Sending data to Kafka from devices (e.g. Raspberry Pi, Tinkerboard)

Hello everyone,
I followed this beautiful guide to install kafka on GCP:

I have followed all the steps and everything works fine. I was wondering: I have a lot of devices around the world that would like to send data to Kafka on my GCP. How can I do it? does it make sense to use Kafka in this way?


Hi @accat , welcome to the forum! This is a very interesting topic, general answer is: yes, it is possible and many companies do that. The how, really depends on your use case. I can point you to some blog posts that will be able to give you an overview way better than I could in this message :slight_smile: Feel free to come back here and add more details for your use-case and let us know if this info was useful!


They are truly “theoretical” posts, I was looking for something more practical with examples of sending data from devices in the field to Kafka

What kind of devices? Practically you’d probably end up using MQTT or maybe HTTP, both of which are supported in the Kafka ecosystem. If you can give us more specifics to go on we can help you better :slight_smile:

1 Like

@rmoff I use Raspberry and TinkerBoard with NodeJS or Python environment.
Yes, with MQTT or HTTP protocol. How can I send data from these devices to Kafka deployed on my GCP?

Hi @accat , then maybe this example could be interesting?

Some more sophisticated demos using specific MQTT technologies

1 Like

With MQTT proxy can I send data from a remote device (for example a Raspberry somewhere in the world) to a Kafka cluster installed on my GCP?

Sure. Probably you will install the proxy somewhere reachable from you devices, and the proxy should have access to Kafka too. Proxy will receive MQTT from your devices and produce to Kafka using the producer API.

I’ll try and I’ll give you a feedback!

1 Like

I finally found the time to try something.
But I don’t understand one thing.
Reading this guide I should have an MQTT Proxy that I don’t actually have inside Kafka …
How do i install it?