I’m working on a telemetry project where I’m using MQTT, Kafka, and Flink, but I need some help designing the ideal production data flow.
My goal is to ingest 100K data points per second from 800 vehicles. I’d love guidance on:
- How to link MQTT, Kafka, and Flink efficiently
- Best practices for partitioning Kafka topics
- Optimizing Flink for high-throughput stream processing
I’m a student and still learning, so any advice from experienced folks in the community would be super valuable. Appreciate any help or references you can share!