Hi,
I’m currently trying to build a pipeline where time-series data flows like this:
file -> Fluent Bit -> Kafka -> Kafka Connect (Prometheus sink or InfluxDB sink) -> Grafana
Which sink (Prometheus vs InfluxDB) would be more appropriate for this kind of setup?
Also, when people integrate time-series databases with Kafka, how do they usually connect and verify the data flow in practice? I’m especially curious about the “middle layer” between Kafka and Grafana—what is the common or recommended pattern there?
Finally, when creating a custom sink connector, do I need to explicitly map all of the log schemas/fields myself?
Thanks!