Solutions can include:
- a KafkaStreams job that works with __offsets and consumer_offsets as KTable
- Kafka Lag Exporter
- a script with CLI that gets the top offsets of partitions and the groups offsets on them
- from the consumer (I am excluding this, due to the fragility - we want lag visible when cluster is down)
Do you have any experience with any consumer-independent solution of tracking lag and emitting into a DataDog / Prometheus, etc?