Using ConsumerTimestampsInterceptor with kafka Streams

Stream Processing Kafka Streams
1

Hi,

Does using ConsumerTimestampsInterceptor work with Kafka streams? We have 2 clusters replicating data across regions with confluent replicator.

If the streams is consuming from east, we want to make sure the offsets are updated for the consumer group ID in west kafka cluster.

I know that “__consumer_timestamps“ topic is replicated by replicator. Any examples on how to achieve it?

2

I guess it should be possible, but with Kafka Streams (if it’s using internal topics like repartition or changelogs, it’s not recommended to mirror these internal topics – it leads to all kind of correctness concerns).

Cf A Tale of Two Data Centers: Kafka Streams Resiliency - Confluent

Instead, if you fail over, you should re-process the input from the beginning to rebuild the state.

If you are stateless, using it on the input topic only should work. Just checkout Using Replicator to migrate topic data across Confluent Cloud clusters | Confluent Documentation