Using one group id for multiple Kafka topics

this is totally normal if you have a consumer that has logic that depends on both topics. In fact Kafka has done a lot of work to make it so the rebalance isn’t as bad.
It really comes down to that first sentence, does your logic depend on two topics? If so you have to have a single consumer group on both topics.

KIP-415: Incremental Cooperative Rebalancing in Kafka Connect - Apache Kafka - Apache Software Foundation is the KIP you should for sure take a look at. Makes the rebalances much easier on SLAs.

As always, Gwen’s talk on the rebalance protocol is a must watch

1 Like