Hi Team
I’m running Kafka Stream Consumer group (Using Spring Cloud function) in Kubernetes (EKS). I’m facing a strange issue sometimes. Whenever a pod (consumer) gets killed/evicted (due to some reasons), a new pod gets added immediately as a new consumer. In this case, rebalance gets started. but my expectation is that after rebalancing, application should run normally. But actually, rebalance continues infinitely. From the logs I get below root cause. But not sure about the fix. Could anyone please guide me here?
Sync group keeps on failing (with different generation id’s) and the rebalance continues to happen.
Triggering the followup rebalance scheduled for 0 ms.
Request joining group due to: rebalance enforced by user
SyncGroup failed: The group began another rebalance. Need to re-join the group. Sent generation was Generation{generationId=131, memberId='exhibit-1-6e823dee-d310-468f-81f9-72962d263c1a-StreamThread-1-consumer-6fe33c8c-826e-4209-afde-23ab7d7b51a8', protocol='stream'}
Session Timeout - 1200000
Max poll interval - 900000
HeartBeat interval - 3000 (edited)
I’m using Kafka-streams 3.0.1