We are using Confluent’s Kafka Python Client for reading messages from a Kafka Topic.
The python code was running perfectly for last 8 months daily without any issue, but in recent days we are sporadically getting issues while consuming data from kafka topic.
Below is the python code behaviour when the issue occurs :
- After consuming some amount of messages, consumer.poll() returns “None” even if messages are present on the topic
- While committing the offset of the last received message, the program crashes with KafkaError{code=_ASSIGNMENT_LOST,val=-142,str=“Commit failed: Local: Group partition assignment lost”}.
This issue only happens sporadically and has started to occur in recent days. Please help.