[Python] How to capture Application maximum poll interval exceeded error?

Clients Non-Java Clients
1

Hello,

My microservice uses confluent-kafka-python. Once in a while it fails with this error

%4|1654121013.314|MAXPOLL|rdkafka#consumer-1| [thrd:main]: Application maximum poll interval (300000ms) exceeded by 67ms (adjust max.poll.interval.ms for long-running message processing): leaving group

Whenever it hits this error, it goes into idle instead of terminating.

Snippet of the consumer code:

    def start(self, callback: Callable[[List[Message]], None]) -> None:
        '''
        start consuming
        '''

        try:
            while self.__terminate_event is None or not self.__terminate_event.is_set():

                if not self.__consumer.assignment():
                    self.__consumer.subscribe(
                        [self.__topic],
                        on_assign=_on_assign,
                        on_revoke=_on_revoke,
                        on_lost=_on_lost
                    )

                    log.info("subscribed to topic: %s", self.__topic)

                message_list = self.__consumer.consume(
                    num_messages=self.__num_messages_per_poll, timeout=KAFKA_TIMEOUT
                )

How can it capture this error, so it terminates and let another service respawn?

Thanks

2

Your listing doesn’t show if you have any except clauses, but I would suggest trying to catch the exception in an except clause and check its message to see if it is the one you are concerned with. You can get more details on the structure of the exceptions put out by confluent-kafka here: confluent-kafka-python/error.py at 80ea78c8e46823b68372448e49153723dfffdbf8 · confluentinc/confluent-kafka-python · GitHub

If that doesn’t help, please post back again.

Thanks,
Dave