Streams app with EOS gets stuck reading Global KTable input topic after upgrade to 2.7+

Hi people!

After upgrading my streams app from 2.5.2 to 2.8.0 the app gets stuck on startup while reading an input topic for a Global KTable. The app uses EOS. Changing this to at_least_once allows the app to start up normally. The last log message produced by the app is:

[Consumer clientId=notification-processor-18e3a69b-623e-4218-b2c7-c3976a88f8a7-global-consumer, groupId=null] Resetting offset for partition event-triggers-v4-19 to position FetchPosition{offset=0, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[xxxx:9092 (id: 3 rack: eu-west-1c)], epoch=38}}.

My suspicion is that the streams app gets stuck on a (broken?) transaction on this topic partition. Downgrading to 2.6.2 allows the app to start up normally with EOS. From 2.7.0 and higher the app will again get stuck.

Two questions:

  • Does anyone know of a change introduced in 2.7 that could cause this?

  • How can I get my streams app past this dead point? Because the app does not commit offsets for the topic as its an input for a GlobalKTable.

Cheers!

This was answered by Matthias on the Slack:

It’s a known bug introduced in 2.7.0 – it will be fixed in upcoming 2.8.1 and 3.0.0 releases: https://issues.apache.org/jira/browse/KAFKA-12951

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.