I have an application with strict requirements to downtime. Currently I have an input topic with three partitions, and I am running three instances of the application to handle the data load. In a ideal distribution of the partitions, each instance will process data from its respective partition. M…

What you observe is by design. Many release back, Kafka Streams changed how it uses the consumer, and it disabled the consumer feature to send a “leave group request” when the consumer is closed. Thus, in your case, only after session.timeout.ms triggers, a rebalance happens. The goal was to avoid …

In our case, the “extra” rebalance when an instance bounces is completely fine as rebalancing usually only takes a few milliseconds. Hence, the internal config sounds like a viable option. Just to clarify, the Stream Properties would look something like this? import org.apache.kafka.streams.Stream…

@hermanjakobsen a more “stable” way of doing it might be the following: streams.close(new CloseOptions().leaveGroup(true)); Edit: This is not true, there is actually a bug. Matthias is correct, and you have to use the internal flag. Please see: [KAFKA-16514] Kafka Streams: stream.close(CloseOption…

How to minimize switchover time between Kafka Streams instances

Stream Processing Kafka Streams

system Closed 10 June 2024 12:51 6

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Kafka Stream high availibility / low latency Kafka Streams	3	3096	29 December 2022
Kafka Stream co-operative rebalance storm and Processing Blocked Kafka Streams	5	4064	16 July 2021
🎧 Smooth Scaling and Uninterrupted Processing with Apache Kafka ft. Sophie Blee-Goldman News and Blogs	0	3253	31 March 2021
Kafka streams rebalance storm Kafka Streams	7	6897	16 July 2021
Syncgroup keeps on failing with message "The group began another rebalance" and it never ends Kafka Streams	0	5033	27 May 2022

How to minimize switchover time between Kafka Streams instances

Related topics