Kafka leader election delay

Hello!

I have a little problem with Kafka in Raft mode when Kafka starting in my K8s cluster.
When first Kafka instance/pod start, it starting election, but after one minute failed, because second instance just fetching image and vice versa.
It cost some time and meantime Kafkas pods are restarting.

I would like to ask have Kafka some settings to extend time for election process? Actually it seems, that is electing one minute and end with error message:

ERROR [BrokerLifecycleManager id=0] Shutting down because we were unable to register with the controller quorum. (kafka.server.BrokerLifecycleManager)

Yes, in the time second Kafka instance isnt ready (due to my slow internet connection) …

Or exist some mechanism to force Kafka instance waiting for another instance?

Thank you very much!

hey @LHozzan

welcome :slight_smile:

never tied by myself
this configs might help:

https://cwiki.apache.org/confluence/display/KAFKA/KIP-595%3A+A+Raft+Protocol+for+the+Metadata+Quorum#KIP595:ARaftProtocolfortheMetadataQuorum-Configurations

best,
michael

Hi @mmuehlbeyer .

Thank you for your reply.

Yop, I try these settings, but I can only achieve, that first started Kafka instance not try it 30 times, but only 15 times (if I doubled one of these settings), but one minute still preserve.
These settings looks fine for me, but I looking for settings to force Kafka attempt election process longer.

Know you please about that settings? Actually it seems, that election process cost 1 minute. I would like to have election process longer, for example 3 minutes before first Kafka instance exit with error state.

Thank you!

Hello.

Finally we founded worked parameter

initial.broker.registration.timeout.ms

By default have the parameter 1 minute and when I extend it to 3 minutes, started instances wait to starting instances and not restarting due to missing voters.

@mmuehlbeyer thank you for your time and effort. Please, consider my question as answered.

Cheers!

1 Like

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.