ConsumerGroups disappearing

Hi all,

I am experiencing ConsumerGroups are disappearing when I restart a kafka broker.
Partitions are often offline/unavailable (lost) and the consumers recreate the ConsumerGroups (which results in huge lags, since it starts from the beginning again).

We have had this for multiple ConsumerGroups, and we have a hard time pinpointing where the problem is.

Do you have any idea or suggestions of what we can try or do to resolve this issue?

Our kafka cluster consists of 5 kafka brokers that each reside on different nodes.

All kafka brokers are configured with the following (relevant, I hope) configuration for HA:

default.replication.factor: 3 0
log.flush.interval.messages: 10000 1 second
min.insync.replicas: 2
offsets.retention.minutes: 1 year and 7 days
offsets.topic.replication.factor: 3
transaction.state.log.min.isr: 2
transaction.state.log.replication.factor: 3

topic: __consumer_offsets
unclean.leader.election.enable: false

Any helps or hints would be greatly appreciated :slight_smile: