We are trying to replicate a topic called “RTNS” on site A to a topic called “10_20_for_backup_RTNS” on site B. Site B also has RTNS topic which is replciated to site A.
A consumer group called “realtime-notifications-ms” is running on both sites and regularly consumes from the topic “RTNS”, which exists on both sites.
Once site A fails, we want that consumer group “realtime-notifications-ms” will also start consuming from topic “10_20_for_backup_RTNS” to handle the backlog that site A did not handle prior to its failure.
The replication of the data works fine, but the replication of the offsets does not work if the consumer group has active consumers on the target cluster. The active consumers we do have are consuming from RTNS topic only, and not from 10_20_for_backup_RTNS.
In mirrormaker2 log we see the following log periodically:
[2025-03-11 16:32:47,339] WARN [MirrorCheckpointConnector|task-0] Unable to sync offsets for consumer group realtime-notifications-ms. This is likely caused by consumers currently using this group in the target cluster. (org.apache.kafka.connect.mirror.MirrorCheckpointTask:376)
431
As per our tests, mirromaker2 fails to replicate offsets of some consumer group if there is an active consumer associated to this group, even from a topic which is not the target one.
Does it make sense and expected, or do we have some misconfiguration?
I attached the properties file that I am using for the mirrormaker2.
Can you please advise?
20_20_for_backup.bootstrap.servers=kafka-service.kafka-service.xpidbo1-bs.iltct8910.eaas.amdocs.com:443
20_20_for_backup.acks=1
20_20_for_backup.batch.size=100
20_20_for_backup.client.id=for_backup_mirror_maker_producer_20
20_20_for_backup.status.storage.replication.factor=1
20_20_for_backup.config.storage.replication.factor=1
20_20_for_backup.offset.storage.replication.factor=1
20_20_for_backup.heartbeats.topic.replication.factor=1
20_20_for_backup.checkpoints.topic.replication.factor=1
20_20_for_backup.offset-syncs.topic.replication.factor=1
20_20_for_backup->20_20_for_backup.enabled=false
20_20_for_backup.security.protocol=SSL
20_20_for_backup.ssl.truststore.location=/run/secrets/keystores/truststore/pki-common-truststore.jks
20_20_for_backup.ssl.truststore.password=changeit
20_20_for_backup.ssl.truststore.type=JKS
20_20_for_backup.ssl.keystore.location=/run/secrets/keystores/kafka-cluster-tcclient/keystore.p12
20_20_for_backup.ssl.keystore.password=xxx
20_20_for_backup.ssl.keystore.type=PKCS12
20_20_for_backup.sync.group.offsets.enabled=true
20_20_for_backup.sync.group.offsets.interval.seconds=10
20_20_for_backup.replication.policy.separator=_
20_20_for_backup.config.action.reload=none
20_20_for_backup.replication.policy.class=com.digital.ms.activeactive.mirrormaker2.business.init.implementation.CustomReplicationPolicyImpl
20_20_for_backup.emit.checkpoints.interval.seconds=10
10_20_for_backup.bootstrap.servers=kafka-service.kafka-service.xpidbo1-bs.iltct8901.eaas.amdocs.com:443
10_20_for_backup->20_20_for_backup.enabled=true
10_20_for_backup.client.id=for_backup_mirror_maker_consumer_10_20
10_20_for_backup->20_20_for_backup.topics=RTNS
10_20_for_backup.status.storage.replication.factor=1
10_20_for_backup.config.storage.replication.factor=1
10_20_for_backup.offset.storage.replication.factor=1
10_20_for_backup.heartbeats.topic.replication.factor=1
10_20_for_backup.checkpoints.topic.replication.factor=1
10_20_for_backup.offset-syncs.topic.replication.factor=1
10_20_for_backup.security.protocol=SSL
10_20_for_backup.ssl.truststore.location=/run/secrets/keystores/truststore/pki-common-truststore.jks
10_20_for_backup.ssl.truststore.password=changeit
10_20_for_backup.ssl.truststore.type=JKS
10_20_for_backup.ssl.keystore.location=/run/secrets/keystores/kafka-cluster-tc-remoteclient/keystore.p12
10_20_for_backup.ssl.keystore.password=xxx
10_20_for_backup.ssl.keystore.type=PKCS12
10_20_for_backup.sync.group.offsets.enabled=true
10_20_for_backup.groups=realtime-notifications-ms
10_20_for_backup.replication.policy.separator=_
10_20_for_backup.config.action.reload=none
10_20_for_backup.sync.topic.acls.enabled=false
10_20_for_backup.replication.policy.class=com.digital.ms.activeactive.mirrormaker2.business.init.implementation.CustomReplicationPolicyImpl
10_20_for_backup.emit.checkpoints.interval.seconds=10
clusters=20_20_for_backup,10_20_for_backup
replication.factor=1
sync.topic.configs.enabled=false
checkpoints.topic.replication.factor=1
offset-syncs.topic.replication.factor=1
sync.group.offsets.enabled=true
sync.group.offsets.interval.seconds=10
consumer.group.sync.interval.ms=10000
offset.sync.interval.ms=10000
offset-syncs.interval.ms=10000
replication.policy.separator=_
config.action.reload=none
sync.topic.acls.enabled=false
replication.policy.class=com.digital.ms.activeactive.mirrormaker2.business.init.implementation.CustomReplicationPolicyImpl
emit.checkpoints.interval.seconds=10