I am using Kafka mirror maker 2.0 with 2 local kafka. I studied the issue that getting rid of cycles without aliases is not possible at the moment. I try to tune mm2.properties so that MM2 replicates topics with the same names without aliases and without cycles messages in topics. Can you tell me …

Hi @EShabakhov welcome to the forum :slight_smile: Just be sure: you try to replicate a topic between you 2 kafka clusters correct? I assume the following: kafka-a → replicates topic test → kafka-b kafka-b → replicates topic test → kafka-a would be nice if you could give some details what you…

Hi @mmuehlbeyer Yes, I try to replicate topic with the same name. For example: Cluster A replicates topic test to cluster B. Cluster B replicates topic test to cluster A. But MirrorMaker’s configuration does not allow to replicate the same topic without aliases and if I do not specify aliases i…

Hi, [image] EShabakhov: But MirrorMaker’s configuration does not allow to replicate the same topic without aliases and if I do not specify aliases in configuration files, then I will have a message loop. basically it’s a feature to prevent cycles https://cwiki.apache.org/confluence/display/…

I’m not well versed in kafka, but as far as I know there are connectors that work with messages in kafka (read / write). These connectors must read specific topics. MirrorMaker can only provide replication if the alias settings are specified in the configuration files. If one kafka cluster fails, t…

basically yes but again its worth to think about your use case and what you would like to achieve with an active/active mirror maker env see also [image] Apache Kafka MirrorMaker 2 (MM2) Part 1: Theory This blog focuses on Kafka MirrorMaker 2 theory (Kafka replication, architecture, co…

I would like to achieve a fault-tolerant system in time and data, so that users can access one of the kafka clusters and have no problems with it, even if one of the kafka clusters goes down. I found these solutions in stackoverflow. May be I thought that they will not solve my problem. I try to us…

Hi I tried to follow recommendations in solution under the question in stackoverflow but the solution the user suggests contains MigrationReplicationPolicy and mirrormaker swears while working on this class, because he cannot find it.

hi, which Kafka version are you running? best, michael

if my understanding is correct you have to create the class by yourself eg. Cloudera has an example in their docs Migrate Kafka Using a Custom Replication Policy How to migrate Kafka with Streams Replication Manager using a custom replication policy. as we…

How to avoid cycles with mirrormaker

Ops Cluster Replication

mmuehlbeyer 23 March 2022 13:06 17

you’re welcome

yes replicator is part of the commercial features

best,
michael

Disaster Recovery Strategy

Topic		Replies	Views
MirrorMaker Endless loop Cluster Replication	5	5126	6 December 2022
Mirrormaker2 creating heartbeats topics recursively in loop in active-active mode Kafka Connect	52	2473	16 July 2024
MirrorMaker 2.0 multiple instances Cluster Replication	2	6342	21 May 2021
MirrorMaker2 not replicating certain topics Cluster Replication	8	218	2 May 2025
Kafka Mirror Maker 2 - source topic keep getting created Kafka Streams	2	3825	11 August 2021

How to avoid cycles with mirrormaker

Related topics