Hello there,
Is there a way to setup MM2 for a simple usage of just moving records between topics? We don’t want it to create the topics, update the configuration, etc. We already have a full setup to handle the definition and configuration of topics. We simply want to copy records from topic A into topic B. Sometimes A and B are on the same cluster, but most of the time they are on different clusters, and they often have the exact same name (no prefix or suffix).
I’ve tried using sync.topic.configs.enabled=false and use a KC transformer to define the target topic (with org.apache.kafka.connect.transforms.RegexRouter), but it still automatically creates another target topic (with the name of the source topic prefixed by the source alias).
The only way around it we’ve found is to define a custom replication policy, to tweak the names. However, we also had to define a custom SourceConnector extending the org.apache.kafka.connect.mirror.MirrorSourceConnector and override the isCycle method, or it fails when the topics have the same names on the two clusters.
That’s a lot of boilerplate for us, for a pretty basic use case. Are we missing something here? Is there an easier way to do this? Should we use something else than MM2? I couldn’t find another way to define a connector from Kafka to Kafka.