Replicator creates topic on source cluster

Hello everybody,

I am currently trying to synchronize one topic from a kafka source cluster to a kafka target cluster. I am using the Confluent Replicator for this task but without the cloud service. Both clusters have a own schema registry.
This is my current configuration:

{
“connector.class”: “io.confluent.connect.replicator.ReplicatorSourceConnector”,
“tasks.max”: “1”,
“src.kafka.bootstrap.servers”: “docker-01:29093”,
“src.schema.registry.url”: “http://mq-schema-registry.docker-01:8081”,
“dest.kafka.bootstrap.servers”: “docker-02:29093”,
“dest.schema.registry.url”: “http://mq-schema-registry.docker-02:8081”,
“topic.whitelist”: “topic_xxx”,
“topic.rename.format”: “TEST_${topic}”,
“confluent.topic.replication.factor”: 1,
“key.converter”: “io.confluent.connect.avro.AvroConverter”,
“value.converter”: “io.confluent.connect.avro.AvroConverter”,
“key.converter.schema.registry.url”: “http://mq-schema-registry.docker-02:8081”,
“value.converter.schema.registry.url”: “http://mq-schema-registry.docker-02:8081”,
“schema.subject.translator.class”: “io.confluent.connect.replicator.schemas.DefaultSubjectTranslator”,
“src.key.converter”: “io.confluent.connect.avro.AvroConverter”,
“src.key.converter.schema.registry.url”: “http://mq-schema-registry.docker-01:8081”,
“src.value.converter”: “io.confluent.connect.avro.AvroConverter”,
“src.value.converter.schema.registry.url”: “http://mq-schema-registry.docker-01:8081”,
“offset.start”: “connect”,
“auto.offset.reset”: “earliest”
}

The sync of my schema from source to target works perfectly fine. But the topic itself makes some troubles. On the target cluster the topic gets created but there is no data synchronized. Additional to that the topic gets created on the source cluster too but with the data in it. Please help me to solve this issue.

Thanks and best regards!

hey @JulianG

both clusters are running on your local machine within docker correct?

anything in the logs?

Hey @mmuehlbeyer,

both clusters are running on separate servers.
The two servers can reach each other because they are in the same sub network.
Both clusters are running within docker.

In the logs I dont get any errors but a warning which says that no offsets could be found. DO you need the exact warning?

Thanks for your help!

This is ther warning I get:
WARN Could not find offset for group replicator, topic xxx, partition 0, timestamp 1707912236111, waiting for replication to catch up. Please check replication lag. (io.confluent.connect.replicator.offsets.ConsumerOffsetsTranslator)

hey @JulianG

sorry for the delay.
would you mind sharing the logs and your replicator config?

hey @mmuehlbeyer the replicator config is in my first post and the warning I get is in my third post.

Thanks for your help!

I see is there an existing consumer group on the topic you’d like to replicate?