I am not able to connect my MSK cluster to my on-prem cluster. Here are the configs for bidirectional sync:
connector.class=org.apache.kafka.connect.mirror.MirrorSourceConnector
replication.factor=1
target.cluster.sasl.jaas.config=software.amazon.msk.auth.iam.IAMLoginModule required awsDebugCreds=true;
auto.create.topics.enable=true
tasks.max=1
source.cluster.alias=source
refresh.groups.enabled=true
target.cluster.producer.sasl.jaas.config=software.amazon.msk.auth.iam.IAMLoginModule required awsDebugCreds=true;
target.cluster.consumer.sasl.client.callback.handler.class=software.amazon.msk.auth.iam.IAMClientCallbackHandler
sync.topic.configs.interval.seconds=60
producer.retry.backoff.ms=1000
target.cluster.producer.security.protocol=SASL_SSL
consumer.auto.offset.reset=earliest
target.cluster.security.protocol=SASL_SSL
target.cluster.consumer.sasl.mechanism=AWS_MSK_IAM
offset.storage.topic=testoffsets
value.converter=org.apache.kafka.connect.converters.ByteArrayConverter
target.cluster.producer.sasl.client.callback.handler.class=software.amazon.msk.auth.iam.IAMClientCallbackHandler
key.converter=org.apache.kafka.connect.converters.ByteArrayConverter
refresh.groups.interval.seconds=60
clusters=source,target
refresh.topics.interval.seconds=60
topics.exclude=.[-.]internal,..replica,.,.-config,.-status,.-offset,source.,heartbeats.
offset-syncs.topic.replication.factor=1
target.cluster.consumer.sasl.jaas.config=software.amazon.msk.auth.iam.IAMLoginModule required awsDebugCreds=true;
emit.checkpoints.enabled=true
topics=stc.*
refresh.topics.enabled=true
producer.max.block.ms=10000
target.cluster.sasl.mechanism=AWS_MSK_IAM
heartbeats.topic.replication.factor=1
target.cluster.sasl.client.callback.handler.class=software.amazon.msk.auth.iam.IAMClientCallbackHandler
source.cluster.bootstrap.servers=-redacted-:9092
target.cluster.alias=target
producer.linger.ms=500
target.cluster.consumer.security.protocol=SASL_SSL
groups.exclude=console-consumer-.,connect-.,.,heartbeats.
target.cluster.bootstrap.servers=-redacted-:9098
checkpoints.topic.replication.factor=1
sync.topic.configs.enabled=true
target.cluster.producer.sasl.mechanism=AWS_MSK_IAM
connector.class=org.apache.kafka.connect.mirror.MirrorSourceConnector
source.cluster.consumer.sasl.jaas.config=software.amazon.msk.auth.iam.IAMLoginModule required awsDebugCreds=true;
replication.factor=1
auto.create.topics.enable=true
tasks.max=1
source.cluster.producer.security.protocol=SASL_SSL
source.cluster.alias=source
refresh.groups.enabled=true
source.cluster.producer.sasl.client.callback.handler.class=software.amazon.msk.auth.iam.IAMClientCallbackHandler
sync.topic.configs.interval.seconds=60
producer.retry.backoff.ms=1000
buffer.memory=10000
source.cluster.producer.sasl.mechanism=AWS_MSK_IAM
source.cluster.sasl.client.callback.handler.class=software.amazon.msk.auth.iam.IAMClientCallbackHandler
source.cluster.consumer.sasl.client.callback.handler.class=software.amazon.msk.auth.iam.IAMClientCallbackHandler
consumer.auto.offset.reset=earliest
offset.storage.topic=testoffsets
value.converter=org.apache.kafka.connect.converters.ByteArrayConverter
key.converter=org.apache.kafka.connect.converters.ByteArrayConverter
refresh.groups.interval.seconds=60
clusters=source,target
refresh.topics.interval.seconds=60
topics.exclude=.[-.]internal,…replica,.,.-config,.-status,.-offset,source.,heartbeats.,source.heartbearts.*
offset-syncs.topic.replication.factor=1
emit.checkpoints.enabled=true
topics=stc.*
refresh.topics.enabled=true
producer.max.block.ms=10000
source.cluster.producer.sasl.jaas.config=software.amazon.msk.auth.iam.IAMLoginModule required awsDebugCreds=true;
heartbeats.topic.replication.factor=1
source.cluster.consumer.security.protocol=SASL_SSL
offset.flush.timeout.ms=60000
source.cluster.consumer.sasl.mechanism=AWS_MSK_IAM
source.cluster.sasl.jaas.config=software.amazon.msk.auth.iam.IAMLoginModule required awsDebugCreds=true;
source.cluster.bootstrap.servers=-redacted-:9098
source.cluster.sasl.mechanism=AWS_MSK_IAM
target.cluster.alias=target
producer.linger.ms=500
groups.exclude=source.,console-consumer-.,connect-.,.,heartbeats.*
target.cluster.bootstrap.servers=-redacted-:9092
checkpoints.topic.replication.factor=1
sync.topic.configs.enabled=true
source.cluster.security.protocol=SASL_SSL
They are used for bidirectional sync, and the on-prem to MSK connector succeeds. But the MSK to on-prem connector does not send messages and fails with this:
WARN [TestDest|task-0] [Producer clientId=connector-producer-TestDest-0] Error while fetching metadata with correlation id 78177 : {source.stctest=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient:1119)
The replicated topic source.stctest is present in on-prem, indicating the connector succeeded in connecting to on-prem to create that remote topic for replication, but no messages are sent. I do not know what may be this issue, I have tried what I could think of to no success.