Hi,
We have 3 nodes kafka cluster with A side, we want to replicate to B side, but mirrormaker 2 couldn’t connect to A side cluster, as below of we have settings and error messages, we don’t have SSL authentication, used with SASL_PLAINTEXT,
server.properties
security.inter.broker.protocol=SASL_PLAINTEXT
sasl.mechanism.inter.broker.protocol=PLAIN
sasl.enabled.mechanisms=PLAIN
authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
listeners=SASL_PLAINTEXT://rota-dr-kafka01:9093
advertised.listeners=SASL_PLAINTEXT://rota-dr-kafka01:9093
super.users=User:admin;User:arpclient
enable.ssl.certificate.verification=false
zookeeper.connect=zookeeper4:2182
Error messages After run to mirror maker 2 on B side node,
Note: Kafka and zookeeper services has been runned properly on A and B side nodes
[2022-08-18 10:19:49,595] WARN The configuration ‘offset.storage.replication.factor’ was supplied but isn’t a known config. (org.apache.kafka.clients.admin.AdminClientConfig:380)
[2022-08-18 10:19:49,595] WARN The configuration ‘admin.sasl.jaas.config’ was supplied but isn’t a known config. (org.apache.kafka.clients.admin.AdminClientConfig:380)
[2022-08-18 10:19:49,595] WARN The configuration ‘admin.bootstrap.servers’ was supplied but isn’t a known config. (org.apache.kafka.clients.admin.AdminClientConfig:380)
[2022-08-18 10:19:49,596] INFO Kafka version: 2.8.0 (org.apache.kafka.common.utils.AppInfoParser:119)
[2022-08-18 10:19:49,596] INFO Kafka commitId: ebb1d6e21cc92130 (org.apache.kafka.common.utils.AppInfoParser:120)
[2022-08-18 10:19:49,596] INFO Kafka startTimeMs: 1660807189595 (org.apache.kafka.common.utils.AppInfoParser:121)
[2022-08-18 10:20:19,600] INFO [AdminClient clientId=adminclient-1] Metadata update failed (org.apache.kafka.clients.admin.internals.AdminMetadataManager:235)
org.apache.kafka.common.errors.TimeoutException: Call(callName=fetchMetadata, deadlineMs=1660807219597, tries=1, nextAllowedTryMs=0) timed out at 1660807219599 after 1 attempt(s)
Caused by: org.apache.kafka.common.errors.DisconnectException: Cancelled fetchMetadata request with correlation id 1 due to node -1 being disconnected
[2022-08-18 10:20:49,601] INFO [AdminClient clientId=adminclient-1] Metadata update failed (org.apache.kafka.clients.admin.internals.AdminMetadataManager:235)
org.apache.kafka.common.errors.TimeoutException: Call(callName=fetchMetadata, deadlineMs=1660807249599, tries=1, nextAllowedTryMs=1660807249701) timed out at 1660807249601 after 1 attempt(s)
Caused by: org.apache.kafka.common.errors.DisconnectException: Cancelled fetchMetadata request with correlation id 3 due to node -1 being disconnected
[2022-08-18 10:20:49,606] INFO App info kafka.admin.client for adminclient-1 unregistered (org.apache.kafka.common.utils.AppInfoParser:83)
[2022-08-18 10:20:49,606] INFO [AdminClient clientId=adminclient-1] Metadata update failed (org.apache.kafka.clients.admin.internals.AdminMetadataManager:235)
org.apache.kafka.common.errors.TimeoutException: Call(callName=fetchMetadata, deadlineMs=1660807279601, tries=1, nextAllowedTryMs=-9223372036854775709) timed out at 9223372036854775807 after 1 attempt(s)
Caused by: org.apache.kafka.common.errors.TimeoutException: The AdminClient thread has exited. Call: fetchMetadata
[2022-08-18 10:20:49,612] INFO Metrics scheduler closed (org.apache.kafka.common.metrics.Metrics:659)
[2022-08-18 10:20:49,612] INFO Closing reporter org.apache.kafka.common.metrics.JmxReporter (org.apache.kafka.common.metrics.Metrics:663)
[2022-08-18 10:20:49,612] INFO Metrics reporters closed (org.apache.kafka.common.metrics.Metrics:669)
[2022-08-18 10:20:49,612] ERROR Stopping due to error (org.apache.kafka.connect.mirror.MirrorMaker:313)
org.apache.kafka.connect.errors.ConnectException: Failed to connect to and describe Kafka cluster. Check worker’s broker connection and security properties.
at org.apache.kafka.connect.util.ConnectUtils.lookupKafkaClusterId(ConnectUtils.java:70)
at org.apache.kafka.connect.util.ConnectUtils.lookupKafkaClusterId(ConnectUtils.java:51)
at org.apache.kafka.connect.mirror.MirrorMaker.addHerder(MirrorMaker.java:236)
at java.lang.Iterable.forEach(Iterable.java:75)
at org.apache.kafka.connect.mirror.MirrorMaker.(MirrorMaker.java:137)
at org.apache.kafka.connect.mirror.MirrorMaker.(MirrorMaker.java:149)
at org.apache.kafka.connect.mirror.MirrorMaker.main(MirrorMaker.java:300)
Caused by: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Call(callName=listNodes, deadlineMs=1660807249602, tries=1, nextAllowedTryMs=1660807249705) timed out at 1660807249605 after 1 attempt(s)
at org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45)
at org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32)
at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89)
at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:260)
at org.apache.kafka.connect.util.ConnectUtils.lookupKafkaClusterId(ConnectUtils.java:64)
… 6 more
Caused by: org.apache.kafka.common.errors.TimeoutException: Call(callName=listNodes, deadlineMs=1660807249602, tries=1, nextAllowedTryMs=1660807249705) timed out at 1660807249605 after 1 attempt(s)
Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. Call: listNodes