Hello,
I am trying to create an http-sink-connector but receive:
*Caused by: org.apache.kafka.common.errors.TimeoutException: License topic could not be created*
*Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. Call: createTopics*
Here is what I see from connector status:
{
"name": "http-sink-test3",
"connector": {
"state": "FAILED",
"worker_id": "<worker-ip>:8083",
"trace": "org.apache.kafka.common.errors.TimeoutException: License topic could not be created\nCaused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. Call: createTopics\n"
},
"tasks": [],
"type": "sink"
}
Here is my connector configuration:
{
"connector.class": "io.confluent.connect.http.HttpSinkConnector",
"confluent.topic.bootstrap.servers": "server1:9095,server2:9096,server3:9097",
"topics": "HTTP_TEST",
"tasks.max": "1",
"http.api.url": "https://<IP-ADDRESS>/api/",
"group.id": "kafka_connect",
"reporter.bootstrap.servers": "server1:9095,server2:9096,server3:9097",
"reporter.error.topic.name": "HTTP_ERROR",
"value.converter.schema.registry.url": "http://schema:8081/",
"reporter.result.topic.name": "HTTP_SUCCESS",
"reporter.error.topic.replication.factor": "1",
"confluent.topic.replication.factor": "1",
"value.converter": "io.confluent.connect.avro.AvroConverter",
"key.converter": "io.confluent.connect.avro.AvroConverter",
"key.converter.schema.registry.url": "http://schema:8081/",
"reporter.result.topic.replication.factor": "1"
}
Here is the error from connector logs:
INFO [http-sink-test3|worker] [AdminClient clientId=http-sink-test3-license-manager] Metadata update failed (org.apache.kafka.clients.admin.internals.AdminMetadataManager:235)
org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. Call: fetchMetadata
[2022-02-24 12:58:51,046] INFO [http-sink-test3|worker] App info kafka.admin.client for http-sink-test3-license-manager unregistered (org.apache.kafka.common.utils.AppInfoParser:83)
[2022-02-24 12:58:51,046] INFO [http-sink-test3|worker] [AdminClient clientId=http-sink-test3-license-manager] Metadata update failed (org.apache.kafka.clients.admin.internals.AdminMetadataManager:235)
org.apache.kafka.common.errors.TimeoutException: Timed out waiting to send the call. Call: fetchMetadata
[2022-02-24 12:58:51,046] INFO [http-sink-test3|worker] [AdminClient clientId=http-sink-test3-license-manager] Timed out 1 remaining operation(s) during close. (org.apache.kafka.clients.admin.KafkaAdminClient:1400)
[2022-02-24 12:58:51,047] INFO [http-sink-test3|worker] Metrics scheduler closed (org.apache.kafka.common.metrics.Metrics:676)
[2022-02-24 12:58:51,047] INFO [http-sink-test3|worker] Closing reporter org.apache.kafka.common.metrics.JmxReporter (org.apache.kafka.common.metrics.Metrics:680)
[2022-02-24 12:58:51,047] INFO [http-sink-test3|worker] Metrics reporters closed (org.apache.kafka.common.metrics.Metrics:686)
[2022-02-24 12:58:51,047] ERROR [http-sink-test3|worker] WorkerConnector{id=http-sink-test3} Error while starting connector (org.apache.kafka.connect.runtime.WorkerConnector:193)
org.apache.kafka.common.errors.TimeoutException: License topic could not be created
Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. Call: createTopics
[2022-02-24 12:58:51,048] ERROR [http-sink-test3|worker] [Worker clientId=connect-1, groupId=eim_connect_sit] Failed to start connector 'http-sink-test3' (org.apache.kafka.connect.runtime.distributed.DistributedHerder:1490)
org.apache.kafka.connect.errors.ConnectException: Failed to start connector: http-sink-test3
at org.apache.kafka.connect.runtime.distributed.DistributedHerder.lambda$startConnector$25(DistributedHerder.java:1461)
at org.apache.kafka.connect.runtime.WorkerConnector.doTransitionTo(WorkerConnector.java:335)
at org.apache.kafka.connect.runtime.WorkerConnector.doRun(WorkerConnector.java:141)
at org.apache.kafka.connect.runtime.WorkerConnector.run(WorkerConnector.java:118)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.apache.kafka.connect.errors.ConnectException: Failed to transition connector http-sink-test3 to state STARTED
... 8 more
Caused by: org.apache.kafka.common.errors.TimeoutException: License topic could not be created
Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. Call: createTopics
I have checked this documentation for ACLs and license topic:
https://docs.confluent.io/kafka-connect-http/current/connector_config.html#confluent-license-properties
The mentioned ACLs were created.
In our deployment we are using the Enterprise (Subscription) License, which can be seen in the _confluent-command topic.
Do you have any ideas?