Jira source connector issue - TimeoutException: License topic could not be created

Kafka Connect Self-Managed Connectors
1

I am trying to add Jira source connector in my development environment (single broker docker container ) and the connector doesn’t start by throwing the below error messages. Other connectors work (MongoDB source connector, File source connector and ElasticSearch sink connector)

Connector configuration

{
  "value.converter.schema.registry.url": "",
  "key.converter.schema.registry.url": "",
  "name": "Jira_Issues",
  "connector.class": "io.confluent.connect.jira.JiraSourceConnector",
  "tasks.max": "1",
  "key.converter": "io.confluent.connect.avro.AvroConverter",
  "value.converter": "io.confluent.connect.avro.AvroConverter",
  "jira.since": "2021-04-17 23:50",
  "jira.tables": [
    "issues"
  ],
  "jira.username": "username",
  "jira.api.token": "Password",
  "jira.url": "https://jira.test.com/",
  "confluent.topic.bootstrap.servers": [
    "broker:9092"
  ],
  "confluent.topic": "jira_issues",
  "confluent.topic.replication.factor": "1"
}

Error message

[2021-04-21 12:51:37,772] INFO App info kafka.admin.client for Jira_Issues-license-manager unregistered (org.apache.kafka.common.utils.AppInfoParser)

[2021-04-21 12:51:37,773] INFO [AdminClient clientId=Jira_Issues-license-manager] Metadata update failed (org.apache.kafka.clients.admin.internals.AdminMetadataManager)

org.apache.kafka.common.errors.TimeoutException: Call(callName=fetchMetadata, deadlineMs=1619009499800, tries=1, nextAllowedTryMs=-9223372036854775709) timed out at 9223372036854775807 after 1 attempt(s)

Caused by: org.apache.kafka.common.errors.TimeoutException: The AdminClient thread has exited. Call: fetchMetadata

[2021-04-21 12:51:37,773] INFO Metrics scheduler closed (org.apache.kafka.common.metrics.Metrics)

[2021-04-21 12:51:37,773] INFO Closing reporter org.apache.kafka.common.metrics.JmxReporter (org.apache.kafka.common.metrics.Metrics)

[2021-04-21 12:51:37,774] INFO Metrics reporters closed (org.apache.kafka.common.metrics.Metrics)

[2021-04-21 12:51:37,774] ERROR WorkerConnector{id=Jira_Issues} Error while starting connector (org.apache.kafka.connect.runtime.WorkerConnector)

org.apache.kafka.common.errors.TimeoutException: License topic could not be created

Caused by: org.apache.kafka.common.errors.TimeoutException: Call(callName=createTopics, deadlineMs=1619009497771, tries=1, nextAllowedTryMs=1619009497872) timed out at 1619009497772 after 1 attempt(s)

Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. Call: createTopics

[2021-04-21 12:51:37,774] ERROR [Worker clientId=connect-1, groupId=compose-connect-group] Failed to start connector 'Jira_Issues' (org.apache.kafka.connect.runtime.distributed.DistributedHerder)

org.apache.kafka.connect.errors.ConnectException: Failed to start connector: Jira_Issues

               at org.apache.kafka.connect.runtime.distributed.DistributedHerder.lambda$startConnector$5(DistributedHerder.java:1305)

               at org.apache.kafka.connect.runtime.WorkerConnector.doTransitionTo(WorkerConnector.java:335)

               at org.apache.kafka.connect.runtime.WorkerConnector.doRun(WorkerConnector.java:140)

               at org.apache.kafka.connect.runtime.WorkerConnector.run(WorkerConnector.java:117)

               at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)

               at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)

               at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)

               at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)

               at java.base/java.lang.Thread.run(Thread.java:834)
2

Make sure that you’ve configured the licence topic & broker connection details correctly, and that the topic has been pre-created if auto creation is disabled.

3

Thank you! I was able to pass the License related error message and hit another one. I used password for authentication purpose (jira.api.token value)

Configuration

{
  "topic.creation.default.partitions": "1",
  "topic.creation.enable": "true",
  "value.converter.schema.registry.url": ,
  "topic.creation.default.replication.factor": "1",
  "key.converter.schema.registry.url": ,
  "name": "Jira_Issues",
  "connector.class": "io.confluent.connect.jira.JiraSourceConnector",
  "tasks.max": "1",
  "key.converter": "io.confluent.connect.avro.AvroConverter",
  "value.converter": "io.confluent.connect.avro.AvroConverter",
  "jira.since": "2021-04-17 12:49",
  "jira.tables": [
    "issues"
  ],
  "jira.username": "username",
  "jira.api.token": "****************",
  "jira.url": https: //jira.test.com/,
  "max.poll.interval.ms": "1000",
  "topic.name.pattern": "jira-topic-${entityName}",
  "confluent.topic.bootstrap.servers": [
    "broker:29092"
  ],
  "confluent.topic": "",
  "confluent.topic.replication.factor": "1"
}

Error Message

[2021-04-22 12:48:22,373] ERROR WorkerSourceTask{id=Jira_Issues-0} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask)
java.lang.IllegalStateException: New RUNNING not compatible with current state FAILED
               at io.confluent.connect.operations.OperationResult.setState(OperationResult.java:300)
               at io.confluent.connect.operations.OperationResult.start(OperationResult.java:91)
               at io.confluent.connect.operations.Operation.start(Operation.java:172)
               at io.confluent.connect.operations.OperationExecutor.startOperation(OperationExecutor.java:411)
               at io.confluent.connect.operations.OperationExecutor.startNextOperations(OperationExecutor.java:390)
               at io.confluent.connect.operations.OperationExecutor.pollForRecords(OperationExecutor.java:301)
               at io.confluent.connect.operations.OperationExecutor.pollForRecords(OperationExecutor.java:276)
               at io.confluent.connect.jira.JiraSourceTask.poll(JiraSourceTask.java:204)
               at org.apache.kafka.connect.runtime.WorkerSourceTask.poll(WorkerSourceTask.java:289)
               at org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:256)
               at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:185)
               at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:234)
               at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
               at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
               at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
               at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
               at java.base/java.lang.Thread.run(Thread.java:834)
[2021-04-22 12:48:22,373] INFO Stopping task and cancelling any currently running operations (io.confluent.connect.jira.JiraSourceTask)
[2021-04-22 12:48:22,374] INFO [Producer clientId=connector-producer-Jira_Issues-0] Closing the Kafka producer with timeoutMillis = 30000 ms.
4

The root cause of the problem is a certificate issue. Problem is resolved by importing the certificate to the trust store. The error message doesn’t give a clue to the root cause. Decompiling the connector lib and trying locally leads to the solution.

2 Likes
5

This page contains some good info on how to add import the cert into a local java trust store:
https://confluence.atlassian.com/kb/how-to-import-a-public-ssl-certificate-into-a-jvm-867025849.html#HowtoimportapublicSSLcertificateintoaJVM-commandline

6

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.