Don't stop connector when event can not be send on topic

Kafka Connect
1

Hi,

I use debezium to send events on kafka topics. If I try to send an event on an unexisting topic, I have the following error.

WorkerSourceTask{id=tempo-orchestrator-cdc-uat1-0} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted

WorkerSourceTask{id=tempo-orchestrator-cdc-uat1-0} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted
 org.apache.kafka.connect.errors.ConnectException: Not authorized to describe topic(s) 'test' on the brokers -----
    at org.apache.kafka.connect.util.TopicAdmin.lambda$describeTopics$0(TopicAdmin.java:471)
    at java.base/java.util.HashMap.forEach(HashMap.java:1336)
    at org.apache.kafka.connect.util.TopicAdmin.describeTopics(TopicAdmin.java:459)
    at org.apache.kafka.connect.runtime.WorkerSourceTask.maybeCreateTopic(WorkerSourceTask.java:403)
    at org.apache.kafka.connect.runtime.WorkerSourceTask.sendRecords(WorkerSourceTask.java:348)
    at org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:248)
    at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:182)
    at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:231)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.apache.kafka.common.errors.TopicAuthorizationException: Topic authorization failed.

And the connector is stoped.

I tried to use "errors.tolerance" : "all" but nothing change.
Should the errors.tolerance manage this type of error?

More detail can be found on this debezium issue: https://issues.redhat.com/projects/DBZ/issues/DBZ-5081

Thanks

2

The error seems to say that you cannot describe the topic (not that it doesn’t exist at all).

You should be able to filter this topic out from Debezium config to allow the connector to continue, otherwise that is a “unrecoverable exception”, so the error tolerance doesn’t apply.

3

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.