On creating the kafka connector, it is getting auto deleted!!! Below are the debug logs
DEBUG Removed target state for connector Avro_Test_GMI_DBO due to null value in topic. (org.apache.kafka.connect.storage.KafkaConfigBackingStore)
[2023-11-05 17:46:22,131] DEBUG Updating configuration for connector ‘Avro_Test_GMI_DBO’ (org.apache.kafka.connect.storage.KafkaConfigBackingStore)
[2023-11-05 17:46:22,132] DEBUG Storing new config for task Avro_Test_GMI_DBO-0; this will wait for a commit message before the new config will take effect. (org.apache.kafka.connect.storage.KafkaConfigBackingStore)
[2023-11-05 17:46:22,133] INFO Successfully processed removal of connector ‘Avro_Test_GMI_DBO’ (org.apache.kafka.connect.storage.KafkaConfigBackingStore)
[2023-11-05 17:46:22,133] DEBUG Removed target state for connector Avro_Test_GMI_DBO due to null value in topic. (org.apache.kafka.connect.storage.KafkaConfigBackingStore)
@confluentdev/ rmoff, Can I hear some updates on this, Looks like any connector created on a kafka connect cluster gets auto deleted. When can this sort of issue happen?
Removed target state for connector Avro_Test_GMI_DBO due to null value in topic.
One thing to keep in mind is that this error message is about the topic where connector configuration itself is stored, i.e., it’s not about the topic that the connector is producing to or consuming from. Hard to say what the issue is without more info but my hunch is an issue with config.storage.topic, e.g., perhaps workers in your cluster are using different values. You can read up on this configuration here.
Thanks @dtroiano , you are right that connector config is not able to be saved in config.storage.topic, I see errors that say is not able to connect to broker but TCP handshake to the same is successful as per below log line.
“SSL handshake completed successfully with peerHost”.
And still errors out as below
TRACE Failed to produce messages to _Connect_Config-0 with base offset -1. (org.apache.kafka.clients.producer.internals.ProducerBatch)
org.apache.kafka.common.errors.BrokerNotAvailableException: The broker is not available.
TCP level seems to be fine but not sure what is going wrong not reaching to broker while producing to connect cluster’s system config topic.
This looks to be an issue with Kafka itself, i.e., a more basic Kafka cluster health or network connectivity issue. To debug this, I would recommend setting aside Connect and looking at Kafka. Can you successfully produce to it with kafka-console-producer from anywhere? From the location where Connect is running? If the answer to those is no, then you’d want to look at Kafka logs for a clue.