librdkafka version => 1.6.0
topic.metadata.propagation.max.ms=60000
Topic auto creation NOT used (in our use case different topics need to have different partition counts). OS = Red-hat Linux 7.9
Message is sent(RdKafka::Producer::produce) immediately after successfully creating the topic (rd_kafka_CreateTopics => rd_kafka_queue_poll => if NOT timed out, check errors with rd_kafka_event_error and rd_kafka_event_error+rd_kafka_CreateTopics_result_topics+rd_kafka_topic_result_error => RD_KAFKA_RESP_ERR_NO_ERROR is taken as success)
Same RdKafka::Producer used to create topics and Produce. Both calls made in in same thread.
But the produce call/delivery fails some times with below errors(other times it is successful),
Produce call fail : ERR__UNKNOWN_TOPIC, ERR__FATAL
Delivery fail (Callback) : ERR__DESTROY, ERR__UNKNOWN_PARTITION
What is the recommended solution here ?
- Add a sleep after each topic creation ?
OR - any other ?