hello everyone, am having a problem that occurs randomly at the time when a producer tries to generate messages to any topic, this is the error that the producer receives and I do not see errors in Kafka’s logs:
Error: Topic XXXX not present in metadata after 60000 ms
This can be given in any topic and at any time, there is no time pattern and the connection to kafka is stable, also the configuration in kafka to automatically create a topic is enabled auto.create.topics.enable: “true”
I leave the configuration of the producer that is being used to create messages:
retries=2700000
retry.backoff.ms=1000
batch.size=131072
buffer.memory=66554432
bootstrap.servers=Ipserver:9094
compression.type=snappy
acks=all
linger.ms=100
request.timeout.ms=60000
enable.idempotence=true
delivery.timeout.ms=180300
metadata.max.idle.ms=180000
I have a cluster of 3 nodes in azure AKS
can it be some setting in the Producer parameters that can be added or changed to test if the behavior changes?
in a blog I saw that they added these parameters but I don’t know if it would be good to apply them
{ "key": "connections.max.idle.ms", "value": "60000" }, { "key": "metadata.max.idle.ms", "value": "60000" }, { "key": "metadata.max.age.ms", "value": "60000" }