There is a producer metric ( request-latency-avg ), this is not updated in producer metrics documentation in confluent.Also can i please get the detail of this metric that records the time spent from which point to which point exactly?
Total time in ms to serve the specified request | LinkThe explanation for this metric is also not clear, can we have a more precise explanation of this and also wanted to ask if can we have this at the topic level?
This is the average amount of time your record batches spend in the send buffer.the batch prior to being flushed
→ how long to fill a batch
double checked once again and it’s a measure of the amount of time between when KafkaProducer.send() was called until the producer receives a response from the broker.
→ so “after” the buffering
Hey there, I think we’ve pretty much nailed it down here, but I wanted to make sure we have this clarified once and for all.
request-latency-avg is a little misleading as it’s not clear exactly what request is referring to. It does not correspond to each call to producer.send() as might be expected. Rather, as you’ve reasoned through here, it refers to the internal request that the producer makes to send records to the broker. record-queue-time-avg is the time the records sit in the producer queue/buffer between the call to producer.send() and the aforementioned request to the broker.
To summarize: request-latency-avg and record-queue-time-avg refer to two completely different time periods in the record send lifecycle; there’s no overlap between the two.
When producer.send() is called, records will first sit in the producer buffer/queue until batch.size or linger.ms is reached or we call producer.flush(). This contributes to record-queue-time-avg. (And this is also why the record-queue-time-avg is so close to linger.ms in the example you printed, @tushargoyal22.)
After this, the producer makes a request to the broker, at which point we start the clock for request-latency-avg.