KafkaProducer.send behaves differently then documented

Clients Java Clients
1

Hi,

We’re seeing a strange behaviour from KafkaProducer.send method.
According to kafka 2.7.0 API :

Asynchronously send a record to a topic and invoke the provided callback when the send has been acknowledged.
The send is asynchronous and this method will return immediately once the record has been stored in the buffer of records waiting to be sent. This allows sending many records in parallel without blocking to wait for the response after each one.

However when we use that it actually blocks as opposed to the documentation.
This does not allow us to utilise the asynchronous mechanism and makes the returned Future unusable.

  1. Anyone know what’s the deal with that? Is that a bug in the documentation or the implementation?
  2. Can we assume that the method is indeed blocking? Are we guaranteed once it returns that records have been acknowledged?
2

Why do you think the method blocks?

3
  1. debugging… it blocks for a few seconds… consumer sees records before it returns
  2. closing the connection directly after the call should result in error AFAIK if not waiting for the futures to complete which it doesn’t. (with a large number of messages that take seconds to send)

Should that happen?

4

Might be because of the debugger, or because it’s the first message. It’s async but it’s not like it immediately returns. There are different ways of closing, by closing the connection you mean closing the app?

You might want to set linger.ms to something like 1000ms, te be convinced it indeed doesn’t send the message directly. The default setting is 0ms, so then it’s pretty much immediate.

5

Thanks I’ll try that.