Understanding Kafka Wire Protocol

Clients
1

This question is more from the broker point of view than client. Hope it reaches the right gurus.

I am diving deep into Kafka wire protocol. To be able to parse the request and generate the response, it makes sense to use existing tools than investigate bytes. I am able to parse API Versions request successfully using org.apache.kafka.common.message.ApiVersionsRequestData after trawling the inter-tubes. I have the following questions.

  1. My pom.xml has the following. If this is the link to javadoc then why can’t I locate the org.apache.kafka.common.message package?
        <dependency>
            <groupId>org.apache.kafka</groupId>
            <artifactId>kafka-clients</artifactId>
            <version>3.8.0</version>
        </dependency>

The following actually works - even though I don’t know where this package is documented!

      ApiVersionsRequestData apiVersionsRequest = new ApiVersionsRequestData();
      apiVersionsRequest.read(accessor, apiVersion);

      String name = apiVersionsRequest.clientSoftwareName();
      String version = apiVersionsRequest.clientSoftwareVersion();
  1. What is the right way to send the API Versions response? Interestingly, the Kafka wire protocol does not document this format! Nor do I see it in the javadoc.
  2. Looking at the ResponseHeader.java, the constructor takes the correlation ID and short header version. Whereas, the Kafka wire protocol does not mention the header version at all!

Any guidance on building the API Versions response will be highly appreciated!

Thanks.