Kafka - Java & Zookeeper compatability

Kafka Connect Managed Connectors
1

Looking for a JDK1.8 & Zk 3.4.9 compatible Kafka version.

Wondering whether the latest kafka doesn’t need Zk and has it built-in and whether it would work with jdk1.8.

TIA

2

hey @jakonconf

welcome :slight_smile:

regarding the java version:
for the latest confluent platform check
https://docs.confluent.io/platform/current/installation/system-requirements.html#java

for the lastest apache kafka have look at

regarding zookeeper:
depending on your needs you may use KRaft instead of Zookeeper with the current version of Apache Kafka or Confluent Platform.
For information and limitations see:

Best,
Michael

3

Thanks Michael for your kind response with all the reference links - quite useful and educative for me to understand the underlying options.

Since we have jdk1.8 dependencies by the other components we had to go with it - for kafka 3.x, I see that we are covered though deprecated.

Also we had to use zookeeper for other hadoop components and would like to share and put that to use here.

From the learning, I plan to use zookeeper 3.4.9 (basis other dependencies) and kafka 3.0.1/3.0.2 - but it uses zookeeper 3.4.14 jars. Does it(kafka) work if I simply replace the zk jars to 3.4.9 jars ?

Not sure it is appropriate for me to use the same thread to ask a different question from the topic?

4

not fully sure whether it will work
though it might be worth to try to connect with the 3.4.14 jars to your zookeeper 3.4.9 cluster.

Go ahead :slight_smile: if needed we could create another thread I think

5

While starting kafka (with DEBUG & resources available), it gets killed.

Here is what I see in the logs but nothing suspectful here. Anything I am missing?

[2023-04-19 18:13:11,925] DEBUG Scheduling task auto-leader-rebalance-task with initial delay 300000 ms and period -1000 ms. (kafka.utils.KafkaScheduler)
[2023-04-19 18:13:33,139] DEBUG Beginning log cleanup… (kafka.log.LogManager)
[2023-04-19 18:13:33,143] DEBUG Log cleanup completed. 0 files deleted in 0 seconds (kafka.log.LogManager)
[2023-04-19 18:13:33,156] DEBUG Checking for dirty logs to flush… (kafka.log.LogManager)
[2023-04-19 18:13:33,192] DEBUG Scheduling task kafka-delete-logs with initial delay 60000 ms and period -1 ms. (kafka.utils.KafkaScheduler)
Killed

6

Ignore the kafka server getting killed error. Looks like a weird issue of resources. Post restart of the node this went away. Now, not able to reproduce it. Hope this doesn’t recur.

7

we have jdk1.8 dependencies by the other components

The server runtime doesn’t matter for external client dependencies.

You can refer the Zookeeper documentation that states that latest version is backwards compatible with older Zookeeper clients which Kafka may use.