Apache Kafka on Windows

Ops
1

This is originally from a Slack thread. Copied here to make it more available (to non-members of the community slack and permanently.)
[You can join the community Slack here]

Jack Thomasson
on Microsoft Windows 10 i installed confluent-community-7.0.1.zip and updated PATH with .../bin/windows. i see there is a kafka-console-producer.bat but no kafka-avro-console-producer.bat. i checked confluent-7.0.1.zip and confluent-community-6.2.2.zip and all of them lack this utility. unfortunately, under git bash .../bin/kafka-avro-console-producergets

Error: Could not find or load main class kafka.tools.ConsoleProducer

Neil Buesing
There are 2 issues with Kafka on windows 1) trying to run under windows shell (command) and running under bash (git-sh for windows)

  • .BAT fixes (to run under command)
    It has been a while since I have done anything with Kafka and Windows, but I had a lot of issues with the installation. I remember having to update kafka-run-class.bat to fix some classpath issues — this felt like they should be fixed, but I just went with it thinking it was unique to me, but maybe it wasn’t — I had to fix classpath so the \share\java\kafka jar files were properly added.

  • fixes to shell scripts so it can run under bash for windows
    Now to get things working with bash for windows that is because a lot of the scripts had issues with figuring out environment. They would incorrectly get CYGWIN wrong — there is some uname in the bash version of kafka-run-class that sets CYGWIN to 0 or 1 depending on the result. I removed that and hard-coded it to be CYGWIN=1 — I also had to comment out any use of the metrics jars (support-metrics?)

Neil Buesing
Again this feels like ages ago for me, but I do remember getting both to work (.bat and .sh) scenarios on a Windows machine.

Jack Thomasson
yeah, i’m a 37 yr Un*x developer with very little experience on Microsoft Windows except frustration. i’ll let cooler heads fix their own utilities.

Neil Buesing
I did get it working - if I had access to a windows machine, I would try to do it again and make fixes and document — it felt like just a few changes to the kafka-run-class both the bat and the other….

Jordan Moore
Suggestion: Use WSL2, not “Git Bash” or CMD

Jack Thomasson
GitHub - jkt628/kafka-kind: local kafka development environment using sigs.k8s.io/kind runs on Linux, macOS, git bash and WSL2. unfortunately, the lack of proper support on Microsoft Windows means the kafka-console-avro-producer examples fail only on Microsoft Windows.

Jordan Moore
If you are using kind, you should use kubectl exec into the schema registry pod to run that command

Jack Thomasson
interesting approach

2

Also, see this blog post: Set Up and Run Apache Kafka on Windows.

1 Like