Language Support for Kafka Streams Library - Exploring Beyond Java and Scala

Kafka Streams
1

Greetings Confluent Community!

Today, I come to you with an intriguing question about the Kafka Streams library. As we all know, Kafka Streams has been widely adopted in Java and Scala ecosystems for building real-time streaming applications. However, I’m eager to delve deeper into the world of languages and explore if there are any other language alternatives available for Kafka Streams.

So, my questions are as follows:

  1. Apart from Java and Scala, are there any official or community-driven language bindings or support for Kafka Streams in languages like Go, Node.js, Python, or any others?
  2. If such language support exists, what are the key features and capabilities available in these implementations?
  3. How does the performance and efficiency of Kafka Streams in these languages compare to the Java and Scala versions? Are there any benchmarks or real-world experiences that can shed light on this aspect?
  4. From a developer’s perspective, what are the advantages and challenges of using Kafka Streams in languages other than Java and Scala? Are there any unique use cases where these alternative language implementations excel?
  5. If you have hands-on experience with Kafka Streams in languages beyond Java and Scala, we would love to hear about your journey, insights, and any lessons learned.

By discussing these aspects, we can gain a comprehensive understanding of the language landscape for Kafka Streams and explore new possibilities for stream processing.

Please feel free to share any resources, articles, or projects that showcase Kafka Streams implementations in different languages. Your contributions will undoubtedly enrich this discussion and open up exciting opportunities for the broader Kafka community.

Thank you all in advance for your participation, and I look forward to a vibrant exchange of ideas!

Let’s embark on this exploration together and unlock the potential of Kafka Streams in various programming languages!

2

Confluent does not have any non-JVM offering at this point.

There is a couple of non-Java projects out in the wild, re-implementing Kafka Streams. For example there is Faust for Python (GitHub - robinhood/faust: Python Stream Processing) and Quix is also building something similar (Quix | Real time stream processing with Kafka and Python). For go there is GitHub - lovoo/goka: Goka is a compact yet powerful distributed stream processing library for Apache Kafka written in Go. and gstream package - github.com/KumKeeHyun/gstream - Go Packages (those are the ones I found just searching the Internet). I would assume there might be more.

I cannot speak to the feature set or quality of any of these projects though.

Personally, I would love to see Kafka Streams being available in as many languages as possible, but it’s a huge investment to build and maintain it, and there are currently no plans at Confluent to invested into other languages for Kafka Streams at this point. – If anybody works on Kafka Streams in a different language, I am open to help out consulting.

3

In general, I find Flink or Beam to have better cross functional language support, even if they fall back to a JVM implementation layer