✍️ What’s New in Apache Kafka 3.1.0

Community News and Blogs
1

:sparkles: There’s a new post on the Confluent blog. Here’s a snippet from it:

On behalf of the Apache Kafka® community, it is my pleasure to announce the release of Apache Kafka 3.1.0. The 3.1.0 release contains many improvements and new features. We’ll highlight […]

:point_right: Head over to the blog to read the full article

2

TL;DR:
While KRaft is still not recommended for production (known gaps), this release contains multiple bugs fixes and new features.

Tiered Storage work continues, with the goal of unlocking infinite scaling and faster rebalance times.

With Apache Kafka 3.1:

  • The FetchRequest supports topic IDs.
  • Enhanced naming consistency for 3 new client metrics with millis and nanos.
  • Built-in and production-grade implementation to allow Kafka to connect to an OpenID identity provider for authentication and token retrieval.
  • Two new metrics are exposed by both the ZooKeeper and KRaft controller: ActiveBrokerCount and FencedBrokerCount to expose the number of active and fenced brokers in the cluster known by the controller.
  • The eager rebalance protocol is deprecated, so users on the eager protocol should prepare to upgrade their applications to the cooperative protocol.
  • A new TaskId field is introduced to the StreamsException class, with a getter API to expose it.
  • Support for foreign-key joins on tables with custom partitioners.
  • Extension of the existing range interfaces semantics in the ReadOnlySessionStore and the ReadOnlyWindowStore to support unbounded ranges. Similarly extension of the existing range and reverseRange interfaces semantics in the ReadOnlyKeyValueStore to support unbounded ranges.
  • A new metric introduced, called blocked-time-total that measures the total time a Kafka Streams thread has spent blocked on Kafka.
  • Additional configuration introduced, to control MirrorMaker2 internal topics naming convention.