Using Zookeeper with Kafka Broker 3.1.0

Cahit · 19 April 2022 00:35

Hi,

I am wondering can we still use zookeeper with Kafka broker version 3.1.0?

mmuehlbeyer · 19 April 2022 05:03

yes you still can use zookeeper and should be used in production as
KRaft mode it’s not ready for production yet (as of Kafka 3.1)

github.com

apache/kafka/blob/26f1c5f91e6d479889dfe5a897ae2de96c576ec0/config/kraft/README.md

KRaft (aka KIP-500) mode Preview Release
=========================================================

# Introduction
It is now possible to run Apache Kafka without Apache ZooKeeper!  We call this the [Kafka Raft metadata mode](https://cwiki.apache.org/confluence/display/KAFKA/KIP-500%3A+Replace+ZooKeeper+with+a+Self-Managed+Metadata+Quorum), typically shortened to `KRaft mode`.
`KRaft` is intended to be pronounced like `craft` (as in `craftsmanship`). It is currently *PREVIEW AND SHOULD NOT BE USED IN PRODUCTION*, but it
is available for testing in the Kafka 3.1 release.

When the Kafka cluster is in KRaft mode, it does not store its metadata in ZooKeeper.  In fact, you do not have to run ZooKeeper at all, because it stores its metadata in a KRaft quorum of controller nodes.

KRaft mode has many benefits -- some obvious, and some not so obvious.  Clearly, it is nice to manage and configure one service rather than two services.  In addition, you can now run a single process Kafka cluster.
Most important of all, KRaft mode is more scalable.  We expect to be able to [support many more topics and partitions](https://www.confluent.io/kafka-summit-san-francisco-2019/kafka-needs-no-keeper/) in this mode.

# Quickstart

## Warning
KRaft mode in Kafka 3.1 is provided for testing only, *NOT* for production.  We do not yet support upgrading existing ZooKeeper-based Kafka clusters into this mode.  
There may be bugs, including serious ones.  You should *assume that your data could be lost at any time* if you try the preview release of KRaft mode.

## Generate a cluster ID

This file has been truncated. show original

hth,
michael

Cahit · 19 April 2022 06:50

Thank you @mmuehlbeyer

Topic		Replies	Views
Apache Kafka 3.1.0 Ops	5	2948	25 February 2022
Bringing Kafka Without Zookeeper Into Production [Kafka Summit 2022] Summit	0	3722	24 April 2022
Apache Kafka beyond Zookeeper Ops	3	3288	3 October 2021
Kafka 3 SASL_SSL ACL Ops	2	3163	17 December 2021
Questions about KRaft use in production Architecture and Design	0	1398	23 January 2024

Using Zookeeper with Kafka Broker 3.1.0

Related topics