In recent times there’s been a proliferation in CLIs for Apache Kafka, and instead of trawling through Reddit to find the announcements I thought I’d collect them here for reference
I’ve copied notes from the project’s github pages below to give an idea of the kind of functionality that they offer.
Think of it as netcat for Kafka.
The grandaddy of them all Still going strong, and my personal favourite.
Produce and Consume messages, and run Metadata queries.
Integrates beautifully with Linux pipes (
Apache Kafka® Configuration Made Easy
Use an approach like Database Migrations to manage your evolutionary configurations for:
The Kafka CLI for humans
- Consume kafka topics from a specific point in time
- Filter data based on content, record metadata, and record headers
- Supports offloading consumption of data to multiple lambda functions, kubernetes pods, etc. for parallelism
- Monitor consumer groups’ offsets.
- Upload avro schemas from a
.avdlfile using different naming strategies.
CLI for Apache Kafka, Zookeeper and Burrow Management.
- Search / Filter for Groups and Topics
- Show Topic Partition Details - Offsets, Replicas, Leaders, etc.
- Show Group Details - Lag, Offsets, Members, etc.
- Display / Modify Topics & Topic Configs
- Delete Consumer Groups
- Reset Partition Offsets for a Group
- Tail Topics in realtime
- Produce to a Specific Topic Partition or all Partitions at once
- Perform a Preferred Replica Election for a target topic or for all topics
- Increase / Decrease Replicas
- Migrate Topics between Brokers
- Query burrow details, Monitor burrow lag
- Explore Zookeeper Paths, Create and Delete Zookeeper Values
- Pass stdin to create Kafka messages or Zookeeper Values
A command-line interface for interaction with Apache Kafka
- command auto-completion for bash, zsh, fish shell including dynamic completion for e.g. topics or consumer groups.
- support for avro schemas
- Configuration of different contexts
- directly access kafka clusters inside your kubernetes cluster
GitHub - devshawn/kafka-shell: ⚡A supercharged, interactive Kafka shell built on top of the existing Kafka CLI tools.
A supercharged, interactive Kafka shell built on top of the existing Kafka CLI tools.
Kafka CLI inspired by kubectl & docker
An interactive CLI tool for managing Kafka topics.
- list topics, configurations, and offsets
- interactively create & delete topics
- modify a topics configuration
Kukulcan provides an API and different sort of REPLs to interact with your streams or administer your deployment.
a Kafka tool that likes JSON
- Consume messages
- Display topic information (e.g., with partition offset and leader info).
- Modify consumer group offsets
- JSON output for easy consumption with tools like kp or jq.
- JSON input to facilitate automation via tools like jsonify.
- No buffering of output.
- Binary keys and payloads can be passed and presented in base64 or hex encoding.
- Support for TLS authentication.
- Basic cluster admin functions: Create & delete topics.
GitHub - twmb/kcl: Your one stop shop to do anything with Kafka. Producing, consuming, transacting, administrating; 0.8.0 through 2.7.0+.
kcl is a complete, pure Go command line Kafka client. Think of it as your one stop shop to do anything you want to do with Kafka. Producing, consuming, transacting, administrating, and so on.
Unlike the small size of kafkacat, this binary is ~12M compiled. It is, however, still fast, has rich consuming and producing formatting options, and a complete Kafka administration interface.
GitHub - batchcorp/plumber: A swiss army knife CLI tool for interacting with Kafka, RabbitMQ and other messaging systems.
plumber is a CLI devtool for inspecting, piping, massaging and redirecting data in message systems like Kafka, RabbitMQ , GCP PubSub and many more.
It’s like curl for messaging systems.
- See what’s passing through your message systems
- Pipe data from one place to another
- Decode protobuf data in real-time