3rd party command line tools for Apache Kafka

In recent times there’s been a proliferation in CLIs for Apache Kafka, and instead of trawling through Reddit to find the announcements I thought I’d collect them here for reference :slight_smile:

I’ve copied notes from the project’s github pages below to give an idea of the kind of functionality that they offer.


GitHub - edenhill/kafkacat: Generic command line non-JVM Apache Kafka producer and consumer

Think of it as netcat for Kafka.

The grandaddy of them all :smiley: Still going strong, and my personal favourite.

Produce and Consume messages, and run Metadata queries.

Integrates beautifully with Linux pipes (stdin / stdout)

GitHub - kattlo/kattlo-cli: Kattlo CLI Project

Apache Kafka® Configuration Made Easy

Use an approach like Database Migrations to manage your evolutionary configurations for:

  • Topics
  • Schemas
  • ACLs
  • ksqlDB
  • Connect
  • Cluster

GitHub - adevinta/zoe: The Kafka CLI for humans

The Kafka CLI for humans

  • Consume kafka topics from a specific point in time
  • Filter data based on content, record metadata, and record headers
  • Supports offloading consumption of data to multiple lambda functions, kubernetes pods, etc. for parallelism
  • Monitor consumer groups’ offsets.
  • Upload avro schemas from a .avsc or .avdl file using different naming strategies.

demo

GitHub - jbvmio/kafkactl: CLI for Apache Kafka Management -

CLI for Apache Kafka, Zookeeper and Burrow Management.

  • Search / Filter for Groups and Topics
  • Show Topic Partition Details - Offsets, Replicas, Leaders, etc.
  • Show Group Details - Lag, Offsets, Members, etc.
  • Display / Modify Topics & Topic Configs
  • Delete Consumer Groups
  • Reset Partition Offsets for a Group
  • Tail Topics in realtime
  • Produce to a Specific Topic Partition or all Partitions at once
  • Perform a Preferred Replica Election for a target topic or for all topics
  • Increase / Decrease Replicas
  • Migrate Topics between Brokers
  • Query burrow details, Monitor burrow lag
  • Explore Zookeeper Paths, Create and Delete Zookeeper Values
  • Pass stdin to create Kafka messages or Zookeeper Values

GitHub - deviceinsight/kafkactl: Command Line Tool for managing Apache Kafka

A command-line interface for interaction with Apache Kafka

  • command auto-completion for bash, zsh, fish shell including dynamic completion for e.g. topics or consumer groups.
  • support for avro schemas
  • Configuration of different contexts
  • directly access kafka clusters inside your kubernetes cluster

asciicast

GitHub - devshawn/kafka-shell: ⚡A supercharged, interactive Kafka shell built on top of the existing Kafka CLI tools.

A supercharged, interactive Kafka shell built on top of the existing Kafka CLI tools.

GitHub - infinimesh/kaf: Modern CLI for Apache Kafka, written in Go.

Kafka CLI inspired by kubectl & docker

GitHub - kdrakon/topiks: An interactive CLI tool for managing Kafka topics

An interactive CLI tool for managing Kafka topics.

  • list topics, configurations, and offsets
  • interactively create & delete topics
  • modify a topics configuration

GitHub - mmolimar/kukulcan: A REPL for Apache Kafka

Kukulcan provides an API and different sort of REPLs to interact with your streams or administer your deployment.

GitHub - fgeller/kt: Kafka command line tool that likes JSON

a Kafka tool that likes JSON

  • Consume messages
  • Display topic information (e.g., with partition offset and leader info).
  • Modify consumer group offsets
  • JSON output for easy consumption with tools like kp or jq.
  • JSON input to facilitate automation via tools like jsonify.
  • No buffering of output.
  • Binary keys and payloads can be passed and presented in base64 or hex encoding.
  • Support for TLS authentication.
  • Basic cluster admin functions: Create & delete topics.

GitHub - twmb/kcl: Your one stop shop to do anything with Kafka. Producing, consuming, transacting, administrating; 0.8.0 through 2.7.0+.

kcl is a complete, pure Go command line Kafka client. Think of it as your one stop shop to do anything you want to do with Kafka. Producing, consuming, transacting, administrating, and so on.

Unlike the small size of kafkacat, this binary is ~12M compiled. It is, however, still fast, has rich consuming and producing formatting options, and a complete Kafka administration interface.

GitHub - batchcorp/plumber: A swiss army knife CLI tool for interacting with Kafka, RabbitMQ and other messaging systems.

plumber is a CLI devtool for inspecting, piping, massaging and redirecting data in message systems like Kafka, RabbitMQ , GCP PubSub and many more.

It’s like curl for messaging systems.

  • See what’s passing through your message systems
  • Pipe data from one place to another
  • Decode protobuf data in real-time
3 Likes

Another one! :slight_smile:

1 Like

Hey @mmolimar, good to see you here on the forum :slight_smile: Thanks for the link, I’ll add it to the original post.

1 Like

(different from the other kafkactl above) but has nice configs for different authentication types.

1 Like

Thanks @sarwar-confluent , I’ve added it to the original post

One more which is the only one that covers the whole Kafka API as far as I know: GitHub - twmb/kcl: Your one stop shop to do anything with Kafka. Producing, consuming, transacting, administrating; 0.8.0 through 2.7.0+.

Thanks @weeco - I thought I had that one already but evidently not :smiley: I’ll add it now

1 Like