Self-hosted Kafka with KRaft, SSL and SASL (scram-sha-256)

Ops
4

I’ve managed to get Kafka v3.6.0 working with SASL_SSL + PLAINTEXT, with a binary build from Apache website. Should also work with confluent-server. Here’s my server.properties:

process.roles=broker,controller
node.id=1
controller.quorum.voters=1@kafka-test-01:9093,2@kafka-test-02:9093,3@kafka-test-03:9093
listeners=BROKER://:9092,CONTROLLER://:9093
advertised.listeners=BROKER://:9092
inter.broker.listener.name=BROKER
controller.listener.names=CONTROLLER
listener.security.protocol.map=BROKER:SASL_SSL,CONTROLLER:SASL_SSL
listener.name.controller.ssl.client.auth=required
listener.name.broker.ssl.client.auth=required
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
ssl.truststore.location=/etc/kafka/jks/keystore.jks
ssl.truststore.password=keystore-pass
ssl.keystore.location=/etc/kafka/jks/keystore.jks
ssl.keystore.password=keystore-pass
ssl.client.auth=required
authorizer.class.name=org.apache.kafka.metadata.authorizer.StandardAuthorizer
super.users=User:admin
sasl.enabled.mechanisms=PLAIN
sasl.mechanism.inter.broker.protocol=PLAIN
sasl.mechanism.controller.protocol=PLAIN
listener.name.controller.plain.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
    username="admin" \
    password="secret000" \
    user_admin="secret000";
listener.name.broker.plain.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
    username="admin" \
    password="secret000" \
    user_admin="secret000";
log.dirs=/var/lib/kafka/kraft-combined-logs
num.partitions=1
num.recovery.threads.per.data.dir=1
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000

Logs dir was bootstrapped with:

sudo -u kafka /opt/kafka/bin/kafka-storage.sh format \
  -t "VfHmcdoORCC689JspCEe_w" -c /etc/kafka/server.properties

PLAIN is not really as comfortable and secure as SCRAM auth – you can only add users with full restart of the cluster. A line like below should be added into both *.sasl.jaas.config=org.apache.* properties for each user on each node:

    user_username="password" \  # or ";" for the last user

Still waiting for the solution for SCRAM authentication.