Kafka KRAFT SASL_PLAINTEXT

Ops Containers
1

Hey, guys.

I’m trying to get Kafka in “kraft” mode up n’ running with SASL_PLAINTEXT

I’ve been able to get a functioning kafka broker/controller up n’ running locally, with this
config/kraft/server.properties

process.roles=broker,controller
node.id=1
controller.quorum.voters=1@localhost:9093
listeners=PLAINTEXT://:9092,CONTROLLER://:9093
inter.broker.listener.name=PLAINTEXT
advertised.listeners=PLAINTEXT://:9092
controller.listener.names=CONTROLLER
listener.security.protocol.map=CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT

I’ve bound ports from the kafka docker container 9092 to 9092 on the host

kafka-topics.sh --list --bootstrap-server localhost:9092 and
kafka-topics.sh --bootstrap-server localhost:9092 --topic test --create --partitions 2 --replication-factor 1 works like a charm.

docker container logs looks like they should.

I need some users to handle ACL on our topics, so I thought it was easy to just replace all PLAINTEXT fields with SASL_PLAINTEXT, I was wrong :smile:

We handle encryption on another level, so SASL_PLAINTEXT is sufficient.

config/kraft/sasl_server.properties

process.roles=broker,controller
node.id=1
controller.quorum.voters=1@localhost:9094
listeners=SASL_PLAINTEXT://:9092,CONTROLLER://:9094
;advertised.listeners=SASL_PLAINTEXT://:9092
controller.listener.names=CONTROLLER
listener.security.protocol.map=CONTROLLER:SASL_PLAINTEXT,SASL_PLAINTEXT:SASL_PLAINTEXT

sasl.enabled.mechanisms=PLAIN
sasl.mechanism.inter.broker.protocol=PLAIN
security.inter.broker.protocol=SASL_PLAINTEXT

sasl.mechanism=PLAIN
security.protocol=SASL_PLAINTEXT
listener.name.controller.plain.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
  username="admin" \
  password="admin-secret" \
  user_admin="admin-secret";
plain.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
   username="admin" \
   password="admin-secret";

I’m getting this error

java.lang.IllegalArgumentException: Could not find a 'KafkaServer' or 'controller.KafkaServer' entry in the JAAS configuration. System property 'java.security.auth.login.config' is not set
2 
process.roles=$KAFKA_PROCESS_ROLES
node.id=$KAFKA_NODE_ID
controller.quorum.voters=$KAFKA_CONTROLLER_QUORUM_VOTERS

listeners=BROKER://:9092,CONTROLLER://:9093
advertised.listeners=BROKER://:9092
listener.security.protocol.map=BROKER:SASL_PLAINTEXT,CONTROLLER:SASL_PLAINTEXT

inter.broker.listener.name=BROKER
controller.listener.names=CONTROLLER

sasl.enabled.mechanisms=PLAIN
sasl.mechanism.controller.protocol=PLAIN
sasl.mechanism.inter.broker.protocol=PLAIN

listener.name.broker.plain.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
    username="admin" \
    password="$KAFKA_ADMIN_PASSWORD" \
    user_admin="$KAFKA_ADMIN_PASSWORD";

listener.name.controller.plain.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
    username="admin" \
    password="$KAFKA_ADMIN_PASSWORD" \
    user_admin="$KAFKA_ADMIN_PASSWORD";

Here is a working configuration.

3

Thank you for sharing this. It returned my sanity after working on this for days. Kafka gives the worst errors when you have even the slightest configuration skew. I wanted to share An addition I made to the configuration to use SCRAM-SHA-256
I did encounter one issue. You cannot set the controller to use SCRAM-SHA-256, or I haven’t been able to figure out the configurations required.

version: "2"
services:
  kafka0:
    image: confluentinc/cp-kafka:7.6.0
    hostname: kafka0
    container_name: kafka0
    ports:
      - "9092:9092"
      - "9093:9093"
    environment:
      CLUSTER_ID: 'MkU3OEVBNTcwNTJENDM2Qk'
      KAFKA_NODE_ID: 1
      KAFKA_PROCESS_ROLES: "broker,controller"
      KAFKA_CONTROLLER_QUORUM_VOTERS: "1@kafka0:9093"
      KAFKA_LISTENERS: BROKER://kafka0:9092,CONTROLLER://kafka0:9093
      KAFKA_ADVERTISED_LISTENERS: BROKER://kafka0:9092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: BROKER:SASL_PLAINTEXT,CONTROLLER:SASL_PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: BROKER
      KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER
      KAFKA_SASL_ENABLED_MECHANISMS: SCRAM-SHA-256,PLAIN
      KAFKA_SASL_MECHANISM_CONTROLLER_PROTOCOL: PLAIN
      KAFKA_SASL_MECHANISM_INTER_BROKER_PROTOCOL: SCRAM-SHA-256
      KAFKA_LISTENER_NAME_BROKER_SCRAM-SHA-256_SASL_JAAS_CONFIG: org.apache.kafka.common.security.scram.ScramLoginModule required username="admin" password="admin-secret" user_admin="admin-secret" ;
      KAFKA_LISTENER_NAME_BROKER_PLAIN_SASL_JAAS_CONFIG: org.apache.kafka.common.security.plain.PlainLoginModule required username="admin" password="admin-secret" user_admin="admin-secret" ;
      KAFKA_LISTENER_NAME_CONTROLLER_PLAIN_SASL_JAAS_CONFIG: org.apache.kafka.common.security.plain.PlainLoginModule required username="admin" password="admin-secret" user_admin="admin-secret" ;
      KAFKA_LISTENER_NAME_CONTROLLER_SCRAM-SHA-256_SASL_JAAS_CONFIG: org.apache.kafka.common.security.scram.ScramLoginModule required username="admin" password="admin-secret" user_admin="admin-secret" ;
      KAFKA_OPTS: "-Djava.security.debug=gssloginconfig,configfile,configparser,logincontext"