Hello,
I am trying to set up SASL with the Kraft mode in Kafka. I have a Docker Compose setup that works fine without ACLs, and I can connect to Kafka using producers. However, when I try to implement ACLs, I run into issues.
Working Docker Compose without ACLs:
---
version: '2'
services:
broker:
image: confluentinc/cp-kafka:7.5.0
hostname: broker
container_name: broker
ports:
- "9092:9092"
- "9101:9101"
environment:
KAFKA_NODE_ID: 1
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
KAFKA_JMX_PORT: 9101
KAFKA_JMX_HOSTNAME: localhost
KAFKA_PROCESS_ROLES: 'broker,controller'
KAFKA_CONTROLLER_QUORUM_VOTERS: '1@broker:29093'
KAFKA_CONTROLLER_LISTENER_NAMES: 'CONTROLLER'
KAFKA_LOG_DIRS: '/tmp/kraft-combined-logs'
# Replace CLUSTER_ID with a unique base64 UUID using "bin/kafka-storage.sh random-uuid"
# See https://docs.confluent.io/kafka/operations-tools/kafka-tools.html#kafka-storage-sh
CLUSTER_ID: 'MkU3OEVBNTcwNTJENDM2Qk'
#KAFKA_LISTENERS: SASL_PLAINTEXT://:9092
KAFKA_SECURITY_INTER_BROKER_PROTOCOL: SASL_PLAINTEXT
KAFKA_SASL_MECHANISM_INTER_BROKER_PROTOCOL: PLAIN
KAFKA_SASL_ENABLED_MECHANISMS: PLAIN
KAFKA_OPTS: -Djava.security.auth.login.config=/etc/kafka/broker_jaas.conf
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: 'CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT,SASL_PLAINTEXT:SASL_PLAINTEXT'
KAFKA_LISTENERS: 'PLAINTEXT://broker:29092,CONTROLLER://broker:29093,PLAINTEXT_HOST://0.0.0.0:9092,SASL_PLAINTEXT://broker:9093'
KAFKA_ADVERTISED_LISTENERS: 'PLAINTEXT://broker:29092,PLAINTEXT_HOST://localhost:9092,SASL_PLAINTEXT://broker:9093'
volumes:
- ./broker_jaas.conf:/etc/kafka/broker_jaas.conf
Docker Compose with ACLs (causing issues):
version: '2'
services:
broker:
image: confluentinc/cp-kafka:7.5.0
hostname: broker
container_name: broker
ports:
- "9092:9092"
- "9101:9101"
environment:
KAFKA_NODE_ID: 1
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
KAFKA_JMX_PORT: 9101
KAFKA_JMX_HOSTNAME: localhost
KAFKA_PROCESS_ROLES: 'broker,controller'
KAFKA_CONTROLLER_QUORUM_VOTERS: '1@broker:29093'
KAFKA_CONTROLLER_LISTENER_NAMES: 'CONTROLLER'
KAFKA_LOG_DIRS: '/tmp/kraft-combined-logs'
# Replace CLUSTER_ID with a unique base64 UUID using "bin/kafka-storage.sh random-uuid"
# See https://docs.confluent.io/kafka/operations-tools/kafka-tools.html#kafka-storage-sh
CLUSTER_ID: 'MkU3OEVBNTcwNTJENDM2Qk'
#KAFKA_SECURITY_INTER_BROKER_PROTOCOL: SASL_PLAINTEXT
KAFKA_SASL_MECHANISM_INTER_BROKER_PROTOCOL: PLAIN
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: 'CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT,SASL_PLAINTEXT:SASL_PLAINTEXT'
KAFKA_LISTENERS: 'PLAINTEXT://broker:29092,CONTROLLER://broker:29093,PLAINTEXT_HOST://0.0.0.0:9092,SASL_PLAINTEXT://broker:9093'
KAFKA_ADVERTISED_LISTENERS: 'PLAINTEXT://broker:29092,PLAINTEXT_HOST://localhost:9092,SASL_PLAINTEXT://broker:9093'
KAFKA_AUTHORIZER_CLASS_NAME: org.apache.kafka.metadata.authorizer.StandardAuthorizer
KAFKA_SUPER_USERS: User:broker
KAFKA_SASL_ENABLED_MECHANISMS: PLAIN
KAFKA_OPTS: -Djava.security.auth.login.config=/etc/kafka/broker_jaas.conf
KAFKA_INTER_BROKER_LISTENER_NAME: SASL_PLAINTEXT
volumes:
- ./broker_jaas.conf:/etc/kafka/broker_jaas.conf
JAAS Configuration (broker_jaas.conf):
KafkaServer {
org.apache.kafka.common.security.plain.PlainLoginModule required
username="broker"
password="broker-password"
user_broker="broker-password";
};
When I try to bring up the Docker container with the ACL configuration, I get the following error:
broker | org.apache.kafka.common.errors.ClusterAuthorizationException: Request Request(processor=0, connectionId=172.25.0.2:29093-172.25.0.2:42614-0, session=Session(User:ANONYMOUS,/172.25.0.2), listenerName=ListenerName(CONTROLLER), securityProtocol=PLAINTEXT, buffer=null, envelope=None) is not authorized.
broker | [2023-10-12 20:22:29,692] INFO [BrokerLifecycleManager id=1] Unable to register broker 1 because the controller returned error CLUSTER_AUTHORIZATION_FAILED (kafka.server.BrokerLifecycleManager)
broker | [2023-10-12 20:22:38,721] ERROR [ControllerApis nodeId=1] Unexpected error handling request RequestHeader(apiKey=BROKER_REGISTRATION, apiVersion=1, clientId=1, correlationId=13, headerVersion=2) -- BrokerRegistrationRequestData(brokerId=1, clusterId='MkU3OEVBNTcwNTJENDM2Qg', incarnationId=bext2mlpSe22Jmw2YvcQ6w, listeners=[Listener(name='PLAINTEXT', host='broker', port=29092, securityProtocol=0), Listener(name='PLAINTEXT_HOST', host='localhost', port=9092, securityProtocol=0), Listener(name='SASL_PLAINTEXT', host='broker', port=9093, securityProtocol=2)], features=[Feature(name='metadata.version', minSupportedVersion=1, maxSupportedVersion=11)], rack=null, isMigratingZkBroker=false) with context RequestContext(header=RequestHeader(apiKey=BROKER_REGISTRATION, apiVersion=1, clientId=1, correlationId=13, headerVersion=2), connectionId='172.25.0.2:29093-172.25.0.2:42614-0', clientAddress=/172.25.0.2, principal=User:ANONYMOUS, listenerName=ListenerName(CONTROLLER), securityProtocol=PLAINTEXT, clientInformation=ClientInformation(softwareName=apache-kafka-java, softwareVersion=7.5.0-ccs), fromPrivilegedListener=false, principalSerde=Optional[org.apache.kafka.common.security.authenticator.DefaultKafkaPrincipalBuilder@4eaf9846]) (kafka.server.ControllerApis)
From the error, it seems like my broker is still trying to register as User:ANONYMOUS. I’m not sure why this is happening. I’m new to Kafka and would appreciate any guidance on this issue.
Thank you!