Docker Compose using SASL_PLAINTEXT and SCRAM-SHA-256

Ops Containers
1

Hi all! I have a pretty niche setup I need to get going and I have not been able to find any specific documentation/threads that cover it. Hopefully someone out there can help :sweat_smile:

To keep it brief, I am attempting to write a Docker Compose that can spin up 1 Zookeeper with 2-3 Brokers that is using SASL_PLAINTEXT with SCRAM-SHA-256 as the consumer authentication. This should also be done using a JAAS config file for specifying the username and password. The version must also be 5.5.X for the Confluent images.

Why this specific setup you might ask? I am developing a service to connect to an environment that already exists in a customerā€™s premise. This is to get a replicated setup that my team can utilise for development and prototyping, as it is our first time getting hands on with Kafka.

I think I have turned every link in Google purple at this stage, and read over pages and pages of Confluent documentation, but nothing out there seems to combine all of these things in one go. The main things tripping me up are which environment variables are actually exposed for this version and how to format them (in regards to the JAAS config, specifying SCRAM and setting up the protocols for the consumers etc.).

This is kind of where Iā€™m at with it now, as you can tell Iā€™ve started to Frankenstein in the SASL_PLAINTEXT and JAAS, not sure if this is on the right track (please correct my mistakes). Iā€™m confused by how to mention the SCRAM mode, because Iā€™ve seen it implemented by overriding the startup command and sometimes by using env variablesā€¦ itā€™s a bit inconsistent.

version: '2'
services:
  zookeeper:
    image: confluentinc/cp-zookeeper:5.5.6
    hostname: zookeeper
    container_name: zookeeper
    restart: unless-stopped
    ports:
      - "2181:2181"
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000

  broker-1:
    image: confluentinc/cp-kafka:5.5.6
    hostname: broker-1
    container_name: broker-1
    depends_on:
      - zookeeper
    restart: unless-stopped
    ports:
      - "9092:9092"
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INTERNAL:PLAINTEXT,EXTERNAL:SASL_PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL
      KAFKA_ADVERTISED_LISTENERS: INTERNAL://broker-1:29092,EXTERNAL://localhost:9092
      KAFKA_DEFAULT_REPLICATION_FACTOR: 2
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_NUM_PARTITIONS: 3
      KAFKA_OPTS: "-Djava.security.auth.login.config=/etc/kafka/secrets/kafka_server_jaas.conf"
    volumes:
        - "./secrets:/etc/kafka/secrets"

If you know how to set up something like this, please reach out to me :pray: I havenā€™t joined the Slack yet but if you want to share your knowledge over a more reactive messaging platform (Discord also works), let me know. Otherwise sharing this knowledge with other community members will help also!

Iā€™ll try report back here if I figure it out myself!

2

Hi,

did you already consider the examples provided by
@vdesabou
available at GitHub - vdesabou/kafka-docker-playground: šŸ³ Fully automated Apache KafkaĀ® and Confluent Docker based examples

especially kafka-docker-playground/environment/sasl-scram at master Ā· vdesabou/kafka-docker-playground Ā· GitHub

HTH,
Michael

2 Likes
3

What a gem this is! I had not found Vincentā€™s GitHub yet during my searching :astonished:

This should hopefully cover it, youā€™re a bloody legend. Iā€™ll report back if thereā€™s something incompatible, but I think all boxes might be ticked here.

1 Like
4

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.