Kafka docker compose stopped working

Ops Containers
1

Hello,
Up until today I’ve been using the official kafka docker compose which I took from GitHub but today out of the blue it stopped working. Right after two seconds the zookeeper goes down and after a minute all other services go down to. It used to work all just fine. I didn’t change anything with it so I don’t know why it’s not working anymore. I’m including the error message below.
Here’s the docker compose file

---
version: '2'
services:
  zookeeper:
    image: confluentinc/cp-zookeeper:7.3.0
    hostname: zookeeper
    container_name: zookeeper
    ports:
      - "2181:2181"
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000

  broker:
    image: confluentinc/cp-server:7.3.0
    hostname: broker
    container_name: broker
    depends_on:
      - zookeeper
    ports:
      - "9092:9092"
      - "9101:9101"
      - "29092:29092"
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://broker:29092,PLAINTEXT_HOST://localhost:9092
      KAFKA_METRIC_REPORTERS: io.confluent.metrics.reporter.ConfluentMetricsReporter
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
      KAFKA_CONFLUENT_LICENSE_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_CONFLUENT_BALANCER_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
      KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
      KAFKA_JMX_PORT: 9101
      KAFKA_JMX_HOSTNAME: localhost
      KAFKA_CONFLUENT_SCHEMA_REGISTRY_URL: http://schema-registry:8081
      CONFLUENT_METRICS_REPORTER_BOOTSTRAP_SERVERS: broker:29092
      CONFLUENT_METRICS_REPORTER_TOPIC_REPLICAS: 1
      CONFLUENT_METRICS_ENABLE: 'true'
      CONFLUENT_SUPPORT_CUSTOMER_ID: 'anonymous'

  schema-registry:
    image: confluentinc/cp-schema-registry:7.3.0
    hostname: schema-registry
    container_name: schema-registry
    depends_on:
      - broker
    ports:
      - "8081:8081"
    environment:
      SCHEMA_REGISTRY_HOST_NAME: schema-registry
      SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS: 'broker:29092'
      SCHEMA_REGISTRY_LISTENERS: http://0.0.0.0:8081

  rest-proxy:
    image: confluentinc/cp-kafka-rest:7.3.0
    depends_on:
      - broker
      - schema-registry
    ports:
      - 8082:8082
    hostname: rest-proxy
    container_name: rest-proxy
    environment:
      KAFKA_REST_HOST_NAME: rest-proxy
      KAFKA_REST_BOOTSTRAP_SERVERS: 'broker:29092'
      KAFKA_REST_LISTENERS: "http://0.0.0.0:8082"
      KAFKA_REST_SCHEMA_REGISTRY_URL: 'http://schema-registry:8081'

  kafka-ui:
    container_name: kafka-ui
    image: provectuslabs/kafka-ui:latest
    ports:
      - 28080:8080
    environment:
      - DYNAMIC_CONFIG_ENABLED=true
    tty: true
    stdin_open: true

Error message in Zookeeper logs:

===> Configuring ...
2025-02-18 11:43:16 ===> Running preflight checks ... 
2025-02-18 11:43:16 ===> Check if /var/lib/zookeeper/data is writable ...
2025-02-18 11:43:16 ===> Check if /var/lib/zookeeper/log is writable ...
2025-02-18 11:43:16 ===> Launching ... 
2025-02-18 11:43:16 ===> Launching zookeeper ... 
2025-02-18 11:38:31     at java.base/java.util.HashMap$ValueSpliterator.forEachRemaining(HashMap.java:1693)
2025-02-18 11:38:31     at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
2025-02-18 11:38:31     at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
2025-02-18 11:38:31     at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
2025-02-18 11:38:31     at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
2025-02-18 11:38:31     at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
2025-02-18 11:38:31     at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
2025-02-18 11:38:31     at java.management/java.lang.management.ManagementFactory.getPlatformMBeanServer(ManagementFactory.java:488)
2025-02-18 11:38:31     at jdk.management.agent/sun.management.jmxremote.ConnectorBootstrap.startLocalConnectorServer(ConnectorBootstrap.java:542)
2025-02-18 11:38:31     at jdk.management.agent/jdk.internal.agent.Agent.startLocalManagementAgent(Agent.java:318)
2025-02-18 11:38:31     at jdk.management.agent/jdk.internal.agent.Agent.startAgent(Agent.java:450)
2025-02-18 11:38:31     at jdk.management.agent/jdk.internal.agent.Agent.startAgent(Agent.java:599)
2025-02-18 11:38:31 Exception thrown by the agent : java.lang.NullPointerException
2025-02-18 11:39:04 java.lang.NullPointerException
2025-02-18 11:39:04     at java.base/jdk.internal.platform.cgroupv2.CgroupV2Subsystem.getInstance(CgroupV2Subsystem.java:81)
2025-02-18 11:39:04     at java.base/jdk.internal.platform.CgroupSubsystemFactory.create(CgroupSubsystemFactory.java:113)
2025-02-18 11:39:04     at java.base/jdk.internal.platform.CgroupMetrics.getInstance(CgroupMetrics.java:167)
2

This looks like a Docker issue impacting a lot of examples. I am able to run the example with Docker Desktop 4.37.1 but hit the same stacktrace after upgrading to 4.38.0. I can also repro the ElasticSearch repro here after upgrading Docker Desktop.

What OS and Docker version do you have? You can downgrade here if you’re on 4.38.0.

1 Like
3

I’m on 4.38.0. This is not good :frowning: I might need to downgrade. Thanks for the update.

4

I’m hoping that this gets resolved in the next Docker Desktop release. Asked here in the GitHub issue.

Another option for you is to switch over to KRaft mode as this seems to only be impacting cp-zookeeper. You might want to consider that regardless of this issue, and also get on the latest version of Confluent Platform. I tested this example on Docker Desktop 4.38.0 and didn’t hit this problem.