# What is wrong with my cluster setup?

**URL:** <https://forum.confluent.io/t/what-is-wrong-with-my-cluster-setup/3681>\
**Category:** Ops\
**Created:** [15 December 2021 22:37 UTC](https://forum.confluent.io/t/what-is-wrong-with-my-cluster-setup/3681 "2021-12-15T22:37:22Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![programista4k](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/programista4k/32/767_2.png) [@programista4k](https://forum.confluent.io/u/programista4k)\
**Post date:** [15 December 2021 22:37 UTC](https://forum.confluent.io/t/what-is-wrong-with-my-cluster-setup/3681/1 "2021-12-15T22:37:22Z")

</div>

Hello frens,  
I want to run kafka cluster with docker compose. It should consist of 3 kafka brokers.  
This is how it looks like

```auto
version: '3.8'
services:

  zookeeper-1:
    image: confluentinc/cp-zookeeper:latest
    container_name: zookeeper-1
    ports:
      - '12181:2181'
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000

  zookeeper-2:
    image: confluentinc/cp-zookeeper:latest
    container_name: zookeeper-2
    ports:
      - '22181:2181'
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000
      
  zookeeper-3:
    image: confluentinc/cp-zookeeper:latest
    container_name: zookeeper-3
    ports:
      - '32181:2181'    
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000
      
  kafka-1:
    image: confluentinc/cp-kafka:latest
    container_name: kafka-1
    ports:
      - '19092:19092'
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper-1:2181,zookeeper-2:2181,zookeeper-3:2181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka-1:9092,PLAINTEXT_HOST://localhost:19092
      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      BOOTSTRAP_SERVERS: kafka1:19092,kafka2:29092,kafka3:39092
    depends_on:
      - zookeeper-1
      - zookeeper-2
      - zookeeper-3
      
  kafka-2:
    image: confluentinc/cp-kafka:latest
    container_name: kafka-2
    ports:
      - '29092:29092'
    environment:
      KAFKA_BROKER_ID: 2
      KAFKA_ZOOKEEPER_CONNECT: zookeeper-1:2181,zookeeper-2:2181,zookeeper-3:2181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka-2:9092,PLAINTEXT_HOST://localhost:29092
      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      BOOTSTRAP_SERVERS: kafka1:19092,kafka2:29092,kafka3:39092
    depends_on:
      - zookeeper-1
      - zookeeper-2
      - zookeeper-3
      
  kafka-3:
    image: confluentinc/cp-kafka:latest
    container_name: kafka-3
    ports:
      - '39092:39092'
    environment:
      KAFKA_BROKER_ID: 3
      KAFKA_ZOOKEEPER_CONNECT: zookeeper-1:2181,zookeeper-2:2181,zookeeper-3:2181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka-3:9092,PLAINTEXT_HOST://localhost:39092
      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      BOOTSTRAP_SERVERS: kafka1:19092,kafka2:29092,kafka3:39092
    depends_on:
      - zookeeper-1
      - zookeeper-2
      - zookeeper-3

```

now when I try to create topic with 3 partitions on it I get error:

```auto
INFO [Admin Manager on Broker 1]: Error processing create topic request   
CreatableTopic(name='__transaction_state', numPartitions=50, replicationFactor=3,  
assignments=[], configs=[CreateableTopicConfig(name='compression.type',  
value='uncompressed'), CreateableTopicConfig(name='cleanup.policy', value='compact'),  
CreateableTopicConfig(name='min.insync.replicas', value='2'),  
CreateableTopicConfig(name='segment.bytes', value='104857600'),  
CreateableTopicConfig(name='unclean.leader.election.enable', value='false')]) 
(kafka.server.ZkAdminManager)

kafka-1 | 
org.apache.kafka.common.errors.InvalidReplicationFactorException: 
Replication factor: 3 larger than available brokers: 2.

```

Why it sees only 2 brokers, not 3? And sometimes the error message indicates it sees only 1 broker.

---

<div class="post-metadata">

**Author:** ![mmuehlbeyer](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/mmuehlbeyer/32/1088_2.png) [@mmuehlbeyer](https://forum.confluent.io/u/mmuehlbeyer)\
**Post date:** [16 December 2021 06:25 UTC](https://forum.confluent.io/t/what-is-wrong-with-my-cluster-setup/3681/2 "2021-12-16T06:25:21Z")

</div>

Hi @programista4k

did you already check the docker logs of the kafka containers?

After a short view on your compose it seems that your advertised listener configuration is not correct  
the port should differ for each container

> [@programista4k](#):
>
> ` KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka-1:9092,PLAINTEXT_HOST://localhost:19092`

> [@programista4k](#):
>
> ` KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka-2:9092,PLAINTEXT_HOST://localhost:29092`

> [@programista4k](#):
>
> ` KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka-3:9092,PLAINTEXT_HOST://localhost:39092`

should be

```auto
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka-1:9092,PLAINTEXT_HOST://localhost:19092

KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka-2:9093,PLAINTEXT_HOST://localhost:29092

KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka-3:9094,PLAINTEXT_HOST://localhost:39092

```

for reference there is a great blog post by @rmoff about advertised\_listener

> **[Kafka Listeners - Explained](https://rmoff.net/2018/08/02/kafka-listeners-explained/)**
>
> How to connect clients to Kafka hosted in separate networks, such as Docker, AWS EC2, GCP, Azure, etc

HTH,  
Michael

---

<div class="post-metadata">

**Author:** ![programista4k](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/programista4k/32/767_2.png) [@programista4k](https://forum.confluent.io/u/programista4k)\
**Post date:** [22 December 2021 02:01 UTC](https://forum.confluent.io/t/what-is-wrong-with-my-cluster-setup/3681/3 "2021-12-22T02:01:51Z")

</div>

Yes Sir, I have read the Kafkas’ and Zookeepers’ logs and there is no error instead of `Replication factor: 3 larger than available brokers: 2.` when I try to create topic with 3 replicas.

I have fixed the ports as ya suggested. Now it looks like this and the problem is still alive:

```auto
version: '3.8'
services:

  zookeeper-1:
    image: confluentinc/cp-zookeeper:latest
    container_name: zookeeper-1
    ports:
      - '12181:2181'
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000

  zookeeper-2:
    image: confluentinc/cp-zookeeper:latest
    container_name: zookeeper-2
    ports:
      - '22181:2181'
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000

  zookeeper-3:
    image: confluentinc/cp-zookeeper:latest
    container_name: zookeeper-3
    ports:
      - '32181:2181'    
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000
      

  kafka-1:
    image: confluentinc/cp-kafka:latest
    container_name: kafka-1
    ports:
      - '19092:19092'
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper-1:2181,zookeeper-2:2181,zookeeper-3:2181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka-1:9092,PLAINTEXT_HOST://localhost:19092
      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      BOOTSTRAP_SERVERS: kafka1:19092,kafka2:29092,kafka3:39092
      KAFKA_LOG_MESSAGE_TIMESTAMP_TYPE: LogAppendTime
    depends_on:
      - zookeeper-1
      - zookeeper-2
      - zookeeper-3

  kafka-2:
    image: confluentinc/cp-kafka:latest
    container_name: kafka-2
    ports:
      - '29092:29092'
    environment:
      KAFKA_BROKER_ID: 2
      KAFKA_ZOOKEEPER_CONNECT: zookeeper-1:2181,zookeeper-2:2181,zookeeper-3:2181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka-2:9093,PLAINTEXT_HOST://localhost:29092
      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      BOOTSTRAP_SERVERS: kafka1:19092,kafka2:29092,kafka3:39092
      KAFKA_LOG_MESSAGE_TIMESTAMP_TYPE: LogAppendTime
    depends_on:
      - zookeeper-1
      - zookeeper-2
      - zookeeper-3

  kafka-3:
    image: confluentinc/cp-kafka:latest
    container_name: kafka-3
    ports:
      - '39092:39092'
    environment:
      KAFKA_BROKER_ID: 3
      KAFKA_ZOOKEEPER_CONNECT: zookeeper-1:2181,zookeeper-2:2181,zookeeper-3:2181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka-3:9094,PLAINTEXT_HOST://localhost:39092
      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      BOOTSTRAP_SERVERS: kafka1:19092,kafka2:29092,kafka3:39092
      KAFKA_LOG_MESSAGE_TIMESTAMP_TYPE: LogAppendTime
    depends_on:
      - zookeeper-1
      - zookeeper-2
      - zookeeper-3

```

---

<div class="post-metadata">

**Author:** ![mmuehlbeyer](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/mmuehlbeyer/32/1088_2.png) [@mmuehlbeyer](https://forum.confluent.io/u/mmuehlbeyer)\
**Post date:** [22 December 2021 09:25 UTC](https://forum.confluent.io/t/what-is-wrong-with-my-cluster-setup/3681/4 "2021-12-22T09:25:16Z")

</div>

hey,  
I’ve tested around with your docker-compose and detected some issues.  
would you please try with the follwing docker-compose.yml

```auto
version: '3.8'
services:

  zookeeper-1:
    image: confluentinc/cp-zookeeper:latest
    container_name: zookeeper-1
    ports:
      - '2181:2181'
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000

  zookeeper-2:
    image: confluentinc/cp-zookeeper:latest
    container_name: zookeeper-2
    ports:
      - '2182:2182'
    environment:
      ZOOKEEPER_CLIENT_PORT: 2182
      ZOOKEEPER_TICK_TIME: 2000

  zookeeper-3:
    image: confluentinc/cp-zookeeper:latest
    container_name: zookeeper-3
    ports:
      - '2183:2183'
    environment:
      ZOOKEEPER_CLIENT_PORT: 2183
      ZOOKEEPER_TICK_TIME: 2000

  kafka-1:
    image: confluentinc/cp-kafka:latest
    container_name: kafka-1
    ports:
      - '19092:19092'
      - '9092:9092'
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper-1:2181
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: LISTENER_INTERNAL:PLAINTEXT,LISTENER_EXTERNAL:PLAINTEXT
      KAFKA_LISTENERS: LISTENER_INTERNAL://kafka-1:19092,LISTENER_EXTERNAL://kafka-1:9092
      KAFKA_ADVERTISED_LISTENERS: LISTENER_INTERNAL://kafka-1:19092,LISTENER_EXTERNAL://kafka-1:9092
      KAFKA_INTER_BROKER_LISTENER_NAME: LISTENER_INTERNAL
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_LOG_MESSAGE_TIMESTAMP_TYPE: LogAppendTime
    depends_on:
      - zookeeper-1
      - zookeeper-2
      - zookeeper-3

  kafka-2:
    image: confluentinc/cp-kafka:latest
    container_name: kafka-2
    ports:
      - '29092:29092'
      - '9093:9093'
    environment:
      KAFKA_BROKER_ID: 2
      KAFKA_ZOOKEEPER_CONNECT: zookeeper-1:2181
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: LISTENER_INTERNAL:PLAINTEXT,LISTENER_EXTERNAL:PLAINTEXT
      KAFKA_LISTENERS: LISTENER_INTERNAL://kafka-2:29092,LISTENER_EXTERNAL://kafka-2:9093
      KAFKA_ADVERTISED_LISTENERS: LISTENER_INTERNAL://kafka-2:29092,LISTENER_EXTERNAL://kafka-2:9093
      KAFKA_INTER_BROKER_LISTENER_NAME: LISTENER_INTERNAL
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_LOG_MESSAGE_TIMESTAMP_TYPE: LogAppendTime
    depends_on:
      - zookeeper-1
      - zookeeper-2
      - zookeeper-3

  kafka-3:
    image: confluentinc/cp-kafka:latest
    container_name: kafka-3
    ports:
      - '9094:9094'
      - '39092:39092'
    environment:
      KAFKA_BROKER_ID: 3
      KAFKA_ZOOKEEPER_CONNECT: zookeeper-1:2181
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: LISTENER_INTERNAL:PLAINTEXT,LISTENER_EXTERNAL:PLAINTEXT
      KAFKA_LISTENERS: LISTENER_INTERNAL://kafka-3:39092,LISTENER_EXTERNAL://kafka-3:9094
      KAFKA_ADVERTISED_LISTENERS: LISTENER_INTERNAL://kafka-3:39092,LISTENER_EXTERNAL://kafka-3:9094
      KAFKA_INTER_BROKER_LISTENER_NAME: LISTENER_INTERNAL
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_LOG_MESSAGE_TIMESTAMP_TYPE: LogAppendTime
    depends_on:
      - zookeeper-1
      - zookeeper-2
      - zookeeper-3

```

the following works (at least in my dev env)

```auto
kafka-topics --create --partitions 3 --replication-factor 3 --topic quickstart-events --bootstrap-server kafka-1:9092

```

---

<div class="post-metadata">

**Author:** ![programista4k](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/programista4k/32/767_2.png) [@programista4k](https://forum.confluent.io/u/programista4k)\
**Post date:** [22 December 2021 16:38 UTC](https://forum.confluent.io/t/what-is-wrong-with-my-cluster-setup/3681/5 "2021-12-22T16:38:53Z")

</div>

Thank you Sir.

 ![zjsx48hhls541](https://us1.discourse-cdn.com/flex019/uploads/confluentcommunity/original/2X/7/727bb078fddba314d1e21e531a0d72a65a00f3a9.jpeg)

---

<div class="post-metadata">

**Author:** ![programista4k](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/programista4k/32/767_2.png) [@programista4k](https://forum.confluent.io/u/programista4k)\
**Post date:** [22 December 2021 19:57 UTC](https://forum.confluent.io/t/what-is-wrong-with-my-cluster-setup/3681/6 "2021-12-22T19:57:55Z")

</div>

And what about those zookeepers, why only first zookeeper is used, Fren?  
Shouldn’t it be:  
`zookeeper-1` for `kafka-1`,  
`zookeeper-2` for `kafka-2`  
etc.?

or `zookeper-1,zookeeper-2,zookeeper-3` for every kafka broker?

---

<div class="post-metadata">

**Author:** ![mmuehlbeyer](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/mmuehlbeyer/32/1088_2.png) [@mmuehlbeyer](https://forum.confluent.io/u/mmuehlbeyer)\
**Post date:** [23 December 2021 06:02 UTC](https://forum.confluent.io/t/what-is-wrong-with-my-cluster-setup/3681/7 "2021-12-23T06:02:19Z")

</div>

> [@programista4k](#):
>
> And what about those zookeepers, why only first zookeeper is used, Fren?

just for ease of configuration

you could also set each zookeeper for every kafka broker

---

<div class="post-metadata">

**Author:** ![programista4k](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/programista4k/32/767_2.png) [@programista4k](https://forum.confluent.io/u/programista4k)\
**Post date:** [24 December 2021 10:06 UTC](https://forum.confluent.io/t/what-is-wrong-with-my-cluster-setup/3681/8 "2021-12-24T10:06:24Z")

</div>

Tank you, Sir.

 ![comment_jsx9s4NiwfehY5RGjqU2lJ7WdY6SlX1v](https://us1.discourse-cdn.com/flex019/uploads/confluentcommunity/original/2X/d/d1c8d93bb09dc407d9600b07c8f88fa8f3794030.jpeg)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex019/uploads/confluentcommunity/original/1X/c49438c90c9df282e9996fdf6971be890c71b65a.svg) [@system](https://forum.confluent.io/u/system)\
**Post date:** [31 December 2021 10:07 UTC](https://forum.confluent.io/t/what-is-wrong-with-my-cluster-setup/3681/9 "2021-12-31T10:07:24Z")

</div>

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.