Hi,
I have installed and running Kafka cluster on local Mac M1 using docker-compose, based on the documentation on Apache Kafka® Docker Quick Start Guide by Confluent
here is my docker-compose:
---
version: '3'
services:
zookeeper:
image: confluentinc/cp-zookeeper:7.3.2
container_name: zookeeper
environment:
ZOOKEEPER_CLIENT_PORT: 2181
ZOOKEEPER_TICK_TIME: 2000
broker:
image: confluentinc/cp-kafka:7.3.2
container_name: broker
ports:
# To learn about configuring Kafka for access across networks see
# https://www.confluent.io/blog/kafka-client-cannot-connect-to-broker-on-aws-on-docker-etc/
- "9092:9092"
depends_on:
- zookeeper
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_INTERNAL:PLAINTEXT
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092,PLAINTEXT_INTERNAL://broker:29092
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
I also have copied Kafka on: /Users/syedahmed/QA/TOOLS/KAFKA/kafka_2.12-3.3.2/bin
and trying to run:
% sh ./kafka-producer-perf-test.sh --topic quickstart --producer-props bootstrap.servers=PLAINTEXT://localhost:9092 --throughput -1 --num-records 2 --payload-file payload.json
Reading payloads from: /Users/syedahmed/QA/TOOLS/KAFKA/kafka_2.12-3.3.2/bin/payload.json
Number of messages read: 1
[2023-06-09 05:22:19,894] WARN [Producer clientId=perf-producer-client] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2023-06-09 05:22:19,894] WARN [Producer clientId=perf-producer-client] Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2023-06-09 05:22:20,001] WARN [Producer clientId=perf-producer-client] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2023-06-09 05:22:20,001] WARN [Producer clientId=perf-producer-client] Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
but cannot connect to the Kafka broker
Is it the incorrect clientId I am using? How do I find out the correct ClientId from the docker instance? or is there something else I am missing?
Apologize for my lack of basic knowledge as this is my first time using Kafka
I would appreciate any help
Thanks,
-Kamal.