Kafka controller failed start while using properties file via docker volume

Ops Containers
1

I am trying to start a controller using docker and passing all the configs via the controller properties file except the cluster_id. But its not responding to the properties file. I am using the following command to run:

$ docker run -d --name=kafka-controller-1 -h kafka-controller-1 -v /root/controller/controller.properties:/etc/kafka/kraft/controller.properties -e CLUSTER_ID='xxxxxxxxxxxxx' confluentinc/cp-kafka:7.5.1

It failed to start because it failed to read the parameters as defined in the properties file and generating the below error:

# docker ps -a
CONTAINER ID   IMAGE                         COMMAND                  CREATED         STATUS                     PORTS     NAMES
ef1d6ca7e3fa   confluentinc/cp-kafka:7.5.1   "/etc/confluent/dock…"   3 seconds ago   Exited (1) 2 seconds ago             kafka-controller-1

# docker logs kafka-controller-1
===> User
uid=1000(appuser) gid=1000(appuser) groups=1000(appuser)
===> Configuring ...
Running in Zookeeper mode...
KAFKA_ZOOKEEPER_CONNECT is required.
Command [/usr/local/bin/dub ensure KAFKA_ZOOKEEPER_CONNECT] FAILED !

But when passing the same parameters via environmental variable, docker run is working fine. So is passing the properties file a standard supported approach or am I doing something incorrectly ?

I am following the steps as mentioned Configure and Monitor KRaft | Confluent Documentation for the properties file based deployment.

2

Hi @Udayendu,

Passing the properties file like this isn’t a supported approach. The cp-kafka Docker image requires environment variables for configuration and, during startup, generates the properties file.

For reference, this is the relevant Dockerfile and this is the script that runs at container startup. Here is where the environment variable-seeded properties file is used to generate the Kafka config passed to kafka-server-start. In your case, the properties file you mapped was ignored and it ran with the default configuration which runs in ZooKeeper mode and requires KAFKA_ZOOKEEPER_CONNECT.

HTH,
Dave

1 Like
3

Thanks Dave.

Could you please let me know how the use of properties file if I want to use the Kafka in the docker mode ?

4

Hi @Udayendu,

Since it’s not a supported approach, you’d need to implement it. E.g., follow the pattern here but for the set of configure, ensure, and launch scripts.

An alternative would be to parse the properties file and export corresponding environment variables so that you don’t have to rewrite those scripts. For example, write a props_to_env helper that does the inverse of this and edit /etc/confluent/docker/run to run it using the pattern referenced above.

HTH,
Dave