# Kafka-connect takes too long to spin up **URL:** \ **Category:** Kafka Connect\ **Created:** [12 March 2021 15:41 UTC](https://forum.confluent.io/t/kafka-connect-takes-too-long-to-spin-up/1196 "2021-03-12T15:41:03Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Otto](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/otto/32/616_2.png) [@Otto](https://forum.confluent.io/u/Otto)\ **Post date:** [12 March 2021 15:41 UTC](https://forum.confluent.io/t/kafka-connect-takes-too-long-to-spin-up/1196/1 "2021-03-12T15:41:04Z")
Greetings, Im using similar docker compose file as [this](https://github.com/confluentinc/demo-scene/blob/c216326ad0b8cd74cf5327e85e66faab85017e66/connect-jdbc/docker-compose.yml) , but kafka connect takes too long to spin up. As you can see Im waiting for 6 mins and it still starting. It seems to be stuck. . ![kafka-connect](https://us1.discourse-cdn.com/flex019/uploads/confluentcommunity/original/1X/7c1e1cb599a10e74f9c673cfee594b92fd8959d0.png) ``` connect part in compose file: kafka-connect-01: image: confluentinc/cp-kafka-connect:5.4.0 container_name: kafka-connect-01 depends_on: - zookeeper - broker - schema-registry ports: - 8083:8083 environment: CONNECT_LOG4J_APPENDER_STDOUT_LAYOUT_CONVERSIONPATTERN: "[%d] %p %X{connector.context}%m (%c:%L)%n" CONNECT_BOOTSTRAP_SERVERS: "broker:29092" CONNECT_REST_PORT: 8083 CONNECT_REST_ADVERTISED_HOST_NAME: "kafka-connect-01" CONNECT_GROUP_ID: compose-connect-group CONNECT_CONFIG_STORAGE_TOPIC: docker-connect-configs CONNECT_OFFSET_STORAGE_TOPIC: docker-connect-offsets CONNECT_STATUS_STORAGE_TOPIC: docker-connect-status CONNECT_KEY_CONVERTER_SCHEMA_REGISTRY_URL: "http://schema-registry:8081" CONNECT_KEY_CONVERTER: org.apache.kafka.connect.storage.StringConverter CONNECT_VALUE_CONVERTER: org.apache.kafka.connect.storage.StringConverter CONNECT_VALUE_CONVERTER_SCHEMA_REGISTRY_URL: "http://schema-registry:8081" CONNECT_INTERNAL_KEY_CONVERTER: "org.apache.kafka.connect.json.JsonConverter" CONNECT_INTERNAL_VALUE_CONVERTER: "org.apache.kafka.connect.json.JsonConverter" CONNECT_LOG4J_ROOT_LOGLEVEL: "ERROR" CONNECT_LOG4J_LOGGERS: "org.apache.kafka.connect.runtime.rest=WARN,org.reflections=ERROR" CONNECT_CONFIG_STORAGE_REPLICATION_FACTOR: "1" CONNECT_OFFSET_STORAGE_REPLICATION_FACTOR: "1" CONNECT_STATUS_STORAGE_REPLICATION_FACTOR: "1" CONNECT_PLUGIN_PATH: "/usr/share/java" # - /my/local/folder/with/jdbc-driver.jar:/usr/share/java/kafka-connect-jdbc/jars/ command: - /bin/bash - -c - | # JDBC Drivers # ------------ # MySQL cd /usr/share/java/kafka-connect-jdbc/ # See https://dev.mysql.com/downloads/connector/j/ curl https://cdn.mysql.com/Downloads/Connector-J/mysql-connector-java-8.0.23.tar.gz | tar xz /etc/confluent/docker/run & sleep infinity ``` Btw, in your demo that I linked above, you have this part for kafka connect: > sleep infinity & > /etc/confluent/docker/run should not it be the opposite(sleep being the last one)? connect logs: > ===\> ENV Variables … > ALLOW\_UNSIGNED=false > COMPONENT=kafka-connect > CONFLUENT\_DEB\_VERSION=1 > CONFLUENT\_PLATFORM\_LABEL= > CONFLUENT\_VERSION=5.4.0 > CONNECT\_BOOTSTRAP\_SERVERS=broker:29092 > CONNECT\_CONFIG\_STORAGE\_REPLICATION\_FACTOR=1 > CONNECT\_CONFIG\_STORAGE\_TOPIC=docker-connect-configs > CONNECT\_GROUP\_ID=compose-connect-group > CONNECT\_INTERNAL\_KEY\_CONVERTER=org.apache.kafka.connect.json.JsonConverter > CONNECT\_INTERNAL\_VALUE\_CONVERTER=org.apache.kafka.connect.json.JsonConverter > CONNECT\_KEY\_CONVERTER=org.apache.kafka.connect.storage.StringConverter > CONNECT\_KEY\_CONVERTER\_SCHEMA\_REGISTRY\_URL=[http://schema-registry:8081](http://schema-registry:8081) > CONNECT\_LOG4J\_APPENDER\_STDOUT\_LAYOUT\_CONVERSIONPATTERN=[%d] %p %X{connector.context}%m (%c:%L)%n > CONNECT\_LOG4J\_LOGGERS=org.apache.kafka.connect.runtime.rest=WARN,org.reflections=ERROR > CONNECT\_LOG4J\_ROOT\_LOGLEVEL=ERROR > CONNECT\_OFFSET\_STORAGE\_REPLICATION\_FACTOR=1 > CONNECT\_OFFSET\_STORAGE\_TOPIC=docker-connect-offsets > CONNECT\_PLUGIN\_PATH=/usr/share/java > CONNECT\_REST\_ADVERTISED\_HOST\_NAME=kafka-connect-01 > CONNECT\_REST\_PORT=8083 > CONNECT\_STATUS\_STORAGE\_REPLICATION\_FACTOR=1 > CONNECT\_STATUS\_STORAGE\_TOPIC=docker-connect-status > CONNECT\_VALUE\_CONVERTER=org.apache.kafka.connect.storage.StringConverter > CONNECT\_VALUE\_CONVERTER\_SCHEMA\_REGISTRY\_URL=[http://schema-registry:8081](http://schema-registry:8081) > CUB\_CLASSPATH=/etc/confluent/docker/docker-utils.jar > HOME=/root > HOSTNAME=59b8cd84053f > KAFKA\_ADVERTISED\_LISTENERS= > KAFKA\_VERSION= > KAFKA\_ZOOKEEPER\_CONNECT= > LANG=C.UTF-8 > PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin > PWD=/usr/share/java/kafka-connect-jdbc > PYTHON\_PIP\_VERSION=8.1.2 > PYTHON\_VERSION=2.7.9-1 > SCALA\_VERSION=2.12 > SHLVL=2 > ZULU\_OPENJDK\_VERSION=8=8.38.0.13 > \_=/usr/bin/env > ===\> User > uid=0(root) gid=0(root) groups=0(root) > ===\> Configuring … > ===\> Running preflight checks … > ===\> Check if Kafka is healthy … > ===\> Launching … > ===\> Launching kafka-connect … after 16 mins, status is unhealthy The log is exactly the same. ---
**Author:** ![rmoff](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/rmoff/32/3_2.png) [@rmoff](https://forum.confluent.io/u/rmoff)\ **Post date:** [12 March 2021 17:24 UTC](https://forum.confluent.io/t/kafka-connect-takes-too-long-to-spin-up/1196/2 "2021-03-12T17:24:51Z")
Hi Otto, How much memory do you have allocated to docker? Sometimes this can be an issue. Is the log you’ve quoted from `docker logs kafka-connect`? > [@Otto](#): > > Btw, in your demo that I linked above, you have this part for kafka connect: > > > sleep infinity & > > /etc/confluent/docker/run > > should not it be the opposite(sleep being the last one)? Good question, but the `sleep` is forked to the background (`&`) so the order in this case is not incorrect. ---
**Author:** ![Otto](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/otto/32/616_2.png) [@Otto](https://forum.confluent.io/u/Otto)\ **Post date:** [15 March 2021 07:06 UTC](https://forum.confluent.io/t/kafka-connect-takes-too-long-to-spin-up/1196/3 "2021-03-15T07:06:20Z")
Thanks Robin, I increased the resources for docker and it seems to work as expected now. I also have some concerns, I will list here: In the demo, for control -center we have > ``` > sleep 120 > /etc/confluent/docker/run > > ``` 1. Since we declare `broker` as a dependency, do we have to still wait 2 mins manually? I excluded it and seems to work. 2. Isnt,` CONNECT_INTERNAL_VALUE_CONVERTER` and `CONNECT_INTERNAL_KEY_CONVERTER` deprecated? 3. When we specify `CONNECT_PLUGIN_PATH`, will connect search for the plugin recursively? For example, if we have `/usr/share/java` will it search in every folder recursively? Thanks ---
**Author:** ![rmoff](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/rmoff/32/3_2.png) [@rmoff](https://forum.confluent.io/u/rmoff)\ **Post date:** [15 March 2021 09:39 UTC](https://forum.confluent.io/t/kafka-connect-takes-too-long-to-spin-up/1196/4 "2021-03-15T09:39:46Z")
> [@Otto](#): > > - Since we declare `broker` as a dependency, do we have to still wait 2 mins manually? I excluded it and seems to work. The dependency is just for Docker Compose in the order in which it initiates each container. It doesn’t make it wait until the container has finished booting up. The two minute wait is a bit of a hacky-but-good-enough approach to make sure that the broker has finished starting before the control center tries to use it > [@Otto](#): > > - Isnt, ` CONNECT_INTERNAL_VALUE_CONVERTER` and `CONNECT_INTERNAL_KEY_CONVERTER` deprecated? Yes. Feel free to submit a PR to the repository to remove these 🙂 > [@Otto](#): > > - When we specify `CONNECT_PLUGIN_PATH` , will connect search for the plugin recursively? For example, if we have `/usr/share/java` will it search in every folder recursively? Yes, I think it must do (although I’m not 100% sure if there’s a limit on the level of recursion). For example, here I would set `plugin.path=/opt/connectors` ```auto /opt/connectors └── jcustenborder-kafka-connect-spooldir ├── doc │ ├── LICENSE │ └── README.md ├── etc … ├── lib … │ ├── javassist-3.21.0-GA.jar │ ├── jsr305-3.0.2.jar │ ├── kafka-connect-spooldir-2.0.43.jar │ ├── listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar … └── manifest.json ``` Since `plugin.path` can be a comma-separated list of paths, I would strongly suggest you set it to point at the folder(s) that contains the connectors (rather than just relying on a broad search from a high level in the file system) simply because it makes it easier to maintain (anyone else inspecting the config will know where the connectors are supposed to be). ---
**Author:** ![system](https://us1.discourse-cdn.com/flex019/uploads/confluentcommunity/original/1X/c49438c90c9df282e9996fdf6971be890c71b65a.svg) [@system](https://forum.confluent.io/u/system)\ **Post date:** [14 April 2021 09:40 UTC](https://forum.confluent.io/t/kafka-connect-takes-too-long-to-spin-up/1196/5 "2021-04-14T09:40:26Z")
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.