Confluent 6.1.1 - Kafka not starting

kkhambadkone · 24 March 2021 16:23

I have installed this on my mac Catalina. I am doing a bin/confluent local services start

Zookeeper starts successfully,
Kafka fails to start. This is what I see in the logs, I have deleted all the kafka related directories in /tmp/

[2021-03-24 09:15:09,079] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$)
[2021-03-24 09:15:09,175] ERROR Exiting Kafka due to fatal exception (kafka.Kafka$)
java.lang.IllegalAccessError: Receiver class scala/collection/mutable/HashSet$ must be the current class or a subtype of interface scala/collection/IterableFactory
        at scala.collection.IterableFactory.apply$(Factory.scala:103)
        at kafka.utils.VerifiableProperties.<init>(VerifiableProperties.scala:28)
        at kafka.server.KafkaServerStartable$.fromProps(KafkaServerStartable.scala:33)
        at kafka.Kafka$.main(Kafka.scala:68)
        at kafka.Kafka.main(Kafka.scala)

mmuehlbeyer · 24 March 2021 20:39

hi,

nor more error messages printed or in the logs?
there should be several logfiles in the created tmp directory.

br,
michael

kkhambadkone · 25 March 2021 01:20

These are all the log files in the kafka/logs folder, only server.log has this message, the other log files are empty

-rw-r–r-- 1 hadoop wheel 0 Mar 24 18:16 metadata-service.log
-rw-r–r-- 1 hadoop wheel 0 Mar 24 18:16 kafka-authorizer.log
-rw-r–r-- 1 hadoop wheel 0 Mar 24 18:16 kafka-request.log
-rw-r–r-- 1 hadoop wheel 0 Mar 24 18:16 log-cleaner.log
-rw-r–r-- 1 hadoop wheel 0 Mar 24 18:16 controller.log
-rw-r–r-- 1 hadoop wheel 0 Mar 24 18:16 state-change.log
-rw-r–r-- 1 hadoop wheel 662 Mar 24 18:16 server.log
-rw-r–r-- 1 hadoop wheel 2552 Mar 24 18:16 kafkaServer-gc.log.0.current

mmuehlbeyer · 26 March 2021 09:00

hmm is there a stdout file in the kafka directory?

after starting which information are shown on your shell?
Should be something like this:

Using CONFLUENT_CURRENT: /var/folders/hj/rb1_1mlj5sj5frs7wq9fjlt00000gn/T/confluent.583529
    Starting ZooKeeper
    ZooKeeper is [UP]
    Starting Kafka
    Kafka is [UP]

Which Java version do you use?

kkhambadkone · 26 March 2021 15:55

yes kafka.stdout has this info. I am using java 8

hadoop@Krishnanands-MacBook-Pro kafka % cat kafka.stdout

SLF4J: Class path contains multiple SLF4J bindings.

SLF4J: Found binding in [jar:file:/Users/hadoop/hadoop-2.7.3/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: Found binding in [jar:file:/Users/hadoop/hbase-1.2.6/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: Found binding in [jar:file:/Users/hadoop/apache-hive-2.1.1-bin/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: Found binding in [jar:file:/Users/hadoop/confluent-6.1.1/share/java/kafka/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: See SLF4J Error Codes for an explanation.

SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]

[2021-03-24 18:16:34,153] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$)

[2021-03-24 18:16:34,253] ERROR Exiting Kafka due to fatal exception (kafka.Kafka$)

java.lang.IllegalAccessError: Receiver class scala/collection/mutable/HashSet$ must be the current class or a subtype of interface scala/collection/IterableFactory

at scala.collection.IterableFactory.apply$(Factory.scala:103)

at kafka.utils.VerifiableProperties.(VerifiableProperties.scala:28)

at kafka.server.KafkaServerStartable$.fromProps(KafkaServerStartable.scala:33)

at kafka.Kafka$.main(Kafka.scala:68)

at kafka.Kafka.main(Kafka.scala)

hadoop@Krishnanands-MacBook-Pro kafka % java -version

java version “1.8.0_271”

Java™ SE Runtime Environment (build 1.8.0_271-b09)

Java HotSpot™ 64-Bit Server VM (build 25.271-b09, mixed mode)

mmuehlbeyer · 26 March 2021 20:24

mmh JDK is installed?
did you try with openjdk?

kkhambadkone · 26 March 2021 22:18

Yes, jdk version is Oracle 1.8_271. Could this be the issue. I read the documentation and it recommends 1.8_202.

mmuehlbeyer · 27 March 2021 19:27

could be worth a try
for me it works with Oracle Java 1.8.0_112 on Mac and Open JDK 1.8.0_282 on Ubuntu

kkhambadkone · 28 March 2021 16:19

Dont seem to be having much luck. I switched to 1.8_112. Scala version is 2.11.0. I also tried with 2.12.

hadoop@Krishnanands-MacBook-Pro confluent-6.1.1 % cat /tmp/confluent.759691/kafka/logs/server.log

[2021-03-28 09:07:44,503] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$)

[2021-03-28 09:07:44,616] ERROR Exiting Kafka due to fatal exception (kafka.Kafka$)

java.lang.IncompatibleClassChangeError: Class scala.collection.mutable.HashSet$ does not implement the requested interface scala.collection.IterableFactory

at scala.collection.IterableFactory.apply(Factory.scala:103)

at scala.collection.IterableFactory.apply$(Factory.scala:103)

at kafka.utils.VerifiableProperties.(VerifiableProperties.scala:28)

at kafka.server.KafkaServerStartable$.fromProps(KafkaServerStartable.scala:33)

at kafka.Kafka$.main(Kafka.scala:68)

at kafka.Kafka.main(Kafka.scala)

[2021-03-28 09:14:50,469] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$)

[2021-03-28 09:14:50,571] ERROR Exiting Kafka due to fatal exception (kafka.Kafka$)

java.lang.IncompatibleClassChangeError: Class scala.collection.mutable.HashSet$ does not implement the requested interface scala.collection.IterableFactory

at scala.collection.IterableFactory.apply(Factory.scala:103)

at scala.collection.IterableFactory.apply$(Factory.scala:103)

at kafka.utils.VerifiableProperties.(VerifiableProperties.scala:28)

at kafka.server.KafkaServerStartable$.fromProps(KafkaServerStartable.scala:33)

at kafka.Kafka$.main(Kafka.scala:68)

at kafka.Kafka.main(Kafka.scala)

hadoop@Krishnanands-MacBook-Pro confluent-6.1.1 % java -version

java version “1.8.0_112”

Java™ SE Runtime Environment (build 1.8.0_112-b16)

Java HotSpot™ 64-Bit Server VM (build 25.112-b16, mixed mode)

hadoop@Krishnanands-MacBook-Pro confluent-6.1.1 % echo $SCALA_HOME

/Users/hadoop/scala-2.11.0

kkhambadkone · 28 March 2021 20:21

It is running now. I had to unset CLASSPATH, was probably picking up some old jar file

Using CONFLUENT_CURRENT: /tmp/confluent.759691

Starting ZooKeeper

ZooKeeper is [UP]

Starting Kafka

Kafka is [UP]

Starting Schema Registry

Schema Registry is [UP]

Starting Kafka REST

Kafka REST is [UP]

Starting Connect

Connect is [UP]

Starting ksqlDB Server

ksqlDB Server is [UP]

Starting Control Center

Control Center is [UP]

system · 4 April 2021 20:21

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Error: ZooKeeper failed to start Kafka Connect	2	999	24 February 2024
Schema Registry fails to start Schema Registry	13	11519	9 March 2023
Not working Confluent Platform 6.1.4 version Containers	0	3381	24 August 2022
Confluent Kafka 7.x unable to start zookeeper Ops	3	3156	30 November 2021
Standalone confluent schema registry not starting Schema Registry	5	1959	24 October 2023

Confluent 6.1.1 - Kafka not starting

Related topics