Hi all…
So I got the confluent 6.1.1 stack downloaded…
I keep on getting errors when starting the stack, with service modules failing.
Anyone willing to help, will be appreciated to diagnose it/fix it.
(Id love to use the cp-all-in-one, but as per a previous thread, it pulls the containers every time I start and I can’t do that atm, with bandwidth and caps I’m sitting with).
Want to replicate Robin’s ship blog.
This is on a MBP
G
hmmm… ok, how can I share a zip file… with the logs from the log directory.
G
ok… determined problems are due to some changes I made so that I can connect/talk to it from pods running in kubernetes on minikube… changes was made following the docs from @rmoff … but clearly I made a mistake some where.
I re-deployed the stack into a different directory and started all, and it worked, but if I start my copy with my changes, well not work, the various modules fail.
here is my server.properties.
if someone with more knowledge can have a look and hopefully see a problem.
(it looks like the kafka cluster comes up, but the other modules can’t connect to it.)
listeners=PLAINTEXT://:9092
# Hostname and port the broker will advertise to producers and consumers. If not set,
# it uses the value for "listeners" if configured. Otherwise, it will use the value
# returned from java.net.InetAddress.getCanonicalHostName().
advertised.listeners=PLAINTEXT://mbp-local:9092
# Maps listener names to security protocols, the default is for them to be the same. See the config documentation for more details
#listener.security.protocol.map=PLAINTEXT:PLAINTEXT,SSL:SSL,SASL_PLAINTEXT:SASL_PLAINTEXT,SASL_SSL:SASL_SSL
Hi,
could you please share some details how you’ve tried to start the
confluent stack?
Did you check the logfiles?
Regards
how can I upload theology file/zip… the site restricts zip files.
how do I start it, as pr instructions “confluent local services start”.
But as per above … think the problem is due to the changes made so the I can access my kafka environment from kubernetes running in minikube, see the above code snippet from server.properties. (implementation of below.
Why I’m saying this… with these changes I can access my kafka topic… even though most of the rest of the stack fails… not idea from my minikube environment,
If I redeploy the distribution/zip files, then all starts fine.
I need to get to a state where the stack starts, and I can access it from minikube development environment.
G
Ideal later would be to get the cp-all-in-one stack to work, well it does, jsut can’t afford this behaviour to pull the entire stack every time I do a docker-compose up -d - there is another thread about that 
so I think you have to adapt the connect information for the failing services
as per default the try to connect on localhost or something like this (afaik)
only changes I made is as per above.
if redeploy the original zip then it all works, and it uses localhost, but then the minikube apps have network problems. (K8S based apps apparently does not like to talk/communicate to services outside their k8s environment)
agree… why I also asked if someone can have a look and commit if they see something wrong…
With these settings the errors I’m seeing is the other modules are having problems connecting to the Kafka “core” if we can call it that.
G
I see
could check the logs of the failing services and post the (relevant) output/errors here?
Regarding the external access did you check:
quite useful imho
See attached diagram…
also the first one that fails as per the diagram is the schema manager.
here is a log snippet.
ssl.truststore.password = [hidden]
ssl.truststore.type = JKS
thread.pool.max = 200
thread.pool.min = 8
websocket.path.prefix = /ws
websocket.servlet.initializor.classes = []
zookeeper.set.acl = false
(io.confluent.kafka.schemaregistry.rest.SchemaRegistryConfig)
[2021-06-12 14:58:52,902] INFO Logging initialized @1137ms to org.eclipse.jetty.util.log.Slf4jLog (org.eclipse.jetty.util.log)
[2021-06-12 14:58:52,918] INFO Initial capacity 128, increased by 64, maximum capacity 2147483647. (io.confluent.rest.ApplicationServer)
[2021-06-12 14:58:53,066] INFO Adding listener: http://0.0.0.0:8081 (io.confluent.rest.ApplicationServer)
[2021-06-12 14:59:53,608] ERROR Error starting the schema registry (io.confluent.kafka.schemaregistry.rest.SchemaRegistryRestApplication)
io.confluent.kafka.schemaregistry.exceptions.SchemaRegistryException: Failed to get Kafka cluster ID
at io.confluent.kafka.schemaregistry.storage.KafkaSchemaRegistry.kafkaClusterId(KafkaSchemaRegistry.java:1288)
at io.confluent.kafka.schemaregistry.storage.KafkaSchemaRegistry.<init>(KafkaSchemaRegistry.java:158)
at io.confluent.kafka.schemaregistry.rest.SchemaRegistryRestApplication.initSchemaRegistry(SchemaRegistryRestApplication.java:69)
at io.confluent.kafka.schemaregistry.rest.SchemaRegistryRestApplication.configureBaseApplication(SchemaRegistryRestApplication.java:88)
at io.confluent.rest.Application.configureHandler(Application.java:255)
at io.confluent.rest.ApplicationServer.doStart(ApplicationServer.java:227)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at io.confluent.kafka.schemaregistry.rest.SchemaRegistryMain.main(SchemaRegistryMain.java:43)
Caused by: java.util.concurrent.TimeoutException
at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:108)
at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:272)
at io.confluent.kafka.schemaregistry.storage.KafkaSchemaRegistry.kafkaClusterId(KafkaSchemaRegistry.java:1286)
... 7 more
hmm which diagram you’re talking about.
seems as the schema registry can’t connect to Kafka brokers
there should be a schema-registry.properties in your install dir
check and adapt the following line
kafkastore.bootstrap.servers=PLAINTEXT://localhost:9092
tried… not having much luck… actually gone through all the .properties files…
decided to wipe out the modified properties files and go back to factory… will then slowly redo the robin changes to make it minibus accessible…
eventually do have to get the cp-all-in-one working.
G