Hi @rick
When I was trying to start this “Introduction to Streaming Application Development” example app (Tutorial: Introduction to Streaming Application Development | Confluent Documentation ), I got this error message:
“ERROR: Failed after 240 seconds. Please troubleshoot and run again.”
even though docker-compose ps shows that all the docker containers are running, however, the URL http://localhost:5601/app/kibana#/dashboard/Microservices will not return anything.
The confluent console also shows 29 topics, 1 ksqldb, 34 clients.
I am wondering what could be the issue.
Thanks,
2 Likes
rick
8 March 2022 14:57
2
@xsqian I’m sorry you’re having issues with this microservices demo. I just ran the demo myself, and did not experience a timeout. The issue can arise when provisioning of cloud resources takes longer than expected. This is non-deterministic and experiences may differ depending on factors including the cloud provider, Confluent Cloud status / traffic, etc…
I’m wondering if you could destroy all the cloud resources created by your initial run of the demo script and try again. And if you encounter the failure again, can you provide a larger snippet of the console output so that I could see which section of the script is failing.
Hi @rick Thank you for testing out on your end and it worked successfully. I actually had run the ./start-ccloud.sh and destroyed the resources and run it again a few times. I will try one more time and grab the console output. I may have to do it later in the evening and will keep you posted. Thanks!
Hi @rick , I re-ran the ./start-ccloud.sh, still got the same error messages:
(base) xsqian@[21:01:24]<microservices-orders>% ./start-ccloud.sh
--------------------------------------------------------------------------------------------
This example runs on Confluent Cloud, sign up here:
https://www.confluent.io/confluent-cloud/tryfree/
The example uses real Confluent Cloud resources that may be billable, including connectors
and ksqlDB applications that may have hourly charges. The end of this script shows a command
you can run to destroy all the cloud resources, and you should verify they are destroyed.
New Confluent Cloud signups receive $400 to spend within Confluent Cloud during their first
60 days. Use Confluent Cloud promo code C50INTEG to receive an additional $50 free usage.
This will sufficiently cover one day of running this example, beyond which you may be billed
for the Confluent Cloud resources until you destroy them.
--------------------------------------------------------------------------------------------
Do you still want to run this script? [y/n] y
✔ Confluent CLI version ok
✔ Logged into the Confluent CLI
====== Create new Confluent Cloud stack
Creating Confluent Cloud stack for service account demo-app-16520, ID: sa-gqr3w3.
Waiting up to 720 seconds for Confluent Cloud cluster to be ready and for credentials to propagate
Sleeping an additional 80 seconds to ensure propagation of all metadata
Principal | Permission | Operation | ResourceType | ResourceName | PatternType
-----------------+------------+-----------+--------------+--------------+--------------
User:sa-gqr3w3 | ALLOW | CREATE | TOPIC | * | LITERAL
Principal | Permission | Operation | ResourceType | ResourceName | PatternType
-----------------+------------+-----------+--------------+--------------+--------------
User:sa-gqr3w3 | ALLOW | DELETE | TOPIC | * | LITERAL
Principal | Permission | Operation | ResourceType | ResourceName | PatternType
-----------------+------------+-----------+--------------+--------------+--------------
User:sa-gqr3w3 | ALLOW | WRITE | TOPIC | * | LITERAL
Principal | Permission | Operation | ResourceType | ResourceName | PatternType
-----------------+------------+-----------+--------------+--------------+--------------
User:sa-gqr3w3 | ALLOW | READ | TOPIC | * | LITERAL
Principal | Permission | Operation | ResourceType | ResourceName | PatternType
-----------------+------------+-----------+--------------+--------------+--------------
User:sa-gqr3w3 | ALLOW | DESCRIBE | TOPIC | * | LITERAL
Principal | Permission | Operation | ResourceType | ResourceName | PatternType
-----------------+------------+------------------+--------------+--------------+--------------
User:sa-gqr3w3 | ALLOW | DESCRIBE_CONFIGS | TOPIC | * | LITERAL
Principal | Permission | Operation | ResourceType | ResourceName | PatternType
-----------------+------------+-----------+--------------+--------------+--------------
User:sa-gqr3w3 | ALLOW | READ | GROUP | * | LITERAL
Principal | Permission | Operation | ResourceType | ResourceName | PatternType
-----------------+------------+-----------+--------------+--------------+--------------
User:sa-gqr3w3 | ALLOW | WRITE | GROUP | * | LITERAL
Principal | Permission | Operation | ResourceType | ResourceName | PatternType
-----------------+------------+-----------+--------------+--------------+--------------
User:sa-gqr3w3 | ALLOW | CREATE | GROUP | * | LITERAL
Principal | Permission | Operation | ResourceType | ResourceName | PatternType
-----------------+------------+-----------+--------------+--------------+--------------
User:sa-gqr3w3 | ALLOW | DESCRIBE | GROUP | * | LITERAL
Principal | Permission | Operation | ResourceType | ResourceName | PatternType
-----------------+------------+-----------+------------------+--------------+--------------
User:sa-gqr3w3 | ALLOW | DESCRIBE | TRANSACTIONAL_ID | * | LITERAL
Principal | Permission | Operation | ResourceType | ResourceName | PatternType
-----------------+------------+-----------+------------------+--------------+--------------
User:sa-gqr3w3 | ALLOW | WRITE | TRANSACTIONAL_ID | * | LITERAL
Principal | Permission | Operation | ResourceType | ResourceName | PatternType
-----------------+------------+------------------+--------------+---------------+--------------
User:sa-gqr3w3 | ALLOW | IDEMPOTENT_WRITE | CLUSTER | kafka-cluster | LITERAL
Principal | Permission | Operation | ResourceType | ResourceName | PatternType
-----------------+------------+-----------+--------------+---------------+--------------
User:sa-gqr3w3 | ALLOW | DESCRIBE | CLUSTER | kafka-cluster | LITERAL
Set API Key "74QWCZZD54RRJHVA" as the active API key for "lkc-9kx085".
Client configuration file saved to: stack-configs/java-service-account-sa-gqr3w3.config
====== Generating Confluent Cloud configurations
Generating component configurations from stack-configs/java-service-account-sa-gqr3w3.config
(If you want to run any of these components to talk to Confluent Cloud, these are the configurations to add to the properties file for each component)
Confluent Platform Components:
delta_configs/schema-registry-ccloud.delta
delta_configs/replicator-to-ccloud-producer.delta
delta_configs/ksqldb-server-ccloud.delta
delta_configs/ksql-datagen.delta
delta_configs/control-center-ccloud.delta
delta_configs/metrics-reporter.delta
delta_configs/rest-proxy.delta
delta_configs/connect-ccloud.delta
delta_configs/connector-ccloud.delta
delta_configs/ak-tools-ccloud.delta
Kafka Clients:
delta_configs/java_producer_consumer.delta
delta_configs/java_streams.delta
delta_configs/librdkafka.delta
delta_configs/python.delta
delta_configs/dotnet.delta
delta_configs/go.delta
delta_configs/node.delta
delta_configs/cpp.delta
delta_configs/env.delta
====== Creating demo topics
Creating topic bronze on Confluent Cloud
Created topic "bronze".
Creating topic customers on Confluent Cloud
Created topic "customers".
Creating topic gold on Confluent Cloud
Created topic "gold".
Creating topic orders on Confluent Cloud
Created topic "orders".
Creating topic orders-enriched on Confluent Cloud
Created topic "orders-enriched".
Creating topic order-validations on Confluent Cloud
Created topic "order-validations".
Creating topic payments on Confluent Cloud
Created topic "payments".
Creating topic platinum on Confluent Cloud
Created topic "platinum".
Creating topic silver on Confluent Cloud
Created topic "silver".
Creating topic warehouse-inventory on Confluent Cloud
Created topic "warehouse-inventory".
Creating topic _confluent-monitoring on Confluent Cloud
Created topic "_confluent-monitoring".
====== Starting local services in Docker
Creating network "microservices-orders_default" with the default driver
Creating volume "microservices-orders_database" with default driver
Building sqlite
[+] Building 1.3s (13/13) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 37B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/alpine:3.4 1.1s
=> [auth] library/alpine:pull token for registry-1.docker.io 0.0s
=> [1/7] FROM docker.io/library/alpine:3.4@sha256:b733d4a32c4da6a00a84df2ca32791bb03df95400243648d8c539e7b4cce329c 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 70B 0.0s
=> CACHED [2/7] WORKDIR /db 0.0s
=> CACHED [3/7] ADD customers.sql /db 0.0s
=> CACHED [4/7] ADD customers.table /db 0.0s
=> CACHED [5/7] RUN apk add --update sqlite 0.0s
=> CACHED [6/7] RUN mkdir /db/data 0.0s
=> CACHED [7/7] RUN sqlite3 /db/data/microservices.db < /db/customers.sql 0.0s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:f24d485129a73a3b8a0ec681ed21b8b7c228e808d5726cd70376486457df6d24 0.0s
=> => naming to docker.io/library/sqlite:latest 0.0s
Creating sqlite ... done
Creating connect ... done
Creating elasticsearch ... done
Creating microservices ... done
Creating kibana ... done
====== Giving services 80 seconds to startup
Waiting up to 240 seconds for connect to start
................................................ERROR: Failed after 240 seconds. Please troubleshoot and run again.
(base) xsqian@[21:14:50]<microservices-orders>%
rick
9 March 2022 13:50
5
Thanks @xsqian
This is telling me that the local Connect service is not running properly (or is delayed in starting).
In the start-ccloud.sh script, you can see where there is a command to check that connect is functioning:
MAX_WAIT=240
echo "Waiting up to $MAX_WAIT seconds for connect to start"
retry $MAX_WAIT check_connect_up connect || exit 1
You could try to increase the MAX_WAIT variable to see if that helps give the connect container more time to initialize. The check_connect_up function looks at the Kafka Connect logs to try and determine if the Connect service is running by doing this:
docker-compose logs connect | grep "Herder started"
After increasing the timeout, if the script still fails you could look into the Connect logs for more information by just running:
docker-compose logs connect
Please let me know how this goes.
Hi @rick MAX_WAIT=360................................................ERROR: Failed after 360 seconds. Please troubleshoot and run again.
all the docker containers are up and running
the Kibana dashboard is working
there are 29 topics, 1 ksqlDB in the confluent cluster
the persist events test successfully mvn compile -Dtest=io.confluent.examples.streams.microservices.OrdersServiceTest test -f kafka-streams-examples/pom.xml
There are 80 clients to the Kafka
Thanks for your help!
1 Like
system
7 April 2022 05:05
7
This topic was automatically closed after 30 days. New replies are no longer allowed.