I’m running Confluent for Kubernetes and the pod ksqldb-0 is being stucked and K8S is restarting due the liveness and readiness probe.
kubectl logs -f ksqldb-0 -n confluent
[INFO] 2022-05-17 20:27:25,515 [ksql-csu-metrics-reporter-0] io.confluent.ksql.utilization.PersistentQuerySaturationMetrics report - reporting node-level saturation 0.0
[INFO] 2022-05-17 20:27:25,516 [main] io.confluent.ksql.function.Blacklist - Blacklist file: /ext/resource-blacklist.txt not found. No classes will be blacklisted
kubectl describe pod ksqldb-0 -n confluent
Events:
Type Reason Age From Message
Normal Scheduled 6m47s default-scheduler Successfully assigned confluent/ksqldb-0 to dev-worker1
Normal Pulled 6m44s kubelet Container image “confluentinc/confluent-init-container:2.3.0” already present on machine
Normal Created 6m44s kubelet Created container config-init-container
Normal Started 6m44s kubelet Started container config-init-container
Normal Started 5m12s (x2 over 6m41s) kubelet Started container ksqldb
Warning Unhealthy 3m44s (x6 over 5m34s) kubelet Liveness probe failed: Get “http://10.67.10.178:8088/chc/live”: dial tcp 10.67.10.178:8088: connect: connection refused
Normal Killing 3m44s (x2 over 5m14s) kubelet Container ksqldb failed liveness probe, will be restarted
Normal Pulled 3m41s (x3 over 6m41s) kubelet Container image “confluentinc/cp-ksqldb-server:7.1.0” already present on machine
Normal Created 3m41s (x3 over 6m41s) kubelet Created container ksqldb
Warning Unhealthy 67s (x10 over 5m37s) kubelet Readiness probe failed: Get “http://10.67.10.178:8088/chc/ready”: dial tcp 10.67.10.178:8088: connect: connection refused
apiVersion: platform.confluent.io/v1beta1
kind: KsqlDB
metadata:
name: ksqldb
spec:
replicas: 1
image:
application: confluentinc/cp-ksqldb-server:7.1.0
init: confluentinc/confluent-init-container:2.3.0
podTemplate:
resources:
requests:
cpu: “300m”
memory: “512Mi”
limits:
cpu: “500m”
memory: “1Gi”
dataVolumeCapacity: 1Gi