Waiting for at-least one connect pod availability

mikereiche · 13 March 2025 22:25

Following Manage Connectors for Confluent Platform Using Confluent for Kubernetes | Confluent Documentation

" waiting for at-least one connect pod availability"

It looks like there is a connect pod that is available.

Edit: the Connect CR in the example does not have “replicas”, and the default replicas is 0.

$ kubectl  describe -f myconnector.yaml 
Name:         test-couchbase-source123123
Namespace:    confluent
Labels:       <none>
Annotations:  <none>
API Version:  platform.confluent.io/v1beta1
Kind:         Connector
Metadata:
  Creation Timestamp:  2025-03-13T22:00:58Z
  Finalizers:
    connector.finalizers.platform.confluent.io
  Generation:        1
  Resource Version:  85303
  UID:               2cb52869-e6d7-4ac0-bfa2-6fb352a8df72
Spec:
  Class:  com.couchbase.connect.kafka.CouchbaseSourceConnector
  Configs:
    couchbase.bootstrap.timeout":            10s
    couchbase.bucket:                        travel-sample
    couchbase.compression:                   ENABLED
    couchbase.event.filter:                  com.couchbase.connect.kafka.filter.AllPassFilter
    couchbase.flow.control.buffer:           16m
    couchbase.log.document.lifecycle:        false
    couchbase.metrics.interval:              10m
    couchbase.password:                      C0uchbase123!
    couchbase.persistence.polling.interval:  0
    couchbase.seed.nodes:                    couchbases://cb.grixsip3rnsvxcl0.cloud.couchbase.com
    couchbase.source.handler:                com.couchbase.connect.kafka.example.MultiRawJsonSourceHandler
    couchbase.stream.from:                   SAVED_OFFSET_OR_BEGINNING
    couchbase.topic:                         test-default
    couchbase.username:                      clientuser
  Connect Cluster Ref:
    Name:  kafka-connect-couchbase
  Name:    test-couchbase-source123123
  Restart Policy:
    Max Retry:  1
    Type:       Never
  Task Max:     2
Status:
  App State:  Unknown
  Conditions:
    Last Probe Time:       2025-03-13T22:00:58Z
    Last Transition Time:  2025-03-13T22:00:58Z
    Message:               Application state is unknown
    Reason:                AppStateUnknown
    Status:                Unknown
    Type:                  platform.confluent.io/app-ready
  State:                   ERROR
Events:
  Type     Reason   Age                   From       Message
  ----     ------   ----                  ----       -------
  Warning  Warning  97s (x26 over 7m52s)  connector  waiting for at-least one connect pod availability

admin@ip-172-31-3-82:~$ kubectl get  pod connect-0
NAME        READY   STATUS    RESTARTS      AGE
connect-0   1/1     Running   4 (20h ago)   3d3h

It’s not clear what spec.connectClusterRef.name should be. Where do I find that? Or if this is defining it, what refers to it?

apiVersion: platform.confluent.io/v1beta1
kind: Connect
metadata:
  name: kafka-connect-couchbase
  namespace: confluent
  annotations:
spec:
  image:
    application: confluentinc/cp-server-connect:7.9.0
    init: confluentinc/confluent-init-container:2.11.0
  keyConverterType: org.apache.kafka.connect.storage.StringConverter
  valueConverterType: org.apache.kafka.connect.converters.ByteArrayConverter
  replicas: 1 <------------------ need to specify at least 1
  build:
    type: onDemand                                   
    onDemand:                                        
      plugins:
        url: 
        - name: Couchbase_Kafka_Connector_4.2.7
          archivePath: https://packages.couchbase.com/clients/kafka/4.2.7/couchbase-kafka-connect-couchbase-4.2.7.zip
          checksum:   b2aff22e3020ed8115bf7e259559c90e
      storageLimit: 4G

apiVersion: platform.confluent.io/v1beta1
metadata:
  name: test-couchbase-source123123
  namespace: confluent
kind: Connector
spec:
  name: test-couchbase-source123123
  taskMax: 2
  class: com.couchbase.connect.kafka.CouchbaseSourceConnector
  configs:
    couchbase.topic: test-default
    couchbase.seed.nodes: couchbases://cb.grixsip3rnsvxcl0.cloud.couchbase.com
    couchbase.username: clientuser
    couchbase.password: C0uchbase123!
    couchbase.bucket: travel-sample
    couchbase.source.handler: com.couchbase.connect.kafka.handler.source.RawJsonSourceHandler
    couchbase.event.filter: com.couchbase.connect.kafka.filter.AllPassFilter
    couchbase.stream.from: SAVED_OFFSET_OR_BEGINNING
    couchbase.compression: ENABLED
    couchbase.flow.control.buffer: 16m
    couchbase.persistence.polling.interval: "0"
    couchbase.log.document.lifecycle: "false"
    couchbase.metrics.interval: 10m
    couchbase.bootstrap.timeout": 10s
  restartPolicy:
    type: Never
    maxRetry: 1
  connectClusterRef:
    name: kafka-connect-couchbase # from connector.yaml

mmuehlbeyer · 14 March 2025 08:47

you need to specifiy the following in your myconnector.yaml

something like
  connectClusterRef:
    name: kafka-connect-couchbase

for ref:

github.com/confluentinc/confluent-kubernetes-examples

connector/http-sink-connector/httpsinkconnector.yaml

df41f1f69

apiVersion: platform.confluent.io/v1beta1
kind: Connector
metadata:
  name: httpsinkconnector
  namespace: confluent
spec:
  class: "io.confluent.connect.http.HttpSinkConnector"
  taskMax: 1
  connectClusterRef:
    name: connect
  configs:
    topics: "http-messages"
    tasks.max: "1"
    connector.class: "io.confluent.connect.http.HttpSinkConnector"
    http.api.url: "http://my-backend-api/messages"
    value.converter: "org.apache.kafka.connect.storage.StringConverter" 
    key.converter: "org.apache.kafka.connect.storage.StringConverter" 
    confluent.topic.bootstrap.servers: "kafka:9071"
    confluent.topic.replication.factor: "3"
    reporter.bootstrap.servers: "kafka:9071"

This file has been truncated. show original

hth,
michael

mikereiche · 14 March 2025 16:39

I did eventually get it sorted out. But even after aligning the cluster name - there was still an issue because the Connect CR provided in the instructions does not have “replicas” - and replicas defaults to 0.

mmuehlbeyer · 14 March 2025 18:56

which instructions are you referring to?
the docs.confluent.io one?
or the one in the github repo?

mikereiche · 14 March 2025 20:24

kind: Connect
metadata:
  annotations:
  platform.confluent.io/connector-wget-extra-args:   --- [1]
spec:
  build:
    type: onDemand                                   --- [2]
    onDemand:                                        --- [3]
      plugins:
        url:                                         --- [4]
        - name:                                      --- [5]
          archivePath:                               --- [6]
          checksum:                                  --- [7]
      storageLimit:                                  --- [8]

At least that’s what I thought the issue is. Then again the event “waiting for at-least one connect pod availability” remains as the last event even after the connector starts working. So it may have been the fix to the cluster-name that got it working. Although I though I removed replicas: 1 and it stopped working.

mmuehlbeyer · 17 March 2025 11:09

hmm so you’ve removed rhe setting
replicas: 1 ?
could you share your config?
replicas should be set to at least 1

mikereiche · 20 March 2025 00:52

replicas is not in the sample in the documentation:

mmuehlbeyer · 20 March 2025 08:45

I see though it’s needed to get the connect cluster started
(feel free to give feedback about the docs at the bottom of the page to enhance the docs)

see the api ref:

and heres the deploy part of the docs:

or kubernetes examples

hth,
michael

Topic		Replies	Views
Self managed Connectors on K8s Self-Managed Connectors	1	3551	9 February 2021
Kafka connect server never returns 200 Self-Managed Connectors	3	1704	14 November 2023
Confluent ClickHouse Connector "Timed out waiting for a node assignment" Self-Managed Connectors	2	103	24 February 2025
Connect Docker Kafka Connect to Confluent Cloud Self-Managed Connectors	6	2042	6 February 2024
Confluent operator for kafa connect Kafka Connect	1	2466	24 April 2023

Waiting for at-least one connect pod availability

Related topics