How to install Confluent for Kubernetes?

Schema Registry
1

It is my first time working with Kafka and I need to work with Schema Registry.
I already tested and could send messages from a producer to a consumer.
Now I am trying to install the Schema Registry, but I still can’t.

In minikube I have some pods related to my-schema-registry-cp-schema-registry and a service my-schema-registry-cp-schema-registry
I added helm confluent with:

 helm repo add confluentinc https://packages.confluent.io/helm

and install confluent with:

helm upgrade --install confluent-operator confluentinc/confluent-for-kubernetes --namespace default

Configure the Schema registry:

helm upgrade --install confluent-operator confluentinc/confluent-for-kubernetes --namespace default --set kafka.bootstrapServers=my-cluster-kafka-bootstrap:9092 --set schemaRegistry.enabled=true

I created a small AVRO file:

{
  "type": "record",
  "name": "MyAvroRecord",
  "fields": [
    {"name": "value", "type": "string"}
  ]
}

Then I move to the cp-helm-charts/charts/cp-schema-registry folder and do:

helm install my-schema-registry . -f values.yaml --namespace default

And I understand that then I have to register the schema in the Schema Registry by doing:

curl -X POST -H "Content-Type: application/vnd.schemaregistry.v1+json" \
--data '{"schema": "{\"type\":\"record\",\"name\":\"MyAvroRecord\",\"fields\":[{\"name\":\"value\",\"type\":\"string\"}]}"}' \
http://my-schema-registry-cp-schema-registry:8081/subjects/test-topic-value/versions

And here is where I got an error and can’t move forward.

curl: (6) Could not resolve host: my-schema-registry-cp-schema-registry

I was working with chatGPT because I couldn’t find a tutorial or an explanation explaining which steps I should follow to work with Confluent for Kubernetes and Schema Registry.

1 Like
2

Hey @nicobonder

I assume you’re trying to setup you env with the helm charts here?

seems as you mixing things up a bit.

I would highly recommend to go with these examples here

hth,
michael

3

@mmuehlbeyer Hi, as you can see in the first link says cp-helm-charts was deprecated and both links talk about Confluent for Kubernetes, and I don’t have a problem installing CFK, but I have a problem installing and using Schema Registry.
Do you have any examples of how to work with Schema Registry?

Thanks for your reply.
Nico

1 Like
4

sure
have look at

it states how to deploy platform including Schema Registry

and the docs

hth,
michael

1 Like
5

Hello, how did you install or deploy Schema Registry? Could you elaborate for my link?

6

I was able to deploy the schema registry on top of confluent operator deployed using confluent for kubernetes as a custom resource.

eg Custom resource defenition

apiVersion: platform.confluent.io/v1beta1
kind: SchemaRegistry
metadata:
  name: kafka-schema-registry
  namespace: confluent
spec:
  image:
    application: confluentinc/cp-schema-registry:7.9.0
    init: confluentinc/confluent-init-container:2.11.0
  dependencies:
    kafka:
      bootstrapEndpoint: "kafka-0:9092,kafka-1:9092,kafka-2:9092"
  replicas: 2
  configOverrides:
    server:
      - kafkastore.topic=_test_schemas

but the confusing part for me is,

CFK ( confluent for kubernetes ) falls under Enterprise licensing. So i am not sure if installing confluent operator using CFK will break the licensing.

I am looking towards deploying it under community licensing. Anyone know if i have to build a custom chart for achieving this ?

1 Like
7

@albin Confluent for Kubernetes falls under the Confluent Enterprise License – there isn’t a way to run it under the Confluent Community License.

1 Like
8

@dtroiano aah okay, then i will have to look at deploying confluent operator with a custom helm chart ?

1 Like
9

Thank you so much for your reply. According to dtroiano, we can’t use CFK for licensing. So, I think that you will install or deploy using Helm. It may be the easiest way than the Custom Resource. I am confused about how to use Helm or your sample custom resource. Is it a complete configuration to run or deploy Schema Registry?

10

@thawzinmyo To deploy standalone schema registry you could probably look at the bitnami helm chart too → schema-registry 25.0.0 · bitnami/bitnami

11

@albin , I configured like that but not success yet

##########
# Secret #
##########
kubectl create secret generic schema-registry-p12 \
  --from-file=truststore.p12=./truststore.p12 \
  --from-file=keystore.p12=./keystore.p12 \
  -n kafka-debezium

###############
# values.yaml #
###############
replicaCount: 1

kafka:
  enabled: false

externalKafka:
  brokers: "SASL_SSL://kafka-***-0.g.db.ondigitalocean.com:25073"
auth:
  enabled: true
  clientProtocol: sasl_tls
  tls:
    certsSecret: schema-registry-p12
    keystorePassword: ***
    truststorePassword: ***
    type: p12
  sasl:
    mechanism: scram-sha-256
    users:
      - doadmin
    passwords:
      - AVNS_1QxaiLBUWP172Nz2MN6

extraEnvVars:
  - name: SCHEMA_REGISTRY_LISTENERS
    value: http://0.0.0.0:8081
  - name: SCHEMA_REGISTRY_HOST_NAME
    value: schema-registry
  - name: SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS
    value: SASL_SSL://kafka-***-0.g.db.ondigitalocean.com:25073
  - name: SCHEMA_REGISTRY_KAFKA_SASL_USERS
    value: ***
  - name: SCHEMA_REGISTRY_KAFKA_SASL_PASSWORDS
    value: ***
  - name: SCHEMA_REGISTRY_KAFKASTORE_SECURITY_PROTOCOL
    value: SASL_SSL
  - name: SCHEMA_REGISTRY_KAFKASTORE_SASL_MECHANISM
    value: SCRAM-SHA-256
  - name: SCHEMA_REGISTRY_KAFKASTORE_SASL_JAAS_CONFIG
    value: >
      org.apache.kafka.common.security.scram.ScramLoginModule required
      username="***" password="***";
  - name: SCHEMA_REGISTRY_SSL_TRUSTSTORE_LOCATION
    value: /etc/kafka/secret/truststore.p12
  - name: SCHEMA_REGISTRY_SSL_TRUSTSTORE_PASSWORD
    value: ***
  - name: SCHEMA_REGISTRY_SSL_TRUSTSTORE_TYPE
    value: PKCS12
  - name: SCHEMA_REGISTRY_SSL_KEYSTORE_LOCATION
    value: /etc/kafka/secret/keystore.p12
  - name: SCHEMA_REGISTRY_SSL_KEYSTORE_PASSWORD
    value: ***
  - name: SCHEMA_REGISTRY_SSL_KEY_PASSWORD
    value: ***
  - name: SCHEMA_REGISTRY_SSL_KEYSTORE_TYPE
    value: PKCS12
  - name: SCHEMA_REGISTRY_SSL_ENDPOINT_IDENTIFICATION_ALGORITHM
    value: ""

volumeMounts:
  - name: kafka-certificates
    mountPath: /etc/kafka/secret
    readOnly: true

volumes:
  - name: kafka-certificates
    secret:
      secretName: schema-registry-p12

I got this error

kubectl logs schema-registry-0 -n ns-lsp-kafka-debezium-shd-01                         
  
schema-registry 03:30:30.35 INFO  ==> 
schema-registry 03:30:30.36 INFO  ==> Welcome to the Bitnami schema-registry container
schema-registry 03:30:30.36 INFO  ==> Subscribe to project updates by watching https://github.com/bitnami/containers
schema-registry 03:30:30.36 INFO  ==> Did you know there are enterprise versions of the Bitnami catalog? For enhanced secure software supply chain features, unlimited pulls from Docker, LTS support, or application customization, see Bitnami Premium or Tanzu Application Catalog. See https://www.arrow.com/globalecs/na/vendors/bitnami/ for more information.
schema-registry 03:30:30.37 INFO  ==> 
schema-registry 03:30:30.38 INFO  ==> ** Starting Schema Registry setup **
schema-registry 03:30:30.41 INFO  ==> Validating settings in SCHEMA_REGISTRY_* env vars
schema-registry 03:30:30.45 WARN  ==> In order to configure the TLS encryption for communication with Kafka brokers, most auth protocols require mounting your schema-registry.keystore.jks and schema-registry.truststore.jks certificates to the /opt/bitnami/schema-registry/certs directory.
/opt/bitnami/scripts/libschemaregistry.sh: line 153: SCHEMA_REGISTRY_KAFKA_SASL_USERS: unbound variable