Schema Registry Inter-Instance Service Discovery

Dear Community, re-posting my question here as my employer decided to block Slack which makes it impossible to continue my previous thread on the Slack Channel.

We are deploying Schema Registry in a multi-dc, active/standby, deployment. We are deploying to Kubernetes deploying two Kafka Clusters and two Schema Registry clusters one in each DC. In the active DC leader.eligibility: true and in the standby DC leader.eligibility: false, both SR clusters pointing the the active cluster’s Kafka endpoint. Note Kubernetes is not stretched and a stretched cluster is not an option.

The question then is how does SR inter-instance service discovery work, or does it, in a multi-DC deployment. Service discovery in the same Kubernetes cluster is fine and SR instances running in that cluster can discover each other but when instance hostname and port is returned to SR instances in another DC as part of leader election in a Kubernetes deployment these hostnames are not routable defaulting to the POD IP address of the SRs.

Can you please explain how to deploy SR in a multi-DC deployment architecture, active/standby to allow the follower SR instances in the standby cluster to discover the SR leader in the active cluster.

Sounds like you’d have to setup the Kubernetes Ingress to be the listener, rather than the pod ip. This is the main way that two clusters communicate with each other, without using external service discovery tools like Consul.