Hello,
I’m trying to make ksqlDB work with Apicurio Registry and have had a few issues with compatibility. (We have an “Azure RedHat Openshift” setup at work which is why we are using Apicurio instead of Confluent’s Schema Registry).
So, Apicurio has 4 different APIs that I can use. The “official” api is called /registry/v2 and it seems to work fine with my Debezium connector. ksqlDB however, when configured to use it, is trying to access “/subjects” which isn’t known by this API. For that to work, I have to point it to the /ccompat/v6 API which is made to be compatible with Confluent Schema Registry clients. When I make ksqlDB point to this one, it works all good… until the IDs (Global-Id and Content-Id) in Apicurio aren’t synced anymore.
Apicurio has 3 types of IDs as seen here :
Debezium uses the “Global-Id” as the “Schema-Id” of each Kafka message it produces. ksqlDB then reads that ID and tries to find it in Apicurio, but using the “Content-Id” instead. So either it’ll never find it or it will find a random schema. In both cases, it gives me a deserialization error :
ERROR {"type":0,"deserializationError":{"target":"value","errorMessage":"Error deserializing message from topic: debezium-topic-1","recordB64":null,"cause":["Failed to deserialize data for topic debezium-topic-1 to Avro: ","Error retrieving Avro value schema for id 76","No content with id/hash 'contentId-76' was found.; error code: 40403"]
In the case of this error, the Global-ID (included with the message) was 76, but the Content-ID in the registry was 64.
Is there a way to make ksqlDB use the official Apicurio Registry API (/registry/v2/) or alternatively, is there a way to force it to use “Global-Id” instead of “Content-Id” ?
Thanks for the help!
