Schema ID returned by cachedSchemaRegistryClient

Schema Registry
1

Is it possible the confluent schema registry return different IDs for the same schema? I am using the below logic to register/get schema ID.

Schema schema = ReflectData.get().getSchema(data.getClass());

    String subjectName = this.getSubjectName(topic, this.isKey, data, schema);

    try {

        if (autoRegisterSchema) {
            id = schemaRegistry.register(subjectName, schema);
        } else {
            id = schemaRegistry.getId(subjectName, schema);
        }

I am facing issues while de-serializing the message on the site2 kafka cluster that keeps mirroring the data from site1

Please guide me here

Thanks
Shrenik

2

Yes, it’s possible to get different IDs because the schema text is compared with a MD5 hash fingerprint. This means any doc comment or whitespace change would result in a unique ID, even if versions are fully compatible.

mirroring the data from site1

Unclear what this means, exactly, but MirrorMaker2 doesn’t copy schemas to a destination registry. You shouldn’t use AvroConverter with it, for example.

3

Thanks @OneCricketeer

I am using ReflectData class from apache.avro library to get the Schema object from the value class object.
Schema schema = ReflectData.get().getSchema(data.getClass());

I have one kafka cluster deployed on active node and other cluster on the standby node. The kafka mirroring service runs on the standby node and it mirrors all the topic data from the active node.

I have kafka streams application(with states store) running on the active node. Now, when I do a switch over and bring up the standby node sometimes I get de-serialization exception while building the state store again. This is because the MAGIC BYTE has the ID which is not present in the schema registry of standby node.

So, I am not sure if the schema registry is returning another Id fro the same schema. Please help me here

4

Thanks for clarifying. As mentioned, mirroring data doesn’t transfer schema ids. You’ll also have to mirror the _schemas topic for the registry, where all the IDs are stored

5

So, the __schemas topic does not get mirrored by default? Do we need any additional configuration for this? I could see schemas present on the standby __schemas topic