Avro schema update keep behind a feature flag

Schema Registry
1

Hi!

Currently I have a Kafka producer application in production which use schema registry.
I need to add two new optional fields to my schema with default value null.

However I need to do my side changes first and keep this behind a feature flag until consumers adjust to this new fields.

I generate classes from my avro schema at maven build time.

Do you see any proper way to feature toggle schema updates at run time?

Because even I don’t set the values to 2 new fields it sends those new fields with Kafka message with null values.

Is there any dynamic way to use versions of schema at runtime from Producer end?

Thanks in advanced!

2

Are you using BACKWARD or BACKWARD_TRANSITIVE compatibility mode?

Assuming you are, then this is typically done as follows:

  1. Schema is at v1
  2. Producers / consumers both on v1
  3. Add optional field and publish v2 schema
  4. Update consumers to use v2
  5. Update producers to use v2

Why isn’t this possible? i.e., instead of this:

However I need to do my side changes first and keep this behind a feature flag until consumers adjust to this new fields.

… why does your side (the producer, right?) have to be updated first?

3

Hi,
Thank you for you answer.
Yes compatibility mode is BACKWARD. However consumers are different teams who will do this their side changes in later time. That is why I was looking for a solution to keep new schema update under a feature flag.
Yes I got your point, in this case consumer has to update first.

Thanks

4

:+1: yes, changing the order of client updates seems best. You could technically choose schema version dynamically at runtime (if using Avro, Avro’s GenericRecord would be a natural fit for doing this) but it feels like an antipattern.