Should/Will Kafka Connect support schema evolution using Avro 1.10.X enum defaults?

Kafka Connect Self-Managed Connectors
8

Thanks for the great description @GuusDeGraeve!

I can see that in checkMaybeCompatible() it is indeed doing an equi-comparison on the parameters, which appears to be the enum parameters:

} else if (!Objects.equals(source.parameters(), target.parameters())) {
            throw new SchemaProjectorException("Schema parameters not equal. source parameters: " + source.parameters() + " and target parameters: " + target.parameters());
}

So any narrowing or expansion of any enum, for any schema, would indeed fail. It’s not clear to me why this must be the case.

Before we file a bug ticket, I have one more question: Are you able to write the data to S3 in AVRO format instead of PARQUET?

I am wondering if there is an issue converting the AVRO enums to PARQUET, which is why this particular check is failing. I am wondering if this results in a failure because PARQUET does not have the same default enum standard that AVRO does. I admit that I am not an expert in the KC code, but if we can validate that this same exception occurs if you’re trying to write the files directly as AVRO format, that may narrow down the scope of the issue.