Command line tools and Avro

kafka-avro-console-consumer is great for working with the Avro. If you are mixing streams where some keys are strings and some keys are Avro, I highly recommend not using the StringDeserializer for deserializing the key, since it will not expose any non-ASCII characters to the output.

Instead, use BytesDeserializer, and if you happen to see the 0x00 magic-byte in the key, it will quickly point out that the contains Avro and that could be the reason your join() in your Kafka Streams application is failing.

Here is an alias I have configured for my local development:

alias kaac='kafka-avro-console-consumer --bootstrap-server localhost:19092 --property print.key=true --property key.separator=\| --key-deserializer=org.apache.kafka.common.serialization.BytesDeserializer --from-beginning --topic '
2 Likes

That’s brilliant! Thanks @nbuesing!