Get unserialized protobuf message as on bus using console consumer?

Kafka Streams
1

We have a publisher which is publishing protobuf messages on kafka. What we want to do is dump some of the serialized message on our side using kafka-console-consumer or kafka-protobuf-console-conumer. So we want hex messages as published on bus instead of the deserialized message. Is it possible using some property on consumer side using kafka-console-consumer or kafka-protobuf-console-conumer ?

Thanks

2

Seems like a “Client” question. Kafka Streams is Kafka’s Java library for stream processing…

But in general yes, using the kafka-console-consumer you can use any deserializer. If you just want to get raw bytes, you can use the ByteArrayDeserializer.

3

Thanks @mjsax i tried using the kafka-consoler-consumer and pass parameter --key.deserializer org.apache.kafka.common.serialization.ByteArrayDeserializer and --value.deserializer org.apache.kafka.common.serialization.ByteArrayDeserializer but this did not work. I am not getting the hex output that i expect to see on the bus which would be something like b'\n\x04ABCD\x12\x0b7\xd0\xabc(\x012\x12\x13987@!)\\\x7f\xc2@' what i get is something like this
[B@abc123 [B@cdf235 and so on

4

Well, as I said, ByteArrayDeserializer gives you raw bytes… I did not expect that it will give you hex encoding… You would need to write you own deserializer.

1 Like