Deserialization of JSON Messages in .NET

[Sunspoticus]

Hi,

Stuck on an issue, hoping the community can help. New to Kafka, experienced .net guy.

The basic scenario:

• Using Confluence.Kafka and with help from users on this forum, I was able to connect and retrieve messages. (using .NET Standard)
• There is no schema for the messages and the response is JSON. I tried to deserialize it using vanilla Newtonsoft.JSON and it threw all manner of fits. After some googlin’ around, I found this code snipnet: .net - Is there a way to use confluent Kafka Dotnet JSON serializer WITHOUT schema registry, - Stack Overflow but couldn’t get it to work standalone or grafted into my client app.

Its been a long road just getting it working and once I get it into a manageable state and I can write the data off and be the hero-of-the-moment…

Any help would be MOST appreciated.

[vnadkarni]

@Sunspoticus

@Sunspoticus

couldn’t get it to work standalone or grafted

What do you see when you try?

[Sunspoticus]

I think its the definition of my custom deserializer. From everything I’ve been able to find out, it’d be much easier if I had a schema to refer to but I’ve been told this particular installation does not utilize the schema registry.

All the examples of deserialization I have been able to find require said registry except for the cited example.

[gklijs]

When just JSON is used, most times you should be able to use the byte / string serializer, and transform that into some JSON object with a library available for that language.
So don’t use the Dotnet serializer, as it’s created specifically for use with schema registry.

[Sunspoticus]

Its frustrating because none of the examples seem to work properly and I’m not sure what I’m doing wrong.

My Kafka message has two top-level fields in it. RequestID and Msg. Both, at this level, are basically strings. I think I’m hung up on the proper definition of the deserializer declaration found in most examples. that is, in the classic using statement where you set options… It refuses to take my SetValueDeserializer(new JsonDeserializer()

[gklijs]

Have you tried using the utf8 serializer, and then use something else to turn the string to JSON? Seems like confluent-kafka-dotnet/Serializers.cs at 4de84f7c3c4b0d140f643dcb21110fd523efa2cc · confluentinc/confluent-kafka-dotnet · GitHub is what you need?