Long / Integer Serde won't output value - Kafka 3.2

Kafka Streams
1

Hello, I am trying something like the word count example and trying to get the value as Long for output. Either to a console or for another consumer application. However, there seems to be an issue with the Long serialization. Basically any version of the word count example I found on GitHub does not seem to work with my current version of Kafka streams / client 3.2.
E.g. this official one from Confluent on Github. The output will only be the key, not the value.

//this below works fine, when using a String to output the number
        ktableoutput.toStream().mapValues(v -> v.toString() + " additional information").to("testoutput", Produced.with(Serdes.String(), Serdes.String()));


//ideally, this should output the Long Number directly, however DOES NOT WORK
        ktableoutput.toStream().to("testoutput", Produced.with(Serdes.String(), Serdes.Long()));

I did try numerous variations (e.g. using Integer) but to no avail.

Using the following relevant config below (also tried it with getName() or without put. )

       props.put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG, Serdes.String().getClass());
       props.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG, Serdes.String().getClass());

So it’s very strange. It seem to not work with any usage of numbers, but works fine when converted to a String.

2

When you say it does not work, what do you observe? Is anything crashing? Are there exceptions? – What command to do you use to read the data?

3

I am also facing the same issue with Kafka v3.0.4.
In a same scenario, when I check the value in Kafka UI (Conduktor UI), I could see the value as ‘Null Null Null Null’ for my key.
So, it is a sequence of Null characters. Ideally it should be a Long value.

Message Details:

{“partition”:0,“offset”:5,“timestamp”:“2023-03-16T19:04:59.117Z”,“timestampType”:“CreateTime”,“key”:{“data”:“who”,“metadata”:{“rawSize”:3,“format”:“String”,“__typename”:“Metadata”},“__typename”:“Deserialized”},“value”:{“data”:“\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0001”,“metadata”:{“rawSize”:8,“format”:“String”,“__typename”:“Metadata”},“__typename”:“Deserialized”},“headers”:,“__typename”:“Record”,“topicName”:“word-count-out”,“uuid”:“3b96edfd-2e8f-4003-ab06-ac5b0ffe0563”,“processedValue”:“\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0001”}

And in the console-consumer I could see blank value.

4

when I check the value in Kafka UI (Conduktor UI)

Based on the output you provide, it seem that it thinks it’s a string in the value: "format":"String"

“value”:{“data”:“\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0001”,“metadata”:{“rawSize”:8,“format”:“String”,“__typename”:“Metadata”},“__typename”:“Deserialized”}

Given that the output is “\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0001” it seems to be indeed 8 bytes and thus a long though.

I would assume that Conduktor assume the incorrect type? I am not familiar with Conduktor, so not sure how you can tell it that it’s a long? In the end, Kafka itself does not store any data type, and thus, it’s not possible to infer it – the client reading the data must know the data type.