KafkaJs schema registry decode wire format

swaroopduth · 19 January 2024 21:21

Hi All

I am using Kafkajs and Kafkajs schema registry for JSON messages. And I use below node library to encode with schema registry as a producer. As you see the below uses wire format to encode the message by adding magic byte and registryId prefixed to the payload.

github.com

kafkajs/confluent-schema-registry/blob/master/src/wireEncoder.ts

const DEFAULT_OFFSET = 0

export const MAGIC_BYTE = Buffer.alloc(1)

export const encode = (registryId: number, payload: Buffer) => {
  const registryIdBuffer = Buffer.alloc(4)
  registryIdBuffer.writeInt32BE(registryId, DEFAULT_OFFSET)

  return Buffer.concat([MAGIC_BYTE, registryIdBuffer, payload])
}

But since the consumer of this message is a Java client. It fails to deserialize it as the message contains prefixed magic byte and registry Anyone faced simillar issue and how to deserialize the same

Topic		Replies	Views
Json with schema registry Schema Registry	1	462	12 March 2024
Deserialization of JSON Messages in .NET Non-Java Clients	5	3307	22 January 2022
Using Schema Registry for JSON Records Schema Registry	4	4490	13 August 2021
Issue with json schema registry Schema Registry	0	1963	7 April 2023
Deserializing a byte[] array from a Kafka ByteArrayDeserializer into a GenericRecord or a SpecificRecord with schema Kafka Streams	1	3144	20 July 2023

KafkaJs schema registry decode wire format

Related Topics