JSON Payload with Schema Validation on Kafka REST Proxy

Schema Registry
1

I tried to achieve the scenario where JSON Payload with Schema Validation on Kafka REST Proxy should be done. I followed the below steps but getting the error, can someone please help me on this,
Step-1 – Created Topic and configured confluent.value.schema.validation = true
Step-2 – Added Avro Schema to that Topic
Step-3 – Tried to produce the below sample message to topic using rest-proxy

{
    "value_schema_id": "100032",
    "records": [
		{
			"key": "1111",
			"value": {
				"id": 11,
				"name": "Maarten",
				"faxNumber": "1234567890"
			}
		}
	]
}

But I am getting below error, can someone please help me on this,

{
    "offsets": [
        {
            "partition": null,
            "offset": null,
            "error_code": 50002,
            "error": "One or more records have been rejected due to 1 record errors in total, and only showing the first three errors at most: [RecordError(batchIndex=0, message='Record DefaultRecord(offset=0, timestamp=1678333249854, key=6 bytes, value=51 bytes) is rejected by the record interceptor io.confluent.cloud.kafka.schemaregistry.validator.CloudRecordSchemaValidator')]"
        }
    ],
    "key_schema_id": null,
    "value_schema_id": null
}
2

Hi, i have quite the same issue, have you found a solution ?

For me the erreor is : interceptor io.confluent.cloud.kafka.schemaregistry.validator.CloudRecordSchemaValidator

3

I also encounter this issue with v3 rest proxy endpoint.

The solution was adding a “type” property for the value and key payload.

{
“value”: {
“type”:“JSON”,
“data”: {
“name”: “Name”,
“age”: 10,
“salary”: 19,
“time”:“August 21 2023”
}
},
“key”:{
“type”:“JSON”,
“data”:“YOUR KEY”
}
}

1 Like
4

I am getting same exception. I have java code base and JSON schema configured. Appreciate if anyone could help me on this.

org.apache.kafka.common.InvalidRecordException: Record DefaultRecord(offset=0, timestamp=1715752932349, key=7 bytes, value=50 bytes) is rejected by the record interceptor io.confluent.cloud.kafka.schemaregistry.validator.CloudRecordSchemaValidator

My JSON Schema
{
“$schema”:“http://json-schema.org/draft-04/schema#”,
“title”:“Employee”,
“type”:“object”,
“additionalProperties”:false,
“javaType”:“com.example.model.Employee”,
“properties”:{
“firstName”:{
“type”:“string”
},
“lastName”:{
“type”:“string”
}
}
}

5
  1. Grant read access to kafka account (Which broker is using to connect to other components) to access schema subject.
  2. Make sure you are using schema aware client, so it adds the 5 magic bytes to your json payload while producing message.
  3. Broker side schema validation should be set to true - set validate value schema to true for that topic.

Post your java code (schema aware client) so we can have a look and suggest changes if needed.

6

iam using REST v3 endpoint and in my topic i have enabled schema validation, but unable to produce message to the topic getting this error
Record DefaultRecord(offset=0, timestamp=1715752932349, key=7 bytes, value=50 bytes) is rejected by the record interceptor.
i have one question does REST V# enpoint support schema validation.

7

Yes, it supports schema validation. If “type” is specified it validates the schema.

8

i am using this uri https://xyz/kafka/v3/clusters/xyz/topics/grocery-anonymous-sessions/records and trying to produce messages
with this payload
‘{“value”:{“type”:“JSON”,“data”:{“session_id”: “123”,“interaction_time”: “123”}}}’
but iam facing this error
{“error_code”:50002,“message”:“Record DefaultRecord(offset=0, timestamp=1743514376471, key=0 bytes, value=45 bytes) is rejected by the record interceptor io.confluent.cloud.kafka.schemaregistry.validator.CloudRecordSchemaValidator”}

9

https://xyz:443/kafka/v3/clusters/Xyz/topics/grocery-anonymous-sessions/records -d ‘{“value”:{“data”:{“session_id”: “123”,“interaction_time”: “123”}}}’
{“error_code”:422,“message”:“Error: 42206 : Payload error. Schema Registry must be configured when using schemas.”}

if iam using like this then iam getting this error , i have just enabled in topic config to validate schema for value and added a data contract and trying to hit this uri to produce message . Is there any more config needed to do this

11

The solution was adding a “type” property for the value and key payload.

{
“value”: {
“type”:“JSON”,
“data”: {
“name”: “Name”,
“age”: 10,
“salary”: 19,
“time”:“August 21 2023”
}
},
“key”:{
“type”:“JSON”,
“data”:“YOUR KEY”
}
}

12

the uri iam using is correct or worng ? and can guide me a little how to use v3 rest proxy to produce a message with schema validation enabled with avro serialization?
Even With type specified it is not