# JDBC Source Connector with JSON payload having a field with array of records

**URL:** <https://forum.confluent.io/t/jdbc-source-connector-with-json-payload-having-a-field-with-array-of-records/13492>\
**Category:** Kafka Connect\
**Created:** [10 December 2024 20:54 UTC](https://forum.confluent.io/t/jdbc-source-connector-with-json-payload-having-a-field-with-array-of-records/13492 "2024-12-10T20:54:08Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![amamgai](https://avatars.discourse-cdn.com/v4/letter/a/ecb155/32.png) [@amamgai](https://forum.confluent.io/u/amamgai)\
**Post date:** [10 December 2024 20:54 UTC](https://forum.confluent.io/t/jdbc-source-connector-with-json-payload-having-a-field-with-array-of-records/13492/1 "2024-12-10T20:54:08Z")

</div>

Hi All,

We are trying to use Kafka Source Connector which read from PostgreSQL Database JSONB column. Payload has a field which is array of records. When we use JsonConvertor we get InvalidRecordException

Caused by: org.apache.kafka.common.InvalidRecordException: Log record DefaultRecord(offset=0, timestamp=1733863040526, key=0 bytes, value=993 bytes) is rejected by the record interceptor io.confluent.kafka.schemaregistry.validator.RecordSchemaValidator\n"}],“type”:“source”}

If we use AvroConvertor we get Not in Union Exception

Schema:

{  
“type”:“record”,  
“namespace”: “com.discover.fraud.card.account.activity”,  
“name”:“AccountServicingAudit”,  
“doc”: “This event send audit details”,  
“fields”:[  
{  
“name”:“accountKey”,  
“type”:“string”,  
“doc”: “Account Key used while processing request.”  
},  
{  
“name”:“transferSeqNumber”,  
“type”:“string”,  
“doc”: “Transfer Sequence Number used while processing request.”  
},  
{  
“name”:“racfId”,  
“type”:“string”,  
“doc”: “Racf Id”  
},  
{  
“name”:“timestamp”,  
“type”:“string”,  
“doc”: “Timestamp of error”  
},  
{  
“name”:“changeType”,  
“type”:“string”,  
“doc”: “Change Type”  
},  
{  
“name”: “changeDetails”,  
“doc”: “List of fields/attributes updated”,  
“type”: [  
{  
“type”: “array”,  
“items”: [  
{  
“type”: “record”,  
“name”: “changeDetails”,  
“fields”: [  
{  
“name”: “name”,  
“type”: “string”,  
“doc”: “Name of the attribute modified”  
},  
{  
“name”: “value”,  
“type”: “string”,  
“doc”: “Value of the attribute modified”  
}  
]  
}  
]  
}  
]  
}  
]  
}

Payload:

{  
“racfId”: “ABCD”,  
“timestamp”: “2024-Jan-25 12:24:16.753”,  
“accountKey”: “1234567890”,  
“changeType”: “TEST”,  
“changeDetails”: [  
{  
“name”: “Status”,  
“value”: “R”  
},  
{  
“name”: “4506TFlag”,  
“value”: “102424”  
},  
{  
“name”: “fcmsQueueName”,  
“value”: “Outsourcing 13”  
},  
{  
“name”: “orderStatus”,  
“value”: “DIRECTREVOKE”  
}  
],  
“transferSeqNumber”: “001”  
}

Kafka Config

curl   
-i -X PUT -H “Accept:application/json”   
-H “Content-Type:application/json” [http://localhost:8080/connectors/cboAudit\_source\_connector/config](http://localhost:8080/connectors/cboAudit_source_connector/config)   
-d ‘{  
“connector.class”: “io.confluent.connect.jdbc.JdbcSourceConnector”,  
“connection.url”: “URL”,  
“connection.user”: “ID”,  
“connection.password”: “PWD”,  
“connection.attempts”: “3”,  
“query.retry.attempts”: “3”,  
“errors.log.enable”: “true”,  
“errors.log.include.messages”: “true”,  
“numeric.mapping”: “best\_fit”,  
“poll.interval.ms”: “5000”,  
“topic.prefix”: “TOPIC”,  
“mode”: “timestamp+incrementing”,  
“timestamp.column.name”: “crte\_tms”,  
“incrementing.column.name”: “process\_seq\_nbr”,  
“key.converter.latest.compatibility.strict”: “false”,  
“key.converter.schemas.enable”: “true”,  
“key.converter.use.latest.version”: “true”,  
“key.converter”: “org.apache.kafka.connect.json.JsonConverter”,  
“key.serializer”: “io.confluent.kafka.serializers.KafkaAvroSerializer”,  
“value.converter.auto.register.schemas”: “false”,  
“value.converter.latest.compatibility.strict”: “false”,  
“value.converter.schema.registry.ssl.truststore.location”: “JKS”,  
“value.converter.schema.registry.ssl.truststore.password”: “pswd”,  
“value.converter.schema.registry.url”: “Registry-loc”,  
“key.converter.schema.registry.url”: “Registry-loc”,  
“value.converter.schemas.enable”: “true”,  
“value.converter.use.latest.version”: “true”,  
“value.converter”: “org.apache.kafka.connect.json.JsonConverter”,  
“value.serializer”: “io.confluent.kafka.serializers.KafkaAvroSerializer”,  
“query”:“SELECT \* FROM (select FLX\_PYLD\_EVNT::json-\>\> '”’“‘accountKey’”‘"’ as accountKey,FLX\_PYLD\_EVNT::json-\>\> ‘"’“‘transferSeqNumber’”‘"’ AS transferSeqNumber,FLX\_PYLD\_EVNT::json-\>\> ‘"’“‘racfId’”‘"’ as racfId,FLX\_PYLD\_EVNT::json-\>\> ‘"’“‘timestamp’”‘"’ as timestamp,FLX\_PYLD\_EVNT::json-\>\> ‘"’“‘changeType’”‘"’ as changeType,FLX\_PYLD\_EVNT::json-\> ‘"’“‘changeDetails’”‘"’ as changeDetails,crte\_tms,process\_seq\_nbr from CBO\_PROCESS\_ACTIVITY where FLUX\_TPC\_NM=‘"’“‘business.direct-banking.cross-product.fraud.cards.account.activity’”‘"’) AS AuditActivity"  
}';

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex019/uploads/confluentcommunity/original/1X/c49438c90c9df282e9996fdf6971be890c71b65a.svg) [@system](https://forum.confluent.io/u/system)\
**Post date:** [9 January 2025 20:54 UTC](https://forum.confluent.io/t/jdbc-source-connector-with-json-payload-having-a-field-with-array-of-records/13492/2 "2025-01-09T20:54:32Z")

</div>

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.