hi all… the below code / curl use to work when i was json.marshalling my msg int byte to post. I’ve now switched to a schema registry client which also calls for using the associated json serialize (does a schema check before posting msg) curl curl -X POST \ -H "Content-Type: application/json" …

Resolved change VALUE_FORMAT='JSON', to VALUE_FORMAT=JSON_SR, changed QUANTITY number to QUANTITY double this last one did not make sense, as it’s just how many items was bought… but it worked so not going to argue…

How were the events in the json_salesbaskets topic serialized when produced? What does the structure look like? The combination of JsonConverter and value.converter.schemas.enable=true in your connector config requires a specific message format with payload and embedded schema root elements, e.g.: {…

Not to make a mess here Can we take this offline… G

My data is being produced by an Golang app that I wrote to test bits, the main purpose is to build understanding. With the new create schema registration client and then the serializer (as per below)… and then post the performance have gone from 10000txn/sec (this was json.marshal) down to 500/sec.…

Types Declares. type Tp_basket struct { InvoiceNumber string `json:"invoiceNumber,omitempty"` SaleDateTime string `json:"saleDateTime,omitempty"` SaleTimestamp string `json:"saleTimestamp,omitempty"` Store TStoreStruct `json:"store,omitempty"` Clerk …

OK I think the issue is that you’re using JSON Schema so should use the io.confluent.connect.json.JsonSchemaConverter converter in your connector config. The converter in your config (org.apache.kafka.connect.json.JsonConverter) plus value.converter.schemas.enable=true is a way to support schemas em…

[image] dtroiano: io.confluent.connect.json.JsonSchemaConverter thanks. seems I thought I was actually doing the non embedded schema version. G

hi … ok so i have 2 topics i build up, a sales basket and a sales payment… done exactly the same, you solution above worked on sales basket… getting the sink to push the payloads into the mongodb… now the weird one, for sales payments I’m getting the below in the mongodb collection, did check the t…

hi hi, back to this one… got the sink to work, using your above… tried to create a ksqldb stream on the topic, getting the following… looks very similar to above. ksqldb-server | [2024-06-24 07:25:17,763] WARN stream-thread [_confluent-ksql-default_transient_transient_JSON_SALESBASKETS_663891254…

Fixed this one… mostly user error… got the sink to Mongo working… getting problems on ksqldb now and think it’s similar to the original sink problem above that we fixed by changing the value converter. [image] georgelza: ObjectId(‘6675d5f6cf58c5310bc9e770’) Paid:null FinTransactionID:null …

Connect sink failing json serialisation

Kafka Connect

georgelza 21 June 2024 16:52 5

See: https://forum.confluent.io/t/json-with-schema-registry/10261

Topic		Replies	Views
HTTP Sink V2 Connector issue Confluent Cloud	3	165	28 January 2025
Json with schema registry Schema Registry	4	4071	20 June 2024
New blog: Why JSON isn't the same as JSON Schema in Kafka Connect converters and ksqlDB (Viewing Kafka messages bytes as hex) ksqlDB	0	3404	9 March 2021
JDBC Sink Connector with JSON message format without schema Kafka Connect	3	2902	23 January 2024
Kafka Connect, RabbitMQ, and JSON Schema Self-Managed Connectors	4	4313	5 November 2021

Connect sink failing json serialisation

Related topics