JDBC sink connector with nested JSON for SQL server

Kafka Connect
1

Hey, I am using JDBC sink connector and want to put nested JSON data in SQL server from Kafka topic. Do I need to flatten it using SMT in this case or there is any other ideal way to deal with?

Thank you.

2

Hi there! I’d personally use the flatten SMT and then rename fields as necessary (using rename SMT) before putting in the database. For more control, you can certainly also implement a streaming application to do these transformation for you. But it depends on your needs.

2 Likes
3

@danicafine how we can specify more than one transform in a connector configuration?

i am attaching one sample below, is it correct?
“transforms”: “flatten,RenameField”,
“transforms.flatten.type”: “org.apache.kafka.connect.transforms.Flatten$Value”,
“transforms.flatten.delimiter”: “_”,
“transforms.RenameField.type”: “org.apache.kafka.connect.transforms.ReplaceField$Value”,
“transforms.RenameField.renames”: “After_Name:Name”

4

Yes, you have the right idea! This blog explains it a little more formally.

2 Likes
5

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.