Getting CL specific fields in database

Kafka Connect Self-Managed Connectors
1

Hi All,

This question is regarding how to get the CL specific fields and store in oracle database table.
I have a topic where XML message is getting published and it is getting stored in database through JDBC Sink connector.
However, I also need to store CL key, CL offset, CL partitions and fingerprint along with XML message.

Is there any Kafka parameter to get these details/values from CL or any other way to store these details in oracle database table.

Thanks in advance.

2

Can you define what CL means in this context? And perhaps give a sample of the data and example of what you are wanting to stream to the target database?

3

CL means communication layer where message is getting published and same is getting stored in database.
However, can we store key, offset, partition and fingerprint data/values from communication layer in database through JDBC sink connector

4

Hi,
Is there any way to achieve this.

5

You can use the InsertField Single Message Transform to add partition, offset, and topic details.

Ref: 🎄 Twelve Days of SMT 🎄 - Day 6: InsertField II

6

Thank you so much.
Will try this solution.

7

It’s working fine.
Also trying to get the Fingerprint from Kafka same way, but it’s giving an error.
Is there any specific name for retrieving fingerprint value.

8

How are you trying to get the values from Kafka?

9

Same way as shown in the video using InsertField$value.
For fingerprint:

"transforms.insertFingerprint.type" : "org.apache.kafka.connect.transforms.InsertField$value",
"transforms.insertFingerprint.Fingerprint.field": "Kafka fingerprint".

Have also added this in “transforms” parameter.
It’s giving an error as No field insertion configured.

10

Is there any different name for that

11

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.