JDBC Connector - differentiating between create and updates

Kafka Connect Self-Managed Connectors
1

I am using the JDBC connector. From the payload of updates from the Kafka topic used in “timestamp+incrementing” mode, how can I differentiate between creates and updates in the database?

2

hey @obelix74

welcome to the forum :slight_smile:

I guess we are talking about the jdbc sink connector?

best,
michael

3

Hi @mmuehlbeyer

Thanks. No, I was talking about the JDBC source connector.

I have a MariaDB database and I configured the source connector to monitor a database and publish the change events to a Kafka topic.

When I look at the payload from the Kafka topic, I can’t tell if it was a CREATE or an UPDATE.

4

understood
afaik it’s not possible see

best,
michael

5

Use Debezium instead. It’ll include an op field in the records that indicates updates, inserts, as well as deletes.

Otherwise, capture all records in a KTable, then check if a key already exists. If so, it’s an update.

6

Hi Jordan

Thanks, I am working on Debezium as well. I need to work with MariaDB + Bitemporal data and Debezium currently does not support that. I am working on an experimental PR on Debezium to support that at the moment.