Best Practice in using Kafka Connect/CDC on a normalized database while still exposing a nice entity API

Architecture and Design
1

As inspired by the Kafka Connect JDBC primary key talk on Youtube. Kafka Connect JDBC sink deep-dive: Working with Primary Keys - YouTube

Hi Robin, thanks for the video. I love the extra details!

I have a question that is not so much about CDC, but on how to share the normalised data of a database as a nice single entity (document) with the organisation.

As an example I imagine an order, which refers to an address and a list of order items each in their own table.

The order, the address and the order items are all mutable - the customer can change it all and as a result I want to send to my clients a single Kafka document with everything in it.

How would you go about this? Both in combining the 3 change streams into a single new stream regardless of what changed, and only sharing the nice API with the organisation?
In short, Iā€™d love a video with a special focus on sharing a maintainable API with the rest of the organisation

1 Like