Using Kafka to Discover Events Hidden in your Database

An oldie but a goodie by Anna McDonald! I love talks like these; for people who are still figuring out how to move to being event-based, it provides perspective and a viable solution.


Is your RDBMS the center of everything? Do you have multiple applications, batch jobs, and direct consumers hitting your RDBMS? In order to cover everywhere an event takes place would you need to update multiple services? Moving to event-driven development can seem overwhelming when faced with this scenario; that’s where Kafka comes in! Using Change Data Capture (CDC) and Kafka Streams gives you access to every single event in your space-if you only know where to look. In this talk we’ll discuss: - The properties in a CDC message used in identifying events - Techniques for defining high-level events and how to sniff them out using Kafka Streams - How to define predicates for CDC events - Recommendations on handling events that require more than one table We’ll wrap up by discussing how to structure your event schema to handle a mix of data derived and more traditionally produced events. Learn to be a certified event investigator FTW!

1 Like