Recording ready to view: SPEAKER Q&A THREAD: - Building Streaming Data Pipelines with Apache Kafka

Community Events
1

Watch the recording and ask @danicafine questions about this event in this thread!

  • :spiral_calendar: When: 2 February 2022
  • :speaking_head: Speaker(s): Danica Fine
  • :notebook_with_decorative_cover: Talk(s): Building Streaming Data Pipelines with Apache Kafka

Data pipeline knowledge is important for streaming data pipeline in Apache Kafka. At this Meetups, Danica Fine (Senior Developer Advocate, Confluent) talks about the process of ingesting data from external systems using Kafka Connect, specifically on Change Data Capture (CDC). During this Meetup, we will learn how to transform Kafka data in real-time with ksqlDB stream processing, how to use ksqlDB to join the stream of real-time data to lookup tables. You will also develop an understanding of how Kafka Connect helps with data egress.

:vhs: Find more Meetups Meetup Hub
:earth_africa: Join the Community

2 Likes
2

@danicafine covers :

  • What is a streaming data pipeline?
  • Using an example of user data and page views events streamed into a Kafka cluster, Danica demonstrates the difference between the main ways of querying this data. (The inverted pyramid in Danica’s presentation really helps make sense of it):
    • the Kafka consumer/producer API,
    • the Kafka Streams API and
    • ksqlDB
  • How you can transform your data, for example filter it, doing Stream x Table Inner joins using ksqlDB
  • Sinking your data into an Elasticsearch

Stick around for the Q&A to find out: When should you use ksqlDB vs Kafka Streams, and when the client for COBOL will be released (kidding :wink: )

1 Like
3

Still eagerly awaiting that COBOL client. :wink:

1 Like