New blog: 🎄 Twelve Days of SMT 🎄 - Day 11: Predicate and Filter

Apache Kafka 2.6 included KIP-585 which adds support for defining predicates against which transforms are conditionally executed, as well as a Filter Single Message Transform to drop messages - which in combination means that you can conditionally drop messages.

As part of Apache Kafka, Kafka Connect ships with pre-built Single Message Transforms and Predicates, but you can also write you own. The API for each is documented: Transformation / Predicate . The predicates that ship with Apache Kafka are:

RecordIsTombstone - The value part of the message is null (denoting a tombstone message)

HasHeaderKey - Matches if a header exists with the name given

TopicNameMatches - Matches based on topic


Read more: https://rmoff.net/2020/12/22/twelve-days-of-smt-day-11-predicate-and-filter/