KTable-KTable foreign key join emits null values when no right matching key found

Kafka Streams
1

Hi,
In ktable-ktable foreign key join, when record arrives in the left side and no matching key found in the right side, results are emitted from the join - [key: k, value: null].
I can’t find this behavior in the documentation. In regular join (not foreign key) when no matching key found in the right side, the join does not emits any result (this is my expected behavior also from the foreign key join).

Kafka 3.8 & Kafka-streams 3.8.1

Thanks in advance.

2

It depends a little bit, but yes, there is a few cases in which a FK-join cannot determine if the old value (before the update) was producing a join result or not, and thus, if the new value (after the update) does not join, a tombstone is emitted to delete a potential result from the old value.

Note: this behavior is not incorrect. If something that does not exist in the result gets “deleted”, this delete is unnecessary but does not change the result, but it’s a no-op semantically.

3

In my opinion it should be added to the documentation, because when your architecture depends on tombstone for deletion logic, this [key: k, value: null] can be dangerous because it can trigger some deletion logic without an intention.

4

Improving the documentation is always a good idea. Feel free to open a PR (or file a ticket).

it can trigger some deletion logic without an intention.

Not totally sure how this could be a problem? Should it not just become a no-op? If you do, eg, DELETE FROM myTable WHERE ID="foo" and there no row with ID=“foo” there won’t be any error – it’s also how most key-value stores works from my understanding, a store.delete("foo") would not throw an error…