KTable-KTable joins, is colocation needed?

Kafka Streams
1

Hi, I think when KStreams is involved in a join, colocation (same number of partitions) is needed?
What about in a KTable-KTable join? Can they have different number of partitions? Is this true in both key-key as well as in foreign key joins?
Thank you

2

Hi,

In KStream-KStream, KStream-KTable, and KTable-KTable joins, both sides of the join need to have the same number of partitions. Kafka Streams will check for this, and if the number of partitions isn’t the same, Kafka Streams will throw a TopologyException.

Foreign key joins don’t require the same number of partitions between the two tables, since you provide a foreignKeyExtractor function that extracts the key for the join from the “calling” or left side table of the join.

HTH,

Bill

2 Likes
3

Hi @nmarasoiu,

Here is also a link to the relevant docs:

Best,
Bruno

1 Like
4

@nmarasoiu Could you please describe the use case for your KTable-KTable join? I haven’t seen an example of this type of join in KS/ksqlDB.

5

Is it possible to dynamically create 1 kstream and 1 ktable and then do a join on them?