Stream composite keys

ksqlDB
1

I’m trying to create a stream with a composite key following the answer from @rmoff in apache kafka connect - How to create KSQL table from a topic with composite key? - Stack Overflow, however, like another recent commenter, I’m running into “MY_COMPOSITE_KEY must be a column in the source schema since PARTITION BY is applied on the input.”

Any suggestions? I can duplicate all the details here if it will help.

As further background, the whole reason I’m trying to do this to deal with the issue that we can’t join on more than one column, as described here by @vxia : KSQLDB: JOINs on multiple conditions are not yet supported · Issue #8574 · confluentinc/ksql · GitHub

Any help is much appreciated?

2

Found a solution:

CREATE STREAM repartitioned_stream AS
   SELECT * FROM my_stream
   PARTITION BY STRUCT(K1 := <expression 1>, K2 := <expression 2>)
   EMIT CHANGES;

See: Keys Unlocked - ksqlDB 0.15 Supports Even More Data Types

3 Likes
3

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.