Utilize topic keys in KSQL Stream

ksqlDB
1

I am looking to incorporate the key value from a topic’s messages into a KSQLDB Stream to be a unique identifier. The keys are unique to what sensor the data is coming from. I am in a situation where the format of the messages coming from our topic cannot be changed and the three columns of data we are receiving do not have anything that would be able to identify what sensor that row came from. Is there a way to include the key from each topic message into the Stream?

I have seen some older posts online that refer to ROWKEY as something I can use in a select statement, but that cannot be resolved when I try it. I am only able to use ROWTIME.

2

How did you declare the schema of the STREAM? If you want a column to take the data from the message key, you can declare the column as KEY:

CREATE STREAM foo (myKey VARCHAR KEY, someOtherColumn INT,...)

Column myKey will get the data from the message key, as it’s declared as KEY column. The message key must be a String for this case (as we used type VARCHAR) – of course, you can also use any other data type. If you have a complex key (eg, JSON), you can also declare multiple KEY columns (similar to “value” columns).

Cf Data definition - ksqlDB Documentation

I have seen some older posts online that refer to ROWKEY

In older versions, it was not possible to just declare some “key columns” in the schema but ksqlDB provided the ROWKEY column. Using ROWKEY column has quite some disadvantages: (1) the name is hard coded and cannot be customized; (2) only primitive data types could be support.

3

This topic was automatically closed after 30 days. New replies are no longer allowed.