Select events from a topic between two dates without emitting changes

ksqlDB
1

Hello,

I am developing an historical information endpoint, so the user can select a device id and see when it was online/offline (daily). So, I would like to select all the events with the same key between two dates from a Kafka topic in ksqlDB without emitting changes. I find the “tumbling window pull query” feature (kind of) similar, but it seems to me that it is only for aggregated data.
Which feature should I use?

Here is a simplified example of the data that I am using:

CREATE STREAM gateway_connections_daily (
    gatewayid VARCHAR KEY,
    connection VARCHAR
) WITH (
    kafka_topic='gateway.fct.connection.0',
    value_format='JSON'
);

Thanks for all in advance.

3

Hi, from the sounds of it a table will be appropriate here, but it’s hard to say for sure without a better understanding of the data. Can you share a sample of data (dummy, if necessary) and expected output?

4

I think this is not supported atm. You would currently issue a push query over the stream, but would need to terminate the query manually, after it reached the end of the topic and thus does not return any more data (it’s of course tricky to know when the end of the topic is reached…).

We did discuss this use case in the past, but not many people requested such a feature yet. Good to know that there would be interest.