Preparing to move away from ksql queries file (= headless mode)…
Our ksql queries file consists of a number of
CSAS commands, as well as
Because headless mode seems to be phased out at some point and we’re unable to actually remove anything from the queries file that we don’t need anymore, we’re trying to plan our migation.
The generated consumer groups contain a prefix, a short form of the command and an increasing integer that represesents which n-th statement the command is in the queries.file. This is also the reason why we can’t just remove something from the middle of the file (the query number in the generated group name will break).
So to move to interactive mode, we either need to have a way of specifying the consumer group name in a command or we’ll need to “rename” the existing group names to the group names that will be autogenerated from the interactive commands, correct? (before actually creating the queries)
Otherwise, everything will start from earliest or latest again, I guess?
What’s the best way to do this? Trying to rename the groups or wait for semantics to allow the group name to be specified in the KSQL command?
INSERT INTO queries seem to get an improvement for that in 0.15. (feat: new WITH(id = '<query-id>') clause to INSERT/SELECT · Issue #6533 · confluentinc/ksql · GitHub)
Asking this here instead of Slack, because this might be a long lived question