I think that the rockdb database is in memory and is lost if/when servers are recycled. Where does the ktable data get persisted? I think the docs insinuate that it’s persisted to kafka topics, but I’m looking to confirm.
Also, what would happen if a ksqlDB cluster was recycled/refreshed? would processing stop while the rockdb key/value store was reloaded?
RocksDB does actually write to local disk. But yes, writing to local disk is not sufficient, and we keep a changelog topic in Kafka.
And yes, if the state for a query must be reloaded from the changelog, no data processing would happen for this query. At least for the parts that need the store. – The query might be partially be processed while state is recovered. In the end, ksqlDB is a data parallel system, and thus, if you have let’s say 4 partitions, and need to restore RocksDB for one partition, the other 3 partitions would still get processed, but only one partition is paused until state is recovered.
This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.