We have a use case where we want an external process/script to initiate a one-time transform on the entire Streams state while the application is running. Our understanding is that there is no way to schedule punctuators dynamically - they must be scheduled at Processor initialization time. We’re considering setting some sort of flag in an external data store and querying this store when our existing scheduled punctuator runs, informing it to potentially take an additional action. But this approach is rather complex, as each punctuator instance must read and register receipt of this flag independently to guarantee the transform runs exactly once on each state store. We’ve also looked into Interactive Queries, but it seems that those are read-only. Are there any mechanisms within Streams that might enable us to achieve this pattern?
Our understanding is that there is no way to schedule punctuators dynamically - they must be scheduled at Processor initialization time.
No. You can register a new punctuation at any time.
Not sure what your end-to-end use-case is. It seems to be an unusual pattern that you want to suddenly change all entries in a state store. Maybe you can give some more background?
The main issue I see anyway it, that it might take a long time to update each row in the store, and thus, no processing would happen during this time. Thus, you would also need to make sure that the thread does not drop out of the consumer group.