Say you have 2 streams - chat(keyed by chatId on the topic but rekeyed to userId for this join) & reaction-info(keyed by userId).
Whenever a new chat event is created I want to join to the latest reaction-info sent before the chat event. Users can have multiple chats & multiple reaction-infos
Can either do a left join to a reaction-info table
Or reduce and compare timestamps of events to ensure we get a reaction info before the chat event.
Or probably lots of other things.
However, I also have a db with historic data for these events and short of passing them in order I don’t know how I can ensure that I get the most up to date reaction info event occuring before each chat event. Guidance on an approach to this would be huuuugely appreciated.
I’d prefer to reduce on two streams to allow for corrective updates but would appreciate suggestions on how to load the history correctly with the reaction-info being a stream or a table.
Is there some handy settings that make this waaay easier to do on a table than it is to do in a reduce?