Hello, I have a following situation when the use case is -
I have a kafka stream application that reads events from topic A, does some transformation on data and stores it inside Compact topic B.
Now I can not understand till the end if it is possible to do reusing of topic B, and load it as a globalKTable , as now for topic B I might have multiple values for same key, and I wonder if before writing to topic B I can know and get the value that was stored before?
Thanks in advance
Like every table, global or “regular”, the used key-value store will have a “primary-key constraint” and thus only have one row per key.
It is possible to write into a topic, and read the same topic as global-KTable, but there won’t be any “synchronization” guarantees. Global stores/KTables do in general not provide any “synchronization” guarantees, and are design for use cases with rather static data, ie, data which does not change regularly.
If you have two input record, <k, v1> and <k,v2> and you modify a state store using <k,v1> to set a state entry <k,s1> which you want to read when processing <k,v2>, you would need to use a regular table/store, and partition the data by k.