When using a left join with a grace period, the non-joined rows are not emitted. If i do not use a grace period, they are emitted. I have an example of the issue here:
The readme has instructions of how to reproduce.
Hoping to figure out if I’m doing something incorrectly here. Thanks!
Copy my reply from the ticket:
I did not look into the details of the link, but does the example process a finite data set? If yes, it could explain what you observe. If input data stop, stream-time which is based on the event-time of the input data does not advance, and thus the join window cannot be closed. Thus the left join result cannot be emitted.
I don’t think that there is a bug.
The difference between using GRACE PERIOD and not using this clause is by design to preserve backward compatibility for existing queries. If you don’t specify the newly added GRACE PERIOD clause we use the old behavior and emit (potentially spurious) left/outer join results eagerly. However, those results are strictly incorrect and we fix this behavior to avoid those (incorrect) spurious results – you enable the fix by using GRACE PERIOD clause. Mid-term, we plan make GRACE PERIOD mandatory and thus want to disallow the old (incorrect) behavior.
Thank you for both of your comments. I have added an additional comment in the ksql issue: Left joins with grace period do not emit non-joined rows · Issue #9084 · confluentinc/ksql · GitHub . We can close this thread out and use the issue for further communication. Thanks again!
This topic was automatically closed after 30 days. New replies are no longer allowed.