I am writing a sink connector, and trying to use the pre-commit function.
I am returning the last offset that was consumed successfully by the sink system, for each partition.
But when i look at the consumers board in the confluent GUI, it will always show me that the consumer is X messages behind (X is number of partitions). Do you have any idea why?
When checking the sink system, i can see that all the records from the topic are there, so the issue is the offset that is returned by precommit.
Should i return the offset that is safe to commit? (the last one that consumed successfully) or the first one i want to consume next?
Thank you.