I am using node-rdkafka library. How was offset management handled if I set ‘auto-commit: off’ and also didn’t set any manual commit using commit()
The offsets for that client won’t advance, which means if there’s a failure or you shut down the application, it will start consuming from the offset stored in the _offsets internal topic; when you started initially, you’ll reprocess everything. If you turn off auto-commit it’s best to make periodic calls to commit() to avoid this situation