Easy way to reset Kafka Connect Connector Offsets

Hi Kafkateers!

Do we have an easy way to reset offset for a connector for a particular topic?
This seems to be a really “must-have” feature, especially when you are debugging your connector configuration, but it seems to be absent.

Probably, a REST API method, looking somewhat similar to this, would be really useful:

POST /connectors/(string:name)/topics/(string:topicName)/reset-offset

I know that there are some workarounds for this, like changing the name of the connector, so it is treated by Kafka Connect as a brand-new one, but this just does not feel a right thing to do.

I know also, that we can reset offset of the connector consumer group using kafka-consumer-groups utility:

./bin/kafka-consumer-groups --bootstrap-server <broker-host>:<broker-port> --group <connector-name> --reset-offsets --to-earliest --execute --topic <topic-name>

But this seems to be not enough, since as it was described here, some really dirty and low-level things should be done to the _connect-offsets topic as well, which is bulky and not really convenient.

Or am I missing something and there IS a simple way to reset connector’s offset for a particular topic, besides the ones I mentioned above?

Hi @whatsupbros,

There is an open KIP-199 for this, and related discussion on the Apache Kafka mailing list

1 Like

Oh, I see! Thank you for letting me know, @rmoff!

Are we going to have also a REST API endpoint for this (which will probably leverage the same tool under the capot) or will this only be available as a CLI tool?

I don’t know if here is the best place to ask - since Kafka Connect is part of Apache Kafka, questions about its development are probably best addressed to the Kafka mailing list.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.