Easy way to reset Kafka Connect Connector Offsets

Kafka Connect
1

Hi Kafkateers!

Do we have an easy way to reset offset for a connector for a particular topic?
This seems to be a really “must-have” feature, especially when you are debugging your connector configuration, but it seems to be absent.

Probably, a REST API method, looking somewhat similar to this, would be really useful:

POST /connectors/(string:name)/topics/(string:topicName)/reset-offset

I know that there are some workarounds for this, like changing the name of the connector, so it is treated by Kafka Connect as a brand-new one, but this just does not feel a right thing to do.

I know also, that we can reset offset of the connector consumer group using kafka-consumer-groups utility:

./bin/kafka-consumer-groups --bootstrap-server <broker-host>:<broker-port> --group <connector-name> --reset-offsets --to-earliest --execute --topic <topic-name>

But this seems to be not enough, since as it was described here, some really dirty and low-level things should be done to the _connect-offsets topic as well, which is bulky and not really convenient.

Or am I missing something and there IS a simple way to reset connector’s offset for a particular topic, besides the ones I mentioned above?

2

Hi @whatsupbros,

There is an open KIP-199 for this, and related discussion on the Apache Kafka mailing list

1 Like
3

Oh, I see! Thank you for letting me know, @rmoff!

Are we going to have also a REST API endpoint for this (which will probably leverage the same tool under the capot) or will this only be available as a CLI tool?

4

I don’t know if here is the best place to ask - since Kafka Connect is part of Apache Kafka, questions about its development are probably best addressed to the Kafka mailing list.

5

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.