Does Confluent s3 connect comsumer resume offset from where it stops?

Kafka Connect
1

Hi everyone. I tried archiving some data on MSK weeks ago, and everything works fine. I retried the process again today but realized the consumer group didn’t start from where it stopped the last time. Is there anything am missing?
below is connect configuration

connector.class=io.confluent.connect.s3.S3SinkConnector
s3.region=ap-southeast-1
flush.size=1000
schema.compatibility=NONE
s3.part.size=5242880
tasks.max=2
topics=kakfa-topics
format.class=io.confluent.connect.s3.format.json.JsonFormat
partitioner.class=io.confluent.connect.storage.partitioner.DefaultPartitioner
value.converter=org.apache.kafka.connect.storage.StringConverter
storage.class=io.confluent.connect.s3.storage.S3Storage
s3.bucket.name=archive-bucket
key.converter=org.apache.kafka.connect.storage.StringConverter

Thanks

2

Have you ran kafka-consumer-groups command to verify offsets are being committed for the connector?

There’s no property specific to the worker properties to control offsets. You’d could override consumer properties, but by default it should resume from previous committed offsets.

3

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.