Control filename in s3 sink connector (repeated query)

Kafka Connect
1

I’m wondering if I can set the file name for something different than:<prefix>/<topic>/year=YYYY/month=MM/day=dd/<topic>+<kafkaPartition>+<startOffset>.<format>
Actually I want to create a file name and in case it is already exists override the old one.

This query was asked earlier but not satisfied with the solution.
(Control filename in s3 sink connector)

Any help will be appreciated.

Thanks

2

Some of the parts of the object names are configurable but the connector must adhere to that pattern. Part of the reason for this constraint is to implement exactly once delivery in the connector – AIUI the partition and offsets in the name are critical in that exactly-once implementation.

3

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.