I have used the “
io.confluent.connect.transforms.ExtractTopic$Value” transforms in my elastic sink connector. but it was failed. it throws the following errors. It was working previous version of the elastic sink connector.
The error is,
The connector doesn't support topic mutating SMTs
Here my question is,
- will this feature come again in future release?
- Shall I create the custom transformation on this?.
Please help me to resolve this issues.
The reason it’s not supported is because the index name will be decided before the transform, I believe, so an index will not get automatically created for the right topic value.
The workaround is to create your ES indicies/mappings ahead of time.
Even if you used a custom transform, the topic name cannot be mutated, as the error says. I’m not sure where this logic is validated, though.
But this feature was working in the elastic sink connector’s older version(11.x).
Why this feature was revoked in the new version.
If you want to mutate the topic, use synchronous flushes