ExtractTopic transformation with S3 Sink Connector

Kafka Connect
1

Hi,
I’m using S3 Sink Connector to read Avro messages from topic. Message itself contains lots of information which I don’t need. How can I extract specific fields from message and write to S3?

{
   "id": 123456,
   "testId": {
      "type": {
         "name": "testType"
      },
      "value": "testValue"
   },
   "hasMetaData": {
      "createdBy": "xxxxx",
      "createdOn": "2022-02-16T16:40:27.820238Z",
      "derivedFrom": "xxxxxx",
      "providerStatus": "NEW"
   }
..... continue
}

I need only “id” and “testId”.

thanks,
Umit

2

Hey @shortcode, welcome back!

You may try to refer to this other thread, where I already answered a similar question.

You are probably looking for whitelist functionality of the ReplaceField SMT.

2 Likes
3

Sorry for duplicate question. Thanks for redirecting me. I didn’t know that I can use Replacefield to drop. Thanks

4

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.