How to using Kafka connector for read and write data with XML format?

Kafka Connect
1

Hello,
I would like to know the solution for processing XML data with Confluent.
My source and sink system be AWS S3 and MQ.

  1. How to read XML data with Kafka connector for Source connector?
    After storing data on Kafka topic and filtering or calculating some value on ksqlDB or KStream before sending to destination.

  2. After step 1), How to send processed data to the destination as XML data may need to restructure XML format file in different from source system.

Thanks
May.R

2

hey @rununrath

did the XML part around 1 year ago.
Followed mostly @rmoff’s blog post:

demo-scence is a good point to start with

best,
michael

3

Hello michael,

I gets some ideas about inbound to read XML and stored as JSON on Kafka.
But I still wonder in part of outbound, in case I expected send JSON data on Kafka topic and write to S3 or MQ as XML format so How to do that?

Thanks
May.R

4

Hi!

You should be able to do that using a Single Message Transformation (SMT). You can implement your own, or you can try this one:

You would need to set post.processing.to.xml to true in order to have the XML output format.

5

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.