Kafka mongo sink. Create _id from key message

Kafka Connect
1

Hello,

I have the following kafka message: key content is {orderid: 123} and the message value as {name: "def", company: "abc"}. I want to insert in mongodb the following document: {_id: 123, name: "def", company: "abc"} where _id takes the value of orderid from the key message. I have tried the following config and it looks ok except that the _id is being an object like this: _id:{orderid: 123}. Any idead how we can achieve this? I have the following config:

name = MongoSinkConnectorConnector_1
connector.class = com.mongodb.kafka.connect.MongoSinkConnector
topics = topicA

key.converter = org.apache.kafka.connect.storage.StringConverter
key.converter.schemas.enable = false

value.converter = org.apache.kafka.connect.json.JsonConverter
value.converter.schemas.enable = false

connection.uri = mongodb://localhost:27017
database = foo
collection = mycollection

# Use a single message as a document
document.id.strategy=com.mongodb.kafka.connect.sink.processor.id.strategy.PartialKeyStrategy

# Post processing of the sink documents
post.processor.chain=com.mongodb.kafka.connect.sink.processor.DocumentIdAdder

# Document ID strategy using the Partial Key Strategy
document.id.strategy.overwrite.existing=true

# Provide a custom setting for which key values to consider for the _id
document.id.strategy.partial.key.projection.list=orderid,customerid
document.id.strategy.partial.key.projection.type=AllowList

Thank you!

2

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.