# Opensearch Sink Connector \_id issue **URL:** \ **Category:** Managed Connectors\ **Created:** [23 January 2025 12:17 UTC](https://forum.confluent.io/t/opensearch-sink-connector-id-issue/36460 "2025-01-23T12:17:15Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![bawwcusr](https://avatars.discourse-cdn.com/v4/letter/b/cc9497/32.png) [@bawwcusr](https://forum.confluent.io/u/bawwcusr)\ **Post date:** [23 January 2025 12:17 UTC](https://forum.confluent.io/t/opensearch-sink-connector-id-issue/36460/1 "2025-01-23T12:17:15Z")
I have recently started working with the Managed Opensearch sink connector. The sink is consuming a topic which is the output of a ksql table. Every message being put in the topic has a defined key. While opensearch sink connector is consuming the data, it generates new document inside the index even for same keys with random \_ids. How can I : 1. Set the \_id of the documents as the topic key. 2. Even if I can’t create the \_id myself, how do I ensure that same key always creates a same \_id ---
**Author:** ![dtroiano](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/dtroiano/32/1961_2.png) [@dtroiano](https://forum.confluent.io/u/dtroiano)\ **Post date:** [23 January 2025 14:43 UTC](https://forum.confluent.io/t/opensearch-sink-connector-id-issue/36460/2 "2025-01-23T14:43:54Z")
What happens if you explicitly set an `_id` field in the value? Either in ksqlDB (see [`AS_VALUE`](https://docs.confluent.io/platform/current/ksqldb/developer-guide/ksqldb-reference/scalar-functions.html#as-value)) or, if you have a document ID already in the value, you can add one of the `ReplaceField$Value` SMTs ([Apache Kafka](https://docs.confluent.io/cloud/current/connectors/transforms/replacefield-ak.html), [Confluent](https://docs.confluent.io/cloud/current/connectors/transforms/replacefield-confluent.html)). ---
**Author:** ![bawwcusr](https://avatars.discourse-cdn.com/v4/letter/b/cc9497/32.png) [@bawwcusr](https://forum.confluent.io/u/bawwcusr)\ **Post date:** [23 January 2025 15:37 UTC](https://forum.confluent.io/t/opensearch-sink-connector-id-issue/36460/3 "2025-01-23T15:37:31Z")
I tried SMTs as well and I did set the \_id explicitly but that didn’t help as well. It still went ahead and created a random \_id in opensearch document. attaching image for reference : ![image](https://us1.discourse-cdn.com/flex019/uploads/confluentcommunity/original/2X/6/62ff705e8030e1bb28efc350210e1f08c85c49cd.png) ---
**Author:** ![dtroiano](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/dtroiano/32/1961_2.png) [@dtroiano](https://forum.confluent.io/u/dtroiano)\ **Post date:** [23 January 2025 15:52 UTC](https://forum.confluent.io/t/opensearch-sink-connector-id-issue/36460/4 "2025-01-23T15:52:12Z")
What `index{#}.batch.size` is in the connector config? It looks like the default of 1 might lead to the issue you are seeing, but anything greater ought to work as long as `_id` is in the value. ---
**Author:** ![bawwcusr](https://avatars.discourse-cdn.com/v4/letter/b/cc9497/32.png) [@bawwcusr](https://forum.confluent.io/u/bawwcusr)\ **Post date:** [23 January 2025 16:11 UTC](https://forum.confluent.io/t/opensearch-sink-connector-id-issue/36460/5 "2025-01-23T16:11:31Z")
It was indeed 1, but I tried bumping it to 10, still no luck ![image](https://us1.discourse-cdn.com/flex019/uploads/confluentcommunity/original/2X/a/ab8da25a2e24745ab3ba173f7e6cd25eee8172ac.png) ---
**Author:** ![bawwcusr](https://avatars.discourse-cdn.com/v4/letter/b/cc9497/32.png) [@bawwcusr](https://forum.confluent.io/u/bawwcusr)\ **Post date:** [23 January 2025 16:18 UTC](https://forum.confluent.io/t/opensearch-sink-connector-id-issue/36460/6 "2025-01-23T16:18:31Z")
Also adding \_id in message value leads to error : ```auto "{\"took\":7,\"errors\":true,\"items\":[{\"index\":{\"_index\":\"order_index_test\",\"_id\":\"t4bzk5QBiA06QGNGvcah\",\"status\":400,\"error\":{\"type\":\"mapper_parsing_exception\",\"reason\":\"failed to parse field [_id] of type [_id] in document with id 't4bzk5QBiA06QGNGvcah'. Preview of field's value: '123'\",\"caused_by\":{\"type\":\"mapper_parsing_exception\",\"reason\":\"Field [_id] is a metadata field and cannot be added inside a document. Use the index API request parameters.\"}}}}]}" ``` ---
**Author:** ![dtroiano](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/dtroiano/32/1961_2.png) [@dtroiano](https://forum.confluent.io/u/dtroiano)\ **Post date:** [23 January 2025 17:07 UTC](https://forum.confluent.io/t/opensearch-sink-connector-id-issue/36460/7 "2025-01-23T17:07:30Z")
Sorry – I took a closer look at the connector source and the OpenSearch REST API and don’t believe that this is currently supported. Confirming that. ---
**Author:** ![bawwcusr](https://avatars.discourse-cdn.com/v4/letter/b/cc9497/32.png) [@bawwcusr](https://forum.confluent.io/u/bawwcusr)\ **Post date:** [23 January 2025 17:18 UTC](https://forum.confluent.io/t/opensearch-sink-connector-id-issue/36460/8 "2025-01-23T17:18:34Z")
ahhh, too bad. I think custom connector would the way to go here then(unless you think otherwise). Thank you for the responses though. ---
**Author:** ![dtroiano](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/dtroiano/32/1961_2.png) [@dtroiano](https://forum.confluent.io/u/dtroiano)\ **Post date:** [23 January 2025 19:23 UTC](https://forum.confluent.io/t/opensearch-sink-connector-id-issue/36460/9 "2025-01-23T19:23:46Z")
That, or the other thing you can try is the HTTP Sink connector to use the [`PUT /_doc/<_id>`](https://opensearch.org/docs/latest/api-reference/document-apis/index-document/) API. Both the [v1](https://docs.confluent.io/cloud/current/connectors/cc-http-sink.html) and [v2](https://docs.confluent.io/cloud/current/connectors/cc-http-sink-v2.html) connectors support template params so you should be able to tack `${key}` onto the URL to serve as the document ID and upsert. I haven’t tested this out but it’s worth a shot.