# Self-Managed Connectors **URL:** https://forum.confluent.io/c/kafka-connect/self-managed-connectors/21.md?page=2 [Latest](https://forum.confluent.io/latest.md) · [Categories](https://forum.confluent.io/categories.md) **Page:** 3 --- ## [How to save XML/JSON string as XMLType/Blob using JDBCSinkConnector](https://forum.confluent.io/t/how-to-save-xml-json-string-as-xmltype-blob-using-jdbcsinkconnector/9999)
**Author:** [@Hrishi](https://forum.confluent.io/u/Hrishi)\ **Replies:** 0\ **Last updated:** [1 February 2024 14:05 UTC](https://forum.confluent.io/t/how-to-save-xml-json-string-as-xmltype-blob-using-jdbcsinkconnector/9999 "2024-02-01T14:05:09Z")
Hello, We have a Kafka topic with AVRO schema as follows, { “type”: “record”, “name”: “CaseInfo”, “namespace”: “case.avro”, “fields”: \[ { “name”: “caseid”, “type”: “string” }, { “name”: “caseData”, “type”: … --- ## [Kubernetes cluster domain name not cluster.local](https://forum.confluent.io/t/kubernetes-cluster-domain-name-not-cluster-local/5730)
**Author:** [@anastazoch](https://forum.confluent.io/u/anastazoch)\ **Replies:** 1\ **Last updated:** [21 January 2024 04:51 UTC](https://forum.confluent.io/t/kubernetes-cluster-domain-name-not-cluster-local/5730 "2024-01-21T04:51:51Z")
We would like to try the Kafka Replicator component on a Kubernetes-based Confluent Platform. Our Kubernetes cluster’s domain name is not default one, i.e. cluster.local, but let’s say \.local. As a resu… --- ## [Custom kafka sink connector always runs with only one task](https://forum.confluent.io/t/custom-kafka-sink-connector-always-runs-with-only-one-task/9812)
**Author:** [@Chid](https://forum.confluent.io/u/Chid)\ **Replies:** 0\ **Last updated:** [12 January 2024 18:41 UTC](https://forum.confluent.io/t/custom-kafka-sink-connector-always-runs-with-only-one-task/9812 "2024-01-12T18:41:45Z")
Hi Team, Firstly I am new to Kafka Connect. I am trying to create a new sink connector. I am able to successfully deploy the connector. However, connector is always running with single task despite of number of tasks gi… --- ## [MirrorMaker 2: Failed to access Avro data from topic](https://forum.confluent.io/t/mirrormaker-2-failed-to-access-avro-data-from-topic/9446)
**Author:** [@JulianG](https://forum.confluent.io/u/JulianG)\ **Replies:** 2\ **Last updated:** [2 January 2024 16:56 UTC](https://forum.confluent.io/t/mirrormaker-2-failed-to-access-avro-data-from-topic/9446 "2024-01-02T16:56:39Z")
Hello everybody, I am currently trying to implement MirrorMaker 2 because I want to synchronize topics between to kafka clusters. I am getting the following Exception: Caused by: org.apache.kafka.common.config.ConfigE… --- ## [Custom replication policy for MirrorSourceConnector](https://forum.confluent.io/t/custom-replication-policy-for-mirrorsourceconnector/9463)
**Author:** [@astronaut42](https://forum.confluent.io/u/astronaut42)\ **Replies:** 1\ **Last updated:** [19 December 2023 14:58 UTC](https://forum.confluent.io/t/custom-replication-policy-for-mirrorsourceconnector/9463 "2023-12-19T14:58:13Z")
Hi Guys, we have an already running Kakfa Connector Cluster. I want to add a MirrorSourceConnector with a custom replication policy: curl --request POST \\ --url http://localhost:9083/connectors \\ --header 'Accept: app… --- ## [Facing issue using ENUM](https://forum.confluent.io/t/facing-issue-using-enum/9043)
**Author:** [@suyog](https://forum.confluent.io/u/suyog)\ **Replies:** 5\ **Last updated:** [13 December 2023 14:17 UTC](https://forum.confluent.io/t/facing-issue-using-enum/9043 "2023-12-13T14:17:18Z")
Hi, I have one field ‘statusCode’ in my schema with dataType as ENUM. When I am trying to publish the event I’m getting “Caused by: org.apache.avro.AvroTypeException: Not an enum: S for schema”. I tried by casting dat… --- ## [Mounting Volumes for Confluent-Connect Self Managed on a Kubernetes cluster](https://forum.confluent.io/t/mounting-volumes-for-confluent-connect-self-managed-on-a-kubernetes-cluster/9278)
**Author:** [@Arash](https://forum.confluent.io/u/Arash)\ **Replies:** 1\ **Last updated:** [5 December 2023 08:57 UTC](https://forum.confluent.io/t/mounting-volumes-for-confluent-connect-self-managed-on-a-kubernetes-cluster/9278 "2023-12-05T08:57:01Z")
Hi, I want to know where I can provide the volumes in the template of yml file for Confluent-Connect Self Managed on a Kubernetes cluster in this link Confluent for Kubernetes API Reference , I found this link Advanced… --- ## [Http sink connector slow consumption](https://forum.confluent.io/t/http-sink-connector-slow-consumption/9217)
**Author:** [@tom3](https://forum.confluent.io/u/tom3)\ **Replies:** 0\ **Last updated:** [30 November 2023 23:11 UTC](https://forum.confluent.io/t/http-sink-connector-slow-consumption/9217 "2023-11-30T23:11:53Z")
I have a http sink connector running in kubernetes. Its making requests to a rest service. The amount of messages consumed and sent to the topic seems low. Is there any metric which the connector publishes which gives th… --- ## [Tabular Kafka Iceberg Sink Connector](https://forum.confluent.io/t/tabular-kafka-iceberg-sink-connector/9057)
**Author:** [@rajcoolguy](https://forum.confluent.io/u/rajcoolguy)\ **Replies:** 1\ **Last updated:** [15 November 2023 15:40 UTC](https://forum.confluent.io/t/tabular-kafka-iceberg-sink-connector/9057 "2023-11-15T15:40:58Z")
Kafka Iceberg Sink Connector is picking default aws credentials and is not picking up the access key and secret key passed in the sink config. Can some one let me know what are the config parameters to pass aws access ke… --- ## [MicrosoftSqlServerSink and SqlServerCdcSource connector](https://forum.confluent.io/t/microsoftsqlserversink-and-sqlservercdcsource-connector/9065)
**Author:** [@aparna](https://forum.confluent.io/u/aparna)\ **Replies:** 0\ **Last updated:** [13 November 2023 07:25 UTC](https://forum.confluent.io/t/microsoftsqlserversink-and-sqlservercdcsource-connector/9065 "2023-11-13T07:25:54Z")
I am having a sink and source connector in my local env and I am attaching the configuration below: Sink: { "connector.class": "io.confluent.connect.jdbc.JdbcSinkConnector", "tasks.max": "1", "topics": "etl-sink-… --- ## [HTTP Sink Connector and DLQ](https://forum.confluent.io/t/http-sink-connector-and-dlq/9015)
**Author:** [@lavalex](https://forum.confluent.io/u/lavalex)\ **Replies:** 0\ **Last updated:** [7 November 2023 11:08 UTC](https://forum.confluent.io/t/http-sink-connector-and-dlq/9015 "2023-11-07T11:08:43Z")
Hello, I have a working HTTP Sink connector 1.7.4 (latest), with these settings: errors.tolerance: “all” errors.deadletterqueue.topic.name: “dlq-httpsink” The topic is created when connector starts, but if I get an e… --- ## [Kafka connect server never returns 200](https://forum.confluent.io/t/kafka-connect-server-never-returns-200/8969)
**Author:** [@lizard](https://forum.confluent.io/u/lizard)\ **Replies:** 2\ **Last updated:** [7 November 2023 05:01 UTC](https://forum.confluent.io/t/kafka-connect-server-never-returns-200/8969 "2023-11-07T05:01:24Z")
I have a self managed distributed connect cluster running locally on docker. I am following the kafka-connect-101 training On container start up I: Install the splunk sink connector from confluent-hub Start kafka con… --- ## [How to consume Topic payload fields into one single column](https://forum.confluent.io/t/how-to-consume-topic-payload-fields-into-one-single-column/8986)
**Author:** [@suyog](https://forum.confluent.io/u/suyog)\ **Replies:** 1\ **Last updated:** [6 November 2023 20:47 UTC](https://forum.confluent.io/t/how-to-consume-topic-payload-fields-into-one-single-column/8986 "2023-11-06T20:47:08Z")
Hi, I have Event-topic with payload of JSON format consisting of 4 fields in it. When I am consuming those events using kafka-jdbc-sink-connectors, it is creating fours different columns in my postgres table. My require… --- ## [Data loss in kafka broker](https://forum.confluent.io/t/data-loss-in-kafka-broker/8960)
**Author:** [@Uche](https://forum.confluent.io/u/Uche)\ **Replies:** 0\ **Last updated:** [30 October 2023 15:24 UTC](https://forum.confluent.io/t/data-loss-in-kafka-broker/8960 "2023-10-30T15:24:32Z")
Please what configuration should I use to ensure that I don’t lose data when replicating? I have a source table that has 7 M rows but only about 6 M plus gets to the destination table. I think the consumer just idles w… --- ## [Mm2 on kafka connect stops replicating topic after DisconnectException](https://forum.confluent.io/t/mm2-on-kafka-connect-stops-replicating-topic-after-disconnectexception/8916)
**Author:** [@papirosko](https://forum.confluent.io/u/papirosko)\ **Replies:** 0\ **Last updated:** [21 October 2023 07:07 UTC](https://forum.confluent.io/t/mm2-on-kafka-connect-stops-replicating-topic-after-disconnectexception/8916 "2023-10-21T07:07:04Z")
I have 3 kafka-connect nodes, running mm2 tasks. Those task replicate 4 topics from remote uk and us clusters into single uk kafka cluster. Everything works fine most of the time, but at some point I noticed that the l… --- ## [JdbcSinkConnector from mysql to postgres invalid byte sequence for encoding "UTF8": 0x00](https://forum.confluent.io/t/jdbcsinkconnector-from-mysql-to-postgres-invalid-byte-sequence-for-encoding-utf8-0x00/8838)
**Author:** [@cssalvador](https://forum.confluent.io/u/cssalvador)\ **Replies:** 8\ **Last updated:** [19 October 2023 14:23 UTC](https://forum.confluent.io/t/jdbcsinkconnector-from-mysql-to-postgres-invalid-byte-sequence-for-encoding-utf8-0x00/8838 "2023-10-19T14:23:32Z")
Hi everyone, just trying to move data from mysql to postgres but this error message shows up in the sink : " org.postgresql.util.PSQLException: ERROR: invalid byte sequence for encoding “UTF8”: 0x00". I want to fix thi… --- ## [Tasks and Partitions Rebalancing Mechanism](https://forum.confluent.io/t/tasks-and-partitions-rebalancing-mechanism/8908)
**Author:** [@michael-ab](https://forum.confluent.io/u/michael-ab)\ **Replies:** 0\ **Last updated:** [19 October 2023 10:34 UTC](https://forum.confluent.io/t/tasks-and-partitions-rebalancing-mechanism/8908 "2023-10-19T10:34:06Z")
Hi, We are in the process of building a custom sink connector using Kafka Connect with the exactly-once delivery semantics feature. Some aspect of the tasks and partitions rebalancing can break the exactly-once delivery… --- ## [Getting error on debezium connector connection with sql server](https://forum.confluent.io/t/getting-error-on-debezium-connector-connection-with-sql-server/8901)
**Author:** [@kunalsingh18495](https://forum.confluent.io/u/kunalsingh18495)\ **Replies:** 2\ **Last updated:** [18 October 2023 10:32 UTC](https://forum.confluent.io/t/getting-error-on-debezium-connector-connection-with-sql-server/8901 "2023-10-18T10:32:16Z")
ERROR \[sql-server-connection|task-0\] WorkerSourceTask{id=sql-server-connection-0} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted (org.apache.kafka.c… --- ## [Debezium on SQL Server - Missing Events](https://forum.confluent.io/t/debezium-on-sql-server-missing-events/8896)
**Author:** [@ssugar](https://forum.confluent.io/u/ssugar)\ **Replies:** 0\ **Last updated:** [17 October 2023 14:42 UTC](https://forum.confluent.io/t/debezium-on-sql-server-missing-events/8896 "2023-10-17T14:42:54Z")
Ran into an issue where SQL Server CDC change tables showed some change records that did not make it to the topic in Confluent. Does anyone have any troubleshooting/investigation tips to dig into why this might have hap… --- ## [Confluent filter condition not just one field but all fields in the topic](https://forum.confluent.io/t/confluent-filter-condition-not-just-one-field-but-all-fields-in-the-topic/8893)
**Author:** [@cssalvador](https://forum.confluent.io/u/cssalvador)\ **Replies:** 0\ **Last updated:** [17 October 2023 13:03 UTC](https://forum.confluent.io/t/confluent-filter-condition-not-just-one-field-but-all-fields-in-the-topic/8893 "2023-10-17T13:03:31Z")
Hello, we have a problem with connect-transforms (connect-transformations | Confluent Hub) I would like to filter out records containing a substring. This substring can be at any field in the topic. How can i use filt… --- ## [Debezium: Lock request time out period exceeded](https://forum.confluent.io/t/debezium-lock-request-time-out-period-exceeded/4381)
**Author:** [@nickbob](https://forum.confluent.io/u/nickbob)\ **Replies:** 6\ **Last updated:** [14 September 2023 05:56 UTC](https://forum.confluent.io/t/debezium-lock-request-time-out-period-exceeded/4381 "2023-09-14T05:56:13Z")
INFO \[sql-server-connection|task-0\] Snapshot - Final stage (io.debezium.pipeline.source.AbstractSnapshotChangeEventSource:88) \[2022-03-08 07:15:47,766\] INFO \[sql-server-connection|task-0\] Removing locking timeout (io.de… --- ## [Help needed with Connecting Confluent Kafka to Salesforce](https://forum.confluent.io/t/help-needed-with-connecting-confluent-kafka-to-salesforce/8500)
**Author:** [@walunjvijay](https://forum.confluent.io/u/walunjvijay)\ **Replies:** 0\ **Last updated:** [14 August 2023 19:00 UTC](https://forum.confluent.io/t/help-needed-with-connecting-confluent-kafka-to-salesforce/8500 "2023-08-14T19:00:10Z")
I experimented with connecting Confluent Kafka to Salesforce Data Cloud using the Confluent-native Sink connector. I set up a connected app and generated the necessary Java keystore files to enable the connector, which i… --- ## [Should/Will Kafka Connect support schema evolution using Avro 1.10.X enum defaults?](https://forum.confluent.io/t/should-will-kafka-connect-support-schema-evolution-using-avro-1-10-x-enum-defaults/3076)
**Author:** [@GuusDeGraeve](https://forum.confluent.io/u/GuusDeGraeve)\ **Replies:** 16\ **Last updated:** [11 August 2023 06:56 UTC](https://forum.confluent.io/t/should-will-kafka-connect-support-schema-evolution-using-avro-1-10-x-enum-defaults/3076 "2023-08-11T06:56:57Z")
We are using Kafka Connect to pipe data from Kafka topics into parquet files on HDFS and/or S3. Our Kafka data is serialised using Avro (schema registry). At the moment we use 2 connectors for this HDFS2 Sink Connecto… --- ## [Could not get Replace Field working](https://forum.confluent.io/t/could-not-get-replace-field-working/8438)
**Author:** [@dramamoorthy](https://forum.confluent.io/u/dramamoorthy)\ **Replies:** 0\ **Last updated:** [2 August 2023 04:52 UTC](https://forum.confluent.io/t/could-not-get-replace-field-working/8438 "2023-08-02T04:52:57Z")
We are using CDC pipeline to stream data from oracle DB to S3 in Avro format. We are able to stream to the data from oracle to kafka with debezium connector and we are able to flatten the data and store it as avro file … --- ## [S3 sync connector error with schema registry](https://forum.confluent.io/t/s3-sync-connector-error-with-schema-registry/8359)
**Author:** [@onlymohan](https://forum.confluent.io/u/onlymohan)\ **Replies:** 5\ **Last updated:** [26 July 2023 16:26 UTC](https://forum.confluent.io/t/s3-sync-connector-error-with-schema-registry/8359 "2023-07-26T16:26:08Z")
Getting below error, when I tried to start S3 sycn connector with schema registry. It works fine when schema registry is not invoked. connect | java.lang.VerifyError: Bad return type connect | Exception Details: co… --- ## [Secrets externalization with Debezium connectors](https://forum.confluent.io/t/secrets-externalization-with-debezium-connectors/8063)
**Author:** [@jaydee](https://forum.confluent.io/u/jaydee)\ **Replies:** 1\ **Last updated:** [23 June 2023 03:36 UTC](https://forum.confluent.io/t/secrets-externalization-with-debezium-connectors/8063 "2023-06-23T03:36:04Z")
Hi, I am getting an error when trying to use secret external file. Error message is below org.apache.kafka.common.config.ConfigException: Could not read properties from file /data/connector\_credentials.properties My wo… --- ## [S3 Sink Connector header schema](https://forum.confluent.io/t/s3-sink-connector-header-schema/8092)
**Author:** [@MathiasSonderfeld](https://forum.confluent.io/u/MathiasSonderfeld)\ **Replies:** 0\ **Last updated:** [14 June 2023 06:46 UTC](https://forum.confluent.io/t/s3-sink-connector-header-schema/8092 "2023-06-14T06:46:22Z")
Hello, Im running an S3 Sink Connector and have a custom Partitioner, that maps records based on some kafka message header values. The headers are parsed into different datatypes and in my specific case, a string is mis… --- ## [Amazon S3 Sink Connector](https://forum.confluent.io/t/amazon-s3-sink-connector/8075)
**Author:** [@agathe-benichou](https://forum.confluent.io/u/agathe-benichou)\ **Replies:** 0\ **Last updated:** [11 June 2023 14:01 UTC](https://forum.confluent.io/t/amazon-s3-sink-connector/8075 "2023-06-11T14:01:46Z")
I want data in my Kafka topic to be delivered to the S3 Bucket every 15 minutes, no matter how much data was generated I have to specify a flush.size to to some super-high number that’s far more than the number of messa… --- ## [Installing Custom Connector](https://forum.confluent.io/t/installing-custom-connector/7992)
**Author:** [@djchapm](https://forum.confluent.io/u/djchapm)\ **Replies:** 9\ **Last updated:** [6 June 2023 20:24 UTC](https://forum.confluent.io/t/installing-custom-connector/7992 "2023-06-06T20:24:45Z")
Hi, I have a java maven project where I’m using an assembly plugin to create a directory containing config, jar file, and all dependencies. I’m mounting this to a directory like /usr/local/share/myconnector and running… --- ## [Is cp-kafka-connect-base image still using confluent forked log4j v1.x?](https://forum.confluent.io/t/is-cp-kafka-connect-base-image-still-using-confluent-forked-log4j-v1-x/7974)
**Author:** [@jerryz](https://forum.confluent.io/u/jerryz)\ **Replies:** 1\ **Last updated:** [4 June 2023 12:46 UTC](https://forum.confluent.io/t/is-cp-kafka-connect-base-image-still-using-confluent-forked-log4j-v1-x/7974 "2023-06-04T12:46:13Z")
Hi there, We are using cp-kafka-connect-base v 5.3. 1 community image for our self managed connectors. From December 2021 Log4j Vulnerabilities Advisory, it says Confluent’s community package does not include or rely… [Previous page](https://forum.confluent.io/c/kafka-connect/self-managed-connectors/21.md?page=1) [Next page](https://forum.confluent.io/c/kafka-connect/self-managed-connectors/21.md?page=3)