# Self-Managed Connectors **URL:** https://forum.confluent.io/c/kafka-connect/self-managed-connectors/21.md?page=3 [Latest](https://forum.confluent.io/latest.md) · [Categories](https://forum.confluent.io/categories.md) **Page:** 4 --- ## [Not able to start Kafka Connect](https://forum.confluent.io/t/not-able-to-start-kafka-connect/4385)
**Author:** [@AneeshBabu](https://forum.confluent.io/u/AneeshBabu)\ **Replies:** 2\ **Last updated:** [1 June 2023 21:40 UTC](https://forum.confluent.io/t/not-able-to-start-kafka-connect/4385 "2023-06-01T21:40:21Z")
Hi there, My use case is : Change Data capture using Debezium When iam trying to start the Kafka Connect ,facing the error as below. Note : Kafka is up & running. \[2022-03-08 18:43:46,573\] ERROR Stopping due to e… --- ## [NonProxyHosts with GCP PubSub - Kafka connector](https://forum.confluent.io/t/nonproxyhosts-with-gcp-pubsub-kafka-connector/7922)
**Author:** [@Nick](https://forum.confluent.io/u/Nick)\ **Replies:** 0\ **Last updated:** [22 May 2023 21:53 UTC](https://forum.confluent.io/t/nonproxyhosts-with-gcp-pubsub-kafka-connector/7922 "2023-05-22T21:53:28Z")
Hi, We have been running into an issue where we need to run the PubSubSourceConnector behind a company Proxy to pull messages from our GCP PubSub topics into our on premise Kafka. We have tried setting the JVM parameter… --- ## [Kafka Consumer not consuming data immediately they're producded](https://forum.confluent.io/t/kafka-consumer-not-consuming-data-immediately-theyre-producded/7695)
**Author:** [@jaydee](https://forum.confluent.io/u/jaydee)\ **Replies:** 3\ **Last updated:** [12 May 2023 16:40 UTC](https://forum.confluent.io/t/kafka-consumer-not-consuming-data-immediately-theyre-producded/7695 "2023-05-12T16:40:50Z")
My consumers are not pushing data to destination DB immediately it is produced. I have set the sink connector to “fetch.max.wait.ms=10”. Please what configuration setting do I need to have it consume the data the minu… --- ## [Kafka connect s3 Source connector stops processing](https://forum.confluent.io/t/kafka-connect-s3-source-connector-stops-processing/7764)
**Author:** [@rajeshkalyanasundara](https://forum.confluent.io/u/rajeshkalyanasundara)\ **Replies:** 0\ **Last updated:** [4 May 2023 09:27 UTC](https://forum.confluent.io/t/kafka-connect-s3-source-connector-stops-processing/7764 "2023-05-04T09:27:02Z")
We have an issue similar to Processing large CSV files using SFTP Source Kafka Connector We are using confluent S3 source connector (cp-kafka-connect docker image version 7.3.3 latest) to restore data from S3 to topics … --- ## [Debezium Oracle to PostgreSQL Date Issue](https://forum.confluent.io/t/debezium-oracle-to-postgresql-date-issue/7689)
**Author:** [@prensgold](https://forum.confluent.io/u/prensgold)\ **Replies:** 0\ **Last updated:** [24 April 2023 13:48 UTC](https://forum.confluent.io/t/debezium-oracle-to-postgresql-date-issue/7689 "2023-04-24T13:48:55Z")
Hello , I am performing cdc copying from Oracle to Postgresql. In Oracle, the date column in all tables goes to postgresql as unix timestamp. I added the following parameters to the json file. “transforms.start\_date.… --- ## [MS SQL connector authentication?](https://forum.confluent.io/t/ms-sql-connector-authentication/7647)
**Author:** [@jbla9028](https://forum.confluent.io/u/jbla9028)\ **Replies:** 0\ **Last updated:** [19 April 2023 00:23 UTC](https://forum.confluent.io/t/ms-sql-connector-authentication/7647 "2023-04-19T00:23:42Z")
Hello, sorry if this is not the right section. We are working to build our Confluent architecture out. One of our first use cases is a on premises MS SQL sink with CDC Microsoft SQL Server Change Data Capture (CDC) Sour… --- ## [Confluent oracle cdc connector doesn't work](https://forum.confluent.io/t/confluent-oracle-cdc-connector-doesnt-work/7642)
**Author:** [@Rami15](https://forum.confluent.io/u/Rami15)\ **Replies:** 0\ **Last updated:** [18 April 2023 08:40 UTC](https://forum.confluent.io/t/confluent-oracle-cdc-connector-doesnt-work/7642 "2023-04-18T08:40:07Z")
I installed Confluent’s oracle cdc connector, configured database (Oracle 19c no-cdb) to activate logminer. the connector got the snapshotting very well. However when it is done from snapshotting, after certain time, a… --- ## [Need help using Regular expressions in table include list](https://forum.confluent.io/t/need-help-using-regular-expressions-in-table-include-list/7606)
**Author:** [@vangariwwt1](https://forum.confluent.io/u/vangariwwt1)\ **Replies:** 2\ **Last updated:** [12 April 2023 12:21 UTC](https://forum.confluent.io/t/need-help-using-regular-expressions-in-table-include-list/7606 "2023-04-12T12:21:26Z")
We are trying to create topics for the database tables in Kafka using table include list and have around 100 tables to be included for the topic creation. Instead of keeping each table name in the connector file, tried… --- ## [Sink from tables with no primary key](https://forum.confluent.io/t/sink-from-tables-with-no-primary-key/7469)
**Author:** [@jaydee](https://forum.confluent.io/u/jaydee)\ **Replies:** 2\ **Last updated:** [29 March 2023 20:14 UTC](https://forum.confluent.io/t/sink-from-tables-with-no-primary-key/7469 "2023-03-29T20:14:36Z")
I am sinking data from a table with no primary key. I am getting this error below. Any ideas to resolve it. Caused by: org.apache.kafka.connect.errors.ConnectException: Sink connector ‘jdbc-debe-azure-sink-itemdata’ is… --- ## [Store changes from changelog topic using Kafka Connect](https://forum.confluent.io/t/store-changes-from-changelog-topic-using-kafka-connect/7234)
**Author:** [@SDev](https://forum.confluent.io/u/SDev)\ **Replies:** 2\ **Last updated:** [22 February 2023 18:52 UTC](https://forum.confluent.io/t/store-changes-from-changelog-topic-using-kafka-connect/7234 "2023-02-22T18:52:17Z")
Can I persist data from changelog topic using Kafka Connect to a database? --- ## [Issue in using Timetsamp Column for a Custom transform](https://forum.confluent.io/t/issue-in-using-timetsamp-column-for-a-custom-transform/7111)
**Author:** [@PitsBee](https://forum.confluent.io/u/PitsBee)\ **Replies:** 0\ **Last updated:** [29 January 2023 22:24 UTC](https://forum.confluent.io/t/issue-in-using-timetsamp-column-for-a-custom-transform/7111 "2023-01-29T22:24:14Z")
Hey guys, Please help me to solve this. I am using Debezium source connector to pull data from SQL databaase. Apart from other fields in the sql table , there is “CreatedDate” ( datetime2(7) ) Before writing to topic… --- ## [Debezium connector mysql task failed](https://forum.confluent.io/t/debezium-connector-mysql-task-failed/7080)
**Author:** [@noname](https://forum.confluent.io/u/noname)\ **Replies:** 13\ **Last updated:** [25 January 2023 07:28 UTC](https://forum.confluent.io/t/debezium-connector-mysql-task-failed/7080 "2023-01-25T07:28:00Z")
Started the docker containers (cp-kafka-connect-base:7.0.1) and installed the self-managed connector (debezium-connector-mysql:latest). Docker containers works good. When I am trying to configure the connector configur… --- ## [Error when trying to create a connector with curl request](https://forum.confluent.io/t/error-when-trying-to-create-a-connector-with-curl-request/7038)
**Author:** [@SeverusP](https://forum.confluent.io/u/SeverusP)\ **Replies:** 23\ **Last updated:** [22 January 2023 00:21 UTC](https://forum.confluent.io/t/error-when-trying-to-create-a-connector-with-curl-request/7038 "2023-01-22T00:21:45Z")
Hello everyone, I’m following the tutorial by Robin Moffat Loading CSV data into Kafka running docker in a windows machine. When i launch this curl request: curl -i -X PUT http://localhost:8083/connectors/source-csv-s… --- ## [SMT debezium to replace a string with another in topic name](https://forum.confluent.io/t/smt-debezium-to-replace-a-string-with-another-in-topic-name/6959)
**Author:** [@PitsBee](https://forum.confluent.io/u/PitsBee)\ **Replies:** 2\ **Last updated:** [17 January 2023 06:03 UTC](https://forum.confluent.io/t/smt-debezium-to-replace-a-string-with-another-in-topic-name/6959 "2023-01-17T06:03:24Z")
hi, I have a Debezium connector that pulls data from SQL server tables. It seems the connector creates the topic name like Server.Database.Table. I want Server.Database.invitation.Sent.V1 - to be transformed to… --- ## [Topics When deploy Kafka Connect](https://forum.confluent.io/t/topics-when-deploy-kafka-connect/6898)
**Author:** [@SeverusP](https://forum.confluent.io/u/SeverusP)\ **Replies:** 1\ **Last updated:** [27 December 2022 10:15 UTC](https://forum.confluent.io/t/topics-when-deploy-kafka-connect/6898 "2022-12-27T10:15:52Z")
If I understand correctly, when you run worker.properties for the first time a Kafka Connect Cluster is instantiated and shares three topics with a Broker. These three are named in the properties: offset.storage.topic … --- ## [Does changing flush.size of the Confluent s3 sink invalidate EOS?](https://forum.confluent.io/t/does-changing-flush-size-of-the-confluent-s3-sink-invalidate-eos/6890)
**Author:** [@filpano](https://forum.confluent.io/u/filpano)\ **Replies:** 2\ **Last updated:** [22 December 2022 20:23 UTC](https://forum.confluent.io/t/does-changing-flush-size-of-the-confluent-s3-sink-invalidate-eos/6890 "2022-12-22T20:23:10Z")
I have a Kafka Connect cluster running the Confluent s3 Sink Connector, configured for EOS. Our current flush.size is set to a rather high number which was ideal for reading in historical data. After having caught up to… --- ## [S3 Source Connector for .CSV files](https://forum.confluent.io/t/s3-source-connector-for-csv-files/6893)
**Author:** [@SeverusP](https://forum.confluent.io/u/SeverusP)\ **Replies:** 1\ **Last updated:** [22 December 2022 20:00 UTC](https://forum.confluent.io/t/s3-source-connector-for-csv-files/6893 "2022-12-22T20:00:36Z")
Hello everyone! I’m wondering if the S3 Source Connector is able to insert .csv files in a topic. Another question is if with a consumer it is possible to read this topic and extrapolate the data of each file. Thank y… --- ## [ValueToKey SMT is not working for initial load but working for real time event transactions](https://forum.confluent.io/t/valuetokey-smt-is-not-working-for-initial-load-but-working-for-real-time-event-transactions/6879)
**Author:** [@shubhamshirur](https://forum.confluent.io/u/shubhamshirur)\ **Replies:** 1\ **Last updated:** [20 December 2022 07:08 UTC](https://forum.confluent.io/t/valuetokey-smt-is-not-working-for-initial-load-but-working-for-real-time-event-transactions/6879 "2022-12-20T07:08:58Z")
Hi, I am trying to use ValueToKey SMT for chaning Key of the record, for data coming from MySQL. Connector being used is debezium CDC source connector for MySQL. When I am running it with SMT it throws error as field d… --- ## [How to increase performanve of JDBC sink connector to sql server in upsert mode](https://forum.confluent.io/t/how-to-increase-performanve-of-jdbc-sink-connector-to-sql-server-in-upsert-mode/6871)
**Author:** [@Kiran](https://forum.confluent.io/u/Kiran)\ **Replies:** 0\ **Last updated:** [19 December 2022 14:32 UTC](https://forum.confluent.io/t/how-to-increase-performanve-of-jdbc-sink-connector-to-sql-server-in-upsert-mode/6871 "2022-12-19T14:32:27Z")
Hi, Im trying to run the JDBC Sink connector for sql server with upsert mode on. I tried various combination of max.poll.records. I still get the error - \[Consumer clientId=connector-consumer-sink-azure-jdbc-sepa-gze-s… --- ## [RabbitMQ Source Connector](https://forum.confluent.io/t/rabbitmq-source-connector/6163)
**Author:** [@nishyanthp](https://forum.confluent.io/u/nishyanthp)\ **Replies:** 1\ **Last updated:** [25 November 2022 12:58 UTC](https://forum.confluent.io/t/rabbitmq-source-connector/6163 "2022-11-25T12:58:32Z")
Hi, I have installed Confluent Platform in my local using Docker images and I am trying to install RabbitMQ Connector plugin using Confluent-Hub, but i am receiving below error(error took from Log) and plugin is no… --- ## [JDBC Connector - differentiating between create and updates](https://forum.confluent.io/t/jdbc-connector-differentiating-between-create-and-updates/6512)
**Author:** [@obelix74](https://forum.confluent.io/u/obelix74)\ **Replies:** 5\ **Last updated:** [24 November 2022 15:15 UTC](https://forum.confluent.io/t/jdbc-connector-differentiating-between-create-and-updates/6512 "2022-11-24T15:15:15Z")
I am using the JDBC connector. From the payload of updates from the Kafka topic used in “timestamp+incrementing” mode, how can I differentiate between creates and updates in the database? --- ## [Regarding connection reset](https://forum.confluent.io/t/regarding-connection-reset/6550)
**Author:** [@Sarath](https://forum.confluent.io/u/Sarath)\ **Replies:** 0\ **Last updated:** [16 November 2022 19:42 UTC](https://forum.confluent.io/t/regarding-connection-reset/6550 "2022-11-16T19:42:52Z")
Here I’m trying to convert my json topic into avro by creating a stream but I’m seeing this error. Can some one please help me on this what change do I need to make This is what it showing up in the KSQL (Could not regi… --- ## [How to save Kafka Headers values in MongoDB Sink?](https://forum.confluent.io/t/how-to-save-kafka-headers-values-in-mongodb-sink/6460)
**Author:** [@zikria35](https://forum.confluent.io/u/zikria35)\ **Replies:** 6\ **Last updated:** [9 November 2022 15:04 UTC](https://forum.confluent.io/t/how-to-save-kafka-headers-values-in-mongodb-sink/6460 "2022-11-09T15:04:16Z")
Right now I have a MongoDB Sink connector. My incoming Kafka messages contain Auditlogs in the Kafka headers. The Kafka headers values are byte arrays and the Kafka message values are Avro. How can I best save the Header… --- ## [Do not get my Data written in the JDBC Sink](https://forum.confluent.io/t/do-not-get-my-data-written-in-the-jdbc-sink/6434)
**Author:** [@oierlings](https://forum.confluent.io/u/oierlings)\ **Replies:** 0\ **Last updated:** [3 November 2022 11:54 UTC](https://forum.confluent.io/t/do-not-get-my-data-written-in-the-jdbc-sink/6434 "2022-11-03T11:54:16Z")
Hi, I’m new to the confluent world. I’m trying to get data from a MySQLDB using the debezium MySQL connector, put the Data and write it to a Oracle DB using the JDBC Sink Connector. I’m able to get the Message into the… --- ## [Any existing SMTs to trim trailing white space from fields?](https://forum.confluent.io/t/any-existing-smts-to-trim-trailing-white-space-from-fields/6425)
**Author:** [@charmon79](https://forum.confluent.io/u/charmon79)\ **Replies:** 0\ **Last updated:** [2 November 2022 15:02 UTC](https://forum.confluent.io/t/any-existing-smts-to-trim-trailing-white-space-from-fields/6425 "2022-11-02T15:02:05Z")
Is anyone aware of an existing Single Message Transform plugin for Connect which can be used to trim trailing spaces from character/string fields? I’d love to be able to trim the values in the source connector, but I’m n… --- ## [Improve S3 sink connector upload speed to S3 bucket](https://forum.confluent.io/t/improve-s3-sink-connector-upload-speed-to-s3-bucket/6379)
**Author:** [@LethoOfGullet](https://forum.confluent.io/u/LethoOfGullet)\ **Replies:** 3\ **Last updated:** [31 October 2022 12:06 UTC](https://forum.confluent.io/t/improve-s3-sink-connector-upload-speed-to-s3-bucket/6379 "2022-10-31T12:06:00Z")
I’m using a 100 Partition topic with 3 Replicas and 2 ISR in a MSK serverless cluster. Datagen is producing the data using the “Orders” quickstart and unlimited throughput. I’m consuming this data using a Confluent S3 Si… --- ## [Postgres debezium self managed kafka connector not able to create topics on confluent cloud](https://forum.confluent.io/t/postgres-debezium-self-managed-kafka-connector-not-able-to-create-topics-on-confluent-cloud/6359)
**Author:** [@Braj04](https://forum.confluent.io/u/Braj04)\ **Replies:** 0\ **Last updated:** [27 October 2022 06:22 UTC](https://forum.confluent.io/t/postgres-debezium-self-managed-kafka-connector-not-able-to-create-topics-on-confluent-cloud/6359 "2022-10-27T06:22:40Z")
What:? I am trying to create a data pipeline to capture change data (CDC) from RDS PostgreSQL to kafka cluster on confluent cloud. How ? I am running a self managed kafka connect cluster in docker container in an EC2 … --- ## [Http Source Connector for Confluent Platform](https://forum.confluent.io/t/http-source-connector-for-confluent-platform/6294)
**Author:** [@nishyanthp](https://forum.confluent.io/u/nishyanthp)\ **Replies:** 3\ **Last updated:** [21 October 2022 09:20 UTC](https://forum.confluent.io/t/http-source-connector-for-confluent-platform/6294 "2022-10-21T09:20:37Z")
Hi, Http Source Connector is available in Confluent platform? As per below link I didn't see Http Source Connector is listed. https://docs.confluent.io/kafka-connectors/self-managed/kafka\_connectors.html#hdfs-3-sour… --- ## [Kafka Connect S3 - Nothing Getting Written to Bucket](https://forum.confluent.io/t/kafka-connect-s3-nothing-getting-written-to-bucket/6128)
**Author:** [@alemeno22](https://forum.confluent.io/u/alemeno22)\ **Replies:** 2\ **Last updated:** [29 September 2022 17:23 UTC](https://forum.confluent.io/t/kafka-connect-s3-nothing-getting-written-to-bucket/6128 "2022-09-29T17:23:35Z")
Hello! I’m trying to get the S3 Connector working with our MSK cluster using Avro. I’ve got it running, no errors in the connector logs, but nothing is getting written to the S3 bucket. I’ve got the IAM policy specified … --- ## [How to pick SAP connectors?](https://forum.confluent.io/t/how-to-pick-sap-connectors/6119)
**Author:** [@pawanjuly3](https://forum.confluent.io/u/pawanjuly3)\ **Replies:** 0\ **Last updated:** [24 September 2022 10:37 UTC](https://forum.confluent.io/t/how-to-pick-sap-connectors/6119 "2022-09-24T10:37:07Z")
Hi, In enterprise its hard to bring everyone to same page and I am not a SAP person , While i know more and sufficient as confluent user. But , I am finding it difficult to decide which SAP connector to use under whic… [Previous page](https://forum.confluent.io/c/kafka-connect/self-managed-connectors/21.md?page=2) [Next page](https://forum.confluent.io/c/kafka-connect/self-managed-connectors/21.md?page=4)