Connect JDBC Driver not listed

Kafka Connect
1

Hi Community.

I’m getting startet with the JDBC Source Connector tutorial, but the connector is not listed.

I’m working on Ubuntu and installed the Confluent platform on-premise using the tar files.

I installed the connector with:

confluent-hub install confluentinc/kafka-connect-jdbc:latest

The following command does list both JDBC source and sink connector:

confluent local services connect plugin list
  {
    "class": "io.confluent.connect.jdbc.JdbcSinkConnector",
    "type": "sink",
    "version": "10.1.0"
  },
  {
    "class": "io.confluent.connect.jdbc.JdbcSourceConnector",
    "type": "source",
    "version": "10.1.0"
  },

But listing the connector does not give me the JDBC connector available for loading:

confluent local services connect connector list

The local commands are intended for a single-node development environment only,
NOT for production usage. https://docs.confluent.io/current/cli/index.html

Bundled Connectors:
  file-sink
  file-source
  replicator

Here’s the output from the REST endpoint:

curl -s localhost:8083/connector-plugins
[{"class":"io.confluent.connect.jdbc.JdbcSinkConnector","type":"sink","version":"10.1.0"},{"class":"io.confluent.connect.jdbc.JdbcSourceConnector","type":"source","version":"10.1.0"},{"class":"io.confluent.connect.replicator.ReplicatorSourceConnector","type":"source","version":"7.0.1"},{"class":"org.apache.kafka.connect.mirror.MirrorCheckpointConnector","type":"source","version":"1"},{"class":"org.apache.kafka.connect.mirror.MirrorHeartbeatConnector","type":"source","version":"1"},{"class":"org.apache.kafka.connect.mirror.MirrorSourceConnector","type":"source","version":"1"}]

In the directory /share/confluent-hub-components I can find the confluentinc-kafka-connect-jdbc

The plugin path seems to be correct I assume:
confluent local services connect log|grep -i plugin.path

	plugin.path = [/home/ubuntu/dev/confluent/confluent-7.0.1/share/java, /home/ubuntu/dev/confluent/confluent-7.0.1/share/confluent-hub-components]
[2022-03-03 18:59:12,996] WARN The configuration 'plugin.path' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:384)
[2022-03-03 18:59:13,449] WARN The configuration 'plugin.path' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:384)
[2022-03-03 18:59:13,478] WARN The configuration 'plugin.path' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:384)
[2022-03-03 18:59:13,615] WARN The configuration 'plugin.path' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:384)
[2022-03-03 18:59:13,635] WARN The configuration 'plugin.path' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:384)
[2022-03-03 18:59:13,649] WARN The configuration 'plugin.path' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:384)
[2022-03-03 18:59:13,676] WARN The configuration 'plugin.path' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:384)
[2022-03-03 18:59:13,709] WARN The configuration 'plugin.path' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:384)
[2022-03-03 18:59:13,753] WARN The configuration 'plugin.path' was supplied but isn't a known config. (org.apache.kafka.clients.producer.ProducerConfig:384)
[2022-03-03 18:59:13,781] WARN The configuration 'plugin.path' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig:384)
[2022-03-03 18:59:13,853] WARN The configuration 'plugin.path' was supplied but isn't a known config. (org.apache.kafka.clients.producer.ProducerConfig:384)
[2022-03-03 18:59:13,858] WARN The configuration 'plugin.path' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig:384)
[2022-03-03 18:59:13,922] WARN The configuration 'plugin.path' was supplied but isn't a known config. (org.apache.kafka.clients.producer.ProducerConfig:384)
[2022-03-03 18:59:13,927] WARN The configuration 'plugin.path' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig:384)

From seeing these warnings, am I missing a configuration?

Is there anything else I am missing? Is there some way I need to create a connector from the plugin first?

2

The terminology is confusing and may be the source of confusion here :slight_smile:

As shown in the output of confluent local services connect plugin list you have the JDBC connector plugin correctly installed. Now you need to go ahead and create an instance of the connector itself. You can do this with confluent local services connect connector load or with the Kafka Connect REST API itself.

This might help clarify things too: Connectors, Configuration, Converters, and Transforms

1 Like
3

Thank for your response, it helped a lot!

2 Likes
4

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.