Docker Compose ps has the following
kafka
kafka-connect
schema-registry
zookeeper
elasticsearch
kibana
ksqldb-cli
primary-ksqldb-server
I have downloaded the zip file for ElasticSearch Sink Connector from confluent.io, hub: confluentinc kafka-connect-elasticsearch
the unzipped folder :
/etc/kafka-connect/jars/confluentinc-kafka-connect-elasticsearch-14.0.1
The docker compose file has
CONNECT_PLUGIN_PATH: “/usr/share/java,/etc/kafka-connect/jars”
volumes:
- $PWD/jars:/etc/kafka-connect/jars
Copied all the jars from /etc/kafka-connect/jars/confluentinc-kafka-connect-elasticsearch-14.0.1/lib to /etc/kafka-connect/jars
$ pwd
/etc/kafka-connect/jars
$ grep io.confluent.connect.elasticsearch.ElasticsearchSinkConnector *
Binary file kafka-connect-elasticsearch-14.0.1.jar matches
$ cat kafka-connect-elastic.json
{
“name”: “elasticsearch-sink”,
“config”: {
“name”: “elasticsearch-sink”,
“connector.class”: “io.confluent.connect.elasticsearch.ElasticsearchSinkConnector”,
“topics”: “ABC”,
“tasks.max”: “1”,
“key.converter”: “org.apache.kafka.connect.storage.StringConverter”,
“value.converter”: “org.apache.kafka.connect.storage.StringConverter”,
“connection.url”: “local host 9200” (new user so i cant post more than 2 links in a post !! ),
“type.name”: “log”,
“key.ignore”: “true”,
“schema.ignore”: “true”
}
}
curl of localhost 8083 / connectors (as a new user I cant post a topic with more than 2 links)
I try to create a connector.
$ curl -s -X POST -H ‘Content-Type: application/json’ --data @kafka-connect-elastic.json http://localhost:8083/connectors
Gives the error
“Failed to find any class that implements Connector and which name matches io.confluent.connect.elasticsearch.ElasticsearchSinkConnector”
Any help is highly appreciated.