SASL_PLAINTEXT Authentication Failed - Kafka Connect Opensearch Plugin

Kafka Connect
1

I am using the Kafka Connect OpenSearch plugin with SASL_PLAINTEXT authentication, while creating a new connector with REST API, I am not able to authenticate created connector, and it’s throwing an error - Authentication Failed while SASL handshake.

I need to understand what key-value pairs I need to pass along with JSON body that is relevant to SASL authentication.

I am sharing the JSON body which I am using while creating a new Connector with REST API -
POST - http://10.30.1.101:8083/connectors

{
    "name": "first-opensearch-connector",
    "config": {
        "name": "first-opensearch-connector",
        "connector.class": "com.dmathieu.kafka.opensearch.OpenSearchSinkConnector",
        "type.name": "_doc",
        "connection.password": "admin",
        "connection.username": "admin",
        "connection.url": "http://10.30.1.101:9200",
        "topics": "my_topic",
        "value.converter": "org.apache.kafka.connect.json.JsonConverter",
        "value.converter.schemas.enable": "false",
        "tasks.max": "1",
        "key.ignore": "true",
        "schema.ignore": "true",
        "drop.invalid.message":"true",
        "behavior.on.malformed.documents":"fail",
        "write.method":"INSERT",
        "read.timeout.ms":"10000",
        "errors.log.enable": "true",
        "errors.log.include.messages": "true",
        "errors.tolerance": "all",
        "transforms":"AddPrefix,TimestampRouter,InsertField",
        "transforms.AddPrefix.type":"org.apache.kafka.connect.transforms.RegexRouter",
        "transforms.AddPrefix.regex":".*",
        "transforms.AddPrefix.replacement":"acme_$0",
        "transforms.TimestampRouter.type": "org.apache.kafka.connect.transforms.TimestampRouter",
        "transforms.TimestampRouter.topic.format": "foo-bar-${topic}-${timestamp}",
        "transforms.TimestampRouter.timestamp.format": "YYYYMMdd",
        "transforms.InsertField.type": "org.apache.kafka.connect.transforms.InsertField$Value",
        "transforms.InsertField.static.field": "MessageSource",
        "transforms.InsertField.static.value": "Kafka Connect framework",
        "sasl.mechanism": "PLAIN",
        "security.protocol": "SASL_PLAINTEXT",
        "sasl.jaas.config": "org.apache.kafka.common.security.plain.PlainLoginModule required username=\"admin\" password=\"admin-secret\""
    }
}
2

@rmoff Can you help me in this?

3

@mann2108 I see you’ve posted an issue here: SASL_PLAINTEXT - Help Needed - Authentication getting failed with connector. · Issue #41 · dmathieu/kafka-connect-opensearch · GitHub

It’s not completely clear to me where you are seeing authentication issues.

For the Connect Worker to Kafka authentication you can provide the SASL details in the connect worker configuration file (connect-distributed.properties per your GH issue). In your GH issue you attempt to provide them in the JSON config payload, but that shouldn’t be necessary (and the format is incorrect, as you break up the sasl.jaas.config value).

Can you provide more clarity where exactly you are seeing this (how are you seeing it, which component is logging it, and any additional context around it)?

4

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.