Let me re run it. I disabled the cognito to make sure it was working again.
The first error I saw before I added the connection.username
was that there was an error authenticating which is understood.
After I added the below config, this was one of the errors I saw.
Error: Failed to chec$ org.apache.kafka.connect.errors.ConnectException: Failed to check if index prod-parser-logs-2021-03-24 exists.
Here is the connector config.
"connector.class": "io.confluent.connect.elasticsearch.ElasticsearchSinkConnector",
"type.name": "_doc",
"tasks.max": "2",
"topics": "prod-parser-logs",
"transforms.routeTS.type": "org.apache.kafka.connect.transforms.TimestampRouter",
"transforms": "routeTS",
"transforms.routeTS.topic.format": "${topic}-${timestamp}",
"connection.username":"test",
"connection.password":"%test%",
"key.ignore": "true",
"schema.ignore": "true",
"transforms.routeTS.timestamp.format": "YYYY-MM-dd",
"value.converter.schemas.enable": "false",
"name": "elastic-search-prod-parser-logs2",
"connection.url": "https://****",
"value.converter": "org.apache.kafka.connect.json.JsonConverter",
"key.converter": "org.apache.kafka.connect.storage.StringConverter"
}
Version of ES I am using is 7.7.
Confluent version I am using 6.0.
Let me know if you want me to test it again and see if there are other errors I missed.
Thank you