Errors with ES sink connector

Kafka Connect
1

I currently run two linux servers running kaftka connect in a cluster. It has been working great.
Once ever 2-3 weeks I get this error below where I need to restart the service. Does anyone know if there is a way to fix this error or prevent it from happening.

[2021-04-07 06:14:08,128] ERROR WorkerSinkTask{id=elastic-search-prod-parser-logs2-0} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted. Error: Bulk request f$
org.apache.kafka.connect.errors.ConnectException: Bulk request failed.
        at io.confluent.connect.elasticsearch.ElasticsearchClient$1.afterBulk(ElasticsearchClient.java:333)
        at org.elasticsearch.action.bulk.BulkRequestHandler$1.onFailure(BulkRequestHandler.java:76)
        at org.elasticsearch.action.bulk.Retry$RetryHandler.onFailure(Retry.java:122)
        at org.elasticsearch.client.RestHighLevelClient$1.onFailure(RestHighLevelClient.java:1603)
        at org.elasticsearch.client.RestClient$FailureTrackingResponseListener.onDefinitiveFailure(RestClient.java:580)
        at org.elasticsearch.client.RestClient$1.failed(RestClient.java:330)
        at org.apache.http.concurrent.BasicFuture.failed(BasicFuture.java:137)
        at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.executionFailed(DefaultClientExchangeHandlerImpl.java:101)
        at org.apache.http.impl.nio.client.AbstractClientExchangeHandler.failed(AbstractClientExchangeHandler.java:426)
        at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.timeout(HttpAsyncRequestExecutor.java:387)
        at org.apache.http.impl.nio.client.InternalIODispatch.onTimeout(InternalIODispatch.java:92)
        at org.apache.http.impl.nio.client.InternalIODispatch.onTimeout(InternalIODispatch.java:39)
        at org.apache.http.impl.nio.reactor.AbstractIODispatch.timeout(AbstractIODispatch.java:175)
        at org.apache.http.impl.nio.reactor.BaseIOReactor.sessionTimedOut(BaseIOReactor.java:263)
        at org.apache.http.impl.nio.reactor.AbstractIOReactor.timeoutCheck(AbstractIOReactor.java:492)
        at org.apache.http.impl.nio.reactor.BaseIOReactor.validate(BaseIOReactor.java:213)
        at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:280)
        at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
        at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: 30,000 milliseconds timeout on connection http-outgoing-1059 [ACTIVE]
        ... 11 more
11
2

The timeout from Elasticsearch could be caused by various things.

Stuff I’d check:

  • Is it a lot of data that you’re ingesting to Elasticsearch?
  • Are there any errors in the Elasticsearch log?
  • How is resource utilisation looking (CPU, network, memory, disk) on the Elasticsearch nodes? Is it hitting any limits?
3

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.