Kafka Connect Elasticsearch sink bandwidth

Kafka Connect
1

I am seeing huge bandwidths when sending data to ES even when the connector is just connected to ES.
When I capture some of the traffic I see
producer-2u0connect-statusF+w9w9Nstatus-connector-elastic-search-topic16{"state":"RUNNING","trace":null,"worker_id":"172.31.133.111:8083","generation":9} Dconnect-status

This seems to get spammed. When I send lets a 100 messages it just keeps running like its constantly sending messages. This never stops until I stop running the worker. Is there a setting that could be causing this to keep sending data like this?

kafka2
kafka2736×101 3.34 KB

Any insight would be appreciated.

2

I was able to figure it out. I ended up deleting the topic configs that were created when running the working. Im assuming something got corrupted or something like that.

3

In case you need more information, you can execute GET _tasks in Elasticsearch to get an idea of what’s going on on the other side of the fence

4

Thank you, I was having issues with tasks running i think something got messed up. Everything seems to be working now.
Thanks again

1 Like
5

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.