Running kafka connector in distributed mode in AWS auto-scale group

Kafka Connect Managed Connectors
1

HI, it look like if we like to setup kafka connector in distributed mode , we will need to have a unique hostname at CONNECT_REST_ADVERTISED_HOST_NAME . However, if we deploy on AWS MSK environment with auto-scala group, there is no known hostname for that, not sure how can I do the setup?

2

Do you mean Amazon MSK Connect - Amazon Web Services OR are you running distributed connect hosted on EC2/EKS connected to MSK?

for Amazon MSK Connect - Amazon Web Services it does the scaling for you. You tell it the amount of vCPU you want and number of tasks, the connect-distributed layer is scaled for you.

If you’re running connect-distributed on EC2 then you provide the ADVERTISTED_HOST_NAME in the configuring for the connect workers. The host name is the hostname of the machine that connect-distributed is running.

3

Thanks for getting back, I mean the connectors that running at difference deployment from the MSK deployment

4

^^^ this is your answer.

5

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.