Connect with external/third-party Kafka Broker

Kafka Connect
1

Can I use S3SinkConnector to read topics from Kafka cluster which deployed at another EC2 instance?

Use case: I’d like to read/subscribe another team’s kafka cluster with Connect which I install on my EC2 instance.

2

So long as you can access the Kafka broker, it doesn’t matter what the consumer is. So yes, you can run your own Kafka Connect worker to pull data from another Kafka broker.

It’s a similar principle to what I show in this article Running a self-managed Kafka Connect worker for Confluent Cloud. The only difference really is that in my example the Kafka broker is a managed one (Confluent Cloud) whereas you’d be connecting to a self-managed Kafka broker. But the concept is the same.

1 Like
3

Can I access that Kafka Connect worker from outside. like calling Connect REST Interface?

4

Yes, so long as you set up your firewall etc correctly.

6

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.