Read/consume from Remote Kafka cluster

Use case:

Read/consume from another team’s /external Kafka cluster topic using Kafka Connect which is hosted on AKS cluster.


Can we use kafka connect without creating below internal topics in remote kafka cluster. if not, what is the alternative?


Please advise.


not 100% sure about your scenario/use-case.
But based on your question I assume that you want to do the following:
“”“You don’t have any internal / own kafka cluster you just want to use a Kafka Sink Connector to get data out from another kafka cluster which is not under your control

If this assumption is correct than I think that there is no way arround these topics if you want to use a distributed connect cluster. Kafka Connect needs topics to manage it’s own state and that is what those topics are used for.

In “stand-alone” mode it might be possible - but I never used that one.

With distributed mode I only see the possibility to get your own kafka cluster, setup mirrormaker2 to replicate the data from your source cluster to yours (if needed with less retention) and config a connector to get the data out of your cluster.

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.