Are supporting to configure HDFS Sink connector with multiple namenodes?

Kafka Connect
1

Hello,

I would like to configure HDFS Sink connector send data from topic to HDFS system.
HDFS system had configure as cluster and contains primary & secondary NameNode.

Are supporting to configure HDFS Sink connector with multiple name node?
If yes, please help to guide which properties to configure to connecting both primary & secondary NameNode.

Thanks
May

2

The connector uses plain HDFS client Java library.

A nameservice can be used, yes, that maps to the active namenode in HA mode.

You’ll need to copy your cluster hdfs-site.xml and core-site.xml into each connect worker and set HADOOP_CONF_DIR directory

3

Thanks for your reply.

So in case failed over of Cloudera service connector will checking from hdfs-site.xml and core-site.xml and auto switch to another active NameNode with auto or needs any manual step to do?

May

4

The HDFS client library knows how to switch

5

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.