Hello there,
I am currently working on setting up a Kafka cluster for a project and could use some advice on configuring it for high availability. I have gone through the Kafka documentation, but I am looking for some real-world insights and best practices from those who have experience with this.
What replication factor do you recommend for a high availability setup? Should I go with a replication factor of 2 or 3?
How many Kafka brokers should I have in my cluster for optimal high availability?
Any recommendations on hardware configuration for Kafka brokers for a high availability setup?
Also, I have gone through this post: https://forum.confluent.io/t/what-is-the-best-configuration-for-apache-kafka-confluent-kafka-if-i-use-kafka-connect/6085tableau which I definitely helped me out a lot.
What tools do you recommend for monitoring Kafka clusters, and what are the key metrics to watch for ensuring high availability?
Thanks in advance for your help and assistance.