# Problem with clusters **URL:** \ **Category:** Ops\ **Created:** [12 July 2024 15:22 UTC](https://forum.confluent.io/t/problem-with-clusters/11054 "2024-07-12T15:22:02Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![psotos](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/psotos/32/3506_2.png) [@psotos](https://forum.confluent.io/u/psotos)\ **Post date:** [12 July 2024 15:22 UTC](https://forum.confluent.io/t/problem-with-clusters/11054/1 "2024-07-12T15:22:02Z")
Hi all! I run into this issue frequently: when I reboot or update my kafka servers some of them fail to start due to the cluster id. 1. I have the cluster ids set the same between all versions 2. Each Broker has a unique id and I have 3 brokers (0, 1, 2) 3. I am NOT using the /tmp directory to store my logs. They are in /opt/kafka/kafka-logs 4. I am running zookeeper on all 3 of my brokers. 5. All server.properties files have the zookeeper.connect pointing to all 3 zookeeper services 6. Only 1 service is failing to start properly with the “The Cluster ID Pwzs3pUqSZCJm79gXzJqGw doesn’t match stored clusterId” 7. The above cluster id was the old one which I blew out to get all of the brokers working together again. Listed below is the new one. 8. My servers all have the new setting cluster.id=3eS38X7oT0aHo10ZJfo9UQ 9. Kafka is kafka\_2.13-3.5.0 10. I have no important data here yet. I dont mind blowing everything away and restarting from scratch if needed. I just want this issue to go away and never occur again between upgrades and reboots of my servers. FYI my kafka-logs dir is NOT underneath my install dir so I can upgrade kafka with no issues. 11. One broker is refusing to come up and its reporting that error that I listed above, 12. All three brokers are running on different servers and there are no docker containers involved. What am I doing wrong here? ---
**Author:** ![dtroiano](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/dtroiano/32/1961_2.png) [@dtroiano](https://forum.confluent.io/u/dtroiano)\ **Post date:** [12 July 2024 18:11 UTC](https://forum.confluent.io/t/problem-with-clusters/11054/2 "2024-07-12T18:11:28Z")
From step 7: > [@psotos](#): > > The above cluster id was the old one which I blew out to get all of the brokers working together again. What was broken in the cluster that caused you to want to change the cluster ID? Were you hitting the `InconsistentClusterIdException` or something else? How did you blow away the old one? Bear in mind that the cluster ID lives in a file called `meta.properties` on the broker in the `log.dirs` directory, e.g.: ```noformat [appuser@broker data]$ cat meta.properties # #Fri Jul 12 18:00:16 UTC 2024 broker.id=1 version=0 cluster.id=vM8Y5EE-SZycFGARoLmVAA ``` _And_ the cluster ID is also stored in ZooKeeper at the `/cluster/id` node: ```noformat get /cluster/id {"version":"1","id":"vM8Y5EE-SZycFGARoLmVAA"} ``` Do you see different values for your deployment? This would cause that error. If they are different, you can either [make them match](https://stackoverflow.com/a/60132830/5587460) to get things working again, or start over with a clean slate (fresh ZooKeeper install). ---
**Author:** ![psotos](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/psotos/32/3506_2.png) [@psotos](https://forum.confluent.io/u/psotos)\ **Post date:** [14 July 2024 02:03 UTC](https://forum.confluent.io/t/problem-with-clusters/11054/3 "2024-07-14T02:03:09Z")
Thank you! This is some good insight. I will post again once I try and resinstall and get it all working. ---
**Author:** ![psotos](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/psotos/32/3506_2.png) [@psotos](https://forum.confluent.io/u/psotos)\ **Post date:** [14 July 2024 17:07 UTC](https://forum.confluent.io/t/problem-with-clusters/11054/4 "2024-07-14T17:07:22Z")
So its still not working. I blew away the directories and reinstalled and configured both zookeeper and kafka. I am using kafka\_2.13-3.7.1 now. I am pointing to the two other brokers in my cluster which now have the correct ids. Why isnt it picking up the correct cluster id from the other brokers? Do I need to list the other brokers first in the server.properties file? Ok I got it working by eliminating the Zookeeper running on my 3rd broker. I updated it to no longer point to localhost but to the other two brokers and kafka came up fine. I am still interested in knowing why this one failed. You mentioned some sort of cluster id setting in Zookeeper but I had no clue which file to edit to change it. Can you be more explicit? Thanks! ---
**Author:** ![dtroiano](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/dtroiano/32/1961_2.png) [@dtroiano](https://forum.confluent.io/u/dtroiano)\ **Post date:** [15 July 2024 14:49 UTC](https://forum.confluent.io/t/problem-with-clusters/11054/5 "2024-07-15T14:49:40Z")
> [@psotos](#): > > You mentioned some sort of cluster id setting in Zookeeper but I had no clue which file to edit to change it. Can you be more explicit? You would need to use the CLI to edit it. E.g., the `get /cluster/id` command above was in the [`zookeeper-shell`](https://docs.confluent.io/kafka/operations-tools/kafka-tools.html#zookeeper-shell-sh) tool. The corresponding command to set the ID in ZooKeeper is `set`: ```noformat set /cluster/id {"version":"1","id":""} ``` ---
**Author:** ![psotos](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/psotos/32/3506_2.png) [@psotos](https://forum.confluent.io/u/psotos)\ **Post date:** [17 July 2024 11:37 UTC](https://forum.confluent.io/t/problem-with-clusters/11054/6 "2024-07-17T11:37:09Z")
> [@dtroiano](#): > > `{"version":"1","id":""}` I’m getting this error: ./zookeeper-shell.sh set /cluster/id {“version”:“1”,“id”:“3eS38X7oT0aHo10ZJfo9UQ”} Connecting to set ZooKeeper -server host:port [-zk-tls-config-file] cmd args [2024-07-17 11:34:48,071] ERROR Unable to resolve address: set/:2181 (org.apache.zookeeper.client.StaticHostProvider) java.net.UnknownHostException: set: Temporary failure in name resolution Command not found: Command not found /cluster/id ---
**Author:** ![dtroiano](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/dtroiano/32/1961_2.png) [@dtroiano](https://forum.confluent.io/u/dtroiano)\ **Post date:** [17 July 2024 13:53 UTC](https://forum.confluent.io/t/problem-with-clusters/11054/7 "2024-07-17T13:53:57Z")
To get into the shell, run: ```noformat zookeeper-shell.sh : ``` `:` would probably be `localhost:2181`. Then run the `set` command inside the shell. ---
**Author:** ![psotos](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/psotos/32/3506_2.png) [@psotos](https://forum.confluent.io/u/psotos)\ **Post date:** [17 July 2024 13:58 UTC](https://forum.confluent.io/t/problem-with-clusters/11054/8 "2024-07-17T13:58:34Z")
Ok I will try that. Thanks so much for this help! It has been a great learning experience!