I have 5 brokers running rocky 9 and ‘confluent-kafka-2.11-2.0.1cp8-1’.
The yum repo is ‘https://packages.confluent.io/rpm/5.0’ - which Im guessing is 3 major versions behind. The goal is to land on the latest (v8.x) without too much disruption of existing data.
I’ve tried looking for guidance on doing the upgrade but it seems that the 5→6 path is too old. All the links go to the same Confluent Documentation | Confluent Documentation which doesn’t appear to have any information on this (v dated) upgrade path.
You can download docs for older Confluent Platform versions (7.2 and under) here. 5 –> 8 is a big jump and you might run into a bunch of pain upgrading in place. I’m not sure how painful it’ll be – just a hunch that it’ll be a hassle. Depending on how the process goes, you may want to keep the idea of data migration to new cluster in your back pocket (i.e., spin up a fresh 8.1 cluster and migrate data).
Unfortunately this is all metal and not handy cloud vms so I’m limited in migration. I’ll have to upgrade each piece separately to the target version.. test the works.. and repeat with the next version.
My main concern is that I don’t miss any mandatory versions on the way.
On second thought.. maybe the mirrormaker approach would (ultimately) be easier. Now just need to figure out how to maintain continuity for connected clients.
Got it – probably goes without saying that you’ll want to do a bunch of reading first (release / upgrade notes for all versions along the way).
To be conservative, you might want to imagine what you would have to do if your Kafka cluster winds up hosed during the process, and proactively create a plan involving backup / restore, etc.