KRaft Controller Disk Size Recommendation

Hello,

I’m doing research to better understand KRaft’s expected resource utilization. I ran into a Confluent doc that provides a recommended disk size of at least 64GB, and I’m looking for help understanding what drives the recommendation.

Confluent’s documentation says

Minimum of 4 GB of RAM

An SSD disk at least 64 GB in size is highly recommended

Apache’s KRaft documentation says

The Kafka controllers store all the metadata for the cluster in memory and on disk. We believe that for a typical Kafka cluster 5GB of main memory and 5GB of disk space on the metadata log directory is sufficient.

Confluent’s RAM minimum is similar to the Apache estimate of KRaft’s RAM use, whereas Confluent’s recommended disk minimum is more than 10x Apache’s estimate of KRaft’s metadata log directory disk use.

Could someone explain what drives the 64 GB recommendation? Is it mainly for operational headroom plus a safety buffer on top of the 5 GB of use expected for KRaft metadata, or is there more to the picture?

Thank you,
Morgan

I can’t speak for Confluent’s sizing, but the gap is probably mostly headroom rather than steady-state usage. The metadata log is only part of what lives on a controller’s disk: you also have periodic snapshots (controlled by metadata.log.max.record.bytes.between.snapshots), segments that stick around until a snapshot covers them, and the usual application logs. Roundup sizes like 64 GB are also common because small SSD volumes often come with limited IOPS and throughput, so a bigger volume is a cheap way to get better performance and avoid running out of space during a spike in metadata changes, such as a mass topic or partition creation. The 5 GB figure from the Apache docs is more of a typical-case estimate. If you want to be more precise, I’d check the size of your metadata log directory under load in a test cluster, and keep an eye on the metadata log size and snapshot metrics to see how much you actually use.