Bounded RocksDB Metrics - Cache Capacity

Kafka Streams
1

Hi all

I am trying to bound the RocksDB usage (Kafka Streams Memory Management for Confluent Platform | Confluent Documentation) and monitor the overall capacity. As I understood the docs (Monitor Kafka Streams Applications in Confluent Platform | Confluent Documentation) for bounded RocksDB usage, in a shared cache setup the metrics should not be shown per state store, but in total:

The block cache metrics report the sum over all RocksDB instances if each instance uses its own block cache, and they report the recorded value from only one instance if a single block cache is shared among all instances.

When I run my Stream app with the rocksdb config, I still see a state store cache (kafka_stream_state_block_cache_capacity) per state store.
I am running a Kafka Streams example with Quarkus 3.12 (Kafka 3.7.0).

What could be the reason for this? I would expect only one cache capacity metric for my machine?

Thank you in advance for your input.

Cheers
Thomas

2

It’s just how metrics are setup. Each store reports this metric. However, because you use a shared block cache, each store would report the total (ie, each store reports the same value).

3

I see, makes sense. Many thanks for the quick feedback & the explanation.
Best regards, Thomas

1 Like
4

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.