Does the Consumer API provide anything similar to state store provided in the Kafka Streams API? If not, what is best practice making Consumers stateful? Would it be a viable option to use rayokota/kcache: An In-Memory Cache Backed by Apache Kafka (github.com)?
The clients module is open source, so you can see it doesn’t pull in many external dependencies.
Hashmap is obviously included in Java standard library, this is all the schema registry uses (multiple nested ConcurrentHashMap), and it’s a fully functional, leader elected REST API. But you’re welcome to serialize data structures to disk, as well to keep a warmer cache, as is the case of KTable with RocksDB.
If you want anything beyond that, you’re going to need to pull external dependencies. And yes, if you want to use kcache, or something similar, you’ll want to ensure you’re using a compacted topic.
1 Like
This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.