Best practice for expose global store to Spring Beans

Kafka Streams
1

Hi,

my case very, very simple: I want expose small compacted topic to Spring Application as key value store. I think very simple option would be a global store from Kafka Streams. I do not want any transformation - just expose messages as-is.

I can resolve all problem with sophisticated Spring setup. The problem that all examples I found focus on transformations, processors, pipes, etc. I do not want it. All I want is public global store, automatically synchronized with topic. I want the store be available to plain @Components

I found one example how to implement that here: Kafka Streams With Spring Boot | Baeldung

the factoryBean is an instance of StreamsBuilderFactoryBean that is wired into the controller. This provides the KafkaStreams instance managed by this factory bean. Therefore, we can obtain the counts key/value state store that we created earlier, represented by KTable . At this point, we can use this to get the current count for the requested word from the local state store.

@GetMapping("/count/{word}")
public Long getWordCount(@PathVariable String word) {
    KafkaStreams kafkaStreams = factoryBean.getKafkaStreams();
    ReadOnlyKeyValueStore<String, Long> counts = kafkaStreams.store(
      StoreQueryParameters.fromNameAndType("counts", QueryableStoreTypes.keyValueStore())
    );
    return counts.get(word);
}

This example presents exactly what I need. But I have a doubt - is it reasonable to call factoryBean.getKafkaStreams().store(...) on each request, over and over again? It looks a bit unefficient…

1 Like
2

Hi,

With Interactive Queries V1, that’s pretty much what you’ll need to do. You could consider caching the result (not the store, as it’s a live resource on the streams instance)
You’d be better off migrating to Interactive Queries V2, where instead of retrieving a state store, you issue a query and receive results. There was a presentation at Current 2022, and there’s a code repo with a sample implementation from the talk.

HTH,
Bill

3

Hi Bill,
I need something simpler. I want to work with global store. It means all data I want is available within JVM and all stuff with remote call is useless.

4

Hi Bill,

For the GlobalTables the KafkaStreams.query return the error (KAFKA-13523), so the store method is only way to use it.

I also interested in the information if it is possible to use the store directly (inject or cache it). Not to call each time factory → streams → store.

Best,
Stas