Hello,
in our application we use kafka-streams to synchronize erp system data with apps in the cluster.
We have a chain consisting of debezium, that produces messages, and a connector that reads via kafka-streams and then publishes the converted data according to our needs.
We also configured the kafka.client.streams.stateDir cache and assigned a persistent volume to that cache.
What we notice is that the memory space using the cache always grows forcing us to keep an eye on the volume at all times.
If in a redeployment phase the volume is cleared then it returns to an acceptable size and then slowly grows again. We have also enlarged the space on the volume.
What we would expect is that the space used by the cache would not only grow, but also go down . Do any of you have experience with this issue? I would appreciate any help.
Thank you.