Kafka Streams Limitations For Complex Transformations

Hi,
As Kafka Streams gives flexibility for lightweight transformations. I want to know that what are the limitations of Kafka Streams in Complex transformation. By complex transformations I mean number of topics going through join and other heavy transformations. What is advisable in this scenario? What are the boundaries of Kafka Streams against which performance will not get affected?

Thank you

I think performance will always be affected by the amount of data being processed and the complexity of that process, regardless of the tool used, but I have seen Kafka Streams used with small and large topologies, some simple, and many more complex. If you run into limitations with the Kafka Streams DSL, you can use the Processor API. Perhaps you can take it for a spin, and build a proof-of-concept with it. Then report back here if you have more questions or run into problems.

Have fun!

Dave

2 Likes

+1 to what @daveklein said.

1 Like