Hello everyone . I am working on a machine learning engine using kafka and kafka streams( vanilla kafka project) .
I am a little bit confused about throughput.Right now i send arround 700000 records and i got ( 30000 records/second ) into my kafka streams class with 4 streams threads. If i send more data like 3x initial data my throughput is even better ( 50000 records /second)
How to use metrics in general? How to simulate an optimal scenario, How i would be sure that i am doing well when eveything changes everytime?