I am looking at an Enterprise confluent cloud system where data is being sent over 50+ topics, representing data from 500+ customers. To improve observability into the platform and incident response, I would like to be able to query the following data via an API endpoint.
- Performance metrics per topic
I would like to be able to see how much data is being processed per minute/hour/etc by a specific topic. This information is available in the UI, but is it queriable? Is there a way to get this information from the ‘metric’ API for confluent?
- Number of items processed per customer, per minute
A little more detailed. In the UI, I can temporarily view messages as they are streaming. In our event stream, we provide a customer ID that I would like to be able to power analytics with. For example, to identify how many requests are being processed for a given customer on a given topic over time. Looking at the documentation, that appears to be the use case for kSQL? Is that possible with kSQL, or am I misinterpreting? Is there another way to achieve this?