Rate Limited when producing message to my cluster

Confluent Cloud
1

Hey folks,

I’m using Rest Endpoint to produce kafka message to my topic in my cluster.

However, I recently hit a 429 too many requests error. I was not aware my cluster has a rate limit. Is there a way to increase the rate limit?

Thanks in advance!

2

Cluster limits are documented here. What kind of cluster do you have? Based on your API usage, do you have a sense of which limit you’d be hitting? I’m guessing that it would either be Kafka REST Produce v3 - Max connection requests (per second) (max 25 except for Dedicated clusters) or Kafka REST Produce v3 - Max streamed requests (per second) (1000 except for Dedicated clusters).

Sometimes it’s possible to increase service quotas through a support ticket. More on that here.

3

Hi @dtroiano , thank you so much for your response. Most likely we are hitting this REST Produce quota. 25 per second is pretty limited.

I’m going to submit a ticket to get it increased. Thank you!