REST proxy best practices

Hi, I m exploring strategies around setting up confluent RESTproxy. Right now we have restproxy nodes for each Kafka Cluster. There are few times we have seen rest proxy nodes crashing due to large messages i.e…few applications sending large messages than topics actually allow (>max.message.bytes) , the error message is below. Every time this happens rest proxy nodes crash, and all the other applications hosted on the same restproxy cluster are impacted.

Got error produce response in correlation id 12871518 on topic-partition topic-iotstabilize-0, splitting and retrying (0 attempts left). Error: MESSAGE_TOO_LARGE

With this happening more often, there are several conversations around setting up dedicated rest proxy cluster for each application despite they all share the same Kafka Cluster.

Just curious how everyone else dealing with this type of scenarios? Is that normal to have several restproxy clusters for each Kafka Cluster? or if there is a way we can prevent the restproxy clusters going down when we come across something like mentioned above

What version of REST Proxy are you running?

Hi, Thanks for the response. We are using old version 4.1 confluent bundle. Probably this is fixed in the latest versions?

That version is from nearly three years ago, so plenty will have been fixed in that time. The latest release is 6.1, I would definitely try and and see if you can reproduce the issue there.

Yeah, we have plans to upgrade to the latest version. Will prioritize that and go from there

Thank you!

1 Like