Kafka: Where can I set max.request.size parameter?

Ops
1

Hello Everyone,

I have MSK cluster which I use in my application.

I am getting error as
Caused by: org.apache.kafka.common.errors.RecordTooLargeException: The message is 2736470 bytes when serialized which is larger than the maximum request size you have configured with the max.request.size configuration.

I am trying to set max.request.size in MSK, but it says that max.request.size is not supported.

Where do I need to set max.request.size?

  • In cluster config
  • In application in kafka producer

Do we have different config for Kafka and MSK?

Please help

2

Seems like you have a single message, that’s more than 1MB. If you want to do this you need to change multiple default values in multiple places. At least message.max.bytes for the brokers, max.message.bytesfor the topic, max.request.size for a producer, max.partition.fetch.bytes for a consumer.

In general it is not recommended to change those, but as long as you don’t send messages bigger than about 16MB, it should work fine.

1 Like