Hello Team,
We want to producer messages of more than 1MB in size for one of our topic.
We checked documentation and found total 4 configs that needs to be changed,
- Consumer side:
fetch.message.max.bytes
- this will determine the largest size of a message that can be fetched by the consumer. - Broker side:
replica.fetch.max.bytes
- this will allow for the replicas in the brokers to send messages within the cluster and make sure the messages are replicated correctly. If this is too small, then the message will never be replicated, and therefore, the consumer will never see the message because the message will never be committed (fully replicated). - Broker side:
message.max.bytes
- this is the largest size of the message that can be received by the broker from a producer. - Broker side (per topic):
max.message.bytes
- this is the largest size of the message the broker will allow to be appended to the topic. This size is validated pre-compression. (Defaults to broker’smessage.max.bytes
.)
But we are worried about 3rd config on broker side that should not affect rest of the topic consumers in a cluster. Because we read one article saying
message.max.bytes 1000000
(1 MB)Maximum message size the broker accepts. When using the old consumer, this property must be lower than the consumer fetch.message.max.bytes, or the consumer cannot consume the message.
For other topic consumer in a cluster if fetch.message.max.bytes is default 1mb. So my question is would this 3rd config change affect for particular topic consumer or all consumer.