Impact of optional Header fields in Kafka

Architecture and Design
1

Hi,

I would like to know the impacts of using optional header fields in Kafka messages. i.e. All the messages will have set of default header fields and in addition to that, a selected set of messages (few messages) will have another set of optional header fields.
e.g.

  • 100,000 messages with default header fields.
  • 1 message with default header + additional header field

By doing so, will there be a header size increase for other messages. And as a result, will there be an overall size increase in the Kafka topic partition log file.

Thanks in advance.

2

Headers are stored inline with the messages. There are no default headers or fields applied. If you add headers to one message, that will not increase the size of any other message.