Adding new partition to a keyed topic

Architecture and Design
1

I need to increase the partition count for a Kafka topic to increase throughput at the consumer. This topic is partition keyed as the consumer expect the messages to reach them in an order.

Kafka is using the default partitioner which is hash key mod by total partitions. If I introduce a new partition, the order of the messages consumed will be messed up. Is there an easy way to introduce new partition and still maintain the order of messages consumed?

We are at Kafka version 2.5

2

You’d have to create a new topic then mirror existing topic into it, then migrate your producers after the mirroring is complete (assuming time order matters as well).

If time order doesn’t matter, you can produce into the new topic while data is mirrored