Before Apache Kafka 2.4, if you didn’t provide the partition number or a key, the partition assignment from the DefaultPartitioner was that of a round-robin approach; in that, it would start with a given partition number and then increment by one until it hit the max number of partitions then start over again at zero.
While this approach resulted in an even distribution, it had the drawback of spreading out records without keys into smaller batches.
But since the 2.4 release, this issue of sparse batches has been solved using a “sticky-partitioner” approach. The DefaultPartitioner now chooses a partition at random and will stick with that partition. This new approach still results in roughly even distribution with the benefit of large batch sizes, which reduces latency. For more details, you can read about KafkaProducer improvements in 2.4
