Kafka events for the same key landing on different pods post-migration causing inconsistent processing

Kafka Streams
1

After migrating from Event Hub to Kafka (Retina), we’re noticing that events with the same partition key are being processed on different pods of the same microservice. This didn’t happen before. There’s no code change and the partition key remains consistent. Could this be related to how Kafka partitions ? Looking for insights on why this might happen and how to ensure event affinity within a pod.

I have checked our consumer and producer have same no. of pods.

2

hi @dev12111

it might be related yes default strategy would be round-robin.
but you’re using a message key right?

for ref:

best,
michael

3

Does not sound like a question about Kafka Streams?

In general, partitioning is done inside the producer, not the broker. So you should check if anything changed if your producer app.

(No idea what “Kafka (Retina)” is…)