I have an Micro-services where we get the event messages from a Kafka messaging system. The event message contains the orders information, quantity, the mode of package and delivery location. At a predetermined time, these events are pushed to the Kafka stream. A special event full delivery event is issued to signify that all events have been sent from the producer. The MS on receiving the full delivery event stops processing the events going forward.
This worked fine until now as
- We have limited number of events around 150.
- Kafka and Micro-services were a single instance and was run on a single machine.
When we use multiple Kafka nodes, a huge amount of events (10,000, for instance), and numerous Micro-services. In those cases, I believe it will lead to error.
How do I avoid,
- What if full delivery event is picked first but there are still a lot of other events that need to be picked up?
What changes should I make to the system to prevent these scenarios?
The benefit which we are looking on explicitly adding an ending event is:
Let us suppose there are two delivery teams (Team1 and Team2). Team1 serves Postal Codes 10243 and 10245, while Team2 serves Postal Codes 10247 and 10249. Now, there is a Sony TV package that needs to be delivered at Postal-Code: 10243, so this falls under Team1’s jurisdiction. However, they only have this one order. T the neighbouring locations 10247 and 10249, which fall under Team2, have 30 orders to deliver. In such cases, the package delivery can be given to Team2. Saving money this way. There is a large minimum amount that must be paid for each Team in order for items to be delivered. These are business decisions over which we have no control.
To deal with such a scenario, we used the logic described above.