Data broker using apache kafka recommended?

I have an architecture question.

I am building a website where people can sell data.

I want to make sure that the logic for this makes sense.

These are the main tables involved:

channel : a table that stores the type of data that you want to sell.

example data:

first name, last name, state, zip code, cigar smoker?, diabetic, male, female …

channel subscription: a table that stores who is interested in buying the data

This is my business logic:

  1. Client A creates the channel XYZ where he describes the data that will be sold through it.

  2. Client B subscribes to channel XYZ.

2.1) Client C subscribes to channel XYZ.

2.2) Client D subscribes to channel XYZ.

  1. Client A sends a ping to our system saying, ‘Hi, I have 30,000 data records I want to sell through channel XYZ. The lowest amount I am willing to sell the data for is $5. Does anyone want to buy this data?’

  2. Our system pings the users subscribe to channel XYZ saying. ‘hi, there are 30,000 records that match the criteria to channel XYZ, do you want to submit a bid?’

  3. The clients subscribed to channel XYZ submit their bid

  4. Our system looks at the submitted bids for channel XYZ, if it meets the minimum, the system:

  • sends a pingback to the seller ‘we have a buyer for your data’
  1. Our system listens for a request from the seller Once a valid API call is registered, the data is validated, saved, and send to the buyer


  • is apache kafka good for this?
  • is node.js good with this? ( i dont really understand if there are different flavors of apache kafka… one of them being with node.js, the other one with python)

Kafka Streams is a Java library for stream processing. Your question seems to be more generally about the architectural considerations when using Kafka, and it’s integration with Node.JS. Those two questions belong into Architecture and Design - Confluent Community and Clients - Confluent Community respectively.

I reposeted it under architecture. i tried to delete the original post but I am running into an issue