Multi-tenant architecture for Kafka and Clickhouse as data sink

Architecture and Design
1

Hello everyone,
I am trying to design a multi-tenant architecture for Kafka and Clickhouse.
I want to be able to read the data of each tenant and insert them in the appropriate database in Clickhouse. For Clickhouse, each tenant would have its own database.
Following are the scenarios that I already considered:

  • Have a dedicated partition for each tenant. This way, I would not mix the data of different tenants in the same partition. On the consumer side, I would simply read the data and insert it in the corresponding database/table in Clickhouse. But then I would have to spawn a new consumer instance and a new partition for each new tenant, does this solution scale well?
  • Have partitions that contain the data of different tenants at the same time. Since reading the events one by one and processing them is not a good thing to do, I would read the data in batches, but then, the consumer reading the data would have a batch containing mixed data of different tenants. I guess this is something that I can managed on the consumer side by writing my own implementation to split the “big” batch of data into multiple smaller batches, each one being associated to one tenant (with the ID for example). At this point, we would have the data of each tenant in its corresponding batch, the remaining task is to find a way to insert the data in the corresponding database. I don’t think it would be a good idea to let multiple database connections alive in one single consumer that would push the data to the right database depending on the Batch ID. I would also have to change the code every time I add a new tenant. I have also thought of pushing those smaller batches to an intermediate partition. Spawn a consumer for each tenant that would read the corresponding batch and insert it. Doing so, each consumer would have one database connection that belongs to a particular tenant. But I don’t really know where to go from here, also don’t know which solution makes most sense?

Please tell me if I should add some details to my post or if something is unclear.

Thanks for your time.