if we pass a JSON array into the topic KSQL DB will count it as a single entity and will not join the data using campaign_id as it is in the objects inside the array.
example of spend side data,
[{
“campaign_id”: “120206348641010775”,
“spend”: 0.06,
“adaccount_id”: “234085952918745”,
“created_time”: “2024-04-05 10:57:35”,
“time_stamp”: 1712832005
}]
- example of revenue side data,
[{
“campaign_id”: “120206348641010775”,
“user_revenue”: 0.73,
}]
The data that I’ll send though topic will be too large (over 30k rows every 10 minutes) and can not be sent as single object and must be sent as array of objects