we had done a POC with Debezium + Postgres + CDC connector + Kstream lookup global Ktable from single table. my design question is, for each master table do we need one topic to store the cdc events from cdc connector for lookups. if the master table is increasing eg 100 tables then the topics (eg 100
topics ) are not manageable.
all cdc events from user table will be loaded in kafka compacted topic user
all cdc events from events table will be loaded in kafka compacted topic events
how can we use address this design issue ,by having limited topic or 1 topic for all master tables . kindly let us know your thoughts.