Can we use single topic for multiple Mysql tables in kafka source connector?


I have set of tables (around 70) and i have spun up a kafka connect in a docker container. I am using mysql jdbc connector for the link between mysql DB and kafka cluster. As per the confluent doc, top define any topic in configuration, we can use topic.profix which will used as perfix for the tables defined in table.whitelist


When i start the connector using API, it auto creates the 2 topics


Auto topic creation is disabled in client’s Kafka cluster and 70 topics against 70 tables, does it make sense? First, can we feed messages from multiple tables in to topic and get exported in their own tables from the sink connector end?

What is the recommendation?

@rmoff Can you please help me here to decide my strategy on topics?

from the post link, i understand, it will create topic or look for topics having topic.prefix-table1, topic.prefix-table2 and so on, which seems to 1-1 mapping or i can have option to have 1 common topic for multiple tables.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.