Can we use single topic for multiple Mysql tables in kafka source connector?

[neeraj]

HI,

I have set of tables (around 70) and i have spun up a kafka connect in a docker container. I am using mysql jdbc connector for the link between mysql DB and kafka cluster. As per the confluent doc, top define any topic in configuration, we can use topic.profix which will used as perfix for the tables defined in table.whitelist

Example:
“topic.prefix”=“test-”
“table.whitelist”=“abc,xyz”

When i start the connector using API, it auto creates the 2 topics

test-abc
test-xyz

Auto topic creation is disabled in client’s Kafka cluster and 70 topics against 70 tables, does it make sense? First, can we feed messages from multiple tables in to topic and get exported in their own tables from the sink connector end?

What is the recommendation?

[neeraj]

@rmoff Can you please help me here to decide my strategy on topics?

From multiple tables to single topic - JDBC source connector

For a source connector on the oracle database. I want to push data in increment mode from multiple tables into a single topic. But it seems table.whitelist and topic.prefix are tightly coupled. Connector would look for as many topics as many table.whitelist. Is there a way to achieve this?

from the post link, i understand, it will create topic or look for topics having topic.prefix-table1, topic.prefix-table2 and so on, which seems to 1-1 mapping or i can have option to have 1 common topic for multiple tables.