If I understand correctly, when you run worker.properties for the first time a Kafka Connect Cluster is instantiated and shares three topics with a Broker.
These three are named in the properties:
offset.storage.topic
config.storage.topic
status.storage.topic.
Well, if I then configure a Source Connector in the cluster without specifying any topic where the offsets are generated, offsets will be generated based on the offset.storage.topic property??
I figured out that I could create my own topics in the broker before running the Connector, and then use them in the configuration.
But if I wanted to create the topics directly by configuring the connector, which property should I use in the configuration JSON?
Thank you all!!
Storage topics are mandatory topics, created for serving different purposes. But when you mention creating topics on its own from the connector configuration file, those are the topics used to store actual events. Make sure your auto.topic.create.enable property is set true in cluster configuration. Connector will take care of creating marked topic in configuration file.
1 Like