Azure Blob Storage Source : Cannot get the topic assignment quite right

Kafka Connect
1

Hello gurus,

I am trying to restore KAFKA messages backed up on AzBlobStorage back on KAFKA but under a new topic, which I have already created manually (with the same number of partitions and replications). The problem: I cannot get the connector to post messages on the new topic and instead it puts them back on the old one (which has the same name as the folder where the messages are stored. Here is my configuration:

name=src00002
connector.class=io.confluent.connect.azure.blob.storage.AzureBlobStorageSourceConnector
azblob.container.name=kafka
format.class=io.confluent.connect.cloud.storage.source.format.CloudStorageAvroFormat
topics.dir=test/
confluent.topic.bootstrap.servers=PLAINTEXT://kafka-broker:29092
tasks.max=1
azblob.account.key=
confluent.topic.replication.factor=1
azblob.account.name=
topic.regex.list=restored_grant:.*

Any ideas/thought would be greatly appreciated as I have been struggling with this way too long by now.
Peace,
Kanellos

2

Hi @Kanellos,

One thing jumps out at me: the topic.regex.list parameter only applies to the generalized mode of the connector. The default mode is RESTORE_BACKUP so try setting mode=GENERIC in your config.

HTH,
Dave

3

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.