Problem with JDBC Source Connector and a hyphen in the table name

Kafka Connect
1

I want to put Data from a MS-SQl Table with the jdbc source connector to kafka.
The data fromat should be avro. The problem ist my table have hyphen in the name.
The avro Convert try to register the schema and use the table name for the avor schema name, but it is not allowed to use a hypen at the avro schema name.

Table name: test-1
Avro:

 type:"record",
 name:"test-1" , < = not allowed
 fields: .....

It is not an option to change the table name.
Can someone help me?

Thx
Marcel

2

Use the RegExRouter Single Message Transform to change the topic name

e.g.

"transforms"                        : "dropHyphen",
"transforms.dropHyphen.type"        : "org.apache.kafka.connect.transforms.RegexRouter",
"transforms.dropHyphen.regex"       : "test-1",
"transforms.dropHyphen.replacement" : "test1"
3

The problem is in my opinion not the topic name.
The problem is the table name.
Your solution only remove the hyphen in the topic name,
but i already remove the table name in the topic name with this:

"topic": "test_topic-"
"transforms":"dropPrefix",			
    "transforms.dropPrefix.type":"org.apache.kafka.connect.transforms.RegexRouter",
    "transforms.dropPrefix.regex":"(.*)-(.*)",
    "transforms.dropPrefix.replacement":"$1"

Error Message:

org.apache.avro.SchemaParseException: Illegal character in: test-1
at org.apache.avro.Schema.validateName(Schema.java:1530)
at org.apache.avro.Schema.access$400(Schema.java:87)
at org.apache.avro.Schema$Name.<init>(Schema.java:673)
at org.apache.avro.Schema.createRecord(Schema.java:212)
at io.confluent.connect.avro.AvroData.fromConnectSchema(AvroData.java:867)
at io.confluent.connect.avro.AvroData.fromConnectSchema(AvroData.java:706)
at io.confluent.connect.avro.AvroData.fromConnectSchema(AvroData.java:700)
4

Hi @rmoff,

i find the correct solution for my problem.
I use this:

"transforms":"SetSchemaName",
"transforms.SetSchemaName.type":"org.apache.kafka.connect.transforms.SetSchemaMetadata$Value",
"transforms.SetSchemaName.schema.name":"test1"
5

Great - glad you got it sorted! :+1:

6

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.