Why are my SMT's ignored?

Kafka Connect Managed Connectors
1

Using ccloud, I can create my connector no problem, but no matter what smt I include in the connector config, I never get any of them to show in the topic messages. I am new to this so maybe I am missing something. Here is the config:

{
    "name": "rdb_connector",
    "validate.non.null": "false",
    "table.whitelist": "account, service_order, customer, account_address, order_, account_bag",
    "db.timezone": "US/Eastern",
    "kafka.api.key": "***********",
    "kafka.region": "us-east1",
    "output.data.format": "JSON",
    "numeric.mapping": "best_fit",   
    "table.types": "TABLE",
    "timestamp.column.name": "modified_at",
    "cloud.environment": "prod",
    "connection.host": "**********",
    "connection.password": "************",
    "kafka.endpoint": "*************",
    "ssl.mode": "prefer",
    "tasks.max": "1",
    "topic.prefix": "admin_",
    "cloud.provider": "gcp",
    "internal.kafka.endpoint": "********",
    "kafka.api.secret": "************",
    "incrementing.column.name": "id",
    "kafka.dedicated": "false",
    "connection.port": "5432",
    "connection.user": "************",
    "connector.class": "PostgresSource",
    "db.name": "admin",
    "transforms":"createKey",
    "transforms.createKey.type":"org.apache.kafka.connect.transforms.ValueToKey",
    "transforms.createKey.fields":"id",
    "transforms.InsertField.type": "org.apache.kafka.connect.transforms.InsertField$Value",
    "transforms.InsertField.static.field": "name",
    "transforms.InsertField.static.value": "rdb.connector"
}

Anything jump out to anyone? Need more info? Let me know if you can help.

2

Yes - Unfortunately Single Message Transform are not available Confluent Cloud yet.

For the moment you’d be recommended to either use ksqlDB in Confluent Cloud to perform the processing that you need, or if that’s not an option you’d need to run your own Kafka Connect worker in order to make use of Single Message Transform.

3

Thanks for the quick answer @rmoff . This is a bummer but I can work around it. Just makes the solution less elegant. Out of curiosity, is there somewhere I can go to see supported / unsupported features for confluent cloud? Or somewhere I could have gotten this answer without posting to the forum?

4

Hi @bakergh.

Out of curiosity, is there somewhere I can go to see supported / unsupported features for confluent cloud? Or somewhere I could have gotten this answer without posting to the forum?

Here are some pointers for current Confluent Cloud limitations:

Going through these myself, I could not find a mention of the lack of SMT support (Single Message Transforms) for connectors in Confluent Cloud. I’ll follow up with our team internally.

5

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.