Hi all!
Just to bring a view of my current (POC).
I have a Postgres Database with a table, which contains multiple columns.
One of these columns is a Jsonb type.
I have a Debezium connect to capture the data changes of the database to send all of them to a Kafka Topic.
I have been trying to configure the connector using multiple different plugins.
The idea is to expand the Jsonb column to have all the key-value data in the Kafka topic also with the data from the rest of the columns.
This is my current Debezium connector configuration:
{
“name”: “basket-connector”,
“config”: {
“connector.class”: “io.debezium.connector.postgresql.PostgresConnector”,
“tasks.max”: “1”,
“database.hostname”: “postgres”,
“database.port”: “5432”,
“database.user”: “postgres”,
“database.password”: “postgres”,
“database.dbname” : “basket”,
“topic.prefix”: “consultancy.dbz”,
“table.include.list”: “public.basket_entity”,
“key.converter”: “io.confluent.connect.avro.AvroConverter”,
“key.converter.schema.registry.url”: “http://schema-reg.biz”,
“value.converter”: “io.confluent.connect.avro.AvroConverter”,
“value.converter.schema.registry.url”: “http://schema-reg.biz”,
“internal.key.converter”:“org.apache.kafka.connect.json.JsonConverter”,
“internal.value.converter”:“org.apache.kafka.connect.json.JsonConverter”,
“internal.key.converter.schemas.enable”:“false”,
“internal.value.converter.schemas.enable”:“false”,
“transforms”: “expand”,
“transforms.expand.type”: “com.redhat.insights.expandjsonsmt.ExpandJSON$Value”,
“transforms.expand.sourceFields”: “basket”
}
}
I hope someone could help me with this to solve the issue.
Regards