Confluent Kafka + Debezium + AWS IAM Authentication

I am trying to find a solution for integrating Kafka + Debezium Connector + AWS RDS with IAM Authentication. Is this supported by Confluent Kafka? I do not want to use AWS MSK Cluster

Hi @Ashish_Piraji

welcome :slight_smile:

if my understanding is correct you would like to transfer data from a kafka cluster
to an AWS RDS database, right?

Where should yor Kafka cluster run?
OnPrem or Cloud?

Is there already a cluster running?

best,
michael

Hi @mmuehlbeyer
Thanks for the quick turnaround.
I am performing a CDC operation using Debezium connector. We are using AWS RDS Aurora Database. Currently we are using Red HAT Openshit to deploy Kafka Cluster, Kafka Connect with Debezium connector. Debezium Connector is able to connect the AWS Aurora DB with Username & Password. But now I am trying integrate with AWS IAM Authentication and no luck on how to integrate. The solutions that I found are all related to AWS MSK Cluster and we do not want to use AWS Managed Kafka Service.

Hi @Ashish_Piraji

just to be sure:
Source is Aurora and target is Kafka right?

Best,
Michael

Hi @mmuehlbeyer
Yes Source is definitely AWS AuroraDB. Debezium Connector is monitoring the CDC event on configured source tables and for any update\insert\delete operation, messages should be published to Kafka topics

ok I see
did you check this blog post?

best,
michael

Hi @mmuehlbeyer Yes we have already implemented what is available on the blog that you provided. Sorry if I had missed mentioning that I’m able to connect Aurora DB with Username & Password. But our info-security team is not allowing to use Password. Instead use IAM Authentication mechanism for connecting to AWS RDS. This is where I am struggling to find a solution

ok I see
there is a discussion on debezium Jira
https://issues.redhat.com/projects/DBZ/issues/DBZ-3547?filter=allopenissues

might helpful

best,
michael

and a example about using aws secrets manager in conjunction with debezium