Snowflake source connector

How to get data from snowflake in kafka topic in confluent UI? Any idea?


afaik there is no Snowflake source connector at the moment.
The JDBC connector might be a solution to try.

out of curiosity:
why do you need to load the data from Snowflake to Kafka?

is it possible to load the data loaded to Snowflake also to Kafka ?


I’d need more information to get more specific but in a general sense, with SLAs measured in minutes/hours.

what I’ve seen done is use Snowflake’s COPY TO command Overview of Data Unloading — Snowflake Documentation to push the data to S3/google blob, then use the S3 source connector or just a process coordinator like airflow, to scrape S3 and push it to kafka. This isn’t a great solution and it’s not really streaming, it can also be costly on the snowflake resources.

Your better option is to fork the pipeline before it hits snowflake and create a standard ‘lambda architecture’. Which I think is what @mmuehlbeyer is saying.

1 Like