Handling Images via Kafka Topics

Kafka Connect
1

Hi All,

Does anyone has experience on handling Images via Kafka Topic? Is it recommended to send images through Kafka topic? Can it be done through a connector?
I have a requirement to source images from Azure blob storage and keep it in a Topic. And then again send/sink the images to Azure blob storage. Here the images can be of any format(.jpg,.jpeg,.png more) and the size can also go upto 10MB.
Please share your insights/experience.

Thank You,
Sandeep

2

Hey,

Technically this may be possible but just keep in mind that you may have to modify some settings as by default a message size can be 1MB. There is a reason behind that as it is considered inefficient to pass big message size through Kafka.
What may be a better approach would be to send the name of the images or links to the images through Kafka which the consumer can use to pull from storage.

Thanks,
Joydeep