Is there any way to get alert when Kafka Connector fails while loading data to BigQuery (Monitoring purpose)?

This is regarding Monitoring and Alerting Purpose. When Confluent Cloud writes data to BigQuery (GCP), using inbuilt Connector provided by platform. Is there any way i can configure emails in case of data write failure / any failure issue with connector.

Hi @rahulanand . This alerting does not exist in Confluent cloud at the moment but it is in the roadmap (though not very imminent).

I agree with @vnadkarni , but we have a work around to monitor connectors via restapi, we built signalfx plugin where it polls restapi every 30s to pull the data and post them in signalfx, thats how we get alerts if connector is down.

https://docs.confluent.io/cloud/current/api.html#operation/readConnectv1ConnectorStatus

1 Like