Evening all,
We’ve attempted to install the datadog connector on a couple of our clusters. One of the connectors failed to install thus it was removed. We noticed later on that although the connector failed to install it did in fact create a topic alongside a consumer group. Removing the connector did not remove the underlying resources created by the connector.
I’ve attempted to remove the consumer group manually via using the kafka-consumer-groups.sh tool by running:
./kafka-consumer-groups --command-config cloud.properties --bootstrap-server pkc-l6wr6.europe-west2.gcp.confluent.cloud:9092 --delete --group <group_name>
Group <group name> could not be deleted due to: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.GroupNotEmptyException: The group is not empty.
It seems that in order to remove the group member we need to set the member in a inactive state where as the current state of the group member is stable.
From what I understand the only way to set the member to an inactive state is to stop the application connecting our kafka cluster. Since this was the datadog managed connector we do not have access to the application or the underlying host.
Are there any potential solutions other than contacting Kafka support?
Thanks!