Issue with Create ACL for Service Account using REST APIs

[NancyGupta]

Hi Team,

I am exploring Confluent Cloud REST APIs and I need some info. My use case is as stated below:

• Create a Service Account
• Create an API key for that service account
• Apply ACL for that service account

I am able to create the Service Account and API key with the REST APIs but for creating ACL, I need principal, which has to be “principalType:principal”.

I have tried giving “ServiceAccount:service-account-id” where service-account-id is the id that I get in the response from creating the service account, but that does not work.

Could you please help with how can I create ACLs on the service account created? What should be send under principal?

As per my exploration, I need the user_id, and I need to send the principal as User:user_id, where user_id is the Account ID which is displayed in the portal under the API KEYS details. But how to get this Account Id? Any other API for it?

[mmuehlbeyer]

hey @NancyGupta

welcome to the forum

which resource type your service account should be able to work with?
There are resource specific api keys (Use API keys in Confluent Cloud | Confluent Documentation) as well as Cloud API Keys(Use API keys in Confluent Cloud | Confluent Documentation)

Regarding RBAC maybe this one is helpful:

Use role-based access control (RBAC) to manage accounts and access in...

Best,
Michael

[NancyGupta]

Hey @mmuehlbeyer

Thanks for the response.
so the service account should be able to work with the resource specific api keys.

My concern mostly is with the REST APIs, I need to apply acl after creating the service account and the resource specific api key. For applying acl, I need principal, and my question is how to get that principal from the API itself.

Thanks,
Nancy

[mmuehlbeyer]

@NancyGupta

For applying acl, I need principal, and my question is how to get that principal from the API itself.

good question
need to check by myself

[NancyGupta]

Hey @mmuehlbeyer , Let me give some more details:

For API key creation, I am using the below API where I am adding the service accountId as the owner id,
https://docs.confluent.io/cloud/current/api.html#operation/createIamV2ApiKey

For ACL, I am using Confluent Cloud API Reference Documentation

where I need the Principal, for example, the one in highlighted:

If I take this from the portal and give the principal as User:AccountId it works. But I need this AccountId from the api response.

[Akovacs]

Hi @mmuehlbeyer !

I’m facing the exact same issue. I’ve tried several ways (through cli and the rest api) of getting the Account Id apart from the UI but I only managed to receive the Resource Id every time.

[mmuehlbeyer]

hey @Akovacs and @NancyGupta

tested as well without success so far.
will check again tomorrow.

best,
michael

[Akovacs]

Any updates with this issue?

[mmuehlbeyer]

sorry wasn’t able to test properly

could you share the code/scripts you are using?

best,
michael

[Akovacs]

We have a custom script that creates the service account and then the api key and then sets some basic acls all with the Confluent CLI tool. The commands used:

confluent iam service-account create <accountName>  --description <accountDescription> -o json

confluent api-key create --environment <environmentId> --resource <resourceId> --service-account <serviceAccountId> --description <description> -o json

confluent kafka acl create --environment <environmentId> --cluster <clusterId> --allow --service-account <serviceAccountId> <operationFlags> --<action> <resource>

We save the response values to a database and later use the service account id returned from the create service account command for the rest api create acls endpoint:
https://docs.confluent.io/cloud/current/api.html#tag/ACL-(v3)

As mentioned before the problem is that using this value for principal in the body like: “principal: User:sa-1a2b3c” creates the acl but doesn’t grant access to the topic for the service account.

Setting the verbosity to trace in the cli tool for creating the acls indicates to me that the cli tool also uses the value listed as Account Id.

[mmuehlbeyer]

hi

understood

mmh what about the following:
confluent kafka acl create --environment env-xxx --cluster lkc-xxx --allow --service-account sa-xxx --operation DESCRIBE --topic '*'

best,
michael

[Akovacs]

ACL creation works as intended using the cli tool. Those can be seen in the UI in the service account’s api key page at the Access → Transactional ID / Topic / etc. tab.

As the last picture from my previous post shows the confluent kafka acl create… command also uses the Account Id (at the principal) when making the call to the rest api.

[asimboruwa]

Hi all,
I had faced the exact same challenge as Nancy. However, upon inspecting the API calls made from the Confluent’s web console in the browser, I found this API:

https://confluent.cloud/api/service_accounts

Pass on the cloud api key to this using basic auth. It will return the list of all service accounts containing the internal numeric ids. This numeric id gets displayed on the UI as “Account ID”. If you use it in the ACL principal it should work - it did for me. Good luck!

I have also submitted a ticket to confluent to get an official solution. It seems like a significant miss in their REST api.

Now, lets hope that they don’t disable this workaround. Cheers.

[visi0nary]

Hey,

good finding!

I have also stumbled upon this issue. Ironically as you already mentioned this renders the complete ACL-REST-API useless on cluster-level because there is no way to programmatically find out the ressource ID of an account.

Have you had any luck with the confluent ticket already? I do not want to use an unofficial API as it can be disabled anytime.

Thank you and best regards,
Jan

[asimboruwa]

Yes, Confluent team confirmed that it is the correct workaround. Their long term solution is to support ACL with the account resource id (User:sa-1a2b3c).

[visi0nary]

Okay, nice. Thank you for finding out! Do you have a link to this ticket or something that I can refer to? Thanks again

[Alayaright]

@NancyGupta

Thanks for the response.
so the service account should be able to work with the resource specific api keys.

My concern mostly is with the REST APIs, I need to apply acl after creating the service account and the resource specific api key. For applying acl, I need principal, and my question is how to get that principal from the API itself.
hdo box gas grill overhead clearance,

Thanks,
Nancy

interesting query:
to my own checking!

[joshrichards37]

Hi,

Is there any update on this? We are using the pulumi provider for confluent cloud and struggling to programatically create ACLs using the service account ID.

[joshrichards37]

For anyone using the pulumi provider for this: you just need to make sure you pass the cloudapikey and secret through to the provider as well as the kafka ones.

export const provider = apiKeySecretArn.apply((arn) => {
  return new Provider(environment, {
    cloudApiKey: apiKey,
    cloudApiSecret: apiSecret,
    kafkaApiKey: apiKeyId,
    kafkaRestEndpoint: restEndpoint,
    kafkaApiSecret: aws.secretsmanager
      .getSecretVersion({
        secretId: arn,
      })
      .then((secret) => secret.secretString),
  });
});```