[FEATURE] New feature request

[sorbra]

Use-cases

I am a DevOps engineer building Azure Databricks infrastructure with Terraform.
As I do not have account level admin permissions in the Databricks account, it seems that I must organize my Terraform code to reflect the following steps:

1. Create as much of the infrastructure I can, without account admin permssions. This includes Entra Id groups and service principals.
2. Request a Databricks account admin to create the account level resources, such as importing groups and service principals into the account.
3. Create the remaining databricks resources.

To be specific, the databricks steps that, afaik, I currently am unable to perform without account admin permissions are:

1. Import Entra Id groups and service principals in the account. This seems to be true although automatic identity management is enabled in the Databricks account.
2. Assign permissions on the imported groups and service principals, which is required for the devops engineer to assign workspace or unity catalog level permission to those groups.
3. Create an NCC and bind it to a workspace.

Attempted Solutions

I have tried to use the databricks_group and databricks_service_principal data sources, but neither seems to work until an account admin has explicitly imported the group/service principal into the account.

Proposal

From a DevOps engineer perspective, one way to handle groups and service principals would be to allow the databricks_group and databricks_service_principal data sources to reference the Entra Id groups using external_id. And, if those groups were not already imported into the account, they would be as a side effect of referencing them in the data source. This seems to be in the spirit of Azure Databricks "automatic identity management". Also, it would be a requirement for the user (devops engineer) calling the data source, to be able to see the groups and service principals, without explicitly having been assigned MANAGE permissions on those.

As for the NCC limitations, I can live with that.

References

[alexott]

Should be handled in #5068