From fa8ea7e195d77e674f84da54165fa813841e2879 Mon Sep 17 00:00:00 2001 From: vishal-av Date: Mon, 11 Aug 2025 12:29:19 +0530 Subject: [PATCH 1/2] [CDS-113003] Creating a cluster with Azure WIF --- .../clusters/create-cluster-with-azure-wif.md | 260 ++++++++++++++++++ .../clusters/create-cluster-with-iam.md | 2 +- 2 files changed, 261 insertions(+), 1 deletion(-) create mode 100644 docs/continuous-delivery/gitops/clusters/create-cluster-with-azure-wif.md diff --git a/docs/continuous-delivery/gitops/clusters/create-cluster-with-azure-wif.md b/docs/continuous-delivery/gitops/clusters/create-cluster-with-azure-wif.md new file mode 100644 index 00000000000..f347fba6691 --- /dev/null +++ b/docs/continuous-delivery/gitops/clusters/create-cluster-with-azure-wif.md @@ -0,0 +1,260 @@ +--- +title: Create a GitOps Cluster with Azure Workload Identity Federation (WIF) +description: Connect Harness GitOps (Argo CD) to Azure Kubernetes Service (AKS) using Workload Identity Federation for secure, keyless authentication. +sidebar_position: 2 +--- + +In this guide, you’ll set up Harness GitOps (powered by Argo CD) to connect securely to an Azure Kubernetes Service (AKS) cluster without storing kubeconfigs or static credentials. + + +## What is Azure WIF? + +Azure **Workload Identity Federation (WIF)** allows Kubernetes ServiceAccounts to securely access Azure resources **without storing credentials or secrets** in the cluster. +Instead, AKS issues short-lived tokens through its **OpenID Connect (OIDC) issuer**, which Azure validates before granting access to the linked Azure AD application. + +With WIF: +- No kubeconfigs or static service principal secrets are stored +- Authentication is short-lived and automatically rotated +- Access is scoped to a specific Kubernetes ServiceAccount + +**Learn more:** [Azure Workload Identity Federation documentation](https://learn.microsoft.com/en-us/azure/aks/workload-identity-overview) + +## Let's Now Set Up the GitOps Cluster with Azure WIF + +The following steps will guide you through configuring Azure Kubernetes Service (AKS) and Harness GitOps (Argo CD) to use Azure Workload Identity Federation. + +**By the end, you will:** +- Create or use an existing AKS cluster +- Enable OIDC + WIF on the cluster +- Create an Azure AD application and federated credential +- Grant RBAC permissions to the Azure app +- Set up an annotated ServiceAccount for GitOps +- Deploy the Harness GitOps Agent and connect the cluster + + +**Prerequisites:** +- Azure CLI and kubectl installed +- Azure subscription access (able to create AKS and assign RBAC) +- Harness GitOps module enabled (you’ll install a GitOps Agent) + +**Variables:** +```bash +RG="gitops-rg" +LOC="eastus" # any supported region +AKS="gitops-aks" +APP_NAME="argo-cd-app" # Azure AD application display name +SA_NS="argocd" # namespace for GitOps Agent / Argo CD +SA_NAME="argocd-manager" # ServiceAccount used for WIF +``` + +### 1. Create or select your AKS cluster +Create or Select an AKS Cluster +If you don’t already have an AKS cluster, you’ll create one. +If you already have one, you’ll just fetch its kubeconfig so you can interact with it. + +**Key points:** +- The resource group (RG) groups your AKS resources in Azure. +- The VM size and node count determine performance and cost — here we use a small example node pool. + +```bash +az group create -n "$RG" -l "$LOC" + +az aks create \ + -g "$RG" -n "$AKS" \ + --node-count 1 \ + --node-vm-size Standard_D4s_v6 \ + --enable-managed-identity \ + --generate-ssh-keys + +az aks get-credentials -g "$RG" -n "$AKS" --overwrite-existing +kubectl get nodes +``` + +If you already have AKS, just fetch credentials: + +```bash +az aks get-credentials -g "$RG" -n "$AKS" --overwrite-existing +``` + +### 2. Enable OIDC Issuer and Workload Identity +Workload Identity requires your AKS cluster to have: +- An OIDC issuer — a trusted identity provider URL that Azure uses to validate Kubernetes-issued tokens +- Workload Identity enabled — the mechanism that links Kubernetes ServiceAccounts to Azure AD applications. + +This step ensures your cluster can issue tokens Azure will trust. + +```bash +az aks update -g "$RG" -n "$AKS" \ + --enable-oidc-issuer \ + --enable-workload-identity + +OIDC_ISSUER_URL=$(az aks show -g "$RG" -n "$AKS" --query "oidcIssuerProfile.issuerUrl" -o tsv) +echo "OIDC_ISSUER_URL=$OIDC_ISSUER_URL" +``` + +### 3. Create an Azure AD Application and Service Principal +The Azure AD application represents your cluster’s identity in Azure AD. +The service principal is the **login** that AKS uses to act as that application. + +This identity will be linked to your Kubernetes ServiceAccount in later steps. + +```bash +AZURE_CLIENT_ID=$(az ad app create --display-name "$APP_NAME" --query appId -o tsv) +echo "AZURE_CLIENT_ID=$AZURE_CLIENT_ID" + +# idempotent: create SP if missing +az ad sp show --id "$AZURE_CLIENT_ID" >/dev/null 2>&1 || az ad sp create --id "$AZURE_CLIENT_ID" +``` + +(If reusing an app, set AZURE_CLIENT_ID=`` and skip the az ad app create.) + +### 4. Add a Federated Credential (WIF) +A federated credential tells Azure: +> **Tokens from this specific Kubernetes ServiceAccount in this cluster are trusted to act as this Azure AD application.** + + +This creates the trust link between: +- **Your AKS OIDC issuer** +- **Your Kubernetes ServiceAccount** +- **Your Azure AD application** + +You will define: +- **Issuer** — The AKS OIDC URL from Step 2. Azure uses this to validate tokens from your cluster. +- **Subject** — The Kubernetes ServiceAccount identity in `system:serviceaccount::` format. Must match the namespace and ServiceAccount you created. +- **Audience** — For Azure WIF, keep `api://AzureADTokenExchange`. This defines who the token is intended for. +- **azure.workload.identity/client-id** — The annotation you’ll add to your ServiceAccount later, pointing to the Azure AD application’s Client ID. + + +```bash +SUBJECT="system:serviceaccount:${SA_NS}:${SA_NAME}" + +# delete any previous cred named 'argo-cd' (safe if none exists) +az ad app federated-credential delete \ + --id "$AZURE_CLIENT_ID" \ + --federated-credential-id "argo-cd" 2>/dev/null || true + +# create federated credential +az ad app federated-credential create \ + --id "$AZURE_CLIENT_ID" \ + --parameters "{ + \"name\": \"argo-cd\", + \"issuer\": \"${OIDC_ISSUER_URL}\", + \"subject\": \"${SUBJECT}\", + \"description\": \"WIF for ArgoCD ServiceAccount\", + \"audiences\": [\"api://AzureADTokenExchange\"] + }" + +# verify +az ad app federated-credential list --id "$AZURE_CLIENT_ID" -o table +``` + +### 5. Grant RBAC Permissions on AKS +Now that your Azure app can authenticate, you must give it permission to perform operations in the cluster. + +We start with the `Azure Kubernetes Service RBAC Cluster Admin role` for simplicity. You can scope this down later. + +```bash +AKS_SCOPE=$(az aks show -g "$RG" -n "$AKS" --query id -o tsv) + +az role assignment create \ + --assignee "$AZURE_CLIENT_ID" \ + --role "Azure Kubernetes Service RBAC Cluster Admin" \ + --scope "$AKS_SCOPE" +``` + +You can scope this down later. + +### 6. Create the Namespace, ServiceAccount, and RBAC + +The Kubernetes ServiceAccount: +- Lives in the namespace you’ll run GitOps in (e.g., argocd) +- Is annotated with the Azure AD application’s Client ID +- Is bound to the necessary Kubernetes RBAC roles +This is the link between Kubernetes and Azure AD. + +```bash +kubectl create namespace "$SA_NS" 2>/dev/null || true + +cat < Date: Tue, 19 Aug 2025 14:27:11 +0530 Subject: [PATCH 2/2] Update create-cluster-with-azure-wif.md --- .../clusters/create-cluster-with-azure-wif.md | 487 +++++++++++------- 1 file changed, 314 insertions(+), 173 deletions(-) diff --git a/docs/continuous-delivery/gitops/clusters/create-cluster-with-azure-wif.md b/docs/continuous-delivery/gitops/clusters/create-cluster-with-azure-wif.md index f347fba6691..c03bb9efae1 100644 --- a/docs/continuous-delivery/gitops/clusters/create-cluster-with-azure-wif.md +++ b/docs/continuous-delivery/gitops/clusters/create-cluster-with-azure-wif.md @@ -4,8 +4,7 @@ description: Connect Harness GitOps (Argo CD) to Azure Kubernetes Service (AKS) sidebar_position: 2 --- -In this guide, you’ll set up Harness GitOps (powered by Argo CD) to connect securely to an Azure Kubernetes Service (AKS) cluster without storing kubeconfigs or static credentials. - +In this guide, you'll set up Harness GitOps (powered by Argo CD) to connect securely to an Azure Kubernetes Service (AKS) cluster without storing kubeconfigs or static credentials. ## What is Azure WIF? @@ -19,242 +18,384 @@ With WIF: **Learn more:** [Azure Workload Identity Federation documentation](https://learn.microsoft.com/en-us/azure/aks/workload-identity-overview) -## Let's Now Set Up the GitOps Cluster with Azure WIF +## Prerequisites -The following steps will guide you through configuring Azure Kubernetes Service (AKS) and Harness GitOps (Argo CD) to use Azure Workload Identity Federation. +Before setting up Azure WIF with GitOps, ensure you have: -**By the end, you will:** -- Create or use an existing AKS cluster -- Enable OIDC + WIF on the cluster -- Create an Azure AD application and federated credential -- Grant RBAC permissions to the Azure app -- Set up an annotated ServiceAccount for GitOps -- Deploy the Harness GitOps Agent and connect the cluster +- **Azure subscription** with appropriate permissions to: + - Create and manage Azure AD applications + - Create and manage Managed Identities + - Assign RBAC roles + - Manage AKS clusters +- **AKS cluster** with specific authentication and authorization configuration +- **Harness GitOps module** enabled in your account +:::important Authentication and Authorization Requirements +Your AKS cluster **must** be configured with one of these specific authentication and authorization combinations: +- **Microsoft Entra ID authentication with Azure RBAC**, OR +- **Microsoft Entra ID authentication with Kubernetes RBAC** -**Prerequisites:** -- Azure CLI and kubectl installed -- Azure subscription access (able to create AKS and assign RBAC) -- Harness GitOps module enabled (you’ll install a GitOps Agent) +Other authentication methods may cause connection failures. +::: -**Variables:** -```bash -RG="gitops-rg" -LOC="eastus" # any supported region -AKS="gitops-aks" -APP_NAME="argo-cd-app" # Azure AD application display name -SA_NS="argocd" # namespace for GitOps Agent / Argo CD -SA_NAME="argocd-manager" # ServiceAccount used for WIF -``` +## Overview of the Setup Process -### 1. Create or select your AKS cluster -Create or Select an AKS Cluster -If you don’t already have an AKS cluster, you’ll create one. -If you already have one, you’ll just fetch its kubeconfig so you can interact with it. +The Azure WIF setup involves these key steps: -**Key points:** -- The resource group (RG) groups your AKS resources in Azure. -- The VM size and node count determine performance and cost — here we use a small example node pool. +1. **Configure AKS Security** - Set up proper authentication and authorization +2. **Enable OIDC and Workload Identity** - Enable your AKS cluster to issue trusted tokens +3. **Create Managed Identity** - Create an Azure managed identity for your GitOps agent +4. **Download and Modify GitOps Agent YAML** - Download the agent YAML and add workload identity annotations +5. **Configure Federated Credentials** - Link your Kubernetes ServiceAccount to the managed identity +6. **Deploy GitOps Agent** - Install the modified agent +7. **Create Cluster Secret** - Configure the cluster connection with workload identity authentication +8. **Verify Setup** - Ensure the cluster appears healthy in Harness UI -```bash -az group create -n "$RG" -l "$LOC" +## Step 1: Configure AKS Security Settings -az aks create \ - -g "$RG" -n "$AKS" \ - --node-count 1 \ - --node-vm-size Standard_D4s_v6 \ - --enable-managed-identity \ - --generate-ssh-keys +Ensure your AKS cluster is configured with the correct authentication and authorization method. -az aks get-credentials -g "$RG" -n "$AKS" --overwrite-existing -kubectl get nodes -``` +### Required Security Configuration -If you already have AKS, just fetch credentials: +Your cluster must use one of these configurations: +**Option 1: Microsoft Entra ID authentication with Azure RBAC** ```bash -az aks get-credentials -g "$RG" -n "$AKS" --overwrite-existing +az aks update -g "$RESOURCE_GROUP" -n "$CLUSTER_NAME" \ + --enable-aad \ + --enable-azure-rbac ``` -### 2. Enable OIDC Issuer and Workload Identity -Workload Identity requires your AKS cluster to have: -- An OIDC issuer — a trusted identity provider URL that Azure uses to validate Kubernetes-issued tokens -- Workload Identity enabled — the mechanism that links Kubernetes ServiceAccounts to Azure AD applications. +**Option 2: Microsoft Entra ID authentication with Kubernetes RBAC** +```bash +az aks update -g "$RESOURCE_GROUP" -n "$CLUSTER_NAME" \ + --enable-aad +``` + +#### Using Azure Portal + +1. Navigate to the [Azure Portal](https://portal.azure.com) +2. Go to **Kubernetes services** and select your AKS cluster +3. In the left menu, click **Authentication and authorization** +4. Under **Authentication method**, select **Microsoft Entra ID** +5. Under **Authorization**, choose either: + - **Azure RBAC** (recommended for most scenarios) + - **Kubernetes RBAC** (if you prefer Kubernetes-native RBAC) +6. Click **Save** to apply the changes + +## Step 2: Enable OIDC Provider and Workload Identity -This step ensures your cluster can issue tokens Azure will trust. +Enable the OIDC issuer and Workload Identity on your AKS cluster: + +#### Using Azure CLI ```bash -az aks update -g "$RG" -n "$AKS" \ +az aks update -g "$RESOURCE_GROUP" -n "$CLUSTER_NAME" \ --enable-oidc-issuer \ --enable-workload-identity - -OIDC_ISSUER_URL=$(az aks show -g "$RG" -n "$AKS" --query "oidcIssuerProfile.issuerUrl" -o tsv) -echo "OIDC_ISSUER_URL=$OIDC_ISSUER_URL" ``` -### 3. Create an Azure AD Application and Service Principal -The Azure AD application represents your cluster’s identity in Azure AD. -The service principal is the **login** that AKS uses to act as that application. +#### Using Azure Portal -This identity will be linked to your Kubernetes ServiceAccount in later steps. +1. In your AKS cluster page, go to **Settings** → **Security** +2. Under **Workload Identity**, toggle **Enable workload identity** to **Enabled** +3. Under **OIDC Issuer**, toggle **Enable OIDC issuer** to **Enabled** +4. Click **Save** to apply the changes +5. After saving, note down the **OIDC Issuer URL** displayed - you'll need this for federated credentials -```bash -AZURE_CLIENT_ID=$(az ad app create --display-name "$APP_NAME" --query appId -o tsv) -echo "AZURE_CLIENT_ID=$AZURE_CLIENT_ID" +### Retrieve the OIDC Issuer URL -# idempotent: create SP if missing -az ad sp show --id "$AZURE_CLIENT_ID" >/dev/null 2>&1 || az ad sp create --id "$AZURE_CLIENT_ID" -``` +Get the OIDC issuer URL, which you'll need for configuring federated credentials: -(If reusing an app, set AZURE_CLIENT_ID=`` and skip the az ad app create.) +#### Using Azure CLI -### 4. Add a Federated Credential (WIF) -A federated credential tells Azure: -> **Tokens from this specific Kubernetes ServiceAccount in this cluster are trusted to act as this Azure AD application.** +```bash +OIDC_ISSUER_URL=$(az aks show -g "$RESOURCE_GROUP" -n "$CLUSTER_NAME" --query "oidcIssuerProfile.issuerUrl" -o tsv) +echo "OIDC Issuer URL: $OIDC_ISSUER_URL" +``` +#### Using Azure Portal -This creates the trust link between: -- **Your AKS OIDC issuer** -- **Your Kubernetes ServiceAccount** -- **Your Azure AD application** +1. In your AKS cluster page, go to **Settings** → **Security** +2. Under **OIDC Issuer**, copy the **Issuer URL** value +3. Save this URL - you'll need it when configuring federated credentials -You will define: -- **Issuer** — The AKS OIDC URL from Step 2. Azure uses this to validate tokens from your cluster. -- **Subject** — The Kubernetes ServiceAccount identity in `system:serviceaccount::` format. Must match the namespace and ServiceAccount you created. -- **Audience** — For Azure WIF, keep `api://AzureADTokenExchange`. This defines who the token is intended for. -- **azure.workload.identity/client-id** — The annotation you’ll add to your ServiceAccount later, pointing to the Azure AD application’s Client ID. +## Step 3: Create Managed Identity +Create an Azure managed identity that will be used by your GitOps agent: ```bash -SUBJECT="system:serviceaccount:${SA_NS}:${SA_NAME}" - -# delete any previous cred named 'argo-cd' (safe if none exists) -az ad app federated-credential delete \ - --id "$AZURE_CLIENT_ID" \ - --federated-credential-id "argo-cd" 2>/dev/null || true - -# create federated credential -az ad app federated-credential create \ - --id "$AZURE_CLIENT_ID" \ - --parameters "{ - \"name\": \"argo-cd\", - \"issuer\": \"${OIDC_ISSUER_URL}\", - \"subject\": \"${SUBJECT}\", - \"description\": \"WIF for ArgoCD ServiceAccount\", - \"audiences\": [\"api://AzureADTokenExchange\"] - }" - -# verify -az ad app federated-credential list --id "$AZURE_CLIENT_ID" -o table +IDENTITY_NAME="gitops-managed-identity" +az identity create --name "$IDENTITY_NAME" --resource-group "$RESOURCE_GROUP" ``` -### 5. Grant RBAC Permissions on AKS -Now that your Azure app can authenticate, you must give it permission to perform operations in the cluster. +#### Using Azure Portal + +1. In the Azure Portal, search for **Managed Identities** and select it +2. Click **+ Create** to create a new managed identity +3. Fill in the details: + - **Subscription**: Select your subscription + - **Resource group**: Select your resource group + - **Region**: Select the same region as your AKS cluster + - **Name**: Enter `gitops-managed-identity` (or your preferred name) +4. Click **Review + create**, then **Create** +5. Once created, click **Go to resource** + +### Get Client ID and Tenant ID -We start with the `Azure Kubernetes Service RBAC Cluster Admin role` for simplicity. You can scope this down later. +Retrieve the client ID and tenant ID that you'll need for the service account annotations: ```bash -AKS_SCOPE=$(az aks show -g "$RG" -n "$AKS" --query id -o tsv) +# Get Client ID from the managed identity +CLIENT_ID=$(az identity show --name "$IDENTITY_NAME" --resource-group "$RESOURCE_GROUP" --query clientId -o tsv) +echo "Client ID: $CLIENT_ID" -az role assignment create \ - --assignee "$AZURE_CLIENT_ID" \ - --role "Azure Kubernetes Service RBAC Cluster Admin" \ - --scope "$AKS_SCOPE" +# Get Tenant ID from your Azure subscription +TENANT_ID=$(az account show --query tenantId -o tsv) +echo "Tenant ID: $TENANT_ID" ``` -You can scope this down later. +**Reference Documentation:** +- [Get Client ID from Managed Identity](https://learn.microsoft.com/en-us/azure/aks/use-managed-identity) +- [Get Tenant ID](https://learn.microsoft.com/en-us/azure/azure-portal/get-subscription-tenant-id) -### 6. Create the Namespace, ServiceAccount, and RBAC +#### Getting Client ID and Tenant ID via Portal -The Kubernetes ServiceAccount: -- Lives in the namespace you’ll run GitOps in (e.g., argocd) -- Is annotated with the Azure AD application’s Client ID -- Is bound to the necessary Kubernetes RBAC roles -This is the link between Kubernetes and Azure AD. +**To get Client ID:** +1. In your managed identity page, go to **Overview** +2. Copy the **Client ID** value +3. Save this value - you'll need it for service account annotations -```bash -kubectl create namespace "$SA_NS" 2>/dev/null || true +**To get Tenant ID:** +1. In the Azure Portal, click on your profile icon (top right) +2. Click **Switch directory** or go to **Microsoft Entra ID** +3. In the **Overview** page, copy the **Tenant ID** value +4. Alternatively, you can find it in your subscription details + +## Step 4: Download and Modify GitOps Agent YAML -cat <