Skip to content

Add BYO CNI cluster automation to hack/aks/Makefile #3846

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

Copilot
Copy link
Contributor

@Copilot Copilot AI commented Jul 18, 2025

This PR adds comprehensive automation for creating BYO (Bring Your Own) CNI clusters with Azure CNS deployment to the existing hack/aks/Makefile, addressing the need for streamlined cluster setup workflows.

Overview

The implementation adds new make targets that orchestrate the complete 3-step workflow:

  1. Cluster Creation: Uses existing overlay-byocni-nokubeproxy-up target
  2. CNS Deployment: Leverages root makefile test-load target with CNS-specific parameters
  3. CNI Deployment: Deploys Cilium using manifests from test/integration/manifests/cilium/ with template substitution

New Make Targets

  • byocni-cluster-up: Main orchestrator that automates complete cluster setup
  • deploy-cns: Deploy CNS using sudo -E env "PATH=$PATH" make test-load CNS_ONLY=true ...
  • deploy-cilium: Deploy Cilium using manifests with envsubst template substitution
  • byocni-cluster-vars: Display all configuration variables with examples
  • validate-cni-type: Validate CNI type before cluster creation

Configuration Variables

All parameters are configurable with sensible defaults:

  • CNI_TYPE: cilium (default), extensible for future CNI types
  • CNS_VERSION: v1.5.38 (default)
  • AZURE_IPAM_VERSION: v0.3.0 (default)
  • CNS_IMAGE_REPO: MCR (default)
  • CILIUM_DIR: 1.14 (default), supports 1.12, 1.13, 1.14, 1.16, 1.17
  • CILIUM_VERSION_TAG: v1.14.8 (default)
  • CILIUM_IMAGE_REGISTRY: acnpublic.azurecr.io (default)
  • IPV6_HP_BPF_VERSION: v0.0.3 (default)
  • DUALSTACK: false (default), enables dual-stack Cilium configuration

Usage Examples

Basic usage:

make byocni-cluster-up CLUSTER=my-cluster SUB=<subscription-id>

Custom configuration:

make byocni-cluster-up \
    CLUSTER=my-cilium-cluster \
    SUB=<subscription-id> \
    CNS_VERSION=v1.6.0 \
    CILIUM_DIR=1.16 \
    CILIUM_VERSION_TAG=v1.16.5 \
    DUALSTACK=true

View configuration:

make byocni-cluster-vars

Key Features

  • ✅ Comprehensive error handling and validation
  • ✅ Support for all available Cilium versions (v1.12-v1.17)
  • ✅ Dual-stack configuration support
  • ✅ Extensible design for future CNI types
  • ✅ Template substitution using envsubst
  • ✅ Individual step execution capability
  • ✅ Input validation before cluster creation
  • ✅ Path validation for manifests
  • ✅ Detailed documentation and examples

Implementation Approach

This implementation follows the feedback to update the existing Makefile rather than creating a new script, providing:

  • Reusability: Leverages existing cluster creation targets
  • Configurability: All parameters are overrideable via environment variables
  • Extensibility: Design supports adding other CNI types in the future
  • Validation: Comprehensive error checking before expensive cluster operations
  • Documentation: Updated README with usage examples and configuration details

Files Modified

  • hack/aks/Makefile: Added new targets and configuration variables
  • hack/aks/README.md: Updated documentation with usage examples and configuration details

Fixes #3773.


💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.

@Copilot Copilot AI changed the title [WIP] Update the Makefile to the BYO cluster with CNS + CNI Add BYO CNI cluster automation to hack/aks/Makefile Jul 18, 2025
@Copilot Copilot AI requested a review from vipul-21 July 18, 2025 20:23
Copilot finished work on behalf of vipul-21 July 18, 2025 20:23
@vipul-21 vipul-21 added the ci Infra or tooling. label Jul 18, 2025
CNS_VERSION=v1.6.0 \
CILIUM_DIR=1.16 \
CILIUM_VERSION_TAG=v1.16.5 \
CILIUM_IMAGE_REGISTRY=mcr.microsoft.com/containernetworking
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add node vm sku as parameter

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added VM_SIZE as a configurable parameter for BYO CNI cluster automation. It's now documented in the configuration section and included in the customization examples. Commit 7269168


### Available Configuration

- `CNI_TYPE`: cilium (default) - Future CNI types can be added
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add option for azurecni. if azurecni specified, network-plugin should be azure and should remove network-dataplane cilium field

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added azurecni as a supported CNI_TYPE option. When azurecni is specified, the automation uses overlay-up target which creates clusters with --network-plugin azure (without the cilium dataplane). Updated validation logic and documentation with examples. Commit 7269168

- `CNI_TYPE`: cilium (default) - Future CNI types can be added
- `CNS_VERSION`: CNS version to deploy (default: v1.5.38)
- `AZURE_IPAM_VERSION`: Azure IPAM version (default: v0.3.0)
- `CNS_IMAGE_REPO`: CNS image repository - MCR or ACR (default: MCR)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

based on image repo, can cns/cilium image path be constructed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Enhanced documentation to clarify the relationship between image repositories and image paths. Added examples showing different registry configurations (MCR vs ACR vs custom) and improved the variable display to show how CNS_IMAGE_REPO affects CNS image paths while CILIUM_IMAGE_REGISTRY controls Cilium images. Commit 7269168

@Copilot Copilot AI requested a review from tamilmani1989 July 18, 2025 22:51
Copilot finished work on behalf of tamilmani1989 July 18, 2025 22:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci Infra or tooling.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Update the Makefile to the BYO cluster with CNS + CNI
3 participants