Skip to content
BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
docs
docs
 
 
kind
kind
 
 
manifests
manifests
 
 
scripts
scripts
 
 
.gitignore
.gitignore
 
 
QUICKSTART.md
QUICKSTART.md
 
 
README.md
README.md
 
 

Repository files navigation

Crossplane Lab: AWS VPC Networking and Content Delivery with KIND

This lab demonstrates how to use Crossplane to manage AWS infrastructure declaratively using Kubernetes APIs. We'll use KIND (Kubernetes-in-Docker) to run a local Kubernetes cluster, while Crossplane manages real AWS resources in your cloud account.

⚠️ IMPORTANT: While the Kubernetes control plane runs locally via KIND, Crossplane will create real AWS resources (VPCs, subnets, NAT gateways, etc.) that incur real costs. Monitor your AWS billing and clean up resources after the lab!

Table of Contents

Prerequisites

Before starting this lab, ensure you have:

  • Docker installed and running (required for KIND)
  • KIND installed (Installation Guide)
  • kubectl installed and configured
  • Helm v3.x installed
  • Git for cloning the repository
  • AWS Account with appropriate permissions
  • AWS CLI configured with credentials (optional, for verification)
  • IAM User with permissions for:
    • EC2 (VPC, subnets, security groups, NAT gateways, etc.)
    • CloudFront (if using CloudFront)
    • S3 (if using VPC endpoints for S3)

AWS IAM Permissions

Your IAM user needs the following permissions (or use AmazonEC2FullAccess and CloudFrontFullAccess for simplicity):

  • ec2:* (VPC, subnets, security groups, NAT gateways, route tables, etc.)
  • cloudfront:* (if using CloudFront)
  • elasticloadbalancing:* (if needed)
  • iam:CreateServiceLinkedRole (for NAT Gateway)

Understanding the Architecture

┌─────────────────────────────────────────┐
│         Your Local Machine              │
│                                         │
│  ┌───────────────────────────────────┐  │
│  │     KIND Cluster (Docker)         │  │
│  │                                   │  │
│  │  ┌──────────────┐                │  │
│  │  │  Kubernetes  │                │  │
│  │  │  Control     │                │  │
│  │  │  Plane       │                │  │
│  │  └──────┬───────┘                │  │
│  │         │                         │  │
│  │  ┌──────▼───────┐                │  │
│  │  │ Crossplane   │                │  │
│  │  │ Controller   │                │  │
│  │  └──────┬───────┘                │  │
│  │         │                         │  │
│  │  ┌──────▼───────┐                │  │
│  │  │ AWS Provider │                │  │
│  │  │ Controller   │                │  │
│  │  └──────┬───────┘                │  │
│  └─────────┼─────────────────────────┘  │
└────────────┼─────────────────────────────┘
             │ HTTPS API Calls
             │ (AWS Credentials)
             ▼
┌─────────────────────────────────────────┐
│         AWS Cloud (us-east-1)           │
│                                         │
│  ┌──────────┐  ┌──────────┐           │
│  │   VPC    │  │ CloudFront│           │
│  │ Subnets  │  │ Distribution│         │
│  │   NAT    │  │           │           │
│  │ Security │  └──────────┘           │
│  │  Groups  │                          │
│  └──────────┘                          │
│                                         │
│    REAL AWS RESOURCES = REAL COSTS      │
└─────────────────────────────────────────┘

Key Points:

  • KIND cluster runs locally in Docker containers
  • Crossplane runs inside the KIND cluster
  • Crossplane makes real API calls to AWS
  • Resources created are real and billable
  • Use AWS Free Tier where possible, but NAT Gateways are not free

Repository Structure

crossplane-aws-vpc-lab/
├── README.md                    # This file
├── docs/
│   ├── aws-concepts.md         # AWS networking concepts
│   └── crossplane-fundamentals.md  # Crossplane concepts
├── kind/
│   └── kind-config.yaml        # KIND cluster configuration
├── manifests/
│   ├── provider-aws.yaml       # AWS provider installation
│   ├── provider-config.yaml    # AWS provider configuration
│   ├── vpc.yaml                # VPC resource
│   ├── subnet.yaml             # Subnet resources
│   ├── elastic-ip.yaml         # Elastic IP for NAT
│   ├── internet-gateway.yaml   # Internet Gateway
│   ├── nat-gateway.yaml        # NAT Gateway
│   ├── security-group.yaml     # Security Groups
│   ├── nacl.yaml               # Network ACLs
│   ├── route-table.yaml        # Route tables and routes
│   ├── vpc-endpoint.yaml       # VPC Endpoint (S3)
│   ├── cloudfront.yaml         # CloudFront distribution
│   └── composition/
│       ├── xrd.yaml            # Composite Resource Definition
│       └── composition.yaml    # Composition template
└── scripts/
    └── setup.sh                # Automated setup script

Lab Exercises

Follow these exercises in sequence. Each exercise builds on the previous one.

Exercise 0: Set Up KIND Cluster

Objective: Create a local Kubernetes cluster using KIND.

Manual Steps

  1. Create the KIND cluster:

    kind create cluster --config kind/kind-config.yaml --name crossplane-lab

  2. Verify kubectl context:

    kubectl cluster-info --context kind-crossplane-lab

  3. Check cluster nodes:

    kubectl get nodes

    You should see one control-plane node running.

Automated Setup

Alternatively, use the setup script:

chmod +x scripts/setup.sh
./scripts/setup.sh

This script will:

  • Check prerequisites
  • Create the KIND cluster
  • Install Crossplane via Helm
  • Verify the installation

Discussion Points:

  • KIND creates a Kubernetes cluster using Docker containers
  • The control plane runs in a Docker container
  • This cluster is completely local but uses standard Kubernetes APIs

Exercise 1: Install Crossplane

Objective: Install Crossplane in the KIND cluster.

If you used the automated setup script, Crossplane is already installed. Otherwise:

  1. Add Crossplane Helm repository:

    helm repo add crossplane-stable https://charts.crossplane.io/stable
helm repo update

  2. Install Crossplane:

    helm install crossplane \
  --namespace crossplane-system \
  --create-namespace \
  crossplane-stable/crossplane \
  --wait

  3. Verify installation:

    kubectl get pods -n crossplane-system

    You should see Crossplane pods running. Wait until all pods are READY.

  4. Check Crossplane components:

    kubectl get all -n crossplane-system

Expected Output:

NAME                                          READY   STATUS    RESTARTS   AGE
pod/crossplane-7d8b9c5c6f-abc123              1/1     Running   0          2m
pod/crossplane-rbac-manager-...               1/1     Running   0          2m

Discussion Points:

  • Crossplane runs as Kubernetes controllers
  • It watches for Custom Resources (CRDs) that represent cloud resources
  • The reconciliation loop continuously ensures desired state matches actual state

Exercise 2: Install AWS Provider

Objective: Install the AWS provider for Crossplane to manage AWS resources.

  1. Check available provider packages: Visit Upbound Marketplace to see the latest versions.

  2. Review the provider manifest:

    cat manifests/provider-aws.yaml

    This installs two providers:

    • provider-aws-ec2: For VPC, subnets, security groups, NAT gateways, etc.
    • provider-aws-cloudfront: For CloudFront distributions

  3. Install the providers:

    kubectl apply -f manifests/provider-aws.yaml

  4. Wait for providers to be ready:

    kubectl get providers

    Watch until both show INSTALLED and HEALTHY:

    kubectl wait --for=condition=Healthy provider/provider-aws-ec2 --timeout=300s
kubectl wait --for=condition=Healthy provider/provider-aws-cloudfront --timeout=300s

  5. Verify provider pods:

    kubectl get pods -n crossplane-system | grep provider

  6. Check available CRDs:

    kubectl get crds | grep aws.upbound.io

    You should see CRDs like:

    • vpcs.ec2.aws.upbound.io
    • subnets.ec2.aws.upbound.io
    • securitygroups.ec2.aws.upbound.io
    • distributions.cloudfront.aws.upbound.io
    • etc.

Discussion Points:

  • Providers install CRDs that extend Kubernetes API
  • Each provider handles a subset of AWS services
  • Family providers (like provider-aws-ec2) are modular and focused

Exercise 3: Configure AWS Provider

Objective: Configure AWS credentials so Crossplane can manage AWS resources.

  1. Create AWS credentials secret:

    kubectl create secret generic aws-credentials \
  --namespace crossplane-system \
  --from-literal=aws_access_key_id=YOUR_ACCESS_KEY \
  --from-literal=aws_secret_access_key=YOUR_SECRET_KEY

    Replace YOUR_ACCESS_KEY and YOUR_SECRET_KEY with your actual AWS credentials.

    Alternative: If you have AWS CLI configured, you can extract credentials:

    # Get your access key ID
aws configure get aws_access_key_id

# Get your secret access key
aws configure get aws_secret_access_key

  2. Create AWS credentials file format: For the provider config, we need credentials in AWS config file format. Update the secret:

    # Create a temporary credentials file
cat > /tmp/aws-credentials <<EOF
[default]
aws_access_key_id = YOUR_ACCESS_KEY
aws_secret_access_key = YOUR_SECRET_KEY
EOF

# Create secret from file
kubectl create secret generic aws-credentials \
  --namespace crossplane-system \
  --from-file=credentials=/tmp/aws-credentials

# Clean up
rm /tmp/aws-credentials

    Or if the secret already exists, delete and recreate it.

  3. Review provider configuration:

    cat manifests/provider-config.yaml

  4. Apply provider configuration:

    kubectl apply -f manifests/provider-config.yaml

  5. Verify provider config:

    kubectl get providerconfig

    You should see default provider config.

Security Best Practices:

  • Never commit AWS credentials to Git
  • Use IAM roles in production (not access keys)
  • Rotate credentials regularly
  • Use least-privilege IAM policies

Discussion Points:

  • ProviderConfig tells providers how to authenticate
  • Secrets store credentials securely in Kubernetes
  • One ProviderConfig can be used by multiple resources

Exercise 4: Create VPC and Subnets

Objective: Create a VPC and subnets using Crossplane.

4.1: Create VPC

  1. Review VPC manifest:

    cat manifests/vpc.yaml

    Key fields:

    • cidrBlock: IP address range for the VPC
    • enableDnsHostnames and enableDnsSupport: DNS configuration
    • providerConfigRef: References the provider config from Exercise 3

  2. Apply VPC manifest:

    kubectl apply -f manifests/vpc.yaml

  3. Monitor VPC creation:

    kubectl get vpc lab-vpc -w

    Watch for SYNCED and READY to become True:

    NAME      SYNCED   READY   EXTERNAL-NAME   AGE
lab-vpc   True     True    vpc-xxxxx       1m

  4. Get VPC details:

    kubectl describe vpc lab-vpc

    Note the EXTERNAL-NAME (the actual AWS VPC ID).

  5. Verify in AWS (optional):

    aws ec2 describe-vpcs --filters "Name=tag:Name,Values=lab-vpc"

4.2: Create Subnets

  1. Review subnet manifest:

    cat manifests/subnet.yaml

    This creates two subnets:

    • Public subnet: 10.0.1.0/24 with mapPublicIpOnLaunch: true
    • Private subnet: 10.0.2.0/24 with mapPublicIpOnLaunch: false

    Notice the vpcIdRef field that references the VPC created earlier.

  2. Apply subnet manifest:

    kubectl apply -f manifests/subnet.yaml

  3. Monitor subnet creation:

    kubectl get subnet -w

  4. Check subnet details:

    kubectl describe subnet lab-public-subnet
kubectl describe subnet lab-private-subnet

Discussion Points:

  • vpcIdRef creates a dependency - Crossplane waits for VPC to be ready
  • Subnets inherit VPC settings
  • Public vs. private subnets differ by IP mapping and routing

Exercise 5: Add Internet Gateway and NAT

Objective: Enable internet connectivity for public and private subnets.

5.1: Create Elastic IP for NAT Gateway

  1. Apply Elastic IP manifest:

    kubectl apply -f manifests/elastic-ip.yaml

  2. Wait for EIP to be ready:

    kubectl get eip lab-eip -w

5.2: Create Internet Gateway

  1. Review Internet Gateway manifest:

    cat manifests/internet-gateway.yaml

  2. Apply Internet Gateway:

    kubectl apply -f manifests/internet-gateway.yaml

  3. Monitor creation:

    kubectl get internetgateway lab-igw -w

5.3: Create NAT Gateway

  1. Review NAT Gateway manifest:

    cat manifests/nat-gateway.yaml

    Notice:

    • subnetIdRef: References the public subnet (NAT must be in public subnet)
    • allocationIdRef: References the Elastic IP

  2. Apply NAT Gateway:

    kubectl apply -f manifests/nat-gateway.yaml

  3. Monitor creation (this takes a few minutes):

    kubectl get natgateway lab-nat -w

    ⚠️ Note: NAT Gateways incur hourly charges (~$0.045/hour) plus data transfer costs.

5.4: Configure Route Tables

  1. Review route table manifest:

    cat manifests/route-table.yaml

    This creates:

    • Public route table with route to Internet Gateway
    • Private route table with route to NAT Gateway
    • Associations between subnets and route tables

  2. Apply route tables:

    kubectl apply -f manifests/route-table.yaml

  3. Verify routes:

    kubectl get routetable
kubectl get route
kubectl get routetableassociation

Discussion Points:

  • Internet Gateway enables internet access for public subnets
  • NAT Gateway allows private subnets outbound internet access
  • Route tables control traffic direction
  • NAT Gateway is a managed service (not free tier)

Exercise 6: Configure Security Groups and NACLs

Objective: Set up firewall rules for network security.

6.1: Create Security Groups

  1. Review security group manifest:

    cat manifests/security-group.yaml

    This creates two security groups:

    • lab-web-sg: Allows HTTP/HTTPS from internet
    • lab-db-sg: Allows MySQL (3306) only from web security group

  2. Apply security groups:

    kubectl apply -f manifests/security-group.yaml

  3. Monitor creation:

    kubectl get securitygroup

  4. Check security group rules:

    kubectl describe securitygroup lab-web-sg
kubectl describe securitygroup lab-db-sg

    Notice how lab-db-sg references lab-web-sg using securityGroupIdsRefs.

6.2: Configure Network ACLs

  1. Review NACL manifest:

    cat manifests/nacl.yaml

    This modifies the default NACL (allows all traffic in both directions).

  2. Apply NACL:

    kubectl apply -f manifests/nacl.yaml

Discussion Points:

  • Security groups are stateful and instance-level
  • NACLs are stateless and subnet-level
  • Security groups can reference other security groups
  • See PDF slides Image ID 5-7, 10 for visual comparisons

Exercise 7: VPC Endpoint and Routing

Objective: Create a VPC endpoint for private S3 access.

  1. Review VPC endpoint manifest:

    cat manifests/vpc-endpoint.yaml

    This creates an S3 Gateway Endpoint:

    • Type: Gateway (free, only for S3 and DynamoDB)
    • Associated with both route tables
    • Allows private access to S3 without internet

  2. Apply VPC endpoint:

    kubectl apply -f manifests/vpc-endpoint.yaml

  3. Monitor creation:

    kubectl get vpcendpoint lab-s3-endpoint -w

  4. Verify endpoint:

    kubectl describe vpcendpoint lab-s3-endpoint

Discussion Points:

  • VPC endpoints keep traffic within AWS network
  • Gateway endpoints are free (S3, DynamoDB only)
  • Interface endpoints use PrivateLink (charges apply)
  • Reduces NAT Gateway data transfer costs for S3 access

Exercise 8: CloudFront Distribution

Objective: Create a CloudFront CDN distribution.

⚠️ Prerequisites: You need an S3 bucket and optionally an Origin Access Identity (OAI) or Origin Access Control (OAC).

  1. Update CloudFront manifest: Edit manifests/cloudfront.yaml:

    • Replace mybucket.s3.amazonaws.com with your S3 bucket domain
    • Replace origin-access-identity/cloudfront/ABCDEFG1234567 with your OAI/OAC

  2. Review CloudFront manifest:

    cat manifests/cloudfront.yaml

  3. Apply CloudFront distribution:

    kubectl apply -f manifests/cloudfront.yaml

  4. Monitor creation (this takes several minutes):

    kubectl get distribution lab-cloudfront -w

  5. Get distribution details:

    kubectl describe distribution lab-cloudfront

    Note the domainName in the status - this is your CloudFront URL.

Discussion Points:

  • CloudFront is a global CDN
  • Reduces latency by caching at edge locations
  • Can use S3, ALB, or custom origins
  • Supports custom SSL certificates

Exercise 9: Advanced - Compositions

Objective: Create a reusable composition that packages multiple resources.

9.1: Install Composition Resources

  1. Apply Composite Resource Definition (XRD):

    kubectl apply -f manifests/composition/xrd.yaml

    This defines a new VPCNetwork resource type.

  2. Verify XRD:

    kubectl get xrd

  3. Apply Composition:

    kubectl apply -f manifests/composition/composition.yaml

    This defines what resources are created when a VPCNetwork is created.

  4. Verify Composition:

    kubectl get composition

9.2: Create a VPCNetwork Instance

  1. Create a VPCNetwork resource:

    cat <<EOF | kubectl apply -f -
apiVersion: aws.example.org/v1alpha1
kind: VPCNetwork
metadata:
  name: my-vpc-network
spec:
  parameters:
    region: us-east-1
    cidrBlock: "10.1.0.0/16"
    publicSubnetCidr: "10.1.1.0/24"
    privateSubnetCidr: "10.1.2.0/24"
    availabilityZone: us-east-1a
EOF

  2. Monitor composition:

    kubectl get vpcnetwork my-vpc-network -w
kubectl get managed

    You should see all the composed resources being created (VPC, subnets, IGW, NAT, routes, etc.).

  3. Check individual resources:

    kubectl get vpc,subnet,internetgateway,natgateway

Discussion Points:

  • Compositions create higher-level abstractions
  • Users don't need to know about individual resources
  • Enforces consistent patterns
  • Reusable across teams

Cleanup

⚠️ IMPORTANT: Clean up AWS resources to avoid ongoing charges!

Delete Resources

  1. Delete all manifests (in reverse dependency order):

    # Delete CloudFront first (takes time to delete)
kubectl delete -f manifests/cloudfront.yaml

# Delete VPC endpoint
kubectl delete -f manifests/vpc-endpoint.yaml

# Delete route tables and associations
kubectl delete -f manifests/route-table.yaml

# Delete NAT Gateway (charges stop when deleted)
kubectl delete -f manifests/nat-gateway.yaml

# Delete security groups and NACLs
kubectl delete -f manifests/security-group.yaml
kubectl delete -f manifests/nacl.yaml

# Delete Internet Gateway
kubectl delete -f manifests/internet-gateway.yaml

# Delete Elastic IP
kubectl delete -f manifests/elastic-ip.yaml

# Delete subnets
kubectl delete -f manifests/subnet.yaml

# Delete VPC (deletes default route table, default NACL, etc.)
kubectl delete -f manifests/vpc.yaml

# Delete compositions
kubectl delete -f manifests/composition/

# Delete provider config
kubectl delete -f manifests/provider-config.yaml

# Delete providers
kubectl delete -f manifests/provider-aws.yaml

# Delete secrets
kubectl delete secret aws-credentials -n crossplane-system

    Or delete all at once (may require retries due to dependencies):

    kubectl delete -f manifests/ --recursive

  2. Verify resources are deleted:

    kubectl get managed

    Should return empty or only resources being deleted.

  3. Check AWS console to ensure resources are deleted (especially NAT Gateways and Elastic IPs).

Delete KIND Cluster

kind delete cluster --name crossplane-lab

Uninstall Crossplane (Optional)

helm uninstall crossplane -n crossplane-system

Troubleshooting

Provider Not Ready

Symptoms: Provider shows INSTALLING or UNHEALTHY

Solutions:

# Check provider logs
kubectl logs -n crossplane-system -l pkg.crossplane.io/provider=provider-aws-ec2

# Check provider status
kubectl describe provider provider-aws-ec2

# Reinstall provider
kubectl delete -f manifests/provider-aws.yaml
kubectl apply -f manifests/provider-aws.yaml

Resource Not Syncing

Symptoms: Resource shows SYNCED=False or stays in SYNCING

Solutions:

# Check resource status
kubectl describe vpc lab-vpc

# Check events
kubectl get events --sort-by='.lastTimestamp'

# Check provider logs
kubectl logs -n crossplane-system -l pkg.crossplane.io/provider=provider-aws-ec2 --tail=100

AWS Authentication Errors

Symptoms: InvalidClientTokenId or UnauthorizedOperation

Solutions:

  • Verify AWS credentials secret exists: 
    kubectl get secret aws-credentials -n crossplane-system
  • Check credentials format in secret
  • Verify IAM user has required permissions
  • Test credentials with AWS CLI: 
    aws sts get-caller-identity

Resource Dependencies

Symptoms: Resource stuck waiting for dependency

Solutions:

  • Check if referenced resource exists and is ready: 
    kubectl get vpc lab-vpc
  • Verify *Ref fields point to correct resource names
  • Check resource names match exactly (case-sensitive)

NAT Gateway Creation Slow

Symptoms: NAT Gateway takes 5+ minutes to create

This is normal - NAT Gateway provisioning takes time. Monitor with:

kubectl get natgateway lab-nat -w

Additional Resources

Documentation

Learning Resources

  • See docs/aws-concepts.md for detailed AWS networking concepts
  • See docs/crossplane-fundamentals.md for Crossplane concepts
  • Crossplane Examples
  • Upbound Blog

PDF References

This lab references PDF slides with images:

  • Image ID 5: Security Group hierarchy
  • Image ID 6: Security Group rules
  • Image ID 7: NACL diagram
  • Image ID 10: Security Group vs. NACL comparison

License

This lab is provided for educational purposes. Use at your own risk regarding AWS costs.

Contributing

Found an issue or want to improve the lab? Please open an issue or submit a pull request!

Happy Learning! 🚀

Remember: This lab creates real AWS resources. Always clean up after completing the exercises to avoid unexpected charges.

About

AWS Networking Stack - Provisioned by Crossplane

Topics

awscrossplanegitopsinfrastructure-as-codekubernetes

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

1 fork
Report repository

Releases

Packages

Contributors

Languages