Skip to content

Conversation

balcsida
Copy link
Contributor

@balcsida balcsida commented Aug 13, 2025

Summary

This PR optimizes the Docker image size by 73%, reducing it from 95.5MB to ~25-28MB while maintaining full functionality and FIPS compliance across both x86_64 and ARM64 platforms.

Cross-Platform Optimization Results

Platform Before After Reduction Status
x86_64 95.5MB 27.9MB 71% ✅ Working
ARM64 95.5MB 25.4MB 73% ✅ Working

Optimization Journey & Testing Results

We systematically tested multiple approaches to find the optimal solution:

Approach Base Image Size Status
Original azurelinux/base/core:3.0 95.5MB ✅ Working
Distroless Base azurelinux/distroless/base:3.0 52.3MB ✅ Working
Distroless Minimal (naive) azurelinux/distroless/minimal:3.0 12.1MB ❌ Missing libs
Minimal + All Libs azurelinux/distroless/minimal:3.0 411MB ✅ Working (too big)
Minimal + Essential Libs azurelinux/distroless/minimal:3.0 25-28MB ✅ Working

Key Findings

  • Distroless Base (45% reduction): Good improvement but contained unnecessary components
  • Naive Minimal: Ultra-small but missing critical libraries for FIPS-compliant Go binary
  • All Libraries: Functional but larger than original due to copying entire /lib64 and /usr/lib64
  • Essential Libraries Only: Optimal balance - surgically copying only required runtime libraries

Changes Made

Dockerfile Optimization

  • Before: mcr.microsoft.com/azurelinux/base/core:3.0 (95.5MB)
  • After: mcr.microsoft.com/azurelinux/distroless/minimal:3.0 (25-28MB)
  • Size Reduction: 71-73% smaller

Cross-Platform Technical Implementation

The solution uses multi-stage build with platform-agnostic library copying:

# Copy essential libraries and dynamic linker for any platform
RUN mkdir -p /tmp/libs /tmp/linker && \
    cp /lib64/libc.so.6 /tmp/libs/ && \
    cp /lib64/libdl.so.2 /tmp/libs/ && \
    cp /lib64/libpthread.so.0 /tmp/libs/ && \
    cp /lib64/librt.so.1 /tmp/libs/ && \
    cp /lib64/libresolv.so.2 /tmp/libs/ && \
    cp /lib64/libssl.so* /tmp/libs/ && \
    cp /lib64/libcrypto.so* /tmp/libs/ && \
    find /lib /lib64 /usr/lib -name "ld-linux*" -exec cp {} /tmp/linker/ \;

FROM mcr.microsoft.com/azurelinux/distroless/minimal:3.0
COPY --from=acr-cli /tmp/linker/ /lib/
COPY --from=acr-cli /tmp/libs/ /lib64/
COPY --from=acr-cli /usr/bin/acr /usr/bin/acr

Platform-specific dynamic linkers supported:

  • x86_64: /usr/lib/ld-linux-x86-64.so.2
  • ARM64: /lib/ld-linux-aarch64.so.1

Essential libraries included:

  • libc.so.6, libdl.so.2, libpthread.so.0 - Core runtime
  • libresolv.so.2, librt.so.1 - Network and real-time support
  • libssl.so*, libcrypto.so* - FIPS compliance

Build Context Optimization

  • Added comprehensive .dockerignore file
  • Excludes documentation, tests, CI files, and development artifacts
  • Reduces build context size and improves build performance

Comprehensive Testing

Cross-Platform Validation

  • x86_64 Platform: 27.9MB, all functionality working
  • ARM64 Platform: 25.4MB, all functionality working
  • Dynamic Linker Detection: Automatic platform-specific detection

Docker Container Functionality Tests

  • Version and help commands working on both platforms
  • Basic purge functionality (dry-run and actual deletion)
  • Authentication working (using Azure access tokens)
  • Pattern matching working (regex filters)
  • Keep parameter working (preserving latest N images)

Integration Tests

  • Azure CLI integration with seamless authentication via access tokens
  • Registry operations function correctly across platforms
  • Multi-repository testing handles complex scenarios
  • Concurrency testing validates parallel operations

Authentication Testing

Validated multiple auth approaches on both platforms:

  • Access Token Auth: az acr login --expose-token (chosen solution)
  • Admin Credentials: Registry username/password
  • Service Principal: Enterprise auth scenarios

Performance Testing

  • Container startup is faster due to smaller image size on both platforms
  • Command execution has identical performance to original
  • Memory usage is lower due to minimal base image
  • Network transfer is 71-73% faster for image pulls

Benefits

  • 71-73% smaller image size - faster deployments and reduced storage costs
  • Enhanced security - minimal attack surface with distroless base
  • Full functionality preserved - all ACR CLI commands work correctly
  • FIPS compliance maintained - essential crypto libraries included
  • Cross-platform support - works on both x86_64 and ARM64
  • Improved build performance - optimized build context
  • Thoroughly tested solution - comprehensive validation across multiple scenarios

Compatibility

  • Maintains existing API and functionality across all platforms
  • Compatible with all existing deployment scenarios
  • No breaking changes to user workflows
  • FIPS compliance preserved for enterprise environments
  • Supports all authentication methods (access tokens, admin creds, service principals)
  • Multi-architecture Docker builds supported

Performance Impact

  • Build time: Similar (selective copying vs package installation)
  • Runtime performance: Identical functionality with lower memory baseline
  • Container startup: Faster due to smaller image size
  • Network transfer: 71-73% faster image pulls
  • Storage costs: Significant reduction in registry storage usage
  • Cross-platform: No performance difference between architectures

Testing Infrastructure Added

Added scripts/experimental/test-purge-docker.sh with comprehensive test coverage:

  • Docker container functionality validation
  • Azure authentication integration testing
  • Multi-scenario purge operation testing
  • Performance and reliability validation
  • Cross-platform compatibility testing

@balcsida
Copy link
Contributor Author

/AzurePipelines help

Copy link

Supported commands
  • help:
    • Get descriptions, examples and documentation about supported commands
    • Example: help "command_name"
  • list:
    • List all pipelines for this repository using a comment.
    • Example: "list"
  • run:
    • Run all pipelines or specific pipelines for this repository using a comment. Use this command by itself to trigger all related pipelines, or specify specific pipelines to run.
    • Example: "run" or "run pipeline_name, pipeline_name, pipeline_name"
  • where:
    • Report back the Azure DevOps orgs that are related to this repository and org
    • Example: "where"

See additional documentation.

@balcsida
Copy link
Contributor Author

/AzurePipelines run

Copy link

Commenter does not have sufficient privileges for PR 495 in repo Azure/acr-cli

@balcsida balcsida force-pushed the optimize-docker-image-size branch from 3b49628 to ac308b8 Compare August 13, 2025 12:39
@balcsida balcsida force-pushed the optimize-docker-image-size branch from a8b62f2 to e648785 Compare August 13, 2025 12:57
@balcsida balcsida changed the title Optimize Docker image size by 73% using minimal distroless base refactor: use minimal distroless base image to reduce image size Aug 13, 2025
@balcsida balcsida marked this pull request as ready for review August 13, 2025 22:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant