Skip to content

Conversation

makemesteaks
Copy link
Member

@makemesteaks makemesteaks commented Sep 12, 2025

Description

This PR implements a build caching system to optimize CI/CD pipeline performance.

  • For Android: It uses a fingerprinting-based cache for the native build artifacts (.apk, .aab). When only JavaScript code changes are detected, it reuses the cached artifacts and repacks them with the new JS bundle using @expo/repack-app. This avoids a full native compilation, significantly reducing build times.
  • For iOS: The JavaScript repack strategy proved to be unreliable due to the build process's tight integration with Xcode. This PR update the iOS workflow to a simpler state and instead introduces a more robust optimization: caching the CocoaPods dependencies. This speeds up the pod install step, which is a common CI bottleneck.

Architecture Diagram

Build Decision Flow (Android)

flowchart TD
    PR[Pull Request Created/Updated] --> Fingerprint[Generate Fingerprint]
    Fingerprint --> Cache{Cache Hit?}
    
    Cache -->|Yes: Fingerprint Matches| JSOnly[JS-Only Changes<br/>Use Cached APK]
    Cache -->|No: Fingerprint Different| Native[Native/Dependency Changes<br/>Full Rebuild]
    
    JSOnly --> Repack[Repack APK with New JS<br/>⚡ Fast: ~4 minutes]
    Native --> FullRebuild[Full Native Compilation<br/>🐢 Slow: ~8 minutes]
    
    FullRebuild --> SaveCache[Save to Cache]
    
    Repack --> Deploy[Deploy to E2E Tests]
    SaveCache --> Deploy
Loading

Build Flow (iOS)

flowchart TD
    PR[Pull Request Created/Updated] --> PodCache{Pod Cache Hit?}
    
    PodCache -->|Yes| FastSetup[Fast Pod Install<br/>~30 seconds]
    PodCache -->|No| SlowSetup[Full Pod Install<br/>~3-5 minutes]
    
    FastSetup --> FullRebuild[Full Native Compilation<br/>🐢 ~8-10 minutes]
    SlowSetup --> FullRebuild
    
    FullRebuild --> Deploy[Deploy to E2E Tests]
Loading

Key Implementation Details

Android Optimization

  • Fingerprint Generation (@expo/fingerprint): Generates a hash of native dependencies to detect when a full rebuild is needed.
  • Smart Repacking (@expo/repack-app): On a cache hit, it repacks the existing .apk with the new JavaScript bundle.
  • Cache Strategy: Caches the .apk and .aab files using a key based on the fingerprint.

iOS Optimization

  • CocoaPods Caching: Caches the ios/Pods directory. The cache is keyed by the ios/Podfile.lock.
  • Reliability: This avoids the complexities of repacking and is a standard, robust method for speeding up iOS builds. pod install becomes significantly faster when dependencies are cached.

4. Performance Gains

Platform Without Cache With Cache Time Saved
Android ~8 min ~4 min (Repack) ~50%
iOS ~10 min ~6-8 min (Pod Cache) ~20-40%

Changelog

CHANGELOG entry: Added build caching for Android and iOS to optimize CI/CD build times. Android uses fingerprinting to repack JS-only changes, and iOS uses caching for CocoaPods dependencies.

Related issues

Fixes: #19620

Manual testing steps

Feature: CI Build Caching

  Scenario: Android JavaScript-only changes trigger fast repack
    Given A PR with only JavaScript/TypeScript changes for Android
    And A native build for the same fingerprint exists in the cache
    When The Android CI workflow runs
    Then The native build is skipped and the APK is repacked
    And The job completes in approximately 4 minutes

  Scenario: iOS dependency changes trigger a full pod install
    Given A PR that modifies the ios/Podfile.lock
    When The iOS CI workflow runs
    Then The CocoaPods cache is missed
    And A full 'pod install' is performed

  Scenario: iOS build with no dependency changes uses cached pods
    Given A PR with no changes to ios/Podfile.lock
    When The iOS CI workflow runs
    Then The CocoaPods cache is hit
    And The 'pod install' step is significantly faster

Pre-merge author checklist

Pre-merge reviewer checklist

  • I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
  • I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

cursor[bot]

This comment was marked as outdated.

cursor[bot]

This comment was marked as outdated.

@makemesteaks makemesteaks marked this pull request as draft September 12, 2025 11:31
Copy link

socket-security bot commented Sep 12, 2025

Caution

MetaMask internal reviewing guidelines:

  • Do not ignore-all
  • Each alert has instructions on how to review if you don't know what it means. If lost, ask your Security Liaison or the supply-chain group
  • Copy-paste ignore lines for specific packages or a group of one kind with a note on what research you did to deem it safe.
    @SocketSecurity ignore npm/PACKAGE@VERSION
Action Severity Alert  (click "▶" to expand/collapse)
Block Medium
npm/@expo/[email protected] has Shell access.

Module: child_process

Location: Package overview

From: package.jsonnpm/@expo/[email protected]

ℹ Read more on: This package | This alert | What is shell access?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at [email protected].

Suggestion: Packages should avoid accessing the shell which can reduce portability, and make it easier for malicious shell access to be introduced.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore npm/@expo/[email protected]. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

View full report

Copy link
Contributor

CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes.

- Fix critical bug where fingerprint file contained CLI output instead of hash
- Add robust error handling and validation for repacking process
- Implement APK backup/restore for failed repack attempts
- Add comprehensive logging showing fingerprints and decision reasoning
- Include cache hit tracking and performance metrics
- Add APK size and structure validation
- Create detailed documentation in docs/build-caching.md
@github-actions github-actions bot added size-L and removed size-M labels Sep 15, 2025
@Qbandev Qbandev requested a review from Copilot September 15, 2025 12:38
@Qbandev Qbandev added the team-dev-ops DevOps team label Sep 15, 2025
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR implements a build caching mechanism using fingerprinting to optimize CI/CD build times. The system can detect when only JavaScript/TypeScript code changes and reuse previously built APKs, saving significant build time by repacking with updated JS bundles instead of rebuilding native code.

  • Introduces fingerprinting scripts to detect native code changes using Expo Fingerprint
  • Adds build optimization logic to choose between full builds, repacking, or skipping builds entirely
  • Implements caching strategies for build artifacts and fingerprints across GitHub Actions workflows

Reviewed Changes

Copilot reviewed 12 out of 13 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
scripts/generate-fingerprint.js New script to generate fingerprint hash for native code dependencies
scripts/check-fingerprint.js New script to compare current vs saved fingerprints for build decisions
package.json Adds fingerprint scripts and new dependencies (@expo/fingerprint, @expo/repack-app)
docs/build-caching.md Comprehensive documentation explaining the fingerprinting system and implementation
README.md Links to new build caching documentation
.github/workflows/temp-ios-workflow.yml Disables temporary iOS workflow by commenting out configuration
.github/workflows/needs-e2e-build.yml Enhanced to output build actions (full/repack/skip) for each platform
.github/workflows/ci.yml Passes build action parameter to Android build workflow
.github/workflows/build-android-e2e.yml Implements fingerprint-based build optimization with repack logic
.github/actions/save-build-fingerprint/action.yml New action to save fingerprints and cache build artifacts
.github/actions/fingerprint-build-check/action.yml New action to check fingerprints and determine build method
.fingerprintignore Configuration to exclude non-native files from fingerprint calculation

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@metamaskbot metamaskbot added the INVALID-PR-TEMPLATE PR's body doesn't match template label Sep 24, 2025
@Qbandev Qbandev requested a review from Copilot September 24, 2025 11:53
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 8 out of 9 changed files in this pull request and generated 4 comments.


Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@Qbandev Qbandev requested a review from Copilot September 24, 2025 12:21
@Qbandev Qbandev changed the title feat(infra-2854): add cross-platform build caching with fingerprinting feat(infra-2854): add cross-platform build caching Sep 24, 2025
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 8 out of 9 changed files in this pull request and generated 2 comments.


Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@Qbandev Qbandev requested a review from Copilot September 24, 2025 12:40
@Qbandev
Copy link
Contributor

Qbandev commented Sep 24, 2025

bugbot run

Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 8 out of 9 changed files in this pull request and generated 2 comments.


Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

cursor[bot]

This comment was marked as outdated.

@Qbandev
Copy link
Contributor

Qbandev commented Sep 24, 2025

bugbot run

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no bugs!


@Qbandev Qbandev removed the INVALID-PR-TEMPLATE PR's body doesn't match template label Sep 24, 2025
@Qbandev Qbandev requested a review from Copilot September 24, 2025 13:15
@metamaskbot metamaskbot added the INVALID-PR-TEMPLATE PR's body doesn't match template label Sep 24, 2025
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 8 out of 9 changed files in this pull request and generated 5 comments.


Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Copy link

@Qbandev Qbandev added the DO-NOT-MERGE Pull requests that should not be merged label Sep 29, 2025
@github-actions github-actions bot locked and limited conversation to collaborators Sep 30, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
DO-NOT-MERGE Pull requests that should not be merged INVALID-PR-TEMPLATE PR's body doesn't match template size-M team-dev-ops DevOps team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

CI/CD: Build native code only when required
5 participants