Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ This file applies to GitHub metadata and GitHub Actions workflows.
- Mobile Android CI copies `.github/gradle/gradle_plugin_repos.init.gradle` into `~/.gradle/init.d` before `flutter build apk`. Flutter's included `flutter_tools/gradle` build has no `pluginManagement` block, so a cold runner only searches the plugin portal for that classpath; the portal 303s Kotlin artifacts to Maven Central and then fails to parse `kotlin-gradle-plugins-bom`. Do not patch `$FLUTTER_ROOT` for this: the Flutter SDK cache would pick up the mutation.
- Rust compiler outputs use `sccache`, installed by `.github/actions/setup-rust-sccache`. Trusted warm-cache, release, and pull-request rust jobs may connect it to a dedicated Cloudflare R2 bucket through the `SCCACHE_R2_ACCOUNT_ID` and `SCCACHE_R2_BUCKET` repository variables and the `SCCACHE_R2_ACCESS_KEY_ID` and `SCCACHE_R2_SECRET_ACCESS_KEY` repository secrets. Those credentials must have access only to the compiler-cache bucket and must never reuse the `alera-updates` bucket or its production credentials. Fork pull requests, incomplete configurations, and R2 authentication failures fall back to the runner-local cache so cache availability cannot block a build.
- Cargokit sets `CARGOKIT_TEMP_DIR` under the Flutter build directory, so the fully linked `alera_native` and `code_forge` outputs under `build/**/cargokit_build` keep a separate `actions/cache` entry. Only `warm-cache.yml` may save that entry, so pull requests and release jobs restore a shared copy instead of writing private copies into their own ref scopes. Windows CI must install the pinned LunarG Vulkan SDK (`1.4.350.0`) from the official installer and verify its SHA-256. Do not use `winget`: GitHub-hosted runners fail with `APPINSTALLER_CLI_ERROR_SOURCE_DATA_MISSING` (`-1978335217`). Windows Whisper/Vulkan jobs map scratch to `R:\c` with native cargo `--target-dir` at `R:\c\n` (not `R:\c\alera_native` or `R:\alera-cargokit`: those prefixes make vulkan-shaders-gen's nested TryCompile object exceed `MAX_PATH` and `cl.exe` reports `C1083` with an empty generated-file name), set `CMAKE_GENERATOR=Ninja` so nested vulkan-shaders-gen cmake inherits Ninja, set `_CL_=/Z7 /FS` because `cl.exe` ignores `CFLAGS`, and set `GGML_CCACHE=OFF` so ggml does not wrap `cl.exe` with the rustc sccache (that pairing drops `.obj` files under Ninja).
- Steps that share nothing run concurrently with `parallel`, or with `background: true` plus a later `wait`/`wait-all`, so a job's wall time is its longest lane instead of the sum of its steps. Composite actions cannot declare either keyword, so the overlap lives in the workflows, which may still run a whole composite as a background step. A background step's outputs, `GITHUB_ENV`, and `GITHUB_PATH` changes apply only at the wait that includes it, so wait before the first step that reads them, and prefer `wait-all` when the background step has an `if:` that can skip it. Never run two Flutter or pub commands that resolve the same package concurrently: steps in a group pass `--no-pub` after one explicit resolution, because an implicit pub get rewrites `package_config.json` under the others. Never overlap two `flutter build` invocations of one project, the native integration suites, or `tool/ci/run_rust_workspace_tests.sh`, whose PTY and app-launch tests run against timing budgets. Overlap network, apt, Docker-daemon, and download work with compilation, not two compilations: a runner has 4 vCPUs (3 on macOS). Desktop Builds run 37696090551 moved the desktop-build workflow `cargo test` suites into the background beside `flutter build` with private target directories. The app build doubled on every platform, and the later debug integration builds lost the dependencies those suites leave in `rust/target`, so every leg got slower (Linux 33 to 46 min, macOS 64 to 78 min). `tune-windows-build` starts in the background before the Flutter setup and is waited on right after it.
- `warm-cache.yml` exists to populate the `main` Cargokit cache scope and the shared R2 compiler cache. It must keep using the same composite actions as the pull request and release jobs, so configuration cannot drift. It must keep running the real release build, since a prologue-only warm job would leave Cargokit cold, and it must keep the scheduled/manual Linux `warm-rust` job, since that release build's rustc command line does not match PR clippy/test. `warm-rust-cache.yml` owns push warming through the same Rust checks action. Rust-only pushes must not trigger the desktop warming matrix, and `warm-cache.yml` must skip its Rust job on pushes to avoid duplicate work.
- Rust checks use `ALERA_BUILD_COMMIT=unknown`; explicit nonempty commit overrides must not consult or watch Git in `alera-cli/build.rs`. Local `make rust-test` exports the stable commit and an empty `ALERA_BUILD_VERSION` only for that target and invokes Cargo directly without Bash. `tool/ci/run_rust_workspace_tests.sh` unsets `ALERA_BUILD_VERSION`; distributed builds retain their real identity. `warm-rust` uses the composite's `test-build` mode, which invokes the same two workspace test commands with `--no-run`. Keep `rust/**`, `makefile` and the Rust check scripts in the `warm-rust-cache.yml` push filters so source changes populate the cache on `main` without executing the suite again. `tool/ci/test_rust_build_cache.sh` verifies default and override identities, Windows stack flags, local checks without Bash on PATH, unaffected CLI build identities and the two runner modes without compiling the full workspace.
- Every workflow must declare an explicit `permissions` block. The repository default grants the whole `GITHUB_TOKEN` scope set to jobs that only read the checkout, so a compromised action or dependency inherits write access it never needed. Workflows that build, analyze, or test declare `contents: read` at the workflow level; a job that genuinely writes something adds the single scope it needs at the job level, as `cloud-deploy.yml` and `release-cut.yml` do. CodeQL's `actions/missing-workflow-permissions` rule reports a workflow that omits it.
Expand Down
60 changes: 35 additions & 25 deletions .github/workflows/cloud-deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -233,6 +233,37 @@ jobs:
fi
done

# The image build is most of this job and needs only the registry login.
# It starts once the preflight has passed and keeps building while the
# state, the current deployment and the relay state are read, then the
# plan waits for its digest.
- name: Setup Docker Buildx
if: ${{ env.DEPLOY_MODE != 'rollback' }}
uses: docker/setup-buildx-action@bb05f3f5519dd87d3ba754cc423b652a5edd6d2c # v4.2.0

- name: Configure Artifact Registry
if: ${{ env.DEPLOY_MODE != 'rollback' }}
run: gcloud auth configure-docker "$GCP_REGION-docker.pkg.dev" --quiet

- name: Build And Publish Image
id: image
if: ${{ env.DEPLOY_MODE != 'rollback' }}
background: true
run: |
set -euo pipefail
repository="$GCP_REGION-docker.pkg.dev/$GCP_PROJECT_ID/alera-cloud/alera-cloud"
image_tag="$repository:$GITHUB_SHA"
docker buildx build \
--file cloud/Dockerfile \
--platform linux/amd64 \
--push \
--tag "$image_tag" \
--metadata-file "$RUNNER_TEMP/alera-cloud-build.json" \
cloud
digest="$(jq -er '."containerimage.digest"' "$RUNNER_TEMP/alera-cloud-build.json")"
echo "target_image=$repository@$digest" >>"$GITHUB_OUTPUT"
echo "digest=$digest" >>"$GITHUB_OUTPUT"

- name: Setup OpenTofu
uses: opentofu/setup-opentofu@a1320f892987e89d278cc92dc5adc984fb93aca4 # v2.0.2
with:
Expand All @@ -244,10 +275,6 @@ jobs:
with:
bun-version: '1.3.14'

- name: Setup Docker Buildx
if: ${{ env.DEPLOY_MODE != 'rollback' }}
uses: docker/setup-buildx-action@bb05f3f5519dd87d3ba754cc423b652a5edd6d2c # v4.2.0

- name: Install Edge Dependencies
working-directory: edge
run: bun install --frozen-lockfile
Expand Down Expand Up @@ -358,27 +385,10 @@ jobs:
fi
echo "state=$relay_state" >>"$GITHUB_OUTPUT"

- name: Configure Artifact Registry
if: ${{ env.DEPLOY_MODE != 'rollback' }}
run: gcloud auth configure-docker "$GCP_REGION-docker.pkg.dev" --quiet

- name: Build And Publish Image
id: image
if: ${{ env.DEPLOY_MODE != 'rollback' }}
run: |
set -euo pipefail
repository="$GCP_REGION-docker.pkg.dev/$GCP_PROJECT_ID/alera-cloud/alera-cloud"
image_tag="$repository:$GITHUB_SHA"
docker buildx build \
--file cloud/Dockerfile \
--platform linux/amd64 \
--push \
--tag "$image_tag" \
--metadata-file "$RUNNER_TEMP/alera-cloud-build.json" \
cloud
digest="$(jq -er '."containerimage.digest"' "$RUNNER_TEMP/alera-cloud-build.json")"
echo "target_image=$repository@$digest" >>"$GITHUB_OUTPUT"
echo "digest=$digest" >>"$GITHUB_OUTPUT"
# A rollback builds nothing, so there is no background step to wait for
# and wait-all returns at once.
- name: Wait for image
wait-all:

- name: Select Deployment Target
id: target
Expand Down
62 changes: 37 additions & 25 deletions .github/workflows/cloud.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,14 @@ jobs:
with:
submodules: false

# The image compiles its own release build inside the Docker daemon and
# needs nothing from the host toolchain, so it runs alongside the cargo
# checks instead of after them. It was half of this job's wall time.
- name: Build Container
id: container
background: true
run: docker build --file Dockerfile .

- name: Setup Rust
uses: dtolnay/rust-toolchain@2c7215f132e9ebf062739d9130488b56d53c060c
with:
Expand All @@ -77,8 +85,8 @@ jobs:
- name: PostgreSQL Contract Tests
run: cargo test --workspace -- --ignored --test-threads=1

- name: Build Container
run: docker build --file Dockerfile .
- name: Wait for container build
wait: container

edge:
runs-on: ubuntu-latest
Expand Down Expand Up @@ -126,29 +134,33 @@ jobs:
with:
tofu_version: 1.12.1

- name: Validate Production Root
working-directory: infra/production
run: |
tofu fmt -check -recursive
tofu init -backend=false -input=false
tofu validate

- name: Validate Bootstrap Root
working-directory: infra/bootstrap/github
run: |
tofu fmt -check -recursive
tofu init -backend=false -input=false
tofu validate

- name: Test Plan Guard
working-directory: tool/cloud
run: python3 -m unittest -v test_validate_tofu_plan.py

- name: Validate Deployment Scripts
run: bash -n tool/cloud/deploy_worker_relay_disabled.sh tool/cloud/test_verify_production.sh tool/cloud/verify_production.sh tool/cloud/verify_worker_namespace.sh

- name: Test Production Verifier
run: bash tool/cloud/test_verify_production.sh
# Each check reads its own root or script. tofu init downloads providers
# into each root's own .terraform directory, so the roots validate
# side by side.
- parallel:
- name: Validate Production Root
working-directory: infra/production
run: |
tofu fmt -check -recursive
tofu init -backend=false -input=false
tofu validate

- name: Validate Bootstrap Root
working-directory: infra/bootstrap/github
run: |
tofu fmt -check -recursive
tofu init -backend=false -input=false
tofu validate

- name: Test Plan Guard
working-directory: tool/cloud
run: python3 -m unittest -v test_validate_tofu_plan.py

- name: Validate Deployment Scripts
run: bash -n tool/cloud/deploy_worker_relay_disabled.sh tool/cloud/test_verify_production.sh tool/cloud/verify_production.sh tool/cloud/verify_worker_namespace.sh

- name: Test Production Verifier
run: bash tool/cloud/test_verify_production.sh

cloud-ready:
name: cloud-ready
Expand Down
24 changes: 19 additions & 5 deletions .github/workflows/desktop-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -117,15 +117,23 @@ jobs:
submodules: false
persist-credentials: false

# Started before the setup so the Defender exclusions already cover the
# Flutter and pub caches while the setup restores them. Its environment
# (Ninja on PATH, the R: scratch drive) applies at the wait below.
- name: Tune Windows build environment
if: runner.os == 'Windows'
background: true
uses: ./.github/actions/tune-windows-build

- name: Setup Flutter workspace
uses: ./.github/actions/setup-flutter-workspace
with:
rust: 'true'
xvfb: 'true'

- name: Tune Windows build environment
if: runner.os == 'Windows'
uses: ./.github/actions/tune-windows-build
# wait-all, not `wait: <id>`: off Windows the tune step is skipped.
- name: Wait for Windows build environment
wait-all:

- name: Verify workflow contracts and native Git integration
run: >-
Expand Down Expand Up @@ -340,13 +348,19 @@ jobs:
submodules: false
persist-credentials: false

# See the build job: the exclusions are in place before the cache restores.
- name: Tune Windows build environment
id: tune
background: true
uses: ./.github/actions/tune-windows-build

- name: Setup Flutter workspace
uses: ./.github/actions/setup-flutter-workspace
with:
rust: 'true'

- name: Tune Windows build environment
uses: ./.github/actions/tune-windows-build
- name: Wait for Windows build environment
wait: tune

- name: Restore cargokit build
uses: actions/cache/restore@v6
Expand Down
13 changes: 11 additions & 2 deletions .github/workflows/landing.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,15 @@ jobs:
- name: Install dependencies
run: bun install --frozen-lockfile

# The browser download and its apt dependencies are the longest step
# before the end-to-end suite and touch nothing under landing/, so they
# install while the unit tests, type check and build run. Those three
# stay sequential: `astro sync` and `astro build` both write .astro/.
- name: Install Playwright Chromium
id: playwright
background: true
run: bunx playwright install --with-deps chromium

- name: Unit tests
run: bun test

Expand All @@ -46,8 +55,8 @@ jobs:
- name: Build
run: bun run build

- name: Install Playwright Chromium
run: bunx playwright install --with-deps chromium
- name: Wait for Playwright Chromium
wait: playwright

- name: End-to-end tests
run: bunx playwright test
Expand Down
Loading
Loading