feat: add --cleanup-bak to cefs gc + weekly GC workflow#2028
Merged
mattgodbolt merged 5 commits intomainfrom Mar 12, 2026
Merged
feat: add --cleanup-bak to cefs gc + weekly GC workflow#2028mattgodbolt merged 5 commits intomainfrom
mattgodbolt merged 5 commits intomainfrom
Conversation
Member
|
Testing... |
The CEFS system creates .bak directories on NFS when replacing compiler dirs with symlinks. These accumulate over time and need periodic cleanup. - Add --cleanup-bak flag to `cefs gc` that removes old .bak and .DELETE_ME_* items before running image GC, reusing the existing discovery infrastructure from fsck - Change gc default --min-age from 1h to 2d (other commands still use the global 1h default) for safer scheduled runs - Add weekly GitHub Actions workflow (Sunday 09:00 UTC) for automated GC with --cleanup-bak - Add cleanup_bak_items() and delete_bak_item() to gc module with tests 🤖 Generated by LLM (Claude, via OpenClaw)
🤖 Generated by LLM (Claude, via OpenClaw)
🤖 Generated by LLM (Claude, via OpenClaw)
mattgodbolt-molty
force-pushed
the
molty/cefs-bak-cleanup
branch
from
5eb1ed9 to
e552fc8
Compare
Instead of os.walk() over all of /opt/compiler-explorer (very slow on EFS), derive the paths to check from CEFS manifests. Each manifest records the exact NFS destination path, so we can do O(n_compilers) lstat() calls rather than an O(all_files_on_NFS) tree walk. DELETE_ME_* items are found via targeted parent.glob() on each known path. 🤖 Generated by LLM (Claude, via OpenClaw)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
.bak/.DELETE_ME_*cleanup tocefs gc, and a new weekly scheduled workflow to run it automatically.Changes
bin/lib/cli/cefs.py--cleanup-bakflag ongc: scans for stale.bakand.DELETE_ME_*items and removes them before running image GC, so the GC sees a clean reference picturefind_files_by_pattern()from fsck infrastructure atNFS_MAX_RECURSION_DEPTH=3--min-ageapplies to both bak cleanup and image GC; default changed from1h→2dforgcspecifically (GC_DEFAULT_MIN_AGE;DEFAULT_MIN_AGEunchanged for other commands)--force(skip confirmation) and dry-run mode throughout.github/workflows/cefs-gc.yml(new)cron: '0 9 * * 0') — after nightly install (05:30) and consolidation (07:15) have finished['admin'](needs NFS access)sudo bin/ce_install cefs gc --force --cleanup-bak --min-age 2dmin-ageandinclude-broken${{ github.event.inputs.min-age || '2d' }}to avoid the empty-default-on-schedule bugbin/test/cefs/bak_cleanup_test.py(new)10 tests covering: file/symlink/dir removal, age filtering, dry-run, mixed types, error handling, edge cases.
Test results
562 passed.
(I'm Molty, an AI assistant acting on behalf of @mattgodbolt)