v18 fresh full audit: reusable AUDIT_CHECKLIST + 3 HF gate fixes

[lucapinello]

Summary

A from-scratch audit with nothing cached or precomputed — fresh notebook re-execution, selenium IGV rendering (so the client-side JS actually loads tracks instead of a placeholder), CDF sanity across all 6 oracle normalizers, device detection across all 6 oracle envs on macOS arm64, and a trace of the HuggingFace gate that AlphaGenome users hit on first use.

Main deliverable — audits/AUDIT_CHECKLIST.md

A reusable 12-section runbook for future audits. Every check has an exact command or grep pattern, and P0/P1/P2 severity tags so you know what blocks ship vs. what's polish:

1. Installation & environment — environment.yml, chorus setup, chorus genome download, per-oracle env existence
2. HuggingFace auth — HF_TOKEN, whoami, license-repo URL consistency across 3 code paths
3. GPU / device detection — per-env probe of TF / PyTorch / JAX, CUDA_VISIBLE_DEVICES respect
4. Per-track CDF / normalization — monotonicity, p50/p95/p99 ordering, signed-flag semantics, track counts
5. Python API sanity — create_oracle, sequence_length, ModelNotLoadedError, ref-allele warning
6. Notebooks — cell-by-cell fresh execution, drift band
7. Shipped HTML reports — selenium full-JS render recipe, IGV visibility, formula badges, cell-type dedup
8. MCP server — 22-tool registry, list_oracles spec sync
9. Error-message quality — every path triggered and inspected
10. Repo-wide consistency — canonical numbers + formula conventions + directory naming
11. Test suite — fast + integration gates
12. Reproducibility — regen scripts, cache paths

Appendix: exactly what artefacts an audit should leave behind in audits/YYYY-MM-DD_vNN_<label>/ so the next auditor can diff mechanically.

Fixes

The HF-gate agent caught 3 live drifts — all in the flow a first-time AlphaGenome user follows:

1. chorus/oracles/alphagenome.py:133 — the "no HF token" error pointed at huggingface.co/google/alphagenome, but the real gated repo is google/alphagenome-all-folds (matches README.md:631, README.md:926, environments/README.md:105). A user clicking the link would not find the license form. Fixed.
2. chorus/oracles/alphagenome_source/templates/load_template.py:49 — env-runner load path raised with no URL at all. Appended the alphagenome-all-folds license URL so direct and env-runner paths give the same actionable guidance.
3. chorus/oracles/alphagenome_source/alphagenome_metadata.py:4 — module docstring said "5,930+ tracks"; library reports 5,731 (matches v17 audit and post-v16 notebooks). Updated.
4. Tightened tests/test_error_recovery.py:169 from a loose substring match to the full correct URL so the test actually catches this drift in CI.

What the audit found healthy

• Fresh notebook run: exit 0, zero errors, zero warnings. The ref-allele warning that used to fire on cell 39 is gone — confirms PR Fix off-by-one in predict_variant_effect ref-allele check #32 flowed through.
• All 6 CDF normalizers load, sort-monotonic, signed% matches expected semantics. Sei + LegNet auto-download from huggingface.co/datasets/lucapinello/chorus-backgrounds works.
• Device detection clean on macOS arm64: TF Metal picked up for Enformer/ChromBPNet, MPS for Borzoi/Sei/LegNet, JAX device list populated for AlphaGenome. Zero hardcoded cuda:0 in live code.
• 5 selenium-rendered HTMLs: IGV tracks load with signal overlays, 0 browser-console JS errors.

Known issues NOT fixed here

• Genome storage hardcoded to <repo>/genomes/ in core/globals.py:13. Should be user-overridable (CHORUS_GENOMES_DIR) and default to ~/.chorus/genomes/.
• EnvironmentManager.install_chorus_primitive can raise with empty stderr on install failure.
• README doesn't document that chorus genome download auto-resumes after a stall.

Each is worth its own focused PR.

Test plan

• pytest tests/ --ignore=tests/test_smoke_predict.py -q → 334 passed / 1 skipped (9m 11s)
• Fresh jupyter nbconvert --execute single_oracle_quickstart.ipynb → exit 0, 0 errors, 0 warnings
• Selenium rendering of 5 HTMLs at 1600×4500 → 0 SEVERE/ERROR JS messages, IGV tracks visible
• CDF sanity script (monotonicity + p50≤p95≤p99) → all 6 oracles pass
• Device probe across 6 oracle envs → all backends detected correctly on macOS arm64
• Tightened test assertion still passes with the new URL

🤖 Generated with Claude Code