Skip to content

test: replace cl out.ll files with LIT checks - #2281

Open
cpunion wants to merge 3 commits into
xgo-dev:mainfrom
cpunion:codex/test-migrate-outll-sentinels
Open

test: replace cl out.ll files with LIT checks#2281
cpunion wants to merge 3 commits into
xgo-dev:mainfrom
cpunion:codex/test-migrate-outll-sentinels

Conversation

@cpunion

@cpunion cpunion commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • replace 25 placeholder out.ll cases with source-embedded LIT checks covering their complete relevant functions and control flow
  • remove the other seven obsolete out.ll files: six empty files already covered by LIT checks and one unused nested artifact
  • preserve all expect.txt runtime-output comparisons and leave llgen, gentests, and the test framework unchanged

Portability

  • keep full instruction ordering with CHECK-LABEL, CHECK-NEXT, and CHECK-EMPTY
  • use patterns only for unstable identifiers and observed platform ABI differences, including nest/swiftself, cgo hashes and C types, libc symbols, pthread layout, and reflect method slots
  • match the defer save-area allocation as alloca i8 without fixing its platform-defined size
  • retain explicit debug checks for function !dbg, DICompileUnit, and DILocalVariable metadata

Companion documentation update: #2279

Validation

  • all 25 affected cases passed on macOS arm64
  • all 25 affected cases passed on Linux amd64 through dev/docker.sh
  • go test ./internal/littest ./internal/llgen ./chore/litgen ./chore/llgen ./chore/gentests -count=1
  • git diff --check

@fennoai fennoai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review summary

This is a clean, well-executed migration from golden out.ll snapshots to source-embedded // LITTEST + // CHECK FileCheck directives. Dead code removal is thorough (no dangling references to SmartDoFile, DoFile, genZip, inCompilerDir, cltest.Pkg, decodeLinkFile, or the Mode enum remain), the internal/littest simplification and new ErrSpecNotFound sentinel are genuine improvements, and the removed TestCgofullGeneratesC2func is now covered by CHECK-DAG in cgofull.go. Moving defer_ir_regression_test.go off a go run ./chore/llgen subprocess to an in-process llgen.GenFrom call is also a correctness/speed win. internal/littest vets clean.

The findings below are all in files the PR did not touch, so they cannot be placed inline.

Findings

1. (High) Stale documentation references the deleted chore/gentests / chore/llgen tooling and the removed out.ll workflow. This PR deletes both chore/gentests/ and chore/llgen/ entirely and removes all out.ll files, but the docs still instruct contributors to use them — these commands will now fail and the described workflow no longer exists:

  • CLAUDE.md:60-102 — the "Understanding out.ll files" / "Required steps after modifying IR generation logic" section still describes out.ll as "generated by llgen" and instructs go install -v ./chore/gentests, go install -v ./chore/llgen, and running gentests / llgen. CLAUDE.md:16 also lists llgen as a chore/ tool.
  • README.md:425-426 — the tools list still documents [llgen](chore/llgen) and [gentests](chore/gentests) (now broken in-repo links) with out.ll/expect.txt regeneration descriptions.
  • dev/README.md:77-97 — the "Refresh test goldens" section documents a ### gentests flow with go run ./chore/gentests for regenerating out.ll.

Suggest updating these to describe the // LITTEST + // CHECK FileCheck workflow (and the surviving litgen tool, go run ./chore/litgen <path>) before merge. Note the internal/llgen library package still exists and is fine — only the chore/llgen command was removed.

2. (Minor) cltest.CaptureMeta is now dead exported code. cl/cltest/cltest.go:370CaptureMeta was only ever called from the deleted chore/gentests/gentests.go; it now has zero callers (its sibling RunAndCapture is still used by cl/compile_test.go:449). Consider removing it (and any orphaned meta-capture plumbing), or adding a comment noting an intended future caller.

3. (Minor / optional, pre-existing) IR checks now shell out per test package. With the ModeSkip short-circuit and the in-process ModeLiteral string compare both removed, every // LITTEST package now routes through filecheck.Matchllvm.New("") (internal/filecheck/filecheck.go:36), which re-execs llvm-config --bindir and then FileCheck on every call. Across the ~200 marked packages that is a meaningful number of redundant llvm-config subprocess spawns per run. This is pre-existing behavior in internal/filecheck (not introduced here), but this PR increases the number of packages routed through it. A process-level sync.Once around the llvm.New("") bindir lookup would remove the redundant execs at zero correctness cost. Not blocking.

No security concerns: removing genZip (a zip shell-out, previously dead behind if false) and decodeLinkFile (an archive/zip decode of committed fixtures) is a net reduction in surface area.

@cpunion cpunion changed the title test: replace legacy out.ll workflow with LIT checks test: replace cl out.ll files with LIT checks Aug 4, 2026
@codecov

codecov Bot commented Aug 4, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@cpunion

cpunion commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks for the review. The review was generated against the initial broader version of this PR. Commit d9b118f has since narrowed the final diff to cl/_test*/** only.

  1. chore/llgen and chore/gentests are restored unchanged, so the documentation links and commands are no longer broken by this PR. The companion documentation cleanup remains in docs: add contributor guide and validation wrappers #2279.
  2. cltest.CaptureMeta is restored to its previous state and is again called by chore/gentests.
  3. ModeSkip, ModeLiteral, and the existing FileCheck plumbing are restored unchanged. Only the 25 newly migrated fixture cases now use source-embedded LIT checks.

The current PR diff contains only fixture sources and out.ll deletions; no tooling or test-framework code remains changed.

@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown

LLGo baseline benchmarks

275b202668d4 | workflow run | long-term charts

Program measurements

Platform Workload File size vs base Build vs base Run vs base
Linux cprintf 18456 B +0.0% 316.176 ms -1.4% (better) 1.314 ms -4.8% (better)
Linux fmtprintf 1829968 B +0.0% 3.196 s +2.2% (worse) 3.708 ms +4.5% (worse)
Linux println 68008 B +0.0% 306.229 ms +0.6% (worse) 1.703 ms -0.9% (better)
macOS cprintf 84672 B +0.0% 438.352 ms -10.5% (better) 3.723 ms +21.7% (worse)
macOS fmtprintf 1869328 B +0.0% 6.376 s +84.8% (worse) 20.222 ms +20.4% (worse)
macOS println 121200 B +0.0% 409.030 ms -11.3% (better) 5.417 ms +18.7% (worse)
Core language and compiler benchmarks
Platform Benchmark ns/op vs base
Linux BenchmarkLookupPCRandom 12.250 ns/op -1.5% (better)
Linux BenchmarkMergeCompilerFlags 146.400 ns/op +0.3% (worse)
Linux BenchmarkMergeLinkerFlags 94.830 ns/op -0.5% (better)
Linux BenchmarkChannelBuffered 37.020 ns/op +2.0% (worse)
Linux BenchmarkChannelHandoff 24299 ns/op -4.1% (better)
Linux BenchmarkDefer 52.550 ns/op +1.6% (worse)
Linux BenchmarkDirectCall 1.758 ns/op -0.1% (better)
Linux BenchmarkGlobalRead 1.757 ns/op -0.1% (better)
Linux BenchmarkGlobalWrite 2.809 ns/op +0.0% (worse)
Linux BenchmarkGoroutine 30847 ns/op +1.0% (worse)
Linux BenchmarkInterfaceCall 9.143 ns/op +0.0% (worse)
Linux BenchmarkRuntimeGetG 2.111 ns/op -0.1% (better)
macOS BenchmarkLookupPCRandom 18.530 ns/op +0.4% (worse)
macOS BenchmarkMergeCompilerFlags 220.400 ns/op +28.5% (worse)
macOS BenchmarkMergeLinkerFlags 109.100 ns/op -9.7% (better)
macOS BenchmarkChannelBuffered 24.540 ns/op -22.3% (better)
macOS BenchmarkChannelHandoff 7615 ns/op -31.2% (better)
macOS BenchmarkDefer 33.750 ns/op -42.1% (better)
macOS BenchmarkDirectCall 1.202 ns/op -9.2% (better)
macOS BenchmarkGlobalRead 1.133 ns/op -24.6% (better)
macOS BenchmarkGlobalWrite 1.237 ns/op -23.8% (better)
macOS BenchmarkGoroutine 56589 ns/op +5.2% (worse)
macOS BenchmarkInterfaceCall 6.229 ns/op -5.1% (better)
macOS BenchmarkRuntimeGetG 2.171 ns/op -17.9% (better)

Compared with d37108c85377 measured in the same runner job.

@cpunion

cpunion commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator Author

Expanded the migrated checks in 275b20266.

Each of the 25 cases now checks its complete relevant function bodies and control-flow blocks instead of only a few symbols. Platform-specific matching is limited to observed ABI differences: nest/swiftself, cgo hashes and C types, sigsetjmp/stderr names, pthread once layout, reflect method slots, and the defer save-area allocation. The debug case also retains explicit function !dbg, DICompileUnit, and DILocalVariable coverage.

All affected cases passed on macOS arm64 and Linux amd64. No out.ll or non-fixture tooling changes were reintroduced.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant