fix(stoch,kama): divide by the value the guard tests, and bound the efficiency ratio (#390) #137

Workflow file for this run

.github/workflows/pr-codegen-gate.yml at 7b32039

	# Free, always-on PR gate: the committed generated tree must match
	# ta_codegen/input/.
	#
	# ta_codegen/input/ is the source of truth and every file it produces is
	# committed, so a PR that edits an input without committing the regenerated
	# output is broken on arrival. Until now the only thing checking that was the
	# 5AM regen-check on dev — i.e. after merge, one branch too late (#211).
	#
	# Cargo + Python for the regen and Rust halves, plus a JDK and a .NET SDK for the
	# two compile jobs. WRITING a .java or a .cs is text emission from the IR — but
	# the text has to compile, and until #326 nothing on the PR path checked that;
	# see the java-compiles job for what that cost. RUNNING the cross-language
	# suites still needs more toolchain than this gate carries and stays in the
	# nightly.
	#
	# `scripts/build.py regen-check` is the first half of the gate, so the same
	# command reproduces that failure locally, exactly.
	#
	# The second half runs the generated crate's doctests and the generator's own
	# test suite. Both are cargo-only, so they keep the no-JDK/no-.NET property
	# above. They are here because the value gates cannot see a whole class of
	# defect: an integer expression that is correct in C, where the counters are
	# signed, and underflows in Rust, where the backend renders them `usize`.
	# Landing #238, `--xlang-hash` reported 5678 cases per language with 0
	# mismatches while the shipped Rust panicked on every debug call -- the servers
	# it compares are release builds, where the underflow wraps instead of trapping.
	# The doctests are built in debug, so Rust's overflow checks are armed, and one
	# is generated per function over a corpus that clears the lookback; that makes
	# them the generic arithmetic-safety gate for all 175 functions. Until now they
	# ran only in the 5AM nightly, i.e. after merge -- the same "one branch too
	# late" this gate was created to fix.
	name: PR Codegen Gate

	on:
	pull_request:

	permissions:
	contents: read

	jobs:
	regen-check:
	name: ta_codegen output up-to-date
	runs-on: ubuntu-latest
	timeout-minutes: 30

	steps:
	# The default merge-commit ref is the right one to gate: it is what lands.
	- uses: actions/checkout@v6

	- name: Install Python
	uses: actions/setup-python@v6
	with:
	python-version: "3.x"

	# cargo is preinstalled on ubuntu-latest; rust-toolchain.toml pins the
	# version rustup installs on first use.
	- name: Regenerate all backends and verify the committed output matches
	shell: bash
	run: \|
	if ! python3 scripts/build.py regen-check; then
	echo "::error::ta_codegen output is out of date. Run 'scripts/build.py generate' and commit the result — see the log above for which files drifted."
	exit 1
	fi

	# Debug build on purpose: `cargo test` defaults to it, and the overflow
	# checks it turns on are the point. Do not add --release here.
	- name: Run the generated crate's doctests (debug, overflow checks armed)
	shell: bash
	run: \|
	if ! cargo test --doc -p ta-lib --manifest-path ta_codegen/output/rust/Cargo.toml; then
	echo "::error::A generated doctest failed. A panic reading 'attempt to subtract with overflow' means an integer expression that is fine in C underflows once the Rust backend renders it as usize -- compare the counters before subtracting, not after."
	exit 1
	fi

	- name: Test the generator tool
	shell: bash
	run: \|
	if ! cargo test --no-fail-fast --manifest-path ta_codegen/generator/Cargo.toml; then
	echo "::error::The generator's own test suite failed. Several suites pin exact-set inventories (FMA fusion, stability classification) that a new indicator must be added to."
	exit 1
	fi

	# Compiling the generated Java is not the same question as generating it, and
	# this gate used to answer only the second. PR #326 passed every check here
	# while NONE of the generated Java compiled: a dispatch arm called a method
	# signature the same PR had removed, `server_gen.rs` was never updated so the
	# JSON-RPC server carried 96 references to a deleted type, and `{@link
	# #value()}` stopped resolving. All four were found by hand after the gate
	# went green.
	#
	# Compile only — no Maven, no tests, no servers started. That is the whole
	# trade: ~20s of javac buys the entire class of "the emitter wrote Java that
	# is not Java", and running anything needs the nightly's toolchain anyway.
	#
	# Three units, because they fail independently and a PR can break exactly one:
	# the shipped library, the hand-written suites under it (NOT generated — the
	# generator preserves them, so an API change has to be carried into them by
	# hand, which is what #326 missed), and the server, whose Java comes from a
	# different emitter (server_gen.rs) than the library's (java_stream.rs).
	# Nothing else compares those two emitters' idea of the same API.
	#
	# `-Xdoclint:all,-missing` on the library is load-bearing and not tidiness:
	# javac's doclint is OFF by default, so a plain compile is blind to it, while
	# the pom turns it on for the DEFAULT build (maven-javadoc-plugin's jar goal
	# binds to `package`). An unresolvable @link therefore fails `./mvnw package`
	# — one of #326's four — and would still reach dev green without this flag.
	# `-missing` matches the pom: streaming handles are deliberately undocumented.
	#
	# `--release 17` everywhere, the same JAVA_RELEASE the generator pins, so this
	# job cannot accept a source the shipped build would reject. The JDK is 21 to
	# match the nightly; any JDK >= 17 satisfies --release.
	java-compiles:
	name: Generated Java compiles
	runs-on: ubuntu-latest
	timeout-minutes: 20

	steps:
	- uses: actions/checkout@v6

	- name: Install JDK
	uses: ./.github/actions/apt-install
	with:
	packages: openjdk-21-jdk

	# The committed sources, not a fresh `generate`: regen-check already
	# proves those are the same tree, and this job then needs no cargo at all.
	- name: Compile the shipped library, with the pom's doclint
	shell: bash
	run: \|
	cd ta_codegen/output/java
	if ! javac --release 17 -Xdoclint:all,-missing -d /tmp/ta-java-main \
	$(find library/src/main -name '*.java'); then
	echo "::error::The generated Java library does not compile. Fix the emitter (ta_codegen/generator/src/backends/java*.rs), never the generated file. 'error: reference not found' is doclint: an @link naming a signature that no longer exists — the pom runs this on the default build, so it fails ./mvnw package too."
	exit 1
	fi

	# Hand-written, and the generator preserves them — so an API change lands
	# here only if someone carries it. Nothing else on this gate reads them.
	- name: Compile the hand-written Java suites against it
	shell: bash
	run: \|
	cd ta_codegen/output/java
	if ! javac --release 17 -nowarn -cp /tmp/ta-java-main -d /tmp/ta-java-test \
	$(find library/src/test -name '*.java'); then
	echo "::error::The hand-written Java suites no longer compile against the generated library. These are NOT generated (ta_codegen/output/java/library/src/test/) — an API change has to be carried into them by hand."
	exit 1
	fi

	# Same command `ta_codegen build --backend=java` runs, so a failure here
	# reproduces with `scripts/build.py servers --language=java`.
	- name: Compile the JSON-RPC server
	shell: bash
	run: \|
	cd ta_codegen/output/java
	if ! javac --release 17 -nowarn --source-path library/src/main/java \
	-d /tmp/ta-java-serve tools/TaCodegenServe.java; then
	echo "::error::The generated Java server does not compile against the generated library. The two come from different emitters — server_gen.rs writes the server, java_stream.rs writes the library — so this is where they disagree about an API. Fix server_gen.rs; reproduce with scripts/build.py servers --language=java."
	exit 1
	fi

	# The same hole for C#: the whole managed tree reaches dev on generation alone.
	#
	# None of the three units is redundant — each is the only one that fails for
	# its own class of defect. Only the library project sets
	# GenerateDocumentationFile, so only it makes CS1591 (a public member with no
	# XML doc) an error — C#'s -Xdoclint. The suites are hand-written, so an API
	# change reaches them only if someone carries it. The server is a different
	# emitter (server_gen.rs, not csharp*.rs), so a defect in its own output is
	# invisible to the other two. Compile only; running them stays in the nightly.
	csharp-compiles:
	name: Generated C# compiles
	runs-on: ubuntu-latest
	timeout-minutes: 20

	steps:
	- uses: actions/checkout@v6

	- name: Install .NET SDK
	uses: actions/setup-dotnet@v6
	with:
	dotnet-version: "10.0.x"

	# The committed sources, not a fresh `generate` — regen-check already
	# proves they match, so this job needs no cargo.
	- name: Build the shipped library, with the csproj's doc gate
	shell: bash
	run: \|
	if ! dotnet build ta_codegen/output/csharp/library/TALib.csproj \
	-c Release --nologo; then
	echo "::error::The generated C# library does not compile. Fix the emitter (ta_codegen/generator/src/backends/csharp*.rs), never the generated file. CS1591 is the doc gate, not a warning to silence: the csproj pairs TreatWarningsAsErrors with GenerateDocumentationFile, so a public member with no XML doc fails the shipped build too."
	exit 1
	fi

	- name: Build the hand-written C# suites against it
	shell: bash
	run: \|
	if ! dotnet build ta_codegen/output/csharp/library/test/TALib.Test.csproj \
	-c Release --nologo; then
	echo "::error::The hand-written C# suites no longer compile against the generated library. These are NOT generated (ta_codegen/output/csharp/library/test/) — an API change has to be carried into them by hand."
	exit 1
	fi

	- name: Build the JSON-RPC server
	shell: bash
	run: \|
	if ! dotnet build ta_codegen/output/csharp/tools/TaCodegenServe.csproj \
	-c Release --nologo; then
	echo "::error::The generated C# server does not compile against the generated library. The two come from different emitters — server_gen.rs writes the server, csharp.rs and csharp_stream.rs write the library — so this is where they disagree about an API. Fix server_gen.rs; reproduce with scripts/build.py servers --language=csharp."
	exit 1
	fi

	# The same hole for C, and the shipped library is the one that lands in the
	# tarball. Nothing on this gate compiled a line of it: regen-check writes the
	# tree and stops, and the C build was nightly-only — so a generator change that
	# makes 178 .c files unbuildable reached dev green, exactly the way #326's
	# Java did before the job above existed.
	#
	# It is also the enforcement point for one contract no other gate can state:
	# a peek frame binds the caller's handle `const`, so a frame that committed a
	# SCALAR or in-struct-array field is `assignment of member '<field>' in
	# read-only object` here rather than a store into memory the caller never
	# reads. `const` says nothing about memory behind a pointer member, so a
	# buffer-element store stays peek_suite's to catch.
	#
	# CMake + gcc, both preinstalled. Release is what ships; the tools build with
	# the library, so ta_regtest and ta_bench are covered too. RUNNING them stays
	# in the nightly.
	c-compiles:
	name: Generated C compiles
	runs-on: ubuntu-latest
	timeout-minutes: 20

	steps:
	- uses: actions/checkout@v6

	# The committed sources, not a fresh `generate` — regen-check already
	# proves they are the same tree, so this job needs no cargo.
	- name: Build the shipped library and the C tools
	shell: bash
	run: \|
	if ! python3 scripts/build.py; then
	echo "::error::The generated C does not compile. Fix the emitter (ta_codegen/generator/src/backends/c*.rs), never the generated file. 'assignment of member ... in read-only object' inside a TA_<N>_Peek means the peek frame writes a state field the emitter failed to localize."
	exit 1
	fi

	# Clippy runs here as its own job, concurrently with regen-check, so it costs
	# the PR no wall-clock: ~50s finishing well inside that job's ~4m. That is
	# also why there is no `paths` filter on Rust — a filter would only save
	# runner minutes, which are free on a public repo, and most PRs here touch
	# .rs anyway because the generated crate is committed.
	#
	# The generator enforces #![deny(clippy::pedantic)] on itself, and rustc
	# ignores the `clippy::` namespace — so the `cargo test` steps above compile
	# the very same code green and can never see it. Only clippy can.
	#
	# This does NOT retire the nightly's clippy job, and nothing here should:
	# most commits reach dev as direct pushes that `on: pull_request` never sees,
	# so the nightly is the only gate that covers them. This job is the earlier,
	# partial signal for the contributor path, not a replacement.
	clippy:
	name: Rust clippy
	runs-on: ubuntu-latest
	timeout-minutes: 20

	steps:
	- uses: actions/checkout@v6

	# rust-toolchain.toml pins the version, so a lint cannot fire differently
	# here than locally. Adding the component is idempotent.
	- name: Ensure clippy component
	shell: bash
	run: rustup component add clippy

	- name: Clippy the generator tool
	shell: bash
	run: \|
	if ! cargo clippy --all-targets --manifest-path ta_codegen/generator/Cargo.toml -- -D warnings; then
	echo "::error::Clippy failed on the generator. It carries #![deny(clippy::pedantic)], so a pedantic lint is an error, not a warning — reproduce with the same command locally on the pinned toolchain."
	exit 1
	fi

	# Rustdoc's lints are a set no other step on this gate can reach, and
	# `rustdoc::broken_intra_doc_links` is the one that matters: only rustdoc
	# resolves a `[`Core::<F>`]` link, so only rustdoc can report that one
	# does not resolve. Measured, on a tree where the doc backend emitted 176
	# unresolvable links: regen-check green (the tree regenerates clean from
	# input), 534 doctests green, the generator's suite green, both clippy
	# steps green, and this step the only red. Everything else on this gate
	# is blind to it by construction.
	#
	# It belongs here, not only in the 5AM nightly, for the reason the whole
	# file exists: the nightly reports after merge, one branch too late. And
	# it is free — it documents a crate this job compiles anyway (~3s
	# locally), inside a job that finishes ~50s against regen-check's ~4m,
	# so the PR waits no longer than it does today.
	#
	# It sits ahead of the generated-crate clippy step rather than at the end
	# of the job so that it and the `cargo test --tests` step proposed for
	# this same job in #211 do not both append at the file's end, where
	# whichever lands second would conflict on nothing but position. The
	# ordering itself is free: doc-then-clippy and clippy-then-doc totalled
	# 17.3s/18.2s against 17.8s/18.0s locally, two runs each from a clean
	# target — indistinguishable.
	#
	# The blast radius is real and growing: the registry alone carries 176
	# generated links, and docs.rs is the first place rustdoc runs on a
	# release, where a broken link renders as literal text (#179 E4).
	- name: Rustdoc the generated crate
	shell: bash
	env:
	RUSTDOCFLAGS: "-D warnings"
	run: \|
	if ! cargo doc --no-deps -p ta-lib --manifest-path ta_codegen/output/rust/Cargo.toml; then
	echo "::error::Rustdoc failed on the generated crate. An 'unresolved link' names a [\`Core::<item>\`] that does not exist — fix the doc backend that emits it (ta_codegen/generator/src/backends/rust_doc.rs), never the generated file. Reproduce locally with RUSTDOCFLAGS='-D warnings' cargo doc --no-deps -p ta-lib --manifest-path ta_codegen/output/rust/Cargo.toml"
	exit 1
	fi

	- name: Clippy the generated crate
	shell: bash
	run: \|
	if ! cargo clippy --all-targets --manifest-path ta_codegen/output/rust/Cargo.toml -- -D warnings; then
	echo "::error::Clippy failed on the generated crate. Its scaffolding emits a crate-level #![allow(clippy::all, clippy::pedantic)], so a failure here is the emitter's, not the emitted code's — fix the backend, never the generated file."
	exit 1
	fi

	# The crate's hand-written test modules (templates/rust/, copied in by
	# `generate`) plus the generated registry's own sweeps: 85 tests today,
	# over types.rs's Core/CoreBuilder API (#144), scratch_election (#146),
	# stream_finite, stream_out_range (#241), div_zero (#249),
	# no_phantom_io (#265), tests/nullable_outputs.rs (#262),
	# tests/stream_open_contract.rs, and abstract_api's registry/binder
	# sweeps — the by-name ASCII fold from #278 among them.
	#
	# Nothing in this gate executed any of them. `clippy --all-targets` above
	# only COMPILES the test target, and `cargo test --doc` in the other job
	# skips it, so until now the only thing that ran them was the 5AM dev
	# nightly — after merge, the one-branch-too-late this file exists to fix
	# (#211).
	#
	# `--tests`, not `--lib`: the latter excludes tests/, and it selects the
	# lib's own unittest binary as well. Same argument the nightly records.
	#
	# It is in THIS job, not next to the doctests, for the reason clippy is:
	# the job finishes far inside regen-check's wall-clock, so the step costs
	# the PR nothing it was not already waiting on.
	- name: Run the generated crate's unit and integration tests
	shell: bash
	run: \|
	if ! cargo test --tests -p ta-lib --manifest-path ta_codegen/output/rust/Cargo.toml; then
	echo "::error::A test in the generated crate failed. These are the crate's own modules, not the cross-language value gates — a failure here is a contract the emitted code no longer keeps (registry lookup, stream range, div-zero, phantom I/O, Core/CoreBuilder). Reproduce with the same command on the pinned toolchain; fix the backend, never the generated file."
	exit 1
	fi

Navigation Menu

Uh oh!

fix(stoch,kama): divide by the value the guard tests, and bound the efficiency ratio (#390) #137

Workflow file

fix(stoch,kama): divide by the value the guard tests, and bound the efficiency ratio (#390) #137

Uh oh!

Workflow file for this run