test(codegen): the sweep sees an inline product as a divisor (#395) - #396
Merged
Merged
Conversation
) TA-Lib#395 reports CCI and CORREL returning TA_SUCCESS with inf/NaN, and notes the divisor sweep does not catch them. It does not, and the reason is not the one the issue gives. The sweep does not match `Expr::Var` on the denominator -- it iterates `names(den)`, every variable the denominator mentions. What it then does wrong is treat "a variable INSIDE the divisor is guarded" as "the divisor is guarded". CCI guards `tempReal2` and divides by `0.015*tempReal2`; CORREL guards `ssX` and `ssY` separately and divides by `sqrt(ssX*ssY)`. Both read as guarded and neither is: the scaling underflows to exactly 0.0 while every operand is still non-zero. Adds a third arm for that: a divisor that is an inline product or quotient (through `sqrt`/`fabs`, which carry their argument's zeroness), with no guard naming the expression itself, but with some guard attempting a bound on an operand. Operand detection here is deliberately looser than the other arms'. CORREL's guard is `ssX > 1e-14*sumX2`, a relative band rather than a literal floor, and the strict test rejects it -- correctly for the arms where soundness is the question, wrongly for this one, where the question is only whether someone tried. The strict test stays where the ER self-test pins it. WILLR, the issue's third function, is NOT covered and should not be: its divisor is `Var("diff")`, it guards `diff` itself, and its problem is that an exact `!= 0.0` is too weak for sub-epsilon residue -- the TA-Lib#107 band-strength question, not this one. Extending this arm to reach it would mean flagging correctly-targeted guards. Beyond the two the issue names, the arm flags seven more with the same shape: the Hilbert family guards `Im != 0.0 && Re != 0.0` and divides by `atan(Im/Re)*rad2Deg`, where a small enough quotient underflows and `atan(0.0)` is exactly 0.0. Recorded as shape-matched with reachability UNPROVEN -- I did not construct an input that reaches it.
kevinlincg
force-pushed
the
issue-395-inline-divisor
branch
from
September 6, 2026 16:02
1eb2965 to
ad5918a
Compare
mario4tier
added a commit
that referenced
this pull request
Sep 6, 2026
…%R's range a guarantee (#395) The three remaining functions carrying #390's shape: a guard that establishes one expression while the arithmetic below it divides by another. All three are reachable through the shipped batch API and all three return a wrong answer under TA_SUCCESS. CCI guards `tempReal2` and divided by `0.015*tempReal2`. The band is RELATIVE and the underflow is ABSOLUTE, so they cross: below |average| ~ 1.6e-308 the guard still answers "not flat" while the scaled copy is exactly 0.0. Dividing by the deviation and scaling after removes the mismatch structurally. It also fixes the silent half, which is larger than the loud one -- the pre-scaled form loses precision long before it reaches Inf, measured relative error 7.7e-08 at price scale 1e-314 rising to 4.8e-01 at 1e-321, against 2e-16 for the new form at every one of those magnitudes. CORREL guards ssX and ssY separately -- correctly, and correl.c explains why -- and divided by `sqrt(ssX*ssY)`. The product carries the fourth power of the window spread, so it leaves the double range at both ends while both factors are still ordinary normals with all 53 bits, and no test of the factors can see it. Where spXY is zero the result is NaN, which the [-1,1] clamp does not catch (`NaN > 1.0` is false); where it is not, the +-Inf is rewritten to exactly +-1.0, a perfect correlation reported from a degenerate window. A root of each factor cannot do either: sqrt maps any finite positive double into [2.3e-162, 1.4e154], so their product is representable at both ends. That also deletes the comment block arguing for the one-root form, whose overflow reasoning the tree's own oracle (ta_test_reference.c) already contradicted. WILLR is two defects, and the maintainer's ruling on the second is on #395. It still carried the exact `diff != 0.0` that #107 replaced in STOCH, so a machine-flat window divided sub-epsilon residue into full-scale noise, and a denormal range underflowed `diff` to 0.0 and answered 0 -- the value meaning a close at the period HIGH -- for a close sitting on the period LOW. Separately, the pre-scaled divisor cost the endpoint exactness that keeps %R in its documented range: the shipped library returns -100.00000000000001 on the reference series at periods 2, 3 and 4. Dividing by the range and scaling after makes the bound a theorem for well-formed input (fl(h-c) <= fl(h-l) by monotonicity, so the quotient is in [0,1] under every rounding mode); the clamp extends it to input TA-Lib does not validate, a close outside its own bar. `diff` leaves the body and the streaming handle of all four backends. Not scoped here, and checked rather than assumed: an exhaustive scan of every division in ta_codegen/input finds no fourth instance of the shape. The Hilbert family matches it lexically and is cleared -- `atan(Im/Re)*rad2Deg` can only reach +-Inf, which the `[0.67*prev, 1.5*prev]` then `[6,50]` clamp absorbs on the next statement -- so #396's seven rows are rewritten from "reachability unproven" to that clearance. The sweep from #396 now has no live instance to point at, so its self-test reconstructs CCI's pre-#395 divisor instead of reading a shipped defect, and pins the third half the other arms pin: silent when nothing bounds an operand. Two soundness fixes on the arm: its whole-divisor test accepted any operator, and its equality arm lacked the exclusivity its siblings carry. CORREL's fixed divisor is annotated rather than special-cased, which makes reverting correl.c:236 fail twice -- the key goes stale and the new divisor arrives unannotated. Measured, not assumed. LEGACY_TOL instrumented to record every function in one run: CCI 1.42e-14 and WILLR 7.11e-15 are new rows at 5e-14 and 3e-14; CORREL's 3.15e-13 is unchanged, so its #242 row stands. --fuzz-064 gives WILLR its own 5e-14 absolute bound rather than leaving it in the FMA bucket, whose 1e-9 is five orders looser than it needs -- the reason #338 gives for naming ATR. Absolute because %R is a bounded dimensionless oscillator: its floor is a ULP of 100 at any input magnitude. Six new legs, each sabotage-proven against a build of the pre-fix arithmetic: CCI subnormal (was +-Inf, now exactly +-1/0.015 -- period 2 makes the mean deviation equal the numerator, so the expectation is an equality); CORREL at 2^-283 (was NaN, -1, +1; now 0 and +-1/sqrt(15)); and WILLR's endpoint, range, denormal window, clamp domain (was +-10000) and machine-flat window. Green: full C suite, generator cargo test and clippy -D warnings, Rust clippy/doc/doctests/tests, --codegen against the frozen oracle (161 passed 0 failed in each of the four backends, CCI and WILLR both value-compared), --xlang-hash (338,957 golden cases, four backends bit-identical), --fuzz-064 (0 failures), regen-check, and the four source/lock/retcode/candle checks. Shipped-build A/B on a Zen 4 laptop, min of 12 alternating rounds x 200 reps, 20k bars, four untouched controls inside +-2%: WILLR batch 0.917 (the batch tier LOSES a division), WILLR streaming 1.054, CCI 1.115, CORREL 1.142. The streaming +5% is the ALT1 tier paying per bar for a hoist it used to amortise, the same order as KAMA's clamp in #390. CCI's and CORREL's costs are one extra divide and one extra sqrt per bar; both were checked for a cheaper correct form and neither has one that does not reintroduce an overflow band or retighten a guard #253 deliberately widened. CORREL's is well under the ~25% the deleted comment claimed. Closes #395. Claude-Session: https://claude.ai/code/session_01GesMuivQUwX4xHr4hEtnS8
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the sweep half of #395. It does not move CCI, CORREL or WILLR.
Correcting the stated cause first
#395 says the sweep misses these because
scan_expr'sif let Expr::Var(v) = &**dennever matches an inline expression. There is no such line — the sweep iterates
names(den), every variable the denominator mentions, so it does look inside0.015*tempReal2.The actual defect is one step further in: it treats "a variable inside the divisor is
guarded" as "the divisor is guarded". Both functions read as guarded and neither is.
That matters because it changes the fix — matching
Expr::Varon the denominator wouldnot have helped.
The third arm
A divisor that is an inline product or quotient (seen through
sqrt/fabs, which carrytheir argument's zeroness), with no guard naming the expression itself, but with some
guard attempting a bound on an operand.
Operand detection here is looser than the other arms use, on purpose. CORREL's guard is
ssX > 1e-14*sumX2— a relative band, not a literal floor — and the stricttests_var_against_zerorejects it. That strictness is right where soundness is thequestion and wrong here, where the question is only whether someone tried to bound the
operand. The strict test stays exactly where
the_sweep_detects_a_reintroduced_er_defectpins it.
Pinned in the direction that matters, as with the other two arms:
the_inline_arm_clears_when_the_guard_names_the_divisorrewrites CCI's guard onto0.015*tempReal2and requires the finding to disappear, so an arm that flagged everyinline product unconditionally would fail it.
WILLR is not covered, and should not be
#395 groups it with the other two; it is a different defect. Its divisor is
Var("diff"), not an inline expression, and it guardsdiffitself — the right target.Its problem is that an exact
!= 0.0is too weak for sub-epsilon residue, which is the#107 band-strength question. Stretching this arm to reach it would mean flagging guards
that name their divisor correctly, which is the one thing the arm exists to distinguish.
Seven more with the same shape, reachability unproven
Beyond CCI and CORREL, the arm flags the Hilbert family — HT_DCPERIOD, HT_DCPHASE,
HT_PHASOR, HT_SINE, HT_TRENDLINE, HT_TRENDMODE and MAMA. All guard
Im != 0.0 && Re != 0.0and then divide byatan(Im/Re)*rad2Deg(
ht_dcperiod.c:318-319and its twins). Both operands being non-zero does not make thequotient representable: a small enough
Im/Reunderflows to 0.0, andatan(0.0)isexactly 0.0.
That is a structural argument, not a measurement. I did not construct an input that
reaches it, and #395 does not mention these functions, so the rows say "shape matches"
and nothing stronger. If you would rather they were not recorded until someone
reproduces one, say so and I will drop them.
All nine annotated OPEN, none patched
Moving these guards changes output on the affected windows — a decision about shipped
functions, the same line drawn for KAMA in #383/#385 and STOCH in #392.
#395alsonotes CCI and WILLR are pre-cutover, so
--codegencompares them against the frozenoracle and
LEGACY_TOLmay need re-measuring; that is part of the fix, not of the sweep.no_annotation_is_stalecovers the follow-through: when a guard does move, its row stopsbeing flagged and the suite says so.
Verification
cargo clippy --all-targetsclean, 938 generator tests green,ta_regtestgreen. Onefile changed; no generated source touched.