Skip to content

Fix lambdify cache-miss on evaluate() (closes #194) + replace flaky timing tests#193

Open
lmoresi wants to merge 2 commits into
developmentfrom
bugfix/lambdify-caching-cachehit
Open

Fix lambdify cache-miss on evaluate() (closes #194) + replace flaky timing tests#193
lmoresi wants to merge 2 commits into
developmentfrom
bugfix/lambdify-caching-cachehit

Conversation

@lmoresi
Copy link
Copy Markdown
Member

@lmoresi lmoresi commented May 18, 2026

Summary

Two coupled changes in the lambdify-evaluate caching area:

  1. Fixes uw.function.evaluate() never hits the lambdify cache (fresh sympy.Dummy per call → +1 cache entry every call) #194uw.function.evaluate() never hit the lambdify cache. _expr_hash used sympy.srepr(expr), which embeds the volatile global dummy_index of any sympy.Dummy. The evaluate() coordinate-substitution path mints a fresh Dummy per call, so an otherwise-identical expression hashed differently every call → _lambdify_cache grew one entry per call (1,2,3,4,…) and sympy.lambdify recompiled every time on a hot path.

    Fix: canonicalise Dummy → Symbol(d.name) in _expr_hash before srepr. This changes only the cache key; the real sympy.lambdify() call still uses the original expr/symbols, so numerics are unchanged. The cache key separately carries the symbol-name tuple, so name-keying is safe and deterministic.

  2. Replaces the flaky wall-clock tests in TestPerformanceExpectations (the original motivation). test_lambdify_caching's assert time2 <= time1*2 was failing CI on unrelated PRs (e.g. Refined-submesh (coarse/fine companion) investigation #192) from pure runner jitter, and its loose tolerance was masking bug uw.function.evaluate() never hits the lambdify cache (fresh sympy.Dummy per call → +1 cache entry every call) #194.

Tests

Validation

Test plan

Closes #194.

Underworld development team with AI support from Claude Code

test_lambdify_caching and test_rbf_false_not_slow asserted one-off
wall-clock timings (time2 <= time1*2, elapsed < 0.01s). These are
inherently flaky on shared CI runners and test_lambdify_caching has
been failing CI on unrelated PRs (0.0107s vs 0.0049s runner jitter).

Replaced with deterministic, timing-free behaviour tests:

- test_lambdify_cache_hit: exercises get_cached_lambdified directly --
  identical request returns the SAME function object and adds no entry
  (true cache hit); a distinct expression is cached separately. This is
  the cache mechanism's actual contract.
- test_rbf_modes_consistent: rbf=True and rbf=False must agree for a
  pure-sympy expression (meaningful, timing-free; replaces the
  "rbf=False not slow" wall-clock check).

Note: writing the cache-hit test surfaced that the high-level
uw.function.evaluate() path does NOT hit the lambdify cache on repeated
identical calls -- _expr_hash(srepr) differs every call, so the cache
grows by one entry per call and never returns a hit. The old loose
timing tolerance was masking this. Not fixed here (evaluate/lambdify is
a performance-critical hot path needing separate benchmarking); see PR
description.

Full module: 19 passed.

Underworld development team with AI support from Claude Code
Copilot AI review requested due to automatic review settings May 18, 2026 12:36
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR replaces flaky wall-clock assertions in lambdify optimization tests with deterministic behavioral checks for cache reuse and evaluation consistency.

Changes:

  • Replaces timing-based lambdify cache test with a direct get_cached_lambdified cache-contract test.
  • Replaces rbf=False timing assertion with a consistency check between rbf=True and rbf=False.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

_expr_hash used sympy.srepr(expr), which embeds the volatile global
dummy_index of any sympy.Dummy. The evaluate() coordinate-substitution
path mints a fresh Dummy per call, so an otherwise-identical expression
hashed differently every call and the lambdify cache never matched --
_lambdify_cache grew one entry per call (1,2,3,4,...) and sympy.lambdify
recompiled every time on a hot path.

Fix: canonicalise Dummy -> name-stable Symbol in _expr_hash before
srepr. This changes only the cache *key*; the real sympy.lambdify()
call still uses the original expr/symbols, so numerics are unchanged.
The cache key separately carries the symbol-name tuple, so name-keying
is safe and deterministic.

Verified: repeated identical evaluate() now holds cache size flat
([1,1,1,...] vs [1,2,3,...] before). test_0720 module 19 passed;
test_0501_integrals 9 passed / 3 pre-existing xfail (unrelated
CellWiseIntegral #172/#174).

test_evaluate_cache_stable_across_calls added as the #194 regression
guard (aggregate cache behaviour + result-consistency, no wall-clock).

Underworld development team with AI support from Claude Code
@lmoresi lmoresi changed the title Replace flaky wall-clock lambdify tests with cache-contract tests Fix lambdify cache-miss on evaluate() (closes #194) + replace flaky timing tests May 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants