JIT constants nested in wrapper expressions are live-rampable — reveal before substitute#416
Conversation
… reveal before substitute (#302) A constant UWexpression listed in constants_manifest could still be baked into the generated C: the lowering substituted constants[] placeholders with a TOP-LEVEL xreplace and only then unwrapped, so a constant hidden inside a nested UWexpression never met the substitution and was folded to a literal by the unwrap. Since every Parameters.* value is template- wrapped in a UWexpression, this froze EVERY constant inside constitutive parameter expressions (broader than the issue's xi case — the reproducer shows even the 'delta pattern' freezes when it arrives via a parameter): manifest said live, C said literal, and only a full rebuild propagated new values. It also explains why rebuilds 'worked': the cache key baked the same values, so a ramp changed the hash and forced recompilation. The lowering is now three-phase, identical in codegen and cache hashing (shared _reveal_constants): 1. expand non-constant UWexpressions to fixpoint, KEEPING truly-constant atoms symbolic (the predicate is the one the manifest scan uses, so the revealed set equals the manifested set by construction); 2. xreplace manifested constants with their constants[] placeholders; 3. resolve remaining UW atoms to values. Two loud guards make this class impossible to reintroduce silently: a truly-constant atom with no manifest slot, or a manifested constant that survives to the final expression, is a RuntimeError — never a silently frozen parameter. Nested-constant ramps no longer invalidate the JIT cache (the placeholder is hashed, not the value). Regression tests (tests/test_0103_jit_rampable_constants.py): top-level and nested constants both ramp via real solves with no rebuild; the ramped solution equals a from-scratch rebuild at the same value to 1e-10; ramping back restores the base solution exactly. Level_1+tier_a gate: 445 passed with the guards active on every kernel. Unblocks the model-owned-time design recorded on #410 (time as a constants[] atom is now guard-protected against silent freezing). Closes #302 Underworld development team with AI support from Claude Code
…ponse) The adversarial review found prepare_for_cache_key swallowed reveal failures into a silent fallback that would hash constant VALUES — re-creating the recompile-per-ramp behaviour for the failing expression while codegen raised. Same input now behaves identically on both paths. Also from the review, filed separately: #415 (pre-existing Max/Min(UWexpression, number) construction crash). Underworld development team with AI support from Claude Code
Adversarial review — round 1 (public record)Independent skeptic pass, live probes in the branch's own build. Instructed to REFUTE; could not. Attack 1 — sympy folding during the reveal phase: HELDEight nested-constant topologies, each ramped via .sym with REAL solves and checked against an independent literal-inlined rebuild (different kernel, separate compile): constant exponent over a field base; constant exponent with numeric base inside a wrapper; Integer*constant Mul auto-simplification; sqrt((T−1)²+δ²) regularisation; Piecewise on a field threshold; DOUBLE nesting; Max(constant, field); and Max(k1,k2) where the ramp switches the runtime branch (kernel evaluates fmax(constants[i],constants[j]) live). Every ramped answer matched truth to 1e−9. No silent bakes; the guards never needed to fire. Attack 2 — guard false positives: HELDtest_0103 + test_1018 (rotated free-slip, consistent-Newton power-law) + test_0817/0818 (ND) + test_0812 (units-active constants): 29 passed, zero guard errors. VEP stability subset: 2 passed. No spurious RuntimeError anywhere. Attack 3 — cache behavior: HELDFirst solve: exactly one new .so. Ramp nested constitutive constant: zero new .so, identical cache key, correct answer. Ramp a BC constant: same. Structural change: recompiles as it must. A fully-constant wrapper chain collapses onto the template constant's slot and inner ramps still propagate (values are re-packed per solve). Attack 4 — consistent Jacobian: HELDPower-law config with consistent_jacobian=True: ramping BOTH a nested amplitude and a nested EXPONENT (the historically-broken family), no rebuild — solution and Newton iteration count bit-identical to a fresh rebuild at the same values (vmean 0.00065017399 both, its=6 both). Attack 5 — performance: HELD20-deep nested chain (20 manifested constants): setup 0.10s / first solve 4.19s vs development's 0.26s / 4.24s. No regression. Attack 6 — BC and boundary kernels: HELDConstants in Dirichlet expressions and natural-BC fluxes are manifested, ramp live without recompile, and match truth (natural-flux ramp: exactly 3x on a linear problem). bd_residual/bd_jacobian slots get the same reveal. Findings → responses
Verdict: MERGE.Underworld development team with AI support from Claude Code |
|
Post-rebase verification (onto development at #404): level_1+tier_a gate 449 passed, 0 failures on a clean rebuild — including #404's four new boundary-flux/topography tests running against this branch's lowering. (An intermediate gate run showed 4 failures that traced to a half-built environment from a timeout-killed build — old compiled extension without the #412 insert paired with post-rebase sources; Underworld development team with AI support from Claude Code |
Closes #302. The prerequisite for the model-owned-time design recorded on #410.
The bug — broader than reported
A constant
UWexpressionlisted inconstants_manifestcould still be baked into the generated C as a literal. The lowering substitutedconstants[]placeholders with a TOP-LEVELxreplaceand only then unwrapped — so a constant hidden inside a nestedUWexpressionnever met the substitution, and the unwrap folded it to a number. Since everyParameters.*value is template-wrapped in aUWexpression, this froze every constant inside constitutive parameter expressions, not just the issue's ξ case: the reproducer shows even the 'δ pattern' freezes when it arrives through a parameter (ramping changed nothing; a rebuild changed the answer x10). It also explains why rebuilds 'worked': the cache key baked the same values, so a ramp changed the hash and forced the recompile a live constant exists to avoid.The fix
Three-phase lowering, identical in codegen and cache hashing (shared
_reveal_constants): (1) expand non-constantUWexpressions to fixpoint while KEEPING truly-constant atoms symbolic — the predicate is the same one the manifest scan uses, so the revealed set equals the manifested set by construction; (2)xreplacemanifested constants →constants[]placeholders; (3) resolve remaining atoms. Two RuntimeError guards make silent baking structurally impossible: a truly-constant atom with no manifest slot, or a manifested constant surviving to the final expression, refuses loudly. Nested-constant ramps no longer invalidate the JIT cache.Adversarial review (public, posted below) — all six attacks HELD
Eight nested topologies probed with real solves against independent literal-rebuild ground truths — constant exponents (the known sibling gotcha),
sqrt(f² + δ²),Piecewise,Maxincluding a ramp that switches the runtime branch, double nesting: every ramp matched truth to 1e-9, zero silent bakes. Guards: zero false positives across rotated free-slip consistent-Newton, ND poisson/stokes, units-active constants, and a VEP subset (31 tests green). Cache: ramp → 0 new .so, structural change → recompiles. Consistent-Jacobian: ramped nested amp AND exponent give bit-identical solution and iteration count vs a rebuild. Performance: 20-deep nesting, no measurable cost vs development. The review's one in-diff finding (a silent fallback in the cache-key path that would have re-created recompile-per-ramp) is fixed in the response commit; its pre-existing discovery is filed as #415 (Max/Min(UWexpression, number)construction crash).Verification
tests/test_0103_jit_rampable_constants.py): top-level and nested constants ramp via real solves with no rebuild; ramped == rebuilt to 1e-10; ramp-back restores base exactly.Manual close on merge: #302.
Underworld development team with AI support from Claude Code