diff --git a/active.md b/active.md index ab06b16d..e4b15fd1 100644 --- a/active.md +++ b/active.md @@ -24,3 +24,56 @@ separate /repo_cleanup sweep so a destructive branch delete never rides a code diff. - repos: - PyAutoHands: feature/hands-hygiene-leftovers + +## jax-grad-local-vs-ci-assertions +- issue: https://github.com/PyAutoLabs/autolens_workspace_test/issues/260 (issued 2026-08-22) +- session: https://claude.ai/code/session_01VEHLT33XpVcRt5YCJGLRMJ (web-github; no local worktree yet) +- status: library-dev (RECLASSIFIED from workspace-dev — the fix is upstream in PyAutoArray; + autolens_workspace_test needs no change and no tolerance change) +- worktree: ~/Code/PyAutoLabs-wt/jax-grad-local-vs-ci-assertions +- prompt: active/jax_grad_local_assertions_fail_but_pass_in_ci.md +- classification: workspace (single repo) — routes to /start_workspace +- strategy: investigate-first. Bug Agent: severity=critical, scope=single-repo, + type=wrong-result, confidence=LOW. Reproduce and confirm root cause BEFORE patching. + Brain sizing disagreement: declared medium, derived large. +- control: imaging/jax_grad/lp.py is the discriminator — the only script known to PASS in + CI and FAIL locally. Every A/B runs against it first (~41s). +- ROOT CAUSE FOUND 2026-08-22 (no laptop needed; findings on issue #260): + PyAutoArray `util/dataset_util.py:72` `should_simulate()` is existence-only and asymmetric — + it force-regenerates under PYAUTO_SMALL_DATASETS=1 but never under full_datasets. `dataset/**` + is gitignored, so CI always simulates fresh and CANNOT hit this; locally the dir persists and + is never refreshed. Any prior smoke run (SMALL_DATASETS=1 is the default for every OTHER + script) rewrites the FITS at 15x15, and the next jax_grad run silently loads them under + full-resolution settings. All three failures reproduced exactly from a clean checkout + (pixelization eager/jit matches the report to 11 s.f.; regularization's tolerance vector + matches exactly). Full chain proven: fresh=PASS -> one SMALL run -> stale=FAIL -> rm -rf =PASS. +- numpy FALSIFIED: lp.py byte-identical across numpy 2.2.6 / 2.4.6 / 2.5.2 (and across + 1-core vs 4-core). The likelihood runs through JAX/XLA; numpy only does the FD bookkeeping. +- NO tolerance change is warranted — all three asserts did their job on a genuinely invalid + dataset. assert_eager_jit_consistent's rtol=1e-10 is vindicated, not under-specified. +- FIX SHIPPED to a branch 2026-08-22: PyAutoArray `claude/jax-grad-assertions-ci-hontn3` @ 5ef6eea. + `should_simulate` now also regenerates on the small->full transition, inferring the regime from + data.fits's shape (the cap emits EXACTLY 16x16). Design went through an adversarial review that + corrected three things: no pixel-scale check (not in the FITS header), `==` not `<=` (the cap + cannot emit 12x12), and data.fits BY NAME not first-FITS-in-dir (PSFs are legitimately 11x11 and + dataset/cluster/test/psf.fits is 5760 B — a glob would have deleted real data every run). + Verified: control-tested, 1168 tests pass, 12 new tests over all four regime transitions, and + end-to-end — poison then full-datasets run now PASSES where it raised. Steady state 0.64 ms. + NO PR opened (not requested); branch is pushed and ready. +- SCOPE: imaging manifestation only. Point-source/weak-lensing (JSON, no FITS) and interferometer + (shape-invariant under the cap, so it fails SILENTLY) remain exposed; stated in the docstring. +- follow-ups filed (not absorbed): + - PyAutoNerves#153 — stamp the regime at the single FITS writer funnel (fitsable.py:89); the only + discriminant that can catch the silent interferometer case. Kept out of the fix deliberately: it + changes a header card on every FITS the stack writes (round-trip tests, file-hash pins). + - PyAutoArray#470 — separate live defect: the small-datasets branch rmtree's + dataset/point_source/simple, which is COMMITTED and allowlisted in .gitignore:13, replacing it + with capped-solver garbage. Recoverable via git checkout but violates the allowlist invariant. +- follow-up (separate defect, not this bug): autolens_workspace_test + `.github/scripts/smoke_install.sh:9` `pip install "jax<0.7" "jaxlib<0.7"` downgrades jax to + 0.6.2 and conflicts with autonerves' jax>=0.7,<0.11; the run only lands on the intended + 0.10.2 because the next line's [optional] extras pull it back up. CI is right by accident. +- out-of-bounds: moving lp.py's evaluation point, adding skip_indices, or widening a + tolerance without a measured basis. All three mask the trap instead of removing it. +- repos: + - PyAutoArray: claude/jax-grad-assertions-ci-hontn3 diff --git a/draft/bug/autolens_workspace_test/jax_grad_local_assertions_fail_but_pass_in_ci.md b/active/jax_grad_local_assertions_fail_but_pass_in_ci.md similarity index 100% rename from draft/bug/autolens_workspace_test/jax_grad_local_assertions_fail_but_pass_in_ci.md rename to active/jax_grad_local_assertions_fail_but_pass_in_ci.md diff --git a/dashboard.html b/dashboard.html index 00f7e868..243c1930 100644 --- a/dashboard.html +++ b/dashboard.html @@ -38,7 +38,7 @@

📋 PyAutoMind Dashboard

Every task the Mind is holding. Tap a task's 📋 and its /start_dev command is on your clipboard — paste it into a Claude Code chat to route Claude straight to that task.

-

In flight 2 · Parked 3 · Planned 6 · Backlog 144 · markdown version

+

In flight 3 · Parked 3 · Planned 6 · Backlog 143 · markdown version

Start here

Highest priority (filed as high) — showing 12 of 18

TRIAGE: needs manual review before routing — medium · safe · high

@@ -59,6 +59,7 @@

In flight Issued — each has an open GitHub issue and usually a branch.

@PyAutoFit TransformedMessage.factor_gradient crashes on first call — issue #1501 — HOLD — do not start dev. Fix-or-delete hangs off the PyAutoFit#1498 logpdf-contract

Hands hygiene: expired announcement dead code, unregistered modules — issue #249 — library-dev

+

jax_grad scripts fail assertions locally that PASS in CI — issue #260 — library-dev (RECLASSIFIED from workspace-dev — the fix is upstream in PyAutoArray;

Parked markdown version

3 task(s) @@ -77,7 +78,7 @@

Planned

latent-nan-guard-honest-run

Backlog markdown version

-

144 filed prompts, not started — sorted most-pickable first (priority, then size). 23 of them belong to an epic and are listed only under Epics below.

+

143 filed prompts, not started — sorted most-pickable first (priority, then size). 23 of them belong to an epic and are listed only under Epics below.

feature — 28

Numba CPU likelihood phase 1: batched MGE convolution + operated-matrix — autoarray · medium · supervised · high

@@ -110,7 +111,7 @@

Backlog

Teach repos_sync --write to stamp organ config surfaces — pyautomind · hard · supervised · low

-bug — 32 +bug — 31

Fix release JAX runtime compatibility and likelihood parity — health_fixes · too-large · supervised · high

Fix JIT quick-update visualization output regressions — health_fixes · too-large · supervised · high

Fix release-profile numerical inversion failures — health_fixes · too-large · supervised · high

@@ -118,7 +119,6 @@

Backlog

pixel_scales given as an int (or np.float64) is never widened — autoarray · small · supervised · medium

jax 0.11 breaks beta/gamma message log_partition under jit ('tuple' object — autofit · small · supervised · medium

Heart script_timing baselines are orphaned by path moves and filled — pyautoheart · small · supervised · medium

-

jax_grad scripts fail assertions locally that PASS in CI — autolens_workspace_test · medium · supervised · medium

Numba PSF gathers derive the y/x kernel shifts from the — autoarray · low · supervised · medium

PyNUFFT dev extra is incompatible with current SciPy on Python — autoarray · small · supervised · normal

LogGaussianPrior misreports its own support as (-inf, inf) — autofit · small · supervised · normal

diff --git a/dashboard.md b/dashboard.md index d1644d8e..86183dcd 100644 --- a/dashboard.md +++ b/dashboard.md @@ -8,10 +8,10 @@ Every task the Mind is holding, on one page: what is in flight, what is parked, | Where | Count | |-------|------:| -| [In flight](#in-flight) (`active/`) | 2 | +| [In flight](#in-flight) (`active/`) | 3 | | [Parked](#parked) (`parked.md`) | 3 | | [Planned](#planned) (`planned.md`) | 6 | -| [Backlog](#backlog) (`draft/`) | 144 | +| [Backlog](#backlog) (`draft/`) | 143 | ## Start here @@ -137,6 +137,14 @@ Issued — each has an open GitHub issue and usually a branch. The full record f

+
📋 jax_grad scripts fail assertions locally that PASS in CI — issue #260 — library-dev (RECLASSIFIED from workspace-dev — the fix is upstream in PyAutoArray; + +``` +/start_dev active/jax_grad_local_assertions_fail_but_pass_in_ci.md +``` + +
+ ## Parked Started or scoped, not currently in flight — resume by moving the row back to `active.md`. Full detail in [`parked.md`](parked.md). @@ -229,7 +237,7 @@ Scoped but not started; some are not yet prompt files. Full detail in [`planned. ## Backlog -**144** filed prompts, not started. Each section is sorted most-pickable first (priority, then size). **23** of them belong to an epic and are listed only under [Epics](#epics) below. +**143** filed prompts, not started. Each section is sorted most-pickable first (priority, then size). **23** of them belong to an epic and are listed only under [Epics](#epics) below.
feature — 28 @@ -461,7 +469,7 @@ Scoped but not started; some are not yet prompt files. Full detail in [`planned.
-bug — 32 +bug — 31
📋 Fix release JAX runtime compatibility and likelihood parity — health_fixes · too-large · supervised · high @@ -519,14 +527,6 @@ Scoped but not started; some are not yet prompt files. Full detail in [`planned.
-
📋 jax_grad scripts fail assertions locally that PASS in CI — autolens_workspace_test · medium · supervised · medium - -``` -/start_dev draft/bug/autolens_workspace_test/jax_grad_local_assertions_fail_but_pass_in_ci.md -``` - -
-
📋 Numba PSF gathers derive the y/x kernel shifts from the — autoarray · low · supervised · medium ```