intake: two prompts from the workspace smoke-gate slowdown investigation - #273
Merged
Conversation
Investigation found the PyAuto smoke gates roughly tripled between mid-July and late August 2026, and that the cause is the smoke *lists*, not the libraries — library import time went the other way (autolens 4.07s -> ~1.25s on 2026-08-19). Measured, py3.12 leg (the critical path), from the CI job logs: autolens_workspace_test run 32605025472 23 entries 553.0s autolens_workspace run 32608556415 37 entries 351.0s Both reconcile to their step wall-clock, so the scripts are the whole cost. The two repos are slow for opposite reasons and get one prompt each: - draft/test/workspaces/slowest_smoke_gate_scripts.md — _test is three scripts (subhalo_recovery 120.7s, database/scrape 63.4s, point_source/jax_likelihood/point 47.7s = 42%); speed up or demote. - draft/test/pyautoheart/smoke_relevance_gate.md — the workspace has no slow scripts at all (max 23.2s, median 9.4s), so the only lever is running the gate less often. Extend Heart's docs-only gate in smoke-tests.yml into a relevance gate. Constraints recorded in both prompts rather than left to be rediscovered: the vacuous-assertion hazard (three prior instances in this repo), and that Heart's readiness reads Smoke Tests on main HEAD with `cancelled` counting as failure — so PR-side narrowing is free but the main-push run is not. Dashboard regenerated (152 prompts). Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_015Mw2KC1M1HKrvTqe5zcEZn
Swept every repo that calls Heart's reusable smoke workflow. One finding
changes the prompt: autogalaxy_workspace_test is a second ~11-minute gate
(557.1s / 37 entries, run 32533004337) that nobody had measured.
It matters because it is the mirror image of autolens_workspace_test —
slowest entry 40.0s, median 13.5s, top-3 only 21% — so the speed-up prompt
has nothing to bite on there and this gate is its only lever. 25 of its 37
entries are a {imaging, interferometer, multi_dataset} x jax_{likelihood,grad}
x {lp, mge, mge_group, rectangular, rectangular_mge, delaunay, delaunay_mge}
cross-product, which tier 2's package narrowing maps onto directly.
Recorded a negative result too: its six jax_grad/ entries were the obvious
suspect (Heart budgets that class at 1800s for running full-resolution) and
measured 86.4s total, 15.5%. The expensive jax_grad scripts are autolens's
and are in no smoke list.
Package table extended to three repos; Repos: gains autogalaxy_workspace_test.
Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_015Mw2KC1M1HKrvTqe5zcEZn
…ries Checked whether the env profile is actually being applied, since "smoke" that takes 11 minutes suggests a misconfiguration. It is not one. autolens_workspace is correct: every entry that reports a mask ran at 80-208 pixels, and its profile additionally sets SKIP_FIT_OUTPUT / SKIP_VISUALIZATION / SKIP_CHECKS. Its two `ENV: full_datasets` smoke entries are guides/galaxies and guides/modeling/cookbook, the two CHEAPEST in the suite at 3.6s and 3.1s. autolens_workspace_test has had PYAUTO_SMALL_DATASETS in its defaults since the profile was created on 2026-04-08. What grew is the exemptions: 10 profile overrides (2026-04-30) -> 23 (05-28) -> 27 (07-23), then migrated in-file by PyAutoHands#187. Today 16 of 23 live entries declare `ENV: [jax] full_datasets` and a 17th is exempted by the surviving override — 516.5s of 553.0s, 93% of the gate, uncapped. Measured masks: subhalo_recovery 5858px, database/scrape 2828px, jax_likelihood 716-952px, vs the workspace's 80-208px. The exemptions are load-bearing, not drift: the jax_likelihood scripts assert hardcoded full-resolution likelihood literals at rtol 1e-4, so capping the dataset fails the assertion (recorded in sph-transform-name-check.md and mge-sigma-min-workspace-sweep.md). So the prompt's real question is not "why is the cap off" but "should the per-PR gate run full-resolution parity assertions at all", with the re-pinning cost named (autolens_workspace_test#257 had to regenerate those literals once already, blocking a nightly release). Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_015Mw2KC1M1HKrvTqe5zcEZn
…rformance-e19kk1 # Conflicts: # dashboard.html
The merge commit regenerated dashboard.md/.html from a PyAutoBrain checkout that was 5 commits behind origin/main, three of which changed the dashboard generator (#249/#250/#251 — the Recent-feed work that PyAutoMind #270/#271/#272 landed). That reverted the paged Recent feed: dashboard.html lost 119 lines. PyAutoBrain updated to ad927eb and both files regenerated. The diff against main is now +24/-6 — the two new prompts entering the backlog, and nothing else. `intake dashboard --check` reports current. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_015Mw2KC1M1HKrvTqe5zcEZn
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Files two PyAutoMind prompts (plus the regenerated dashboard) from an investigation into why the workspace smoke gates got slow. Prompts only — no code or CI change here.
What the investigation found
The gates roughly tripled between mid-July and late August 2026, and the cause is the smoke lists, not the libraries. Library import time moved the other way over the same period (
import autolens4.07s → ~1.25s on 2026-08-19), which was partly masking the growth.Measured from the CI job logs, py3.12 leg (the critical path). Both totals reconcile to their step wall-clock to the second, so the scripts are the entire cost:
autolens_workspace_testautogalaxy_workspace_testautolens_workspaceThe repos are slow for opposite reasons, so they get one prompt each.
The two prompts
draft/test/workspaces/slowest_smoke_gate_scripts.md—autolens_workspace_testis three scripts:imaging/subhalo_recovery.py(120.7s),misc/database/scrape/general.py(63.4s),point_source/jax_likelihood/point.py(47.7s) = 42% of the run. Speed up or demote, per script.draft/test/pyautoheart/smoke_relevance_gate.md—autolens_workspacehas no slow scripts at all (max 23.2s, median 9.4s), so the only lever is running the gate less often. Extends Heart's existing docs-only gate insmoke-tests.ymlinto a relevance gate.autogalaxy_workspace_testis the same shape (slowest entry 40.0s, top-3 only 21%) and folds into the same change.Constraints recorded rather than left to be rediscovered
latent_nan_robustnesspassing vacuously under the smoke profile).main-push run.required_workflowsreadsSmoke TestsonmainHEAD andcancelledcounts as failure — so narrowing PR-side runs is free, but skipping or cancelling themainrun is not.on.pull_request.paths. A job skipped by a path filter never reports a conclusion, so a required check sits pending forever. The in-workflowchangesjob emits a realskipped.PYAUTO_SMALL_DATASETShas been in_test's defaults since the profile was created (2026-04-08). What grew is the exemptions: 10 profile overrides (2026-04-30) → 23 (05-28) → 27 (07-23), then migrated in-file by PyAutoHands#187. Today 16 of 23 live entries declareENV: [jax] full_datasetsand a 17th is exempted by the surviving override — 93% of the gate runs uncapped. Those exemptions are load-bearing: thejax_likelihoodscripts assert hardcoded full-resolution likelihood literals at rtol 1e-4, so capping the dataset fails the assertion.Also swept, not filed
Two findings are reported but deliberately left unfiled pending a decision:
run_smoke.pyignoressmoke_tests.txtand runs every script viarun_python.py. HowToLens went from ~2m30 to ~8–11m on 2026-08-04 via PR ci(arxiv-digest): earlier cron for reliable pre-wake UK delivery #66, "run every script in smoke, not a 6-of-40 allowlist" — a deliberate coverage decision, not drift.autocti_workspacehas no.github/directory at all, while Heart'srequired_workflows.workspacesexpectsSmoke Testsfor that group.🤖 Generated with Claude Code
https://claude.ai/code/session_015Mw2KC1M1HKrvTqe5zcEZn
Generated by Claude Code