refactor: collapse run_smoke.py onto the PyAutoHands runners (356 → 119 lines) - #147
Merged
Merged
Conversation
…119 lines) Phase 2 of PyAutoHands#260, and the last of the ten vendored copies. This file becomes a thin delegator over `autohands/run_python.py` (scripts) and `autohands/run.py` (notebooks) — the shape the HowTo repos have used all along, and the shape the four *_workspace_test repos took in phase 1. The copies were not merely duplicated, they were EXPENSIVE. Each of the last three fixes had to be swept across every copy by hand: the env-resolver fork (PyAutoHands#185), the per-script timeout and process-group kill (#226/#227), and the jupyter guard. The HowTo tier needed none of those sweeps, precisely because it holds no logic. The notebook leg is preserved in full, promoted into PyAutoHands rather than reimplemented: - `--no-write-back` executes a throwaway copy, so a smoke run never modifies the committed notebooks. The shared runner already pins the kernel's cwd to the repo root, which is what this file's staged-copy-at-root trick existed to achieve — so that trick is superseded, not ported. - `--retry-from scripts` keeps the regenerate-from-source recovery for a stale notebook. A TIMEOUT is never retried, a clean sys.exit(0) skip guard is already a PASS and never reaches it, and the retry's verdict replaces the first attempt's so one notebook yields one result. - A missing notebook toolchain stays one FAIL with the run continuing: the shared runner invokes `sys.executable run_notebook.py`, never a bare `jupyter`, so the abort-with-no-summary failure mode is structurally absent and JUPYTER_MISSING_RC is not needed. Behaviour preserved, verified rather than assumed: - Env resolution is IDENTICAL. The old runner passed a relative path to build_env_for_script, the shared runners pass an absolute one. Every listed script was resolved both ways and diffed: 0 differences, in env and in args. - No profile here sets per-script `args`, so extra-args support changes nothing. - Every entry in smoke_tests.txt and smoke_notebooks.txt resolves to a file that exists, and every listed notebook's source script exists for the retry path. - The exit code is the worst of the two legs, so a failing notebook cannot be masked by passing scripts. `config/build/no_run.yaml` is deliberately NOT applied to either allowlist. It is policy for the release mega-run and notebook generation; the smoke lists are policy for this gate, and an entry legitimately appears in both — measured, and fixed in PyAutoHands#262 before this landed. Requires PyAutoHands#261, #262 and #263, all merged. #263 matters here in particular: every listed notebook is in a subdirectory, and before it the regenerate-and-retry looked its source up by bare filename and never found it. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01UpSFum81Jeq9KZ9wdKtaeZ
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Phase 2 of PyAutoHands#260 — the last of the ten vendored copies. This file
becomes a thin delegator over
autohands/run_python.py(scripts) andautohands/run.py(notebooks): the shape the HowTo repos have used all along,and the shape the four
*_workspace_testrepos took in phase 1.The copies were not merely duplicated, they were expensive. Each of the last
three fixes had to be swept across every copy by hand — the env-resolver fork
(PyAutoHands#185), the per-script timeout and process-group kill (#226/#227),
the jupyter guard. The HowTo tier needed none of those sweeps, precisely because
it holds no logic.
The notebook leg is preserved in full
Promoted into PyAutoHands rather than reimplemented:
--no-write-back— executes a throwaway copy, so a smoke run nevermodifies the committed notebooks. The shared runner already pins the kernel's
cwd to the repo root, which is what this file's staged-copy-at-root trick
existed to achieve — so that trick is superseded, not ported.
--retry-from scripts— keeps the regenerate-from-source recovery for astale notebook. A TIMEOUT is never retried, a clean
sys.exit(0)skip guardis already a PASS and never reaches it, and the retry's verdict replaces the
first attempt's so one notebook yields one result.
runner invokes
sys.executable run_notebook.py, never a barejupyter, sothe abort-with-no-summary failure mode is structurally absent.
Behaviour preserved — verified, not assumed
old (relative) and new (absolute) path form and diffed: 0 differences.
notebooks' source scripts exist for the retry path.
be masked by passing scripts.
no_run.yamlis deliberately not applied — and this repo shows whyIt is policy for the release mega-run and notebook generation; the smoke lists
are policy for this gate. This repo has
searches/mcmc.pyin both itsallowlist and
no_run.yaml, and it runs in smoke today — as doessearches/mcmc.ipynb, whose source script is that same file. Filtering theallowlist by
no_runwould have silently dropped both. Measured and fixed inPyAutoHands#262.
Test Plan
leg, a stale nested notebook regenerating from its source and passing, the
committed notebooks byte-identical afterwards, and the exit code correct in
all three pass/fail combinations
*_workspace_testcollapses are merged and greenRequires PyAutoHands#261, #262 and #263, all merged. #263 matters here in
particular: both listed notebooks are in subdirectories, and before that fix
the regenerate-and-retry looked its source up by bare filename and never found it.
Generated by the PyAutoLabs agent workflow.
Generated by Claude Code