Skip to content

ci: one canonical Heart-owned arcticpy install action for every CTI repo - #171

Merged
Jammy2211 merged 1 commit into
mainfrom
feature/arcticpy-install-standardisation
Aug 24, 2026
Merged

ci: one canonical Heart-owned arcticpy install action for every CTI repo#171
Jammy2211 merged 1 commit into
mainfrom
feature/arcticpy-install-standardisation

Conversation

@Jammy2211

Copy link
Copy Markdown
Contributor

Closes #170 (Heart's leg — the four consumer PRs are listed below).

import autocti hard-requires arcticpy, which is not a pip dependency: source-only C++ sdist, libgsl-dev + a toolchain to build, and its own requirements downgrade numpy below 2.0. The install recipe had drifted into four shell copies with no owner. This adds the canonical step and repoints the two copies that live here.

What's here

.github/actions/install-arcticpy/ — the canonical step, and the single arcticpy pin (input version, default 2.6). Bumping arcticpy is now a one-line change instead of four. Deliberately self-contained: it upgrades pip/setuptools/wheel itself rather than relying on a caller having done so first.

.github/workflows/arcticpy-action.yml + .github/scripts/arcticpy_smoke.py — self-test. The action is consumed cross-repo at @main, so a recipe change would otherwise reach four repos with nothing having exercised it. This job references the action by local path (so it builds the branch's version) and doesn't stop at "pip exited 0" — it clocks a single bright pixel through arctic and asserts a decaying trail with charge conserved.

.github/workflows/lib-tests.yml — both inline copies (jobs unittest and unittest-nojax) replaced by the action, keeping the inputs.package == 'autocti' gate.

.github/workflows/smoke-tests.yml — new arcticpy boolean input (default false) running the action before the workspace epilogue, so autocti_workspace_test can drop its copy. Default-false leaves every non-CTI caller byte-identical.

Two things the specified recipe got wrong

Both caught by building arcticpy 2.6 in a clean container and actually running it, not by reading:

  1. --no-deps suppresses arcticpy's runtime dependencies too. arcticpy/read_noise.py imports scipy and matplotlib at import time, and __init__.py imports read_noise. Installing only the build deps leaves import arcticpy raising ModuleNotFoundError: No module named 'scipy'. The issue's recipe listed scipy in its pip line but the prose only called out matplotlib; the action installs both.
  2. arcticpy exposes no __version__ attribute. The assertion the task specifies — import arcticpy; print(arcticpy.__version__) — raises AttributeError on a perfectly good install. The action reads importlib.metadata instead. (The same broken command was already documented in autocti_assistant/skills/ac_setup_environment.md; fixed in that repo's PR.)

Audit correction

The issue flagged PyAutoCTI's own lib-tests CI as UNREAD. Audited: PyAutoCTI/.github/workflows/main.yml is a thin caller of this repo's lib-tests.yml and carries no recipe of its own — so the copies were 2 (here) + 1 (workspace_test) + 1 (assistant) = four, not two.

Also: smoke_install.sh was not one runner-image change from breaking. smoke-tests.yml runs pip install --upgrade pip setuptools wheel immediately before invoking the epilogue, so setuptools was present. The real defect was weaker but still worth removing — the epilogue's correctness depended implicitly on a step in another repository's workflow, with nothing stating the dependency.

Verification

Built arcticpy 2.6 from a bare venv in a clean container:

  • Negative case reproduced — with setuptools uninstalled, the build dies at BackendUnavailable: Cannot import 'setuptools.build_meta'. Python 3.12+ venvs no longer ship setuptools, so the omission was a real hazard.
  • Canonical recipe buildsSuccessfully installed arcticpy-2.6.
  • arcticpy_smoke.py passes against that build, exactly as committed: bright pixel 1000.0 → 997.24, trail [1.18, 0.78, 0.51, 0.33, 0.21] (decaying), charge conserved to 5.8e-4.
  • The action's version command returns 2.6.
  • All five changed/added YAML files parse; the smoke script compiles.

Writing the self-test is what surfaced both recipe bugs above — the first draft of it failed locally on ModuleNotFoundError: scipy, then on AttributeError: __version__, then on two API mistakes of my own (add_cti needs a CCD, not a CCDPhase, and parallel_roe is not optional in practice).

Merge order

This PR must merge first. The four consumer PRs reference the action at PyAutoLabs/PyAutoHeart/.github/actions/install-arcticpy@main, so their CI cannot pass until the action exists on main here.

Deliberately not done

Caching the built wheel (actions/cache keyed on arcticpy version + Python + runner image). arcticpy compiles from source on every CI run in every CTI repo, so there is real time to reclaim — but the task asks for a measured proposal with before/after timings rather than an assumed win, and cache invalidation is a genuine cost. Left as a follow-up rather than guessed at.


Generated by Claude Code

`import autocti` hard-requires arcticpy, which is not a pip dependency: its
sdist is source-only C++ (libgsl-dev + a toolchain + Cython) and its own
requirements downgrade numpy below 2.0. The install recipe had drifted into
four separate shell copies with no owner. This adds the canonical step and
repoints the two copies that live here.

.github/actions/install-arcticpy/
  The canonical step, and THE single arcticpy pin (input `version`, default
  2.6) — bumping it is now a one-line change instead of four. Deliberately
  self-contained: it upgrades pip/setuptools/wheel itself rather than relying
  on a caller having done so in a preceding step, which is what
  autocti_workspace_test's epilogue was silently doing across a repo boundary.

Two things the recipe as previously specified got wrong, both caught by
building arcticpy 2.6 in a clean container and running it:

  * `--no-deps` suppresses arcticpy's RUNTIME dependencies too, and
    arcticpy/read_noise.py imports scipy and matplotlib at import time (via
    __init__.py). Installing only the build deps leaves `import arcticpy`
    raising ModuleNotFoundError, so the action installs scipy + matplotlib as
    well. The CTI stack pulls both in anyway; naming them here is what makes
    the step verifiable standing alone.
  * arcticpy exposes NO __version__ attribute. The specified assertion
    `import arcticpy; print(arcticpy.__version__)` raises AttributeError on a
    perfectly good install. The action reads importlib.metadata instead.

Without setuptools the build dies at `BackendUnavailable: Cannot import
'setuptools.build_meta'` — reproduced here, and Python 3.12+ venvs no longer
ship setuptools, so its omission was a real hazard.

.github/workflows/arcticpy-action.yml + .github/scripts/arcticpy_smoke.py
  Self-test. The action is consumed cross-repo at @main, so a recipe change
  would otherwise reach four repos with nothing having exercised it. This job
  references the action by LOCAL path (so it builds the branch's version) and
  does not stop at "pip exited 0": it clocks a single bright pixel through
  arctic and asserts a decaying trail with charge conserved. Verified locally —
  1000.0 -> 997.24 in the bright pixel, trail [1.18, 0.78, 0.51, ...].

.github/workflows/lib-tests.yml
  Both inline copies (jobs `unittest` and `unittest-nojax`) replaced by the
  action, keeping the `inputs.package == 'autocti'` gate. This is the path
  PyAutoCTI's own CI takes — its main.yml is a thin caller of this workflow and
  carries no recipe of its own.

.github/workflows/smoke-tests.yml
  New `arcticpy` boolean input (default false) running the action before the
  workspace epilogue, so autocti_workspace_test can drop its copy. Default-false
  leaves every non-CTI caller byte-identical. Gated exactly as lib-tests.yml
  already gates its own arcticpy step.

Refs #170

Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_018nDAxBEavkzb6Zkz1cYHef

Copy link
Copy Markdown
Contributor Author

The caching question, answered with a measurement

The task asked for a measured proposal rather than an assumed win. The self-test job gives that measurement for free — GitHub's own per-step timings from run 32769526194, job build (3.12):

Step Duration
Install GSL headers (apt-get) 15.9 s
Build dependencies (pip, setuptools/wheel/numpy/cython) 9.0 s
Runtime dependencies (pip, scipy/matplotlib) 9.9 s
Build and install arcticpy 2.6 16.8 s
Verify import 0.8 s
Total action ~52 s

Within that 16.8 s, the actual C++ compile — Building wheel for arcticpy … finished — is ~14 s (19:41:20.7 → 19:41:34.7). The rest is sdist download and metadata prep.

Recommendation: don't cache the wheel. The thing a cache would eliminate is ~14 s. Against that:

  • actions/cache restore + save is itself a few seconds, so the net saving is smaller than 14 s.
  • The cache key would need arcticpy version + Python minor + runner image, and a stale-key miss silently costs more than no cache at all.
  • The larger costs in this step are apt-get (15.9 s) and the pip downloads (18.9 s) — both already served by GitHub's own mirrors, and neither addressed by caching the arcticpy wheel.

Caching would add real invalidation complexity to buy back roughly a quarter of a step that runs for under a minute. The premise that "arcticpy compiles from source on every CI run" is true, but the compile turns out to be cheap — arctic is a small C++ extension, not a heavyweight build.

If this is ever revisited, the more promising target is apt-get (a cached .deb, or a prepared image), not the wheel.


Generated by Claude Code

@Jammy2211
Jammy2211 merged commit 83983e4 into main Aug 24, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

refactor: one canonical Heart-owned arcticpy CI install for every CTI repo

2 participants