From 3f77e2fb8707562b2b8b42f9d8e34c15ae6499fc Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 24 Aug 2026 19:37:03 +0000 Subject: [PATCH] ci: one canonical Heart-owned arcticpy install action for every CTI repo MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `import autocti` hard-requires arcticpy, which is not a pip dependency: its sdist is source-only C++ (libgsl-dev + a toolchain + Cython) and its own requirements downgrade numpy below 2.0. The install recipe had drifted into four separate shell copies with no owner. This adds the canonical step and repoints the two copies that live here. .github/actions/install-arcticpy/ The canonical step, and THE single arcticpy pin (input `version`, default 2.6) — bumping it is now a one-line change instead of four. Deliberately self-contained: it upgrades pip/setuptools/wheel itself rather than relying on a caller having done so in a preceding step, which is what autocti_workspace_test's epilogue was silently doing across a repo boundary. Two things the recipe as previously specified got wrong, both caught by building arcticpy 2.6 in a clean container and running it: * `--no-deps` suppresses arcticpy's RUNTIME dependencies too, and arcticpy/read_noise.py imports scipy and matplotlib at import time (via __init__.py). Installing only the build deps leaves `import arcticpy` raising ModuleNotFoundError, so the action installs scipy + matplotlib as well. The CTI stack pulls both in anyway; naming them here is what makes the step verifiable standing alone. * arcticpy exposes NO __version__ attribute. The specified assertion `import arcticpy; print(arcticpy.__version__)` raises AttributeError on a perfectly good install. The action reads importlib.metadata instead. Without setuptools the build dies at `BackendUnavailable: Cannot import 'setuptools.build_meta'` — reproduced here, and Python 3.12+ venvs no longer ship setuptools, so its omission was a real hazard. .github/workflows/arcticpy-action.yml + .github/scripts/arcticpy_smoke.py Self-test. The action is consumed cross-repo at @main, so a recipe change would otherwise reach four repos with nothing having exercised it. This job references the action by LOCAL path (so it builds the branch's version) and does not stop at "pip exited 0": it clocks a single bright pixel through arctic and asserts a decaying trail with charge conserved. Verified locally — 1000.0 -> 997.24 in the bright pixel, trail [1.18, 0.78, 0.51, ...]. .github/workflows/lib-tests.yml Both inline copies (jobs `unittest` and `unittest-nojax`) replaced by the action, keeping the `inputs.package == 'autocti'` gate. This is the path PyAutoCTI's own CI takes — its main.yml is a thin caller of this workflow and carries no recipe of its own. .github/workflows/smoke-tests.yml New `arcticpy` boolean input (default false) running the action before the workspace epilogue, so autocti_workspace_test can drop its copy. Default-false leaves every non-CTI caller byte-identical. Gated exactly as lib-tests.yml already gates its own arcticpy step. Refs PyAutoLabs/PyAutoHeart#170 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018nDAxBEavkzb6Zkz1cYHef --- .github/actions/install-arcticpy/action.yml | 125 ++++++++++++++++++++ .github/scripts/arcticpy_smoke.py | 64 ++++++++++ .github/workflows/arcticpy-action.yml | 59 +++++++++ .github/workflows/lib-tests.yml | 23 ++-- .github/workflows/smoke-tests.yml | 23 ++++ 5 files changed, 281 insertions(+), 13 deletions(-) create mode 100644 .github/actions/install-arcticpy/action.yml create mode 100644 .github/scripts/arcticpy_smoke.py create mode 100644 .github/workflows/arcticpy-action.yml diff --git a/.github/actions/install-arcticpy/action.yml b/.github/actions/install-arcticpy/action.yml new file mode 100644 index 0000000..7ed1810 --- /dev/null +++ b/.github/actions/install-arcticpy/action.yml @@ -0,0 +1,125 @@ +name: Install arcticpy +description: >- + Build and install arcticpy, the C++ arctic clocking code that `import autocti` + hard-requires. The canonical recipe for the whole organism — every CTI repo's + CI consumes this action rather than carrying its own copy. + +# WHY THIS EXISTS +# +# arcticpy is deliberately NOT a pip dependency of autocti: +# +# * its PyPI distribution is a source-only C++ sdist — it needs libgsl-dev +# headers and a toolchain to build, and there is no wheel; +# * its own requirements downgrade numpy below 2.0, which breaks a modern +# PyAuto stack. +# +# So it is built with --no-build-isolation (reusing the numpy already present +# instead of resolving its own) and --no-deps (so it cannot drag numpy back +# down). Those two flags are what make the recipe fiddly, and each one costs +# something that has to be paid back explicitly: +# +# --no-build-isolation pip does NOT create an isolated build environment and +# does NOT read pyproject's build-system.requires, so +# every BUILD dependency must already be installed. +# arcticpy declares none of them. +# --no-deps pip installs no RUNTIME dependencies either, so the +# packages arcticpy imports at import time must also be +# installed by hand. +# +# Both sets are installed below, and the step ends by proving the result +# actually imports. +# +# Verified 2026-08-24 in a clean container against a bare venv: each missing +# build dependency failed the build naming the next one. Without setuptools the +# build dies at `BackendUnavailable: Cannot import 'setuptools.build_meta'` — +# and Python 3.12+ venvs no longer ship setuptools by default, which is why the +# recipe that omitted it was a real hazard rather than a style nit. +# +# This step is intentionally SELF-CONTAINED: it upgrades pip/setuptools/wheel +# itself rather than relying on a caller having done so in a preceding step. +# The previous arrangement had autocti_workspace_test's smoke_install.sh +# depending on a `pip install --upgrade pip setuptools wheel` line living in a +# different repository's workflow, with nothing stating the dependency. + +inputs: + version: + description: "arcticpy version to build. THE single pin for the organism — bump it here." + required: false + default: "2.6" + sudo: + description: >- + Use sudo for the apt-get leg. GitHub-hosted runners need it; a container + already running as root does not have it. Set to 'false' there. + required: false + default: "true" + install-gsl: + description: >- + Install libgsl-dev via apt. Set to 'false' when GSL headers are already + present (a prepared image, or a no-root local extraction). + required: false + default: "true" + +outputs: + version: + description: "The arcticpy version actually installed, per importlib.metadata." + value: ${{ steps.verify.outputs.version }} + +runs: + using: composite + steps: + - name: Install GSL headers + if: ${{ inputs.install-gsl == 'true' }} + shell: bash + run: | + set -euo pipefail + SUDO="" + if [ "${{ inputs.sudo }}" = "true" ]; then SUDO="sudo"; fi + $SUDO apt-get update + $SUDO apt-get install -y libgsl-dev + + - name: Install arcticpy build dependencies + shell: bash + run: | + set -euo pipefail + # --no-build-isolation reads NOTHING from arcticpy's build-system + # requires, so these must be present before the build starts. + python -m pip install --upgrade pip setuptools wheel + python -m pip install numpy cython + + - name: Install arcticpy runtime dependencies + shell: bash + run: | + set -euo pipefail + # --no-deps suppresses these, but arcticpy/read_noise.py imports both at + # import time (`from scipy.optimize import curve_fit`, `import + # matplotlib as mpl`) and __init__.py imports read_noise. So `import + # arcticpy` fails without them — which is why the verify step below, + # and every downstream `import autocti`, needs them present. + # + # The CTI stack installs scipy and matplotlib anyway as ordinary + # dependencies; naming them here is what makes this step verifiable on + # its own rather than only inside an already-built stack. + python -m pip install scipy matplotlib + + - name: Build and install arcticpy ${{ inputs.version }} + shell: bash + run: | + set -euo pipefail + python -m pip install "arcticpy==${{ inputs.version }}" \ + --no-build-isolation --no-deps + + - name: Verify arcticpy imports + id: verify + shell: bash + run: | + set -euo pipefail + # Assert here rather than letting a broken build surface much later as a + # confusing `import autocti` failure in an unrelated job. + # + # NOTE: arcticpy exposes no __version__ attribute — the obvious + # `import arcticpy; print(arcticpy.__version__)` raises AttributeError + # even on a perfectly good install. The distribution metadata is the + # supported way to ask. + VERSION="$(python -c 'import arcticpy; from importlib.metadata import version; print(version("arcticpy"))')" + echo "arcticpy $VERSION imported successfully" + echo "version=$VERSION" >> "$GITHUB_OUTPUT" diff --git a/.github/scripts/arcticpy_smoke.py b/.github/scripts/arcticpy_smoke.py new file mode 100644 index 0000000..6cfa474 --- /dev/null +++ b/.github/scripts/arcticpy_smoke.py @@ -0,0 +1,64 @@ +"""Prove the arcticpy install actually clocks charge, not just that pip exited 0. + +Run by .github/workflows/arcticpy-action.yml against the install-arcticpy +composite action. A build can succeed, import, and still be useless if the +compiled arctic extension is broken — so this clocks a single bright pixel and +checks the result has the four properties every CTI repo depends on. + +Kept as a file rather than a heredoc inside the workflow so it can be run +locally against any environment: + + python .github/scripts/arcticpy_smoke.py +""" + +import numpy as np +import arcticpy as ac + + +def main() -> None: + # One bright pixel in an otherwise empty column. CTI drags charge out of it + # and releases it into the pixels clocked after it. + image = np.zeros((20, 1)) + image[10, 0] = 1000.0 + + traps = [ac.TrapInstantCapture(density=10.0, release_timescale=2.0)] + # NOTE: add_cti wants a CCD, not a CCDPhase — a bare CCDPhase raises + # AttributeError on fraction_of_traps_per_phase. Likewise parallel_roe is + # not optional in practice: omitting it raises on roe.dwell_times. + ccd = ac.CCD(full_well_depth=1e5, well_notch_depth=0.0, well_fill_power=0.8) + roe = ac.ROE() + + clocked = ac.add_cti( + image, + parallel_traps=traps, + parallel_ccd=ccd, + parallel_roe=roe, + parallel_express=5, + ) + + trail = clocked[11:, 0] + print(f"bright pixel : {image[10, 0]} -> {clocked[10, 0]}") + print(f"trail : {trail[:5]}") + print(f"total charge : {image.sum()} -> {clocked.sum()}") + + # 1. Charge left the bright pixel. + assert clocked[10, 0] < image[10, 0], "no charge was trapped out of the bright pixel" + + # 2. It reappeared behind it, in pixels that started empty. + assert trail[0] > 0, "no trail behind the bright pixel" + + # 3. The trail decays — the signature of exponential trap release. + assert trail[0] > trail[1] > trail[2], f"trail is not decaying: {trail[:3]}" + + # 4. Charge is conserved. arctic's express approximation is not exactly + # conserving (measured ~6e-4 relative at express=5), so this is a + # gross-error check, not a precision one. + assert np.isclose(clocked.sum(), image.sum(), rtol=5e-3), ( + f"charge not conserved: {image.sum()} -> {clocked.sum()}" + ) + + print("OK: arcticpy produced a decaying CTI trail with charge conserved") + + +if __name__ == "__main__": + main() diff --git a/.github/workflows/arcticpy-action.yml b/.github/workflows/arcticpy-action.yml new file mode 100644 index 0000000..4d684e1 --- /dev/null +++ b/.github/workflows/arcticpy-action.yml @@ -0,0 +1,59 @@ +name: arcticpy Action Self-Test + +# The install-arcticpy composite action is consumed cross-repo at +# `PyAutoLabs/PyAutoHeart/.github/actions/install-arcticpy@main`, so a change to +# the recipe only reaches its consumers once it is ON main — which means nothing +# would exercise it before merge, and a broken recipe would land red in four +# repos at once (PyAutoHeart lib-tests, PyAutoCTI, autocti_workspace_test smoke, +# autocti_assistant wiki-currency). +# +# This job closes that window. It references the action by LOCAL path, so it +# builds the version on the branch under review, and it does not stop at +# "pip exited 0": it clocks a single bright pixel through arctic and asserts the +# result is a real CTI trail. That is the property the CTI repos actually +# depend on. + +on: + pull_request: + paths: + - ".github/actions/install-arcticpy/**" + - ".github/workflows/arcticpy-action.yml" + workflow_dispatch: + inputs: + version: + description: "arcticpy version to test (default: the action's own pin)" + required: false + type: string + +concurrency: + group: arcticpy-action-${{ github.ref }} + cancel-in-progress: ${{ github.ref != 'refs/heads/main' }} + +jobs: + build: + runs-on: ubuntu-latest + strategy: + fail-fast: false + matrix: + # The build depends on the interpreter (3.12+ venvs no longer ship + # setuptools), so both supported minors are exercised. + python-version: ["3.12", "3.13"] + steps: + - uses: actions/checkout@v4 + + - name: Set up Python ${{ matrix.python-version }} + uses: actions/setup-python@v5 + with: + python-version: ${{ matrix.python-version }} + + - name: Install arcticpy (the action under test, from this branch) + id: arctic + uses: ./.github/actions/install-arcticpy + with: + version: ${{ inputs.version || '2.6' }} + + - name: Report the version the action installed + run: echo "action reported arcticpy ${{ steps.arctic.outputs.version }}" + + - name: Clock a bright pixel and assert a CTI trail + run: python .github/scripts/arcticpy_smoke.py diff --git a/.github/workflows/lib-tests.yml b/.github/workflows/lib-tests.yml index 1a3f537..a55c09f 100644 --- a/.github/workflows/lib-tests.yml +++ b/.github/workflows/lib-tests.yml @@ -78,16 +78,13 @@ jobs: python-version: ${{ matrix.python-version }} cache: pip + # The canonical arcticpy install for the whole organism, including its + # version pin, lives in the action — not in this file and not in any + # consumer repo. Referenced at @main rather than by local path because + # this is a REUSABLE workflow: it runs in the caller's context. - name: Install arcticpy (autocti only — source-only C++ sdist) if: ${{ inputs.package == 'autocti' }} - run: | - # arcticpy needs GSL headers to compile, and installing it with its - # own requirements downgrades numpy below 2.0 — so numpy is installed - # first and arcticpy is built without deps or build isolation. - sudo apt-get update && sudo apt-get install -y libgsl-dev - pip install --upgrade pip setuptools wheel - pip install numpy cython - pip install arcticpy==2.6 --no-build-isolation --no-deps + uses: PyAutoLabs/PyAutoHeart/.github/actions/install-arcticpy@main - name: Install (deps + package from source, [optional] extras) run: | @@ -177,13 +174,13 @@ jobs: python-version: "3.13" cache: pip + # The canonical arcticpy install for the whole organism, including its + # version pin, lives in the action — not in this file and not in any + # consumer repo. Referenced at @main rather than by local path because + # this is a REUSABLE workflow: it runs in the caller's context. - name: Install arcticpy (autocti only — source-only C++ sdist) if: ${{ inputs.package == 'autocti' }} - run: | - sudo apt-get update && sudo apt-get install -y libgsl-dev - pip install --upgrade pip setuptools wheel - pip install numpy cython - pip install arcticpy==2.6 --no-build-isolation --no-deps + uses: PyAutoLabs/PyAutoHeart/.github/actions/install-arcticpy@main - name: Install ([optional] extras), then strip the jax package family run: | diff --git a/.github/workflows/smoke-tests.yml b/.github/workflows/smoke-tests.yml index d7b37e4..f3e818f 100644 --- a/.github/workflows/smoke-tests.yml +++ b/.github/workflows/smoke-tests.yml @@ -16,6 +16,13 @@ name: Smoke Tests (reusable) # (.github/scripts/smoke_install.sh, receiving PYTHON_VERSION — extras, # pins, version conditionals) and the smoke runner itself. # +# One deliberate exception: `arcticpy: true`. arcticpy is a stack-level system +# dependency (source-only C++ sdist, GSL headers, a numpy-downgrade trap) whose +# recipe had drifted into four separate copies across the CTI repos. That is +# not workspace-specific variation, it is one recipe with no owner — so Heart +# owns it, in .github/actions/install-arcticpy, and the CTI workspace asks for +# it with a flag instead of carrying a copy in its epilogue. +# # `runner` lets a caller point that same ceremony at a DIFFERENT workspace # script — a re-timing harness, a one-off diagnostic sweep — instead of # copying the chain-checkout/install steps into a second workflow. The copy is @@ -61,6 +68,13 @@ on: required: false type: string default: "" + arcticpy: + description: >- + Install arcticpy (the C++ arctic clocking code `import autocti` + requires) before the workspace epilogue. CTI workspaces only. + required: false + type: boolean + default: false jobs: # Docs-only gate: skip the matrix when the diff touches nothing but prose. @@ -141,6 +155,15 @@ jobs: with: python-version: ${{ matrix.python-version }} + # arcticpy is a stack-level system dependency with a fiddly, easy-to-drift + # build recipe — it belongs to Heart, not to each workspace's epilogue. + # Gated the same way lib-tests.yml gates its own arcticpy step, so every + # non-CTI caller is unaffected. This runs BEFORE the epilogue: the + # epilogue installs autocti, and `import autocti` needs arcticpy present. + - name: Install arcticpy (CTI workspaces only — source-only C++ sdist) + if: ${{ inputs.arcticpy }} + uses: PyAutoLabs/PyAutoHeart/.github/actions/install-arcticpy@main + - name: Install (base + the workspace's own epilogue) env: PYTHON_VERSION: ${{ matrix.python-version }}