Skip to content

ci: gate --check-version on removals, and make its report diagnosable - #26

Merged
Jammy2211 merged 1 commit into
mainfrom
feature/wiki-currency-check-version-gate
Aug 24, 2026
Merged

ci: gate --check-version on removals, and make its report diagnosable#26
Jammy2211 merged 1 commit into
mainfrom
feature/wiki-currency-check-version-gate

Conversation

@Jammy2211

Copy link
Copy Markdown
Contributor

Summary

Closes #25.

--check-version compared a hash of the entire public surface of autonerves, autoarray, autofit, autocti and autocti.plot against a committed baseline. This assistant documents almost none of that surface, and the workflow installs those libraries from their main source clones — so the clock was every upstream merge that exported a name. The red fixed in #23 was 12 additions and 2 removals, and not one of the 14 symbols was cited anywhere in wiki/, skills/ or modes/. That baseline would have rotted again within weeks.

The decision, and the fact that drove it

The task offered two options: gate on removals (1), or demote --check-version and let --scope all be the only gate (2). Option 2 was the leaner candidate. Option 1 won on a fact the prompt did not have: the baseline stored only a hash and a count, never the symbol names.

So a red printed exactly this, and nothing more:

[drift] API DRIFT vs baseline (baseline generated 2026-07-23):
  - public API surface changed: autoarray, autofit

Working out what moved meant creating worktrees at the baseline's own commits, installing the libraries from them, and diffing the symbol sets by hand — which is what the last investigation actually did. Recording the names (449 across five modules, 12 KB) kills the noise and makes every future red self-diagnosing. That combination is worth more than the simplification option 2 buys.

What changed

Baseline records symbols per module alongside hash and n_symbols. _api_hash was split so both derive from one sorted list and cannot disagree.

The gate distinguishes additions from removals. Additions print and pass; any removal fails and names the symbol:

[drift] public API surface moved — additions only (not gated) (baseline generated 2026-08-24):
  autofit  160 -> 171
    + added:   AbstractClipper, AbstractScaler, ApproxUpdater, ...
  Nothing was removed, so no cited symbol can have stopped resolving. Not gating.
[drift] public API surface moved — REMOVALS (baseline generated 2026-08-24):
  autoarray  122 -> 121
    - removed: TransformerNUFFTPyNUFFT
  A removed symbol can break a doc that cites it. Run `--scope all` to find out
  whether any of the above is actually cited, then `--write-baseline` to re-pin.

It deliberately does not narrow further to removals of cited symbols. That is exactly what --scope all already computes, against the real citation set; two mechanisms answering one question is how they drift apart. --check-version stays the cheap whole-surface tripwire.

Two backward-compatibility paths, both gating rather than guessing. A baseline predating symbol recording keeps the old all-or-nothing behaviour and says why — treating an undiagnosable change as additive would wave a real removal through. A module absent from the baseline entirely is reported distinctly, so adding one to BASELINE_MODULES does not read as "your baseline is old".

No workflow edit is needed: wiki-currency.yml's uniform run() helper fails on any non-zero exit, so the exit code is the mechanism. AGENTS.md's session-start prose is updated to the narrowed meaning.

The caveat, stated plainly

This would not have prevented the original red. Both removed symbols (TransformerNUFFTPyNUFFT, autofit.database) were uncited, so a removals-only gate still goes red on them. What changes is that the red now arrives with the names attached, and the twelve irrelevant additions no longer trigger it at all.

The regenerated baseline is a pure schema upgrade

Regenerated against the real stack, then diffed against the one #23 wrote:

module hash n_symbols symbols recorded
autonerves same 32 → 32 32
autoarray same 122 → 122 122
autofit same 160 → 160 160
autocti same 103 → 103 103
autocti.plot same 32 → 32 32

Every hash and every version byte-identical. This records no API change — only the names behind the hashes that were already there. --check-version was already clean against today's main before regenerating, so #23's fix is confirmed still valid.

Scope note — skills/ac_audit_skill_apis.md did not exist

The task said to document the baseline-regeneration story there. The file was never written — it is an unfinished clone item in PENDING.md ("regenerate for this domain, reference: skills/al_audit_skill_apis.md"), and AGENTS.md (×3), modes/maintainer.md and skills/ac_setup_environment.md all link to it. Five broken links.

So it is written here, from audit_skill_apis.py itself rather than adapted from the lensing sibling, so it describes this repo's actual checks. It carries the thing the task asked for — the regenerate-or-investigate decision procedure — plus the five checks, how to read a drift report, and why pinning to a released stack stays rejected. Registered in skills/README.md, symlinked into .claude/skills/ (committed as mode 120000), and both PENDING.md entries ticked.

Test Plan

Against the real CTI stack — autonerves/autoarray/autofit 2026.8.17.1 from source main, autocti 2024.11.13.2, arcticpy 2.6, i.e. the same shape wiki-currency.yml builds. All five checks:

check result
--check-version exit 0 — clean
--scope all exit 0 — 24 files, 31/31 unique symbols, 0 missing
--lint-idioms exit 0 — 52 files, no defunct idioms (52 not 51: the new skill is scanned and clean)
--check-provenance exit 0 — 8 pages, 0 errors, 9 warnings (pre-existing, non-gating)
--check-citations exit 0 — 27 files, 29 citations, 0 missing

Unit teststest_check_version.py rewritten, 11 tests covering: identical surface; version stamp differing alone; additions-only passing and being named; a removal gating; a removal alongside additions; a removal in one module while another only gains; both legacy-baseline paths; a module absent from the baseline; a missing baseline; and that the shipped baseline is on the new schema with hashes consistent with its recorded names.

  • 11/11 test_check_version.py pass
  • Full suite failure set identical to main — verified by diffing FAILED lines against a clean main worktree; the 13 pre-existing failures are all "no installed stack" (test_api_gate, test_install_preflight, test_benchmark), and this branch adds none
  • All five wiki-currency checks pass locally against the real stack
  • wiki-currency green in CI on this PR

Explicitly rejected

Pinning to a released stack. autocti's PyPI release is the pre-resurrection 2024.11.13.2, so pinning would grade today's docs against an API predating the work they describe — vacuously green, worse than noisily red. Considered and rejected 2026-08-24; the workflow's own install-step comment says the same. The stack_version input still exists for the release-time workflow_call path, where a pin is the right thing.


Generated by Claude Code

`--check-version` compared a hash of the ENTIRE public surface of autonerves,
autoarray, autofit, autocti and autocti.plot against a committed baseline. The
assistant documents almost none of that surface, and the workflow installs those
libraries from their `main` source clones — so the clock was every upstream merge
that exported a name. The red that prompted this was 12 additions and 2 removals,
and not one of the 14 symbols was cited anywhere in wiki/, skills/ or modes/.

Two changes, and the second is why option 1 beat option 2 (demote the check and
let --scope all be the only gate):

1. The baseline now records the sorted public names per module, not just a hash
   and a count. Previously a red printed exactly "public API surface changed:
   autoarray, autofit" — undiagnosable. Working out what moved meant creating
   worktrees at the baseline's own commits, installing the libraries from them,
   and diffing by hand. That is what the last investigation actually had to do.
   449 names, 12 KB.

2. With names available, the gate distinguishes additions from removals. A symbol
   appearing cannot break a doc; a symbol disappearing can. Additions print and
   pass; any removal fails and names the symbol.

It does NOT narrow further to "removals of cited symbols" — that is exactly what
--scope all already computes, and two mechanisms answering one question is how
they drift apart.

A baseline predating symbol recording keeps the old all-or-nothing behaviour and
says so: treating an undiagnosable change as additive would wave a real removal
through. A module absent from the baseline entirely is reported distinctly, so
adding one to BASELINE_MODULES does not read as "your baseline is old".

The regenerated baseline is a pure schema upgrade — every per-module hash and
every version is byte-identical to the one PR #23 wrote, so this records no API
change, only the names behind the existing hashes.

Also writes skills/ac_audit_skill_apis.md, which did not exist. It was an
unfinished clone item in PENDING.md, and AGENTS.md, modes/maintainer.md and
skills/ac_setup_environment.md all linked to it — five broken links. Written from
audit_skill_apis.py itself rather than adapted from the lensing sibling, and it
carries the thing the task asked to document: when regenerating the baseline is
right and when it is papering over a real break.

Verified against the real CTI stack (autonerves/autoarray/autofit 2026.8.17.1
from source main, autocti 2024.11.13.2, arcticpy 2.6) — all five wiki-currency
checks pass: --check-version clean, --scope all 31/31 symbols, --lint-idioms
52 files clean, --check-provenance 0 errors, --check-citations 0 missing.

Refs #25

Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_014wiN7R1yaeGj6k1Pa1FEq4
@Jammy2211
Jammy2211 merged commit de4ac2d into main Aug 24, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci: make --check-version report a real symbol diff and gate on removals only

2 participants