ci: gate --check-version on removals, and make its report diagnosable - #26
Merged
Merged
Conversation
`--check-version` compared a hash of the ENTIRE public surface of autonerves, autoarray, autofit, autocti and autocti.plot against a committed baseline. The assistant documents almost none of that surface, and the workflow installs those libraries from their `main` source clones — so the clock was every upstream merge that exported a name. The red that prompted this was 12 additions and 2 removals, and not one of the 14 symbols was cited anywhere in wiki/, skills/ or modes/. Two changes, and the second is why option 1 beat option 2 (demote the check and let --scope all be the only gate): 1. The baseline now records the sorted public names per module, not just a hash and a count. Previously a red printed exactly "public API surface changed: autoarray, autofit" — undiagnosable. Working out what moved meant creating worktrees at the baseline's own commits, installing the libraries from them, and diffing by hand. That is what the last investigation actually had to do. 449 names, 12 KB. 2. With names available, the gate distinguishes additions from removals. A symbol appearing cannot break a doc; a symbol disappearing can. Additions print and pass; any removal fails and names the symbol. It does NOT narrow further to "removals of cited symbols" — that is exactly what --scope all already computes, and two mechanisms answering one question is how they drift apart. A baseline predating symbol recording keeps the old all-or-nothing behaviour and says so: treating an undiagnosable change as additive would wave a real removal through. A module absent from the baseline entirely is reported distinctly, so adding one to BASELINE_MODULES does not read as "your baseline is old". The regenerated baseline is a pure schema upgrade — every per-module hash and every version is byte-identical to the one PR #23 wrote, so this records no API change, only the names behind the existing hashes. Also writes skills/ac_audit_skill_apis.md, which did not exist. It was an unfinished clone item in PENDING.md, and AGENTS.md, modes/maintainer.md and skills/ac_setup_environment.md all linked to it — five broken links. Written from audit_skill_apis.py itself rather than adapted from the lensing sibling, and it carries the thing the task asked to document: when regenerating the baseline is right and when it is papering over a real break. Verified against the real CTI stack (autonerves/autoarray/autofit 2026.8.17.1 from source main, autocti 2024.11.13.2, arcticpy 2.6) — all five wiki-currency checks pass: --check-version clean, --scope all 31/31 symbols, --lint-idioms 52 files clean, --check-provenance 0 errors, --check-citations 0 missing. Refs #25 Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_014wiN7R1yaeGj6k1Pa1FEq4
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #25.
--check-versioncompared a hash of the entire public surface ofautonerves,autoarray,autofit,autoctiandautocti.plotagainst a committed baseline. This assistant documents almost none of that surface, and the workflow installs those libraries from theirmainsource clones — so the clock was every upstream merge that exported a name. The red fixed in #23 was 12 additions and 2 removals, and not one of the 14 symbols was cited anywhere inwiki/,skills/ormodes/. That baseline would have rotted again within weeks.The decision, and the fact that drove it
The task offered two options: gate on removals (1), or demote
--check-versionand let--scope allbe the only gate (2). Option 2 was the leaner candidate. Option 1 won on a fact the prompt did not have: the baseline stored only a hash and a count, never the symbol names.So a red printed exactly this, and nothing more:
Working out what moved meant creating worktrees at the baseline's own commits, installing the libraries from them, and diffing the symbol sets by hand — which is what the last investigation actually did. Recording the names (449 across five modules, 12 KB) kills the noise and makes every future red self-diagnosing. That combination is worth more than the simplification option 2 buys.
What changed
Baseline records
symbolsper module alongsidehashandn_symbols._api_hashwas split so both derive from one sorted list and cannot disagree.The gate distinguishes additions from removals. Additions print and pass; any removal fails and names the symbol:
It deliberately does not narrow further to removals of cited symbols. That is exactly what
--scope allalready computes, against the real citation set; two mechanisms answering one question is how they drift apart.--check-versionstays the cheap whole-surface tripwire.Two backward-compatibility paths, both gating rather than guessing. A baseline predating symbol recording keeps the old all-or-nothing behaviour and says why — treating an undiagnosable change as additive would wave a real removal through. A module absent from the baseline entirely is reported distinctly, so adding one to
BASELINE_MODULESdoes not read as "your baseline is old".No workflow edit is needed:
wiki-currency.yml's uniformrun()helper fails on any non-zero exit, so the exit code is the mechanism.AGENTS.md's session-start prose is updated to the narrowed meaning.The caveat, stated plainly
This would not have prevented the original red. Both removed symbols (
TransformerNUFFTPyNUFFT,autofit.database) were uncited, so a removals-only gate still goes red on them. What changes is that the red now arrives with the names attached, and the twelve irrelevant additions no longer trigger it at all.The regenerated baseline is a pure schema upgrade
Regenerated against the real stack, then diffed against the one #23 wrote:
Every hash and every version byte-identical. This records no API change — only the names behind the hashes that were already there.
--check-versionwas already clean against today'smainbefore regenerating, so #23's fix is confirmed still valid.Scope note —
skills/ac_audit_skill_apis.mddid not existThe task said to document the baseline-regeneration story there. The file was never written — it is an unfinished clone item in
PENDING.md("regenerate for this domain, reference:skills/al_audit_skill_apis.md"), andAGENTS.md(×3),modes/maintainer.mdandskills/ac_setup_environment.mdall link to it. Five broken links.So it is written here, from
audit_skill_apis.pyitself rather than adapted from the lensing sibling, so it describes this repo's actual checks. It carries the thing the task asked for — the regenerate-or-investigate decision procedure — plus the five checks, how to read a drift report, and why pinning to a released stack stays rejected. Registered inskills/README.md, symlinked into.claude/skills/(committed as mode120000), and bothPENDING.mdentries ticked.Test Plan
Against the real CTI stack — autonerves/autoarray/autofit
2026.8.17.1from sourcemain, autocti2024.11.13.2, arcticpy 2.6, i.e. the same shapewiki-currency.ymlbuilds. All five checks:--check-version--scope all--lint-idioms--check-provenance--check-citationsUnit tests —
test_check_version.pyrewritten, 11 tests covering: identical surface; version stamp differing alone; additions-only passing and being named; a removal gating; a removal alongside additions; a removal in one module while another only gains; both legacy-baseline paths; a module absent from the baseline; a missing baseline; and that the shipped baseline is on the new schema with hashes consistent with its recorded names.test_check_version.pypassmain— verified by diffingFAILEDlines against a cleanmainworktree; the 13 pre-existing failures are all "no installed stack" (test_api_gate,test_install_preflight,test_benchmark), and this branch adds nonewiki-currencygreen in CI on this PRExplicitly rejected
Pinning to a released stack. autocti's PyPI release is the pre-resurrection
2024.11.13.2, so pinning would grade today's docs against an API predating the work they describe — vacuously green, worse than noisily red. Considered and rejected 2026-08-24; the workflow's own install-step comment says the same. Thestack_versioninput still exists for the release-timeworkflow_callpath, where a pin is the right thing.Generated by Claude Code