Skip to content

fix: regenerate the stale API audit baseline; make wiki-currency diagnosable - #23

Merged
Jammy2211 merged 1 commit into
mainfrom
fix/wiki-currency-baseline-drift
Aug 24, 2026
Merged

fix: regenerate the stale API audit baseline; make wiki-currency diagnosable#23
Jammy2211 merged 1 commit into
mainfrom
fix/wiki-currency-baseline-drift

Conversation

@Jammy2211

Copy link
Copy Markdown
Contributor

wiki-currency has been red on main independently of any PR, so every PR against this repo opened already-red.

Which sub-check fails

The job log never says, because the workflow redirects each check into drift-report.md. The drift-report artifact is served from a blob host this session's network policy blocks, so instead I reproduced the run locally against the stack the workflow builds (autonerves/autoarray/autofit 2026.8.17.1 from source main, autocti 2024.11.13.2, arcticpy 2.6). Exactly one of the five fails:

Check Result
--check-version FAIL — API DRIFT vs baseline (generated 2026-07-23)
--scope all pass — 31/31 cited symbols resolve, 0 missing
--lint-idioms pass — 51 files, no defunct idioms
--check-provenance pass — 8 pages, 0 errors (9 warnings)
--check-citations pass — 29 citations, 0 missing paths

It is baseline staleness, not a stale doc claim

The task rightly warned not to regenerate the baseline to paper over a genuine divergence, so I checked rather than assumed. I diffed the public surface at the baseline's own commits (PyAutoFit daa4e39, PyAutoArray 85c57eb, both 2026-07-23) against today's main:

autoarray  120 -> 121   + InterpolatorDelaunayNN, validate
                        - TransformerNUFFTPyNUFFT
autofit    149 -> 159   + AbstractClipper, AbstractScaler, ApproxUpdater,
                          ClipperNone, ClipperPriorBox, DynamicUpdater,
                          FactorUpdater, NSS, ScalerNone, ScalerPriorWidth,
                          SimplerUpdater
                        - database

So it is not blindly additive — two symbols were removed. But:

  • Neither removed symbol is cited. TransformerNUFFTPyNUFFT appears nowhere in wiki/, skills/ or modes/. The database matches are prose about the workspace's advanced/database/ script directory, not the autofit.database symbol — there is no af.database / from autofit import database anywhere.
  • The symbol audit agrees--scope all reports "All cited symbols resolve cleanly. No drift detected."
  • autonerves, autocti and autocti.plot hashes are byte-identical to the baseline. The surfaces this assistant actually documents have not moved at all.

The wiki makes no stale claim. The whole-surface hash simply aged. Baseline regenerated; all five checks then pass locally.

The report was mislabelled — and that caused the misdiagnosis

The drift report printed stack_version: latest released whenever the input was empty. That is false on the native PR/dispatch path. The install step does:

if [ -n "$VER" ]; then pip install "autocti==$VER"
else                   pip install ./sources/PyAutoNerves ./sources/PyAutoFit ...

— it builds from the main source clones, and the workflow's own comment explains why (the CTI release train is not wired, so PyPI would grade the modern docs against a pre-resurrection wheel).

This matters beyond tidiness: the task that filed this bug reasoned "the workflow audits against latest released, so it rots on a clock" and pointed at release timing. The real clock is the libraries' main branches, which move far faster than releases. The label sent the diagnosis to the wrong hypothesis. Both input descriptions and the report header now state what is actually installed.

Diagnosability

The report named no source refs at all, so a red run couldn't be diagnosed after the fact — you could not distinguish a genuine divergence from an aged baseline. The clone step now records each source tree's short SHA into the report header:

Cited/installed source trees:
- PyAutoArray: `68a69ae`
- PyAutoCTI: `7376d14`
- PyAutoFit: `438f56f`
- PyAutoNerves: `0ecefa0`
- autocti_workspace: `93ac60c`

Verified by running that step's shell locally.

On pinning to a released stack version — recommend not doing this

The task asked whether pinning would stop the check rotting on timing alone. I don't think it should be done, and the reason is in the workflow already: autocti's PyPI release is the pre-resurrection 2024.11.13.2, so pinning would grade today's docs against an API that predates the work they describe — vacuously green, which is worse than noisily red.

The structural issue is real but different. --check-version gates on a hash of the entire public surface of autoarray and autofit, almost none of which this assistant documents. Today's red is 12 additions and 2 removals, and not one of them touches a cited symbol. Any autofit/autoarray main merge that exports a new name turns this repo red again — this baseline will rot the same way within weeks.

Meanwhile --scope all already answers the question that matters ("do the symbols the docs cite still resolve?") and is immune to that noise.

A worthwhile follow-up would be to make --check-version gate only on symbols removed from the surface and report additions as informational — or to accept that --scope all subsumes it as a gate. That is a change to what the check means, so I've left it as a proposal rather than making it unilaterally under this task's supervised autonomy. Happy to file it.

Unblocks

autocti_assistant#21 was held unmerged at /prm because of this red. Note PR #22 (the arcticpy consumer leg) also touches wiki-currency.yml, so these two will need merging in order — this one is independent of #22 and can go first or second, but whichever lands second will need a base merge.


Generated by Claude Code

…nosable

wiki-currency has been red on main independently of any PR. Reproduced the run
locally against the stack the workflow builds and identified the failure: of the
five sub-checks, exactly one fails.

  --check-version    FAIL  API DRIFT vs baseline (generated 2026-07-23)
  --scope all        pass  31/31 cited symbols resolve, 0 missing
  --lint-idioms      pass  51 files, no defunct idioms
  --check-provenance pass  8 pages, 0 errors
  --check-citations  pass  29 citations, 0 missing paths

The job log never named it because each check is redirected into drift-report.md.

WHY IT IS BASELINE STALENESS, NOT A DOC DIVERGENCE

Diffed the public surface at the baseline's own commits (PyAutoFit daa4e39,
PyAutoArray 85c57eb, both 2026-07-23) against today's main:

  autoarray  120 -> 121   added InterpolatorDelaunayNN, validate
                          removed TransformerNUFFTPyNUFFT
  autofit    149 -> 159   added AbstractClipper, AbstractScaler, ApproxUpdater,
                          ClipperNone, ClipperPriorBox, DynamicUpdater,
                          FactorUpdater, NSS, ScalerNone, ScalerPriorWidth,
                          SimplerUpdater
                          removed database

Two symbols were removed, so this is not blindly additive — but neither is cited
anywhere in wiki/, skills/ or modes/ (the `database` matches in the docs are
prose about the workspace's advanced/database/ script directory, not the
autofit.database symbol). The symbol audit agrees: all 31 cited symbols resolve
cleanly. And autonerves, autocti and autocti.plot — the surfaces this assistant
actually documents — are byte-identical to the baseline.

So the wiki makes no stale claim; the whole-surface hash simply aged. Baseline
regenerated against the stack the workflow builds (autonerves/autoarray/autofit
2026.8.17.1 from source main, autocti 2024.11.13.2, arcticpy 2.6). All five
checks then pass.

THE REPORT WAS MISLABELLED

The drift report printed `stack_version: latest released` whenever the input was
empty. That is false on the native PR/dispatch path: the install step builds the
stack from the sources/ `main` clones, and the workflow's own comment explains
why (the CTI release train is not wired, so PyPI would grade the modern docs
against a pre-resurrection wheel). The label sent the diagnosis of this very
failure looking for a release that had moved the API, when the actual clock is
the libraries' main branches — much faster. Both input descriptions and the
report header now say what is really installed.

The report also named no source refs at all, so a red run could not be diagnosed
after the fact. The clone step now records each source tree's short SHA into the
report header.

Refs PyAutoMind draft/bug/autocti/wiki_currency_baseline_drift.md

Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_018nDAxBEavkzb6Zkz1cYHef
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants