Skip to content

docs: close the release-blocking documentation gaps before tagging 2.4.0 - #54

Merged
MuteJester merged 7 commits into
masterfrom
docs/release-gaps-2.4.0
Nov 6, 2026
Merged

MuteJester merged 7 commits into
masterfrom
docs/release-gaps-2.4.0

Conversation

@MuteJester

Copy link
Copy Markdown
Owner

Found while checking the docs ahead of the 2.4.0 tag. Everything here is
pre-existing and most of it is already live, so this is a fix pass, not
new writing. One commit per piece.

What a reader currently gets wrong

The blinded handoff was not blinded. export-results.md told readers
to_fasta "emits only sequence_id + the assembled sequence, so it's
implicitly blinded". It does not — both to_fasta and to_fastq
annotate every header with the generating calls:

>seq0|v_call=IGHVF6-G23*03|j_call=IGHJ1*01

Anyone who followed that page handed the answer to the tool they were
scoring, inside the input of their own benchmark. The page documented
the header correctly fifteen lines earlier, so it contradicted itself
and the wrong half was the one a benchmarking reader acts on.

Allele usage was undocumented and inverts a usage table.
v_allele_weights appeared on no page of the site while the homepage
advertises controllable V-gene usage. An allele you do not list keeps
its default weight of 1.0, so asking for 0.30 and 0.20 on two alleles
puts 0.300% of 2,000 records on them rather than half — they end up
rarer than the 196 alleles you never mentioned. 0.0 is rejected, and
restrict_alleles cannot be combined with weights. The guide now gives
a full-support recipe measured at 0.501/0.299/0.200 with zero leakage.

The bundled cartridges do not carry data-fitted allele usage.
concepts/reference-cartridge.md said all four human cartridges did.
allele_usage.available is False on every one of them, and the legacy
gene_use_dict is not read by the recombination pass, so default V
usage is uniform over 198 alleles — the wrong prior for any
usage-sensitive benchmark.

What shipped without documentation

GenAIRR.reference_sources is a public module with its own __all__,
added after the 2.3.1 tag, with zero mentions across the site, the
changelog, the migration guide and the README — while the 2.4.0 release
commit is itself about the packaged dossier. New guide, a ### Added
entry, and a section in MIGRATING.md saying what is new and worth
adopting, which it had no room for while being entirely about breakage.

Every claim on the new page is executed against the shipped build. The
fetch and read examples are written against the real signatures
rather than the obvious guesses: release is an int with no "latest",
profile and cache are required, and write_dossier takes a
directory rather than an archive.

Why the live site is worse than this repo

22 of the 28 redirect_maps entries pointed at pages whose whole body
said the page had not been written, and the deployed site still serves
an older placeholder text naming a contributor-only planning document
and then sending readers to the page they just came from. A citation to
/guide-benchmark-aligner.html used to serve 1,059 words.

All 22 now point at the nearest page with content and all 37
placeholders are deleted, which also removes them from the search index
— five were the only title match for their own obvious query. This
does not bring the content back:
16,251 words of the original prose
are still in website/, one written page behind each of these URLs.
Porting it needs modernising rather than copying and is separate work.

The deploy has been failing on every merge to master since 2026-10-21 —
15 consecutive runs — because the workflow installed GenAIRR from PyPI
and mkdocstrings cannot collect a class that does not exist in 2.3.1.
That is the dependency the wrong way round: site_docs/ is versioned
with the source, so a page documenting anything newer than the last
release cannot build and the site silently keeps serving the last green
run. It now builds the checkout. The deploy job itself was never broken
and resumes once the build does.

Guards

Two new pins: every redirect target must exist, and none may carry a
placeholder marker. Verified by reintroducing a marker and watching the
second pin name all four affected redirects. They deliberately do not
read the contributor-only migration plan the sibling contract module
parses — that file is absent from every clone, which skips all 26 of its
pins in CI, so the only existing check on this contract has never run
where it matters.

Verification

  • pytest tests/ -q — 5888 passed, 17 skipped
  • ruff check tests/ src/ tools/ — clean
  • mkdocs build --strict — exits 0, and no page is left outside the nav
  • tools/inventory_gate.py check --since origin/master --declarations — clean
  • Every code block added here was executed against this checkout

Deferred

Porting the 22 legacy pages; the nav regrouping; lineage sizing guidance
(n_max is inert at the default max_generations=10, and n_sample
over-asks silently); four genotype traps that produce wrong output with
no exception; rate= and Q-score conversion; s5f_model kernels;
stream_records; .trim() examples; curate_refdata policies; the
reference/experiment.md member list; ASC-vs-IMGT allele naming.

The export page claimed `to_fasta` "emits only `sequence_id` + the
assembled sequence, so it's implicitly blinded". It does not. Both
`to_fasta` and `to_fastq` annotate every header with the generating V
and J calls:

    >seq0|v_call=IGHVF6-G23*03|j_call=IGHJ1*01

So a reader who followed this page handed the answer to the tool they
were scoring, inside the input of their own benchmark. The page already
documented that header correctly fifteen lines earlier, under FASTA,
which made the claim a self-contradiction rather than a stale fact --
and the wrong half was the one a benchmarking reader acts on, because
FASTA is the format an aligner wants.

Replaced the claim with a table of what each exporter carries, and a
blinded-FASTA writer that emits `sequence_id` and bases only. The
`.upper()` in it is deliberate: the corruption passes lowercase the
bases they touch and some aligners read case as a soft-mask.

Two things a scorer has to settle are now stated where they are needed
rather than left to be discovered: that the default human IGH cartridge
reports ASC cluster names while an IMGT-referenced aligner reports IMGT
names, with both escape hatches; and that `v_call` and `recalled_v_call`
answer different questions, diverging exactly where mutation has made
the generating allele unrecoverable.

The illustrative header under FASTA was also stale -- it showed
space-separated calls with no `key=` prefixes. It now shows what the
exporter writes for this page's own example parameters.
`index.md` advertises controllable V-gene usage and no page of the site
mentioned the knob that controls it. `v_allele_weights`,
`d_allele_weights` and `j_allele_weights` appeared nowhere outside one
paragraph of the README about the cartridge estimator, and
`biology-map.md` -- which claims to map every biological mechanism --
had no row for allele usage at all.

The knob also reads like the opposite of what it does. An allele you do
not list keeps its default weight of 1.0, so passing the shares from a
real usage table makes the alleles you asked for rarer than the ones you
never mentioned. Measured on human IGH over 2,000 records, asking for
0.30 and 0.20 puts 6 records -- 0.300% -- on those two alleles instead
of half of them. `0.0` is rejected as not finite-positive, so exclusion
has no spelling here either, and `restrict_alleles` cannot be combined
with weights: that raises `ValueError`.

So the section now gives the three surfaces, states that they do not
compose, and carries a full-support recipe that reproduces a target
distribution exactly -- 0.501/0.299/0.200 against 0.5/0.3/0.2, with no
records outside the three.

While measuring it, the default turned out to be worth stating too.
`allele_usage.available` is False on all four human cartridges the
concepts page named as carrying data-fitted distributions, and the
legacy `gene_use_dict` they do carry is not read by the recombination
pass. Default V usage is therefore uniform over 198 alleles on human
IGH, which is the wrong prior for any usage-sensitive benchmark and the
opposite of what that page promised. Corrected there, and cross-linked.

The same page also pointed at three `docs/*.md` files that have not
existed under that name since the audit documents were renamed, so
three links on the published site were dead.
`GenAIRR.reference_sources` is a public module with its own `__all__`,
added after the 2.3.1 tag, and it had zero mentions across the docs
site, the changelog, the migration guide and the README. The release
commit for 2.4.0 is itself about the packaged dossier. The only shipped
build id was discoverable solely by asking for one that does not exist
and reading the error.

So the capability it exists for -- pinning a named published germline
release and handing a reviewer a dossier they can verify with no
network and no trust in its author -- was unreachable for anyone who
had not read the source.

New guide, with every claim executed against the shipped build:
`bundled()` runs under `Experiment.on(...)` and reports IMGT names,
which also makes it the shortest answer to the ASC-naming problem;
`write_dossier` takes a directory, not an archive, and emits the nine
named members plus their content-addressed blobs; `read_dossier`
regenerates every derived file from the locked bytes and the recipe
rather than re-reading stored hashes, so one altered byte comes back as
`kind='digest_mismatch'` with the member's path.

The `fetch` and `read` examples are written against the real signatures
rather than the obvious guesses: `release` is an int and there is no
"latest", `profile` and `cache` are required keyword arguments, and the
standard-library HTTPS client ignores proxy settings, which is why
`from_files` exists.

Also added the changelog entry this should have had, and a section to
the migration guide saying what is new and worth adopting -- it was
entirely about breakage, so a reader following it to the end learned
nothing about what they had just gained.
22 of the 28 `redirect_maps` entries pointed at pages whose entire body
was an admonition saying the page had not been written. The live site is
worse still: it serves an older placeholder text that names
`docs/docs_mkdocs_migration_plan.md`, a contributor-only document no
reader can open, and then tells them to go look at the old site -- which
is the page they just came from, because the old URL now redirects here.

So a bookmark or a citation to, say, `/guide-benchmark-aligner.html`
used to serve a 1,059-word recipe and now serves 33 words of apology.
That is worse than a 404, which at least tells a reader to search.

Each of the 22 now points at the nearest page that has content: the
lessons and the four concept placeholders at their section indexes, the
three trace/replay guides at `guides/trace-replay.md`, the four
remaining validation recipes at `validation/index.md`, and the rest at
the specific page that absorbed their subject -- `guide-v-usage` at the
allele-universe section written earlier in this branch. All 37
placeholder pages are deleted, which also takes them out of the search
index, where five of them were the only title match for their own
obvious query.

Two new pins enforce it, because nothing did: every redirect target must
exist, and no redirect target may carry a placeholder marker. Verified
by reintroducing a marker and watching the second one name all four
affected redirects. They deliberately do not read the contributor-only
migration plan that the sibling contract module parses -- that file is
absent from every clone, which skips all 26 of its pins in CI, so the
one existing check on this contract has never run where it matters.

What this does not do is bring the content back. 16,251 words of the
original prose are still in `website/`, one written page behind each of
these URLs. Porting it is a separate piece of work; it needs
modernising, not copying.
The docs deploy has failed on every merge to master since 2026-10-21 --
15 consecutive runs. `mkdocstrings` could not collect
`GenAIRR._reference.artifact.Reference`, because the workflow installed
GenAIRR from PyPI and that class does not exist in 2.3.1.

That is the dependency the wrong way round. `site_docs/` is versioned
with the source, so a page documenting anything added since the last
release cannot build, and the site silently keeps serving whatever the
last green run produced. It had been serving a fortnight-old build --
which is why the published placeholder pages still named a
contributor-only planning document, long after the repo stopped
mentioning it.

Building the checkout keeps the rendered reference and the prose around
it in step by construction, at the cost of a Rust build in the docs job;
the cargo cache and toolchain action are the same ones `test.yml`
already uses. This is the configuration the local `mkdocs build
--strict` has been passing under all along.

The deploy job itself was never the problem: it last succeeded on
2026-10-21, so GitHub Pages is configured correctly and resumes once
the build does.
…gone

Both entries explained the 2.4.0 dossier regeneration, which is now in
the baseline. A declaration left behind after its change merges excuses
the next edit to the same name, so the gate reports them as explaining
nothing on this run.
…dule

`test_no_module_builds_a_path_from_file_except_the_recorded_ones` fails
on a module that derives the repo root from `__file__` without a
recorded reason, and this one did. `tests.support.paths.repo_root()` is
anchored to a fixed location and is what the rest of the suite uses, so
the fix is to follow the convention rather than record an exception for
a test that needs nothing special.
@MuteJester
MuteJester merged commit cdbabdf into master Nov 6, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant