Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 28 additions & 7 deletions .github/workflows/deploy-docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,15 +44,36 @@ jobs:
- name: Install docs dependencies
run: pip install -r requirements-docs.txt

- name: Install GenAIRR from PyPI (mkdocstrings needs to import it)
- name: Set up Rust
uses: dtolnay/rust-toolchain@stable

- name: Cache cargo build
uses: actions/cache@v4
with:
path: |
~/.cargo/registry
~/.cargo/git
engine_rs/target
key: ${{ runner.os }}-docs-cargo-${{ hashFiles('engine_rs/Cargo.lock') }}
restore-keys: ${{ runner.os }}-docs-cargo-

- name: Install GenAIRR from this checkout (mkdocstrings needs to import it)
# The reference pages use `:::` mkdocstrings directives to render
# signatures + docstrings from real classes (Experiment,
# SimulationResult, ReferenceCartridgeBuilder, etc.). For those
# to resolve, GenAIRR must be importable in the build environment.
# Installing from PyPI is fast (precompiled wheels) and means the
# rendered API reference matches the latest released version,
# which is what published docs should reflect.
run: pip install GenAIRR
# SimulationResult, ReferenceCartridgeBuilder, etc.). For those to
# resolve, GenAIRR must be importable in the build environment.
#
# This used to install from PyPI, on the reasoning that published
# docs should reflect the published release. That inverts the
# dependency: `site_docs/` is versioned with the source, so a page
# documenting anything newer than the last release cannot build. It
# broke on `GenAIRR._reference.artifact.Reference` and the deploy
# then failed on every merge to master from 2026-10-21 until the
# 2.4.0 release work, which is why the live site served
# pre-release text for a fortnight. Building the checkout keeps the
# rendered reference and the prose describing it in step by
# construction.
run: pip install .

- name: Build site with mkdocs build --strict
run: make docs-build
Expand Down
22 changes: 22 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,28 @@ have.

### Added

- **Published germline sources, and dossiers anyone can verify offline.** The
new public module `GenAIRR.reference_sources` locks one *named* published
germline release and keeps the evidence. `OgrdbSource` acquires an explicit
OGRDB germline-set release -- an integer release, never "latest" -- and pins
its exact bytes in a lock; `fetch` is the only operation in GenAIRR that uses
the network, and `from_files` and `read` cover the offline and
behind-a-proxy cases. `source.build(recipe)` projects an explicit recipe into
a `ReferenceBuild`: a runtime `Reference` you pass straight to
`Experiment.on(...)`, plus the lock, the recipe, the build report and a
manifest recording the GenAIRR version that produced it.
`build.write_dossier(directory)` writes the whole thing as a portable
dossier, and `ReferenceBuild.read_dossier(directory)` checks one with no
network and no trust in its author: it regenerates every derived file from
the locked bytes and the recipe and compares, so a single tampered byte comes
back as `ReferenceSourceError(kind='digest_mismatch', path=...)` naming the
member. Failures on this surface all carry a stable `kind` and a `path`.
One build ships inside the wheel --
`ReferenceBuild.bundled("human_igh_ogrdb_10_observed_v1")` -- so the workflow
is available with no network at all; it also reports IMGT allele names, where
`"human_igh"` reports ASC cluster names. See
[Published germline sources](https://mutejester.github.io/GenAIRR/guides/published-references.html).

- **Two clonal models beside the deprecated star.**
`Experiment.clonal_lineage(...)` grows a real BCR genealogy per clone: a
generation-synchronous birth-death process with per-division S5F
Expand Down
22 changes: 22 additions & 0 deletions MIGRATING.md
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,28 @@ Two further counter changes to know about:
A genotype run with novel alleles must be validated against
`compiled.refdata`.

## 7. What you may now want to use

Everything above is about breakage. These are the 2.4.0 additions most
likely to change how you work, in case the migration is the only part of
the release you read:

- **Pin a published germline release, and ship the evidence.**
`GenAIRR.reference_sources` locks one named OGRDB release and writes a
dossier a reviewer can verify offline, with no network and no trust in
you. `ReferenceBuild.bundled("human_igh_ogrdb_10_observed_v1")` needs
no network at all. See
[Published germline sources](https://mutejester.github.io/GenAIRR/guides/published-references.html).
- **Real clonal genealogies.** `clonal_lineage()` replaces the star
topology with a birth-death process and hands back the ground-truth
tree per clone; `clonal_repertoire()` covers TCR and flat-BCR
abundance. Both are the intended replacements for `expand_clones()`
(§3).
- **Per-individual diploid genotypes**, with population priors, cohorts
and a genotype truth sidecar for scoring inference.
- The full list is the `### Added` section of
[CHANGELOG.md](CHANGELOG.md).

---

# Migrating from GenAIRR 1.x to 2.0
Expand Down
45 changes: 23 additions & 22 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -137,33 +137,33 @@ plugins:
# NOTE: no 'learn.md' redirect — learn.md is a real nav page
# ("Choose your path"); a redirect here would clobber the built
# learn.html with a stub and make the page unreachable.
"lesson-1.md": "getting-started/lesson-1-vdj-recombination-from-bases-up.md"
"lesson-2.md": "getting-started/lesson-2-the-pipeline-scrubber.md"
"lesson-3.md": "getting-started/lesson-3-s5f-mutation-isnt-uniform.md"
"lesson-4.md": "getting-started/lesson-4-sequencing-artifacts-dialed-in.md"
"lesson-5.md": "getting-started/lesson-5-the-ground-truth-payoff.md"
"lesson-1.md": "getting-started/index.md"
"lesson-2.md": "getting-started/index.md"
"lesson-3.md": "getting-started/index.md"
"lesson-4.md": "getting-started/index.md"
"lesson-5.md": "getting-started/index.md"
"concepts.md": "concepts/index.md"
"concept-pipeline.md": "concepts/pipeline.md"
"concept-persistent-ir.md": "concepts/persistent-ir.md"
"concept-contracts.md": "concepts/contracts.md"
"concept-pipeline.md": "concepts/index.md"
"concept-persistent-ir.md": "concepts/index.md"
"concept-contracts.md": "concepts/index.md"
"concept-airr-record.md": "concepts/airr-record.md"
"concept-live-call.md": "concepts/live-call.md"
"concept-live-call.md": "concepts/index.md"
"guides.md": "guides/index.md"
"guide-build-config.md": "guides/build-config.md"
"guide-build-config.md": "guides/build-reference-cartridge.md"
"guide-productive.md": "guides/productive.md"
"guide-clonal-families.md": "guides/clonal-families.md"
"guide-v-usage.md": "guides/v-usage.md"
"guide-tune-corruption.md": "guides/tune-corruption.md"
"guide-export.md": "guides/export.md"
"guide-streaming.md": "guides/streaming.md"
"guide-with-metadata.md": "guides/with-metadata.md"
"guide-audit-realism.md": "validation/audit-realism.md"
"guide-benchmark-aligner.md": "validation/benchmark-aligner.md"
"guide-compare-shm.md": "validation/compare-shm.md"
"guide-replay.md": "validation/replay.md"
"guide-reproduce.md": "validation/reproduce.md"
"guide-smoke-test.md": "validation/smoke-test.md"
"guide-trace-introspection.md": "validation/trace-introspection.md"
"guide-v-usage.md": "guides/recombination-junction.md"
"guide-tune-corruption.md": "guides/corruption-sequencing.md"
"guide-export.md": "getting-started/export-results.md"
"guide-streaming.md": "reference/index.md"
"guide-with-metadata.md": "getting-started/export-results.md"
"guide-audit-realism.md": "validation/index.md"
"guide-benchmark-aligner.md": "validation/index.md"
"guide-compare-shm.md": "validation/index.md"
"guide-replay.md": "guides/trace-replay.md"
"guide-reproduce.md": "guides/trace-replay.md"
"guide-smoke-test.md": "validation/index.md"
"guide-trace-introspection.md": "guides/trace-replay.md"
"reference.md": "reference/index.md"
- mkdocstrings:
handlers:
Expand Down Expand Up @@ -214,6 +214,7 @@ nav:
- Corruption + sequencing artefacts: guides/corruption-sequencing.md
- Paired-end reads and FASTQ: guides/paired-end-fastq.md
- Portable local references: guides/portable-references.md
- Published germline sources: guides/published-references.md
- Reference Cartridge Authoring:
- cartridges/index.md
- Reference evidence and quarantine: cartridges/reference-evidence.md
Expand Down
8 changes: 0 additions & 8 deletions site_docs/architecture/adding-a-pass.md

This file was deleted.

8 changes: 0 additions & 8 deletions site_docs/architecture/audit-index.md

This file was deleted.

8 changes: 0 additions & 8 deletions site_docs/architecture/engine.md

This file was deleted.

8 changes: 0 additions & 8 deletions site_docs/architecture/validation-matrix.md

This file was deleted.

11 changes: 0 additions & 11 deletions site_docs/cartridges/build-from-fasta.md

This file was deleted.

5 changes: 0 additions & 5 deletions site_docs/cartridges/empirical-models.md

This file was deleted.

8 changes: 0 additions & 8 deletions site_docs/cartridges/estimators/allele-usage.md

This file was deleted.

8 changes: 0 additions & 8 deletions site_docs/cartridges/estimators/index.md

This file was deleted.

8 changes: 0 additions & 8 deletions site_docs/cartridges/estimators/np-base-model.md

This file was deleted.

8 changes: 0 additions & 8 deletions site_docs/cartridges/estimators/np-length.md

This file was deleted.

8 changes: 0 additions & 8 deletions site_docs/cartridges/estimators/p-nucleotide-length.md

This file was deleted.

8 changes: 0 additions & 8 deletions site_docs/cartridges/estimators/trim-distributions.md

This file was deleted.

5 changes: 0 additions & 5 deletions site_docs/cartridges/rules-and-identity.md

This file was deleted.

8 changes: 0 additions & 8 deletions site_docs/cartridges/v-subregion-annotations.md

This file was deleted.

5 changes: 0 additions & 5 deletions site_docs/concepts/contracts.md

This file was deleted.

5 changes: 0 additions & 5 deletions site_docs/concepts/live-call.md

This file was deleted.

5 changes: 0 additions & 5 deletions site_docs/concepts/persistent-ir.md

This file was deleted.

5 changes: 0 additions & 5 deletions site_docs/concepts/pipeline.md

This file was deleted.

17 changes: 11 additions & 6 deletions site_docs/concepts/reference-cartridge.md
Original file line number Diff line number Diff line change
Expand Up @@ -175,8 +175,13 @@ ship typed defaults that users override per-experiment.
!!! warning "Which bundled cartridges carry *real* data-derived distributions"
Only the human **IGH**, **IGK**, **IGL**, and **TCRB** cartridges
ship with empirical distributions fitted from **real repertoire
data** — trim lengths, NP-region lengths, the NP base / Markov
model, and allele usage.
data** — trim lengths, NP-region lengths, and the NP base / Markov
model.

Allele usage is **not** among them. No bundled cartridge ships an
`allele_usage` plane, so allele draws are uniform over the
catalogue until you weight them yourself — see
[Controlling the allele universe](../guides/recombination-junction.md#controlling-the-allele-universe).

Every **other** bundled species/locus cartridge carries a real
germline **allele catalogue** (from IMGT), but its data-derived
Expand Down Expand Up @@ -417,10 +422,10 @@ legacy fields are sitting on the cartridge unused.

For the original design audit and the engine-side
`RefDataConfig` bridge, see
[`docs/reference_cartridge.md`](https://github.com/MuteJester/GenAIRR/blob/master/docs/reference_cartridge.md)
[`audit-docs/reference_cartridge.md`](https://github.com/MuteJester/GenAIRR/blob/master/audit-docs/reference_cartridge.md)
and
[`docs/reference_cartridge_authoring_audit.md`](https://github.com/MuteJester/GenAIRR/blob/master/docs/reference_cartridge_authoring_audit.md).
[`audit-docs/reference_cartridge_authoring_audit.md`](https://github.com/MuteJester/GenAIRR/blob/master/audit-docs/reference_cartridge_authoring_audit.md).
For per-estimator audits (allele usage, trims, NP length, NP base,
P-nucleotide length), see the matching design docs in `docs/`. For
P-nucleotide length), see the matching design docs in `audit-docs/`. For
the full engine-wide validation matrix, see
[`docs/validation_matrix.md`](https://github.com/MuteJester/GenAIRR/blob/master/docs/validation_matrix.md).
[`audit-docs/validation_matrix.md`](https://github.com/MuteJester/GenAIRR/blob/master/audit-docs/validation_matrix.md).
Loading
Loading