Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,34 @@ All notable changes to GenAIRR are documented here.

### Added

- `result.to_genotype_tsv(path)` and `CohortResult.to_genotype_tsv(path)`:
a run writes the genotype it was given beside the records it produced, one
row per carried allele, with `subject_id`, `locus`, `segment`, `gene`,
`allele`, `chromosome`, `novel` and `gene_state`. Until now the planted
truth and the records left the library by separate doors -- the records from
the result, the genotype from the `Genotype` object the caller still had to
be holding -- so a dataset handed to someone else arrived without the thing
it was meant to be scored against. A gene deleted on a chromosome keeps its
row with the literal allele `(none)`, because an empty TSV cell cannot be
told apart from a missing value, and a result with no genotype raises rather
than writing a zero-row file that looks like success. Whether a given read
drew a novel allele is a join of `truth_v_call` against this table on
`(subject_id, allele)`, which the genotype guide now shows; it is
deliberately not a record column, since the record already carries both
halves.

The writer refuses more than an absent genotype. It refuses a permissive
genotype that still leaves reference genes unspecified, because the engine
samples those genes over the whole reference and their alleles would reach
the records but no row of the file: measured on `HUMAN_IGH_OGRDB`, a
permissive genotype with one gene pinned writes two rows while the records
draw 119 alleles that are not in it, and a benchmark scoring a caller
against that file marks correct calls wrong. It also refuses a cohort where
only some subjects carry a genotype, and two subjects sharing one
`subject_id`, since both produce a table that looks complete and is not.
Per-chromosome expression weight (`chromosome_weights`) is *not* in the
sidecar and is documented as such.

- `determinism_version` on a trace file as well as on every exported record.
A trace recorded a seed and a plan signature, which together say what to
draw and from where, but not which model does the drawing: replaying a file
Expand Down
11 changes: 8 additions & 3 deletions site_docs/guides/genotype-benchmarking.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,9 @@ The recipe is the same for any tool:

1. Build a `Genotype`, simulate a repertoire, write the AIRR table
(`result.to_tsv(...)`) and/or reads FASTA, and the ground truth
(`genotype.to_tsv(...)`).
(`genotype.to_tsv(...)` for the wide per-gene table, or
`result.to_genotype_tsv(...)` for one row per carried allele, keyed on
`subject_id` so it joins to the records).
2. Run the inference tool to recover the per-individual allele set.
3. Compare recovered vs planted: presence/absence, zygosity, and (for
discovery tools) any novel alleles.
Expand Down Expand Up @@ -106,6 +108,8 @@ res = (

res.to_tsv("repertoire.tsv") # AIRR table → TIgGER
g.to_tsv("truth_genotype.tsv") # ground truth to score against
res.to_genotype_tsv("truth_long.tsv") # same truth, long form, joins
# to the records on subject_id

with open("reads.fasta", "w") as fh: # raw reads → IgDiscover / partis
for r in res:
Expand Down Expand Up @@ -162,8 +166,9 @@ deletion calls.

## Where to go next

- [Genotypes (overview)](genotype.md) - building genotypes, novel alleles, and
the `to_table()` / `to_tsv()` ground truth you score against.
- [Genotypes (overview)](genotype.md) - building genotypes, novel alleles, the
`to_table()` / `to_tsv()` ground truth you score against, and the
per-record join that answers whether a given read drew a novel allele.
- [Genotype cohorts](genotype-cohorts.md) - scale this recipe to many donors in
one `run_cohort` call.

52 changes: 52 additions & 0 deletions site_docs/guides/genotype.md
Original file line number Diff line number Diff line change
Expand Up @@ -390,13 +390,65 @@ A genotype experiment emits, by construction, everything an evaluation needs:
(`homozygous` / `heterozygous` / `hemizygous` / `deleted`), the carried alleles
per haplotype, and per-haplotype `allele:copies:weight` detail. This is the
reference a genotype-inference benchmark compares against.
- **`result.to_genotype_tsv(path)`:** the same truth in **long** form, written
as a sidecar beside the records: one row per carried allele, with
`subject_id`, `locus`, `segment`, `gene`, `allele`, `chromosome` (`0`/`1`),
`novel` and `gene_state`. A gene deleted on a chromosome keeps its row with
the literal allele `(none)`, so an absence is distinguishable from a missing
value. `CohortResult.to_genotype_tsv(path)` writes every subject into one
table, which joins to the concatenated records file on `subject_id`. A run
with no genotype raises rather than writing an empty file.

```python
for row in result.genotypes[0].to_table():
if row["zygosity"] != "homozygous": # show the interesting genes
print(row["gene"], row["zygosity"], row["haplotype_0"], row["haplotype_1"])
```

### Was this read's allele novel?

There is deliberately no per-read novelty column. The question is a join: the
record names the allele it was generated from, and the sidecar says whether
that allele is novel in that subject. Keeping it a join is what stops the
record carrying a field derived from two others it already has.

```python
import csv

result.to_tsv("repertoire.tsv")
result.to_genotype_tsv("truth_genotype.tsv")

with open("truth_genotype.tsv", encoding="utf-8") as fh:
novel = {(row["subject_id"], row["allele"])
for row in csv.DictReader(fh, delimiter="\t")
if row["novel"] == "True"}

# `truth_v_call` needs expose_provenance=True; prefer it over `v_call`, which
# may be a comma-separated tie list.
for rec in result.records:
drew_novel_v = (rec["subject_id"], rec["truth_v_call"]) in novel
```

The `gene_state` column answers the zygosity question per read the same way.
The deletion question is **not** a per-read question: a deleted gene is never
drawn from, so no record's `truth_v_call` can ever match a `(none)` row. Ask
it of the subject, by reading the rows whose `gene_state` is `deleted`.

!!! warning "Join on the allele, not on the locus"
A record carries one `locus` for the whole rearrangement, taken from its V
call. A sidecar row carries the locus of *its own* gene. On a cartridge
whose V pool mixes loci — `HUMAN_TCRD_IMGT` has both `TRAV` and `TRDV`
genes — a read with a `TRAV` V call is labelled `TRA` while the `TRDJ`
allele it was generated from is a `TRD` row. Filtering the sidecar by the
record's locus before joining therefore drops the D and J truth for most
reads. Join on `(subject_id, allele)`.

!!! note "Per-chromosome expression weight is not in the sidecar"
`chromosome_weights(w0, w1)` changes how often each chromosome is drawn
from, and no export carries it. With `(1.0, 0.0)` the table still lists
every haplotype-1 allele as carried, while no record can draw one. If you
set non-equal weights, record them alongside the run yourself.

## More genotype topics

- **[Genotype sampling & population priors](genotype-priors.md)** - draw a
Expand Down
2 changes: 2 additions & 0 deletions site_docs/reference/simulation-result.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ The eight methods you'll reach for in real pipelines:
| `.to_dataframe(*, airr_strict=False)` | Return a pandas DataFrame with canonical AIRR columns |
| `.to_tsv(path, *, airr_strict=True)` | Write TSV with AIRR coordinates, CIGAR placement offsets, and `T` / `F` booleans |
| `.to_csv(path, *, airr_strict=False)` | Write CSV with Python record conventions by default |
| `.to_genotype_tsv(path)` | Write the run's genotype ground truth as a sidecar, one row per carried allele, joined to the records on `subject_id` + allele. Raises if the run carried no genotype |
| `.to_fasta(path, *, prefix="seq")` | Write assembled sequences as FASTA |
| `.to_fastq(...)` / `.to_paired_fastq(...)` | Write FASTQ; paired-end requires `read_layout="paired_end"` |

Expand Down Expand Up @@ -90,5 +91,6 @@ output file unless you pass `overwrite=True`.
- to_dataframe
- to_tsv
- to_csv
- to_genotype_tsv
- to_fasta
- from_outcomes
210 changes: 210 additions & 0 deletions src/GenAIRR/_genotype_truth.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,210 @@
"""The long projection of a genotype's carried-allele truth.

``Genotype.to_table()`` is one row per gene, with the carried alleles as
list-valued columns (``haplotype_0``, ``haplotype_1``). That is the right
shape for a Python caller and the wrong shape for a TSV a benchmark joins
against: a list inside a flat cell is a nested format inside a flat one, and
every consumer ends up writing its own splitter.

This module projects that table into the long form -- one row per (gene,
chromosome, carried allele) -- which is what a join wants. ``to_table()`` is
left exactly as it is; it is a public, tested surface with its own callers,
and this is a projection of it rather than a replacement.

Two properties matter more than the shape:

**A deleted gene still gets rows.** This is the one point where a long table
can lose information the wide one keeps. "Gene absent from the table" and
"gene present and deleted on both chromosomes" are different truths, and a
genotype caller is scored on exactly that difference. A gene deleted on one
chromosome gets one allele row and one ``DELETED_ALLELE`` row; deleted on
both gets two. ``gene_state`` carries the zygosity verbatim, so the wide form
stays recoverable from the long one.

**The locus names the gene's locus, which is not always the record's.**
``locus_of`` is a port of the engine's naming rule (``locus_prefix`` in
``engine_rs/src/airr_record/locus.rs``): the uppercased three-character
prefix, accepted only when it is one of the seven AIRR loci, and empty
otherwise. A synthetic reference whose allele names are not locus-prefixed
gets an empty locus here, as it does on the record.

The two values are *not* interchangeable, and the difference is real rather
than theoretical. A record carries one locus for the whole rearrangement,
taken from its V call first (``derive_locus``). A sidecar row carries the
locus of the gene in that row. On a cartridge whose V pool mixes loci --
``HUMAN_TCRD_IMGT`` has both ``TRAV`` and ``TRDV`` genes, as do the two
mouse TCRD cartridges -- a read with a ``TRAV`` V call and a ``TRDJ`` J call
is labelled ``TRA``, while the sidecar row for the very ``TRDJ`` allele it
was generated from says ``TRD``. Measured on that cartridge: 134 of 200
records. So **join on ``(subject_id, allele)`` and do not filter the sidecar
by the record's locus first** -- that filter silently drops the D and J
truth for most reads. The locus column is there to make a row readable on
its own, not to be matched against the record's.
"""
from typing import Dict, List, Tuple

from .genotype._common import _SEGMENTS, _alleles_by_gene

# The seven AIRR loci, as the engine spells them. TCR loci are TRA/TRB/TRG/TRD
# here, not the IMGT-style TCRA/TCRB -- the engine emits the former.
AIRR_LOCI = ("IGH", "IGK", "IGL", "TRA", "TRB", "TRG", "TRD")

#: What a deleted chromosome's allele cell holds. A literal token, not an
#: empty cell: an empty TSV cell cannot be told apart from a missing value,
#: which is the one confusion a ground-truth file must not create.
DELETED_ALLELE = "(none)"

#: The long form's columns, in order.
COLUMNS = (
"subject_id",
"locus",
"segment",
"gene",
"allele",
"chromosome",
"novel",
"gene_state",
)


def locus_of(name: str) -> str:
"""The AIRR locus a gene or allele name belongs to, or ``""``.

A port of the engine's ``locus_prefix``, kept deliberately identical so
the sidecar's ``locus`` column equals the ``locus`` on the records it is
joined to. Returning ``""`` rather than guessing is the point: a name
outside the seven loci names no locus, and inventing one would put a
value in the truth file that the records disagree with.
"""
if len(name) < 3:
return ""
prefix = name[:3].upper()
return prefix if prefix in AIRR_LOCI else ""


def genotype_truth_rows(genotype) -> List[Dict]:
"""One row per (gene, chromosome, carried allele) for one subject.

Deleted chromosomes contribute a ``DELETED_ALLELE`` row rather than no
row. A novel allele that was defined but never placed on a chromosome
contributes nothing: it is not carried, so it is not truth about this
subject's genome.
"""
rows: List[Dict] = []
for wide in genotype.to_table():
novel = set(wide["novel"])
gene = wide["gene"]
locus = locus_of(gene)
for chromosome in (0, 1):
carried = wide[f"haplotype_{chromosome}"]
# A chromosome with no allele is a deletion, and it is reported.
alleles = carried if carried else [DELETED_ALLELE]
for allele in alleles:
rows.append({
"subject_id": wide["subject_id"],
"locus": locus,
"segment": wide["segment"],
"gene": gene,
"allele": allele,
"chromosome": chromosome,
"novel": allele in novel,
"gene_state": wide["zygosity"],
})
return rows


def unspecified_genes(genotype) -> List[Tuple[str, str]]:
"""The reference genes this genotype does not pin, as ``(segment, gene)``.

Empty for a strict or completed genotype. Non-empty only for a
``Genotype.permissive`` that was never completed, where the engine samples
the unspecified genes over the whole reference: those alleles reach the
records but no row of the projection, which is the one way this table can
be a silent subset of the truth.

Keyed on what is specified rather than on ``is_permissive``, because
``permissive().complete_from_reference()`` stays permissive while its
table becomes complete.
"""
cfg = getattr(genotype, "_cfg", None)
if cfg is None:
return []
missing: List[Tuple[str, str]] = []
for segment in _SEGMENTS:
for gene in _alleles_by_gene(cfg, segment):
if not genotype.is_specified(segment, gene):
missing.append((segment, gene))
return missing


def write_genotype_tsv(path: str, genotypes) -> None:
"""Write the long truth table for ``genotypes`` to ``path`` as TSV.

Refuses, before opening the file, when there is nothing to write or when
a genotype cannot be joined. Both refusals exist for the same reason: a
truth file that is empty, or whose ``subject_id`` is blank, sits beside a
records file looking like an export that worked. Raising leaves the
caller with an error instead of a plausible artifact.
"""
import csv

supplied = list(genotypes or ())
carried = [g for g in supplied if g is not None]
if not carried:
raise ValueError(
"this result has no genotype, so there is no genotype truth to "
"write; run with `.with_genotype(...)` or `run_cohort(...)` to "
"produce a result that carries one")

# Dropping the Nones would write a table that is short by whole subjects
# and says nothing about it -- a partial file that looks complete.
missing_genotype = [i for i, g in enumerate(supplied) if g is None]
if missing_genotype:
raise ValueError(
f"subject(s) at position(s) {missing_genotype} have no genotype, so "
f"the sidecar would be missing those subjects entirely while "
f"looking complete; give every subject a genotype or export the "
f"ones that have them separately")

unjoinable = [i for i, g in enumerate(carried) if not getattr(g, "subject_id", None)]
if unjoinable:
raise ValueError(
f"genotype(s) at position(s) {unjoinable} carry no subject_id, so "
f"the sidecar could not be joined to the records; set one with "
f"`.with_subject(...)`")

seen: Dict[str, int] = {}
for index, genotype in enumerate(carried):
sid = genotype.subject_id
if sid in seen:
raise ValueError(
f"subject_id {sid!r} appears on genotypes at positions "
f"{seen[sid]} and {index}, so their rows would merge under one "
f"join key and neither subject's truth would be recoverable; "
f"give each subject a distinct id with `.with_subject(...)`")
seen[sid] = index

for genotype in carried:
missing = unspecified_genes(genotype)
if missing:
shown = ", ".join(f"{seg}:{gene}" for seg, gene in missing[:3])
raise ValueError(
f"genotype {getattr(genotype, 'subject_id', None)!r} leaves "
f"{len(missing)} reference gene(s) unspecified (e.g. {shown}), "
f"so this table is not the set of alleles the records were "
f"generated from: the engine samples those genes over the "
f"whole reference, unphased, and they would be missing from "
f"the file. Call `complete_from_reference()` to make the "
f"carried set explicit, or export the partial picture with "
f"`Genotype.to_tsv(...)`, which records that it is permissive")

rows = [row for g in carried for row in genotype_truth_rows(g)]
with open(path, "w", newline="", encoding="utf-8") as fh:
# LF, matching `_write_delimited`'s deliberate `lineterminator="\n"`.
# The excel dialect's default CRLF would leave a stray \r on
# `gene_state`, the last column, in any shell pipeline that joins the
# sidecar to the records file.
writer = csv.writer(fh, delimiter="\t", lineterminator="\n")
writer.writerow(COLUMNS)
for row in rows:
writer.writerow([row[c] for c in COLUMNS])
16 changes: 16 additions & 0 deletions src/GenAIRR/_result_export.py
Original file line number Diff line number Diff line change
Expand Up @@ -328,6 +328,22 @@ def to_tsv(self, path: str, *, airr_strict: bool = True) -> None:
"""
self._write_delimited(path, "\t", airr_strict=airr_strict)

def to_genotype_tsv(self, path: str) -> None:
"""Write this run's genotype ground truth beside its records.

One row per (gene, chromosome, carried allele), joined to the records
on ``subject_id``. A gene deleted on a chromosome keeps its row with
``(none)`` in the allele cell -- "absent from the table" and "present
and deleted" are different truths.

Raises ``ValueError`` on a result with no genotype rather than writing
an empty file: a zero-row truth file beside a million records is a
failure that looks like success.
"""
from ._genotype_truth import write_genotype_tsv

write_genotype_tsv(path, getattr(self, "_genotypes", None))

def to_csv(self, path: str, *, airr_strict: bool = False) -> None:
"""Write the records as comma-separated values. Convenience
alongside :meth:`to_tsv` — most analysis tooling prefers TSV
Expand Down
11 changes: 11 additions & 0 deletions src/GenAIRR/cohort.py
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,17 @@ def to_csv(self, path: str, *, airr_strict: bool = False) -> None:
DataFrame)."""
self.to_dataframe(airr_strict=airr_strict).to_csv(path, index=False)

def to_genotype_tsv(self, path: str) -> None:
"""Write every subject's genotype ground truth into one table.

The cohort counterpart of :meth:`SimulationResult.to_genotype_tsv`:
the same long form, with ``subject_id`` distinguishing the subjects,
so one sidecar joins to the concatenated records file.
"""
from ._genotype_truth import write_genotype_tsv

write_genotype_tsv(path, self.genotypes)

def to_fasta(self, path: str) -> None:
"""Write one FASTA record per concatenated record. Header is exactly
``>{sequence_id}`` (the namespaced id); body is the record's
Expand Down
Loading
Loading