diff --git a/CHANGELOG.md b/CHANGELOG.md index 0fbec5d7..ae1e1ea0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,34 @@ All notable changes to GenAIRR are documented here. ### Added +- `result.to_genotype_tsv(path)` and `CohortResult.to_genotype_tsv(path)`: + a run writes the genotype it was given beside the records it produced, one + row per carried allele, with `subject_id`, `locus`, `segment`, `gene`, + `allele`, `chromosome`, `novel` and `gene_state`. Until now the planted + truth and the records left the library by separate doors -- the records from + the result, the genotype from the `Genotype` object the caller still had to + be holding -- so a dataset handed to someone else arrived without the thing + it was meant to be scored against. A gene deleted on a chromosome keeps its + row with the literal allele `(none)`, because an empty TSV cell cannot be + told apart from a missing value, and a result with no genotype raises rather + than writing a zero-row file that looks like success. Whether a given read + drew a novel allele is a join of `truth_v_call` against this table on + `(subject_id, allele)`, which the genotype guide now shows; it is + deliberately not a record column, since the record already carries both + halves. + + The writer refuses more than an absent genotype. It refuses a permissive + genotype that still leaves reference genes unspecified, because the engine + samples those genes over the whole reference and their alleles would reach + the records but no row of the file: measured on `HUMAN_IGH_OGRDB`, a + permissive genotype with one gene pinned writes two rows while the records + draw 119 alleles that are not in it, and a benchmark scoring a caller + against that file marks correct calls wrong. It also refuses a cohort where + only some subjects carry a genotype, and two subjects sharing one + `subject_id`, since both produce a table that looks complete and is not. + Per-chromosome expression weight (`chromosome_weights`) is *not* in the + sidecar and is documented as such. + - `determinism_version` on a trace file as well as on every exported record. A trace recorded a seed and a plan signature, which together say what to draw and from where, but not which model does the drawing: replaying a file diff --git a/site_docs/guides/genotype-benchmarking.md b/site_docs/guides/genotype-benchmarking.md index cf50615e..f389d8c1 100644 --- a/site_docs/guides/genotype-benchmarking.md +++ b/site_docs/guides/genotype-benchmarking.md @@ -9,7 +9,9 @@ The recipe is the same for any tool: 1. Build a `Genotype`, simulate a repertoire, write the AIRR table (`result.to_tsv(...)`) and/or reads FASTA, and the ground truth - (`genotype.to_tsv(...)`). + (`genotype.to_tsv(...)` for the wide per-gene table, or + `result.to_genotype_tsv(...)` for one row per carried allele, keyed on + `subject_id` so it joins to the records). 2. Run the inference tool to recover the per-individual allele set. 3. Compare recovered vs planted: presence/absence, zygosity, and (for discovery tools) any novel alleles. @@ -106,6 +108,8 @@ res = ( res.to_tsv("repertoire.tsv") # AIRR table → TIgGER g.to_tsv("truth_genotype.tsv") # ground truth to score against +res.to_genotype_tsv("truth_long.tsv") # same truth, long form, joins + # to the records on subject_id with open("reads.fasta", "w") as fh: # raw reads → IgDiscover / partis for r in res: @@ -162,8 +166,9 @@ deletion calls. ## Where to go next -- [Genotypes (overview)](genotype.md) - building genotypes, novel alleles, and - the `to_table()` / `to_tsv()` ground truth you score against. +- [Genotypes (overview)](genotype.md) - building genotypes, novel alleles, the + `to_table()` / `to_tsv()` ground truth you score against, and the + per-record join that answers whether a given read drew a novel allele. - [Genotype cohorts](genotype-cohorts.md) - scale this recipe to many donors in one `run_cohort` call. diff --git a/site_docs/guides/genotype.md b/site_docs/guides/genotype.md index 5564156d..fd4df706 100644 --- a/site_docs/guides/genotype.md +++ b/site_docs/guides/genotype.md @@ -390,6 +390,14 @@ A genotype experiment emits, by construction, everything an evaluation needs: (`homozygous` / `heterozygous` / `hemizygous` / `deleted`), the carried alleles per haplotype, and per-haplotype `allele:copies:weight` detail. This is the reference a genotype-inference benchmark compares against. +- **`result.to_genotype_tsv(path)`:** the same truth in **long** form, written + as a sidecar beside the records: one row per carried allele, with + `subject_id`, `locus`, `segment`, `gene`, `allele`, `chromosome` (`0`/`1`), + `novel` and `gene_state`. A gene deleted on a chromosome keeps its row with + the literal allele `(none)`, so an absence is distinguishable from a missing + value. `CohortResult.to_genotype_tsv(path)` writes every subject into one + table, which joins to the concatenated records file on `subject_id`. A run + with no genotype raises rather than writing an empty file. ```python for row in result.genotypes[0].to_table(): @@ -397,6 +405,50 @@ for row in result.genotypes[0].to_table(): print(row["gene"], row["zygosity"], row["haplotype_0"], row["haplotype_1"]) ``` +### Was this read's allele novel? + +There is deliberately no per-read novelty column. The question is a join: the +record names the allele it was generated from, and the sidecar says whether +that allele is novel in that subject. Keeping it a join is what stops the +record carrying a field derived from two others it already has. + +```python +import csv + +result.to_tsv("repertoire.tsv") +result.to_genotype_tsv("truth_genotype.tsv") + +with open("truth_genotype.tsv", encoding="utf-8") as fh: + novel = {(row["subject_id"], row["allele"]) + for row in csv.DictReader(fh, delimiter="\t") + if row["novel"] == "True"} + +# `truth_v_call` needs expose_provenance=True; prefer it over `v_call`, which +# may be a comma-separated tie list. +for rec in result.records: + drew_novel_v = (rec["subject_id"], rec["truth_v_call"]) in novel +``` + +The `gene_state` column answers the zygosity question per read the same way. +The deletion question is **not** a per-read question: a deleted gene is never +drawn from, so no record's `truth_v_call` can ever match a `(none)` row. Ask +it of the subject, by reading the rows whose `gene_state` is `deleted`. + +!!! warning "Join on the allele, not on the locus" + A record carries one `locus` for the whole rearrangement, taken from its V + call. A sidecar row carries the locus of *its own* gene. On a cartridge + whose V pool mixes loci — `HUMAN_TCRD_IMGT` has both `TRAV` and `TRDV` + genes — a read with a `TRAV` V call is labelled `TRA` while the `TRDJ` + allele it was generated from is a `TRD` row. Filtering the sidecar by the + record's locus before joining therefore drops the D and J truth for most + reads. Join on `(subject_id, allele)`. + +!!! note "Per-chromosome expression weight is not in the sidecar" + `chromosome_weights(w0, w1)` changes how often each chromosome is drawn + from, and no export carries it. With `(1.0, 0.0)` the table still lists + every haplotype-1 allele as carried, while no record can draw one. If you + set non-equal weights, record them alongside the run yourself. + ## More genotype topics - **[Genotype sampling & population priors](genotype-priors.md)** - draw a diff --git a/site_docs/reference/simulation-result.md b/site_docs/reference/simulation-result.md index 949ea6d6..e79cf32b 100644 --- a/site_docs/reference/simulation-result.md +++ b/site_docs/reference/simulation-result.md @@ -22,6 +22,7 @@ The eight methods you'll reach for in real pipelines: | `.to_dataframe(*, airr_strict=False)` | Return a pandas DataFrame with canonical AIRR columns | | `.to_tsv(path, *, airr_strict=True)` | Write TSV with AIRR coordinates, CIGAR placement offsets, and `T` / `F` booleans | | `.to_csv(path, *, airr_strict=False)` | Write CSV with Python record conventions by default | +| `.to_genotype_tsv(path)` | Write the run's genotype ground truth as a sidecar, one row per carried allele, joined to the records on `subject_id` + allele. Raises if the run carried no genotype | | `.to_fasta(path, *, prefix="seq")` | Write assembled sequences as FASTA | | `.to_fastq(...)` / `.to_paired_fastq(...)` | Write FASTQ; paired-end requires `read_layout="paired_end"` | @@ -90,5 +91,6 @@ output file unless you pass `overwrite=True`. - to_dataframe - to_tsv - to_csv + - to_genotype_tsv - to_fasta - from_outcomes diff --git a/src/GenAIRR/_genotype_truth.py b/src/GenAIRR/_genotype_truth.py new file mode 100644 index 00000000..ef0e184a --- /dev/null +++ b/src/GenAIRR/_genotype_truth.py @@ -0,0 +1,210 @@ +"""The long projection of a genotype's carried-allele truth. + +``Genotype.to_table()`` is one row per gene, with the carried alleles as +list-valued columns (``haplotype_0``, ``haplotype_1``). That is the right +shape for a Python caller and the wrong shape for a TSV a benchmark joins +against: a list inside a flat cell is a nested format inside a flat one, and +every consumer ends up writing its own splitter. + +This module projects that table into the long form -- one row per (gene, +chromosome, carried allele) -- which is what a join wants. ``to_table()`` is +left exactly as it is; it is a public, tested surface with its own callers, +and this is a projection of it rather than a replacement. + +Two properties matter more than the shape: + +**A deleted gene still gets rows.** This is the one point where a long table +can lose information the wide one keeps. "Gene absent from the table" and +"gene present and deleted on both chromosomes" are different truths, and a +genotype caller is scored on exactly that difference. A gene deleted on one +chromosome gets one allele row and one ``DELETED_ALLELE`` row; deleted on +both gets two. ``gene_state`` carries the zygosity verbatim, so the wide form +stays recoverable from the long one. + +**The locus names the gene's locus, which is not always the record's.** +``locus_of`` is a port of the engine's naming rule (``locus_prefix`` in +``engine_rs/src/airr_record/locus.rs``): the uppercased three-character +prefix, accepted only when it is one of the seven AIRR loci, and empty +otherwise. A synthetic reference whose allele names are not locus-prefixed +gets an empty locus here, as it does on the record. + +The two values are *not* interchangeable, and the difference is real rather +than theoretical. A record carries one locus for the whole rearrangement, +taken from its V call first (``derive_locus``). A sidecar row carries the +locus of the gene in that row. On a cartridge whose V pool mixes loci -- +``HUMAN_TCRD_IMGT`` has both ``TRAV`` and ``TRDV`` genes, as do the two +mouse TCRD cartridges -- a read with a ``TRAV`` V call and a ``TRDJ`` J call +is labelled ``TRA``, while the sidecar row for the very ``TRDJ`` allele it +was generated from says ``TRD``. Measured on that cartridge: 134 of 200 +records. So **join on ``(subject_id, allele)`` and do not filter the sidecar +by the record's locus first** -- that filter silently drops the D and J +truth for most reads. The locus column is there to make a row readable on +its own, not to be matched against the record's. +""" +from typing import Dict, List, Tuple + +from .genotype._common import _SEGMENTS, _alleles_by_gene + +# The seven AIRR loci, as the engine spells them. TCR loci are TRA/TRB/TRG/TRD +# here, not the IMGT-style TCRA/TCRB -- the engine emits the former. +AIRR_LOCI = ("IGH", "IGK", "IGL", "TRA", "TRB", "TRG", "TRD") + +#: What a deleted chromosome's allele cell holds. A literal token, not an +#: empty cell: an empty TSV cell cannot be told apart from a missing value, +#: which is the one confusion a ground-truth file must not create. +DELETED_ALLELE = "(none)" + +#: The long form's columns, in order. +COLUMNS = ( + "subject_id", + "locus", + "segment", + "gene", + "allele", + "chromosome", + "novel", + "gene_state", +) + + +def locus_of(name: str) -> str: + """The AIRR locus a gene or allele name belongs to, or ``""``. + + A port of the engine's ``locus_prefix``, kept deliberately identical so + the sidecar's ``locus`` column equals the ``locus`` on the records it is + joined to. Returning ``""`` rather than guessing is the point: a name + outside the seven loci names no locus, and inventing one would put a + value in the truth file that the records disagree with. + """ + if len(name) < 3: + return "" + prefix = name[:3].upper() + return prefix if prefix in AIRR_LOCI else "" + + +def genotype_truth_rows(genotype) -> List[Dict]: + """One row per (gene, chromosome, carried allele) for one subject. + + Deleted chromosomes contribute a ``DELETED_ALLELE`` row rather than no + row. A novel allele that was defined but never placed on a chromosome + contributes nothing: it is not carried, so it is not truth about this + subject's genome. + """ + rows: List[Dict] = [] + for wide in genotype.to_table(): + novel = set(wide["novel"]) + gene = wide["gene"] + locus = locus_of(gene) + for chromosome in (0, 1): + carried = wide[f"haplotype_{chromosome}"] + # A chromosome with no allele is a deletion, and it is reported. + alleles = carried if carried else [DELETED_ALLELE] + for allele in alleles: + rows.append({ + "subject_id": wide["subject_id"], + "locus": locus, + "segment": wide["segment"], + "gene": gene, + "allele": allele, + "chromosome": chromosome, + "novel": allele in novel, + "gene_state": wide["zygosity"], + }) + return rows + + +def unspecified_genes(genotype) -> List[Tuple[str, str]]: + """The reference genes this genotype does not pin, as ``(segment, gene)``. + + Empty for a strict or completed genotype. Non-empty only for a + ``Genotype.permissive`` that was never completed, where the engine samples + the unspecified genes over the whole reference: those alleles reach the + records but no row of the projection, which is the one way this table can + be a silent subset of the truth. + + Keyed on what is specified rather than on ``is_permissive``, because + ``permissive().complete_from_reference()`` stays permissive while its + table becomes complete. + """ + cfg = getattr(genotype, "_cfg", None) + if cfg is None: + return [] + missing: List[Tuple[str, str]] = [] + for segment in _SEGMENTS: + for gene in _alleles_by_gene(cfg, segment): + if not genotype.is_specified(segment, gene): + missing.append((segment, gene)) + return missing + + +def write_genotype_tsv(path: str, genotypes) -> None: + """Write the long truth table for ``genotypes`` to ``path`` as TSV. + + Refuses, before opening the file, when there is nothing to write or when + a genotype cannot be joined. Both refusals exist for the same reason: a + truth file that is empty, or whose ``subject_id`` is blank, sits beside a + records file looking like an export that worked. Raising leaves the + caller with an error instead of a plausible artifact. + """ + import csv + + supplied = list(genotypes or ()) + carried = [g for g in supplied if g is not None] + if not carried: + raise ValueError( + "this result has no genotype, so there is no genotype truth to " + "write; run with `.with_genotype(...)` or `run_cohort(...)` to " + "produce a result that carries one") + + # Dropping the Nones would write a table that is short by whole subjects + # and says nothing about it -- a partial file that looks complete. + missing_genotype = [i for i, g in enumerate(supplied) if g is None] + if missing_genotype: + raise ValueError( + f"subject(s) at position(s) {missing_genotype} have no genotype, so " + f"the sidecar would be missing those subjects entirely while " + f"looking complete; give every subject a genotype or export the " + f"ones that have them separately") + + unjoinable = [i for i, g in enumerate(carried) if not getattr(g, "subject_id", None)] + if unjoinable: + raise ValueError( + f"genotype(s) at position(s) {unjoinable} carry no subject_id, so " + f"the sidecar could not be joined to the records; set one with " + f"`.with_subject(...)`") + + seen: Dict[str, int] = {} + for index, genotype in enumerate(carried): + sid = genotype.subject_id + if sid in seen: + raise ValueError( + f"subject_id {sid!r} appears on genotypes at positions " + f"{seen[sid]} and {index}, so their rows would merge under one " + f"join key and neither subject's truth would be recoverable; " + f"give each subject a distinct id with `.with_subject(...)`") + seen[sid] = index + + for genotype in carried: + missing = unspecified_genes(genotype) + if missing: + shown = ", ".join(f"{seg}:{gene}" for seg, gene in missing[:3]) + raise ValueError( + f"genotype {getattr(genotype, 'subject_id', None)!r} leaves " + f"{len(missing)} reference gene(s) unspecified (e.g. {shown}), " + f"so this table is not the set of alleles the records were " + f"generated from: the engine samples those genes over the " + f"whole reference, unphased, and they would be missing from " + f"the file. Call `complete_from_reference()` to make the " + f"carried set explicit, or export the partial picture with " + f"`Genotype.to_tsv(...)`, which records that it is permissive") + + rows = [row for g in carried for row in genotype_truth_rows(g)] + with open(path, "w", newline="", encoding="utf-8") as fh: + # LF, matching `_write_delimited`'s deliberate `lineterminator="\n"`. + # The excel dialect's default CRLF would leave a stray \r on + # `gene_state`, the last column, in any shell pipeline that joins the + # sidecar to the records file. + writer = csv.writer(fh, delimiter="\t", lineterminator="\n") + writer.writerow(COLUMNS) + for row in rows: + writer.writerow([row[c] for c in COLUMNS]) diff --git a/src/GenAIRR/_result_export.py b/src/GenAIRR/_result_export.py index ff28060d..e2e42dfe 100644 --- a/src/GenAIRR/_result_export.py +++ b/src/GenAIRR/_result_export.py @@ -328,6 +328,22 @@ def to_tsv(self, path: str, *, airr_strict: bool = True) -> None: """ self._write_delimited(path, "\t", airr_strict=airr_strict) + def to_genotype_tsv(self, path: str) -> None: + """Write this run's genotype ground truth beside its records. + + One row per (gene, chromosome, carried allele), joined to the records + on ``subject_id``. A gene deleted on a chromosome keeps its row with + ``(none)`` in the allele cell -- "absent from the table" and "present + and deleted" are different truths. + + Raises ``ValueError`` on a result with no genotype rather than writing + an empty file: a zero-row truth file beside a million records is a + failure that looks like success. + """ + from ._genotype_truth import write_genotype_tsv + + write_genotype_tsv(path, getattr(self, "_genotypes", None)) + def to_csv(self, path: str, *, airr_strict: bool = False) -> None: """Write the records as comma-separated values. Convenience alongside :meth:`to_tsv` — most analysis tooling prefers TSV diff --git a/src/GenAIRR/cohort.py b/src/GenAIRR/cohort.py index f0a2f198..0b4200b5 100644 --- a/src/GenAIRR/cohort.py +++ b/src/GenAIRR/cohort.py @@ -147,6 +147,17 @@ def to_csv(self, path: str, *, airr_strict: bool = False) -> None: DataFrame).""" self.to_dataframe(airr_strict=airr_strict).to_csv(path, index=False) + def to_genotype_tsv(self, path: str) -> None: + """Write every subject's genotype ground truth into one table. + + The cohort counterpart of :meth:`SimulationResult.to_genotype_tsv`: + the same long form, with ``subject_id`` distinguishing the subjects, + so one sidecar joins to the concatenated records file. + """ + from ._genotype_truth import write_genotype_tsv + + write_genotype_tsv(path, self.genotypes) + def to_fasta(self, path: str) -> None: """Write one FASTA record per concatenated record. Header is exactly ``>{sequence_id}`` (the namespaced id); body is the record's diff --git a/src/GenAIRR/genotype/model.py b/src/GenAIRR/genotype/model.py index 5400c5d8..0333a3ac 100644 --- a/src/GenAIRR/genotype/model.py +++ b/src/GenAIRR/genotype/model.py @@ -519,7 +519,9 @@ def _fmt(detail): return ";".join(f"{a}:{c}:{w}" for (a, c, w) in detail) rows = self.to_table() - with open(path, "w", newline="") as fh: + # Explicit encoding: the default is locale-dependent, so on Windows + # this wrote cp1252 and a non-ASCII subject_id raised. + with open(path, "w", newline="", encoding="utf-8") as fh: w = csv.writer(fh, delimiter="\t") w.writerow( [ diff --git a/tests/biology/catalog/_stages.py b/tests/biology/catalog/_stages.py index f582f783..30850611 100644 --- a/tests/biology/catalog/_stages.py +++ b/tests/biology/catalog/_stages.py @@ -31,7 +31,7 @@ #: field nobody is answerable for is a field nobody has to get right, so the #: alternative to a claim is an explicit entry here -- not silence. #: -#: Nine of these are also listed in the ``truth_fields`` of the mechanism they +#: Fourteen of these are also listed in the ``truth_fields`` of the mechanism they #: belong to, which is deliberate and is not a contradiction: that is where a #: reader looks for them. An entry here means the field does not count toward #: its mechanism's coverage, because nothing a test could do would establish a @@ -40,8 +40,19 @@ UNCLAIMED_FIELDS = { "sequence_id": "a label the harness assigns, not simulated biology; " "clonality.founder claims it for clone identity instead", - "locus": "claimed by germline.genotype as the only exported field it has; " - "see that mechanism's truth_gap", + "locus": "the locus the rearrangement was drawn from, echoed onto every record " + "from the cartridge's identity rather than sampled per record; " + "claimed by germline.genotype because that is where a reader " + "looks for it", + "subject_id": "a label the caller supplied, not a drawn outcome: the " + "engine samples nothing per record for it and it is the same " + "string on every record of a subject, the same shape as " + "sequence_id. Excluded because no test can establish its " + "VALUE, which is not the same as nothing being asserted " + "about it -- the export tests do hold it to the join, " + "requiring the sidecar's key to equal the records' key. It " + "is listed beside germline.genotype because that join is " + "what makes it worth exporting at all", "productivity_policy": "configuration echoed back, not an outcome", "productivity_scope": "configuration echoed back, not an outcome", "read_layout": "the configured layout name echoed back -- 'paired_end' " @@ -99,14 +110,38 @@ stage="germline", summary="Which alleles this individual carries, across both haplotypes, " "including whether a haplotype is functional at all.", - truth_fields=("locus",), + truth_fields=("locus", "haplotype", "subject_id"), engine_passes=("SampleGenotypePass",), - truth_gap="The genotype itself is not exported. There is no haplotype or " - "chromosome field, no carried-allele set, and no per-read " - "novel-allele or deletion truth, so `locus` is the only field " - "available to claim. Until that truth is exported, " - "genotype and haplotype inference tools (TIgGER, piglet, " - "RAbHIT) cannot be benchmarked against GenAIRR.", + truth_gap="Per-chromosome expression weight is not exported. " + "`Genotype.chromosome_weights(w0, w1)` decides how often " + "each chromosome is drawn from, and no export carries it: " + "not the record, not `to_table`, not the sidecar, and " + "`to_metadata` records only that the value was set " + "manually, never the value. So a genotype with weights " + "(1.0, 0.0) projects 85 rows on this cartridge claiming " + "haplotype-1 alleles are carried while no record can ever " + "draw one, and a caller that correctly reports them absent " + "is scored wrong with nothing in the truth files to explain " + "why. This is the remaining part of the summary's clause " + "\"whether a haplotype is functional at all\".", + notes="This mechanism carried a wider truth_gap saying the genotype " + "\"is not exported\" and that there is \"no haplotype or " + "chromosome field\". The second half was simply wrong: " + "`haplotype` -- the chromosome this rearrangement drew from -- " + "has been recorded by the engine as a `sample_haplotype` trace " + "value and stamped onto every record of a genotype run by " + "`_compiled/plain.py` all along, and no mode in the field " + "census attached a genotype, so the completeness checks could " + "not see it. The carried-allele set " + "is now exported too, beside the records rather than in them: " + "`to_genotype_tsv` writes one row per carried allele, keyed on " + "`subject_id`, with a chromosome column, a novel flag and a " + "`(none)` sentinel where a gene is deleted. It is a sidecar " + "because it is per subject, not per read -- a million reads " + "would otherwise carry the same table a million times. Whether " + "a given read's allele was novel is answered by joining " + "`truth_v_call` to that table on (subject_id, allele), which is " + "why no per-read novelty field exists.", ), Mechanism( id="germline.allele_choice", diff --git a/tests/biology/test_catalog_integrity.py b/tests/biology/test_catalog_integrity.py index 8d9f3171..4f5da1b8 100644 --- a/tests/biology/test_catalog_integrity.py +++ b/tests/biology/test_catalog_integrity.py @@ -188,7 +188,7 @@ def test_a_truth_gap_stays_visible_even_when_the_mechanism_verifies(): def test_a_field_the_catalog_calls_unclaimed_does_not_count_toward_coverage(): - """Thirteen fields are in both UNCLAIMED_FIELDS and some mechanism's truth + """Fourteen fields are in both UNCLAIMED_FIELDS and some mechanism's truth fields, and the entries say why: a configuration echo, a harness label, a derived ratio, or an inference output of a search this tool does not run. Each is listed beside the mechanism it belongs to but is not something that @@ -234,6 +234,7 @@ def test_the_fields_excluded_from_coverage_are_pinned(): assert excluded == { "sequence_id", # clonality.founder "locus", # germline.genotype + "subject_id", # germline.genotype "productivity_policy", # expression.productivity "productivity_scope", # expression.productivity "read_layout", # assay.paired_end @@ -589,7 +590,28 @@ def test_the_coverage_report_numbers_are_pinned(): that check on every record. A partition claim needs a test that the buckets *differ*, not only that they add up. - So the surface is now: 22 verified, 3 at a declared gapped ceiling, 4 + germline.genotype then moved from `no-truth-exported` to a declared + gapped ceiling, which is a narrowing of its gap rather than a test being + added. The old gap claimed the genotype "is not exported" and that there + is "no haplotype or chromosome field". The second half had never been + true: `haplotype` is recorded by the engine and stamped on every record + of a genotype run, and the field census had no genotype mode, so no + completeness check here could see it -- the same blind spot the duplex + mode was added to fix, and the reason a wrong statement about the record + survived in the catalog at all. The first half is closed by + `to_genotype_tsv`, which writes the carried-allele set beside the records. + + What is left is narrower and real, so the gap stays in that reduced form: + per-chromosome expression weight is not exported anywhere. A genotype + with weights (1.0, 0.0) projects rows claiming haplotype-1 alleles are + carried while no record can ever draw one. That is the remaining piece of + this mechanism's own summary clause, "whether a haplotype is functional + at all", which is why deleting the gap outright would have over-claimed. + The row is covered on `haplotype` by the phasing test, which reads the + drawn chromosome off a genotype it constructed itself; `locus` and + `subject_id` are excluded from the denominator, so one field carries it. + + So the surface is now: 22 verified, 4 at a declared gapped ceiling, 3 with no exported truth at all, and 2 consistent. `unverified` is empty. """ tests, covered, bases = _declared_registrations() @@ -605,7 +627,7 @@ def test_the_coverage_report_numbers_are_pinned(): status = {row.mechanism_id: row.status for row in rows} expected = { "germline.allele_choice": "verified", - "germline.genotype": "no-truth-exported", + "germline.genotype": "gap-limited", "recombination.assembly": "consistent", "recombination.d_inversion": "verified", "recombination.trimming": "verified", @@ -649,7 +671,7 @@ def test_the_coverage_report_numbers_are_pinned(): "field moved without anyone noticing.") counts = {name: sum(1 for value in status.values() if value == name) for name in sorted(set(status.values()))} - assert counts == {"consistent": 2, "gap-limited": 3, "no-truth-exported": 4, + assert counts == {"consistent": 2, "gap-limited": 4, "no-truth-exported": 3, "verified": 22}, counts @@ -786,3 +808,20 @@ def test_a_registered_test_that_can_skip_its_assertions_proves_it_did_not(): + "\n".join(f" {name}" for name in offenders) + "\n\nAdd an assertion on the count compared, or on the collection " "filled, so a run that compared nothing fails.") + + +def test_the_field_census_includes_a_genotype_run(): + """A genotype run stamps two fields no other mode produces. + + ``subject_id`` and ``haplotype`` are written onto every record when a + genotype is attached, and no mode in the census attaches one -- so the + completeness checks above cannot see them. That is the same failure the + duplex mode was added to fix, and it is how ``germline.genotype``'s + truth_gap came to say there is "no haplotype or chromosome field" while + the engine had been stamping one all along. + """ + from tests.support.fixtures.record_modes import fields_by_mode + modes = fields_by_mode() + assert "genotype" in modes, f"census modes: {sorted(modes)}" + assert {"subject_id", "haplotype"} <= modes["genotype"], ( + f"genotype mode produced: {sorted(modes['genotype'])}") diff --git a/tests/components/genotype/test_genotype_engine.py b/tests/components/genotype/test_genotype_engine.py index ec59a963..23cf98dd 100644 --- a/tests/components/genotype/test_genotype_engine.py +++ b/tests/components/genotype/test_genotype_engine.py @@ -2,6 +2,7 @@ import GenAIRR as ga import GenAIRR.data as gdata from GenAIRR.genotype import Genotype +from tests.biology.catalog import verifies import pytest @@ -98,6 +99,16 @@ def test_heterozygous_expression_is_roughly_balanced(): assert 0.35 < frac0 < 0.65, frac0 +@verifies("germline.genotype", + claim="`haplotype` names the chromosome the rearrangement actually " + "drew from. The genotype is phased so that each chromosome " + "carries exactly one V gene and one J gene, which makes the " + "drawn chromosome readable off the calls themselves, and the " + "field agrees with it on every record. The phasing comes from " + "the genotype this test constructs, so a wrong `haplotype` " + "cannot also move what it is checked against", + fields=("haplotype",), + basis="reference") def test_vdj_phasing_links_v_and_j_to_one_chromosome(): """The core phasing guarantee: with hap0 = {V:gv0, J:gj0} and hap1 = {V:gv1, J:gj1}, every rearrangement's V and J must come from @@ -123,6 +134,11 @@ def test_vdj_phasing_links_v_and_j_to_one_chromosome(): res = ga.Experiment.on(cfg).with_genotype(g).recombine().run_records( n=200, seed=4, expose_provenance=True ) + # Both chromosomes must actually be drawn from, or the haplotype claim + # below holds in one direction only: a run that never left hap0 satisfies + # every assertion while proving nothing about hap1. + assert {r["haplotype"] for r in res} == {0, 1}, ( + f"only drew from {sorted({r['haplotype'] for r in res})}") for r in res: v_is_0 = r["truth_v_call"].startswith(gv0 + "*") j_is_0 = r["truth_j_call"].startswith(gj0 + "*") diff --git a/tests/components/genotype/test_genotype_truth_completeness.py b/tests/components/genotype/test_genotype_truth_completeness.py new file mode 100644 index 00000000..cc5e04bc --- /dev/null +++ b/tests/components/genotype/test_genotype_truth_completeness.py @@ -0,0 +1,208 @@ +"""The test that makes the genotype sidecar trustworthy. + +A sidecar is only ground truth if it accounts for every allele the records +were actually generated from. Measuring that once, by hand, proves it for one +run; this turns the measurement into a standing guarantee. + +The contract: for every record, the allele named in `truth_v_call` / +`truth_d_call` / `truth_j_call` appears in that subject's sidecar rows as +carried on some chromosome. A sidecar that misses one is worse than absent -- +a benchmark scoring a caller against it would count a correct call as wrong. + +Every assertion here is paired with a guard on its own premise. A completeness +test runs green on a degenerate repertoire: one subject, one allele per +segment, no deletions. Those guards are what stop this from passing for the +wrong reason. +""" +import pytest + +import GenAIRR as ga +import GenAIRR.data as gdata +from GenAIRR._genotype_truth import DELETED_ALLELE, genotype_truth_rows +from GenAIRR.genotype import Genotype + +pytestmark = pytest.mark.behavior + +SUBJECTS = 3 +PER_SUBJECT = 150 + + +def _cfg(): + return gdata.HUMAN_IGH_OGRDB + + +@pytest.fixture(scope="module") +def cohort(): + """Three sampled subjects, one carrying a deliberately deleted gene. + + The deletion is forced rather than hoped for: whether `Genotype.sample` + happens to delete a gene at these seeds is not something this test should + depend on, and a sidecar that mishandles deletions is exactly the defect + worth catching. + """ + cfg = _cfg() + genotypes = [Genotype.sample(cfg, seed=i, subject_id=f"S{i}") for i in range(SUBJECTS)] + gene = next(iter(cfg.v_alleles)) + genotypes[-1] = genotypes[-1].delete_gene(gene, haplotype="both") + return ga.Experiment.on(cfg).recombine().run_cohort( + genotypes, n_per_subject=PER_SUBJECT, seed=7, expose_provenance=True) + + +def _carried_by_subject(cohort): + carried = {} + for subject in cohort.subjects: + rows = genotype_truth_rows(subject.genotype) + carried[subject.subject_id] = { + r["allele"] for r in rows if r["allele"] != DELETED_ALLELE} + return carried + + +class TestThePremisesThisRunMustSatisfy: + """Guards first. If any of these fails, the completeness assertions below + would be passing on a repertoire too degenerate to mean anything.""" + + def test_the_run_produced_records_for_every_subject(self, cohort): + counts = {s.subject_id: len(s.result) for s in cohort.subjects} + assert len(counts) == SUBJECTS, counts + assert all(n == PER_SUBJECT for n in counts.values()), counts + + def test_the_records_carry_the_truth_call_fields(self, cohort): + rec = cohort.records[0] + for field in ("truth_v_call", "truth_d_call", "truth_j_call", "subject_id"): + assert field in rec, f"{field} missing; expose_provenance did not take effect" + + def test_more_than_one_allele_was_drawn_per_segment(self, cohort): + """A run that drew a single V allele would satisfy completeness + trivially.""" + for segment in ("v", "d", "j"): + drawn = {r[f"truth_{segment}_call"] for r in cohort.records} + drawn.discard("") + assert len(drawn) > 1, f"only {drawn} drawn for {segment}; too degenerate" + + def test_at_least_one_subject_carries_a_fully_deleted_gene(self, cohort): + """Fully deleted, not merely one chromosome short. + + `test_a_fully_deleted_gene_is_never_drawn_from` skips any subject + with no gene deleted on BOTH chromosomes. A guard keyed on the + `(none)` sentinel alone is satisfied by a hemizygous deletion, which + would leave that test asserting nothing while this one still passed, + so the guard has to match the state the test actually needs. + """ + deleted = [ + (s.subject_id, r["gene"]) + for s in cohort.subjects + for r in genotype_truth_rows(s.genotype) + if r["allele"] == DELETED_ALLELE and r["gene_state"] == "deleted" + ] + assert deleted, "no fully deleted gene anywhere; that path is untested" + + def test_the_subjects_genotypes_are_not_identical(self, cohort): + signatures = { + s.subject_id: frozenset( + (r["gene"], r["chromosome"], r["allele"]) + for r in genotype_truth_rows(s.genotype)) + for s in cohort.subjects + } + assert len(set(signatures.values())) == SUBJECTS, ( + "subjects share a genotype; a per-subject sidecar could not be " + "told apart from one table repeated") + + +class TestEveryDrawnAlleleIsInThatSubjectsSidecar: + def test_no_record_names_an_allele_its_subject_does_not_carry(self, cohort): + carried = _carried_by_subject(cohort) + missing = [] + for rec in cohort.records: + sid = rec["subject_id"] + for segment in ("v", "d", "j"): + allele = rec[f"truth_{segment}_call"] + if allele and allele not in carried[sid]: + missing.append((sid, segment, allele)) + assert not missing, ( + f"{len(missing)} record/allele pairs name an allele absent from " + f"the subject's genotype truth: {sorted(set(missing))[:10]}") + + def test_a_subjects_alleles_are_not_satisfied_by_another_subjects_sidecar(self, cohort): + """The join is per subject. If completeness only held when pooling + every subject's alleles, the sidecar would not be per-subject truth. + """ + carried = _carried_by_subject(cohort) + pooled = set().union(*carried.values()) + assert any(carried[sid] != pooled for sid in carried), ( + "every subject carries the pooled allele set, so this test cannot " + "distinguish per-subject truth from pooled truth") + + def test_a_deleted_gene_is_never_drawn_from(self, cohort): + """The other direction: a gene the genotype deleted must not appear in + any of that subject's records.""" + for subject in cohort.subjects: + rows = genotype_truth_rows(subject.genotype) + fully_deleted = { + r["gene"] for r in rows if r["gene_state"] == "deleted"} + if not fully_deleted: + continue + drawn_genes = set() + for rec in subject.result: + for segment in ("v", "d", "j"): + allele = rec[f"truth_{segment}_call"] + if allele: + drawn_genes.add(allele.split("*")[0]) + assert not (drawn_genes & fully_deleted), ( + f"{subject.subject_id} drew from deleted gene(s): " + f"{sorted(drawn_genes & fully_deleted)}") + + +class TestNovelAllelesAreMarkedInTheSidecar: + def test_a_carried_novel_allele_is_flagged_and_reaches_the_records(self): + """Pins both halves at once: the sidecar marks the novel allele, and + the records really did draw it, so the flag describes live truth. + + The novel gene is made the subject's *only* V gene, because every V + gene in this cartridge carries the same usage weight -- drawing one + specific gene is a ~0.3% event per record, so a test that merely hoped + for the draw would need thousands of records and still flake. Deleting + the rest makes the draw a certainty rather than a probability. + """ + cfg = _cfg() + genes = list(cfg.v_alleles) + target = genes[0] + novel = f"{target}*i01" + genotype = (Genotype.from_dataconfig(cfg) + .complete_from_reference() + .add_novel_allele(novel, base=cfg.v_alleles[target][0].name, + mutations=[(10, "A")], allow_nonfunctional=True) + .homozygous(target, novel)) + for other in genes[1:]: + genotype = genotype.delete_gene(other, haplotype="both") + genotype = genotype.with_subject("N0") + + result = (ga.Experiment.on(cfg).with_genotype(genotype).recombine() + .run_records(n=30, seed=3, expose_provenance=True)) + + drawn = {r["truth_v_call"] for r in result} + assert drawn == {novel}, ( + f"premise: the novel allele must be the only V drawn; got {sorted(drawn)[:5]}") + + rows = genotype_truth_rows(genotype) + flagged = {r["allele"] for r in rows if r["novel"]} + assert flagged == {novel}, flagged + carried = {r["allele"] for r in rows if r["allele"] != DELETED_ALLELE} + assert drawn <= carried, "the drawn allele is absent from the sidecar" + + def test_the_other_v_genes_are_present_as_deleted_not_absent(self): + """The same genotype's 51 deleted genes must still be in the table. + This is the completeness property that a long form could silently + drop, checked on a genotype where deletions are the majority.""" + cfg = _cfg() + genes = list(cfg.v_alleles) + genotype = Genotype.from_dataconfig(cfg).complete_from_reference() + for other in genes[1:]: + genotype = genotype.delete_gene(other, haplotype="both") + genotype = genotype.with_subject("N0") + + rows = genotype_truth_rows(genotype) + deleted_genes = {r["gene"] for r in rows if r["allele"] == DELETED_ALLELE} + + assert deleted_genes == set(genes[1:]), ( + f"{len(set(genes[1:]) - deleted_genes)} deleted V genes vanished " + f"from the truth table instead of being reported as deleted") diff --git a/tests/components/genotype/test_genotype_truth_export.py b/tests/components/genotype/test_genotype_truth_export.py new file mode 100644 index 00000000..0fe5c25a --- /dev/null +++ b/tests/components/genotype/test_genotype_truth_export.py @@ -0,0 +1,338 @@ +"""`to_genotype_tsv` on a single-subject result and on a cohort. + +A records file carries `subject_id` and nothing that says what that subject's +genome was. These tests pin the sidecar that closes the join, and they pin the +refusal harder than the happy path: a zero-row truth file written beside a +million records is the failure mode that looks like success, so a result with +no genotype must raise by name and leave no file behind. +""" +import csv + +import pytest + +import GenAIRR as ga +import GenAIRR.data as gdata +from GenAIRR._genotype_truth import COLUMNS, DELETED_ALLELE, genotype_truth_rows +from GenAIRR.cohort import CohortResult, CohortSubjectResult +from GenAIRR.genotype import Genotype + +pytestmark = pytest.mark.behavior + + +def _cfg(): + return gdata.HUMAN_IGH_OGRDB + + +def _read(path): + with open(path, newline="", encoding="utf-8") as fh: + return list(csv.DictReader(fh, delimiter="\t")) + + +def _run(genotype, n=6, seed=1): + return ga.Experiment.on(_cfg()).with_genotype(genotype).recombine().run_records(n=n, seed=seed) + + +class TestASingleSubjectSidecar: + def test_its_subject_id_matches_every_record(self, tmp_path): + g = Genotype.sample(_cfg(), seed=0, subject_id="S0") + result = _run(g) + out = tmp_path / "genotype.tsv" + + result.to_genotype_tsv(str(out)) + + rows = _read(out) + assert rows, "premise: the sidecar must not be empty" + record_subjects = {r["subject_id"] for r in result} + assert record_subjects == {"S0"}, "premise: the records must carry the subject" + assert {r["subject_id"] for r in rows} == record_subjects + + def test_the_header_is_the_long_form_columns(self, tmp_path): + g = Genotype.sample(_cfg(), seed=0, subject_id="S0") + out = tmp_path / "genotype.tsv" + + _run(g).to_genotype_tsv(str(out)) + + with open(out, newline="", encoding="utf-8") as fh: + header = next(csv.reader(fh, delimiter="\t")) + assert header == list(COLUMNS) + + def test_the_file_is_the_projection_and_nothing_else(self, tmp_path): + g = Genotype.sample(_cfg(), seed=0, subject_id="S0") + out = tmp_path / "genotype.tsv" + + _run(g).to_genotype_tsv(str(out)) + + expected = genotype_truth_rows(g) + rows = _read(out) + assert len(rows) == len(expected) + assert [r["allele"] for r in rows] == [r["allele"] for r in expected] + assert [int(r["chromosome"]) for r in rows] == [r["chromosome"] for r in expected] + + def test_a_deleted_gene_reaches_the_file_as_the_sentinel(self, tmp_path): + """End to end for the decision that a deleted gene stays in the table. + + Built from a complete genotype, not a permissive one. A permissive + genotype reaches the writer's refusal now, and it was the wrong + premise before that: the deletion was real, but every unspecified + gene was missing from the file the test then read, so it asserted + the sentinel inside a table that was already a subset of the truth. + """ + cfg = _cfg() + gene = next(iter(cfg.v_alleles)) + g = (Genotype.from_dataconfig(cfg).complete_from_reference() + .delete_gene(gene, haplotype="both") + .with_subject("S0")) + out = tmp_path / "genotype.tsv" + + _run(g).to_genotype_tsv(str(out)) + + deleted = [r for r in _read(out) if r["gene"] == gene] + assert len(deleted) == 2, deleted + assert {r["allele"] for r in deleted} == {DELETED_ALLELE} + assert {r["gene_state"] for r in deleted} == {"deleted"} + + +class TestACohortSidecar: + def test_it_carries_every_subject_and_nothing_else(self, tmp_path): + cfg = _cfg() + g0 = Genotype.sample(cfg, seed=0, subject_id="A") + g1 = Genotype.sample(cfg, seed=1, subject_id="B") + cohort = CohortResult([ + CohortSubjectResult(subject_id="A", genotype=g0, result=_run(g0, n=3, seed=1), + refdata=object(), seed=11, count=3), + CohortSubjectResult(subject_id="B", genotype=g1, result=_run(g1, n=3, seed=2), + refdata=object(), seed=22, count=3), + ]) + out = tmp_path / "cohort_genotype.tsv" + + cohort.to_genotype_tsv(str(out)) + + rows = _read(out) + assert {r["subject_id"] for r in rows} == {"A", "B"} + assert len(rows) == len(genotype_truth_rows(g0)) + len(genotype_truth_rows(g1)) + + def test_two_subjects_genotypes_are_not_collapsed(self, tmp_path): + """Premise and point in one: the two sampled genotypes must actually + differ, or the test could not tell a per-subject table from a + duplicated one.""" + cfg = _cfg() + g0 = Genotype.sample(cfg, seed=0, subject_id="A") + g1 = Genotype.sample(cfg, seed=1, subject_id="B") + a_truth = {(r["gene"], r["chromosome"], r["allele"]) for r in genotype_truth_rows(g0)} + b_truth = {(r["gene"], r["chromosome"], r["allele"]) for r in genotype_truth_rows(g1)} + assert a_truth != b_truth, "premise: two subjects must differ somewhere" + + cohort = CohortResult([ + CohortSubjectResult(subject_id="A", genotype=g0, result=_run(g0, n=3, seed=1), + refdata=object(), seed=11, count=3), + CohortSubjectResult(subject_id="B", genotype=g1, result=_run(g1, n=3, seed=2), + refdata=object(), seed=22, count=3), + ]) + out = tmp_path / "cohort_genotype.tsv" + cohort.to_genotype_tsv(str(out)) + + rows = _read(out) + got_a = {(r["gene"], int(r["chromosome"]), r["allele"]) for r in rows + if r["subject_id"] == "A"} + got_b = {(r["gene"], int(r["chromosome"]), r["allele"]) for r in rows + if r["subject_id"] == "B"} + assert got_a != got_b + + +class TestARunWithNoGenotypeRefuses: + """A zero-row truth file beside a million records is the failure that + looks like success. It must raise, by name, and write nothing.""" + + def test_a_plain_result_raises_naming_the_cause(self, tmp_path): + result = ga.Experiment.on(_cfg()).recombine().run_records(n=4, seed=1) + out = tmp_path / "genotype.tsv" + + with pytest.raises(ValueError, match="no genotype"): + result.to_genotype_tsv(str(out)) + + def test_the_refusal_leaves_no_file_behind(self, tmp_path): + result = ga.Experiment.on(_cfg()).recombine().run_records(n=4, seed=1) + out = tmp_path / "genotype.tsv" + + with pytest.raises(ValueError): + result.to_genotype_tsv(str(out)) + + assert not out.exists(), "a refused export must not leave a partial file" + + def test_a_genotype_with_no_subject_id_refuses(self, tmp_path): + """A sidecar whose subject_id is blank cannot be joined to anything, + so it is the same failure as an empty one: plausible and useless.""" + cfg = _cfg() + gene = next(iter(cfg.v_alleles)) + nameless = Genotype.permissive(cfg).homozygous(gene, cfg.v_alleles[gene][0].name) + assert not nameless.subject_id, "premise: the genotype must carry no subject_id" + out = tmp_path / "genotype.tsv" + + with pytest.raises(ValueError, match="no subject_id"): + _run(nameless).to_genotype_tsv(str(out)) + + assert not out.exists() + + def test_a_cohort_with_no_genotypes_raises(self, tmp_path): + plain = ga.Experiment.on(_cfg()).recombine().run_records(n=2, seed=1) + cohort = CohortResult([ + CohortSubjectResult(subject_id="A", genotype=None, result=plain, + refdata=object(), seed=11, count=2), + ]) + out = tmp_path / "cohort_genotype.tsv" + + with pytest.raises(ValueError, match="no genotype"): + cohort.to_genotype_tsv(str(out)) + assert not out.exists() + + +class TestAnIncompleteGenotypeRefuses: + """A permissive genotype is not the set of alleles the records came from. + + `Genotype.permissive` leaves unspecified genes to be sampled over the + whole reference, unphased -- its own docstring says never to mistake it + for a complete genotype. Projecting one yields a row only for the genes + somebody happened to constrain, so the file is silently a subset of the + truth: measured on this cartridge, a permissive genotype with one gene + pinned writes two rows while the records draw 119 alleles that are not in + it. A benchmark scoring a caller against that file marks correct calls + wrong, which is worse than having no file, and it is the same failure the + empty-result refusal already guards. + """ + + def test_a_permissive_genotype_with_unspecified_genes_refuses(self, tmp_path): + g = Genotype.permissive(_cfg()).with_subject("P0") + result = _run(g) + out = tmp_path / "genotype.tsv" + with pytest.raises(ValueError, match="unspecified"): + result.to_genotype_tsv(str(out)) + + def test_the_refusal_says_how_to_complete_the_genotype(self, tmp_path): + g = Genotype.permissive(_cfg()).with_subject("P0") + result = _run(g) + with pytest.raises(ValueError, match="complete_from_reference"): + result.to_genotype_tsv(str(tmp_path / "genotype.tsv")) + + def test_the_refusal_leaves_no_file_behind(self, tmp_path): + g = Genotype.permissive(_cfg()).with_subject("P0") + result = _run(g) + out = tmp_path / "genotype.tsv" + with pytest.raises(ValueError): + result.to_genotype_tsv(str(out)) + assert not out.exists() + + def test_one_pinned_gene_does_not_buy_a_sidecar(self, tmp_path): + """The dangerous shape: enough specified to look deliberate.""" + cfg = _cfg() + gene = next(iter(cfg.v_alleles)) + g = (Genotype.permissive(cfg) + .homozygous(gene, cfg.v_alleles[gene][0].name) + .with_subject("P1")) + result = _run(g, n=40, seed=2) + with pytest.raises(ValueError, match="unspecified"): + result.to_genotype_tsv(str(tmp_path / "genotype.tsv")) + + def test_a_completed_permissive_genotype_is_accepted(self, tmp_path): + """Completing it is what makes the table the carried set. + + `permissive` stays set, so the guard must test what is specified + rather than which constructor was used -- refusing every permissive + genotype would refuse this one, whose table is complete. + """ + cfg = _cfg() + g = (Genotype.permissive(cfg).complete_from_reference() + .with_subject("P2")) + result = _run(g, n=40, seed=3) + out = tmp_path / "genotype.tsv" + result.to_genotype_tsv(str(out)) + carried = {row["allele"] for row in _read(str(out))} + assert g.is_permissive, "premise: this genotype is still permissive" + drawn = {rec[f] for rec in result + for f in ("v_call", "d_call", "j_call") if rec.get(f)} + assert drawn, "premise: the run drew some alleles" + assert drawn <= carried, f"drawn but absent: {sorted(drawn - carried)}" + + +class TestTheFileIsJoinableAsAFile: + """Byte-level properties a consumer joins on, not just parsed values. + + The projection tests assert Python objects; these assert what lands on + disk. Four of the eight columns had no file-level assertion, including + `novel` -- whose serialized spelling the documented join compares against + the literal string "True". A writer that emitted 1/0 or true would make + the published snippet silently return False for every read while every + other test stayed green. + """ + + def test_the_novel_column_is_spelled_the_way_the_documented_join_reads_it( + self, tmp_path): + cfg = _cfg() + gene = next(iter(cfg.v_alleles)) + base = cfg.v_alleles[gene][0].name + g = (Genotype.from_dataconfig(cfg).complete_from_reference() + .add_novel_allele(f"{gene}*i01", base=base, mutations=[(10, "A")], + allow_nonfunctional=True) + .heterozygous(gene, base, f"{gene}*i01") + .with_subject("N0")) + out = tmp_path / "genotype.tsv" + _run(g).to_genotype_tsv(str(out)) + + rows = _read(str(out)) + spellings = {row["novel"] for row in rows} + assert spellings <= {"True", "False"}, spellings + novel_rows = [r for r in rows if r["novel"] == "True"] + assert [r["allele"] for r in novel_rows] == [f"{gene}*i01"], novel_rows + + def test_every_column_carries_a_value_on_every_row(self, tmp_path): + """`locus`, `segment` and `gene` had no file-level assertion either.""" + g = Genotype.sample(_cfg(), seed=0, subject_id="S0") + out = tmp_path / "genotype.tsv" + _run(g).to_genotype_tsv(str(out)) + rows = _read(str(out)) + assert rows, "premise: the file has rows" + for column in COLUMNS: + blank = [r for r in rows if r[column] == ""] + assert blank == [], f"{column} is blank on {len(blank)} row(s)" + assert {r["locus"] for r in rows} == {"IGH"} + assert {r["segment"] for r in rows} == {"V", "D", "J"} + + def test_its_line_endings_match_the_records_file_it_joins_to(self, tmp_path): + """`_write_delimited` pins LF deliberately. A CRLF sidecar beside an LF + records file leaves a stray \\r on the last column, so `gene_state == + "deleted"` fails in any shell pipeline that joins the two.""" + g = Genotype.sample(_cfg(), seed=0, subject_id="S0") + out = tmp_path / "genotype.tsv" + _run(g).to_genotype_tsv(str(out)) + raw = out.read_bytes() + assert b"\r\n" not in raw, "sidecar is CRLF; the records writer pins LF" + + +class TestAnUnjoinableCohortRefuses: + def test_a_missing_genotype_is_refused_not_skipped(self, tmp_path): + """Dropping a `None` writes a partial table that looks complete.""" + cfg = _cfg() + g = Genotype.sample(cfg, seed=0, subject_id="S0") + result = _run(g) + cohort = CohortResult([ + CohortSubjectResult(subject_id="S0", genotype=g, result=result, + refdata=object(), seed=11, count=6), + CohortSubjectResult(subject_id="S1", genotype=None, result=result, + refdata=object(), seed=22, count=6), + ]) + with pytest.raises(ValueError, match="no genotype"): + cohort.to_genotype_tsv(str(tmp_path / "genotype.tsv")) + + def test_two_subjects_sharing_one_id_are_refused(self, tmp_path): + """Both subjects' rows would merge under one join key.""" + cfg = _cfg() + g0 = Genotype.sample(cfg, seed=0, subject_id="DUP") + g1 = Genotype.sample(cfg, seed=1, subject_id="DUP") + result = _run(g0) + cohort = CohortResult([ + CohortSubjectResult(subject_id="DUP", genotype=g0, result=result, + refdata=object(), seed=11, count=6), + CohortSubjectResult(subject_id="DUP", genotype=g1, result=result, + refdata=object(), seed=22, count=6), + ]) + with pytest.raises(ValueError, match="subject_id"): + cohort.to_genotype_tsv(str(tmp_path / "genotype.tsv")) diff --git a/tests/components/genotype/test_genotype_truth_projection.py b/tests/components/genotype/test_genotype_truth_projection.py new file mode 100644 index 00000000..dd2be0d9 --- /dev/null +++ b/tests/components/genotype/test_genotype_truth_projection.py @@ -0,0 +1,235 @@ +"""The long, per-carried-allele projection of a genotype's truth. + +`Genotype.to_table()` is per gene, with the alleles as list-valued columns +(`haplotype_0`, `haplotype_1`). That shape is wrong for a TSV a benchmark +joins against: a list inside a flat cell makes every consumer write its own +splitter. These tests pin the long form -- one row per (gene, chromosome, +carried allele) -- and, above all, pin the two things a long form can lose: + +* a **deleted** gene still gets rows. "Absent from the table" and "present + and deleted" are different truths, and a genotype caller is scored on + exactly that difference. +* the `locus` column agrees with the locus the engine puts on the records, + because the sidecar is useless if the join key disagrees with the join. +""" +import pytest + +import GenAIRR.data as gdata +from GenAIRR._genotype_truth import DELETED_ALLELE, genotype_truth_rows +from GenAIRR.genotype import Genotype + +pytestmark = pytest.mark.behavior + + +def _cfg(): + return gdata.HUMAN_IGH_OGRDB + + +def _gene_with_at_least(n, cfg=None): + """A V gene carrying at least `n` distinct alleles, or skip. + + Guards the premise: a heterozygous test built on a single-allele gene + would construct a homozygote and pass for the wrong reason. + """ + cfg = cfg or _cfg() + for gene, alleles in cfg.v_alleles.items(): + if len({a.name for a in alleles}) >= n: + return gene, sorted({a.name for a in alleles}) + pytest.skip(f"no V gene with >= {n} distinct alleles in this cartridge") + + +def _rows_for(genotype, gene): + return [r for r in genotype_truth_rows(genotype) if r["gene"] == gene] + + +class TestOneRowPerCarriedAllele: + def test_a_homozygous_gene_gives_the_same_allele_on_both_chromosomes(self): + gene, alleles = _gene_with_at_least(1) + g = Genotype.from_dataconfig(_cfg()).homozygous(gene, alleles[0]).with_subject("S1") + + rows = _rows_for(g, gene) + + assert [r["chromosome"] for r in rows] == [0, 1] + assert {r["allele"] for r in rows} == {alleles[0]} + assert {r["gene_state"] for r in rows} == {"homozygous"} + assert {r["subject_id"] for r in rows} == {"S1"} + + def test_a_heterozygous_gene_gives_a_different_allele_per_chromosome(self): + gene, alleles = _gene_with_at_least(2) + a0, a1 = alleles[0], alleles[1] + assert a0 != a1, "premise: the two alleles must differ" + g = Genotype.from_dataconfig(_cfg()).heterozygous(gene, a0, a1).with_subject("S1") + + rows = _rows_for(g, gene) + + assert [(r["chromosome"], r["allele"]) for r in rows] == [(0, a0), (1, a1)] + assert {r["gene_state"] for r in rows} == {"heterozygous"} + + def test_every_row_names_its_segment(self): + gene, alleles = _gene_with_at_least(1) + g = Genotype.from_dataconfig(_cfg()).homozygous(gene, alleles[0]).with_subject("S1") + + assert {r["segment"] for r in _rows_for(g, gene)} == {"V"} + + +class TestADeletedGeneStillGetsRows: + """The one place a long table can lose what a wide one keeps.""" + + def test_deleted_on_one_chromosome_keeps_one_allele_row_and_one_sentinel(self): + gene, alleles = _gene_with_at_least(1) + g = (Genotype.from_dataconfig(_cfg()) + .homozygous(gene, alleles[0]) + .delete_gene(gene, haplotype=1) + .with_subject("S1")) + + rows = _rows_for(g, gene) + + assert [(r["chromosome"], r["allele"]) for r in rows] == [ + (0, alleles[0]), (1, DELETED_ALLELE)] + assert {r["gene_state"] for r in rows} == {"hemizygous"} + + def test_deleted_on_both_chromosomes_gives_two_sentinel_rows(self): + gene, alleles = _gene_with_at_least(1) + g = (Genotype.from_dataconfig(_cfg()) + .homozygous(gene, alleles[0]) + .delete_gene(gene, haplotype="both") + .with_subject("S1")) + + rows = _rows_for(g, gene) + + assert [(r["chromosome"], r["allele"]) for r in rows] == [ + (0, DELETED_ALLELE), (1, DELETED_ALLELE)] + assert {r["gene_state"] for r in rows} == {"deleted"} + + def test_the_sentinel_is_a_literal_token_and_not_an_empty_cell(self): + """An empty TSV cell cannot be told apart from a missing value, which + is the one confusion a truth file must not create.""" + assert DELETED_ALLELE == "(none)" + assert DELETED_ALLELE.strip() != "" + + +class TestNovelAlleles: + """`allow_nonfunctional=True` throughout: a single substitution at a fixed + offset lands in a stop codon for some genes, and the projection does not + read functionality at all. Hunting for a position that stays functional + would couple these tests to the cartridge's codon layout for no gain.""" + + def test_a_placed_novel_allele_is_marked_novel(self): + cfg = _cfg() + gene, alleles = _gene_with_at_least(1, cfg) + base = alleles[0] + g = (Genotype.from_dataconfig(cfg) + .add_novel_allele(f"{gene}*i01", base=base, mutations=[(10, "A")], + allow_nonfunctional=True) + .homozygous(gene, f"{gene}*i01") + .with_subject("S1")) + + rows = _rows_for(g, gene) + + assert rows, "premise: the placed novel allele must produce rows" + assert all(r["novel"] is True for r in rows), rows + assert {r["allele"] for r in rows} == {f"{gene}*i01"} + + def test_a_carried_reference_allele_is_not_marked_novel(self): + gene, alleles = _gene_with_at_least(1) + g = Genotype.from_dataconfig(_cfg()).homozygous(gene, alleles[0]).with_subject("S1") + + assert all(r["novel"] is False for r in _rows_for(g, gene)) + + def test_an_unplaced_novel_allele_has_no_row(self): + """Defined but never assigned to a chromosome. It is not carried, so + it is not truth about this subject's genome.""" + cfg = _cfg() + gene, alleles = _gene_with_at_least(1, cfg) + g = (Genotype.from_dataconfig(cfg) + .add_novel_allele(f"{gene}*i01", base=alleles[0], mutations=[(10, "A")], + allow_nonfunctional=True) + .homozygous(gene, alleles[0]) + .with_subject("S1")) + + # Premise: the allele really was defined. Without this the test + # passes just as well against an `add_novel_allele` that did nothing. + assert g.has_novel(), "premise: a novel allele must be defined" + assert f"{gene}*i01" in g.novel_allele_names() + + rows = genotype_truth_rows(g) + + assert rows, "premise: the gene must still produce rows" + assert all(r["allele"] != f"{gene}*i01" for r in rows), ( + "an unplaced novel allele must not appear as carried") + + +class TestTheLocusColumn: + """What the locus column is, and what it is not. + + The class was once named for the records and never looked at one, which + is how it missed the case below: on a cartridge whose V pool mixes loci, + a record's locus and its own allele's sidecar locus genuinely differ. + The record carries one locus for the whole rearrangement, taken from the + V call; a row carries the locus of its gene. The join key is + `(subject_id, allele)`, so the difference is a readability property of + the column rather than a broken join -- but it has to be pinned, because + the obvious wrong move is to filter the sidecar by the record's locus. + """ + + def test_the_locus_is_the_prefix_the_engine_derives(self): + gene, alleles = _gene_with_at_least(1) + g = Genotype.from_dataconfig(_cfg()).homozygous(gene, alleles[0]).with_subject("S1") + + assert {r["locus"] for r in _rows_for(g, gene)} == {"IGH"} + + def test_a_name_with_no_airr_locus_prefix_yields_an_empty_locus(self): + """`derive_locus` in the engine accepts only the seven AIRR loci and + returns empty otherwise; the projection must not invent one.""" + from GenAIRR._genotype_truth import locus_of + + assert locus_of("IGHVF1-G2") == "IGH" + assert locus_of("TRBV20-1") == "TRB" + assert locus_of("igkv1-5") == "IGK" + assert locus_of("SYNTHETIC1") == "" + assert locus_of("XX") == "" + + def test_on_a_single_locus_cartridge_it_agrees_with_every_record(self): + """The property the class name used to assert, now against records.""" + import GenAIRR as ga + + g = (Genotype.from_dataconfig(_cfg()).complete_from_reference() + .with_subject("S1")) + result = (ga.Experiment.on(_cfg()).with_genotype(g).recombine() + .run_records(n=20, seed=3)) + rows = genotype_truth_rows(g) + assert rows, "premise: the genotype projects to rows" + record_loci = {rec["locus"] for rec in result} + assert record_loci == {"IGH"}, record_loci + assert {r["locus"] for r in rows} == record_loci + + def test_on_a_mixed_locus_cartridge_a_rows_locus_is_its_genes_not_the_reads(self): + """`HUMAN_TCRD_IMGT` carries both TRAV and TRDV genes, so a read with + a TRAV V call is labelled TRA while the TRDJ allele it was generated + from is a TRD row. Pinned so nobody 'fixes' the column to match the + record and loses the D/J truth.""" + import GenAIRR as ga + import GenAIRR.data as gdata + + cfg = gdata.HUMAN_TCRD_IMGT + v_prefixes = {gene[:4] for gene in cfg.v_alleles} + assert v_prefixes >= {"TRAV", "TRDV"}, ( + f"premise: this cartridge's V pool mixes loci, got {v_prefixes}") + + g = (Genotype.from_dataconfig(cfg).complete_from_reference() + .with_subject("T0")) + result = (ga.Experiment.on(cfg).with_genotype(g).recombine() + .run_records(n=40, seed=5, expose_provenance=True)) + locus_by_allele = {r["allele"]: r["locus"] for r in genotype_truth_rows(g)} + + disagreeing = [rec for rec in result + if locus_by_allele.get(rec["truth_j_call"]) != rec["locus"]] + assert disagreeing, ( + "premise: this cartridge is supposed to produce reads whose J " + "allele sits in a different locus than the read's own label") + # The join is on the allele, and it holds for every record regardless. + for rec in result: + for field in ("truth_v_call", "truth_d_call", "truth_j_call"): + allele = rec.get(field) + if allele: + assert allele in locus_by_allele, allele diff --git a/tests/support/fixtures/record_modes.py b/tests/support/fixtures/record_modes.py index 671f8c6f..c12d04fc 100644 --- a/tests/support/fixtures/record_modes.py +++ b/tests/support/fixtures/record_modes.py @@ -43,6 +43,24 @@ def _duplex(): return set(exp.run(n=1, seed=1)[0].airr_record(refs)) +def _genotype(): + """A record from a run with a genotype attached. + + Attaching one stamps ``subject_id`` and ``haplotype`` (the chromosome the + rearrangement drew from) onto every record, and those two fields exist in + no other mode. Leaving the mode out is how the catalog came to describe + them as unexported while the engine was exporting them. + """ + import GenAIRR.data as gdata + from GenAIRR.genotype import Genotype + + cfg = gdata.HUMAN_IGH_OGRDB + genotype = Genotype.permissive(cfg).with_subject("census") + result = (ga.Experiment.on(cfg).recombine() + .with_genotype(genotype).compile().run_records(n=1, seed=1)) + return set(result.records[0]) + + @functools.lru_cache(maxsize=1) def fields_by_mode() -> dict[str, frozenset[str]]: """Mode name -> the record fields that mode produces.""" @@ -55,6 +73,7 @@ def fields_by_mode() -> dict[str, frozenset[str]]: ga.Experiment.on("human_igh").recombine() .clonal_repertoire(n_clones=2, max_size=4))), "duplex": frozenset(_duplex()), + "genotype": frozenset(_genotype()), } diff --git a/tools/inventory-declarations.json b/tools/inventory-declarations.json index 38e046b3..24e73c65 100644 --- a/tools/inventory-declarations.json +++ b/tools/inventory-declarations.json @@ -1,5 +1,13 @@ { - "revised": {}, + "revised": { + "test_a_field_the_catalog_calls_unclaimed_does_not_count_toward_coverage": "docstring only: the overlap between UNCLAIMED_FIELDS and some mechanism's truth fields grew from thirteen to fourteen when germline.genotype claimed subject_id. The assertions are unchanged.", + "test_the_fields_excluded_from_coverage_are_pinned": "subject_id added to the pinned exclusion set. It is the subject label the caller supplied, identical on every record of a subject with no per-record outcome in it -- the same shape as read_layout -- and it is listed beside germline.genotype because it is the key the genotype sidecar joins on. The pin is deliberately widened, which is what this test exists to force.", + "test_the_coverage_report_numbers_are_pinned": "germline.genotype moves from no-truth-exported to verified, so the per-mechanism pin and the distribution move with it: verified 22 -> 23, no-truth-exported 4 -> 3. The reason is in the docstring: its truth_gap claimed there is 'no haplotype or chromosome field' while the engine had been stamping one, and the field census had no genotype mode to show it.", + "test_vdj_phasing_links_v_and_j_to_one_chromosome": "gains a @verifies registration for germline.genotype on `haplotype` (reference basis), and a premise guard that both chromosomes are actually drawn from. Without the guard the haplotype claim holds in one direction only: a run that never left hap0 satisfies every existing assertion while proving nothing about hap1. No existing assertion was weakened or removed.", + "fields_by_mode": "a `genotype` mode is added to the field census. subject_id and haplotype are stamped on every record of a genotype run and no mode produced them, so the catalog's completeness checks could not see them -- the same blind spot the duplex mode was added to fix. The census grows; nothing it asserted was relaxed.", + "MECHANISMS": "germline.genotype's truth_fields grow from (locus,) to (locus, haplotype, subject_id) and its truth_gap is removed in favour of a notes field. The gap said the genotype 'is not exported' and that there is 'no haplotype or chromosome field'; the second half was never true and the first is closed by to_genotype_tsv. Widening truth_fields lowers a coverage ratio rather than raising it, and the coverage pin is updated in the same commit.", + "UNCLAIMED_FIELDS": "the `locus` entry no longer points at a truth_gap that has been removed, and `subject_id` is added with its reason. The added entry raises germline.genotype's ratio, so it is also pinned in test_the_fields_excluded_from_coverage_are_pinned, which is the argument this gate asks for." + }, "replaced": {}, "allow": {} }