Cross-family calls: family decided once per genome by mean identity; no near-tie margin; family_ambiguous

results/2026-10-29_near_tie/CANDIDATE_RECORDS.md

@@ -0,0 +1,38 @@
+# Candidate reference records for the SLA2-type genera (draft, 2026-10-29)
+
+Search: PubMed (via the PubMed MCP) and NCBI Protein and Assembly (E-utilities) for MAT loci in Barnettozyma, Petasospora (Cyberlindnera fabianii), Wickerhamomyces, Ambrosiozyma, Saccharomycopsis, Citeromyces, Williopsis (Cyberlindnera jadinii), Hansenula, Kuraishia and Pachysolen. Only three leads are real; nothing here is curated yet.
+
+## 1. *Saccharomycopsis schoenii*: best candidate (published locus, RNA-seq, chromosome-scale assembly)
+
+- **Paper:** Kriti D, Barazandeh M, Ogbede JU, Giaever G, Nislow C. G3 2026;16(5), [10.1093/g3journal/jkag067](https://doi.org/10.1093/g3journal/jkag067) (PMID 41849659, PMC13148404). Chromosome-scale assembly (6 nuclear chromosomes, PacBio HiFi + Hi-C), BRAKER2 annotation, RNA-seq.
+- **What it says about MAT:** an expanded MAT system, with SLA2-MAT cassettes (MATa1, MATa2, MATalpha1, MATalpha2) on every chromosome; SLA2 stays tightly linked to the MAT genes on every chromosome while DIC1 is scattered; the cassettes are co-transcribed (189-247 bp splice junctions between MATa1 and SLA2; conserved 58 bp introns between MATa2 and MATa1). The active locus is on chromosome IV: a leftmost alpha-dominant sub-cluster (470,610-475,000: MATalpha2, MATalpha1) and a rightmost palindromic cassette (528,044-537,777; SLA2 anchor flanked by mirrored MATa1 and MATa2 arrays). The species is homothallic (Dost et al., Microbiol Res 2024, [10.1016/j.micres.2024.127691](https://doi.org/10.1016/j.micres.2024.127691), PMID 38492364; *S. lassenensis* is heterothallic).
+- **Assembly:** GenBank JBTFFQ000000000, BioProject PRJNA1364152 (data availability). The v0.7.0 genome table has GCA_054953955.1 (NCYC 2975, chromosome level, submitted 2026-02-09), which I expect to be this assembly (not checked against the WGS prefix). Our replay calls six `MTL alpha` loci there (CM143126-CM143131), each with SLA2 (40-45% identity) inside the locus, no DIC1 and no PIK1/OBP1/PAP1: the published SLA2-MAT architecture, not MTL.
+- **Record plan:** one record per active locus, `system` homothallic or `unknown`, the gene models of the deposit (annotation plus RNA-seq, tier 2), a family of its own or an SLA2-MAT family (curator).
+- **Needs:** the deposit's annotation (GFF) and exact coordinates of the chromosome IV loci; the MATa1/MATa2 intron structure.
+
+## 2. *Barnettozyma californica* and relatives
+
+- **Paper:** Hession C, Byrne KP, Wolfe KH, Butler G. PLoS Genet 2025;21(12):e1011814, [10.1371/journal.pgen.1011814](https://doi.org/10.1371/journal.pgen.1011814) (PMID 41359681, PMC12711049). A centromere formed at the MAT locus of *B. californica* (250 kb from the old one); the same chromosome carries MAT and MRP17-PUT2 about 250 kb apart in all three *Barnettozyma* species studied.
+- **MAT structure (S3 Fig, text only read):** a 460 bp centromere-associated sequence lies in a 2.8 kb intergenic region **between MATa1 and MATa2 on one side and MATalpha1 and MATalpha2 on the other**, so one locus carries both gene pairs.
+- **Our replay:** five *B. californica* assemblies (UCD09 GCA_003123585.1, UCD89 GCA_003123885.1, NRRL Y-17395 GCA_003706655.x, PB4207 GCA_023627795.1) give one MTL call of 16 kb with SLA2 inside and **DIC1 8.2 kb away**, no PIK1: the SLA2-MAT-DIC1 arrangement.
+- **Assembly with the best contiguity:** *B. discipulorum* GCA_049863505.1 (PYCC:10016, complete genome, 2025-04-23; not in our run). The accessions the paper used are in its S1 Table, which was not read.
+- **Record plan:** a record from the paper's locus (both idiomorph gene pairs at one locus), pseudohomothallic or `unknown`, tier 2; family decision for the curator (a Phaffomycetales/Barnettozyma family, or MATog-like).
+- **Needs:** S1 Table and S3 Fig (supplement) for the assembly and coordinates.
+
+## 3. *Petasospora (Cyberlindnera) fabianii*: annotation-only
+
+- Protein ONH67106 "Mating-type-like protein ALPHA1" (strain 65, MPUK01000005.1, locus tag BON22_3092, MATalpha_HMGbox region): a draft-genome annotation with no paper on the locus (PubMed finds none), tier 3 at best.
+- Chromosome-level assemblies: GCA_022641835.1 ("Missouri", CP048715.1): our MATsc call is at 630,345-685,843 with DIC1 1.1 kb from the locus and SLA2 inside; GCA_050575385.1 (TBRC4498, chromosome) is uncalled in v0.7.0.
+- **Record plan:** a manual annotation of the CP048715.1 locus (the same procedure as for CBS 573) would make this a real record; needs the curator's go-ahead on the evidence bar (no publication, no RNA-seq).
+
+## Leads not followed
+- *Williopsis (Cyberlindnera) jadinii* XP_020071302: hypothetical HMG-box protein, nothing on MAT.
+- *Wickerhamomyces anomalus* (isolate KG16, JAHTLX01) is one of the seven Phaffomycetales genomes of Hession et al.; whether the supplement describes its MAT locus was not checked.
+- *Ambrosiozyma*, *Citeromyces*, *Hansenula*, *Kuraishia*, *Pachysolen*: no MAT-named protein records or papers found.
+
+## Next steps
+1. Fetch the S. schoenii annotation (GCA_054953955.1 or JBTFFQ) and the Hession et al. supplement (S1 Table, S3 Fig) for the coordinates.
+2. Draft records for S. schoenii (chromosome IV) and *B. californica* with the existing `curate-db propose` workflow, plus a family decision (separate or MATog-like).
+3. Curator: the family structure (one SLA2-MAT family for these genera, or per genus) and the evidence bar for the annotation-only *P. fabianii* lead.
+
+Sources (PubMed): 41849659, 41359681, 38492364.

results/2026-10-29_near_tie/GENUS_CURATION_LIST.md

@@ -0,0 +1,84 @@
+# Genera that need a curator ruling on MATsc against MTL (draft, 2026-10-29)
+
+From the 354-genome replay with the genome-level rule (`genus_table.tsv`, one row per genus: genomes, species, loci, family kept, mean best core-gene identity per family, gap, loci with MTL flank support). 156 `family_ambiguous` loci, 140 genomes, 27 genera. Identity gap = MATsc minus MTL, median over the genus's loci, in percentage points; the scores are best-hit identities of one gene (see README: about 6.5 points of noise per locus), so a gap under about 5 points is weak.
+
+Nothing here is a ruling. The choice per genus is one of: **leave** (the genome-level identity rule decides, with `family_ambiguous`), **scope** (the genus is added to the `taxonomic_scope` of a family, as for MATkp, MATog and MATpk; needs at least one reference record from the genus or evidence that the references transfer), or **hold** (leave unresolved and report both).
+
+## A. Consistent in the data, large gap (MATsc side): leave
+
+| Genus | genomes (species) | kept | gap | note |
+|---|---|---|---|---|
+| Williopsis | 9 (6) | MATsc 9 | +9.7 | |
+| Taiozyma | 3 (3) | MATsc 3 | +12.8 | |
+| Millerago | 2 (2) | MATsc 2 | +15.7 | |
+| Sporopachydermia | 3 (2) | MATsc 3 | +8.3 | |
+| Kregervanrija | 4 (3) | MATsc 4 | +7.3 | one near tie called different idiomorphs under the old rule |
+| Hansenula | 6 (6) | MATsc 6 | +3.4 | gap small |
+| Candida (sensu lato, 3 species) | 3 (3) | MATsc 3 | +5.5 | |
+| *Pichia* other than *P. kudriavzevii* (11 species) | 27 (11) | MATsc 28 | +4.3 | not routed to MATpk (species scope); MATpk call at 31-60% identity in the genus-scope trial, MATsc `medium` before |
+
+Proposed: **leave**. A ruling would need a reference record from the genus.
+
+## B. Lean MTL, no flank support: curator to decide
+
+| Genus | genomes (species) | kept | gap | loci with PAP1/OBP1/PIK1 flank |
+|---|---|---|---|---|
+| Saccharomycopsis | 20 (6) | MTL 22, MATsc 5 | -5.0 | 0 |
+| Ambrosiozyma | 13 (11) | MTL 15, MATsc 4 | -3.3 | 0 |
+| Citeromyces | 3 (3) | MTL 3 | -3.1 | 0 |
+| Saturnispora, Wenyingozyma | 1 each | MTL | -5.3, -11.2 | 0 |
+
+The family is split inside the genus for Saccharomycopsis and Ambrosiozyma (a few loci keep MATsc), which the per-locus rule produced and the genome-level rule only partly removes (a genome with a single cross-called locus is decided by it). No genome has two MTL flank genes, so MTL here is a choice by identity alone, the same weakness as the MTL-in-Serinales finding of #188. Proposed: **hold** and check synteny (SLA2/DIC1 against PAP1/OBP1/PIK1 neighbourhood) for one genome per genus before any scope.
+
+## C. Split or tied genera: needs the evidence
+
+| Genus | genomes (species) | kept | gap | flank loci |
+|---|---|---|---|---|
+| Petasospora | 11 (7) | MATsc 6, MTL 5 | 0.0 | 2 |
+| Wickerhamomyces | 9 (9) | MATsc 8, MTL 1 | +4.3 | 0 |
+| Barnettozyma | 8 (5) | MTL 6, MATsc 2 | -6.2 | 6 |
+| Gotozyma, Komagataea, Ruyongia, Xingzhongia | 2 each | split | -0.2 to +6.4 | 0-1 |
+| Saccharomycodes | 3 (2) | MTL 3 | -2.3 | 3 (one gene each) |
+
+Barnettozyma is the one genus where MTL flank genes show up in most loci (6 of 8, never two): worth a synteny look first. Saccharomycodes is close to Saccharomycetaceae by phylogeny, so MATsc would be the expected family although the identity leans MTL. Proposed: **hold**.
+
+## What would settle each
+
+1. Synteny of one genome per genus B and C (a figure with the flanks of both families).
+2. A profile score of each locus's genes against both reference sets (alpha-box and HMG-box profiles), instead of best-hit identity.
+3. A curated reference record from a genus (as for MATpk) before it is scoped.
+
+## Questions for the curator
+
+1. Accept **leave** for the genera of A?
+2. For B and C: **hold** now, with a synteny check as the next step, or scope any genus now?
+3. Is the order of work: Barnettozyma, Saccharomycopsis, Ambrosiozyma, Petasospora?
+
+## Synteny check (curator's order: Barnettozyma, Saccharomycopsis, Ambrosiozyma, Petasospora; all 156 loci were run)
+
+`flank_synteny.py` cuts each ambiguous locus +/- 40 kb and searches 35 flank proteins from the curated records with miniprot (SLA2, CHA1, APN2, DIC1 from the MATsc, MATyl and MATog records; PAP1, OBP1, PIK1 from the MTL records). A hit counts at 40% identity or more; `flank_synteny.tsv` has the nearest hit of each gene (identity@distance to the locus edge, 0 = inside the locus span). Cluster job 31430330.
+
+| Genus | loci | kept MATsc / MTL | SLA2 or CHA1 within 10 kb | DIC1 within 10 kb | PIK1, OBP1 or PAP1 within 20 kb |
+|---|---|---|---|---|---|
+| Barnettozyma | 8 | 2 / 6 | 7 | 7 | **0** |
+| Saccharomycopsis | 27 | 5 / 22 | 21 | 0 | **0** |
+| Ambrosiozyma | 19 | 4 / 15 | 13 | 13 | **0** |
+| Petasospora | 11 | 6 / 5 | 11 | 11 | **0** |
+| Wickerhamomyces | 9 | 8 / 1 | 9 | 9 | **0** |
+| all 27 genera | 156 | 91 / 65 | 129 | 71 | **0** |
+
+Findings:
+
+1. **No MTL flank gene lies near any of the 156 loci, including the 65 where MTL was kept.** The MTL architecture has PAP1, OBP1 and PIK1 beside the locus; none is found within 20 kb at 40% identity or more. (The detector's own flank rule found one flank gene beside 38 MTL calls, never two; a hit below 40% identity is not counted here, and a divergent PIK1 in a distant genus could be missed. This is the weak point of the check.)
+2. **SLA2 or CHA1 sits beside the locus in 129 of 156 loci**, and **DIC1 beside 71**, the same side-by-side as the *Komagataella* (MATkp) and *Ogataea* (MATog) cassettes: SLA2 on one side, DIC1 on the other. DIC1 is present in the Pichiales and Phaffomycetales genera (Barnettozyma 7 of 8, Ambrosiozyma 13 of 19, Petasospora 11 of 11, Wickerhamomyces 9 of 9) and absent in Saccharomycopsis and Saccharomycodes (Ascoideales, Saccharomycodales), which have SLA2 only.
+3. So the identity preference for MTL in Saccharomycopsis, Ambrosiozyma, Barnettozyma and Citeromyces is not backed by the neighbourhood: these loci sit in an SLA2 (+ DIC1) neighbourhood, a MATsc, MATog or MATkp-like arrangement. The MTL calls are probably the closest reference by sequence only.
+
+### Proposed (for the curator; not implemented)
+
+- **Genera with SLA2 + DIC1 beside the locus** (Barnettozyma, Ambrosiozyma, Petasospora, Wickerhamomyces, Hansenula, Williopsis, Ruyongia, ...): a MATog/MATkp-type architecture. Candidate action: a reference record from one genus (a *Barnettozyma* or *Petasospora* species with a manual locus check) before scoping; until then the MATsc/MATog calls stand.
+- **Rule change to consider:** an MTL call outside Serinales with no flank support should not win a cross-call against a family whose own flank genes are present (MATsc: SLA2). Applied to this replay it would keep MATsc in all 65 loci where MTL won. It makes the flank rule decide the family, the asymmetric evidence that the evaluation warned about, but here the asymmetry is complete (0 of 156). This needs the curator's decision.
+
+## Flank rule implemented (curator, 2026-10-30)
+An out-of-scope MTL call with fewer than two of PAP1/OBP1/PIK1 within the window cannot win a cross-call against a family whose own flank gene (SLA2, DIC1) is in the call; it is set aside as `cross_call` (`flank_rule: true`), is left out of the genome mean, and the kept call's `family_ambiguous.reason` is `cross_call_flank_rule` (`flank_rule_set_aside` names the family). `src/MATPredict/detect/family_status.py`, 6 tests in `tests/detect/test_family_status.py`.
+
+Estimated effect on the 156 cross-called pairs of the v0.7.0 replay (`cross_call_pairs.tsv`, before the per-genome pooling): of the 51 pairs kept as MTL, **40 flip to MATsc** (Pichiaceae 14, Saccharomycopsidaceae 12, Wickerhamomycetaceae 7, Phaffomycetaceae 6, Pachysolenaceae 1); 11 stay MTL because the MATsc call has no SLA2/DIC1 (Saccharomycopsidaceae 8, Pichiaceae 3). This is not ground truth: MATsc is not the right family for these genera either (S. schoenii has its own `MATss` family now); the rule only removes the MTL call that has no flank gene of its own where a competitor has one. A v0.8 replay (`--preset v0.8`) is needed for exact numbers.

results/2026-10-29_near_tie/README.md

@@ -0,0 +1,42 @@
+# The near-tie margin: replaced by a genome-level decision (curator decision of 2026-10-29)
+
+`--cross-family-policy on` (PR #214) kept, for two families calling one locus where neither is scoped to the genome, the call with the higher best core-gene identity, except that within `NEAR_TIE_POINTS` = 3 points the family without a `confirm_outside_scope` rule (MATsc) was kept and flagged `family_unconfirmed`. The 3 was a draft value, never validated. The curator asked for it to be evaluated.
+
+## Evidence
+
+`cross_call_pairs.tsv`: the 156 cross-called locus pairs of the 354-genome replay (v5), 140 genomes, all MATsc against MTL; identity gap = kept minus other best core-gene identity. An independent evaluation (a second model, read-only, 23 tool calls, from this table and the full reports) found:
+
+- **No natural break at 3 points.** Per species the gaps run continuously from -3.3 to +1.7. The cutoff splits congeners: *Ambrosiozyma oregonensis* (-2.6) is kept as MATsc, *A. angophorae*, *A. neoplatypodis* and *A. ambrosiae* (-3.2 to -3.3) as MTL.
+- **The score is too coarse for a 3-point rule.** The kept call's best alignment has a median 59 aa at 38% coverage, so one residue is about 1.7 points and the standard error of one identity is about 6.5 points. The two identities come from different reference proteins, often different genes (41 of 105 kept MATsc calls take their best identity from MATA2 or MATALPHA2, which the registry marks `idiomorph_informative: false`).
+- **The MATsc tie-break is biased.** `sla2` is searched only by MATsc and PAP1/OBP1/PIK1 only by MTL, so gene counts and flank evidence are not comparable; no MTL call reached two flank genes. In 4 near ties the two families call different idiomorphs, so the tie-break silently sets MATa against alpha.
+- **Per-locus decisions give genomes with two families** (4 genomes in v5), which cannot be right for one genome.
+- **The flag measured "MTL was involved", not uncertainty**: every MTL call outside Serinales is flagged by the flank rule anyway, and every margin option flags 65-108 calls.
+
+The evaluation's counts for 156 pairs: margin 0: 5 near ties; 1: 19; 2: 34; 3: 42; 5: 72; genome-level mean with no margin: 0 mixed genomes and 1 lower-identity call kept, against 4 and 12 at 3 points.
+
+## Decision (option A)
+
+1. No margin and no preference for either family.
+2. For every group of overlapping cross-family calls with no family scoped to the genome, the family is decided once for the genome: the higher mean, over those groups, of each family's best core-gene identity (equal means: the smaller family name).
+3. Every such kept call carries `family_ambiguous` (identities of both families, their gap, the genome means and locus counts): GFF3 attribute `family_ambiguous=<family:identity|...>` and a YAML block. `family_unconfirmed` is now only the PAP1/OBP1/PIK1 flank rule for MTL outside Serinales.
+4. The set-aside call's gene evidence (gene, role, identity, coverage, status) is kept in its `cross_call` note, so the two families can be compared gene for gene after a run.
+
+## Replay with the new rule (job 31429403; `compare_v5_v8.py`, `compare_v5_v8.txt`)
+
+354 genomes, `--preset v0.8`, the database of `main` (no MATpk):
+
+| | v5 (3-point margin) | v8 (genome-level) |
+|---|---|---|
+| genomes whose cross-called loci keep two families | 4 | 0 |
+| `cross_call_near_tie` flags | 42 | 0 |
+| `family_ambiguous` loci | - | 156 in 140 genomes (MATsc kept 91, MTL 65) |
+| kept call has the lower identity | 12 | 1 (gap -1.5) |
+| `mtl_outside_scope` unconfirmed / confirmed | 75 / 5 | 89 / 5 |
+| genomes whose calls change | | 14 |
+
+The 14 genomes change family (and so idiomorph names, MATa against A and so on); for most the per-locus rule had kept MATsc where the genus leans to MTL. The extra 14 `mtl_outside_scope` flags are those MTL calls, which have no PAP1/OBP1/PIK1 support.
+
+## Open
+
+- There is still no ground truth: the family is a choice between two reference systems by best-hit identity. The evaluation suggests per-genus curator rulings (Ambrosiozyma, Saccharomycopsis, Pichia, the Wickerhamomycetaceae, as for MATkp, MATog and MATpk), HMG/alpha-box profile scoring against both reference sets, and synteny of a sample per genus.
+- With the genome-level rule a genome with only one cross-called locus is decided by that locus alone (as before).