Skip to content

CHANGELOG: v0.9.0 header, fixes moved out of Unreleased, missing entries #346

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
hyphaltip merged 1 commit into from
Nov 12, 2026
Merged

CHANGELOG: v0.9.0 header, fixes moved out of Unreleased, missing entries #346

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 19 additions & 5 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,9 @@ release completed.

## [Unreleased]

### Fixed
- An EMPTY taxonomy answer is now recorded (#324): when the lineage and phylum lookups return nothing (no exception, for example a cache miss on a compute node without network) the report's `routing_error` says so and the not-searched reason reads "taxonomy lookup failed (...)", where it said only "no taxid, or no phylum could be resolved". Seven *C. fimbriata* assemblies (taxid 5158, a valid taxon) were left unsearched this way in the v0.9.0 run. Routing is unchanged. The panel script's taxonomy warm-up now counts an empty answer as a failure, warns, and stops the job when `TAXONOMY_WARMUP_STRICT=1` (default `0`: warn only). Tests: `tests/detect/test_empty_taxonomy_lookup.py`, `tests/test_clade_panel_warmup.py`.
- `genes_found` listed optional genes of different slots (GAP1 and BGL1 in *Cochliobolus*) in hash-seed order, so the same genome could report them in either order from run to run (found 2026-11-12 in the 60-genome determinism check of the v0.9.0 run: 1 genome). They now follow the roster order. Output order only; no call, tier, label or coordinate changes. Test: `tests/detect/test_genes_found_order.py`.

## [0.9.0] — TBD — `v0.9.0`

> DRAFT release prep (do not tag until the definitive run is integrated). The entries below are those of the merged pull requests of the v0.9.0 cycle; the definitive run is `--preset v0.9.1 --withheld-detail-policy on` with the `tblastn` outlier policy (`MATPREDICT_TBLASTN_SEG=yes`, `MATPREDICT_TBLASTN_SEG_GENES=SLA2,APN2,MAT1-1-1`) and its counts are filled in once the tables are built. Rules that exist but are in no preset: `--weak-fragment-cap-policy`, `--weak-coverage-noise-policy`, `--strong-single-gene-policy`, `--confidence-cap-policy`.
> Not yet tagged: the date is set when the tag is cut. The entries below are those of the merged pull requests of the v0.9.0 cycle. The definitive run is described in the next section: it is the frozen commit `7d73be5` with `--preset v0.9.1 --withheld-detail-policy on`, and the 372 genomes that moved under the `tblastn` outlier policy come from reruns with it off. Rules that exist but are in no preset: `--weak-fragment-cap-policy`, `--weak-coverage-noise-policy`, `--strong-single-gene-policy`, `--confidence-cap-policy`.

### The definitive run (the 372 genomes that moved under the `tblastn` outlier policy are taken from reruns with it OFF; curator's choice, 2026-11-12)
Frozen commit `7d73be5`, `--preset v0.9.1 --withheld-detail-policy on`, no weak-fragment cap. 23,579 reports (19,343 Ascomycota, 3,263 Basidiomycota, 973 Mucoromycotina), every one stamped with the same commit, database hash and preset. Against v0.8.1:
Expand All @@ -29,6 +25,24 @@ Frozen commit `7d73be5`, `--preset v0.9.1 --withheld-detail-policy on`, no weak-
- Opt-in `--weak-fragment-cap-policy off|on` (#275; default `off`, in no preset): an `Ascomycota:MAT` call at `medium` or `high` whose strongest core gene (`MAT1-*`) is not a both-tools-agree model and scores under 50 bits is capped at tier `low`; the idiomorph, the locus and the class are never changed, and the report's `weak_fragment_cap` records the gene, its status and bitscore. On the v0.8.1 run it matches 108 of 12,051 loci and would change the tier of 68 (23 medium, 45 high). Details and the other triggers measured: `docs/notes/2026-11-06_weak-fragment-cap.md`.
- `docs/detect-options.md`: one page listing every `detect` option, policy, preset and environment variable with its default and what it does (tables generated from `--help`), and what is not in a release yet; `tests/test_docs_detect_options.py` fails when a policy, preset or environment variable is added without being listed.
- Opt-in `tblastn` filtering policy for the slowest genomes (#276): `MATPREDICT_TBLASTN_SEG=yes` with `MATPREDICT_TBLASTN_SEG_GENES=SLA2,APN2,MAT1-1-1` (or `MATPREDICT_TBLASTN_SEG_MIN_AA`) runs the genome-wide localization as two `tblastn` calls: `-seg no` for every other query and `-seg yes -soft_masking true` (masked regions used for seeds only) for the listed gene names. On the two genomes that hit the 3,600 s limit in the v0.8.1 run it takes 3,063 s and 1,685 s to 468 s and 288 s with identical loci; on 300 Ascomycota genomes 296 are identical (no change of family, idiomorph, confidence or gene set) and on 120 Basidiomycota genomes all 120. It gives no speed-up on typical genomes. The default is unchanged (`-seg no`). Measurements and what does not work: `docs/notes/2026-11-06_tblastn-seg-policy.md`.
- Opt-in `--weak-coverage-noise-policy off|on` (#149; default `off`, in no preset): an unpolished gene whose every hit covers under 30% of its reference is noise, whatever its bitscore, so it no longer caps the tier at `medium`. It acts only on a locus whose idiomorph is not `undetermined` and whose best core-gene coverage is at least 50% (curator, 2026-11-12). Tier only (`medium` to `high`). A first version without the two limits moved 109 loci from `medium` to `high` in 970 genomes of the v0.8.1 run; the limited version has not been run on a panel, so its count is not measured. It is not in the frozen v0.9.0 run.
- Opt-in `--withheld-detail-policy off|on` (#313; default `off`, in no preset; report only): with `on`, each `suppressed_loci` entry also lists the genes of the withheld locus (name, role, polish status, identity, coverage, bitscore). No call, tier or coordinate changes. The definitive run uses it.
- Label tables: a `nearest_reference` column (#200, #311; curator's ruling 2026-11-07). For each core gene it gives `GENE=SYSTEM`, the system code of the class of the matched record's organism, genes joined by `|`. A matched record whose class has no code is written as its `record_id`. `loci.tsv` gets a `core_reference` column with `gene:record_id:identity:bitscore` per gene. Tables only; no report, call or tier changes.
- Raw-reads gate causes (#195, #196; PR #325). `datasets/reads/sra_survey/pilot/causes.tsv` holds one cause per species: `homothallic` and `silent_cassettes` give `NOT_TYPABLE`, `zygosity` joins the zygosity gate, and a species with no cause is `UNVALIDATED:cause_unchecked`. The 50%-`both` shortcut no longer gives `PROVISIONAL_HOMOTHALLIC`. `cause_evidence.tsv` records the evidence for 42 species. The reads pages show the cause wording of the ruling of 2026-11-09.
- Raw-reads gate, thin evidence (#327; curator's ruling 2026-11-09): a species with one strain or one to three runs is typed from reads and capped at `UNVALIDATED`, with the reason shown, instead of being held. A species is held only when an idiomorph has no locus or no run.
- Raw-reads clone groups (#320) use the error-tolerant solid-containment distance of `sketch_query.py`: 202 species, clone ids at 0.9999, 0.9995, 0.999 and 0.998. The site uses these tables in place of the older plain-containment tables.
- *A. fumigatus* population report from the sketch store (#326): 4,849 isolates (4,736 kept); MAT1-1 in 50.4% of clones (95% CI 48.2-52.5%, 2,077 clones). Also an ENA FTP fallback for runs where ENA answered HTTPS with 403 (#326).
- Sketch store (`datasets/reads/sketch_store/`; #315, #319, #329, #330, #333): one `sourmash` sketch per isolate (k=21 and k=31, scaled 100) answers MAT, target-screen, population-structure and QC questions without downloading reads again; Starship and bafA prevalence, co-occurrence and gain/loss on the tree; a baf family screen. Research tooling; `detect` does not use it.
- Landing page: Fola and *A. fumigatus* read-typing charts, tables and notes (#301). They describe the v0.7.0 reads code. #172 stays open for the Verticillium, Cryptococcus and other Fusarium rows.
- FungiDB annotation check (#290, #336): `scripts/annotation_by_tier.py` and `docs/notes/2026-11-12_290-annotation-by-tier.md`. No gene-level verdict changed on the 1,234 genes present in both the v0.8.1 and v0.9.0 runs; 15 genes appear only in v0.9.0 and none is lost. Annotation is a cross-check, not the truth. #290 stays open for the remaining steps.

### Changed
- Clean-run tables: the Basidiomycota defective flag is a model check (#168, #304; curator's decision 2026-11-07; report only). `defective_genes` lists only a core gene at identity of at least 60% and coverage of at least 80%; every other flagged gene moves to the new column `low_confidence_model_genes`. In the v0.8.1 run, 1,040 loci were flagged; 153 keep a listed gene and 887 moved. Outside Basidiomycota nothing changes. Tiers, calls and reports are untouched.
- Speed: the curated YAML (`order.yml` and the `metadata.yaml` records) is parsed with libyaml's C loader, with the pure-Python loader as fallback (#287, #292). Detection is identical to the frozen v0.8.1 reports on 823 genomes.

### Fixed
- An EMPTY taxonomy answer is now recorded (#324): when the lineage and phylum lookups return nothing (no exception, for example a cache miss on a compute node without network) the report's `routing_error` says so and the not-searched reason reads "taxonomy lookup failed (...)", where it said only "no taxid, or no phylum could be resolved". Seven *C. fimbriata* assemblies (taxid 5158, a valid taxon) were left unsearched this way in the v0.9.0 run. Routing is unchanged. The panel script's taxonomy warm-up now counts an empty answer as a failure, warns, and stops the job when `TAXONOMY_WARMUP_STRICT=1` (default `0`: warn only). Tests: `tests/detect/test_empty_taxonomy_lookup.py`, `tests/test_clade_panel_warmup.py`.
- `genes_found` listed optional genes of different slots (GAP1 and BGL1 in *Cochliobolus*) in hash-seed order, so the same genome could report them in either order from run to run (found 2026-11-12 in the 60-genome determinism check of the v0.9.0 run: 1 genome). They now follow the roster order. Output order only; no call, tier, label or coordinate changes. Test: `tests/detect/test_genes_found_order.py`.

## [0.8.1] — 2026-11-06 — `v0.8.1`

Expand Down
Loading