Skip to content

Mint pre-mapped VRS for every variant, independent of dcd-mapping #795

Description

@bencap

Summary

Give every Variant a pre-mapped VRS allele object and its ga4gh digest, minted by MaveDB itself at upload validation time, for both non-accession (locally stored target sequence) and accession-based (CDOT-resolved) targets. This value does not depend on the dcd-mapping service having run or succeeded.

Problem

A variant's only VRS representation today comes from MappedVariant, written by the dcd-mapping service after a successful score-set-level mapping call. Score sets that dcd-mapping cannot process at all get zero VRS for any of their variants, with no path to recovery short of making dcd-mapping itself succeed. Non-human targets are the largest concrete case: we unconditionally reject any score set containing a non-human, non-accession target before attempting any mapping, so none of those variants can ever get a MappedVariant row under the current architecture.

Proposed behavior

  • Mint a pre-mapped VRS allele inline during upload validation, for every variant, independent of dcd-mapping — reusing the parsed/resolved objects validation already builds rather than re-deriving them in a separate pass.
  • Non-accession targets: build the allele from the variant's hgvs_nt/hgvs_pro and the target's locally stored sequence (TargetSequence.sequence). No external service calls.
  • Accession-based targets: resolve the named accession's sequence via CDOT, then build the allele the same way. Memoize the CDOT fetch per (target, accession) rather than per variant.
  • Store the result on Variant, not on a mapping table:
    • pre_mapped_vrs (JSONB) — the pre-mapped VRS allele object.
    • pre_mapped_vrs_digest (indexed string) — the ga4gh digest.
    • pre_mapped_vrs_version (string) — the VRS spec version used to mint.
  • Record each mint attempt's outcome (present/failed, with reason) through the existing AnnotationStatusManager / AnnotationStatus / AnnotationFailureCategory machinery already used for AnnotationType.VRS_MAPPING on this branch. Do not add new status or error columns to Variant.
  • A per-variant minting failure is isolated and recorded; it must never abort creation of the rest of the score set's variants.
  • Leave MappedVariant and the dcd-mapping pipeline unchanged. MappedVariant.pre_mapped/post_mapped keep being written exactly as they are today.
  • Serving reads pre-mapped VRS from Variant unconditionally and post-mapped VRS from MappedVariant where a row exists. The two are never merged or used as fallbacks for each other.
  • Provide a backfill for all pre-existing variants.

Acceptance criteria

  • Variant has pre_mapped_vrs (JSONB), pre_mapped_vrs_digest (indexed), and pre_mapped_vrs_version columns.
  • Uploading a score set mints a pre-mapped VRS for every variant during validation, for both accession-based and non-accession targets, with no dependency on dcd-mapping having run.
  • For a non-accession target, minting succeeds using the DB-stored target sequence with no external service call, and pre_mapped_vrs_digest is a valid ga4gh digest.
  • For an accession-based target, minting succeeds using a CDOT-resolved sequence for the named accession, with the CDOT fetch made at most once per (target, accession) per upload.
  • For every variant that already has a dcd-mapping-produced MappedVariant.pre_mapped, the independently minted pre_mapped_vrs_digest is byte-identical to the digest already stored in MappedVariant.pre_mapped.
  • On a minting or normalization error for a given variant, the outcome is recorded via AnnotationStatusManager with a failure category; the variant's pre_mapped_vrs* columns are left null; the rest of the score set's variants are still created and minted.
  • A backfill mints pre-mapped VRS for all pre-existing variants; variants that cannot be minted get a recorded failure rather than being silently skipped.
  • The variant view model exposes pre_mapped_vrs, pre_mapped_vrs_digest, and pre_mapped_vrs_version, alongside the existing MappedVariant-sourced post-mapped fields where present.
  • The dcd-mapping pipeline, MappedVariant, and AnnotationType.VRS_MAPPING are unchanged by this work.

Implementation notes

  • Minting is constructed inline during upload validation (validate_transgenic_variant and validate_genomic_variant in lib/validation/dataframe/variant.py), reusing the parsed/resolved objects those functions already build, rather than as a separate pass after variant creation. Both files are unchanged between release-2026.3.0 and feature/bencap/allele-centric-mapping-and-storage today, so this is safe for that branch's next merge from release-2026.3.0. The minting logic itself stays a pure function, callable with an already-parsed variant — only its call site lives inside validation.
  • A minting failure must never be escalated into ValidationError (which rejects the whole upload) — it is carried forward as a per-row recorded failure, the same way a syntax failure is not.
  • Model the backfill on alembic/manual_migrations/migrate_target_gene_mapping_qc.py (the migrate/verify/rollback CLI shape): batched, idempotent, safe to re-run.
  • Before trusting the accession or non-accession minting path for variants that were never mappable before, run each against the full existing corpus of MappedVariant.pre_mapped rows and confirm digest equality. Treat any mismatch as a defect in the new minting code, not in the existing mapped data.
  • Scope is release-2026.3.0 only. Integrating this onto feature/bencap/allele-centric-mapping-and-storage — swapping the status-recording call site to Disposition/AnnotationEvent, extending the variant-subject CHECK constraint to admit a pre-mapped-VRS annotation type, and deciding whether to add SeqRepo write-on-mint — is out of scope for this issue and its sub-issues.
  • Work is split into the following sub-issues, tracked under this one: Variant migration; non-accession minting; accession-based (CDOT) minting; attaching minted results to created Variant rows; backfill; API/CSV serving of the new fields.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    app: backendTask implementation touches the backendapp: databaseTask implementation requires database changes

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions