Repository navigation
feat(export): IMAS-shaped replay export adapter (#49) - #71
Merged
Merged
Conversation
Export-only adapter over ShotBundle v1: a static in-repo mapping from the
bundle's arrays to IMAS data-dictionary paths, served by a new sibling of
the USD route, GET /replay/{shot_id}/imas. The payload is the replay
bundle's own values path-for-path - units converted once at this edge
(MA->A, 1e20 m^-3 -> m^-3, keV->eV, MW->W, MJ->J) - never a recomputation.
What the dictionary has no honest home for rides a sibling
fusionlab_annotations.json: the catalog's useful/abort flags, the
per-series provenance labels (measured/model/learned/computed), the
machine-specific eq_inputs block, and the series IMAS cannot carry
(n_e20 as line density, the Thomson radii, R_m, P_rad_MW, dWdt_MW).
The IMAS payload points at the sidecar through ids_properties.comment
and carries no flag or label as data.
Groups the bundle lacks are omitted, never emitted as empty IDS stubs -
the same graceful degradation /replay applies. Dictionary conventions
were verified against the IMAS Data Dictionary (DD 4.2.x) before the
conversion table locked; the vacuum_toroidal_field r0 is omitted because
the bundle does not record the B_T reference radius. No new dependency.
Fixes #49
Co-authored-by: Tapan Ganatma Nakkina <[email protected]>
…for the IMAS adapter
- tests/test_imas_export.py: the mapping table partitions the SPEC exactly
once (an IMAS home or a sidecar entry - a future field fails here until
its home is decided); the conversion table is pinned against the
api_replay._UNIT oracle the house already keeps; a committed golden
snapshot of the shot 30166 export pins shape and values (a snapshot
diff is the failure mode; regenerate with the one-liner in the module
docstring); minimal and LCFS-only synthetic bundles prove degraded
shots export core traces only - no empty IDS stubs; the sidecar test
asserts flags, provenance, meta and unmapped series stay out of the
IMAS payload, which mentions them only in the comment.
- tests/test_api.py: the /replay/{id}/imas attachment equals the replay
bundle at IMAS paths (factor-1 traces exactly, Ip within rounding),
carries the replay's own archive flags and provenance map, is GET-only
with clean 404s, and never emits a non-finite float.
Full suite: 489 passed, 1 skipped (baseline 476 + 1); ruff and mypy clean.
Co-authored-by: Tapan Ganatma Nakkina <[email protected]>
DATA_CONTRACT.md's "future adapter" paragraph framed IMAS as producing
bundles (the import side); it now states the export direction - the
/replay/{id}/imas adapter emits the bundle's own values at IMAS paths
with units converted once at that edge - and the rule that IMAS is never
an internal format or an import spine. Route maps in .obvious pick up
the new endpoint and the imas_export module.
Co-authored-by: Tapan Ganatma Nakkina <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #49.
Human author: Tapan Ganatma Nakkina ([email protected])
Why
Issue #49 asks FusionLab's replay bundle to speak to IMAS dictionaries. Today the export surfaces are derived CSV/JSON, OpenUSD, and the replay bundle itself; none names quantities the way the fusion community's data dictionary does. The backlog program's locked decision: IMAS is an export direction only — an adapter at the edge, never an internal data spine and never an import path. This PR ships that adapter.
What
fusionlab/imas_export.py(new, on theusd_export.pypattern): a pure function over the same bundle dictusd_export.export_shottakes, mapping the ShotBundle v1 arrays to IMAS data-dictionary paths via a static, in-repo table. Units convert once at this edge — house units (m, T, MA, 1e20 m^-3, keV, MW, MJ) → the dictionary's (A, W, m^-3, eV) — withapi_replay._UNITas the conversion table's test oracle. Nothing is recomputed: every emitted value is the bundle's own.GET /replay/{shot_id}/imas(sibling of the USD route): a strict-JSON attachment,imas_{id}.json, whose values equal the/replaybundle path-for-path, plus the siblingfusionlab_annotations.jsonin the same body: the catalog'suseful/abortflags, the per-series provenance labels (measured/model/learned/computed), the machine-specificeq_inputsblock, and the series the dictionary has no home for (n_e20line density, Thomson radii,R_m,P_rad_MW,dWdt_MW, midplane-radiusq_mid). The IMAS payload points at the sidecar viaids_properties.commentand carries no flag or label as data./replayapplies.equilibrium.time_slice.global_quantities.ipin A;q_95/beta_tor_normdimensionless;boundary.elongation/minor_radiusin m;profiles_1d.fin T.m with psi in Wb;profiles_2d.psiin Wb on a rectangular grid (grid_type.index = 1, dim1=R, dim2=Z);pf_activecoil rectangle geometry fields r/z/width/height (geometry_type.index = 2); wall limiterunit[].outliner/z in m;ids_properties.commentexists as free text. Two honest omissions:vacuum_toroidal_field.r0is left out because the bundle does not record the reference radius B_T was measured at (noted in the IDS comment); the walldescription.typeenum is left unfilled because no option honestly describes a single open limiter..obviousroute maps updated.omasnot taken, not even as an extra — a dev cross-check can be added later if wanted); basepyproject.tomluntouched.How to Review
fusionlab/imas_export.pyare the whole design — everything else is mechanics.test_mapping_table_partitions_the_specguarantees every future SPEC field gets classified (IMAS home or sidecar entry), never silently unexported.tests/fixtures/imas/golden_30166.jsonis a committed full-value snapshot (~6 MB, one value per line). When the export changes deliberately, regenerate with the one-liner in the test module's docstring and review the diff — the snapshot diff is the failure mode.omasas a dependency, any change to/replayor the bundle spec.Test Evidence
main@a7a6e87): 539 passed, 1 skipped (main alone: 526; +13 new — 11 intests/test_imas_export.py, 2 intests/test_api.py),uv run ruff checkclean,uv run mypy fusionlabclean./replaybundle at IMAS paths; degraded bundles (minimal and LCFS-only) exporting core traces only; sidecar contents; the comment-pointer rule; GET-only + clean 404s; no non-finite float in the attachment.🔗 Obvious Project · 🧵 Obvious Thread · 🔀 Obvious PR Artifact