Repository navigation
feat(model): QLKNN7_11 third virtual closure — first increment - #73
Merged
Merged
Conversation
DeepMind's fusion_surrogates ships QLKNN7_11 under Apache-2.0; the conversion script (ephemeral jax/flax tooling) writes a torch checkpoint loaded weights_only=True on the surrogate.py pattern. Conversion is deterministic (re-run reproduces sha256 03f50b19...) and reproduces the 25 upstream reference pairs to 8.5e-14. The bundle is 295 KiB — 0.4% of the worker's 64 MiB artifact cap. qlknn.py keeps torch lazy (app import never loads it), predicts raw gyro-Bohm fluxes, and reduces them to the (K, alpha) tau_E interface through a documented, iteration-capped power-balance fixed point. The reduction's assumptions are pinned in the module docstring; the closure is compared with MAST data by the re-fly gate, never treated as predictive. First increment of #48: no transport solver, TGLF excluded (licensing). Co-authored-by: Tapan Ganatma Nakkina <[email protected]>
All additive, no protocol change: run_qlknn body in compute/local.py TASKS, pydantic input schema + worker.TASKS entry (both execute one implementation), ArtifactRef.role gains "qlknn.pt", shipped_models() and _weights_for_job learn the module, _TASK_WEIGHTS maps the task so SshProvider hash-uploads the weight once, and capabilities gain qlknn_available on both providers. Co-authored-by: Tapan Ganatma Nakkina <[email protected]>
virtual.CLOSURES gains "qlknn"; baseline() dispatches through the run_qlknn task (mirroring _learned_correction's ride through run_task), and integrate() broadcasts per-slice alpha across scenarios while keeping scalar-alpha behavior for the existing closures. available_closures() filters by weight presence, so /virtual/gate advertises the closure only while models/qlknn.pt exists and the route 422s it otherwise; the agent's run_whatif enum picks the closure up from virtual.CLOSURES with no edits. Evidence lands only through scripts/validate_virtual.py: the regenerated models/virtual_metrics.json adds the QLKNN re-fly row (flat-top median |ln(W_refly/W)| = 0.590 over 117 held-out shots vs 0.174 learned) and docs/RESULTS.md's table is regenerated from that script's output — the widest of the four closures, published rather than hidden, learned stays default. Co-authored-by: Tapan Ganatma Nakkina <[email protected]>
…eline New tests/test_qlknn.py pins pristine import (no torch at import time), upstream reference-pair fidelity, checkpoint stats, the per-slice (K, alpha) solve on shot 30166, the tau-decreases-with-loss-power sign, and the agent-enum pickup. The rails get: provider-passthrough equivalence, worker roundtrip with the shipped weight plus the upload-over-shipped drift contract, loopback upload-then-run parity, the weightless-worker fallback that still answers locally, and the gate/422/panel behavior at the route level — including the weights-missing case hiding the closure while the default still answers. Enumeration pins (task set, model set, hash-read count) grow with the registry. .obvious/obvious.md's test baseline is regenerated from the final fresh run (543 passed, 1 skipped). Co-authored-by: Tapan Ganatma Nakkina <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #48
Human author: Tapan Ganatma Nakkina ([email protected])
Why
Issue #48 asks for a third virtual-shot model alongside the learned correction and the scaling laws. QLKNN is a local turbulent-flux model — it outputs gyro-Bohm-normalized fluxes, not a confinement time — so the roadmap priced the full version (a 1D transport loop) as scale work. This PR ships the honest first increment: the weights vendored, the evaluation wired through the compute rails end to end, and a new
qlknnclosure whose flux-to-confinement reduction is a documented, iteration-capped power-balance fixed point entering through the same evidence gates as every other closure. Scope boundary: first increment; the transport loop stays out. TGLF is excluded outright (redistribution rights unresolved).What
scripts/convert_qlknn_weights.py(ephemeral jax/flax tooling) converts DeepMind'sfusion_surrogatesQLKNN7_11 bundle (Apache-2.0, pinned commit + sha256) into a torch checkpoint loadedweights_only=Trueon thesurrogate.pypattern. Conversion is deterministic (re-run reproduces sha25603f50b19…) and reproduces all 25 upstream reference pairs to max abs err 8.5e-14. Bundle: 295 KiB — 0.4% of the worker's 64 MiB artifact cap. License and attribution committed beside the weights.fusionlab/qlknn.py:MODEL_FILE/available()/load()/predict(); torch stays lazy (app import never loads it).predict()returns raw fluxes; a documented power-balance fixed point (capped iterations, fixed profile shapes, balance surface x=0.7) reduces them to the(K, alpha)interfacebaseline()consumes. Assumptions (fixed shapes, Ti=Te, impurity composition, q95 scaling, all loss power crossing the surface) ride the docstring and CAVEATS.run_qlknnbody incompute/local.pyTASKS; pydantic schema + entry inworker.pyTASKS;ArtifactRef.rolegains"qlknn.pt";shipped_models()/_weights_for_job/_TASK_WEIGHTSlearn the module so the provider hash-uploads the weight once; capabilities gainqlknn_availableon both providers.virtual.CLOSURESgains"qlknn";baseline()dispatches through therun_qlknntask (mirroring_learned_correction);integrate()broadcasts per-slice alpha (existing closures keep scalar-alpha behavior).available_closures()filters by weight presence —/virtual/gateadvertises the closure only while the weights exist, and the route 422s it otherwise. The agent'srun_whatifenum picks the closure up fromvirtual.CLOSURESwith zero edits; the UI renders from the gate payload (no hardcoded lists).scripts/validate_virtual.py→models/virtual_metrics.json. Held-out re-fly (117 shots): QLKNN flat-top median |ln(W_refly/W)| 0.590 vs 0.174 learned — the widest of the four closures, published rather than hidden; the learned correction stays the default.docs/RESULTS.md's table is regenerated from the script's output (house rule: numbers come from scripts).How to Review
fusionlab/qlknn.py— the reduction's assumptions and cap are the review surface; the net evaluation is verified against upstream to 8.5e-14.fusionlab/virtual.py— the per-slice-alpha broadcast inintegrate()is the one behavioral change to existing paths (kept no-op for scalar closures; existing re-fly numbers unchanged within print precision).fusionlab/worker.py/compute/— purely additive registration; protocol version untouched.Test Evidence
Gate panel renders the QLKNN7_11 closure from the /virtual/gate payload (caveat line in the what-if drawer)

data/shots/, which trips the pre-existing cache-iterating unit-sanity test; not reproducible on a clean cache, and CI is immune).uv run ruff checkclean;uv run mypy fusionlabclean (34 source files).obvious autobuild uploadagainst the tested head SHA.🔗 Obvious Project · 🧵 Obvious Thread · 🔀 Obvious PR Artifact