fix(seam): correct the eval-seam contract for vpc 3.3.0's per-model source; floors vpc>=3.3.0, views-evaluation>=2.0.0 (#287) - #297
Merged
Conversation
…ource; floors vpc>=3.3.0, views-evaluation>=2.0.0 (#287) views-pipeline-core 3.3.0 (PyPI 2026-09-19) fixes C-215's root: its reporting stage now constructs PerModelMetricFrameSource — one of OUR MetricFrameFileSources per model, each rooted at THAT model's own data/generated (vpc #485, PR #522, their C-328) — instead of a single source rooted at the subject, which is why every baseline/constituent row vanished from every ensemble report. Our locked on-disk layout is untouched; only the root per model changed. Our half, in the same window as the register's sequencing note required: the seam test that had faithfully locked the DEFECTIVE subject-rooted call (test_stage_constructs_source_with_our_init_contract) is replaced by two real guards: - the stage constructs PerModelMetricFrameSource with EXACTLY {primary_model, primary_root=<subject>.data_generated, run_type, target} — no root_of= override, so the C-215 shape cannot be re-expressed through the new class (verified against five mutant stages, all fail); - a real on-disk per-model round-trip: frames saved under TWO roots resolve through the public metric_frame() with values read back exactly, and the subject-rooted shape provably returns None for the other model — never a shared root, the fixture blind spot that hid C-215 for months. Pins: views-pipeline-core>=3.3.0 (a <3.3.0 stage would silently blank rows again); views-evaluation>=2.0.0 — the floor held on 09-18 lands now because vpc 3.3.0 itself requires >=2.0.0, so the shared env co-resolves. Lock 129→126 packages (stepshift, xarray, views-transformation-library left; vpc 3.3.0 dropped the last). Suite green on the RELEASED 3.3.0 + 2.0.0: 443. Register: C-215 → Resolved (with what it does NOT do: C-216/C-217 configs and pins, C-223's one-time baseline evaluation, C-219/C-222 announce and vintage layers, and the end-to-end proof still owed); C-192 correction note closed; Cluster I updated honestly — the C-220/C-225 "land WITH the fix" constraint was NOT honoured (upstream shipped alone), so C-220 is now ARMED and its scrub is owed before C-223's evaluation run; Cluster H: eval floor landed, and the next trigger (views-frames 2.0.0, gated by TWO ceilings — ours and views-evaluation's [frames] extra, eval#91) recorded. No vpc ADR records the per-model rule yet; our 08-21 draft should be filed. CICs (evaluation_source, evaluation_report_template) updated for the injected source. 91 concerns (69 resolved, 22 open). Closes #287 item 1. Items 2-4 (announce layer, arm-on-fix traps, vintage semantics) remain open. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01N7YBtiS2kSVj27spMKDYfh
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
pipeline-core 3.3.0 (PyPI today) fixes the root of the baselines-vanishing cluster (our C-215, their #485/C-328): the reporting stage now constructs
PerModelMetricFrameSource— one of ourMetricFrameFileSources per model, each rooted at that model's owndata/generated— instead of a single source rooted at the subject. Our locked on-disk layout is untouched.This PR is our half, landed in the same window as the register's sequencing note required.
The seam test that locked the defect is replaced by two real guards. The pasted tests from #287 were a good start but weaker than their docstrings claimed (round-1 review): one would have passed a stage re-expressing the old defect through
root_of=, the other never touched disk. Now:{primary_model, primary_root=<subject>.data_generated, run_type, target}— noroot_ofoverride. Verified against five mutant stages; all fail.metric_frame()with values read back exactly, and the subject-rooted shape provably returnsNonefor the other model. Never a shared root — the fixture blind spot that hid C-215 for months.Pins:
views-pipeline-core>=3.3.0(an older stage would silently blank rows again);views-evaluation>=2.0.0— the floor held yesterday lands now, since vpc 3.3.0 itself requires>=2.0.0and the shared env co-resolves. Lock 129→126 packages. Suite 443 passed on the released 3.3.0 + 2.0.0.Register (honest)
C-215 → Resolved, with what it does not do: views-models' config drift and pins (C-216/C-217), the one-time baseline evaluation run (C-223), the announce/vintage layers (C-219/C-222), and the end-to-end proof — a real ensemble report with baseline rows — still owed. The original "C-220/C-225 land WITH the fix" constraint was not honoured (upstream shipped alone); C-220 is therefore now armed and its cross-level scrub is owed before C-223's evaluation run. Next Cluster H trigger recorded: views-frames 2.0.0 is out but gated by two ceilings (ours + views-evaluation's
[frames]extra, eval#91). No vpc ADR records the per-model rule yet — our 08-21 draft should be filed.Review
Three adversarial rounds: round 1 found the two test weaknesses above plus two false register claims of mine (wrong ADR credited, wrong package named as departed) — all fixed; rounds 2–3 doc-precision only, sibling CICs swept.
Closes #287 item 1 (items 2–4 stay open).
🤖 Generated with Claude Code
https://claude.ai/code/session_01N7YBtiS2kSVj27spMKDYfh