diff --git a/.github/workflows/run_pytest.yml b/.github/workflows/run_pytest.yml index e8dd1b2..e287262 100644 --- a/.github/workflows/run_pytest.yml +++ b/.github/workflows/run_pytest.yml @@ -79,13 +79,29 @@ jobs: # and CI still reports green. Guarded by # tests/test_falsification_extras_actually_installed.py. - name: Verify optional extras are installed - run: poetry run python -c "import views_frames" + run: poetry run python -c "import views_frames, importlib.metadata as m; v = m.version('views-frames'); print('resolved views-frames', v); assert v.split('.')[0] != '1', 'the resolver picked the floor major; the 2.x end of the range would go untested'" - name: Run tests run: | set -e poetry run pytest tests/ + # The `frames` extra admits two views-frames majors (>=1.10.2,<3; #91). The step + # above runs on whatever the resolver picks (asserted to be the 2.x end in the + # verify step), so the FLOOR would be untested. This reads the floor from + # pyproject.toml (one source, no literal to drift), pins it, and runs the whole + # suite on it. Nothing after this step touches Python, so nothing is restored; + # a later Python-touching step must reinstall the extras first (the held workflow + # text forces that review). Guarded by the held workflow text. + - name: Run the suite on the views-frames floor + run: | + set -e + FLOOR=$(poetry run python -c "import tomllib, re; d = tomllib.load(open('pyproject.toml', 'rb')); p = d.get('project', {}); e = [x for x in p.get('dependencies', []) + sum(p.get('optional-dependencies', {}).values(), []) if x.startswith('views-frames')]; spec = re.sub(r'^views-frames(\[[^\]]*\])?', '', e[0]) if e else d['tool']['poetry']['dependencies']['views-frames']['version']; print([s for s in spec.replace(' ', '').split(',') if s.startswith('>=')][0][2:])") + echo "views-frames floor from pyproject.toml: $FLOOR" + poetry run pip install --quiet "views-frames==$FLOOR" + poetry run python -c "import importlib.metadata as m, sys; assert m.version('views-frames') == sys.argv[1], m.version('views-frames')" "$FLOOR" + poetry run pytest tests/ -q + # Structural documentation checks (CIC references, cross-ADR integrity, unfilled # placeholders). Complements tests/test_documentation_contracts.py, which asserts # that documented claims match code. Neither ran in CI before 2026-08-02, which is diff --git a/CHANGELOG.md b/CHANGELOG.md index f362b77..3bb9467 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -17,7 +17,23 @@ provided they were announced here. ## [Unreleased] -_Nothing yet._ +### Changed + +- **The `frames` extra no longer excludes views-frames 2.x** (`>=1.10.2,<3`, was `<2`). + Reported by views-pipeline-core's 3.3.0 range review (#91): views-frames 2.0.0 has been + published since 2026-08-18 and this cap, carried into any environment that requests the + extra, excluded it. What this changes is this package's own contract only — a consumer + whose own bounds exclude views-frames 2.x must widen those too before it resolves. + Measured before widening: the full suite (at 2.0.0's count) on views-frames 1.10.2, + 1.11.0 and 2.0.0, and a `MetricFrame` saved under either major loads under the other + with identical rows and values. Safe because this package's only runtime import from + views-frames is the `FrameMetadata` dataclass (its tests also call the envelope contract + `views_frames.conformance.assert_frame_envelope`, whose body is unchanged between the + majors); views-frames 2.0.0's ADR-028 changes (index type, read-only `.values`, + `map_estimate` refusals) never reach it. Guards now pin the declared range and that + import surface, and CI runs the whole suite on the range's floor as well as on the + resolved version, asserting the resolved one is the 2.x end. Nothing changes in what + this package emits; MINOR under ADR-022 §5. --- diff --git a/documentation/ADRs/022_evolution_and_stability.md b/documentation/ADRs/022_evolution_and_stability.md index 641be5b..b600725 100644 --- a/documentation/ADRs/022_evolution_and_stability.md +++ b/documentation/ADRs/022_evolution_and_stability.md @@ -127,6 +127,7 @@ This policy is checked at two points: - [ ] Does the version bump match the change class (rule 5)? - [ ] Do the release notes list every breaking change with its migration? - [ ] Does `MetricFrame`'s format or axis vocabulary change? If so, has it been agreed with views-reporting and views-pipeline-core? +- [ ] Has any dependency this package caps (`numpy`, `scipy`, `views-frames`) published a release outside the declared range since the last release? If so, is the cap measured and deliberate, or stale? *(Added 2026-09-19, #91: the `frames` extra's `<2` cap outlived views-frames 2.0.0 by a month and was noticed by a consumer, not by a release of ours.)* ## Consequences diff --git a/documentation/CICs/MetricFrame.md b/documentation/CICs/MetricFrame.md index d6812bc..055586b 100644 --- a/documentation/CICs/MetricFrame.md +++ b/documentation/CICs/MetricFrame.md @@ -42,7 +42,7 @@ It is a string-keyed value object — **not** a spatiotemporal `(time, unit)` fr - **`identifiers`**: dict containing exactly the keys in `AXES`, each a 1-D length-`N` array of strings. - **`metadata`**: optional `MetricFrameMetadata`; defaults to an empty one. - **Assumes** the caller has already decided what belongs in the frame. Construction is a structural gate, not a semantic one. -- **Requires** the optional `views-frames` dependency (`pip install views-evaluation[frames]`). The module is import-gated in `views_evaluation/__init__.py` on `find_spec`, so the core API stays importable without it (ADR-011 minimal core). +- **Requires** the optional `views-frames` dependency (`pip install views-evaluation[frames]`), any version in `>=1.10.2,<3` — both majors measured 2026-09-19 (#91). The module is import-gated in `views_evaluation/__init__.py` on `find_spec`, so the core API stays importable without it (ADR-011 minimal core). This package's only runtime import from views-frames is the `FrameMetadata` dataclass (six keyword fields, `to_dict`, `from_dict`); its second dependency is the envelope contract `views_frames.conformance.assert_frame_envelope`, called by the tests, whose body is unchanged between the majors although views-frames moved its `CONFORMANCE_FLOOR` to 2.0.0. `tests/test_falsification_extras_actually_installed.py::TestViewsFramesRange` pins the range and the one-name import surface, `tests/test_metric_frame.py::TestViewsFramesSurface` pins `FrameMetadata`'s behaviour on the installed major, and CI runs the whole suite on the range's floor as well as on the resolved version. --- diff --git a/pyproject.toml b/pyproject.toml index 6b497f4..7a179a9 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -13,7 +13,7 @@ readme = "README.md" python = ">=3.11,<3.15" numpy = "^1.26.4" scipy = "^1.11" # Level-0 kernels (EMD, Pearson); ADR-011 permits numpy and scipy -views-frames = {version = ">=1.10.2,<2", optional = true} +views-frames = {version = ">=1.10.2,<3", optional = true} # both majors measured 2026-09-19 (#91); the only name used is FrameMetadata [tool.poetry.extras] frames = ["views-frames"] diff --git a/reports/technical_risk_register.md b/reports/technical_risk_register.md index b4975ab..ccd468c 100644 --- a/reports/technical_risk_register.md +++ b/reports/technical_risk_register.md @@ -1,6 +1,6 @@ # Technical Risk Register — views-evaluation -**Last updated:** 2026-09-18 +**Last updated:** 2026-09-19 **Total open concerns:** 14 **Governing ADR:** ADR-023 **Citation convention:** `Location` fields name files and symbols (functions, classes, sections), not line numbers — line numbers drift as soon as anything is inserted above them. @@ -14,6 +14,21 @@ > review then found four gaps in the rewrite (linked-worktree `commondir`, locale-dependent > reads, unchecked ref contents, an unpinned name gate) — all fixed and tested before merge. 18 → 17. +> **2026-09-19 — #91: the `frames` extra widened to views-frames `<3`.** views-pipeline-core's 3.3.0 +> range review found that our `<2` cap excluded views-frames 2.0.0 (published 2026-08-18) from any +> environment requesting the extra. (The review of this change found that the report's premise — that +> our cap was the *only* thing holding reporting environments on 1.x — was false: that consumer also +> declares its own `<2.0.0`; so the note below states only this repo's contract.) Measured on +> 1.10.2, 1.11.0 and 2.0.0 (full suite green on each; cross-major save/load identical) and widened; +> guards pin the range, the one-name import surface and `FrameMetadata`'s behaviour; CI runs the whole +> suite on the floor and asserts the resolved version is the 2.x end. Recorded under C-38 as its mirror +> case, C-38's trigger widened, C-41 updated, and ADR-022 §7 gained a checklist line so a capped +> dependency's new major is noticed at the next release rather than by a consumer. The guard audit +> found survivors on four of the new guards (a PEP 621 table read second, `seed=0` dropped by a falsy +> `to_dict`, an environment marker on the entry, `getattr`/re-aliasing on a views_frames alias, and +> floor-step checks that a `-k` or `|| true` slipped past), all closed and re-verified red. Ships as +> 2.1.0 (MINOR). Count unchanged. + > **2026-09-18 — release 2.0.0 (S8, #72).** The MAJOR that ends the pandas-free epic: `to_dataframe()` > and the `dataframe` extra gone one release cycle after 1.1.0's `DeprecationWarning` (ADR-022 §2). > §3.2 notifications posted before the tag and cited by comment ID in the checklist (views-pipeline-core @@ -288,10 +303,11 @@ Root-cause groupings (added 2026-06-26; expanded then largely closed 2026-08-02 - **Description:** On 2026-08-02 the `views-frames` floor was raised from `^1.4` to `>=1.10.2,<2` and merged to `development` and `main`. **`views-reporting` consumes this repo by git branch**, not by version (`views-evaluation = { git = "...", branch = "development" }` in its `[tool.uv.sources]`), with no release, no version bump, and no opportunity for it to opt in. **Nobody checked the consumer's resolution before tightening the constraint** — that is the whole entry. Three separate attempts to write down what that consumer's lock actually held were each wrong, and the third went stale 62 seconds after it was written when the consumer committed, pushed and tagged a release. The correct conclusion is not a better narration of someone else's lockfile: it is that a constraint change here must not be made on an assumption about resolution there, and that a consumer tracking a branch receives such changes with none of a release's gates. - **The sharper case.** A `[tool.uv.sources]` override drops only the *first-party* specifier; it collapses the package's candidate set graph-wide, so any **other** package requiring an excluded version makes the whole resolve unsatisfiable. Widening one repo's pin can therefore break a *different* repo's resolve, and widening must proceed in dependency order. - **Why this is not C-35:** C-35 is about not exercising a consumer's *call path*. This is about not checking a consumer's *dependency resolution*. A library can be perfectly compatible at the API level and still be unresolvable, and no test in either repo would show it — the failure appears at install time, in their CI, after our merge. -- **Trigger:** The next time any constraint in `pyproject.toml` is narrowed — a raised floor, a lowered ceiling, a new required dependency — while any consumer resolves this repo from a branch rather than a published version. Branch-tracking makes every merge a release for that consumer, without any of a release's gates. +- **Trigger:** The next time any constraint in `pyproject.toml` is narrowed — a raised floor, a lowered ceiling, a new required dependency — while any consumer resolves this repo from a branch rather than a published version; **and (added 2026-09-19, #91) whenever a dependency this repo caps publishes a release outside the declared range** — the ADR-022 §7 checklist now asks at every release, because a month passed between views-frames 2.0.0 and anyone noticing the `<2` cap. Branch-tracking makes every merge a release for that consumer, without any of a release's gates. - **Location:** `pyproject.toml` (dependency constraints); `views-reporting/pyproject.toml` (formerly the `[tool.uv.sources]` branch-tracking entry; **as of 2026-09-16 it declares `views-evaluation[frames]>=1.0.0,<2.0.0`**, a plain version range) - **Source:** falsification audit round 3 (2026-08-02); consumer state re-checked 2026-09-16 (repo-assimilation) - **State update (2026-09-16):** views-reporting has executed this entry's cheapest mitigation — it now pins a published version range rather than a branch, so this repo's merges no longer reach it un-gated. The trigger remains valid for any *other* consumer that resolves from a branch, and the sharper resolver-collapse case above is unchanged. No branch-tracking consumer is known at the time of writing; that is an observation, not a guarantee (the 2026-08-02 narration failures above are the reason this entry does not describe consumer state in more detail). Inward-facing counterpart: C-41. +- **The mirror case (2026-09-19, #91):** a constraint can also be too *narrow* for too long. The `frames` extra capped views-frames `<2` for a month after views-frames 2.0.0 shipped, and the cap travels through the extra into every environment that requests it — an environment we do not own. Found by views-pipeline-core's range review, not ours; and their report's premise that our cap was the *only* bound in play was itself wrong (the consumer concerned also declares its own `<2.0.0`), which is the C-38 narration hazard in the other direction. Widened to `<3` on measurement (suite green on 1.10.2, 1.11.0 and 2.0.0; cross-major save/load round trip); `TestViewsFramesRange` and `TestViewsFramesSurface` pin the range, the one-name import surface and `FrameMetadata`'s behaviour, and CI runs the whole suite on the floor and asserts the resolved version is the 2.x end. The lesson is the same as this entry's: a dependency constraint in this repo is a decision about a consumer's environment, in both directions. - **Mitigation path:** Cheapest durable fix is to remove the reason for branch-tracking: now that the emit is published, views-reporting can revert to a plain version pin, at which point our merges stop reaching it un-gated. Failing that, treat narrowing a constraint as a breaking change under ADR-022 §3 and check the locks of known branch-tracking consumers first — the same "execute the real consumer's path" rule as C-35, applied to resolution rather than to calls. - **Note:** ADR-022 §3.2 already requires notifying known consumers before a release. Branch-tracking sidesteps that entirely, because there is no release to notify about. See C-35, C-36. @@ -300,7 +316,7 @@ Root-cause groupings (added 2026-06-26; expanded then largely closed 2026-08-02 ### C-41 — CI resolves dependencies fresh each run, tests one interpreter of four declared, under a numpy 1.x ceiling - **Tier:** 3 (Medium) — reproducibility and coverage gap that raises the cost of every dependency change and every green-build claim; no correctness impact today. -- **Description:** No `poetry.lock` is committed (the `.gitignore` comment leaves it optional and none exists), so each `poetry install --all-extras` in `run_pytest.yml` resolves anew, and "CI passed" does not name the versions it passed against. `pyproject.toml` declares `python = ">=3.11,<3.15"` while `run_pytest.yml` pins `python-version: "3.11"` — three of the four claimed interpreters are never exercised. `numpy = "^1.26.4"` caps at the last 1.x line while `scikit-learn` (`^1.6.0`), `scipy` (`^1.11`) and `views-frames` (`>=1.10.2,<2`) each admit every later minor release below 2.0, so a *minor* release of any of them that drops numpy 1.x support makes the resolve unsatisfiable, or — worse — resolves to a different combination between two runs with no diff to show it. The local environment used for the 2026-09-16 assimilation resolved to numpy 1.26.4, scipy 1.15.1, scikit-learn 1.7.2, pandas 1.5.3, views-frames 1.10.2; nothing records that CI resolved the same. +- **Description:** No `poetry.lock` is committed (the `.gitignore` comment leaves it optional and none exists), so each `poetry install --all-extras` in `run_pytest.yml` resolves anew, and "CI passed" does not name the versions it passed against. `pyproject.toml` declares `python = ">=3.11,<3.15"` while `run_pytest.yml` pins `python-version: "3.11"` — three of the four claimed interpreters are never exercised. `numpy = "^1.26.4"` caps at the last 1.x line while `scikit-learn` (`^1.6.0`), `scipy` (`^1.11`) and `views-frames` (`>=1.10.2,<2`; **`<3` since 2026-09-19**, #91 — the admitted set now spans a whole second major line, and `run_pytest.yml`'s verify step is the first CI step that prints and asserts which views-frames it resolved, a partial mitigation of this entry) each admit every later minor release below 2.0, so a *minor* release of any of them that drops numpy 1.x support makes the resolve unsatisfiable, or — worse — resolves to a different combination between two runs with no diff to show it. The local environment used for the 2026-09-16 assimilation resolved to numpy 1.26.4, scipy 1.15.1, scikit-learn 1.7.2, pandas 1.5.3, views-frames 1.10.2; nothing records that CI resolved the same. - **Trigger:** When scikit-learn, scipy or views-frames publishes a release that requires numpy ≥ 2, or when a maintainer adds Python 3.12+ to the CI matrix, or when a CI run goes red with no change to this repo — check what CI actually resolved, and whether it can be reproduced. - **Location:** `pyproject.toml` (`numpy = "^1.26.4"`, `python = ">=3.11,<3.15"`, caret ranges on `scikit-learn` and `scipy` that admit future 1.x minors); `.github/workflows/run_pytest.yml` (`python-version: "3.11"`; `poetry install --all-extras` without a lock); absence of `poetry.lock` - **Source:** repo-assimilation (2026-09-16) diff --git a/tests/test_falsification_extras_actually_installed.py b/tests/test_falsification_extras_actually_installed.py index fe11048..9d73486 100644 --- a/tests/test_falsification_extras_actually_installed.py +++ b/tests/test_falsification_extras_actually_installed.py @@ -107,6 +107,116 @@ def _strip_comment(s): return "".join(out).rstrip() +def _views_frames_specifier(): + """The declared views-frames version specifier, from either pyproject layout — + PEP 621 FIRST, as `_extras_table` reads it and as poetry >= 2 does, so a stale + poetry entry beside an authoritative `[project]` table cannot be the one read + (guard audit, 2026-09-19). Rejects an environment marker on the entry: a + `python = ">=3.12"` key or a `; python_version` marker leaves the extra unresolved + on the 3.11 runner with the specifier text unchanged.""" + import tomllib + data = tomllib.loads(PYPROJECT.read_text(encoding="utf-8")) + project = data.get("project", {}) + for entry in project.get("dependencies", []) + sum(project.get("optional-dependencies", {}).values(), []): + if re.match(r"views-frames\b", entry): + assert ";" not in entry, f"views-frames carries an environment marker: {entry!r}" + return re.sub(r"^views-frames(?:\[[^\]]*\])?\s*", "", entry).replace(" ", "") + poetry = data.get("tool", {}).get("poetry", {}).get("dependencies", {}).get("views-frames") + if isinstance(poetry, dict): + assert set(poetry) <= {"version", "optional"}, f"views-frames poetry entry carries extra keys: {sorted(poetry)}" + return poetry["version"].replace(" ", "") + if isinstance(poetry, str): + return poetry.replace(" ", "") + raise AssertionError("views-frames is not declared in pyproject.toml") + + +class TestViewsFramesRange: + """The `frames` extra admits two views-frames majors (`>=1.10.2,<3`, #91), measured + on both ends. Two static pins, hosted here because this module never import-skips: + the declared range itself, and the one-name import surface that justifies it. The + behavioural half is `tests/test_metric_frame.py::TestViewsFramesSurface`.""" + + def test_the_declared_range_is_the_measured_one(self): + """Compared as specifier sets, so `<3,>=1.10.2` or a space is not a false alarm.""" + from packaging.specifiers import SpecifierSet + spec = _views_frames_specifier() + assert SpecifierSet(spec) == SpecifierSet(">=1.10.2,<3"), ( + f"views-frames is declared {spec!r}; the measured range is >=1.10.2,<3 (#91). " + f"Widen or narrow only with the suite green on both ends of the new range." + ) + + def test_the_ci_floor_step_derives_the_floor_from_pyproject(self): + """The floor step must read the floor from pyproject.toml and carry no version + literal of its own — a hand-copied `1.10.2` in the step (there were two) is a + second source that a floor raise can miss. Comment-stripped, so a commented-out + line cannot satisfy it.""" + steps = _workflow_steps(WORKFLOW.read_text(encoding="utf-8")) + floor = [s for s in steps if "floor" in s["name"].lower()] + assert len(floor) == 1, [s["name"] for s in steps] + lines = _significant_lines(floor[0]["run"]) + run = "\n".join(lines) + # The derivation reads pyproject.toml with tomllib and names views-frames; a + # literal `FLOOR=1.10.2` beside an `echo ... pyproject.toml` satisfied a + # substring check (guard audit, 2026-09-19). + derivation = [line for line in lines if line.startswith("FLOOR=")] + assert len(derivation) == 1 and "tomllib" in derivation[0] and "pyproject.toml" in derivation[0], derivation + # Both layouts must key on views-frames and on nothing else: a derivation from + # the numpy entry still mentioned 'views-frames' in its PEP 621 branch. + assert re.findall(r"\['([\w-]+)'\]\['version'\]", derivation[0]) == ["views-frames"], derivation + assert "startswith('views-frames')" in derivation[0], derivation + assert "views-frames==$FLOOR" in run, run + assert not re.search(r"views-frames==\d", run), "the floor step hard-codes a version" + # The suite command exactly: `-k`, `--ignore`, `--deselect`, `|| true` all + # passed a `pytest tests/` prefix check. + assert "poetry run pytest tests/ -q" in lines, lines + assert not re.search(r"\|\||;\s*(true|:)\s*$", run, re.M), "a suppressed exit in the floor step" + assert not re.search(r"^\s*continue-on-error", floor[0].get("raw", ""), re.M) + + def test_the_only_name_imported_from_views_frames_is_frame_metadata(self): + """AST over every package file: `from views_frames import X`, `import + views_frames[.sub] as y`, and `views_frames.X` attribute access may name only + FrameMetadata. A second name is a second thing that can change between majors + and is a red build until measured. (`import views_frames as vf; vf.X` was + invisible to the first version, and `getattr(views_frames, "X")` and + `_vf = views_frames; _vf.X` to the second — release review and guard audit, + 2026-09-19. Import machinery — `import_module("views_frames").X` — is forbidden + package-wide by `TestLevelZeroImportPurity`.)""" + import ast + import views_evaluation + names = set() + for path in Path(views_evaluation.__file__).parent.rglob("*.py"): + tree = ast.parse(path.read_text(encoding="utf-8")) + aliases = {"views_frames"} + for node in ast.walk(tree): + if isinstance(node, ast.Import): + for a in node.names: + if a.name.split(".")[0] == "views_frames": + names.add(a.name) if "." in a.name else None + aliases.add(a.asname or a.name.split(".")[0]) + # Re-aliasing by assignment (`_vf = views_frames`), to a fixed point. + grew = True + while grew: + grew = False + for node in ast.walk(tree): + if isinstance(node, ast.Assign) and isinstance(node.value, ast.Name) and node.value.id in aliases: + for t in node.targets: + if isinstance(t, ast.Name) and t.id not in aliases: + aliases.add(t.id) + grew = True + for node in ast.walk(tree): + if isinstance(node, ast.ImportFrom) and (node.module or "").split(".")[0] == "views_frames": + names |= {f"{node.module}.{a.name}" for a in node.names} + elif isinstance(node, ast.Attribute) and isinstance(node.value, ast.Name) and node.value.id in aliases: + names.add(f"views_frames.{node.attr}") + elif (isinstance(node, ast.Call) and isinstance(node.func, ast.Name) and node.func.id == "getattr" + and node.args and isinstance(node.args[0], ast.Name) and node.args[0].id in aliases): + # `getattr(views_frames, "X")`: a literal is a name; anything else is + # a name this guard cannot read, which is as bad. + attr = node.args[1] if len(node.args) > 1 else None + names.add(f"views_frames.{attr.value}" if isinstance(attr, ast.Constant) else "views_frames.") + assert names == {"views_frames.FrameMetadata"}, sorted(names) + + def _commands(run): """Executable command lines from a run block, with inert ones dropped. @@ -466,11 +576,19 @@ class TestPublishGateIsReal: PY poetry install --all-extras - name: Verify optional extras are installed - run: poetry run python -c "import views_frames" + run: poetry run python -c "import views_frames, importlib.metadata as m; v = m.version('views-frames'); print('resolved views-frames', v); assert v.split('.')[0] != '1', 'the resolver picked the floor major; the 2.x end of the range would go untested'" - name: Run tests run: | set -e poetry run pytest tests/ + - name: Run the suite on the views-frames floor + run: | + set -e + FLOOR=$(poetry run python -c "import tomllib, re; d = tomllib.load(open('pyproject.toml', 'rb')); p = d.get('project', {}); e = [x for x in p.get('dependencies', []) + sum(p.get('optional-dependencies', {}).values(), []) if x.startswith('views-frames')]; spec = re.sub(r'^views-frames(\[[^\]]*\])?', '', e[0]) if e else d['tool']['poetry']['dependencies']['views-frames']['version']; print([s for s in spec.replace(' ', '').split(',') if s.startswith('>=')][0][2:])") + echo "views-frames floor from pyproject.toml: $FLOOR" + poetry run pip install --quiet "views-frames==$FLOOR" + poetry run python -c "import importlib.metadata as m, sys; assert m.version('views-frames') == sys.argv[1], m.version('views-frames')" "$FLOOR" + poetry run pytest tests/ -q - name: Validate documentation consistency run: bash documentation/validate_docs.sh """, diff --git a/tests/test_metric_frame.py b/tests/test_metric_frame.py index 71e6264..a16f3da 100644 --- a/tests/test_metric_frame.py +++ b/tests/test_metric_frame.py @@ -969,3 +969,52 @@ def test_submodule_relative_gitdir_resolves_against_the_git_file(self, tmp_path, monkeypatch.setattr(mf_mod, "__file__", str(_fake_package_file(repo))) assert mf_mod._source_git_sha() == "1234567" + + + +class TestViewsFramesSurface: + """The `frames` extra admits two views-frames majors (`>=1.10.2,<3`, #91). That is + safe only because this package's runtime surface into views-frames is one name, + `FrameMetadata`, used as a plain dataclass: six keyword fields, `to_dict` (omitting + None), `from_dict` (ignoring keys it does not own, which `MetricFrame.load` relies + on), default construction. views-frames 2.0.0's ADR-028 changes (a frame's index + type, read-only `.values`, `map_estimate` refusals) never reach it. Measured + 2026-09-19: the full suite (at its 2.0.0 count, before these tests) green on 1.10.2, + 1.11.0 and 2.0.0, and a frame saved + under either major loads under the other. The declared range and the one-name + import surface are pinned in tests/test_falsification_extras_actually_installed.py + (outside this module's import-skip); this class pins the BEHAVIOUR, against + whichever views-frames is installed.""" + + def test_installed_views_frames_is_inside_the_declared_range(self): + """Not `major in (1, 2)`: that is a third copy of the ceiling and admits a + below-floor release (1.9.0 passed it). The declared specifier is the one source.""" + import importlib.metadata as md + from packaging.specifiers import SpecifierSet + from tests.test_falsification_extras_actually_installed import _views_frames_specifier + installed = md.version("views-frames") + assert SpecifierSet(_views_frames_specifier()).contains(installed, prereleases=True), ( + f"views-frames {installed} is outside the declared range; measure before widening" + ) + + def test_frame_metadata_offers_the_surface_this_package_uses(self): + """Behavioural, against the installed major: the six keyword fields + `to_metric_frame` passes; default construction (reached through + `MetricFrame(..., metadata=None)` → `default_factory=FrameMetadata`); `to_dict` + omitting None; `from_dict` ignoring the keys `MetricFrameMetadata.to_dict()` + adds beside the generic ones (`schema_version`, `scoring_code_version`, + `evaluation_timestamp`) — which is what `MetricFrame.load()` hands it — and an + int timestamp surviving the round trip as an int, not a float that compares equal.""" + fm = FrameMetadata(model="m", run_type="test", timestamp=1758153600, seed=7, run_id="r", data_version="v") + assert FrameMetadata.from_dict(fm.to_dict()) == fm + back = FrameMetadata.from_dict({**fm.to_dict(), "schema_version": "1.0.0", "scoring_code_version": "2.1.0", + "evaluation_timestamp": "2026-09-19T00:00:00"}) + assert back == fm, "from_dict must ignore the MetricFrameMetadata-owned keys load() passes through" + assert type(back.timestamp) is int + assert FrameMetadata().to_dict() == {}, "to_dict must omit None so metadata.json keeps its shape" + assert FrameMetadata(model="m").to_dict() == {"model": "m"} + # None, not falsy: `seed=0` and `timestamp=0` are values and must survive the + # trip, or a run seeded 0 would load with `seed=None` (guard audit, 2026-09-19). + zero = FrameMetadata(model="m", run_type="t", timestamp=0, seed=0, run_id="", data_version="") + assert FrameMetadata.from_dict(zero.to_dict()) == zero + assert zero.to_dict()["seed"] == 0 and zero.to_dict()["timestamp"] == 0