From ade7d77c56041baec149e3b6b1a7380c714b8cb7 Mon Sep 17 00:00:00 2001 From: Jonathan Borduas Date: Tue, 8 Sep 2026 06:53:39 +0100 Subject: [PATCH] =?UTF-8?q?test=5Fbootstrap=5Faudit:=20the=20SUITE-DEPENDS?= =?UTF-8?q?=20marker=20was=20stale=20=E2=80=94=20#20's=20gated=20caller=20?= =?UTF-8?q?now=20runs?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit #20's close condition names this file by name, and its parenthetical was still true 19 days later: "CALLER THAT STILL RUNS IT (#381): tools/bootstrap-audit.py classifying a deliberately broken bootstrap and reporting the failure, IN A GATED SUITE. ⚠ Note test_bootstrap_audit currently declares `# SUITE-DEPENDS:` and is therefore gated nowhere (#286) — so the caller exists and does not run." ⇒ MEASURED BOTH DIRECTIONS, because "it passes" is not enough. A suite can pass by SKIPPING the legs that need the thing — which is exactly what the marker's own reason warns of ("a fake would make its control pass without the thing it controls for"): with a live registry rc 0 · 12 check lines env -i, HOME=/nonexistent rc 0 · 12 check lines · output BYTE-IDENTICAL ⛔ CONTROL, same bare env test_stranded_branches.py -> rc 1, FileNotFoundError: 'gh' (a real dependency) ⇒ No leg reads the registry. The declaration excluded the suite from the gate for a dependency it does not have. AT THE GATE, and the number MOVES with the change — which is what makes it a measurement: before ran 57 hermetic suite(s) · 57 passed · skipped 4 after ran 58 hermetic suite(s) · 58 passed · skipped 3 · 0 FINDINGS ★ I SWEPT THE OTHER THREE with the same probe, and only this one was stale: test_runmarker bare rc 1, output DIFFERS genuine (injects into pane-binding, which exits 2 on an empty registry) test_stranded_branches bare rc 1, output DIFFERS genuine (gh, network + auth) test_grant_check bare rc 0, IDENTICAL ⚠ NOT stale — its marker states a deliberate judgement: "MEASURED PASSING on a runner (run 32345860216); listed here only because a network-dependent gate is a flake source." That is an author's call about flakiness, not a claim I can refute by running it once. Left alone. ⚠ This ADDS a suite to the gating population. It passes hermetically today; if it ever fails there, that is a finding rather than a regression in this change. Co-Authored-By: Claude Opus 5 (1M context) --- tools/test_bootstrap_audit.py | 24 +++++++++++++++++++++++- 1 file changed, 23 insertions(+), 1 deletion(-) diff --git a/tools/test_bootstrap_audit.py b/tools/test_bootstrap_audit.py index 7e97cd8..7f3a565 100644 --- a/tools/test_bootstrap_audit.py +++ b/tools/test_bootstrap_audit.py @@ -28,7 +28,29 @@ Run: python3 tools/test_bootstrap_audit.py """ -# SUITE-DEPENDS: needs a live Daintree pane registry; a fake would make its control pass without the thing it controls for +# ⛔ THE `# SUITE-DEPENDS:` MARKER WAS REMOVED HERE, 2026-09-08, and the reason it +# carried is no longer true of this suite. It read: +# +# needs a live Daintree pane registry; a fake would make its control pass +# without the thing it controls for +# +# Measured, both directions, because "it passes" is not enough — a suite can pass by +# SKIPPING the legs that need the thing, which is precisely what that reason warns of: +# +# with a live registry rc 0 · 12 check lines +# env -i, HOME=/nonexistent rc 0 · 12 check lines · output BYTE-IDENTICAL +# ⛔ CONTROL, same bare env: test_stranded_branches.py -> rc 1, +# FileNotFoundError: 'gh' (a real dependency) +# +# ⇒ No leg of this suite reads the registry. The declaration excluded it from the +# gating population for a dependency it does not have. +# +# ★ WHY THAT MATTERED: #20's close condition names this file by name — "CALLER THAT +# STILL RUNS IT (#381): tools/bootstrap-audit.py classifying a deliberately broken +# bootstrap and reporting the failure, IN A GATED SUITE. Note test_bootstrap_audit +# currently declares # SUITE-DEPENDS: and is therefore gated nowhere (#286) — so the +# caller exists and does not run." That was written 2026-08-20 and was still true +# 19 days later. import importlib.util import os