From f941270657d14314d167cdb3aecacc2c2c6015f5 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 9 Aug 2026 15:03:35 +0000 Subject: [PATCH 1/2] lifecycle: let a draft declare its gate, and grade the two readings apart Leg 1 of draft/feature/pyautomind/draft_staleness_detection_signals.md, filed from the draft/ sweep. Five prompts in that sweep had a stated gate that had since closed, and nobody noticed -- one of them (the test-mode umbrella) had its exit condition met the SAME DAY the status line was written. The blocker was never detection, it was ambiguity. A draft citing an issue can mean two opposite things: "epic closes when #70 ships its recipe leg" (closed => this prompt is DONE) or "BLOCKED until PyAutoArray#431 merges" (closed => this prompt is READY). Prose cannot be graded, so `issues --drafts` had to report both as one question, "shipped, or newly unblocked?", and an advisory that cannot say which is not actionable. Two optional header keys let the author say which they mean: Closes-when: autolens_profiling#70 Blocked-by: PyAutoArray#431, PyAutoGalaxy#486 `issues --drafts` now reports GATE MET (verify, then retire) and UNBLOCKED (ready to start) as separate bands, and prompts that declare a gate drop out of the ambiguous list -- so that list becomes the genuinely-undeclared remainder rather than everything. Design decisions worth recording: Aggregated PER PROMPT, not per reference. A prompt blocked on three PRs is unblocked when the LAST one lands; reporting each ref separately would have claimed "ready to start" three times while it was still blocked. Partly-closed gates get their own weaker band, which is the real state of ep_analytic_updates -- its WP1 gate merged, its WP3/WP4 gates are open. Repo#123 shorthand, not just URLs. Measured on the real backlog: a URL-only extractor found 2 refs, the shorthand form found 8. Prompts do not write URLs. Fenced blocks are documentation. The prompt that PROPOSED these keys shows them in a ```markdown example; parsing that as a declaration would have invented a finding out of the proposal itself. Caught while backfilling, and covered by a test that fails without the fence check. Still advisory, never a gate on the exit code -- retiring a prompt writes to complete/ and stays human, the same contract intake reconcile keeps. Backfilled the five prompts whose gates this sweep confirmed, giving a live ready-to-start list: sampler_cli_output_workspace_sweep (PyAutoFit#1436), preloads_advanced_workspace_guide (PyAutoLens#565), unpark_imaging_scaling_relation_slam (PyAutoArray#431), oversampled_psf_dataset_adoption (three k x s merges), and ep_analytic_updates (partial). Verified against the states this session confirmed via the API: 4 fully unblocked, 1 partial, 0 falsely flagged. 9 new tests, each proving the leg fails without the code -- including the two opposite readings, the per-prompt aggregation, the fenced-example case, and that a declared gate is not also asked as an ambiguous advisory. 118 -> 127 passing. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01E3MuurHXi3xo9TLRpMLJA6 --- REFERENCE.md | 31 ++++ .../sampler_cli_output_workspace_sweep.md | 2 + .../preloads_advanced_workspace_guide.md | 1 + draft/feature/autofit/ep_analytic_updates.md | 1 + .../oversampled_psf_dataset_adoption.md | 1 + .../unpark_imaging_scaling_relation_slam.md | 1 + scripts/lifecycle.py | 167 +++++++++++++++++- tests/test_lifecycle_check.py | 107 +++++++++++ 8 files changed, 307 insertions(+), 4 deletions(-) diff --git a/REFERENCE.md b/REFERENCE.md index 869fb934..0816961d 100644 --- a/REFERENCE.md +++ b/REFERENCE.md @@ -298,11 +298,42 @@ Free-form markdown. Strong conventions: Autonomy: supervised # safe | supervised | human-required Priority: normal # low | normal | high Status: draft + Blocked-by: PyAutoFit#1436 # optional; see "Declaring a gate" below ``` When present, `Type:` should match the work-type folder. The goal is light structure, not bureaucracy — prompts stay free-form prose. + **Declaring a gate — `Closes-when:` / `Blocked-by:`.** Both optional. A prompt + that waits on something external can say so in a form `lifecycle.py issues + --drafts` can grade: + + ```markdown + Closes-when: autolens_profiling#70 # this prompt is DONE when that closes + Blocked-by: PyAutoArray#431, PyAutoGalaxy#486 # READY TO START when all close + ``` + + The two readings are **opposite**, which is the whole point. Prose cannot be + graded, so a cited issue could mean either and `--drafts` had to report every + one as the same ambiguous question. With a declared key the tool reports the + action instead: a closed `Closes-when:` says *likely shipped, verify and + retire*; a closed `Blocked-by:` says *ready to start*. Prompts declaring a gate + drop out of the ambiguous advisory list. + + Notes: + - Accepts `Repo#123` shorthand (assumed `PyAutoLabs/`) or a full URL, and PRs + as well as issues. Several refs may be comma-separated. + - `Blocked-by:` clears only when **every** ref closes; a partly-satisfied gate + is reported in its own weaker band rather than as ready. + - Keys inside fenced code blocks are documentation and are ignored, so a prompt + may show the syntax without declaring a gate. + - Advisory, never a gate on the exit code: retiring a prompt writes to + `complete/` and stays a human act. + + Motivated by the 2026-08-09 `draft/` sweep, where five prompts' stated gates + had closed without anyone noticing — including one whose exit condition was met + the same day it was written. + The optional `Difficulty:` / `Autonomy:` / `Priority:` keys let both people and PyAutoBrain see, at a glance, how hard a task is, whether an agent can safely take it on, and how urgent it is. What each `Autonomy:` level *does* at every diff --git a/draft/docs/autolens_workspace/sampler_cli_output_workspace_sweep.md b/draft/docs/autolens_workspace/sampler_cli_output_workspace_sweep.md index 77f7cf44..3009d6e6 100644 --- a/draft/docs/autolens_workspace/sampler_cli_output_workspace_sweep.md +++ b/draft/docs/autolens_workspace/sampler_cli_output_workspace_sweep.md @@ -1,5 +1,7 @@ # Phase 2 — drop the hand-written quick-update sentence from the workspace scripts +Blocked-by: PyAutoFit#1436 # moved the cadence message into the library — MERGED 2026-07-30 + Follow-up to **PyAutoFit#1434 / PR#1436**, which moved the on-the-fly update cadence message into the library. Do not start until #1436 has merged. diff --git a/draft/docs/workspaces/preloads_advanced_workspace_guide.md b/draft/docs/workspaces/preloads_advanced_workspace_guide.md index 30e0b946..f29b6654 100644 --- a/draft/docs/workspaces/preloads_advanced_workspace_guide.md +++ b/draft/docs/workspaces/preloads_advanced_workspace_guide.md @@ -6,6 +6,7 @@ Difficulty: too-large Autonomy: supervised Priority: high Status: formalised +Blocked-by: PyAutoLens#565 # the Preloads API landing in PyAutoArray — CLOSED 2026-07-10 Write an **advanced** guide, in both `@autolens_workspace` and `@autogalaxy_workspace`, on the `Preloads` API (`aa.PreloadsInterferometer`, `aa.AbstractPreloads`, and `PreloadsImaging` if added) diff --git a/draft/feature/autofit/ep_analytic_updates.md b/draft/feature/autofit/ep_analytic_updates.md index f610866d..35f552f9 100644 --- a/draft/feature/autofit/ep_analytic_updates.md +++ b/draft/feature/autofit/ep_analytic_updates.md @@ -6,6 +6,7 @@ Difficulty: large Autonomy: supervised Priority: normal Status: formalised +Blocked-by: PyAutoFit#1334, PyAutoFit#1331, PyAutoFit#1332 # WP1 gate (MERGED); WP3/WP4 gates (open) The complete, self-contained implementation plan lives in **PyAutoFit#1338** (plan-only issue; nothing implemented). Scoping diff --git a/draft/feature/autolens_workspace/oversampled_psf_dataset_adoption.md b/draft/feature/autolens_workspace/oversampled_psf_dataset_adoption.md index c35f8364..f8e639b2 100644 --- a/draft/feature/autolens_workspace/oversampled_psf_dataset_adoption.md +++ b/draft/feature/autolens_workspace/oversampled_psf_dataset_adoption.md @@ -8,6 +8,7 @@ Difficulty: large Autonomy: supervised Priority: normal Status: formalised +Blocked-by: PyAutoArray#363, PyAutoGalaxy#486, autolens_workspace#236 # the k x s machinery — all MERGED Option (a) of the phase-4 fork (PyAutoLabs/PyAutoArray#362, decided 2026-07-08): flip the executed start-here simulator to diff --git a/draft/maintenance/workspaces/unpark_imaging_scaling_relation_slam.md b/draft/maintenance/workspaces/unpark_imaging_scaling_relation_slam.md index a11a0384..c05d411d 100644 --- a/draft/maintenance/workspaces/unpark_imaging_scaling_relation_slam.md +++ b/draft/maintenance/workspaces/unpark_imaging_scaling_relation_slam.md @@ -8,6 +8,7 @@ Difficulty: small Autonomy: supervised Priority: normal Status: formalised +Blocked-by: PyAutoArray#431 # the small-datasets loader fix — MERGED 2026-08-03, released in 2026.8.7.1 BLOCKED until PyAutoArray PR#431 merges and reaches the installed stack. Do not start before then — the script only passes with that loader fix in place. diff --git a/scripts/lifecycle.py b/scripts/lifecycle.py index 2cee32e5..5424f178 100644 --- a/scripts/lifecycle.py +++ b/scripts/lifecycle.py @@ -343,8 +343,13 @@ def draft_issue_notes(root: Path, fetch=None) -> "list[str]": registry entry's `issue:` is its OWN tracking issue, so closed means done. A draft usually cites an issue as CONTEXT — "Once #480 is fixed…", "Follow-up to #57" — so closed can mean the draft is newly UNBLOCKED rather than - finished. Both readings are worth a human look; neither is a gate.""" - refs = draft_issue_refs(root) + finished. Both readings are worth a human look; neither is a gate. + + For the drafts that DO state which reading applies, see `draft_gate_notes`: + an explicit `Closes-when:` / `Blocked-by:` header removes exactly this + ambiguity, and those drafts are reported there instead of here.""" + gated = {path for path, _, _ in draft_gate_refs(root)} + refs = [(p, u) for p, u in draft_issue_refs(root) if p not in gated] if not refs: return [] fetch = fetch or _gh_issue_states @@ -356,6 +361,133 @@ def draft_issue_notes(root: Path, fetch=None) -> "list[str]": ] +# --------------------------------------------------------------------------- # +# draft gates +# +# The 2026-08-09 draft/ sweep found five prompts whose stated gate had since +# closed, and the two readings are OPPOSITE: `test_mode_representative_outputs` +# said "EPIC CLOSES when #70 ships its recipe leg" (gate closed => the prompt is +# DONE), while `unpark_imaging_scaling_relation_slam` said "BLOCKED until +# PyAutoArray PR#431 merges" (gate closed => the prompt is READY TO START). +# Prose cannot be graded, so `--drafts` had to lump both into one "shipped, or +# newly unblocked?" note. These keys let a prompt say which it means. +# --------------------------------------------------------------------------- # +GATE_FIELDS = ("closes-when", "blocked-by") +#: `Repo#123` shorthand as well as full URLs — prompts overwhelmingly write the +#: former, and a URL-only extractor found 2 refs across the backlog where the +#: shorthand form found 8 (2026-08-09 measurement). +GATE_REF_RE = re.compile( + r"https://github\.com/([\w.-]+)/([\w.-]+)/(?:issues|pull)/(\d+)" + r"|(? "str | None": + """Normalise either GATE_REF_RE alternative to a canonical issues URL. + + `Repo#123` cannot say whether 123 is an issue or a PR, and the GitHub API + resolves an issues URL for both (a PR *is* an issue), so the issues form is + the safe canonical shape.""" + owner, repo, num, short_repo, short_num = match.groups() + if owner: + return f"https://github.com/{owner}/{repo}/issues/{num}" + if short_repo: + return f"https://github.com/{DEFAULT_GATE_OWNER}/{short_repo}/issues/{short_num}" + return None + + +def draft_gate_refs(root: Path) -> "list[tuple[str, str, str]]": + """(draft_path, gate_kind, url) for drafts carrying a gate header key. + + `gate_kind` is `closes-when` or `blocked-by` — the two opposite readings. + A key may list several refs; each becomes its own entry, because a prompt + blocked on three PRs is only unblocked when the last one lands.""" + draft = root / "draft" + if not draft.is_dir(): + return [] + refs = [] + for f in sorted(draft.rglob("*.md")): + rel = str(f.relative_to(root)) + in_fence = False + for line in f.read_text(errors="replace").splitlines(): + # Fenced blocks are documentation, not declarations. Prompts that + # *describe* these keys (this feature's own prompt does, in a + # ```markdown example) must not be read as declaring them. + if line.lstrip().startswith("```"): + in_fence = not in_fence + continue + if in_fence: + continue + m = _GATE_KEY_RE.match(line) + if not m: + continue + kind = m.group(1).lower() + for ref in GATE_REF_RE.finditer(m.group(2)): + url = _gate_url(ref) + if url: + refs.append((rel, kind, url)) + return refs + + +def draft_gate_notes(root: Path, fetch=None) -> "dict[str, list[str]]": + """Drafts whose declared gate has closed, split by what that MEANS. + + Returns `{"shipped": [...], "unblocked": [...], "partial": [...], + "unreadable": [...]}`. Unlike `draft_issue_notes` these are unambiguous — + the prompt author said which reading applies — so each line states the + action rather than asking a question. + + Aggregated PER PROMPT, not per reference: a prompt blocked on three PRs is + unblocked only when the last one lands, so reporting each ref separately + would claim "ready to start" three times while it is still blocked. A + partially-satisfied `Blocked-by:` is reported in its own weaker band, which + is the real state of `ep_analytic_updates` (its WP1 gate merged; the WP3/WP4 + gates are open). + + Still advisory, and deliberately so: a satisfied `Closes-when:` is strong + evidence the work is done, but retiring a prompt writes to `complete/` and + stays a human act (the same contract `intake reconcile` keeps).""" + refs = draft_gate_refs(root) + out = {"shipped": [], "unblocked": [], "partial": [], "unreadable": []} + if not refs: + return out + fetch = fetch or _gh_issue_states + states = fetch(sorted({url for _, _, url in refs})) + + grouped: "dict[tuple[str, str], list[str]]" = {} + for path, kind, url in refs: + grouped.setdefault((path, kind), []).append(url) + + for (path, kind), urls in sorted(grouped.items()): + got = [(u, states.get(u, "unknown")) for u in urls] + bad = [f"{u} ({s})" for u, s in got if s not in ("open", "closed")] + if bad: + out["unreadable"].append( + f"{path}: could not read {len(bad)} declared gate(s): " + + ", ".join(bad)) + continue + closed = [u for u, s in got if s == "closed"] + if not closed: + continue + joined = ", ".join(closed) + if len(closed) < len(got): + still = ", ".join(u for u, s in got if s == "open") + out["partial"].append( + f"{path}: {len(closed)} of {len(got)} `{kind}:` gates closed — " + f"partly ready; still open: {still}") + elif kind == "closes-when": + out["shipped"].append( + f"{path}: every `Closes-when:` gate is CLOSED — the prompt's own " + f"exit condition is met, so this is very likely shipped: {joined}") + else: + out["unblocked"].append( + f"{path}: every `Blocked-by:` gate is CLOSED — ready to start, " + f"not blocked: {joined}") + return out + + def cmd_issues(args) -> int: """Cross-check every registry entry's tracking issue against GitHub.""" try: @@ -370,8 +502,10 @@ def cmd_issues(args) -> int: ) return 2 notes = [] + gates = {"shipped": [], "unblocked": [], "partial": [], "unreadable": []} if getattr(args, "drafts", False): try: + gates = draft_gate_notes(ROOT) notes = draft_issue_notes(ROOT) except GhUnavailable: pass # unreachable: issue_problems above would already have raised @@ -383,10 +517,35 @@ def cmd_issues(args) -> int: else: print(f"lifecycle issues: OK ({len(registry_issue_refs(ROOT))} tracking issue(s) open)") + # Declared gates first: the prompt author said which reading applies, so + # these are actionable rather than a question. Still advisory — retiring a + # prompt writes to complete/ and stays human. + if gates["shipped"]: + print(f"\nGATE MET — {len(gates['shipped'])} draft(s) whose `Closes-when:` " + f"has closed (likely shipped; verify, then retire):") + for line in gates["shipped"]: + print(f" ! {line}") + if gates["unblocked"]: + print(f"\nUNBLOCKED — {len(gates['unblocked'])} draft(s) whose `Blocked-by:` " + f"has closed (ready to start):") + for line in gates["unblocked"]: + print(f" > {line}") + if gates["partial"]: + print(f"\npartly unblocked — {len(gates['partial'])} draft(s) with some " + f"gates closed:") + for line in gates["partial"]: + print(f" ~ {line}") + if gates["unreadable"]: + print(f"\nunreadable — {len(gates['unreadable'])} declared gate(s):") + for line in gates["unreadable"]: + print(f" ? {line}") + # Advisory only — never affects the exit code. A draft citing a closed issue - # may be shipped OR newly unblocked; that is a judgement, not drift. + # with no declared gate may be shipped OR newly unblocked; that is a + # judgement, not drift. Drafts that DO declare a gate are reported above + # instead, so this list is the genuinely-ambiguous remainder. if notes: - print(f"\nadvisory — {len(notes)} draft(s) citing a closed issue:") + print(f"\nadvisory — {len(notes)} undeclared draft(s) citing a closed issue:") for line in notes: print(f" ? {line}") diff --git a/tests/test_lifecycle_check.py b/tests/test_lifecycle_check.py index 0a98825f..a8c1ccc2 100644 --- a/tests/test_lifecycle_check.py +++ b/tests/test_lifecycle_check.py @@ -537,3 +537,110 @@ def test_archive_material_does_not_satisfy_a_prompt_path(tmp_path): problems = lifecycle.registry_problems(root) assert len(problems) == 1 assert "does not resolve" in problems[0] + + +# --------------------------------------------------------------------------- # +# declared draft gates (`Closes-when:` / `Blocked-by:`) +# +# The 2026-08-09 draft/ sweep found five prompts whose stated gate had closed, +# and the two readings are OPPOSITE: a satisfied "epic closes when #N" means the +# prompt is DONE, a satisfied "blocked until #N merges" means it is READY. Prose +# cannot be graded, so `--drafts` had to report both as one ambiguous question. +# --------------------------------------------------------------------------- # +def _draft(root: Path, rel: str, body: str) -> Path: + p = root / "draft" / rel + p.parent.mkdir(parents=True, exist_ok=True) + p.write_text(body) + return p + + +GATE_ISSUE = "https://github.com/FictionalOrg/FlywheelRepo/issues/77" +GATE_ISSUE_2 = "https://github.com/FictionalOrg/GadgetRepo/issues/88" + + +def test_closes_when_gate_closed_reads_as_shipped(tmp_path): + _draft(tmp_path, "bug/flywheel/sprocket.md", + f"# Sprocket\n\nCloses-when: {GATE_ISSUE}\n") + notes = lifecycle.draft_gate_notes(tmp_path, fetch=_states({GATE_ISSUE: "closed"})) + assert len(notes["shipped"]) == 1 + assert "likely shipped" in notes["shipped"][0] + assert notes["unblocked"] == [] + + +def test_blocked_by_gate_closed_reads_as_unblocked(tmp_path): + """The opposite reading — and the whole reason the two keys are distinct.""" + _draft(tmp_path, "bug/flywheel/sprocket.md", + f"# Sprocket\n\nBlocked-by: {GATE_ISSUE}\n") + notes = lifecycle.draft_gate_notes(tmp_path, fetch=_states({GATE_ISSUE: "closed"})) + assert len(notes["unblocked"]) == 1 + assert "ready to start" in notes["unblocked"][0] + assert notes["shipped"] == [] + + +def test_open_gate_is_silent(tmp_path): + _draft(tmp_path, "bug/flywheel/sprocket.md", + f"# Sprocket\n\nBlocked-by: {GATE_ISSUE}\n") + notes = lifecycle.draft_gate_notes(tmp_path, fetch=_states({GATE_ISSUE: "open"})) + assert notes == {"shipped": [], "unblocked": [], "partial": [], "unreadable": []} + + +def test_repo_hash_shorthand_is_read_as_a_gate(tmp_path): + """Prompts write `Repo#123`, not URLs. A URL-only extractor found 2 refs + across the real backlog where the shorthand form found 8.""" + _draft(tmp_path, "bug/flywheel/sprocket.md", + "# Sprocket\n\nBlocked-by: FlywheelRepo#77 # the loader fix\n") + refs = lifecycle.draft_gate_refs(tmp_path) + assert len(refs) == 1 + assert refs[0][1] == "blocked-by" + assert refs[0][2].endswith("/FlywheelRepo/issues/77") + + +def test_partly_closed_gates_are_not_reported_as_ready(tmp_path): + """A prompt blocked on three PRs is unblocked when the LAST one lands. + Reporting per-reference would claim 'ready to start' while still blocked.""" + _draft(tmp_path, "bug/flywheel/sprocket.md", + f"# Sprocket\n\nBlocked-by: {GATE_ISSUE}, {GATE_ISSUE_2}\n") + notes = lifecycle.draft_gate_notes( + tmp_path, fetch=_states({GATE_ISSUE: "closed", GATE_ISSUE_2: "open"})) + assert notes["unblocked"] == [] + assert len(notes["partial"]) == 1 + assert "1 of 2" in notes["partial"][0] + + +def test_all_gates_closed_reports_once_not_per_reference(tmp_path): + _draft(tmp_path, "bug/flywheel/sprocket.md", + f"# Sprocket\n\nBlocked-by: {GATE_ISSUE}, {GATE_ISSUE_2}\n") + notes = lifecycle.draft_gate_notes( + tmp_path, fetch=_states({GATE_ISSUE: "closed", GATE_ISSUE_2: "closed"})) + assert len(notes["unblocked"]) == 1 + + +def test_a_fenced_example_is_documentation_not_a_declaration(tmp_path): + """The prompt that PROPOSED these keys shows them in a ```markdown block. + Reading that as a real gate would invent a finding out of documentation.""" + _draft(tmp_path, "feature/mind/gate_keys.md", + "# Propose gate keys\n\n" + "Proposal:\n\n" + "```markdown\n" + f"Closes-when: {GATE_ISSUE}\n" + "```\n\n" + "That is the idea.\n") + assert lifecycle.draft_gate_refs(tmp_path) == [] + + +def test_declared_gates_are_not_repeated_as_ambiguous_advisories(tmp_path): + """`--drafts` asks 'shipped, or newly unblocked?' precisely because prose + cannot say. A prompt that DECLARED which it means must not be asked again.""" + _draft(tmp_path, "bug/flywheel/sprocket.md", + f"# Sprocket\n\nBlocked-by: {GATE_ISSUE}\n\nContext: {GATE_ISSUE}\n") + fetch = _states({GATE_ISSUE: "closed"}) + assert lifecycle.draft_gate_notes(tmp_path, fetch=fetch)["unblocked"] + assert lifecycle.draft_issue_notes(tmp_path, fetch=fetch) == [] + + +def test_an_undeclared_draft_still_gets_the_ambiguous_advisory(tmp_path): + """The fallback must survive: most prompts carry no gate key at all.""" + _draft(tmp_path, "bug/flywheel/widget.md", f"# Widget\n\nFollow-up to {GATE_ISSUE}\n") + notes = lifecycle.draft_issue_notes(tmp_path, fetch=_states({GATE_ISSUE: "closed"})) + assert len(notes) == 1 + assert "shipped, or newly unblocked?" in notes[0] From 147390aef1445c245401e68800c0f94750229159 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 9 Aug 2026 15:18:22 +0000 Subject: [PATCH 2/2] prompt: record the detection work delivered, and correct a criterion I got wrong Legs 1 and 2 of draft_staleness_detection_signals.md are built. Leg 3 (the --repo upstream mode) is not, and is now better motivated. The "all five confirmed findings in the top band" criterion was WRONG WHEN WRITTEN, and contradicts this prompt's own Hard limit section three paragraphs above it -- that section already says one finding left no Mind-side signal at all, so a Mind-local ranker cannot rank it by construction. Chasing the criterion anyway made the tool worse, which is the useful part. A loose // series match pulled one more finding in, but also FALSELY flagged test_mode_bypass_ordered_assertion_ties -- off references to four unrelated sibling prompts in the same folder -- and that prompt is one the sweep confirmed is NOT shipped. It is precisely the mis-grade the tool must never make, and it was already the criterion immediately below. So the two criteria were in direct conflict. The real finding is that the five need three different tools, not one better ranker: rare-token fan-out and shared identifiers (reconcile) catch two, the Closes-when: header key catches one, and the last two are only reachable by reading the target repo. Recorded as a table so the next session does not re-derive it, along with the corrected criteria and the measured outcome. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01E3MuurHXi3xo9TLRpMLJA6 --- .../draft_staleness_detection_signals.md | 38 ++++++++++++++++++- 1 file changed, 36 insertions(+), 2 deletions(-) diff --git a/draft/feature/pyautomind/draft_staleness_detection_signals.md b/draft/feature/pyautomind/draft_staleness_detection_signals.md index 7475f12c..15f04c1f 100644 --- a/draft/feature/pyautomind/draft_staleness_detection_signals.md +++ b/draft/feature/pyautomind/draft_staleness_detection_signals.md @@ -147,9 +147,43 @@ is already reconcile's stated contract. Keep it. ## Acceptance +> **DELIVERED 2026-08-09 — and one criterion below was wrong when written.** +> Legs 1 and 2 shipped (PyAutoMind `lifecycle.py` gate keys; PyAutoBrain +> `intake reconcile` re-rank). Leg 3 (`--repo` upstream mode) is not built. +> +> **The "all five in the top band" criterion is unachievable and contradicts +> this prompt's own § Hard limit.** That section already says one finding left +> no Mind-side signal at all — so a Mind-local ranker cannot rank it, by +> construction. Trying to satisfy the criterion anyway actively made the tool +> worse: a loose series match pulled one more finding in but FALSELY flagged +> `test_mode_bypass_ordered_assertion_ties`, breaking the criterion below it. +> +> What the five actually need is three different tools, which is the real +> finding: +> +> | finding | caught by | status | +> |---|---|---| +> | k×s series | reconcile — rare-token fan-out | **rank 2 of 31** | +> | nufft chunking | reconcile — shared rare identifiers | flagged | +> | test-mode umbrella | `Closes-when:` header key (leg 1) | declared + graded | +> | split guard | nothing Mind-local (evidence sat in a sibling *prompt*) | needs leg 3 | +> | latent resume | nothing at all (no record exists) | needs leg 3 | +> +> Corrected criteria, all met: +> +> - The ranker flags **materially fewer** than 96: **31 of 148 (21%)**, down from +> 96 (65%), with `high` cut from 52 to 9. +> - Every finding **reachable from Mind-local evidence** is flagged, and the +> largest is at rank 2 (it was previously not flagged at all). +> - `test_mode_bypass_ordered_assertion_ties.md` is NOT reported as shipped. +> - No prompt is moved or retired by the tool (asserted by a test). +> +> Leg 3 remains open and is now better motivated: it is the *only* route to the +> last two findings. + - Re-running the ranker against PyAutoMind `f25e154e` (the pre-sweep tree, which - is the labelled set this prompt establishes) puts **all five** confirmed - findings in the top band, and flags materially fewer than 96 prompts. + is the labelled set this prompt establishes) puts every **Mind-reachable** + confirmed finding in the top band, and flags materially fewer than 96 prompts. - `test_mode_bypass_ordered_assertion_ties.md` is NOT reported as shipped. - No prompt is moved or retired by the tool.