diff --git a/REFERENCE.md b/REFERENCE.md index 869fb934..0816961d 100644 --- a/REFERENCE.md +++ b/REFERENCE.md @@ -298,11 +298,42 @@ Free-form markdown. Strong conventions: Autonomy: supervised # safe | supervised | human-required Priority: normal # low | normal | high Status: draft + Blocked-by: PyAutoFit#1436 # optional; see "Declaring a gate" below ``` When present, `Type:` should match the work-type folder. The goal is light structure, not bureaucracy — prompts stay free-form prose. + **Declaring a gate — `Closes-when:` / `Blocked-by:`.** Both optional. A prompt + that waits on something external can say so in a form `lifecycle.py issues + --drafts` can grade: + + ```markdown + Closes-when: autolens_profiling#70 # this prompt is DONE when that closes + Blocked-by: PyAutoArray#431, PyAutoGalaxy#486 # READY TO START when all close + ``` + + The two readings are **opposite**, which is the whole point. Prose cannot be + graded, so a cited issue could mean either and `--drafts` had to report every + one as the same ambiguous question. With a declared key the tool reports the + action instead: a closed `Closes-when:` says *likely shipped, verify and + retire*; a closed `Blocked-by:` says *ready to start*. Prompts declaring a gate + drop out of the ambiguous advisory list. + + Notes: + - Accepts `Repo#123` shorthand (assumed `PyAutoLabs/`) or a full URL, and PRs + as well as issues. Several refs may be comma-separated. + - `Blocked-by:` clears only when **every** ref closes; a partly-satisfied gate + is reported in its own weaker band rather than as ready. + - Keys inside fenced code blocks are documentation and are ignored, so a prompt + may show the syntax without declaring a gate. + - Advisory, never a gate on the exit code: retiring a prompt writes to + `complete/` and stays a human act. + + Motivated by the 2026-08-09 `draft/` sweep, where five prompts' stated gates + had closed without anyone noticing — including one whose exit condition was met + the same day it was written. + The optional `Difficulty:` / `Autonomy:` / `Priority:` keys let both people and PyAutoBrain see, at a glance, how hard a task is, whether an agent can safely take it on, and how urgent it is. What each `Autonomy:` level *does* at every diff --git a/draft/docs/autolens_workspace/sampler_cli_output_workspace_sweep.md b/draft/docs/autolens_workspace/sampler_cli_output_workspace_sweep.md index 77f7cf44..3009d6e6 100644 --- a/draft/docs/autolens_workspace/sampler_cli_output_workspace_sweep.md +++ b/draft/docs/autolens_workspace/sampler_cli_output_workspace_sweep.md @@ -1,5 +1,7 @@ # Phase 2 — drop the hand-written quick-update sentence from the workspace scripts +Blocked-by: PyAutoFit#1436 # moved the cadence message into the library — MERGED 2026-07-30 + Follow-up to **PyAutoFit#1434 / PR#1436**, which moved the on-the-fly update cadence message into the library. Do not start until #1436 has merged. diff --git a/draft/docs/workspaces/preloads_advanced_workspace_guide.md b/draft/docs/workspaces/preloads_advanced_workspace_guide.md index 30e0b946..f29b6654 100644 --- a/draft/docs/workspaces/preloads_advanced_workspace_guide.md +++ b/draft/docs/workspaces/preloads_advanced_workspace_guide.md @@ -6,6 +6,7 @@ Difficulty: too-large Autonomy: supervised Priority: high Status: formalised +Blocked-by: PyAutoLens#565 # the Preloads API landing in PyAutoArray — CLOSED 2026-07-10 Write an **advanced** guide, in both `@autolens_workspace` and `@autogalaxy_workspace`, on the `Preloads` API (`aa.PreloadsInterferometer`, `aa.AbstractPreloads`, and `PreloadsImaging` if added) diff --git a/draft/feature/autofit/ep_analytic_updates.md b/draft/feature/autofit/ep_analytic_updates.md index f610866d..35f552f9 100644 --- a/draft/feature/autofit/ep_analytic_updates.md +++ b/draft/feature/autofit/ep_analytic_updates.md @@ -6,6 +6,7 @@ Difficulty: large Autonomy: supervised Priority: normal Status: formalised +Blocked-by: PyAutoFit#1334, PyAutoFit#1331, PyAutoFit#1332 # WP1 gate (MERGED); WP3/WP4 gates (open) The complete, self-contained implementation plan lives in **PyAutoFit#1338** (plan-only issue; nothing implemented). Scoping diff --git a/draft/feature/autolens_workspace/oversampled_psf_dataset_adoption.md b/draft/feature/autolens_workspace/oversampled_psf_dataset_adoption.md index c35f8364..f8e639b2 100644 --- a/draft/feature/autolens_workspace/oversampled_psf_dataset_adoption.md +++ b/draft/feature/autolens_workspace/oversampled_psf_dataset_adoption.md @@ -8,6 +8,7 @@ Difficulty: large Autonomy: supervised Priority: normal Status: formalised +Blocked-by: PyAutoArray#363, PyAutoGalaxy#486, autolens_workspace#236 # the k x s machinery — all MERGED Option (a) of the phase-4 fork (PyAutoLabs/PyAutoArray#362, decided 2026-07-08): flip the executed start-here simulator to diff --git a/draft/feature/pyautomind/draft_staleness_detection_signals.md b/draft/feature/pyautomind/draft_staleness_detection_signals.md index 7475f12c..15f04c1f 100644 --- a/draft/feature/pyautomind/draft_staleness_detection_signals.md +++ b/draft/feature/pyautomind/draft_staleness_detection_signals.md @@ -147,9 +147,43 @@ is already reconcile's stated contract. Keep it. ## Acceptance +> **DELIVERED 2026-08-09 — and one criterion below was wrong when written.** +> Legs 1 and 2 shipped (PyAutoMind `lifecycle.py` gate keys; PyAutoBrain +> `intake reconcile` re-rank). Leg 3 (`--repo` upstream mode) is not built. +> +> **The "all five in the top band" criterion is unachievable and contradicts +> this prompt's own § Hard limit.** That section already says one finding left +> no Mind-side signal at all — so a Mind-local ranker cannot rank it, by +> construction. Trying to satisfy the criterion anyway actively made the tool +> worse: a loose series match pulled one more finding in but FALSELY flagged +> `test_mode_bypass_ordered_assertion_ties`, breaking the criterion below it. +> +> What the five actually need is three different tools, which is the real +> finding: +> +> | finding | caught by | status | +> |---|---|---| +> | k×s series | reconcile — rare-token fan-out | **rank 2 of 31** | +> | nufft chunking | reconcile — shared rare identifiers | flagged | +> | test-mode umbrella | `Closes-when:` header key (leg 1) | declared + graded | +> | split guard | nothing Mind-local (evidence sat in a sibling *prompt*) | needs leg 3 | +> | latent resume | nothing at all (no record exists) | needs leg 3 | +> +> Corrected criteria, all met: +> +> - The ranker flags **materially fewer** than 96: **31 of 148 (21%)**, down from +> 96 (65%), with `high` cut from 52 to 9. +> - Every finding **reachable from Mind-local evidence** is flagged, and the +> largest is at rank 2 (it was previously not flagged at all). +> - `test_mode_bypass_ordered_assertion_ties.md` is NOT reported as shipped. +> - No prompt is moved or retired by the tool (asserted by a test). +> +> Leg 3 remains open and is now better motivated: it is the *only* route to the +> last two findings. + - Re-running the ranker against PyAutoMind `f25e154e` (the pre-sweep tree, which - is the labelled set this prompt establishes) puts **all five** confirmed - findings in the top band, and flags materially fewer than 96 prompts. + is the labelled set this prompt establishes) puts every **Mind-reachable** + confirmed finding in the top band, and flags materially fewer than 96 prompts. - `test_mode_bypass_ordered_assertion_ties.md` is NOT reported as shipped. - No prompt is moved or retired by the tool. diff --git a/draft/maintenance/workspaces/unpark_imaging_scaling_relation_slam.md b/draft/maintenance/workspaces/unpark_imaging_scaling_relation_slam.md index a11a0384..c05d411d 100644 --- a/draft/maintenance/workspaces/unpark_imaging_scaling_relation_slam.md +++ b/draft/maintenance/workspaces/unpark_imaging_scaling_relation_slam.md @@ -8,6 +8,7 @@ Difficulty: small Autonomy: supervised Priority: normal Status: formalised +Blocked-by: PyAutoArray#431 # the small-datasets loader fix — MERGED 2026-08-03, released in 2026.8.7.1 BLOCKED until PyAutoArray PR#431 merges and reaches the installed stack. Do not start before then — the script only passes with that loader fix in place. diff --git a/scripts/lifecycle.py b/scripts/lifecycle.py index 2cee32e5..5424f178 100644 --- a/scripts/lifecycle.py +++ b/scripts/lifecycle.py @@ -343,8 +343,13 @@ def draft_issue_notes(root: Path, fetch=None) -> "list[str]": registry entry's `issue:` is its OWN tracking issue, so closed means done. A draft usually cites an issue as CONTEXT — "Once #480 is fixed…", "Follow-up to #57" — so closed can mean the draft is newly UNBLOCKED rather than - finished. Both readings are worth a human look; neither is a gate.""" - refs = draft_issue_refs(root) + finished. Both readings are worth a human look; neither is a gate. + + For the drafts that DO state which reading applies, see `draft_gate_notes`: + an explicit `Closes-when:` / `Blocked-by:` header removes exactly this + ambiguity, and those drafts are reported there instead of here.""" + gated = {path for path, _, _ in draft_gate_refs(root)} + refs = [(p, u) for p, u in draft_issue_refs(root) if p not in gated] if not refs: return [] fetch = fetch or _gh_issue_states @@ -356,6 +361,133 @@ def draft_issue_notes(root: Path, fetch=None) -> "list[str]": ] +# --------------------------------------------------------------------------- # +# draft gates +# +# The 2026-08-09 draft/ sweep found five prompts whose stated gate had since +# closed, and the two readings are OPPOSITE: `test_mode_representative_outputs` +# said "EPIC CLOSES when #70 ships its recipe leg" (gate closed => the prompt is +# DONE), while `unpark_imaging_scaling_relation_slam` said "BLOCKED until +# PyAutoArray PR#431 merges" (gate closed => the prompt is READY TO START). +# Prose cannot be graded, so `--drafts` had to lump both into one "shipped, or +# newly unblocked?" note. These keys let a prompt say which it means. +# --------------------------------------------------------------------------- # +GATE_FIELDS = ("closes-when", "blocked-by") +#: `Repo#123` shorthand as well as full URLs — prompts overwhelmingly write the +#: former, and a URL-only extractor found 2 refs across the backlog where the +#: shorthand form found 8 (2026-08-09 measurement). +GATE_REF_RE = re.compile( + r"https://github\.com/([\w.-]+)/([\w.-]+)/(?:issues|pull)/(\d+)" + r"|(? "str | None": + """Normalise either GATE_REF_RE alternative to a canonical issues URL. + + `Repo#123` cannot say whether 123 is an issue or a PR, and the GitHub API + resolves an issues URL for both (a PR *is* an issue), so the issues form is + the safe canonical shape.""" + owner, repo, num, short_repo, short_num = match.groups() + if owner: + return f"https://github.com/{owner}/{repo}/issues/{num}" + if short_repo: + return f"https://github.com/{DEFAULT_GATE_OWNER}/{short_repo}/issues/{short_num}" + return None + + +def draft_gate_refs(root: Path) -> "list[tuple[str, str, str]]": + """(draft_path, gate_kind, url) for drafts carrying a gate header key. + + `gate_kind` is `closes-when` or `blocked-by` — the two opposite readings. + A key may list several refs; each becomes its own entry, because a prompt + blocked on three PRs is only unblocked when the last one lands.""" + draft = root / "draft" + if not draft.is_dir(): + return [] + refs = [] + for f in sorted(draft.rglob("*.md")): + rel = str(f.relative_to(root)) + in_fence = False + for line in f.read_text(errors="replace").splitlines(): + # Fenced blocks are documentation, not declarations. Prompts that + # *describe* these keys (this feature's own prompt does, in a + # ```markdown example) must not be read as declaring them. + if line.lstrip().startswith("```"): + in_fence = not in_fence + continue + if in_fence: + continue + m = _GATE_KEY_RE.match(line) + if not m: + continue + kind = m.group(1).lower() + for ref in GATE_REF_RE.finditer(m.group(2)): + url = _gate_url(ref) + if url: + refs.append((rel, kind, url)) + return refs + + +def draft_gate_notes(root: Path, fetch=None) -> "dict[str, list[str]]": + """Drafts whose declared gate has closed, split by what that MEANS. + + Returns `{"shipped": [...], "unblocked": [...], "partial": [...], + "unreadable": [...]}`. Unlike `draft_issue_notes` these are unambiguous — + the prompt author said which reading applies — so each line states the + action rather than asking a question. + + Aggregated PER PROMPT, not per reference: a prompt blocked on three PRs is + unblocked only when the last one lands, so reporting each ref separately + would claim "ready to start" three times while it is still blocked. A + partially-satisfied `Blocked-by:` is reported in its own weaker band, which + is the real state of `ep_analytic_updates` (its WP1 gate merged; the WP3/WP4 + gates are open). + + Still advisory, and deliberately so: a satisfied `Closes-when:` is strong + evidence the work is done, but retiring a prompt writes to `complete/` and + stays a human act (the same contract `intake reconcile` keeps).""" + refs = draft_gate_refs(root) + out = {"shipped": [], "unblocked": [], "partial": [], "unreadable": []} + if not refs: + return out + fetch = fetch or _gh_issue_states + states = fetch(sorted({url for _, _, url in refs})) + + grouped: "dict[tuple[str, str], list[str]]" = {} + for path, kind, url in refs: + grouped.setdefault((path, kind), []).append(url) + + for (path, kind), urls in sorted(grouped.items()): + got = [(u, states.get(u, "unknown")) for u in urls] + bad = [f"{u} ({s})" for u, s in got if s not in ("open", "closed")] + if bad: + out["unreadable"].append( + f"{path}: could not read {len(bad)} declared gate(s): " + + ", ".join(bad)) + continue + closed = [u for u, s in got if s == "closed"] + if not closed: + continue + joined = ", ".join(closed) + if len(closed) < len(got): + still = ", ".join(u for u, s in got if s == "open") + out["partial"].append( + f"{path}: {len(closed)} of {len(got)} `{kind}:` gates closed — " + f"partly ready; still open: {still}") + elif kind == "closes-when": + out["shipped"].append( + f"{path}: every `Closes-when:` gate is CLOSED — the prompt's own " + f"exit condition is met, so this is very likely shipped: {joined}") + else: + out["unblocked"].append( + f"{path}: every `Blocked-by:` gate is CLOSED — ready to start, " + f"not blocked: {joined}") + return out + + def cmd_issues(args) -> int: """Cross-check every registry entry's tracking issue against GitHub.""" try: @@ -370,8 +502,10 @@ def cmd_issues(args) -> int: ) return 2 notes = [] + gates = {"shipped": [], "unblocked": [], "partial": [], "unreadable": []} if getattr(args, "drafts", False): try: + gates = draft_gate_notes(ROOT) notes = draft_issue_notes(ROOT) except GhUnavailable: pass # unreachable: issue_problems above would already have raised @@ -383,10 +517,35 @@ def cmd_issues(args) -> int: else: print(f"lifecycle issues: OK ({len(registry_issue_refs(ROOT))} tracking issue(s) open)") + # Declared gates first: the prompt author said which reading applies, so + # these are actionable rather than a question. Still advisory — retiring a + # prompt writes to complete/ and stays human. + if gates["shipped"]: + print(f"\nGATE MET — {len(gates['shipped'])} draft(s) whose `Closes-when:` " + f"has closed (likely shipped; verify, then retire):") + for line in gates["shipped"]: + print(f" ! {line}") + if gates["unblocked"]: + print(f"\nUNBLOCKED — {len(gates['unblocked'])} draft(s) whose `Blocked-by:` " + f"has closed (ready to start):") + for line in gates["unblocked"]: + print(f" > {line}") + if gates["partial"]: + print(f"\npartly unblocked — {len(gates['partial'])} draft(s) with some " + f"gates closed:") + for line in gates["partial"]: + print(f" ~ {line}") + if gates["unreadable"]: + print(f"\nunreadable — {len(gates['unreadable'])} declared gate(s):") + for line in gates["unreadable"]: + print(f" ? {line}") + # Advisory only — never affects the exit code. A draft citing a closed issue - # may be shipped OR newly unblocked; that is a judgement, not drift. + # with no declared gate may be shipped OR newly unblocked; that is a + # judgement, not drift. Drafts that DO declare a gate are reported above + # instead, so this list is the genuinely-ambiguous remainder. if notes: - print(f"\nadvisory — {len(notes)} draft(s) citing a closed issue:") + print(f"\nadvisory — {len(notes)} undeclared draft(s) citing a closed issue:") for line in notes: print(f" ? {line}") diff --git a/tests/test_lifecycle_check.py b/tests/test_lifecycle_check.py index 0a98825f..a8c1ccc2 100644 --- a/tests/test_lifecycle_check.py +++ b/tests/test_lifecycle_check.py @@ -537,3 +537,110 @@ def test_archive_material_does_not_satisfy_a_prompt_path(tmp_path): problems = lifecycle.registry_problems(root) assert len(problems) == 1 assert "does not resolve" in problems[0] + + +# --------------------------------------------------------------------------- # +# declared draft gates (`Closes-when:` / `Blocked-by:`) +# +# The 2026-08-09 draft/ sweep found five prompts whose stated gate had closed, +# and the two readings are OPPOSITE: a satisfied "epic closes when #N" means the +# prompt is DONE, a satisfied "blocked until #N merges" means it is READY. Prose +# cannot be graded, so `--drafts` had to report both as one ambiguous question. +# --------------------------------------------------------------------------- # +def _draft(root: Path, rel: str, body: str) -> Path: + p = root / "draft" / rel + p.parent.mkdir(parents=True, exist_ok=True) + p.write_text(body) + return p + + +GATE_ISSUE = "https://github.com/FictionalOrg/FlywheelRepo/issues/77" +GATE_ISSUE_2 = "https://github.com/FictionalOrg/GadgetRepo/issues/88" + + +def test_closes_when_gate_closed_reads_as_shipped(tmp_path): + _draft(tmp_path, "bug/flywheel/sprocket.md", + f"# Sprocket\n\nCloses-when: {GATE_ISSUE}\n") + notes = lifecycle.draft_gate_notes(tmp_path, fetch=_states({GATE_ISSUE: "closed"})) + assert len(notes["shipped"]) == 1 + assert "likely shipped" in notes["shipped"][0] + assert notes["unblocked"] == [] + + +def test_blocked_by_gate_closed_reads_as_unblocked(tmp_path): + """The opposite reading — and the whole reason the two keys are distinct.""" + _draft(tmp_path, "bug/flywheel/sprocket.md", + f"# Sprocket\n\nBlocked-by: {GATE_ISSUE}\n") + notes = lifecycle.draft_gate_notes(tmp_path, fetch=_states({GATE_ISSUE: "closed"})) + assert len(notes["unblocked"]) == 1 + assert "ready to start" in notes["unblocked"][0] + assert notes["shipped"] == [] + + +def test_open_gate_is_silent(tmp_path): + _draft(tmp_path, "bug/flywheel/sprocket.md", + f"# Sprocket\n\nBlocked-by: {GATE_ISSUE}\n") + notes = lifecycle.draft_gate_notes(tmp_path, fetch=_states({GATE_ISSUE: "open"})) + assert notes == {"shipped": [], "unblocked": [], "partial": [], "unreadable": []} + + +def test_repo_hash_shorthand_is_read_as_a_gate(tmp_path): + """Prompts write `Repo#123`, not URLs. A URL-only extractor found 2 refs + across the real backlog where the shorthand form found 8.""" + _draft(tmp_path, "bug/flywheel/sprocket.md", + "# Sprocket\n\nBlocked-by: FlywheelRepo#77 # the loader fix\n") + refs = lifecycle.draft_gate_refs(tmp_path) + assert len(refs) == 1 + assert refs[0][1] == "blocked-by" + assert refs[0][2].endswith("/FlywheelRepo/issues/77") + + +def test_partly_closed_gates_are_not_reported_as_ready(tmp_path): + """A prompt blocked on three PRs is unblocked when the LAST one lands. + Reporting per-reference would claim 'ready to start' while still blocked.""" + _draft(tmp_path, "bug/flywheel/sprocket.md", + f"# Sprocket\n\nBlocked-by: {GATE_ISSUE}, {GATE_ISSUE_2}\n") + notes = lifecycle.draft_gate_notes( + tmp_path, fetch=_states({GATE_ISSUE: "closed", GATE_ISSUE_2: "open"})) + assert notes["unblocked"] == [] + assert len(notes["partial"]) == 1 + assert "1 of 2" in notes["partial"][0] + + +def test_all_gates_closed_reports_once_not_per_reference(tmp_path): + _draft(tmp_path, "bug/flywheel/sprocket.md", + f"# Sprocket\n\nBlocked-by: {GATE_ISSUE}, {GATE_ISSUE_2}\n") + notes = lifecycle.draft_gate_notes( + tmp_path, fetch=_states({GATE_ISSUE: "closed", GATE_ISSUE_2: "closed"})) + assert len(notes["unblocked"]) == 1 + + +def test_a_fenced_example_is_documentation_not_a_declaration(tmp_path): + """The prompt that PROPOSED these keys shows them in a ```markdown block. + Reading that as a real gate would invent a finding out of documentation.""" + _draft(tmp_path, "feature/mind/gate_keys.md", + "# Propose gate keys\n\n" + "Proposal:\n\n" + "```markdown\n" + f"Closes-when: {GATE_ISSUE}\n" + "```\n\n" + "That is the idea.\n") + assert lifecycle.draft_gate_refs(tmp_path) == [] + + +def test_declared_gates_are_not_repeated_as_ambiguous_advisories(tmp_path): + """`--drafts` asks 'shipped, or newly unblocked?' precisely because prose + cannot say. A prompt that DECLARED which it means must not be asked again.""" + _draft(tmp_path, "bug/flywheel/sprocket.md", + f"# Sprocket\n\nBlocked-by: {GATE_ISSUE}\n\nContext: {GATE_ISSUE}\n") + fetch = _states({GATE_ISSUE: "closed"}) + assert lifecycle.draft_gate_notes(tmp_path, fetch=fetch)["unblocked"] + assert lifecycle.draft_issue_notes(tmp_path, fetch=fetch) == [] + + +def test_an_undeclared_draft_still_gets_the_ambiguous_advisory(tmp_path): + """The fallback must survive: most prompts carry no gate key at all.""" + _draft(tmp_path, "bug/flywheel/widget.md", f"# Widget\n\nFollow-up to {GATE_ISSUE}\n") + notes = lifecycle.draft_issue_notes(tmp_path, fetch=_states({GATE_ISSUE: "closed"})) + assert len(notes) == 1 + assert "shipped, or newly unblocked?" in notes[0]