Skip to content

process(skills): paraphrase first, not-run auditors, second-pass labels - #2746

Merged
antonsynd merged 9 commits into
devfrom
claude/verify-pass-definitions
Nov 19, 2026
Merged

antonsynd merged 9 commits into
devfrom
claude/verify-pass-definitions

Conversation

@antonsynd

@antonsynd antonsynd commented Nov 19, 2026 •

Copy link
Copy Markdown
Owner

Three small additions to the verify skills. The ideas come from reading google/elephant-goldfish-model (fresh-context document review); the wording is ours and nothing is copied from it. The contract, CLAUDE.md and docs/process/ are unchanged.

1. /verify-plan paraphrases the plan before grading (c1ad899, ee83e2e)

Before any probe or grade, from the plan's body alone (never a prior Verification Summary), the verifier writes a few sentences on what the plan changes and how the code works today by the plan's account, into a new ### Paraphrase subsection of the Verification Summary. A section it cannot paraphrase is a Dimension 4 error: NEEDS REVISION, never corrected inline, listed in the subsection's Could not paraphrase slot. A nine-dimension pass run by the plan's own session writes not written: author's session instead: a paraphrase by the author proves nothing. The paraphrase leads the verifier's summary to the user, and /plan-flow relays it after each stage.

Why. A claim-by-claim refutation can pass a plan whose references are all accurate but whose parts do not connect.

2. /verify-implementation: not run is not nothing found (5985851, a7c791b)

An auditor the tier spawns (Agents 1–4) that times out, errors or returns no verdict has not audited. Its worktrees are removed and it is re-spawned once; failing again it is not run: its categories read not audited, never zero findings, and the round skips to Cleanup steps 1, 2 and 6 (worktrees, agents returned, the report), with no remediation, record, merge or issue close. The report template gains an **Auditors:** status line. Agent 1 at light tier stays not spawned, which is the tier, not a failure; an optional Agent 5 that fails is recorded on that line and does not stop the round.

Why. The skill said to wait for every auditor but not what an auditor that never reports means; contract §12's Check already calls a skipped execution auditor red.

3. /verify-plan labels second-pass findings (63a9596, b37ae42)

When the ## Revision log shows a prior pass, the verifier grades fresh and then labels each finding against that pass's commit:

  • settled: an owner ruling decides it; not raised. Nothing else is dropped.
  • declined: the Revision log declined it and it is still drawn; raised with the log's reason quoted.
  • re-opened: the log says the revision answered it and it is still drawn.
  • churn: its cure would undo the revision's answer to a prior finding. It is also listed under ### Unchecked Claims, with both texts, as needing the owner's word, which is what /create-plan revise step 3 and /plan-flow's owner-only batch already ask about before any rewrite.
  • new: the rest.

The labels change no stamp. /plan-flow › The pass cap carries the label when it brings a second NEEDS REVISION to the owner; it still adds no rule to a stage.

Why. Two fresh readers sample differently. At the two-pass cap the owner needs to know whether the second pass found the same defect still there, a new one, or a disagreement between two readers.

Owner's words (2026-11-19, in this session)

  • "do 1, 2, and 3 (3 = definitions to sharpen the second pass) and link integrity for skills. use opus subagent(s), then raise PR(s) with /code-review after. merge on green CI"

The link-integrity check is #2747.

Read set: +1853 bytes

/verify-plan +1273, /verify-implementation +490, /plan-flow +90.

  • Baseline raise (51621b1, then 71f4ce8): procedure_bytes 143707 → 145560 (+1853). The goals document's stop rule (docs/process/handoff-2026-11-17-arc.md §4) puts any rise to the owner. Asked how the three additions should land, with the options "Raise by exact delta", "Net zero" (an equal cut elsewhere in the counted skills) and "Definitions in the contract", and an estimate of about +1.5 KB, the owner answered: "Raise by exact delta (Recommended)". The measured delta is 353 bytes over that estimate: +1796 as first written, +57 more from the review fixes.

Measured

Independent review

Independent review @ 51621b1: merge after fixes; 10 applied, 0 declined (notes below); fixes 51621b1..71f4ce8 read by the lead

/code-review (high) ran on this pull request at 51621b1 in a forked context that had the diff, not this session's reasoning. It returns findings, not a verdict; "merge after fixes" is the lead's reading of six findings marked important. The fix range changes eight lines of skill text and one baseline number, adds no guard or abstraction, and so had no second review.

Applied:

  1. settled contradicted "labels change no stamp", and let a reviser waive a finding by declining it. Settled is now owner-ruled only; a declined finding is raised with the log's reason.
  2. re-opened "at the same text" could never apply, since a revision rewrites the text. Defined by the defect: answered per the log, still drawn.
  3. churn caught plain errors the revision introduced, and "never a revision on your word alone" contradicted the routing. Churn is now a cure that would undo the revision's answer to a prior finding; a plain error is new; the routing is the existing Unchecked Claims path. One residue: a churn finding that is also deferrable under a CONVERGED stamp reaches the owner through /plan-flow's batch, but a standalone /implement-plan reads CONVERGED as PASS.
  4. re-spawn collided with the failed auditor's fixed-name worktree. Its worktrees are removed first.
  5. "counts as reported" let remediation start on a partial picture and dead-end at Final Verification 7. The round now skips to Cleanup steps 1, 2 and 6.
  6. an unparaphrasable section had no severity. It is an error: NEEDS REVISION, never corrected inline.
  7. optional Agent 5 failing twice blocked a round that never needed it. The stop is scoped to Agents 1–4. One residue: nothing says Agent 5's category reads not audited when it fails.
  8. "else → 4" contradicted "the dimension grading it" and had no home in an --adequacy summary. One dimension (4), listed in ### Paraphrase itself.
  9. the paraphrase could be anchored on a prior pass's paraphrase, or go stale after inline corrections. Body only; the template says "as read, before corrections".
  10. nothing delivered the paraphrase to the author or owner. It leads the verifier's user summary and /plan-flow's after-stage relay.

🤖 Generated with Claude Code

https://claude.ai/code/session_01LGMKZWuQhE7ZvVZDE3FdDs

A fresh reader checks claims one at a time and can pass a plan whose
references are all accurate but whose parts do not connect. Before any
probe or grade, the verifier writes from the plan text alone what the
plan changes and how the code works today by its account, into a new
### Paraphrase subsection of the Verification Summary (also listed for
--adequacy). A section it cannot paraphrase is a finding on the
dimension that grades it (Defect Class 6, guards 7, otherwise 4), so
the Convergence rule still has a dimension. A nine-dimension pass run
by the plan's own session writes "not written: author's session".

Co-Authored-By: Claude Fable 5.2 <[email protected]>
Claude-Session: https://claude.ai/code/session_01LGMKZWuQhE7ZvVZDE3FdDs
The skill waited for every auditor but said nothing of one that times
out, errors or returns no verdict, so its categories could read as
none found. Such an auditor has not audited: it is re-spawned once,
and if it fails again it is "not run". Its categories read "not
audited", never zero findings, and the round is incomplete: no
verification record (Cleanup step 3), no merge and no issue closed
(step 5), consistent with contract §12's Check that an execution
auditor skipped at any tier is red. Agent 1 at light tier is "not
spawned", not "not run". The report header gains an Auditors line.

Co-Authored-By: Claude Fable 5.2 <[email protected]>
Claude-Session: https://claude.ai/code/session_01LGMKZWuQhE7ZvVZDE3FdDs
/plan-flow caps a plan at two verify passes and takes the second
NEEDS REVISION to the owner as a decision; fresh readers sample
differently, so the owner needs to know what kind of finding a second
pass raised. When the Revision log shows a prior pass, the verifier
grades fresh, then labels each finding in the Verification Summary:
settled (owner-ruled, never raised; declined with a reason, raised
only with what refuted the reason), re-opened (the revision changed a
decision to answer it and the defect stands), churn (against rewritten
text the prior pass's commit would not have drawn: an owner decision
with both texts) or new. The labels change no stamp; an unraised
settled finding is not a finding. /plan-flow's pass cap carries each
open finding's label to the owner, churn with both texts.

Co-Authored-By: Claude Fable 5.2 <[email protected]>
Claude-Session: https://claude.ai/code/session_01LGMKZWuQhE7ZvVZDE3FdDs
The three skill commits on this branch add 1796 procedure bytes:
/verify-plan +1271 (paraphrase first, +639; second-pass finding
labels, +632), /verify-implementation +463 (an auditor not run is not
nothing found) and /plan-flow +62 (the pass cap carries each open
finding's label).

The lead asked the owner on 2026-11-19 how the three additions should
land given the ratchet, offering "Raise by exact delta", "Net zero" and
"Definitions in the contract"; the owner answered "Raise by exact delta
(Recommended)".

Co-Authored-By: Claude Fable 5.2 <[email protected]>
Claude-Session: https://claude.ai/code/session_01LGMKZWuQhE7ZvVZDE3FdDs
Review fixes to the paraphrase step. It reads the plan's body alone,
never a prior Verification Summary, so a second pass cannot anchor on
the first pass's paraphrase. A section it cannot paraphrase is a
Dimension 4 error (NEEDS REVISION, never corrected inline) instead of
a per-section mapping whose "else" had no home in an --adequacy
summary. The template records the body as read, before corrections,
and the sections that could not be paraphrased. The summary to the
user (Output §3) and /plan-flow's relay after each stage carry it.

Co-Authored-By: Claude Fable 5.2 <[email protected]>
Claude-Session: https://claude.ai/code/session_01LGMKZWuQhE7ZvVZDE3FdDs
Review fixes to the second-pass labels. Only an owner ruling settles a
finding; a finding the Revision log declined is raised as declined,
quoting the log's reason, so a revision cannot waive it by declining
it. re-opened is defined by the defect still being drawn after the log
says it was answered, not by unchanged text. churn is a finding whose
cure would undo the revision's answer to a prior finding; it is also
listed under Unchecked Claims with both texts as needing the owner's
word, which /create-plan revise step 3 and /plan-flow's owner-only
batch already ask before any rewrite. A plain error the revision
introduced is new. Labels change no stamp, without exception.

Co-Authored-By: Claude Fable 5.2 <[email protected]>
Claude-Session: https://claude.ai/code/session_01LGMKZWuQhE7ZvVZDE3FdDs
Review fixes to the not-run rule. It covers the auditors the tier
spawns (Agents 1-4); optional Agent 5 not run is recorded on the
Auditors line and stops nothing. Before the single re-spawn the lead
removes the worktrees the failed auditor's brief names (each brief
creates a fixed-name one). An auditor not run after it stops the round
at collection: only Cleanup steps 1, 2 and 6 run, so no remediation
starts on a partial picture and no record, merge or issue close
follows. "It counts as reported" is gone.

Co-Authored-By: Claude Fable 5.2 <[email protected]>
Claude-Session: https://claude.ai/code/session_01LGMKZWuQhE7ZvVZDE3FdDs
Shorter wording, same rule: /verify-plan's re-opened label and
/verify-implementation's stop on an auditor not run (29 bytes).

Co-Authored-By: Claude Fable 5.2 <[email protected]>
Claude-Session: https://claude.ai/code/session_01LGMKZWuQhE7ZvVZDE3FdDs
The review fixes on this branch add 57 procedure bytes over the
previous raise (145503 -> 145560). Over origin/dev the three additions
now add 1853: /verify-plan +1273, /verify-implementation +490,
/plan-flow +90 (baseline 143707 -> 145560).

The lead asked the owner on 2026-11-19 how the three additions should
land given the ratchet, offering "Raise by exact delta", "Net zero" and
"Definitions in the contract"; the owner answered "Raise by exact delta
(Recommended)".

Co-Authored-By: Claude Fable 5.2 <[email protected]>
Claude-Session: https://claude.ai/code/session_01LGMKZWuQhE7ZvVZDE3FdDs
@antonsynd
antonsynd merged commit 9941200 into dev Nov 19, 2026
22 of 23 checks passed
@antonsynd
antonsynd deleted the claude/verify-pass-definitions branch November 19, 2026 04:32
antonsynd pushed a commit that referenced this pull request Nov 19, 2026
Brings in #2746 (skill text) so the reference check runs against it.

Co-Authored-By: Claude Fable 5.2 <[email protected]>
Claude-Session: https://claude.ai/code/session_01LGMKZWuQhE7ZvVZDE3FdDs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants