Skip to content

feat(om-review-loop): review-and-fix with fresh-eyes subagents until it goes quiet - #49

Merged
jtomaszewski merged 1 commit into
mainfrom
jtomaszewski/om-review-loop-skill
Aug 25, 2026
Merged

feat(om-review-loop): review-and-fix with fresh-eyes subagents until it goes quiet#49
jtomaszewski merged 1 commit into
mainfrom
jtomaszewski/om-review-loop-skill

Conversation

@jtomaszewski

Copy link
Copy Markdown
Contributor

Adds skills/om-review-loop/SKILL.md — an Open Mercato review-and-fix loop that keeps running om-code-review until N consecutive rounds turn up nothing new.

Why

om-auto-review-pr --autofix already loops, in-process, and posts the result. The context that wrote a fix then re-reviews it against its own intent: excellent at catching self-contradiction, structurally blind to what it never considered. This skill is the subagent, fresh-eyes version of that loop — every round is read by contexts that have never heard the argument for the code — and it is local only.

The skill states the three axes where the two differ rather than restating the pipeline: reviewer context (new subagents vs. one iterating context), exit condition (N consecutive quiet rounds + the full gate vs. one actionable-empty pass + CI), and side effects (none vs. reviews, labels, claims, handoff).

The design points it pins down

  • Reviewers never learn the loop exists. No round number, no ledger, no fix rationale — "round 5, nothing since round 3" is an instruction to find nothing. Prompts are identical every round except the diff; dedupe happens orchestrator-side, after the review returns, so it can never leak in as a hint.
  • An append-only ledger, rejected findings included. Drop them and each round re-raises them from a reviewer who never heard the refutation, the counter never advances, and the loop runs until the budget dies. Fingerprints key on file + enclosing symbol, never line numbers — those move with the first commit, after which dedupe silently stops matching.
  • Verify before fixing. One adversarial refute-by-default verifier per new finding, accepting either a concrete failure path or a fact confirmed in the repo. Each finding costs a code change, and a code change on a wrong premise is a new defect. The refutation rate is reported, because converging by dismissal looks identical to converging by fixing.
  • The validation gate, split three ways. Codegen prefix once at the start; a per-round subset chosen from validation.commands and keeping every prerequisite of what it keeps (subset below the prerequisites and you invent failures the loop then "fixes"); the full gate in configured order exactly once, when the counter first reaches the threshold. A failure there is a blocker by om-code-review's own rule and resets the counter to zero — there is no finishing around it.
  • Judgement calls go up, not into the diff. Design disagreements, scope questions, out-of-diff findings — handed to the user, never decided silently and never dropped. "Quiet" means nothing new, not nothing left, and the report carries the still-open ones so the exit can't read as a clean bill.
  • om-code-review's severity scale and verdict rule verbatim. No second rubric.

Exit reports the findings-per-round curve, and hitting the round cap is reported as non-convergence rather than dressed up as a result.

What it claims

The report's closing section says N passes by independent fresh contexts raised nothing new and the gate passes — and then says what that isn't: not "this change is correct" (the reviewers share a model and therefore a blind spot), no runtime evidence, and no prediction about what a human reviewer will find.

Assumptions

Consuming repo is Open Mercato with the OM skills installed and .ai/agentic.config.json present. Missing either → the skill stops and says so. It never installs anything and never runs om-setup-agent-pipeline. Nothing here goes upstream to open-mercato/skills.

Also in this PR

README skill-table row, and the minor plugin.json bump AGENTS.md requires (1.3.0 → 1.4.0).

🤖 Generated with Claude Code

…it goes quiet

Adds `skills/om-review-loop/SKILL.md`: an Open Mercato review loop that runs
`om-code-review` in fresh subagent contexts each round, fixes what survives
verification, and exits on N *consecutive* quiet rounds rather than a fixed
iteration count.

The mechanism is the fresh context. A context that wrote a fix re-reviews it
against its own intent — good at self-contradiction, blind to what it never
considered. `om-auto-review-pr --autofix` already loops in-process and posts;
this is the subagent, fresh-eyes, local-only version, and the skill states the
three axes where they differ instead of restating the pipeline.

Design points the skill pins down:

- Reviewers never see the round number, the ledger, or the fix rationale —
  "round 5, nothing since round 3" is an instruction to find nothing. Dedupe
  happens orchestrator-side, after the review returns.
- An append-only ledger fingerprinted on file + enclosing symbol (never line
  numbers, which move with the first commit) holds every finding ever raised,
  rejected ones included, so a refuted finding stops returning each round from
  a reviewer who never heard the argument against it.
- One adversarial, refute-by-default verifier per new finding, since each one
  costs a code change; refutation-rate is reported because converging by
  dismissal looks identical to converging by fixing.
- The validation gate splits: codegen prefix once, a subset-with-its-
  prerequisites per round, and the full `validation.commands` exactly once when
  the counter reaches the threshold — a failure there is a blocker that resets
  the counter to zero, so there is no finishing around it.
- Judgement calls and out-of-diff findings are handed to the user, never fixed
  and never dropped; "quiet" means nothing new, not nothing left.

Local only: no reviews, comments, labels, pushes, or merges — it leaves a
branch, a ledger and a report. The report says what N quiet rounds prove
(N passes found nothing new) and what they don't (not "clean", not a
prediction about a human reviewer).

Also: README skill-table row and the minor version bump AGENTS.md requires.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
@jtomaszewski
jtomaszewski merged commit e312ee9 into main Aug 25, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant