Repository navigation
Run critic rounds as real multi-agent Workflows (not simulated critics) - #10
Merged
Merged
Conversation
The critic commands said "multi-agent" but collapsed into one agent simulating ten critics in a single context (or one sub-agent at a time) — no independence, no parallelism, no real maker/checker isolation. Replace that with deterministic Workflow-tool orchestration. - .claude/workflows/critic-panel.js: fan out the 10 critics as PARALLEL subagents (own context each, structured findings), dedup, then adversarially verify every blocker/important finding with a SEPARATE skeptic; return a synthesized matrix. - .claude/workflows/scan-docs.js: pipeline each story through an evidence-gathering agent + a separate status-verifier agent (evidence-based status, no overclaiming). - Rewire /critic-round, /multi-agent-dev, /multi-agent-e2e, /qa-pass to invoke critic-panel via the Workflow tool; /scan-project-docs to invoke scan-docs. - Named the workflow `critic-panel` (not `critic-round`) so it doesn't collide with the /critic-round command in the skills list. - Documented in CLAUDE.md, README, skills-index, adoption inputs, memory wiki + log. Workflow scripts validated (syntax OK in the async runtime context). Docs/config only; no product code touched. Co-Authored-By: Claude Opus 4.8 <[email protected]> Claude-Session: https://claude.ai/code/session_014xUCrJdREvUMjoVzSX5T8G
There was a problem hiding this comment.
Pull request overview
This PR migrates “multi-agent” critic and doc-scanning flows from single-context simulation to deterministic Workflow-tool orchestration, adding real sub-agent fan-out plus verifier/“skeptic” confirmation, and updating docs/commands to use the new workflows.
Changes:
- Added two Workflow scripts:
critic-panel(parallel critics + adversarial verification) andscan-docs(evidence gatherer + independent status verifier per story). - Rewired existing slash commands and documentation to invoke these workflows via the Workflow tool.
- Updated repository guidance (README/CLAUDE/memory/skills index/log) to reflect the new “real multi-agent” approach.
Reviewed changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| README.md | Documents the new .claude/workflows/* scripts and explains “real workflow orchestration” vs simulated critics. |
| CLAUDE.md | Updates core rules to require invoking the new Workflow-based critic/doc-scan flows. |
| .claude/workflows/scan-docs.js | New workflow to pipeline per-story evidence gathering + independent status verification. |
| .claude/workflows/critic-panel.js | New workflow to run critics in parallel and verify blocker/important findings via a skeptic agent. |
| .claude/skills-index.md | Adds routing note pointing critic rounds and doc-scan to workflows. |
| .claude/memory/topics/workflow.md | Records the workflow naming + invocation guidance (avoid command/workflow name collisions). |
| .claude/memory/log.md | Logs the migration to deterministic multi-agent workflows. |
| .claude/commands/scan-project-docs.md | Updates procedure to invoke the scan-docs workflow for parallel evidence/status gathering. |
| .claude/commands/qa-pass.md | Updates QA pass to run post-implementation critic round via critic-panel workflow. |
| .claude/commands/multi-agent-e2e.md | Updates E2E command to surface gaps via critic-panel workflow. |
| .claude/commands/multi-agent-dev.md | Updates dev loop to run pre/post critic rounds via critic-panel workflow. |
| .claude/commands/critic-round.md | Replaces simulated critic-round instructions with Workflow-tool invocation steps. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- critic-panel.js: fail-safe the adversarial verifier — a missing/failed verdict now defaults real:false (do NOT confirm unverified findings); unique verify labels (verify:<critic>:<i>) so one critic's multiple findings don't collide; non-undefined verifyReason fallback. - scan-docs.js: restrict proposed/final status enums to repo-justifiable statuses (REPO_STATUSES, excluding "Browser verified"/"Done"); align the verifier prompt to confirm at most "E2E tested" from repo evidence (browser pass is the caller's job); deterministic justification fallback so the record stays well-formed. - multi-agent-e2e.md: keep the example args.focus JSON on one line (copy-safe). Co-Authored-By: Claude Opus 4.8 <[email protected]> Claude-Session: https://claude.ai/code/session_014xUCrJdREvUMjoVzSX5T8G
Extend the deterministic multi-agent orchestration to the remaining fan-out commands: - gap-analysis.js: one agent per gap dimension (docs↔code↔tests, edge cases, UX/design, ambiguity) in parallel → synthesized next-implementation order. Wired into /story-gap-analysis (was single-agent). - e2e-design.js: one agent per path category (happy/failure/edge/a11y/regression) enumerates concrete Playwright cases → dedup. Wired into /multi-agent-e2e. - loop-iteration.js: maker/checker for /loop at L2/L3 — implementer subagent in an isolated worktree → SEPARATE verifier approves/rejects each item. Wired into .claude/loop.md. Applied the PR-10 review lessons to all new scripts up front: fail-safe verifier defaults, unique per-item labels (map index), schema-required fallback strings. Docs: README tree + command→workflow table, CLAUDE.md, skills-index, and a new "Multi-agent orchestration (Workflows)" section in the adoption guide; memory topics/workflow.md + log entry (incl. the validate-by-wrapping gotcha). All scripts syntax-validated. Docs/config only; no product code touched. Co-Authored-By: Claude Opus 4.8 <[email protected]> Claude-Session: https://claude.ai/code/session_014xUCrJdREvUMjoVzSX5T8G
Dogfooded a critic-panel-style Workflow (18 agents) over .claude/workflows/* —
1 blocker + 10 important confirmed, 0 refuted. Fixes:
- scan-docs.js: BLOCKER — guard the first-stage `evidence` result (unguarded
deref crashed the whole pipeline on a null/failed evidence agent); fail CLOSED
on a missing verifier verdict (cap the maker's self-proposed status) instead of
accepting it unverified; add phase('Evidence') + index-safe labels.
- critic-panel.js: .filter(Boolean) the verified array before reading f.real;
MERGE duplicate findings (keep corroborating critics + evidence) instead of
dropping; optional args.uiInScope to skip the 5 UI critics on non-UI changes;
optional args.priorEvidence so critics verify against already-gathered evidence
instead of re-running; coalesce title before trim.
- loop-iteration.js: implementer returns the REAL git diff; the verifier reviews
the actual diff (not the maker's self-report); the caller applies the returned
patch; add a hard programmatic DENYLIST gate on changedFiles (defense-in-depth);
phase('Implement') + index-safe labels.
- e2e-design.js: dedup key includes category (don't drop a failure case that
shares a title with a happy case); coalesce title before trim.
- multi-agent-dev.md / qa-pass.md: pass args.priorEvidence to the post-impl panel.
loop.md: apply the returned diff patch for approved items.
All scripts re-validated. Memory log records the round + lessons. Docs/config only.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_014xUCrJdREvUMjoVzSX5T8G
The label-uniqueness fix from the dogfooded critic round was only
half-applied: `${id || i}` is null-safe but two work items / stories
sharing the same caller-supplied id still produced identical labels.
Switch loop-iteration and scan-docs to always append the map index
(`${id || 'item'}-${i}`, `${id || 'story'}-${i}`) so labels are unique
even when ids collide.
Record the closing lessons in the memory wiki: always include the map
index in labels, guard the first pipeline stage before deref, fail
closed when the checker is missing, hand the maker/checker the real
artifact (diff) not a self-report, and that the DSL auto-caps
concurrency.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_014xUCrJdREvUMjoVzSX5T8G
| .claude/loop.md Recurring watchdog loop (used by /loop) | ||
| .claude/memory/ Knowledge wiki (index.md + topics/ + log.md) | ||
| .claude/workflows/* Multi-agent orchestration scripts (critic-panel, scan-docs, ...) | ||
| .claude/loop.md loop-checklist.md Recurring watchdog loop + readiness rubric (used by /loop) |
Comment on lines
+113
to
+115
| // Fail safe: a missing/failed verdict must NOT confirm the finding. | ||
| real: v ? v.real : false, | ||
| verifyReason: (v && v.reason) || 'verifier returned no verdict — not confirmed', |
| @@ -1,46 +1,35 @@ | |||
| --- | |||
| description: Read-only multi-agent critic review of the current localhost app state — returns a severity-classified findings matrix, makes no code changes. Use when you want an assessment without implementing. | |||
| description: Run the multi-agent critic round as a deterministic Workflow — critics fan out as parallel subagents (own context each), each blocker/important finding is adversarially verified by a separate skeptic, then a synthesized severity matrix is returned. Read-only; changes no code. | |||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The critic commands said "multi-agent" but collapsed into one agent simulating ten
critics in a single context (or one sub-agent at a time) — no independence, no
parallelism, no real maker/checker isolation. Replace that with deterministic
Workflow-tool orchestration.
(own context each, structured findings), dedup, then adversarially verify every
blocker/important finding with a SEPARATE skeptic; return a synthesized matrix.
agent + a separate status-verifier agent (evidence-based status, no overclaiming).
critic-panel via the Workflow tool; /scan-project-docs to invoke scan-docs.
critic-panel(notcritic-round) so it doesn't collide withthe /critic-round command in the skills list.
Workflow scripts validated (syntax OK in the async runtime context). Docs/config
only; no product code touched.
Co-Authored-By: Claude Opus 4.8 [email protected]
Claude-Session: https://claude.ai/code/session_014xUCrJdREvUMjoVzSX5T8G