From 2a1dc241b911d6cad7c81ab05fef080bff2fe04f Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 29 Jun 2026 17:34:30 +0000 Subject: [PATCH 1/5] Run critic rounds as real multi-agent Workflows (not simulated critics) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The critic commands said "multi-agent" but collapsed into one agent simulating ten critics in a single context (or one sub-agent at a time) — no independence, no parallelism, no real maker/checker isolation. Replace that with deterministic Workflow-tool orchestration. - .claude/workflows/critic-panel.js: fan out the 10 critics as PARALLEL subagents (own context each, structured findings), dedup, then adversarially verify every blocker/important finding with a SEPARATE skeptic; return a synthesized matrix. - .claude/workflows/scan-docs.js: pipeline each story through an evidence-gathering agent + a separate status-verifier agent (evidence-based status, no overclaiming). - Rewire /critic-round, /multi-agent-dev, /multi-agent-e2e, /qa-pass to invoke critic-panel via the Workflow tool; /scan-project-docs to invoke scan-docs. - Named the workflow `critic-panel` (not `critic-round`) so it doesn't collide with the /critic-round command in the skills list. - Documented in CLAUDE.md, README, skills-index, adoption inputs, memory wiki + log. Workflow scripts validated (syntax OK in the async runtime context). Docs/config only; no product code touched. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_014xUCrJdREvUMjoVzSX5T8G --- .claude/commands/critic-round.md | 65 ++++++-------- .claude/commands/multi-agent-dev.md | 70 ++++++--------- .claude/commands/multi-agent-e2e.md | 45 ++++------ .claude/commands/qa-pass.md | 15 ++-- .claude/commands/scan-project-docs.md | 11 ++- .claude/memory/log.md | 9 ++ .claude/memory/topics/workflow.md | 7 ++ .claude/skills-index.md | 5 ++ .claude/workflows/critic-panel.js | 118 ++++++++++++++++++++++++++ .claude/workflows/scan-docs.js | 71 ++++++++++++++++ CLAUDE.md | 11 +++ README.md | 11 +++ 12 files changed, 318 insertions(+), 120 deletions(-) create mode 100644 .claude/workflows/critic-panel.js create mode 100644 .claude/workflows/scan-docs.js diff --git a/.claude/commands/critic-round.md b/.claude/commands/critic-round.md index daf51ea..acb7df4 100644 --- a/.claude/commands/critic-round.md +++ b/.claude/commands/critic-round.md @@ -1,46 +1,35 @@ --- -description: Read-only multi-agent critic review of the current localhost app state — returns a severity-classified findings matrix, makes no code changes. Use when you want an assessment without implementing. +description: Run the multi-agent critic round as a deterministic Workflow — critics fan out as parallel subagents (own context each), each blocker/important finding is adversarially verified by a separate skeptic, then a synthesized severity matrix is returned. Read-only; changes no code. argument-hint: [focus, e.g. "checkout flow on localhost"] --- -Run a multi-agent critic review of the current local web app state. +Run a **genuine multi-agent critic round** — not a single agent simulating critics. +Use the **Workflow tool** so each critic runs as its own subagent in its own context +and every blocker/important finding is verified by a *separate* skeptic. Focus: $ARGUMENTS -Use these critics: - -1. First-Time User Critic -2. UX Flow Critic -3. Designer Critic -4. Artistic Direction Critic -5. Frontend Architecture Critic -6. QA / E2E Critic -7. Accessibility Critic -8. Performance Critic -9. Security Critic -10. Regression Critic - -Procedure: - -1. Inspect relevant code. -2. Start or reuse localhost. -3. Open the relevant page in Chrome or Playwright MCP. -4. Test the flow as a real user. -5. Check console errors. -6. Check failed network requests. -7. Ask each critic to independently review. -8. Return a critic matrix. - -Output format: - -| Critic | Severity | Finding | Evidence | Recommended action | -|---|---|---|---|---| - -Classify severity: - -- blocker: must fix before task is complete -- important: should fix in this task if related -- nice-to-have: record only, do not fix unless asked - -Do not edit files unless explicitly asked. +## Procedure +1. Make sure context exists for the critics: confirm `CLAUDE.md` and (if a UI flow + is in scope) that localhost is running for the browser-driven critics. +2. **Invoke the Workflow tool** with the predefined critic-panel workflow: + - `scriptPath: .claude/workflows/critic-panel.js` + - `args: { "focus": "$ARGUMENTS" }` + This command explicitly opts into multi-agent orchestration — calling Workflow + here is expected. +3. The workflow returns `{ focus, counts, confirmed, niceToHaves, refuted }`. + Render `confirmed` as the critic matrix and list `niceToHaves` separately: + + | Critic | Severity | Finding | Evidence | Recommended action | + |---|---|---|---|---| + +4. Note any `refuted` findings (a skeptic could not substantiate them) so they are + not actioned. Do **not** edit files — this is review only. + +## Notes +- This is the heavier, deterministic path (the workflow spawns ~10 critic agents + + one verifier per blocker/important finding). That is intended: real independent + review, not one context role-playing ten critics. +- Other commands (`/multi-agent-dev`, `/multi-agent-e2e`, `/qa-pass`) reuse this same + workflow for their critic rounds. diff --git a/.claude/commands/multi-agent-dev.md b/.claude/commands/multi-agent-dev.md index c215cc6..bf47023 100644 --- a/.claude/commands/multi-agent-dev.md +++ b/.claude/commands/multi-agent-dev.md @@ -8,53 +8,35 @@ Run a full multi-agent development loop for this local web app. User task: $ARGUMENTS -You are the Lead Agent. - -You must use the following critic agents before implementation: - -1. First-Time User Critic -2. UX Flow Critic -3. Designer Critic -4. Artistic Direction Critic -5. Frontend Architecture Critic -6. QA / E2E Critic -7. Accessibility Critic -8. Performance Critic -9. Security Critic -10. Regression Critic +You are the Lead Agent. The critic rounds run as a **deterministic multi-agent +Workflow** (critics fan out as parallel subagents in their own contexts; each +blocker/important finding is verified by a separate skeptic) — **not** one agent +simulating critics. Invoking the Workflow tool here is expected (this command opts +into multi-agent orchestration). Procedure: -1. Read CLAUDE.md. -2. Read package.json. -3. Inspect the relevant routes, components, API handlers, tests, and styles. -4. Start or reuse the local dev server. -5. Open the relevant localhost page using Chrome or Playwright MCP. -6. Observe the current behavior. -7. Run the critic round before implementation. - -For each critic, produce: - -- severity: blocker / important / nice-to-have -- finding -- evidence -- recommended fix - -Then: - -8. Synthesize findings as Lead Agent. -9. Choose the smallest safe implementation plan. -10. Implement only blockers and important issues directly related to the task. -11. Run the smallest relevant automated checks first. -12. Verify in browser on localhost. -13. Check console errors. -14. Check failed network requests. -15. Add or update Playwright E2E tests for the verified flow. -16. Run relevant E2E tests. -17. Run typecheck, lint, unit tests, and build if appropriate. -18. Run a second critic round after implementation. -19. Fix remaining blockers. -20. Report final evidence. +1. Read CLAUDE.md and package.json. +2. Inspect the relevant routes, components, API handlers, tests, and styles. +3. Start or reuse the local dev server; open the relevant localhost page (Chrome/ + Playwright MCP) and observe current behavior. +4. **Pre-implementation critic round** — invoke the Workflow tool with + `scriptPath: .claude/workflows/critic-panel.js`, + `args: { "focus": "$ARGUMENTS (current behavior + changed files)" }`. + Render the returned `confirmed` findings as the critic matrix; record + `niceToHaves`; ignore `refuted`. +5. As Lead Agent, choose the smallest safe plan from the confirmed blockers + + directly-related important findings. +6. Implement only those. Keep the **maker/checker split**: implementation is your + pass; the post-implementation workflow below is the independent checker. +7. Run the smallest relevant automated checks first. +8. Verify in browser on localhost; check console errors and failed network requests. +9. Add or update Playwright E2E tests for the verified flow; run them. +10. Run typecheck, lint, unit tests, and build if appropriate. +11. **Post-implementation critic round** — invoke the same Workflow again with + `args.focus` set to the changed flow + `git diff`. Fix any remaining confirmed + blockers (re-run the workflow until none remain). +12. Report final evidence. Never mark complete unless: diff --git a/.claude/commands/multi-agent-e2e.md b/.claude/commands/multi-agent-e2e.md index 304f2c2..30eccbb 100644 --- a/.claude/commands/multi-agent-e2e.md +++ b/.claude/commands/multi-agent-e2e.md @@ -3,40 +3,29 @@ description: Design and run Playwright E2E tests for a user flow via a critic wo argument-hint: [target flow, e.g. "onboarding"] --- -Design and run E2E tests using a multi-agent critic workflow. +Design and run E2E tests using a **deterministic multi-agent Workflow** (the critic +panel runs as parallel subagents in their own contexts) — not one agent simulating +critics. Target flow: $ARGUMENTS -Roles: - -1. QA / E2E Critic - - Defines critical paths and edge cases. - -2. First-Time User Critic - - Defines what a new user would try first. - -3. UX Flow Critic - - Defines success, failure, loading, and recovery states. - -4. Accessibility Critic - - Defines keyboard and screen-reader relevant checks. - -5. Regression Critic - - Identifies existing flows that could break. - Procedure: -1. Inspect existing Playwright tests. -2. Inspect app routes and components. -3. Start or reuse localhost. -4. Manually verify the target flow in browser. -5. Draft E2E test cases from critic findings. -6. Implement Playwright tests. -7. Run the specific E2E tests. -8. Fix test or app failures, but do not weaken assertions. -9. Run browser verification again. -10. Report evidence. +1. Inspect existing Playwright tests, app routes, and components. +2. Start or reuse localhost; manually verify the target flow in browser. +3. **Surface coverage gaps via the critic panel** — invoke the Workflow tool with + `scriptPath: .claude/workflows/critic-panel.js`, + `args: { "focus": "E2E coverage of the $ARGUMENTS flow: critical paths, failure/ + edge/recovery states, accessibility, and regressions" }`. The QA/E2E, + First-Time User, UX Flow, Accessibility, and Regression critics each report in + their own context; findings are adversarially verified. +4. Draft E2E test cases from the workflow's `confirmed` findings (happy + at least + one failure/edge path). +5. Implement Playwright tests; run the specific specs. +6. Fix test or app failures, but do not weaken assertions. +7. Run browser verification again. +8. Report evidence. Required output: diff --git a/.claude/commands/qa-pass.md b/.claude/commands/qa-pass.md index b9f0135..b55209b 100644 --- a/.claude/commands/qa-pass.md +++ b/.claude/commands/qa-pass.md @@ -24,13 +24,12 @@ Checklist: - buttons have accessible names - forms have labels - keyboard navigation works for critical flow -12. Run a post-implementation critic round using: - - First-Time User Critic - - UX Flow Critic - - Designer Critic - - QA / E2E Critic - - Accessibility Critic - - Regression Critic -13. Report blockers, non-blockers, and recommended next actions. +12. Run a **post-implementation critic round as a Workflow** — invoke the Workflow + tool with `scriptPath: .claude/workflows/critic-panel.js`, + `args: { "focus": "the changed flows for $ARGUMENTS + git diff" }`. The critics + run as parallel subagents and each blocker/important finding is verified by a + separate skeptic (no single agent simulating the panel). +13. Report the workflow's `confirmed` blockers/non-blockers and recommended next + actions. Never mark as complete if build, tests, or critical browser flow fails. diff --git a/.claude/commands/scan-project-docs.md b/.claude/commands/scan-project-docs.md index 8df1056..3d27bbd 100644 --- a/.claude/commands/scan-project-docs.md +++ b/.claude/commands/scan-project-docs.md @@ -38,8 +38,15 @@ Procedure: pile stories into a single file. 6. Split large stories into independently testable stories. 7. Add measurable acceptance criteria. -8. Search the codebase for implementation evidence. -9. Search the test suite for unit, integration, and E2E evidence. +8. **Gather evidence in parallel via a Workflow** — once you have the story list, + invoke the Workflow tool with `scriptPath: .claude/workflows/scan-docs.js` and + `args: { "stories": [ {id, title, source, acceptanceCriteria}, ... ] }`. It + pipelines each story through an evidence-gathering agent (searches code + tests) + and a *separate* status-verifier agent (evidence-based status, no overclaiming), + returning one record per story. This is real multi-agent orchestration, not one + agent judging every story in a single context. (Invoking Workflow here is expected.) +9. Use the returned records as the source of truth for status; do browser + verification yourself where it raises a status to `Browser verified`/`Done`. 10. Start or reuse localhost when browser verification is possible. 11. Open relevant pages using Chrome or Playwright MCP. 12. Verify implemented user flows in browser when possible. diff --git a/.claude/memory/log.md b/.claude/memory/log.md index 571e30e..eec4fcd 100644 --- a/.claude/memory/log.md +++ b/.claude/memory/log.md @@ -76,3 +76,12 @@ change), `rotate` (archived old entries). Format: `## [YYYY-MM-DD] | 500-line rotation rule). Added the maker/checker no-self-approve rule and `Agent isolation: worktree` note to CLAUDE.md. Filed the principle in topics/workflow.md. + +## [2026-06-29] migrate | Critic rounds → deterministic multi-agent Workflows +- Replaced "simulate critics" (one agent role-playing the panel) with real + orchestration: added `.claude/workflows/critic-panel.js` (parallel critics → + adversarial verify → matrix) and `.claude/workflows/scan-docs.js` (per-story + parallel evidence → separate status verifier). Rewired `/critic-round`, + `/multi-agent-dev`, `/multi-agent-e2e`, `/qa-pass`, `/scan-project-docs` to invoke + them via the Workflow tool. Named the workflow `critic-panel` (not `critic-round`) + to avoid colliding with the command in the skills list. diff --git a/.claude/memory/topics/workflow.md b/.claude/memory/topics/workflow.md index c568494..99da051 100644 --- a/.claude/memory/topics/workflow.md +++ b/.claude/memory/topics/workflow.md @@ -19,6 +19,13 @@ routing); the grouped "when to use" catalog is `.claude/skills-index.md`. A catalog file must NOT live in `.claude/commands/` — anything there auto-registers as its own `/command`. +- **Critic rounds and the doc-scan run as deterministic multi-agent Workflows**, not + one agent simulating critics: `.claude/workflows/critic-panel.js` (fan-out critics + → adversarial verify → matrix) and `.claude/workflows/scan-docs.js` (per-story + parallel evidence → separate status verifier). The critic commands invoke these + via the Workflow tool (the sanctioned multi-agent opt-in). Name the workflow + distinctly from any command (e.g. `critic-panel` vs the `/critic-round` command) + to avoid a duplicate entry in the skills list. - A command's `description` also lets Claude **auto-invoke** it. Add `disable-model-invocation: true` to heavy/code-changing commands that should be manual-only. diff --git a/.claude/skills-index.md b/.claude/skills-index.md index f2a3691..36a08cd 100644 --- a/.claude/skills-index.md +++ b/.claude/skills-index.md @@ -27,6 +27,11 @@ catalog so both you and Claude can route to the right command. `.claude/memory/loop-run-log.md`) ## Development loop + +> Critic rounds run as real multi-agent **Workflows** (`.claude/workflows/critic-panel.js`), +> and the doc-scan uses `.claude/workflows/scan-docs.js` — parallel subagents + +> adversarial verification, invoked via the Workflow tool. Not one agent simulating critics. + | Command | When to use | Args | |---|---|---| | `/write-goal` | Start of a non-trivial task: turn a described outcome into a production-grade `/goal` (or `/loop`) prompt to review and run | `[outcome]` (say "loop" for a /loop) | diff --git a/.claude/workflows/critic-panel.js b/.claude/workflows/critic-panel.js new file mode 100644 index 0000000..a92f594 --- /dev/null +++ b/.claude/workflows/critic-panel.js @@ -0,0 +1,118 @@ +export const meta = { + name: 'critic-panel', + description: + 'Deterministic multi-agent critic round: fan out the critics as parallel subagents (each in its own context), adversarially verify every blocker/important finding with a separate skeptic, then return a synthesized severity matrix.', + phases: [ + { title: 'Review', detail: 'each critic reviews independently, in its own context' }, + { title: 'Verify', detail: 'a separate skeptic tries to refute each blocker/important finding' }, + ], +} + +// What to review (focus) + how deep, passed via the Workflow `args`. +const focus = + (args && (args.focus || (typeof args === 'string' ? args : null))) || + 'the current app/flow and the changed files' + +// The standard critic roster. UI-facing critics should drive the running app via +// Playwright/Chrome MCP (reachable through ToolSearch) when a localhost flow exists. +const CRITICS = [ + { key: 'first-time-user', label: 'First-Time User Critic', lens: 'Can a new user understand the first screen and complete the flow without help? Labels, errors, hidden assumptions.' }, + { key: 'ux-flow', label: 'UX Flow Critic', lens: 'Beginning/middle/end of the journey, feedback on every action, confirm destructive actions, loading/success/recovery states.' }, + { key: 'designer', label: 'Designer Critic', lens: 'Layout hierarchy, spacing, alignment, typography, color, button/empty/loading/error states, mobile responsiveness.' }, + { key: 'artistic-direction', label: 'Artistic Direction Critic', lens: 'Mood, visual identity, beauty, coherence — is it intentional and memorable or generic?' }, + { key: 'frontend-arch', label: 'Frontend Architecture Critic', lens: 'Component boundaries, state, hooks, server/client split, TypeScript correctness, SOLID/DRY/KISS, no duplicated logic.' }, + { key: 'qa-e2e', label: 'QA / E2E Critic', lens: 'Critical paths, missing Playwright tests, meaningful vs superficial assertions, edge cases, did we actually verify localhost?' }, + { key: 'accessibility', label: 'Accessibility Critic', lens: 'Keyboard nav, focus states, accessible names, labels, form-error linkage, contrast, semantic HTML, dialog a11y.' }, + { key: 'performance', label: 'Performance Critic', lens: 'Unnecessary re-renders, large client bundles, heavy images, blocking fetches, overuse of client components.' }, + { key: 'security', label: 'Security Critic', lens: 'Unsafe input handling, secrets exposed to the client, auth-bypass risk, missing server-side validation, unsafe redirects, XSS, insecure storage.' }, + { key: 'regression', label: 'Regression Critic', lens: 'Existing routes/flows affected, shared components/APIs changed, snapshots, styling side effects, backward compatibility.' }, +] + +const FINDINGS_SCHEMA = { + type: 'object', + additionalProperties: false, + required: ['findings'], + properties: { + findings: { + type: 'array', + items: { + type: 'object', + additionalProperties: false, + required: ['severity', 'title', 'evidence', 'recommendation'], + properties: { + severity: { type: 'string', enum: ['blocker', 'important', 'nice-to-have'] }, + title: { type: 'string' }, + evidence: { type: 'string', description: 'Concrete observation: file:line, command output, or browser observation.' }, + recommendation: { type: 'string' }, + }, + }, + }, + }, +} + +const VERDICT_SCHEMA = { + type: 'object', + additionalProperties: false, + required: ['real', 'reason'], + properties: { + real: { type: 'boolean', description: 'true if the finding is genuine after attempting to refute it' }, + reason: { type: 'string' }, + }, +} + +// Phase 1 — every critic reviews independently and in parallel (a barrier: we want +// the full set before deduping and verifying). +phase('Review') +const reviews = await parallel( + CRITICS.map((c) => () => + agent( + `You are the **${c.label}** for this project. Read CLAUDE.md and the relevant memory topic page(s), inspect ${focus} (read the code; if a localhost flow is in scope, drive it via Playwright/Chrome MCP and check console + network), then review strictly through your lens:\n${c.lens}\n\nReturn only findings you can back with concrete evidence. An empty list is a valid, honest answer.`, + { label: `critic:${c.key}`, phase: 'Review', schema: FINDINGS_SCHEMA }, + ).then((r) => ({ critic: c.key, label: c.label, findings: (r && r.findings) || [] })), + ), +) + +// Flatten + light dedup (same severity + near-same title) across critics. +const all = reviews + .filter(Boolean) + .flatMap((r) => r.findings.map((f) => ({ ...f, critic: r.critic, criticLabel: r.label }))) +const seen = new Set() +const deduped = all.filter((f) => { + const k = `${f.severity}::${f.title.trim().toLowerCase()}` + if (seen.has(k)) return false + seen.add(k) + return true +}) +log(`${deduped.length} findings from ${CRITICS.length} critics (${all.length} before dedup)`) + +// Phase 2 — adversarially verify each blocker/important finding with a SEPARATE +// skeptic agent (the reviewer never confirms its own finding). Nice-to-haves are +// recorded as-is, not verified. +phase('Verify') +const toVerify = deduped.filter((f) => f.severity === 'blocker' || f.severity === 'important') +const verified = await parallel( + toVerify.map((f) => () => + agent( + `Adversarially verify this ${f.severity} finding from the ${f.criticLabel}. Try to REFUTE it by inspecting the actual code/flow — default to real:false if the evidence does not hold up.\n\nTitle: ${f.title}\nEvidence: ${f.evidence}\nRecommendation: ${f.recommendation}`, + { label: `verify:${f.critic}`, phase: 'Verify', schema: VERDICT_SCHEMA }, + ).then((v) => ({ ...f, real: v ? v.real : true, verifyReason: v && v.reason })), + ), +) + +const niceToHaves = deduped.filter((f) => f.severity === 'nice-to-have') +const confirmed = verified.filter((f) => f.real !== false) +const refuted = verified.filter((f) => f.real === false) + +return { + focus, + counts: { + critics: CRITICS.length, + confirmedBlockers: confirmed.filter((f) => f.severity === 'blocker').length, + confirmedImportant: confirmed.filter((f) => f.severity === 'important').length, + niceToHave: niceToHaves.length, + refuted: refuted.length, + }, + confirmed, // verified blocker/important findings — the actionable matrix + niceToHaves, // recorded only + refuted, // findings a skeptic could not substantiate +} diff --git a/.claude/workflows/scan-docs.js b/.claude/workflows/scan-docs.js new file mode 100644 index 0000000..eb268ee --- /dev/null +++ b/.claude/workflows/scan-docs.js @@ -0,0 +1,71 @@ +export const meta = { + name: 'scan-docs', + description: + 'Deterministic doc-scanner: for each user story (passed in), pipeline an evidence-gathering agent then a separate status-verifier agent. Returns evidence-based per-story status records; the caller writes the docs/stories files.', + phases: [ + { title: 'Evidence', detail: 'gather implementation + test evidence per story' }, + { title: 'Verify', detail: 'a separate agent confirms an evidence-based status (no overclaiming)' }, + ], +} + +// args.stories: [{ id, title, source, acceptanceCriteria? }] +// The caller extracts the story list from the PRD/docs first, then runs this. +const stories = (args && args.stories) || [] +if (!stories.length) { + return { error: 'No stories provided. Extract stories from docs first, then pass args.stories = [{id,title,source}].', records: [] } +} + +const STATUSES = [ + 'Not started', 'Partially implemented', 'Implemented', + 'Unit tested', 'E2E tested', 'Browser verified', 'Done', 'Blocked', 'Deprecated', +] + +const EVIDENCE_SCHEMA = { + type: 'object', + additionalProperties: false, + required: ['implementationFiles', 'unitTests', 'e2eTests', 'proposedStatus', 'notes'], + properties: { + implementationFiles: { type: 'array', items: { type: 'string' }, description: 'files that implement this story (empty if none found)' }, + unitTests: { type: 'string', enum: ['Missing', 'Partial', 'Present'] }, + e2eTests: { type: 'string', enum: ['Missing', 'Partial', 'Present'] }, + proposedStatus: { type: 'string', enum: STATUSES }, + notes: { type: 'string' }, + }, +} + +const VERDICT_SCHEMA = { + type: 'object', + additionalProperties: false, + required: ['finalStatus', 'justification'], + properties: { + finalStatus: { type: 'string', enum: STATUSES }, + justification: { type: 'string', description: 'why the evidence supports this status; downgrade if the proposed status overclaims' }, + }, +} + +// Pipeline: each story flows through evidence → verify independently (no barrier). +const records = await pipeline( + stories, + (story) => + agent( + `Gather IMPLEMENTATION and TEST evidence for this user story by searching the codebase and tests (do not write any files):\n${JSON.stringify(story)}\n\nReport which files implement it, whether unit and E2E tests cover it, and the most defensible status. "Not started" is correct when no implementation is found.`, + { label: `evidence:${story.id}`, phase: 'Evidence', schema: EVIDENCE_SCHEMA }, + ), + (evidence, story) => + agent( + `You are a strict status verifier. Confirm an EVIDENCE-BASED status for story ${story.id} (${story.title}). A story is only "Done" with implementation + passing tests + E2E/browser verification recorded. Downgrade if the proposed status overclaims.\n\nProposed: ${JSON.stringify(evidence)}`, + { label: `verify:${story.id}`, phase: 'Verify', schema: VERDICT_SCHEMA }, + ).then((v) => ({ + id: story.id, + title: story.title, + source: story.source, + implementationFiles: evidence.implementationFiles, + unitTests: evidence.unitTests, + e2eTests: evidence.e2eTests, + finalStatus: v ? v.finalStatus : evidence.proposedStatus, + justification: v && v.justification, + notes: evidence.notes, + })), +) + +return { count: records.filter(Boolean).length, records: records.filter(Boolean) } diff --git a/CLAUDE.md b/CLAUDE.md index 59cfda0..2baab41 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -124,6 +124,17 @@ Before implementation, the Lead Agent must consult these critic perspectives: 9. Security Critic 10. Regression Critic +**Critic rounds run as a deterministic multi-agent Workflow, not one agent +simulating critics.** Use the Workflow tool with +`.claude/workflows/critic-panel.js`: each critic runs as its own subagent in its own +context, and every blocker/important finding is adversarially verified by a +*separate* skeptic before it counts. The doc-scanner uses +`.claude/workflows/scan-docs.js` the same way (parallel evidence-gathering + +separate status verifier). The commands (`/critic-round`, `/multi-agent-dev`, +`/multi-agent-e2e`, `/qa-pass`, `/scan-project-docs`) invoke these workflows — that +is the sanctioned multi-agent opt-in. This is heavier (many subagents) by design; +it buys genuine independent review instead of a single context role-playing the panel. + ## Development loop For every task: diff --git a/README.md b/README.md index 3a55f2b..a570d7a 100644 --- a/README.md +++ b/README.md @@ -26,6 +26,9 @@ CLAUDE.md Project rules + multi-agent workflow + doc-sca loop-run-log.md Append-only /loop run history (the loop's durable state) topics/ One page per subject, read on demand settings.local.json Allowed commands for the workflow + workflows/ Deterministic multi-agent orchestration scripts (Workflow tool) + critic-panel.js Fan-out critics → adversarial verify → severity matrix + scan-docs.js Per-story parallel evidence-gathering → status verifier commands/ Each command has a description used for routing write-goal.md Write a production-grade /goal (or /loop) prompt for a task multi-agent-dev.md Full critic-led development pass @@ -107,6 +110,14 @@ The documentation-scanner workflow adds: Product Requirements, User Story, Acceptance Criteria, Code Evidence, Unit Test Evidence, E2E Test Evidence, and Documentation Consistency critics. +**These run as real multi-agent orchestration, not one agent simulating critics.** +Critic rounds invoke the `.claude/workflows/critic-panel.js` Workflow — each critic +is its own subagent in its own context, and every blocker/important finding is +adversarially verified by a *separate* skeptic before it counts. The doc-scanner +uses `.claude/workflows/scan-docs.js` (parallel per-story evidence + a separate +status verifier). It's heavier by design — independent review beats a single context +role-playing the panel. + ## Everyday workflow Let Claude write the `/goal` for you — it knows the project's gates better than a From 0c655587bea777243dcf83f159c9f07f2555a259 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 29 Jun 2026 17:46:05 +0000 Subject: [PATCH 2/5] workflows: address Copilot review on PR #10 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - critic-panel.js: fail-safe the adversarial verifier — a missing/failed verdict now defaults real:false (do NOT confirm unverified findings); unique verify labels (verify::) so one critic's multiple findings don't collide; non-undefined verifyReason fallback. - scan-docs.js: restrict proposed/final status enums to repo-justifiable statuses (REPO_STATUSES, excluding "Browser verified"/"Done"); align the verifier prompt to confirm at most "E2E tested" from repo evidence (browser pass is the caller's job); deterministic justification fallback so the record stays well-formed. - multi-agent-e2e.md: keep the example args.focus JSON on one line (copy-safe). Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_014xUCrJdREvUMjoVzSX5T8G --- .claude/commands/multi-agent-e2e.md | 9 ++++----- .claude/workflows/critic-panel.js | 11 ++++++++--- .claude/workflows/scan-docs.js | 15 +++++++++------ 3 files changed, 21 insertions(+), 14 deletions(-) diff --git a/.claude/commands/multi-agent-e2e.md b/.claude/commands/multi-agent-e2e.md index 30eccbb..8c93e27 100644 --- a/.claude/commands/multi-agent-e2e.md +++ b/.claude/commands/multi-agent-e2e.md @@ -15,11 +15,10 @@ Procedure: 1. Inspect existing Playwright tests, app routes, and components. 2. Start or reuse localhost; manually verify the target flow in browser. 3. **Surface coverage gaps via the critic panel** — invoke the Workflow tool with - `scriptPath: .claude/workflows/critic-panel.js`, - `args: { "focus": "E2E coverage of the $ARGUMENTS flow: critical paths, failure/ - edge/recovery states, accessibility, and regressions" }`. The QA/E2E, - First-Time User, UX Flow, Accessibility, and Regression critics each report in - their own context; findings are adversarially verified. + `scriptPath: .claude/workflows/critic-panel.js` and (on one line) + `args: { "focus": "E2E coverage of the $ARGUMENTS flow: critical paths, failure/edge/recovery states, accessibility, and regressions" }`. + The QA/E2E, First-Time User, UX Flow, Accessibility, and Regression critics each + report in their own context; findings are adversarially verified. 4. Draft E2E test cases from the workflow's `confirmed` findings (happy + at least one failure/edge path). 5. Implement Playwright tests; run the specific specs. diff --git a/.claude/workflows/critic-panel.js b/.claude/workflows/critic-panel.js index a92f594..bffff1e 100644 --- a/.claude/workflows/critic-panel.js +++ b/.claude/workflows/critic-panel.js @@ -91,11 +91,16 @@ log(`${deduped.length} findings from ${CRITICS.length} critics (${all.length} be phase('Verify') const toVerify = deduped.filter((f) => f.severity === 'blocker' || f.severity === 'important') const verified = await parallel( - toVerify.map((f) => () => + toVerify.map((f, i) => () => agent( `Adversarially verify this ${f.severity} finding from the ${f.criticLabel}. Try to REFUTE it by inspecting the actual code/flow — default to real:false if the evidence does not hold up.\n\nTitle: ${f.title}\nEvidence: ${f.evidence}\nRecommendation: ${f.recommendation}`, - { label: `verify:${f.critic}`, phase: 'Verify', schema: VERDICT_SCHEMA }, - ).then((v) => ({ ...f, real: v ? v.real : true, verifyReason: v && v.reason })), + { label: `verify:${f.critic}:${i}`, phase: 'Verify', schema: VERDICT_SCHEMA }, + ).then((v) => ({ + ...f, + // Fail safe: a missing/failed verdict must NOT confirm the finding. + real: v ? v.real : false, + verifyReason: (v && v.reason) || 'verifier returned no verdict — not confirmed', + })), ), ) diff --git a/.claude/workflows/scan-docs.js b/.claude/workflows/scan-docs.js index eb268ee..b25abf7 100644 --- a/.claude/workflows/scan-docs.js +++ b/.claude/workflows/scan-docs.js @@ -15,9 +15,12 @@ if (!stories.length) { return { error: 'No stories provided. Extract stories from docs first, then pass args.stories = [{id,title,source}].', records: [] } } -const STATUSES = [ +// This workflow only inspects the repo (code + tests). Statuses that require a +// human/MCP browser pass — "Browser verified" and "Done" — are intentionally OUT of +// scope: the caller raises a story to those after verifying it in a browser. +const REPO_STATUSES = [ 'Not started', 'Partially implemented', 'Implemented', - 'Unit tested', 'E2E tested', 'Browser verified', 'Done', 'Blocked', 'Deprecated', + 'Unit tested', 'E2E tested', 'Blocked', 'Deprecated', ] const EVIDENCE_SCHEMA = { @@ -28,7 +31,7 @@ const EVIDENCE_SCHEMA = { implementationFiles: { type: 'array', items: { type: 'string' }, description: 'files that implement this story (empty if none found)' }, unitTests: { type: 'string', enum: ['Missing', 'Partial', 'Present'] }, e2eTests: { type: 'string', enum: ['Missing', 'Partial', 'Present'] }, - proposedStatus: { type: 'string', enum: STATUSES }, + proposedStatus: { type: 'string', enum: REPO_STATUSES }, notes: { type: 'string' }, }, } @@ -38,7 +41,7 @@ const VERDICT_SCHEMA = { additionalProperties: false, required: ['finalStatus', 'justification'], properties: { - finalStatus: { type: 'string', enum: STATUSES }, + finalStatus: { type: 'string', enum: REPO_STATUSES }, justification: { type: 'string', description: 'why the evidence supports this status; downgrade if the proposed status overclaims' }, }, } @@ -53,7 +56,7 @@ const records = await pipeline( ), (evidence, story) => agent( - `You are a strict status verifier. Confirm an EVIDENCE-BASED status for story ${story.id} (${story.title}). A story is only "Done" with implementation + passing tests + E2E/browser verification recorded. Downgrade if the proposed status overclaims.\n\nProposed: ${JSON.stringify(evidence)}`, + `You are a strict status verifier for story ${story.id} (${story.title}). Confirm a status that is justified by REPO evidence only (code + tests). From repo evidence you can confirm at most "E2E tested"; "Browser verified" and "Done" require a separate browser pass and are OUT of scope here — never assign them. Downgrade if the proposed status overclaims.\n\nProposed: ${JSON.stringify(evidence)}`, { label: `verify:${story.id}`, phase: 'Verify', schema: VERDICT_SCHEMA }, ).then((v) => ({ id: story.id, @@ -63,7 +66,7 @@ const records = await pipeline( unitTests: evidence.unitTests, e2eTests: evidence.e2eTests, finalStatus: v ? v.finalStatus : evidence.proposedStatus, - justification: v && v.justification, + justification: (v && v.justification) || 'verifier returned no verdict — using gathered repo evidence', notes: evidence.notes, })), ) From 48a38d9b600d6030a6c3795c2e130979265612e0 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 29 Jun 2026 17:54:01 +0000 Subject: [PATCH 3/5] workflows: add gap-analysis, e2e-design, loop-iteration (+ wire + docs) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Extend the deterministic multi-agent orchestration to the remaining fan-out commands: - gap-analysis.js: one agent per gap dimension (docs↔code↔tests, edge cases, UX/design, ambiguity) in parallel → synthesized next-implementation order. Wired into /story-gap-analysis (was single-agent). - e2e-design.js: one agent per path category (happy/failure/edge/a11y/regression) enumerates concrete Playwright cases → dedup. Wired into /multi-agent-e2e. - loop-iteration.js: maker/checker for /loop at L2/L3 — implementer subagent in an isolated worktree → SEPARATE verifier approves/rejects each item. Wired into .claude/loop.md. Applied the PR-10 review lessons to all new scripts up front: fail-safe verifier defaults, unique per-item labels (map index), schema-required fallback strings. Docs: README tree + command→workflow table, CLAUDE.md, skills-index, and a new "Multi-agent orchestration (Workflows)" section in the adoption guide; memory topics/workflow.md + log entry (incl. the validate-by-wrapping gotcha). All scripts syntax-validated. Docs/config only; no product code touched. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_014xUCrJdREvUMjoVzSX5T8G --- .claude/commands/multi-agent-e2e.md | 21 ++++--- .claude/commands/story-gap-analysis.md | 42 ++++++-------- .claude/loop.md | 20 ++++--- .claude/memory/log.md | 10 ++++ .claude/memory/topics/workflow.md | 22 +++++--- .claude/skills-index.md | 8 ++- .claude/workflows/e2e-design.js | 55 ++++++++++++++++++ .claude/workflows/gap-analysis.js | 77 ++++++++++++++++++++++++++ .claude/workflows/loop-iteration.js | 63 +++++++++++++++++++++ CLAUDE.md | 28 ++++++---- README.md | 24 ++++++-- docs/adoption-guide.md | 30 +++++++++- 12 files changed, 329 insertions(+), 71 deletions(-) create mode 100644 .claude/workflows/e2e-design.js create mode 100644 .claude/workflows/gap-analysis.js create mode 100644 .claude/workflows/loop-iteration.js diff --git a/.claude/commands/multi-agent-e2e.md b/.claude/commands/multi-agent-e2e.md index 8c93e27..01d63e9 100644 --- a/.claude/commands/multi-agent-e2e.md +++ b/.claude/commands/multi-agent-e2e.md @@ -14,17 +14,16 @@ Procedure: 1. Inspect existing Playwright tests, app routes, and components. 2. Start or reuse localhost; manually verify the target flow in browser. -3. **Surface coverage gaps via the critic panel** — invoke the Workflow tool with - `scriptPath: .claude/workflows/critic-panel.js` and (on one line) - `args: { "focus": "E2E coverage of the $ARGUMENTS flow: critical paths, failure/edge/recovery states, accessibility, and regressions" }`. - The QA/E2E, First-Time User, UX Flow, Accessibility, and Regression critics each - report in their own context; findings are adversarially verified. -4. Draft E2E test cases from the workflow's `confirmed` findings (happy + at least - one failure/edge path). -5. Implement Playwright tests; run the specific specs. -6. Fix test or app failures, but do not weaken assertions. -7. Run browser verification again. -8. Report evidence. +3. **Enumerate test cases via a Workflow** — invoke the Workflow tool with + `scriptPath: .claude/workflows/e2e-design.js`, `args: { "flow": "$ARGUMENTS" }`. + One agent per path category (happy, failure, edge, accessibility, regression) + enumerates concrete cases in its own context; results are deduped. (Invoking + Workflow here is expected.) It returns `{ flow, count, cases }`. +4. Implement Playwright tests from the returned `cases` (cover the happy path + at + least one failure/edge path); run the specific specs. +5. Fix test or app failures, but do not weaken assertions. +6. Run browser verification again. +7. Report evidence. Required output: diff --git a/.claude/commands/story-gap-analysis.md b/.claude/commands/story-gap-analysis.md index 02ea837..31ec7b5 100644 --- a/.claude/commands/story-gap-analysis.md +++ b/.claude/commands/story-gap-analysis.md @@ -13,27 +13,21 @@ the topic pages relevant to the gaps you're assessing (e.g. `topics/testing.md`, `topics/build-and-verify.md`) so "tested / not tested" and "edge cases" judgments reflect the project's real conventions. -Produce: - -1. Requirements found in docs but not implemented. -2. Implemented features not described in docs. -3. Stories implemented but not unit-tested. -4. Stories implemented but not E2E-tested. -5. Stories E2E-tested but missing important edge cases. -6. Stories that work functionally but fail UX/design/artistic direction review. -7. Ambiguous or contradictory requirements. -8. Recommended next implementation order. - -Use critic agents: - -- Product Requirements Critic -- Code Evidence Critic -- Unit Test Evidence Critic -- E2E Test Evidence Critic -- UX / First-Time User Critic -- Designer Critic -- Artistic Direction Critic -- Regression Critic -- Documentation Consistency Critic - -Do not edit files unless explicitly asked. +**Run the analysis as a deterministic multi-agent Workflow** — invoke the Workflow +tool with `scriptPath: .claude/workflows/gap-analysis.js`, +`args: { "scope": "$ARGUMENTS" }`. One agent analyzes each gap dimension in its own +context, in parallel; a final agent synthesizes the recommended order. (Invoking +Workflow here is expected.) + +The workflow returns `{ scope, totalGaps, byDimension, recommendedOrder }` covering: + +1. Requirements found in docs but not implemented (`docs-not-impl`). +2. Implemented features not described in docs (`impl-not-docs`). +3. Stories implemented but not unit-tested (`impl-not-unit`). +4. Stories implemented but not E2E-tested (`impl-not-e2e`). +5. Stories E2E-tested but missing important edge cases (`e2e-missing-edges`). +6. Stories that work but fail UX/design/artistic-direction review (`ux-design`). +7. Ambiguous or contradictory requirements (`ambiguous`). +8. Recommended next implementation order (`recommendedOrder`). + +Render the dimensions + the recommended order. Do not edit files unless explicitly asked. diff --git a/.claude/loop.md b/.claude/loop.md index 0db50e3..6abd9f7 100644 --- a/.claude/loop.md +++ b/.claude/loop.md @@ -22,16 +22,20 @@ budget, denylist, escalation, stop conditions). Before running, fix the loop's 4. Start or reuse the dev server. 5. Open the relevant localhost page in Chrome or Playwright MCP. 6. Check browser console errors and failed network requests. -7. Run a critic round (First-Time User, UX Flow, Designer, Artistic Direction, - Frontend Architecture, QA / E2E, Accessibility, Performance, Security, - Regression) and produce a critic matrix. -8. **Maker/checker:** the critic that verifies a fix must be a *separate* pass from - the one that made it — the implementer never marks its own work "done". +7. Run the critic round as the **critic-panel Workflow** + (`.claude/workflows/critic-panel.js`, `args.focus` = the current task/flow): + critics fan out as parallel subagents and findings are adversarially verified. + Render the returned `confirmed` matrix. +8. **Maker/checker:** the pass that verifies a fix must be *separate* from the one + that made it — the implementer never marks its own work "done". 9. Act **only within the trust level**: - L1: record findings only. - - L2/L3: fix blockers directly related to the current task; fix important issues - only if low-risk and in scope; re-run the smallest relevant test; re-check - browser. Never fix nice-to-haves automatically; never start unrelated refactors. + - L2/L3: fix the confirmed blockers + directly-related important findings via the + **loop-iteration Workflow** (`.claude/workflows/loop-iteration.js`, + `args.items` = the triaged, denylist-cleared items): an implementer subagent + fixes each in an isolated worktree and a *separate* verifier approves/rejects. + Apply approved changes; escalate the rest. Never fix nice-to-haves + automatically; never start unrelated refactors. 10. **Append a run entry** to `.claude/memory/loop-run-log.md` (see format below). ## Denylist — never touch autonomously (escalate instead) diff --git a/.claude/memory/log.md b/.claude/memory/log.md index eec4fcd..d2546cc 100644 --- a/.claude/memory/log.md +++ b/.claude/memory/log.md @@ -85,3 +85,13 @@ change), `rotate` (archived old entries). Format: `## [YYYY-MM-DD] | Critic rounds run as real multi-agent **Workflows** (`.claude/workflows/critic-panel.js`), -> and the doc-scan uses `.claude/workflows/scan-docs.js` — parallel subagents + -> adversarial verification, invoked via the Workflow tool. Not one agent simulating critics. +> Fan-out commands run as real multi-agent **Workflows** under `.claude/workflows/` +> (parallel subagents + adversarial/maker-checker verification), invoked via the +> Workflow tool — not one agent simulating critics: `critic-panel` (critic rounds), +> `scan-docs` (`/scan-project-docs`), `gap-analysis` (`/story-gap-analysis`), +> `e2e-design` (`/multi-agent-e2e`), `loop-iteration` (`/loop` L2/L3). | Command | When to use | Args | |---|---|---| diff --git a/.claude/workflows/e2e-design.js b/.claude/workflows/e2e-design.js new file mode 100644 index 0000000..e542d82 --- /dev/null +++ b/.claude/workflows/e2e-design.js @@ -0,0 +1,55 @@ +export const meta = { + name: 'e2e-design', + description: + 'Deterministic E2E test design: fan out one agent per path category (happy, failure, edge, accessibility, regression) to enumerate concrete Playwright test cases for a flow, then dedup. The caller implements the cases.', + phases: [{ title: 'Design', detail: 'one agent per path category enumerates test cases in parallel' }], +} + +const flow = (args && (args.flow || (typeof args === 'string' ? args : null))) || 'the target flow' + +const CATEGORIES = [ + { key: 'happy', prompt: 'the happy path(s) a user follows when everything works' }, + { key: 'failure', prompt: 'validation/failure paths and error states' }, + { key: 'edge', prompt: 'boundary/edge cases (limits, empty input, special characters, reload/persistence)' }, + { key: 'accessibility', prompt: 'keyboard navigation and accessible-name/role assertions' }, + { key: 'regression', prompt: 'existing flows this change could break' }, +] + +const CASES_SCHEMA = { + type: 'object', additionalProperties: false, required: ['cases'], + properties: { + cases: { + type: 'array', + items: { + type: 'object', additionalProperties: false, required: ['title', 'steps', 'expected'], + properties: { + title: { type: 'string' }, + steps: { type: 'string' }, + expected: { type: 'string', description: 'visible outcome to assert — not just page load' }, + }, + }, + }, + }, +} + +phase('Design') +const results = await parallel( + CATEGORIES.map((c) => () => + agent( + `Enumerate concrete Playwright E2E test cases for "${flow}" covering ${c.prompt}. Inspect the routes/components and existing specs first. Each case needs: title, steps, and the visible expected outcome to assert.`, + { label: `e2e:${c.key}`, phase: 'Design', schema: CASES_SCHEMA }, + ).then((r) => ((r && r.cases) || []).map((x) => ({ ...x, category: c.key }))), + ), +) + +const all = results.filter(Boolean).flat() +const seen = new Set() +const cases = all.filter((c) => { + const k = c.title.trim().toLowerCase() + if (seen.has(k)) return false + seen.add(k) + return true +}) +log(`${cases.length} test cases (${all.length} before dedup) for "${flow}"`) + +return { flow, count: cases.length, cases } diff --git a/.claude/workflows/gap-analysis.js b/.claude/workflows/gap-analysis.js new file mode 100644 index 0000000..fe8ff8f --- /dev/null +++ b/.claude/workflows/gap-analysis.js @@ -0,0 +1,77 @@ +export const meta = { + name: 'gap-analysis', + description: + 'Deterministic gap analysis: fan out one agent per gap dimension (docs↔code↔tests, edge cases, UX/design, ambiguity) in parallel, then synthesize a recommended next-implementation order. Read-only.', + phases: [ + { title: 'Dimensions', detail: 'one agent per gap dimension, in parallel' }, + { title: 'Synthesize', detail: 'merge into a prioritized next-implementation order' }, + ], +} + +const scope = (args && (args.scope || (typeof args === 'string' ? args : null))) || 'the whole project' + +const DIMENSIONS = [ + { key: 'docs-not-impl', prompt: 'Requirements in docs (docs/prd.md, docs/stories/*, README) with NO implementation evidence in the codebase.' }, + { key: 'impl-not-docs', prompt: 'Implemented features/behaviors NOT described in any product doc.' }, + { key: 'impl-not-unit', prompt: 'Stories/behaviors implemented but lacking meaningful unit tests.' }, + { key: 'impl-not-e2e', prompt: 'Stories/flows implemented but lacking Playwright E2E coverage.' }, + { key: 'e2e-missing-edges', prompt: 'Flows with E2E coverage that miss important failure/edge cases.' }, + { key: 'ux-design', prompt: 'Flows that work functionally but would fail a UX / Designer / Artistic Direction review.' }, + { key: 'ambiguous', prompt: 'Ambiguous or contradictory requirements across the docs.' }, +] + +const GAP_SCHEMA = { + type: 'object', additionalProperties: false, required: ['gaps'], + properties: { + gaps: { + type: 'array', + items: { + type: 'object', additionalProperties: false, required: ['item', 'detail', 'priority'], + properties: { + item: { type: 'string' }, + detail: { type: 'string', description: 'concrete evidence: file/story/test reference' }, + priority: { type: 'string', enum: ['high', 'medium', 'low'] }, + }, + }, + }, + }, +} + +const ORDER_SCHEMA = { + type: 'object', additionalProperties: false, required: ['order'], + properties: { + order: { + type: 'array', + items: { + type: 'object', additionalProperties: false, required: ['next', 'why'], + properties: { next: { type: 'string' }, why: { type: 'string' } }, + }, + }, + }, +} + +phase('Dimensions') +const results = await parallel( + DIMENSIONS.map((d) => () => + agent( + `Analyze the gap for "${scope}" along this dimension by reading docs/, src/, e2e/, and the story map:\n${d.prompt}\nReturn only gaps backed by concrete references; an empty list is a valid, honest answer.`, + { label: `gap:${d.key}`, phase: 'Dimensions', schema: GAP_SCHEMA }, + ).then((r) => ({ dimension: d.key, gaps: (r && r.gaps) || [] })), + ), +) + +const byDimension = {} +let total = 0 +for (const r of results.filter(Boolean)) { + byDimension[r.dimension] = r.gaps + total += r.gaps.length +} +log(`${total} gaps across ${DIMENSIONS.length} dimensions`) + +phase('Synthesize') +const synth = await agent( + `Given these gaps grouped by dimension, propose the recommended NEXT-implementation order (highest-leverage first), each with a one-line "why". Be concrete.\n\n${JSON.stringify(byDimension)}`, + { label: 'synthesize', phase: 'Synthesize', schema: ORDER_SCHEMA }, +) + +return { scope, totalGaps: total, byDimension, recommendedOrder: (synth && synth.order) || [] } diff --git a/.claude/workflows/loop-iteration.js b/.claude/workflows/loop-iteration.js new file mode 100644 index 0000000..bf1c07f --- /dev/null +++ b/.claude/workflows/loop-iteration.js @@ -0,0 +1,63 @@ +export const meta = { + name: 'loop-iteration', + description: + 'Maker/checker for an L2/L3 loop iteration: for each already-triaged, denylist-cleared work item, an implementer subagent (isolated git worktree) applies the smallest safe fix and runs the relevant checks there; a SEPARATE verifier subagent adversarially reviews the reported change and approves or rejects. Returns per-item outcomes; the caller applies approved changes and escalates the rest.', + phases: [ + { title: 'Implement', detail: 'implementer per item, in an isolated worktree' }, + { title: 'Verify', detail: 'a separate verifier reviews the change and approves/rejects' }, + ], +} + +// args.items: [{ id, description }] — already triaged and denylist-cleared by the loop. +const items = (args && args.items) || [] +const level = (args && args.level) || 'L2' +if (!items.length) { + return { error: 'No work items. Triage + denylist-check first, then pass args.items = [{id, description}].', applied: [], rejected: [] } +} + +const IMPL_SCHEMA = { + type: 'object', additionalProperties: false, required: ['changedFiles', 'diffSummary', 'checksPassed', 'notes'], + properties: { + changedFiles: { type: 'array', items: { type: 'string' } }, + diffSummary: { type: 'string' }, + checksPassed: { type: 'boolean', description: 'did the smallest relevant checks pass in the worktree' }, + notes: { type: 'string' }, + }, +} + +const VERDICT_SCHEMA = { + type: 'object', additionalProperties: false, required: ['pass', 'reason'], + properties: { + pass: { type: 'boolean' }, + reason: { type: 'string' }, + }, +} + +// Pipeline: each item flows implement → verify independently. +const outcomes = await pipeline( + items, + (item) => + agent( + `Implement the SMALLEST safe fix for this loop work item and run the smallest relevant checks (tests/typecheck/lint). Stay in scope; NEVER touch denylist paths (auth, payments, secrets/.env, infra, CI config, migrations). Report the changed files, a diff summary, and whether the checks passed.\n\n${JSON.stringify(item)}`, + { label: `impl:${item.id}`, phase: 'Implement', schema: IMPL_SCHEMA, isolation: 'worktree' }, + ), + (impl, item) => + agent( + `You are a SEPARATE verifier (not the implementer). Adversarially review this proposed change for correctness, scope creep, denylist violations, and whether its "checksPassed" claim is credible. Default to pass:false if you cannot confirm it is safe and correct.\n\nItem: ${JSON.stringify(item)}\nProposed change: ${JSON.stringify(impl)}`, + { label: `verify:${item.id}`, phase: 'Verify', schema: VERDICT_SCHEMA }, + ).then((v) => ({ + id: item.id, + // Fail safe: no verdict / verifier died → not approved. + applied: !!(v && v.pass), + status: v && v.pass ? 'applied' : 'rejected', + changedFiles: (impl && impl.changedFiles) || [], + diffSummary: impl && impl.diffSummary, + checksPassed: !!(impl && impl.checksPassed), + verdict: (v && v.reason) || 'verifier returned no verdict — rejected', + })), +) + +const settled = outcomes.filter(Boolean) +const applied = settled.filter((o) => o.applied) +const rejected = settled.filter((o) => !o.applied) +return { level, applied, rejected, escalate: rejected.map((o) => o.id) } diff --git a/CLAUDE.md b/CLAUDE.md index 2baab41..a183267 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -124,16 +124,24 @@ Before implementation, the Lead Agent must consult these critic perspectives: 9. Security Critic 10. Regression Critic -**Critic rounds run as a deterministic multi-agent Workflow, not one agent -simulating critics.** Use the Workflow tool with -`.claude/workflows/critic-panel.js`: each critic runs as its own subagent in its own -context, and every blocker/important finding is adversarially verified by a -*separate* skeptic before it counts. The doc-scanner uses -`.claude/workflows/scan-docs.js` the same way (parallel evidence-gathering + -separate status verifier). The commands (`/critic-round`, `/multi-agent-dev`, -`/multi-agent-e2e`, `/qa-pass`, `/scan-project-docs`) invoke these workflows — that -is the sanctioned multi-agent opt-in. This is heavier (many subagents) by design; -it buys genuine independent review instead of a single context role-playing the panel. +**Critic rounds and other fan-out work run as deterministic multi-agent Workflows, +not one agent simulating critics.** The commands invoke `Workflow`-tool scripts in +`.claude/workflows/` — parallel subagents (own context each) with adversarial / +maker-checker verification — and that command-driven invocation is the sanctioned +multi-agent opt-in: + +- `critic-panel.js` — fan-out critics → adversarial verify → matrix + (`/critic-round`, `/multi-agent-dev`, `/qa-pass`, `/loop`). +- `scan-docs.js` — per-story parallel evidence → separate status verifier + (`/scan-project-docs`). +- `gap-analysis.js` — per-dimension gaps → synthesized order (`/story-gap-analysis`). +- `e2e-design.js` — per-category E2E case enumeration → dedup (`/multi-agent-e2e`). +- `loop-iteration.js` — worktree implementer → separate verifier per item + (`/loop` at L2/L3). + +This is heavier (many subagents) by design; it buys genuine independent review +instead of one context role-playing the panel. Commands that don't fan out +(`/fix-localhost`, `/write-goal`, `/dream`, `/memory-audit`) stay single-agent. ## Development loop diff --git a/README.md b/README.md index a570d7a..a987219 100644 --- a/README.md +++ b/README.md @@ -29,6 +29,9 @@ CLAUDE.md Project rules + multi-agent workflow + doc-sca workflows/ Deterministic multi-agent orchestration scripts (Workflow tool) critic-panel.js Fan-out critics → adversarial verify → severity matrix scan-docs.js Per-story parallel evidence-gathering → status verifier + gap-analysis.js Per-dimension gap analysis → synthesized next-order + e2e-design.js Per-category E2E test-case enumeration → dedup + loop-iteration.js Maker/checker: worktree implementer → separate verifier commands/ Each command has a description used for routing write-goal.md Write a production-grade /goal (or /loop) prompt for a task multi-agent-dev.md Full critic-led development pass @@ -111,12 +114,21 @@ Acceptance Criteria, Code Evidence, Unit Test Evidence, E2E Test Evidence, and Documentation Consistency critics. **These run as real multi-agent orchestration, not one agent simulating critics.** -Critic rounds invoke the `.claude/workflows/critic-panel.js` Workflow — each critic -is its own subagent in its own context, and every blocker/important finding is -adversarially verified by a *separate* skeptic before it counts. The doc-scanner -uses `.claude/workflows/scan-docs.js` (parallel per-story evidence + a separate -status verifier). It's heavier by design — independent review beats a single context -role-playing the panel. +The critic-bearing commands invoke deterministic `Workflow`-tool scripts under +`.claude/workflows/` — parallel subagents (own context each) with adversarial / +maker-checker verification: + +| Workflow | Backs | What it does | +|---|---|---| +| `critic-panel.js` | `/critic-round`, `/multi-agent-dev`, `/qa-pass`, `/loop` | fan-out critics → adversarial verify → severity matrix | +| `scan-docs.js` | `/scan-project-docs` | per-story parallel evidence → separate status verifier | +| `gap-analysis.js` | `/story-gap-analysis` | per-dimension gap analysis → synthesized next-order | +| `e2e-design.js` | `/multi-agent-e2e` | per-category E2E case enumeration → dedup | +| `loop-iteration.js` | `/loop` (L2/L3) | worktree implementer → separate verifier per item | + +It's heavier by design — independent review beats a single context role-playing the +panel. Commands that don't fan out (`/fix-localhost`, `/write-goal`, `/dream`, +`/memory-audit`) stay single-agent on purpose. ## Everyday workflow diff --git a/docs/adoption-guide.md b/docs/adoption-guide.md index fb2e86f..f470003 100644 --- a/docs/adoption-guide.md +++ b/docs/adoption-guide.md @@ -23,8 +23,9 @@ The portable part is everything **except** the starter app. The starter app ```txt CLAUDE.md Project rules, critic roster, completion gates .claude/commands/* Slash commands (/multi-agent-dev, /critic-round, /dream, ...) -.claude/loop.md Recurring watchdog loop (used by /loop) -.claude/memory/ Knowledge wiki (index.md + topics/ + log.md) +.claude/workflows/* Multi-agent orchestration scripts (critic-panel, scan-docs, ...) +.claude/loop.md loop-checklist.md Recurring watchdog loop + readiness rubric (used by /loop) +.claude/memory/ Knowledge wiki (index.md + topics/ + log.md + loop-run-log.md) .claude/settings.local.json Allowed commands (adapt to your tools) docs/*.md docs/stories/ Story-map scaffolds (user-stories index, stories/ per-story files, ...) ``` @@ -491,6 +492,31 @@ Adapt it to your stack: --- +## Multi-agent orchestration (Workflows) + +The critic-bearing commands don't simulate critics in one context — they invoke +deterministic **`Workflow`-tool scripts** under `.claude/workflows/` that fan out +parallel subagents (own context each) with adversarial / maker-checker verification: +`critic-panel`, `scan-docs`, `gap-analysis`, `e2e-design`, and `loop-iteration`. + +To adapt them: + +- **They're project-aware via prompts, not config** — the agents read `CLAUDE.md`, + the memory wiki, and your `src/`/`e2e/`/`docs/` paths. Adjust those paths in the + scripts if your layout differs (e.g. a monorepo package dir). +- **Cost is real.** A critic round spawns ~10 critic agents + a verifier per + blocker/important finding; `/multi-agent-dev` runs that twice (pre + post). That's + the price of genuine independent review — scale the roster down if you want it + lighter, or reserve the workflows for substantive changes. +- **Name workflows distinctly from commands** — a workflow whose `meta.name` + matches a command shows up as a duplicate in the skills list (that's why the + critic workflow is `critic-panel`, not `critic-round`). +- **Requires a Claude Code build with the `Workflow` tool.** If yours lacks it, the + commands still work as a single Lead agent simulating the panel — just without the + independent contexts. + +--- + ## Tune the critics to your product The 10 critics are general-purpose. Adjust emphasis to your product type: From 2a760dc56eb014435ed6876292d12a0ad2703060 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 29 Jun 2026 18:09:17 +0000 Subject: [PATCH 4/5] workflows: fix defects found by a real multi-agent critic round (PR #10) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Dogfooded a critic-panel-style Workflow (18 agents) over .claude/workflows/* — 1 blocker + 10 important confirmed, 0 refuted. Fixes: - scan-docs.js: BLOCKER — guard the first-stage `evidence` result (unguarded deref crashed the whole pipeline on a null/failed evidence agent); fail CLOSED on a missing verifier verdict (cap the maker's self-proposed status) instead of accepting it unverified; add phase('Evidence') + index-safe labels. - critic-panel.js: .filter(Boolean) the verified array before reading f.real; MERGE duplicate findings (keep corroborating critics + evidence) instead of dropping; optional args.uiInScope to skip the 5 UI critics on non-UI changes; optional args.priorEvidence so critics verify against already-gathered evidence instead of re-running; coalesce title before trim. - loop-iteration.js: implementer returns the REAL git diff; the verifier reviews the actual diff (not the maker's self-report); the caller applies the returned patch; add a hard programmatic DENYLIST gate on changedFiles (defense-in-depth); phase('Implement') + index-safe labels. - e2e-design.js: dedup key includes category (don't drop a failure case that shares a title with a happy case); coalesce title before trim. - multi-agent-dev.md / qa-pass.md: pass args.priorEvidence to the post-impl panel. loop.md: apply the returned diff patch for approved items. All scripts re-validated. Memory log records the round + lessons. Docs/config only. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_014xUCrJdREvUMjoVzSX5T8G --- .claude/commands/multi-agent-dev.md | 6 ++- .claude/commands/qa-pass.md | 9 ++-- .claude/loop.md | 8 ++-- .claude/memory/log.md | 18 ++++++++ .claude/workflows/critic-panel.js | 64 ++++++++++++++++++----------- .claude/workflows/e2e-design.js | 4 +- .claude/workflows/loop-iteration.js | 49 ++++++++++++++++------ .claude/workflows/scan-docs.js | 39 ++++++++++++------ 8 files changed, 136 insertions(+), 61 deletions(-) diff --git a/.claude/commands/multi-agent-dev.md b/.claude/commands/multi-agent-dev.md index bf47023..10e5b78 100644 --- a/.claude/commands/multi-agent-dev.md +++ b/.claude/commands/multi-agent-dev.md @@ -34,8 +34,10 @@ Procedure: 9. Add or update Playwright E2E tests for the verified flow; run them. 10. Run typecheck, lint, unit tests, and build if appropriate. 11. **Post-implementation critic round** — invoke the same Workflow again with - `args.focus` set to the changed flow + `git diff`. Fix any remaining confirmed - blockers (re-run the workflow until none remain). + `args.focus` set to the changed flow and `args.priorEvidence` set to what you + already gathered (git diff summary + test/lint/build results + browser/console + findings) so the critics verify against it instead of re-running everything. + Fix any remaining confirmed blockers (re-run the workflow until none remain). 12. Report final evidence. Never mark complete unless: diff --git a/.claude/commands/qa-pass.md b/.claude/commands/qa-pass.md index b55209b..731022c 100644 --- a/.claude/commands/qa-pass.md +++ b/.claude/commands/qa-pass.md @@ -25,10 +25,11 @@ Checklist: - forms have labels - keyboard navigation works for critical flow 12. Run a **post-implementation critic round as a Workflow** — invoke the Workflow - tool with `scriptPath: .claude/workflows/critic-panel.js`, - `args: { "focus": "the changed flows for $ARGUMENTS + git diff" }`. The critics - run as parallel subagents and each blocker/important finding is verified by a - separate skeptic (no single agent simulating the panel). + tool with `scriptPath: .claude/workflows/critic-panel.js` and + `args: { "focus": "the changed flows for $ARGUMENTS", "priorEvidence": "" }`. + The critics run as parallel subagents and verify against that evidence; each + blocker/important finding is checked by a separate skeptic (no single agent + simulating the panel). 13. Report the workflow's `confirmed` blockers/non-blockers and recommended next actions. diff --git a/.claude/loop.md b/.claude/loop.md index 6abd9f7..51cf619 100644 --- a/.claude/loop.md +++ b/.claude/loop.md @@ -33,9 +33,11 @@ budget, denylist, escalation, stop conditions). Before running, fix the loop's - L2/L3: fix the confirmed blockers + directly-related important findings via the **loop-iteration Workflow** (`.claude/workflows/loop-iteration.js`, `args.items` = the triaged, denylist-cleared items): an implementer subagent - fixes each in an isolated worktree and a *separate* verifier approves/rejects. - Apply approved changes; escalate the rest. Never fix nice-to-haves - automatically; never start unrelated refactors. + fixes each in an isolated worktree and a *separate* verifier reviews the real + diff and approves/rejects (a hard denylist gate also forces escalation). For + each **approved** item, apply its returned `diff` patch and re-run the smallest + relevant check before committing; escalate `rejected`/`escalate` items. Never + fix nice-to-haves automatically; never start unrelated refactors. 10. **Append a run entry** to `.claude/memory/loop-run-log.md` (see format below). ## Denylist — never touch autonomously (escalate instead) diff --git a/.claude/memory/log.md b/.claude/memory/log.md index d2546cc..740f361 100644 --- a/.claude/memory/log.md +++ b/.claude/memory/log.md @@ -95,3 +95,21 @@ change), `rotate` (archived old entries). Format: `## [YYYY-MM-DD] |