diff --git a/.claude/commands/dream.md b/.claude/commands/dream.md index 97b99ce..b81b1cf 100644 --- a/.claude/commands/dream.md +++ b/.claude/commands/dream.md @@ -65,6 +65,11 @@ $ARGUMENTS `.claude/commands/*.md` has a `description` and a row in the index; **flag** (report, don't auto-remove) any index rows for commands that no longer exist and any near-duplicate commands. +2b. Health-check the **workflow diagrams** (`docs/workflow-graphs.md`) against + `.claude/workflows/*.js`: for each workflow, confirm the diagram's node labels, + model tier (🟒/πŸ”΅/βšͺ/▫️), and edges (parallel/pipeline/gates) match the script's + `CRITICS`/`LENSES`/`DIMENSIONS`/`CATEGORIES` arrays and `FANOUT_MODEL`/`JUDGE_MODEL` + usage; **report (don't auto-fix)** any diagram that has drifted from its script. 3. Auto-fix only additive/clerical issues (broken links, a missing index row, a missing `description`). Everything else β€” removing a dead index row, deleting a command, merging duplicates, or removing/rewriting a fact β€” is **reported only** diff --git a/.claude/commands/improve-skills.md b/.claude/commands/improve-skills.md index c919438..39e3d14 100644 --- a/.claude/commands/improve-skills.md +++ b/.claude/commands/improve-skills.md @@ -21,12 +21,18 @@ $ARGUMENTS instruction-following risk, safety/gating, consistency/DRY, workflow DSL, routing) runs in its own context, in parallel; a *separate* skeptic adversarially verifies every blocker/important finding. Invoking Workflow here is expected. -2. The workflow returns `{ scope, targets, counts, applyReady, needsDesign, niceToHaves, refuted }`. +2. The workflow returns `{ scope, targets, budgetStop, counts, applyReady, needsDesign, niceToHaves, refuted }`. Render `applyReady` and `needsDesign` as a matrix and list `niceToHaves` separately: | Lens | Severity | File | Finding | Evidence | Proposed edit | Edit verified safe? | |---|---|---|---|---|---|---| + **If `budgetStop` is true**, tell the user up front that the adversarial verify phase + was skipped due to the token-budget floor: every `needsDesign` item in this run is + **UNVERIFIED** (not merely design-pending β€” neither its reality nor its edit-safety + was checked), `applyReady` is empty by design, and a follow-up `/improve-skills` pass + should re-run verification before anything is applied. + 3. **Confirmation gate (hard rule).** This command may propose edits to core prompts but must **never apply them without explicit confirmation** β€” the same rule `/dream` follows for creating/rewriting skills. Present the proposed edits and ask diff --git a/.claude/memory/log.md b/.claude/memory/log.md index b67e856..6a0e280 100644 --- a/.claude/memory/log.md +++ b/.claude/memory/log.md @@ -151,3 +151,47 @@ change), `rotate` (archived old entries). Format: `## [YYYY-MM-DD] | `scan-docs` (`/scan-project-docs`), `gap-analysis` (`/story-gap-analysis`), > `e2e-design` (`/multi-agent-e2e`), `loop-iteration` (`/loop` L2/L3), > `improve-skills` (`/improve-skills` β€” meta-critic over our own prompts/workflows). +> Pass `args.models = { fanout, judge }` when invoking any of these to override the +> default `sonnet`/`opus` model tiering (e.g. for cost control on a large fan-out). | Command | When to use | Args | |---|---|---| diff --git a/.claude/workflows/critic-panel.js b/.claude/workflows/critic-panel.js index 522ea73..3e95bbb 100644 --- a/.claude/workflows/critic-panel.js +++ b/.claude/workflows/critic-panel.js @@ -8,16 +8,32 @@ export const meta = { ], } +// Normalize args: the Workflow tool may hand args as an object, a plain string, or a +// JSON-encoded string. Parse the JSON-string case so structured fields (models, +// priorEvidence, uiInScope) resolve instead of silently no-opping. +let a = args +if (typeof a === 'string') { + const s = a.trim() + if (s.startsWith('{') || s.startsWith('[')) { + try { a = JSON.parse(s) } catch { /* keep the string as a plain focus note */ } + } +} + // What to review (focus) + how deep, passed via the Workflow `args`. const focus = - (args && (args.focus || (typeof args === 'string' ? args : null))) || + (a && (a.focus || (typeof a === 'string' ? a : null))) || 'the current app/flow and the changed files' // Optional: evidence the Lead already gathered (test/lint/build output, browser // findings, git diff). Critics verify against it instead of re-deriving from scratch. -const priorEvidence = (args && args.priorEvidence) || null +const priorEvidence = (a && a.priorEvidence) || null // Optional: skip the 5 UI/browser critics for non-UI (backend/docs/config) changes. // Defaults to including them (safe). Callers pass `uiInScope: false` to save fan-out. -const uiInScope = !(args && args.uiInScope === false) +const uiInScope = !(a && a.uiInScope === false) + +// Model tiering (graph-engineering): high-volume fan-out on the fast tier, the +// high-stakes adversarial gate on the strong tier. Override via args.models. +const FANOUT_MODEL = (a && a.models && a.models.fanout) || 'sonnet' +const JUDGE_MODEL = (a && a.models && a.models.judge) || 'opus' // The standard critic roster. `ui: true` marks browser-facing critics. const ALL_CRITICS = [ @@ -70,10 +86,10 @@ const VERDICT_SCHEMA = { // the full set before deduping and verifying). phase('Review') const reviews = await parallel( - CRITICS.map((c) => () => + CRITICS.map((c, i) => () => agent( `You are the **${c.label}** for this project. Read CLAUDE.md and the relevant memory topic page(s), inspect ${focus} (read the code; if a localhost flow is in scope, drive it via Playwright/Chrome MCP and check console + network), then review strictly through your lens:\n${c.lens}\n${priorEvidence ? `\nEvidence already gathered by the Lead β€” verify against it instead of re-deriving from scratch:\n${typeof priorEvidence === 'string' ? priorEvidence : JSON.stringify(priorEvidence)}\n` : ''}\nReturn only findings you can back with concrete evidence. An empty list is a valid, honest answer.`, - { label: `critic:${c.key}`, phase: 'Review', schema: FINDINGS_SCHEMA }, + { label: `critic:${c.key}-${i}`, phase: 'Review', schema: FINDINGS_SCHEMA, model: FANOUT_MODEL }, ).then((r) => ({ critic: c.key, label: c.label, findings: (r && r.findings) || [] })), ), ) @@ -107,7 +123,7 @@ const verified = await parallel( toVerify.map((f, i) => () => agent( `Adversarially verify this ${f.severity} finding from the ${f.criticLabel}. Try to REFUTE it by inspecting the actual code/flow β€” default to real:false if the evidence does not hold up.\n\nTitle: ${f.title}\nEvidence: ${f.evidence}\nRecommendation: ${f.recommendation}`, - { label: `verify:${f.critic}:${i}`, phase: 'Verify', schema: VERDICT_SCHEMA }, + { label: `verify:${f.critic}:${i}`, phase: 'Verify', schema: VERDICT_SCHEMA, model: JUDGE_MODEL }, ).then((v) => ({ ...f, // Fail safe: a missing/failed verdict must NOT confirm the finding. diff --git a/.claude/workflows/e2e-design.js b/.claude/workflows/e2e-design.js index 8d8f539..a13b4f4 100644 --- a/.claude/workflows/e2e-design.js +++ b/.claude/workflows/e2e-design.js @@ -5,7 +5,22 @@ export const meta = { phases: [{ title: 'Design', detail: 'one agent per path category enumerates test cases in parallel' }], } -const flow = (args && (args.flow || (typeof args === 'string' ? args : null))) || 'the target flow' +// Normalize args: object, plain string, or JSON-encoded string. Parse the JSON case +// so structured fields (models) resolve instead of silently no-opping. +let a = args +if (typeof a === 'string') { + const s = a.trim() + if (s.startsWith('{') || s.startsWith('[')) { + try { a = JSON.parse(s) } catch { /* keep the string as a plain flow note */ } + } +} + +const flow = (a && (a.flow || (typeof a === 'string' ? a : null))) || 'the target flow' + +// Model tiering (graph-engineering): this workflow is pure fan-out enumeration +// (the dedup is plain code, not an agent), so every node runs on the fast tier. +// Override via args.models.fanout. +const FANOUT_MODEL = (a && a.models && a.models.fanout) || 'sonnet' const CATEGORIES = [ { key: 'happy', prompt: 'the happy path(s) a user follows when everything works' }, @@ -34,10 +49,10 @@ const CASES_SCHEMA = { phase('Design') const results = await parallel( - CATEGORIES.map((c) => () => + CATEGORIES.map((c, i) => () => agent( `Enumerate concrete Playwright E2E test cases for "${flow}" covering ${c.prompt}. Inspect the routes/components and existing specs first. Each case needs: title, steps, and the visible expected outcome to assert.`, - { label: `e2e:${c.key}`, phase: 'Design', schema: CASES_SCHEMA }, + { label: `e2e:${c.key}-${i}`, phase: 'Design', schema: CASES_SCHEMA, model: FANOUT_MODEL }, ).then((r) => ((r && r.cases) || []).map((x) => ({ ...x, category: c.key }))), ), ) diff --git a/.claude/workflows/gap-analysis.js b/.claude/workflows/gap-analysis.js index fe8ff8f..63c4191 100644 --- a/.claude/workflows/gap-analysis.js +++ b/.claude/workflows/gap-analysis.js @@ -8,7 +8,22 @@ export const meta = { ], } -const scope = (args && (args.scope || (typeof args === 'string' ? args : null))) || 'the whole project' +// Normalize args: object, plain string, or JSON-encoded string. Parse the JSON case +// so structured fields (models) resolve instead of silently no-opping. +let a = args +if (typeof a === 'string') { + const s = a.trim() + if (s.startsWith('{') || s.startsWith('[')) { + try { a = JSON.parse(s) } catch { /* keep the string as a plain scope note */ } + } +} + +const scope = (a && (a.scope || (typeof a === 'string' ? a : null))) || 'the whole project' + +// Model tiering (graph-engineering): per-dimension gap discovery on the fast tier, +// the synthesis/prioritization judgment on the strong tier. Override via args.models. +const FANOUT_MODEL = (a && a.models && a.models.fanout) || 'sonnet' +const JUDGE_MODEL = (a && a.models && a.models.judge) || 'opus' const DIMENSIONS = [ { key: 'docs-not-impl', prompt: 'Requirements in docs (docs/prd.md, docs/stories/*, README) with NO implementation evidence in the codebase.' }, @@ -52,10 +67,10 @@ const ORDER_SCHEMA = { phase('Dimensions') const results = await parallel( - DIMENSIONS.map((d) => () => + DIMENSIONS.map((d, i) => () => agent( `Analyze the gap for "${scope}" along this dimension by reading docs/, src/, e2e/, and the story map:\n${d.prompt}\nReturn only gaps backed by concrete references; an empty list is a valid, honest answer.`, - { label: `gap:${d.key}`, phase: 'Dimensions', schema: GAP_SCHEMA }, + { label: `gap:${d.key}-${i}`, phase: 'Dimensions', schema: GAP_SCHEMA, model: FANOUT_MODEL }, ).then((r) => ({ dimension: d.key, gaps: (r && r.gaps) || [] })), ), ) @@ -71,7 +86,7 @@ log(`${total} gaps across ${DIMENSIONS.length} dimensions`) phase('Synthesize') const synth = await agent( `Given these gaps grouped by dimension, propose the recommended NEXT-implementation order (highest-leverage first), each with a one-line "why". Be concrete.\n\n${JSON.stringify(byDimension)}`, - { label: 'synthesize', phase: 'Synthesize', schema: ORDER_SCHEMA }, + { label: 'synthesize', phase: 'Synthesize', schema: ORDER_SCHEMA, model: JUDGE_MODEL }, ) return { scope, totalGaps: total, byDimension, recommendedOrder: (synth && synth.order) || [] } diff --git a/.claude/workflows/improve-skills.js b/.claude/workflows/improve-skills.js index 949e679..e0c603a 100644 --- a/.claude/workflows/improve-skills.js +++ b/.claude/workflows/improve-skills.js @@ -27,6 +27,7 @@ const scope = const DEFAULT_TARGETS = [ '.claude/commands/*.md', '.claude/workflows/*.js', + 'docs/workflow-graphs.md', // the Mermaid graphs β€” reviewed for drift vs the scripts 'CLAUDE.md', '.claude/skills-index.md', '.claude/memory/index.md', @@ -36,14 +37,19 @@ const DEFAULT_TARGETS = [ const targets = (a && Array.isArray(a.targets) && a.targets.length ? a.targets : DEFAULT_TARGETS) const targetList = targets.join(', ') +// Model tiering (graph-engineering): the fan-out of meta-critics on the fast tier, +// the two-axis adversarial verifier on the strong tier. Override via args.models. +const FANOUT_MODEL = (a && a.models && a.models.fanout) || 'sonnet' +const JUDGE_MODEL = (a && a.models && a.models.judge) || 'opus' + // Meta-critic lenses β€” each reviews the SYSTEM'S OWN instructions, not product code. // `code: true` lenses focus on the workflow .js orchestration; the rest on prose prompts. const LENSES = [ { key: 'clarity', code: false, label: 'Clarity & Actionability Critic', lens: 'Is every instruction unambiguous and written as an imperative the agent will actually follow? Flag vague/hedged steps, undefined terms, missing acceptance criteria, and steps with no checkable outcome.' }, { key: 'instruction-following', code: false, label: 'Instruction-Following Risk Critic', lens: 'Where would an LLM plausibly misread, skip, or over-comply? Conflicting directives, a critical rule buried mid-paragraph, ambiguous ordering, "should" where "must" is meant, or prompts so long the key constraint is lost.' }, { key: 'safety-gating', code: false, label: 'Safety & Gating Critic', lens: 'Are destructive or auto-applied actions properly confirmation-gated and denylist-aware? Flag any prompt that could auto-create/rewrite/delete files, commit/push, or self-modify core prompts WITHOUT an explicit confirmation gate. This is the most important lens for a self-improving system.' }, - { key: 'consistency-dry', code: false, label: 'Consistency & DRY Critic', lens: 'Contradictions or duplication ACROSS CLAUDE.md, the commands, skills-index.md, and the memory wiki. Flag facts/lists stated in multiple places that can drift apart (single-source-of-truth risk), and any command whose body contradicts CLAUDE.md or the wiki.' }, - { key: 'workflow-dsl', code: true, label: 'Workflow DSL & Schema Critic', lens: 'For .claude/workflows/*.js only: fail-safe verifiers (a null/failed verdict must NOT confirm), unique agent labels that include the map index, guarded first pipeline stage before deref, fail-closed defaults, maker/checker given the REAL artifact (not a self-report), valid JSON Schemas, and a meta block that is a pure literal. Flag deviations.' }, + { key: 'consistency-dry', code: false, label: 'Consistency & DRY Critic', lens: 'Contradictions or duplication ACROSS CLAUDE.md, the commands, skills-index.md, and the memory wiki. Flag facts/lists stated in multiple places that can drift apart (single-source-of-truth risk), and any command whose body contradicts CLAUDE.md or the wiki. Also: for each diagram in docs/workflow-graphs.md, verify its node list, model-tier annotations (🟒/πŸ”΅/βšͺ/▫️), and edges match the actual agent() calls (CRITICS/LENSES/DIMENSIONS/CATEGORIES arrays and FANOUT_MODEL/JUDGE_MODEL usage) in the corresponding .claude/workflows/*.js β€” flag any diagram that has drifted from its script.' }, + { key: 'workflow-dsl', code: true, label: 'Workflow DSL & Schema Critic', lens: 'For .claude/workflows/*.js only: fail-safe verifiers (a null/failed verdict must NOT confirm), unique agent labels that include the map index, guarded first pipeline stage before deref, fail-closed defaults, maker/checker given the REAL artifact (not a self-report), valid JSON Schemas, and a meta block that is a pure literal. Also check the graph-engineering conventions: correct FANOUT_MODEL/JUDGE_MODEL tiering (fan-out nodes on the fast tier; gates/judgment/synthesis on the strong tier; any code-writing node explicitly exempted rather than silently downgraded); and, where a budget guard exists, that it fails safe (skips or escalates on a floor breach and never auto-approves unverified work). Flag deviations.' }, { key: 'routing', code: false, label: 'Routing & Discoverability Critic', lens: 'Does each command have a one-line description (for model routing) and a matching skills-index row? Flag near-duplicate commands, dead index rows, missing argument-hints, and heavy/code-changing commands that lack disable-model-invocation where they should be manual-only.' }, ] @@ -85,10 +91,10 @@ const VERDICT_SCHEMA = { // (a barrier: we want the full set before deduping and verifying). phase('Review') const reviews = await parallel( - LENSES.map((c) => () => + LENSES.map((c, i) => () => agent( `You are the **${c.label}** auditing this project's OWN Claude Code configuration β€” its custom prompts, skills, and multi-agent orchestration. You are NOT reviewing product/application code; you are reviewing the instructions that drive the agent.\n\nRead these files (the prompt surface): ${targetList}\n${c.code ? 'Focus on the workflow orchestration scripts (.claude/workflows/*.js).' : 'Focus on the prose prompts (commands, CLAUDE.md, skills-index, loop files).'}\n\nReview strictly through your lens:\n${c.lens}\n\nScope: ${scope}\n\nFor each issue return concrete evidence (a quote or file:line) AND a specific proposedEdit (what to change to what). Propose improvements to the prompts/workflows themselves β€” do not propose product-code changes. An empty list is a valid, honest answer.`, - { label: `meta:${c.key}`, phase: 'Review', schema: FINDINGS_SCHEMA }, + { label: `meta:${c.key}-${i}`, phase: 'Review', schema: FINDINGS_SCHEMA, model: FANOUT_MODEL }, ).then((r) => ({ critic: c.key, label: c.label, findings: (r && r.findings) || [] })), ), ) @@ -118,20 +124,37 @@ log(`${deduped.length} findings from ${LENSES.length} meta-critics (${all.length // a safe improvement (so we never "improve" a prompt into being weaker or less safe). phase('Verify') const toVerify = deduped.filter((f) => f.severity === 'blocker' || f.severity === 'important') -const verified = await parallel( - toVerify.map((f, i) => () => - agent( - `You are a SEPARATE skeptic (not the critic who raised this). Read the actual file and try to REFUTE this ${f.severity} finding about the project's own prompts/workflows.\n\nJudge two things independently:\n1. real β€” is the problem genuine? Default real:false if the evidence does not hold up.\n2. editSafe β€” would the proposed edit be a real improvement that does NOT weaken instruction-following, remove a safety/confirmation gate, or break a workflow? Default editSafe:false if you cannot confirm it is safe.\n\nFile: ${f.file}\nTitle: ${f.title}\nEvidence: ${f.evidence}\nProposed edit: ${f.proposedEdit}`, - { label: `verify:${f.critic}:${i}`, phase: 'Verify', schema: VERDICT_SCHEMA }, - ).then((v) => ({ - ...f, - // Fail safe: a missing/failed verdict must NOT confirm the finding or its edit. - real: v ? v.real : false, - editSafe: v ? v.editSafe : false, - verifyReason: (v && v.reason) || 'verifier returned no verdict β€” not confirmed', - })), - ), -) + +// Budget as a HARD stop (soft + hard controls, graph-engineering): if the turn's token +// target is nearly spent, do NOT spawn the verifier fan-out. Route findings to +// needsDesign (real:true, editSafe:false) so nothing is auto-applied unverified. +const VERIFY_FLOOR = 60_000 // tokens to leave for the verify phase +const budgetStop = !!(budget.total && budget.remaining() < VERIFY_FLOOR) +let verified +if (budgetStop) { + log(`budget floor reached (${Math.round(budget.remaining() / 1000)}k left) β€” skipping adversarial verify; ${toVerify.length} findings routed to needsDesign (unverified, never auto-applied)`) + verified = toVerify.map((f) => ({ + ...f, + real: true, + editSafe: false, // unverified β‡’ not apply-ready; surfaced for human review only + verifyReason: 'unverified β€” token budget floor reached before the verify phase', + })) +} else { + verified = await parallel( + toVerify.map((f, i) => () => + agent( + `You are a SEPARATE skeptic (not the critic who raised this). Read the actual file and try to REFUTE this ${f.severity} finding about the project's own prompts/workflows.\n\nJudge two things independently:\n1. real β€” is the problem genuine? Default real:false if the evidence does not hold up.\n2. editSafe β€” would the proposed edit be a real improvement that does NOT weaken instruction-following, remove a safety/confirmation gate, or break a workflow? Default editSafe:false if you cannot confirm it is safe.\n\nFile: ${f.file}\nTitle: ${f.title}\nEvidence: ${f.evidence}\nProposed edit: ${f.proposedEdit}`, + { label: `verify:${f.critic}:${i}`, phase: 'Verify', schema: VERDICT_SCHEMA, model: JUDGE_MODEL }, + ).then((v) => ({ + ...f, + // Fail safe: a missing/failed verdict must NOT confirm the finding or its edit. + real: v ? v.real : false, + editSafe: v ? v.editSafe : false, + verifyReason: (v && v.reason) || 'verifier returned no verdict β€” not confirmed', + })), + ), + ) +} const niceToHaves = deduped.filter((f) => f.severity === 'nice-to-have') const settled = verified.filter(Boolean) @@ -146,6 +169,7 @@ const refuted = settled.filter((f) => f.real === false) return { scope, targets, + budgetStop, // true β‡’ verify was skipped for budget; applyReady is empty by design counts: { lenses: LENSES.length, confirmedBlockers: confirmed.filter((f) => f.severity === 'blocker').length, diff --git a/.claude/workflows/loop-iteration.js b/.claude/workflows/loop-iteration.js index ee235b6..07808a4 100644 --- a/.claude/workflows/loop-iteration.js +++ b/.claude/workflows/loop-iteration.js @@ -8,13 +8,42 @@ export const meta = { ], } +// Normalize args: object, plain string, or JSON-encoded string. Parse the JSON case +// so structured fields (items, level, models) resolve instead of silently no-opping. +let a = args +if (typeof a === 'string') { + const s = a.trim() + if (s.startsWith('{') || s.startsWith('[')) { + try { a = JSON.parse(s) } catch { /* no structured fields available */ } + } +} + // args.items: [{ id, description }] β€” already triaged and denylist-cleared by the loop. -const items = (args && args.items) || [] -const level = (args && args.level) || 'L2' +const items = (a && a.items) || [] +const level = (a && a.level) || 'L2' + +// Model tiering (graph-engineering): the adversarial verifier (the safety gate) runs +// on the strong tier. The IMPLEMENTER intentionally has NO override β€” it writes real +// code in a worktree, so it inherits the session model rather than being silently +// downgraded (correctness of the change matters more than fan-out throughput here). +// Override the verifier tier via args.models.judge. +const JUDGE_MODEL = (a && a.models && a.models.judge) || 'opus' if (!items.length) { return { error: 'No work items. Triage + denylist-check first, then pass args.items = [{id, description}].', applied: [], rejected: [] } } +// Budget as a HARD stop (soft + hard controls, graph-engineering): worktree +// implementers are expensive, so if the turn's token target is nearly spent, do NOT +// start any of them β€” escalate every item for a later run instead of failing mid-fix. +const ITER_FLOOR = 80_000 // tokens to leave before starting the implement/verify fan-out +if (budget.total && budget.remaining() < ITER_FLOOR) { + return { + level, applied: [], rejected: [], escalate: items.map((it) => it.id), + budgetStopped: true, + note: `budget floor reached (${Math.round(budget.remaining() / 1000)}k left < ${ITER_FLOOR / 1000}k) β€” no items processed; all ${items.length} escalated for a later run`, + } +} + const IMPL_SCHEMA = { type: 'object', additionalProperties: false, required: ['changedFiles', 'diff', 'checksPassed', 'notes'], properties: { @@ -47,6 +76,19 @@ const DENYLIST = [ ] const hitsDenylist = (files) => (files || []).some((f) => DENYLIST.some((re) => re.test(f))) +// Parse the file paths out of the ACTUAL unified diff (the artifact the caller +// applies). The gate must not trust the implementer's self-reported `changedFiles` +// alone: a maker that under-reports its changed files while its diff still touches a +// denylisted path would otherwise slip past. We check the UNION of both. +function parseDiffPaths(diff) { + const paths = [] + const s = String(diff || '') + for (const m of s.matchAll(/^diff --git a\/(.+?) b\/(.+)$/gm)) { paths.push(m[1], m[2]) } + for (const m of s.matchAll(/^\+\+\+ b\/(.+)$/gm)) { if (m[1] !== '/dev/null') paths.push(m[1]) } + for (const m of s.matchAll(/^--- a\/(.+)$/gm)) { if (m[1] !== '/dev/null') paths.push(m[1]) } + return paths +} + phase('Implement') // Pipeline: each item flows implement β†’ verify independently. const outcomes = await pipeline( @@ -58,17 +100,20 @@ const outcomes = await pipeline( ), (impl, item, i) => { const im = impl || {} - // Hard denylist gate β€” independent of the verifier. - if (hitsDenylist(im.changedFiles)) { + // Hard denylist gate β€” independent of the verifier. Check the UNION of the + // self-reported changedFiles AND the paths parsed from the real diff (the artifact + // the caller applies), so an under-reported file list can't bypass the gate. + const gatedFiles = [...(im.changedFiles || []), ...parseDiffPaths(im.diff)] + if (hitsDenylist(gatedFiles)) { return { id: item.id, applied: false, status: 'escalated-denylist', changedFiles: im.changedFiles || [], diff: im.diff || '', checksPassed: !!im.checksPassed, - verdict: 'changed files match the denylist β€” forced escalation regardless of verdict', + verdict: 'changed files or diff paths match the denylist β€” forced escalation regardless of verdict', } } return agent( `You are a SEPARATE verifier (not the implementer). Adversarially review the ACTUAL diff below for correctness, scope creep, and denylist violations, and judge whether the change is safe to apply. Default to pass:false if you cannot confirm it from the diff.\n\nItem: ${JSON.stringify(item)}\nChanged files: ${JSON.stringify(im.changedFiles || [])}\nchecksPassed (self-reported): ${!!im.checksPassed}\nDiff:\n${im.diff || '(no diff returned)'}`, - { label: `verify:${item.id || 'item'}-${i}`, phase: 'Verify', schema: VERDICT_SCHEMA }, + { label: `verify:${item.id || 'item'}-${i}`, phase: 'Verify', schema: VERDICT_SCHEMA, model: JUDGE_MODEL }, ).then((v) => ({ id: item.id, // Fail safe: no verdict / verifier died β†’ not approved. diff --git a/.claude/workflows/scan-docs.js b/.claude/workflows/scan-docs.js index 529576f..c9b09dc 100644 --- a/.claude/workflows/scan-docs.js +++ b/.claude/workflows/scan-docs.js @@ -8,9 +8,19 @@ export const meta = { ], } +// Normalize args: object, plain string, or JSON-encoded string. Parse the JSON case +// so structured fields (stories, models) resolve instead of silently no-opping. +let a = args +if (typeof a === 'string') { + const s = a.trim() + if (s.startsWith('{') || s.startsWith('[')) { + try { a = JSON.parse(s) } catch { /* no structured fields available */ } + } +} + // args.stories: [{ id, title, source, acceptanceCriteria? }] // The caller extracts the story list from the PRD/docs first, then runs this. -const stories = (args && args.stories) || [] +const stories = (a && a.stories) || [] if (!stories.length) { return { error: 'No stories provided. Extract stories from docs first, then pass args.stories = [{id,title,source}].', records: [] } } @@ -23,6 +33,11 @@ const REPO_STATUSES = [ 'Unit tested', 'E2E tested', 'Blocked', 'Deprecated', ] +// Model tiering (graph-engineering): per-story evidence gathering on the fast tier, +// the strict status verifier (the gate) on the strong tier. Override via args.models. +const FANOUT_MODEL = (a && a.models && a.models.fanout) || 'sonnet' +const JUDGE_MODEL = (a && a.models && a.models.judge) || 'opus' + const EVIDENCE_SCHEMA = { type: 'object', additionalProperties: false, @@ -60,7 +75,7 @@ const records = await pipeline( (story, i) => agent( `Gather IMPLEMENTATION and TEST evidence for this user story by searching the codebase and tests (do not write any files):\n${JSON.stringify(story)}\n\nReport which files implement it, whether unit and E2E tests cover it, and the most defensible status. "Not started" is correct when no implementation is found.`, - { label: `evidence:${story.id || 'story'}-${i}`, phase: 'Evidence', schema: EVIDENCE_SCHEMA }, + { label: `evidence:${story.id || 'story'}-${i}`, phase: 'Evidence', schema: EVIDENCE_SCHEMA, model: FANOUT_MODEL }, ), (evidence, story, i) => { // Guard the first-stage result like every sibling workflow does β€” a null/failed @@ -68,7 +83,7 @@ const records = await pipeline( const ev = evidence || {} return agent( `You are a strict status verifier for story ${story.id} (${story.title}). Confirm a status that is justified by REPO evidence only (code + tests). From repo evidence you can confirm at most "E2E tested"; "Browser verified" and "Done" require a separate browser pass and are OUT of scope here β€” never assign them. Downgrade if the proposed status overclaims.\n\nProposed: ${JSON.stringify(ev)}`, - { label: `verify:${story.id || 'story'}-${i}`, phase: 'Verify', schema: VERDICT_SCHEMA }, + { label: `verify:${story.id || 'story'}-${i}`, phase: 'Verify', schema: VERDICT_SCHEMA, model: JUDGE_MODEL }, ).then((v) => ({ id: story.id, title: story.title, diff --git a/docs/workflow-graphs.md b/docs/workflow-graphs.md new file mode 100644 index 0000000..ae677e4 --- /dev/null +++ b/docs/workflow-graphs.md @@ -0,0 +1,143 @@ +# Workflow graphs + +Visual map of every deterministic multi-agent Workflow under `.claude/workflows/`. +Each is a **graph**: nodes = subagents (own context), edges = data hand-offs. This is +the "see the graph before you run it" view (the diagram-first idea from graph +engineering), kept in-repo so it version-controls alongside the scripts. + +**Model tiering** (graph-engineering convention): high-volume fan-out runs on the +**fast tier** (Sonnet); gates, judgment, and synthesis run on the **strong tier** +(Opus). The loop implementer is the deliberate exception β€” it writes real code, so it +inherits the session model instead of being downgraded. All tiers are overridable via +`args.models = { fanout, judge }`. + +> Legend: 🟒 fast tier (Sonnet) Β· πŸ”΅ strong tier (Opus) Β· βšͺ inherits session model Β· +> ▫️ plain code (no agent). Barriers (`parallel`) collect all nodes before the next +> stage; pipelines (`pipeline`) flow each item through independently. + +--- + +## `critic-panel` β€” critic round (`/critic-round`, `/multi-agent-dev`, `/qa-pass`, `/loop`) + +```mermaid +flowchart LR + focus([focus / changed files]) --> R + subgraph R["Review β€” parallel barrier Β· 🟒 Sonnet"] + c1[First-Time User] + c2[UX Flow] + c3[Designer] + c4[Artistic Direction] + c5[Frontend Arch] + c6[QA / E2E] + c7[Accessibility] + c8[Performance] + c9[Security] + c10[Regression] + end + R --> dedup[▫️ merge-dedup
keep corroborating critics] + dedup --> V + subgraph V["Verify β€” one skeptic per blocker/important Β· πŸ”΅ Opus"] + v1[refute finding #1] + v2[refute finding #2] + vn[refute finding #N] + end + V --> out([confirmed matrix + niceToHaves + refuted]) +``` + +> `uiInScope: false` drops the 5 UI-facing critics (First-Time User, UX Flow, Designer, +> Artistic Direction, Accessibility) from the Review fan-out for non-UI (backend / docs / +> config) changes β€” the 10-node graph above is the default/full case. + +## `improve-skills` β€” meta-critic over our own prompts (`/improve-skills`) + +```mermaid +flowchart LR + surface([.claude prompt + workflow surface]) --> R + subgraph R["Review β€” per lens, parallel Β· 🟒 Sonnet"] + l1[Clarity] + l2[Instruction-following risk] + l3[Safety / gating] + l4[Consistency / DRY] + l5[Workflow DSL / schema] + l6[Routing / discoverability] + end + R --> dedup[▫️ merge-dedup] + dedup --> gate{budget floor?} + gate -- "under floor" --> skip[▫️ skip verify β†’
needsDesign, unverified] + gate -- ok --> V + subgraph V["Verify β€” 2-axis skeptic per finding Β· πŸ”΅ Opus"] + v1[real? + editSafe?] + end + V --> out([applyReady + needsDesign + refuted]) + skip --> out +``` + +## `scan-docs` β€” evidence-based story status (`/scan-project-docs`) + +```mermaid +flowchart LR + stories([stories]) --> P + subgraph P["pipeline β€” per story, independent"] + e[🟒 gather code+test evidence] --> v[πŸ”΅ strict status verifier
fail-closed cap] + end + P --> out([per-story status records]) +``` + +## `gap-analysis` β€” docs↔code↔tests gaps (`/story-gap-analysis`) + +```mermaid +flowchart LR + scope([scope]) --> D + subgraph D["Dimensions β€” parallel barrier Β· 🟒 Sonnet"] + d1[docs-not-impl] + d2[impl-not-docs] + d3[impl-not-unit] + d4[impl-not-e2e] + d5[e2e-missing-edges] + d6[ux-design] + d7[ambiguous] + end + D --> s[πŸ”΅ synthesize prioritized order] + s --> out([recommendedOrder]) +``` + +## `e2e-design` β€” Playwright case enumeration (`/multi-agent-e2e`) + +```mermaid +flowchart LR + flow([target flow]) --> D + subgraph D["Design β€” per path category, parallel Β· 🟒 Sonnet"] + d1[happy] + d2[failure] + d3[edge] + d4[accessibility] + d5[regression] + end + D --> dedup[▫️ dedup on category+title] + dedup --> out([concrete test cases]) +``` + +## `loop-iteration` β€” maker/checker for `/loop` L2/L3 + +```mermaid +flowchart LR + items([triaged, denylist-cleared items]) --> budget{budget floor?} + budget -- "under floor" --> esc0[▫️ escalate all] + budget -- ok --> P + subgraph P["pipeline β€” per item, independent"] + impl[βšͺ implementer
isolated worktree, real git diff] --> dl{▫️ denylist gate
.claude / CLAUDE.md / auth / secrets…} + dl -- hit --> escd[escalated-denylist] + dl -- clear --> ver[πŸ”΅ verifier reviews real diff] + end + P --> out([applied + rejected + escalate]) + esc0 --> out +``` + +--- + +## How to regenerate + +These diagrams are hand-maintained to mirror `.claude/workflows/*.js`. When you add a +node, change a tier, or add an edge, update the matching diagram here. `/improve-skills` +(Consistency lens) and `/dream lint` should flag a diagram that has drifted from its +script.