diff --git a/.claude/loop-checklist.md b/.claude/loop-checklist.md new file mode 100644 index 0000000..1341629 --- /dev/null +++ b/.claude/loop-checklist.md @@ -0,0 +1,68 @@ +# Loop readiness checklist + +Ship-rubric for any recurring `/loop` before you raise its trust level. Adapted for +this repo from the "loop engineering" loop-design checklist +(https://github.com/cobusgreyling/loop-engineering). Pair with `.claude/loop.md`. + +## Trust levels (what the loop may DO) +| Level | Scope | Requires | +|---|---|---| +| **L0 — Draft** | Intent documented | §1 | +| **L1 — Report** | Triage → run-log, no code changes | §1–3, §5 | +| **L2 — Assisted** | Small in-scope fixes + verifier; no auto-merge | §1–7 | +| **L3 — Unattended** | Acts without a human watching | all sections | + +## Checklist +**1. Purpose & scope** +- [ ] Single clear goal (one sentence) + explicit non-goals +- [ ] Watched scope defined (repos, branches, PRs, tickets, paths) +- [ ] Phased-rollout plan (report-only before acting) + +**2. Scheduling** +- [ ] Cadence matches urgency; off-hours behavior defined +- [ ] First-run + restart-durability behavior specified +- [ ] Self-stops when the watchlist empties + +**3. Skills** +- [ ] Triage step has a tight output format +- [ ] Action steps match project conventions; build/test commands known + +**4. Maker/checker split** +- [ ] Implementer and verifier are separate passes +- [ ] Implementer cannot mark its own work "done" +- [ ] Verifier runs tests in isolation before approving + +**5. State / memory** +- [ ] Run state documented (`.claude/memory/loop-run-log.md`) +- [ ] Prior state read at every iteration start +- [ ] Outcomes + timestamps written; resolved items pruned; human overrides recorded + +**6. Human handoff** +- [ ] Escalation triggers explicit (max attempts, risk paths, ambiguity) +- [ ] Denylist paths identified (auth, payments, secrets, infra, CI config, migrations) +- [ ] Notify only when action is needed + +**7. Connectors (MCP)** +- [ ] Minimum permissions; bot identity clear on comments +- [ ] Loop may open/update PRs or tickets only if acting (L2+) + +**8. Cost & limits** +- [ ] Token/turn budget estimated; per-run cap + kill switch +- [ ] Append-only run log; remaining budget checked each iteration (against the per-run cap) +- [ ] Max iterations per item per run; max auto-PRs per day + +**9. Observability** +- [ ] Each iteration logged (started, items found, actions, escalations) +- [ ] Success metric chosen; state inspectable without reading chat logs + +**10. Safety** +- [ ] No auto-merge without an explicit allowlist +- [ ] Secrets/env in denylist +- [ ] Flakes handled by root-cause, not retry-only + +## Red flags (stop & fix before raising the level) +- Same PR/issue with >3 automated fix attempts without progress. +- Verifier is the same session as the implementer. +- No run-log (memory loss each iteration). +- Notifications on every run regardless of findings. +- Auto-merge without a path allowlist. diff --git a/.claude/loop.md b/.claude/loop.md index 215d165..0db50e3 100644 --- a/.claude/loop.md +++ b/.claude/loop.md @@ -1,48 +1,79 @@ You are running a recurring multi-agent development watchdog loop for this localhost web app. -On each loop iteration, act as Lead Agent and perform this process: - -1. Identify the current active task from the conversation. -2. Check git status and changed files. -3. Start or reuse the dev server. -4. Open the relevant localhost page in Chrome or Playwright MCP. -5. Check browser console errors. -6. Check failed network requests. -7. Run a critic round using these perspectives: - - - First-Time User Critic - - UX Flow Critic - - Designer Critic - - Artistic Direction Critic - - Frontend Architecture Critic - - QA / E2E Critic - - Accessibility Critic - - Performance Critic - - Security Critic - - Regression Critic - -8. Produce a critic matrix. -9. If blockers exist: - - fix only blockers directly related to the current task - - re-run the smallest relevant test - - re-check browser -10. If important issues exist and are directly related: - - fix them only if low-risk and within scope -11. Do not fix nice-to-have issues automatically. -12. Do not start unrelated refactors. -13. Do not push, deploy, delete data, or modify secrets. -14. Stop when: - - no blockers remain - - browser verification passes - - relevant tests pass - - the current task has clear evidence of completion - -Each loop report must include: - -- current task -- localhost status -- critic matrix -- fixes applied -- browser evidence -- test evidence -- remaining risks +This loop is **hardened with "loop engineering" operating rules** (trust levels, +budget, denylist, escalation, stop conditions). Before running, fix the loop's +**trust level** and stay within it. + +## Trust level (declare it before the first iteration) +- **L1 — Report only.** Triage and report findings; make NO code changes, commits, + or PRs. The default when the level is unstated. +- **L2 — Assisted.** May apply small, in-scope, low-risk fixes with a verifier + (post-implementation critic) confirming; never auto-merge. +- **L3 — Unattended.** May act without a human watching; requires every section of + `.claude/loop-checklist.md` to be satisfied first. + +## Per-iteration process (act as Lead Agent) +1. **Read state first:** open `.claude/memory/loop-run-log.md` (prior iterations, + outcomes, human overrides) and check **remaining budget** against the per-run cap + (turns or tokens — see *Budget & limits*) before doing anything; stop and + escalate if it is already exhausted. +2. Identify the current active task from the conversation. +3. Check git status and changed files. +4. Start or reuse the dev server. +5. Open the relevant localhost page in Chrome or Playwright MCP. +6. Check browser console errors and failed network requests. +7. Run a critic round (First-Time User, UX Flow, Designer, Artistic Direction, + Frontend Architecture, QA / E2E, Accessibility, Performance, Security, + Regression) and produce a critic matrix. +8. **Maker/checker:** the critic that verifies a fix must be a *separate* pass from + the one that made it — the implementer never marks its own work "done". +9. Act **only within the trust level**: + - L1: record findings only. + - L2/L3: fix blockers directly related to the current task; fix important issues + only if low-risk and in scope; re-run the smallest relevant test; re-check + browser. Never fix nice-to-haves automatically; never start unrelated refactors. +10. **Append a run entry** to `.claude/memory/loop-run-log.md` (see format below). + +## Denylist — never touch autonomously (escalate instead) +Auth, payments, secrets/`.env`, infrastructure/deploy, CI workflow config, and +database migrations. Also: do not push, deploy, delete data, modify secrets, or +auto-merge without an explicit allowlist. + +## Escalate to a human (pause and ask) when +- A fix would touch a denylist path. +- An item hits the **max attempts** (see red flags) without progress. +- The right fix is ambiguous or architecturally significant (use AskUserQuestion). +- The budget is exhausted. +Notify only when action is needed — do not ping on a no-op run. + +## Red-flag stop conditions (halt and escalate) +- The same issue/PR has **>3 fix attempts** without forward progress. +- The verifier is the same pass/session as the implementer. +- There is no run-log/state (memory loss between runs). +- You'd notify on every run regardless of findings. +- Auto-merge is enabled without a path allowlist. + +## Budget & limits +- Declare a **per-run cap** with the trust level — a turn budget (e.g. 25 turns) or + a token ceiling — and record remaining budget in every run-log entry; stop and + escalate when reached. +- Max iterations per item per run: small (e.g. 3). Max auto-PRs per day: small. +- A loop with an empty watchlist should self-stop, not spin. + +## Stop when +- No blockers remain, browser verification passes, relevant tests pass, and the + current task has clear evidence of completion — or a stop condition above fires. + +## Run-log entry (append to `.claude/memory/loop-run-log.md`) +``` +## [YYYY-MM-DD HH:MM] L | +- found: +- actions: +- budget: / (turns or tokens) +- escalations: +- evidence: +- next: +``` + +Each loop report must include: current task, trust level, localhost status, critic +matrix, fixes applied, browser evidence, test evidence, escalations, and remaining risks. diff --git a/.claude/memory/log.md b/.claude/memory/log.md index 048253f..571e30e 100644 --- a/.claude/memory/log.md +++ b/.claude/memory/log.md @@ -67,3 +67,12 @@ change), `rotate` (archived old entries). Format: `## [YYYY-MM-DD] | 500-line rotation rule). Added the maker/checker no-self-approve rule + and `Agent isolation: worktree` note to CLAUDE.md. Filed the principle in + topics/workflow.md. diff --git a/.claude/memory/loop-run-log.md b/.claude/memory/loop-run-log.md new file mode 100644 index 0000000..db06164 --- /dev/null +++ b/.claude/memory/loop-run-log.md @@ -0,0 +1,29 @@ +# Loop Run Log + +Append-only history of `/loop` runs (the loop's durable **state**, separate from +the wiki's `log.md` which records `/dream` operations). Read this at the start of +every loop iteration; write an entry at the end. Newest entries at the bottom. + +Entry format (see `.claude/loop.md`): + +``` +## [YYYY-MM-DD HH:MM] L | +- found: +- actions: +- budget: / (turns or tokens) +- escalations: +- evidence: +- next: +``` + +Rules: +- One entry per loop iteration; record outcomes, timestamps, and any human override. +- Prune resolved watchlist items; keep the decision trail. +- **Rotation:** when this file passes **500 lines**, roll older entries into + `.claude/memory/loop-run-log/.md` — **create that directory/file on demand** + if absent (same rule as `log.md`); the file is not `@imported`, so this is about + read/append cost only. + +--- + +_No loop runs recorded yet._ diff --git a/.claude/memory/topics/workflow.md b/.claude/memory/topics/workflow.md index 1b78320..c568494 100644 --- a/.claude/memory/topics/workflow.md +++ b/.claude/memory/topics/workflow.md @@ -10,6 +10,11 @@ (copy `docs/stories/_TEMPLATE.md`); `docs/user-stories.md` is the index and `docs/implementation-status.md` is the evidence dashboard. Don't pile stories into one file — same scaling reason as the memory wiki and log. +- **Loops are hardened ("loop engineering"):** every `/loop` declares a trust level + (L1 report → L2 assisted → L3 unattended), reads/writes + `.claude/memory/loop-run-log.md`, and obeys the denylist + red-flag stop + conditions in `.claude/loop.md`. Raise a level only after `.claude/loop-checklist.md` + passes. The verifier pass is always separate from the implementer (no self-approve). - Custom commands live in `.claude/commands/` (each with a `description` for routing); the grouped "when to use" catalog is `.claude/skills-index.md`. A catalog file must NOT live in `.claude/commands/` — anything there auto-registers diff --git a/.claude/skills-index.md b/.claude/skills-index.md index 7456954..f2a3691 100644 --- a/.claude/skills-index.md +++ b/.claude/skills-index.md @@ -22,7 +22,9 @@ catalog so both you and Claude can route to the right command. - **Map what's built vs documented vs tested** → `/scan-project-docs`, then `/sync-story-status`, `/story-gap-analysis` - **Capture/curate what was learned** → `/dream`, `/memory-audit` -- **Keep watching localhost on an interval** → `/loop` (uses `.claude/loop.md`) +- **Keep watching localhost on an interval** → `/loop` (uses `.claude/loop.md`; + gate trust level with `.claude/loop-checklist.md`, state in + `.claude/memory/loop-run-log.md`) ## Development loop | Command | When to use | Args | diff --git a/CLAUDE.md b/CLAUDE.md index bda4f2c..59cfda0 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -59,7 +59,17 @@ building. Triggers: "write me the `/goal` for this", "turn this into a `/loop`", "what `/goal` should we build based on how this project works?". For recurring/watchdog work, the standing loop prompt is `.claude/loop.md` -(used by `/loop`). +(used by `/loop`). Loops are **hardened** ("loop engineering"): declare a trust +level — **L1 report-only → L2 assisted → L3 unattended** — and stay within it; read +`.claude/memory/loop-run-log.md` at the start of each iteration and append an entry +per iteration; respect the denylist (auth, payments, secrets, infra, CI config, migrations), +the escalation triggers, and the red-flag stop conditions. Before raising a loop's +trust level, satisfy `.claude/loop-checklist.md`. + +**Maker/checker rule:** the pass that verifies a fix must be separate from the pass +that made it — the implementer never marks its own work "done" (post-implementation +critic round is the checker). For loops that mutate files in parallel, run the +implementer via `Agent` with `isolation: worktree`. ## Project type This is a local web application developed and tested on localhost. diff --git a/README.md b/README.md index 5c40ff2..3a55f2b 100644 --- a/README.md +++ b/README.md @@ -17,11 +17,13 @@ The workflow starts with critique, not coding — and ends with proof, not confi ```txt CLAUDE.md Project rules + multi-agent workflow + doc-scanner workflow .claude/ - loop.md Recurring multi-agent watchdog loop (used by /loop) + loop.md Recurring watchdog loop, hardened: trust levels + denylist + stop rules + loop-checklist.md Readiness rubric to raise a loop's trust level (L0→L3) skills-index.md Grouped "when to use which command" catalog (kept in sync by /dream) memory/ Knowledge wiki (Karpathy LLM-wiki pattern), maintained by /dream index.md Small catalog imported by CLAUDE.md (always loaded) log.md Append-only history of wiki operations + loop-run-log.md Append-only /loop run history (the loop's durable state) topics/ One page per subject, read on demand settings.local.json Allowed commands for the workflow commands/ Each command has a description used for routing @@ -38,6 +40,7 @@ CLAUDE.md Project rules + multi-agent workflow + doc-sca story-gap-analysis.md Gap analysis: docs vs code vs E2E docs/ adoption-guide.md How to adopt this template in a real project + loop-integration-guide.md How to add hardened recurring /loop automations to a project prd.md Product requirements (source of record for stories) product-docs-index.md Source documents scanned user-stories.md Story index (one row per story → its file) diff --git a/docs/adoption-guide.md b/docs/adoption-guide.md index 53a840e..fb2e86f 100644 --- a/docs/adoption-guide.md +++ b/docs/adoption-guide.md @@ -444,6 +444,24 @@ E2E coverage, no critic blockers) — once you've wired your own commands (above # → review the /goal it prints, then run it (don't skip to /multi-agent-dev) ``` +### Recurring loops are hardened + +For recurring/watchdog work (`/loop`, used via `.claude/loop.md`), the template +applies "loop engineering" operating rules (adapted from +[loop-engineering](https://github.com/cobusgreyling/loop-engineering)): declare a +**trust level** — **L1 report-only → L2 assisted → L3 unattended** — and keep the +loop inside it; read/write the append-only `.claude/memory/loop-run-log.md` each +iteration; honor the **denylist** (auth, payments, secrets, infra, CI config, migrations), +the **escalation triggers**, and the **red-flag stop conditions** (e.g. >3 fix +attempts on one item, verifier == implementer, auto-merge without an allowlist). +Before raising a loop to a higher trust level, satisfy `.claude/loop-checklist.md`. +Adopters should start every new loop at **L1** and only promote it once the +checklist passes. + +For a full step-by-step (copy list, scope/cadence, budget, loop-pattern recipes, +worktree isolation, and a first-loop walkthrough), see +[`loop-integration-guide.md`](./loop-integration-guide.md). + --- ## Continuous integration (CI) diff --git a/docs/loop-integration-guide.md b/docs/loop-integration-guide.md new file mode 100644 index 0000000..f84f7dc --- /dev/null +++ b/docs/loop-integration-guide.md @@ -0,0 +1,110 @@ +# Integrating hardened recurring loops into an existing project + +A practical guide to adding safe, recurring `/loop` automations ("loop +engineering") to a project that already uses this template's workflow. Adapted from +[cobusgreyling/loop-engineering](https://github.com/cobusgreyling/loop-engineering). + +> **A loop is a recurring goal:** an agent iterates toward an objective on a cadence +> (triage → act → verify → record), escalating to a human when needed. The risk is +> the same project's caveat — *unattended loops make unattended mistakes and token +> costs explode* — so a loop is only as safe as its operating rules. + +**Prerequisite:** you've adopted the base workflow (see +[`adoption-guide.md`](./adoption-guide.md)). This guide adds the *recurring-loop* +layer on top. + +--- + +## 1. What you're adding (copy these) + +```txt +.claude/loop.md Hardened loop operating prompt (trust levels, denylist, stop rules, run-log step) +.claude/loop-checklist.md Readiness rubric to promote a loop's trust level (L0→L3) +.claude/memory/loop-run-log.md Durable, append-only loop run state +``` + +Then merge the loop paragraph from this template's `CLAUDE.md` ("Writing `/goal` and +`/loop` prompts" → the trust-level + maker/checker note) into your `CLAUDE.md`. + +## 2. The operating model in one breath + +Declare a **trust level** and stay inside it; keep **durable state** in the run-log; +obey a **denylist**, **escalation triggers**, **red-flag stop conditions**, and a +**budget cap**; and keep the **verifier separate from the implementer** (no +self-approve). + +| Level | The loop may… | Promote only when | +|---|---|---| +| **L1 — Report** | triage → write the run-log; **no** code changes | checklist §1–3, §5 | +| **L2 — Assisted** | small in-scope fixes with a verifier; **no** auto-merge | checklist §1–7 | +| **L3 — Unattended** | act without a human watching | **all** checklist sections | + +## 3. Step-by-step + +1. **Copy the three files** above; merge the `CLAUDE.md` loop guidance. +2. **Define purpose & scope** for the loop: one-sentence goal, explicit non-goals, + and the watched scope (which repos, branches, PRs, tickets, paths). Fill §1 of + `.claude/loop-checklist.md`. +3. **Set cadence & lifecycle:** an interval that matches urgency, first-run and + restart behavior, off-hours behavior, and **self-stop when the watchlist empties**. +4. **Adapt the denylist to your repo:** keep auth, payments, secrets/`.env`, + infra/deploy, CI config, and DB migrations off-limits; add anything + project-specific. Adjust the localhost/command references to your stack. +5. **Set the budget & limits:** a **per-run cap** (turn budget, e.g. 25 turns, or a + token ceiling), max iterations per item, max auto-PRs/day, and a kill switch. + Record remaining budget in each run-log entry. +6. **Wire the state:** read `.claude/memory/loop-run-log.md` at the **start of each + iteration** and append an entry at the **end of each iteration**; prune resolved + items; record human overrides. (Rotates past 500 lines, like the memory log.) +7. **Start at L1 (report-only).** Run it, read the run-log, confirm it triages + correctly and changes nothing. Notifications should fire **only** when action is + needed — not every run. +8. **Promote to L2** only after checklist §1–7 pass: allow small, in-scope, low-risk + fixes, each confirmed by a **separate verifier** pass (the implementer never + marks its own work done); still no auto-merge. +9. **Promote to L3** only after **all** checklist sections pass and you trust it + unattended. Re-read the red flags first. + +## 4. Pick a loop pattern (recipes) + +Common recurring loops — each is a `/loop` with its own scope, cadence, and budget: + +| Pattern | What it does | Start at | Cadence | +|---|---|---|---| +| **Daily triage** | scan for issues/regressions, report | L1 | 1–2 h | +| **PR babysitter** | watch a PR's CI + reviews, fix red, re-push | L2 | 5–15 m | +| **CI sweeper** | keep `main` green | L2 | 5–15 m | +| **Dependency sweeper** | propose dep bumps with passing `verify` | L1→L2 | 6 h–1 d | +| **Changelog / post-merge cleanup / issue triage** | housekeeping | L1 | 1 d | + +> In this template, **PR babysitter** is already available via +> `subscribe_pr_activity` + a scheduled self check-in — the same maker/checker + +> escalation rules apply. + +## 5. Parallel / file-mutating loops → worktrees + +If a loop (or its sub-agents) mutates files in parallel, run the **implementer via +`Agent` with `isolation: worktree`** so concurrent work can't clobber a shared tree, +and have the **verifier** run tests in its own isolation before approving. + +## 6. Red flags — stop and fix before raising a level + +- Same PR/issue with **>3 fix attempts** without progress. +- Verifier is the **same session** as the implementer. +- **No run-log** (memory loss between iterations). +- Notifications on **every** run regardless of findings. +- **Auto-merge** without a path allowlist. + +## 7. Adapt to where the loop runs + +- **Local / Claude Code on the web:** `/loop ` drives `.claude/loop.md`; + the run-log is the durable state across iterations. +- **CI / scheduled (GitHub Actions, cron):** the same operating rules apply; the + run-log lives in the repo, and the denylist/budget/escalation must be explicit + because no human is watching. + +## 8. Before you call a loop "ready" + +Run through [`.claude/loop-checklist.md`](../.claude/loop-checklist.md) and only +raise the trust level when its sections for that level pass. **When in doubt, stay +at L1** — report-only loops are almost always safe; acting loops are not.