From 4d91355fcdfcd07b67316052ae66682567506168 Mon Sep 17 00:00:00 2001 From: Jacek Tomaszewski Date: Mon, 24 Aug 2026 18:32:11 +0200 Subject: [PATCH 1/2] feat(docs-audit): audit, fix and gate a repo's documentation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds the `docs-audit` skill: grade a repository's docs and agent instructions against the house conventions, apply the mechanical fixes, propose the judgment calls, and optionally install the CI gate. The failure it targets is silent. An agent loads AGENTS.md from the repo root down to its working directory and stops at 32,768 bytes; everything past that offset is dropped with no warning. A rule at the bottom of a 33 KB file is not low-priority, it is absent — and the symptom is an agent that "ignores the conventions" while a human writes the rule a third time, further down. - `scripts/doc-audit.sh` — read-only, bash 3.2, ~1s on a typical repo: instruction-budget chains, CLAUDE.md/AGENTS.md wiring, the Skill profile, commands that no longer resolve, dead links, unindexed docs, spec convention, state-vs-record candidates, size outliers, whether anything is CI-enforced. Verified against 17 checkouts. - `references/conventions.md` — the rubric, with the failure each rule was bought with, so an edit can be argued for rather than asserted. - `templates/check-agents-md-budget.mjs` — the gate the skill installs into the target repo: root hard limit plus a per-chain ratchet, so existing debt freezes visibly instead of hiding. Co-Authored-By: Claude Opus 5 (1M context) --- .claude-plugin/plugin.json | 4 +- README.md | 5 +- skills/docs-audit/SKILL.md | 112 ++++++ skills/docs-audit/references/conventions.md | 232 +++++++++++ skills/docs-audit/scripts/doc-audit.sh | 374 ++++++++++++++++++ .../templates/check-agents-md-budget.mjs | 136 +++++++ 6 files changed, 859 insertions(+), 4 deletions(-) create mode 100644 skills/docs-audit/SKILL.md create mode 100644 skills/docs-audit/references/conventions.md create mode 100755 skills/docs-audit/scripts/doc-audit.sh create mode 100644 skills/docs-audit/templates/check-agents-md-budget.mjs diff --git a/.claude-plugin/plugin.json b/.claude-plugin/plugin.json index cc9d3aa..5151c33 100644 --- a/.claude-plugin/plugin.json +++ b/.claude-plugin/plugin.json @@ -1,7 +1,7 @@ { "name": "fsh", - "description": "Full Stack House Claude Code skills (idea brainstorming & kickoff, delivery workflow, bug/flake hunting, change explanation, spec/ticket/PR polishing, status reporting, design, document redlines)", - "version": "1.0.0", + "description": "Full Stack House Claude Code skills (idea brainstorming & kickoff, delivery workflow, bug/flake hunting, change explanation, spec/ticket/PR polishing, documentation & agent-instruction audit, status reporting, design, document redlines)", + "version": "1.1.0", "author": { "name": "Full Stack House", "url": "https://fullstack.house" diff --git a/README.md b/README.md index 42b0a33..865b06d 100644 --- a/README.md +++ b/README.md @@ -18,6 +18,7 @@ read what a counterparty changed in a contract. | [`ticket-refresh`](./skills/ticket-refresh/SKILL.md) | `pr-polish` for a tracker ticket: re-verify its body against reality — resolve every linked PR/issue (following supersessions), check whether an "upstream" fix already ships in the installed version, and rewrite claims the world has overtaken, then post one comment so watchers learn what changed. Body + comment only; never touches Status/Assignee. | | [`ticket-polish`](./skills/ticket-polish/SKILL.md) | `ticket-refresh`'s complement: refresh makes a body true, polish makes it legible. Restructure an accreted ticket — one canonical enumeration instead of parallel numbering, open work leads, history compressed to its surviving reasoning, a checkable DoD of only the remaining work, title re-trued. Form only, facts unchanged — it runs `ticket-refresh` itself when the body's facts have gone stale, so a neglected ticket needs one invocation, not two. | | [`spec-polish`](./skills/spec-polish/SKILL.md) | `ticket-polish` for a spec or design doc: a first screen a newcomer can stop after, the decision the doc asks of its reader up front, the argument before the evidence, catalogues and traceability ids moved to appendices, mandated sections kept but in reader order. Form only, facts unchanged — spot-checks the spec's evidence against the code first and refuses to polish stale claims. | +| [`docs-audit`](./skills/docs-audit/SKILL.md) | Audit a repo's documentation and agent instructions against the house conventions, then fix the mechanical and propose the rest: instruction-budget overflow (the rules an agent never receives), `CLAUDE.md`/`AGENTS.md` drift, commands that no longer resolve, dead links, unindexed docs, spec-convention breaks, state docs narrating their own history. Ships a CI gate so the rules hold without re-running it. `--audit-only` writes nothing. | | [`review-queue`](./skills/review-queue/SKILL.md) | Triage every PR awaiting your review: classify the queue, fan out one read-only reviewer subagent per PR, merge into a linked triage table (verdicts, draft comments, cross-PR conflicts). Posts nothing without explicit per-action approval. | | [`bug-hunt`](./skills/bug-hunt/SKILL.md) | Reproduce → diagnose → failing-test → fix a reported bug at the narrowest correct layer. Forbids speculative fixes; files a tracker task on give-up. | | [`flake-hunt`](./skills/flake-hunt/SKILL.md) | Root-cause and fix a flaky Playwright e2e test. Forbids timeouts/retries/skip; files a tracker task on give-up. | @@ -28,7 +29,7 @@ read what a counterparty changed in a contract. | [`bro`](./skills/bro/SKILL.md) | Restate the last message in plain human language — no jargon, one human talking to another. Manual-invoke only. | | [`zoom-out`](./skills/zoom-out/SKILL.md) | Break mid-task tunnel vision: restate the goal from the original request, mark sunk work ignorable, measure the decision space, get a fresh-context second opinion (subagent that never sees the current approach), present 2–3 options-in-kind + a recommendation. Analysis only until the user picks. | -`explain`, `brainstorm`, `kickoff`, `overnight`, `deliver`, `upstream-pr`, `pr-polish`, `ticket-refresh`, `ticket-polish`, `spec-polish`, `review-queue`, `bug-hunt`, `flake-hunt`, `project-status`, `design-polish`, and `design-explore` are **repo-agnostic** — they derive +`explain`, `brainstorm`, `kickoff`, `overnight`, `deliver`, `upstream-pr`, `pr-polish`, `ticket-refresh`, `ticket-polish`, `spec-polish`, `docs-audit`, `review-queue`, `bug-hunt`, `flake-hunt`, `project-status`, `design-polish`, and `design-explore` are **repo-agnostic** — they derive project-specific commands, paths, and policy at runtime (see [Skill profile](#skill-profile) below). A repo with its own sharper, hardcoded variant can keep it in its `.claude/skills/` alongside these (plugin skills are namespaced, so they don't collide — see Install). @@ -106,7 +107,7 @@ lesser extent) derive most specifics at runtime from the consuming repo's `CLAUD (e.g. `docs/specs/YYYY-MM-DD-slug.md`) or a tracker/Notion location, plus how deep a spec is expected to go. `kickoff` writes its spec there when the work warrants one, and `brainstorm` keeps its handoff briefs beside them (`/briefs/`); `spec-polish` - reads the same knob to find the specs on a branch and the sections the repo mandates. Without the + reads the same knob to find the specs on a branch and the sections the repo mandates; `docs-audit` uses it to locate the spec directory it grades for template, index and naming. Without the knob or a discoverable convention, `kickoff` writes the plan into the tracker ticket itself (when a Tracker is configured) or the PR description, and `brainstorm` falls back to `.context/briefs/` when `.context/` exists (otherwise it asks where briefs go). diff --git a/skills/docs-audit/SKILL.md b/skills/docs-audit/SKILL.md new file mode 100644 index 0000000..5399193 --- /dev/null +++ b/skills/docs-audit/SKILL.md @@ -0,0 +1,112 @@ +--- +name: docs-audit +description: Audit a repository's documentation and agent instructions against the house conventions, fix what is mechanical, and leave the rest as a ranked proposal — measured, not eyeballed: instruction-budget overflow (the rules an agent never receives), CLAUDE.md/AGENTS.md drift, stale commands, dead links, unindexed docs, spec-convention breaks, and state docs narrating their own history. Optionally installs the CI gate so the rules hold without re-running it. Use when onboarding a repo, when agents keep ignoring documented rules, when someone asks "is our documentation any good / bring this repo up to standard", or before handing a repo to a new contributor. Args: a repo path (default: the current repo), plus `--audit-only` to write nothing. +--- + +# docs-audit + +You are running the **docs-audit** skill. Goal: make a repository's documentation *work on the reader it actually has* — half of whom are agents that read a fixed number of bytes and then stop. + +The failure this exists to catch is silent. A rule at the bottom of a 33 KB `AGENTS.md` is not low-priority, it is **absent**: the agent's context ends before it. Nobody notices, because the symptom is an agent that "ignores the conventions" and a human who writes the rule a third time, further down. + +**Scope: structure, wiring, indexes and enforcement. Never the claims.** You may move a paragraph, split a file, add an index row, fix a link, install a gate. Correcting what a doc *asserts* is a different job with a different review — report it, don't fold it in. + +The rubric is [`references/conventions.md`](./references/conventions.md). **Read it before Phase 3**; it carries the reasoning each rule was bought with, and you will need that to argue for an edit. + +## 1. Resolve the target and the mode + +- **Target** — the path given, else the current repo. Always work from the repo root (`git rev-parse --show-toplevel`). +- **Mode** — write by default: mechanical fixes applied (Phase 4), judgment calls proposed (Phase 5). `--audit-only` writes nothing at all. +- **Ownership decides how far you go.** `git remote -v` plus the repo's own docs: + - **Ours** — audit and fix. + - **A client's, that we work in** — audit and fix, but their stated conventions win over this rubric (see Hard rule 5). + - **Upstream we don't own** — **propose only**, however wrong it looks. Reformatting a maintainer's `AGENTS.md` in a drive-by PR is how a contribution gets closed unread. Report, and offer to open one narrow PR for the single highest-value fix. +- **Read the repo's own rules first.** A root `AGENTS.md`/`CLAUDE.md`, a `docs/README.md`, a spec-directory `README.md`, `CONTRIBUTING.md`. A repo that has *deliberately* chosen differently is not in violation. + +## 2. Measure + +Facts first — the byte arithmetic and the link graph are not things to eyeball. + +```bash +"${CLAUDE_SKILL_DIR}/scripts/doc-audit.sh" [repo-path] | tee /tmp/doc-audit.txt +``` + +Read-only, a couple of seconds on a large repo. It reports ten sections: instruction-budget chains, `CLAUDE.md`/`AGENTS.md` wiring, the `## Skill profile`, commands that no longer resolve, dead relative links, unindexed docs, spec convention, state-vs-record candidates, size outliers, and whether anything is CI-enforced. + +Two of its sections are **heuristics, not findings** — §8 (state-vs-record) matches prose patterns and will flag a doc that merely *discusses* revision history, and §6 (orphans) flags raw corpora (`sources/`, `_archive/`, imported material) that are legitimately unlinked. Judge each; never paste them through as verdicts. + +The other eight are arithmetic. Trust them. + +## 3. Diagnose + +Read the root doc and every nested agent doc in full. Skim the doc tree. Then name the shapes — these recur: + +- **Fork at the top.** `AGENTS.md` and `CLAUDE.md` both carrying content. They have already drifted or will; the drift shows up as an agent that behaves differently depending on which tool ran it. +- **Claude-only repo.** No `AGENTS.md` anywhere. Every non-Claude agent — Codex, Cursor, most CI review bots — starts this repo with no instructions at all, and nothing in the repo says so. +- **Budget overflow.** A chain over 32 KB. Say *which rules* fall past the cutoff, by reading down to that byte offset — it is nearly always the `Never` list and the validation commands, because they sit at the bottom. +- **Procedure in the routing file.** The root doc carries worked examples, option tables, and step-by-step recipes. Those are what a referenced doc is for; the root's job is hard rules, boundaries, and where to look next. +- **No router.** Three or more nested agent docs and no Task Router table at the root. They get found by accident, if at all. +- **Absent or stale `## Skill profile`.** Every repo-agnostic skill then interrogates the user for the base branch, the check commands, the tracker, the reviewer. +- **Commands that lie.** A documented `yarn typecheck` that no longer exists sends an agent into a diagnostic detour before it can begin. +- **Journal in a state doc.** "corrected 2026-…", "an earlier revision said", a struck-through paragraph left in place, a diagram plus a note explaining the diagram is wrong. See rubric §1 — this is the rule most repos have never stated, and the one that compounds fastest. +- **Doc grown by bug fixes.** A contract or mapping doc that gains a section every time something breaks. The measurements belong in the spec (dated, historical by design); the doc gets the one-line conclusion. +- **Spec sprawl.** No template, no index, reused numbers, or every spec permanently `Draft` — a status field nobody moves is a status field nobody reads. +- **Index as a file listing.** An index that says *what exists* rather than *what each doc is good for* saves nobody a directory walk. +- **Nothing gated.** Every rule above holds only as long as someone re-reads it. See rubric §5. + +Rank by what it costs: **rules that never arrive** > **instructions that mislead** (stale commands, drifted forks) > **friction** (missing profile, no router) > **hygiene** (dead links, orphans, naming). + +## 4. Fix what is mechanical + +Apply these directly — they have one correct answer and a reviewable diff: + +- **The pointer.** Replace a forked or inverted `CLAUDE.md` with the two-line pointer; where only `CLAUDE.md` exists, `git mv` it to `AGENTS.md` and leave the pointer behind. Content unchanged, so the diff is pure wiring. +- **Dead links** — repoint to the moved file, or remove the link if its target is gone. +- **Index rows** for orphans that should be reachable; a one-clause description each, saying what the doc is good for. +- **Spec scaffolding** — the template and the index `README.md` when missing, from the rubric's section list. Backfill the index table from the specs already there. +- **`## Skill profile`** — write the skeleton and fill only what the repo can prove: default branch from `git`, check commands from `package.json` and the CI workflow, reviewer from recent PRs. Leave the rest as explicit `TODO:` lines rather than plausible guesses; a wrong tracker id is worse than a missing one. + +Run whatever the repo uses to validate docs (link checkers, markdown lint) after editing. + +## 5. Propose what is judgment + +These change meaning or lose information. **Show the plan, apply after agreement** — and for a large one, do the first instance and let the user react before the rest. + +- **Splitting an over-budget doc.** Propose the split line: what stays (hard rules, boundaries, router) and what moves to a referenced doc, with the byte count each side lands on. Never summarize a rule while moving it — move the text. +- **Deleting or archiving orphans.** Say which look superseded and by what. +- **Rewriting journal narration.** Quote the passage and give the state-doc replacement beside it. One example is more persuasive than the rule. +- **Handoffs, not duplicates.** A spec that is accurate but unreadable → `/spec-polish`. A ticket → `/ticket-polish`. A stale PR description → `/pr-polish`. Say so; don't do their job here. + +## 6. Enforce + +If the repo is over budget, or within ~15% of it, offer the gate — it is the difference between a rule and a preference: + +```bash +mkdir -p scripts && cp "${CLAUDE_SKILL_DIR}/templates/check-agents-md-budget.mjs" scripts/ +node scripts/check-agents-md-budget.mjs # see where it stands +node scripts/check-agents-md-budget.mjs --update-baseline # only if debt exists already +``` + +Then wire it in: a `package.json` script (`"agents:check-budget"`), and a step in the CI job that already runs lint/typecheck — never a new workflow for one check. + +The baseline is a **ratchet**: chains inside the budget grow freely; a chain already over it may only shrink, and a *new* over-budget chain fails outright. That freezes existing debt visibly instead of hiding it. Record the baseline only when the repo starts out over — a clean repo needs no baseline file. + +## 7. Report + +- **What never reaches an agent** — first, and quantified: which chains, how many bytes, which specific rules fall past the cutoff. This is the finding people act on. +- **Ranked list** of everything else, by the cost ranking in Phase 3. +- **Applied vs. proposed**, as two lists. Every applied edit in one commit per category, so each is separately revertible. +- **Deliberately left alone** — a convention the repo has chosen differently on purpose, a corpus directory that is meant to be unlinked, a client repo where we don't set policy. +- **What needs a human**: a doc whose facts look stale (name it, don't fix it), a rule that contradicts another rule, a spec directory nobody has touched in a year. + +To ship it: `/deliver` (or `/deliver --no-merge` where a human should read the doc changes first). + +## Hard rules + +1. **Never change what a doc claims.** Structure, wiring, indexes and gates only. A factual correction inside a restructuring diff is an unreviewed edit — report it separately. +2. **Never delete a doc on your own judgment.** Propose, with what supersedes it. +3. **Never restructure a repo we don't own.** Report; offer one narrow PR for the single highest-value fix. +4. **The repo's deliberate convention beats this rubric.** Record the conflict in the report and move on. "Different from the house style" is not a finding. +5. **Never paste a heuristic through as a finding.** §6 and §8 of the script output are candidates; you judge them. +6. **Never add a manual "verify X" step** to a checklist as the fix. Gate it (Phase 6) or leave it — a checklist item nobody can fail is a preference with extra words. +7. **Confidentiality.** If the audit report, or any doc you touch, will land anywhere public — a public repo, an upstream PR — apply the gate in this repo's `AGENTS.md`: no client name, repo, path, ticket id, name-carrying identifier or infrastructure of theirs unless already public in their own material. A leak inherited from the old text becomes yours once you re-save the file. diff --git a/skills/docs-audit/references/conventions.md b/skills/docs-audit/references/conventions.md new file mode 100644 index 0000000..8a35a66 --- /dev/null +++ b/skills/docs-audit/references/conventions.md @@ -0,0 +1,232 @@ +# Documentation conventions + +The rubric `docs-audit` grades against. Every rule here was extracted from repos that had +already paid for getting it wrong, so each one carries the failure it prevents rather than a +preference. + +Two tests decide most questions: + +- **Genre** — is this artifact *state* or *record*? The rules invert between them (§1). +- **Budget** — will an agent actually receive this text, or does it fall off the end (§2.2)? + +--- + +## 1. State and record + +Everything written is one of two genres. Decide which before editing. + +**State** — architecture and contract docs, mapping tables, runbooks, every `README`, **the +body of every spec**, and **code comments**. They describe the system *as it is today*. Git +holds the history; the tracker holds the status. A reader must never have to subtract earlier +revisions to work out the current rule. + +**Record** — incident reports, discovery notes from a meeting, options/estimate/proposal docs, +task findings, and each spec's `## Changelog`. These *are* journals, correctly so: a record is +dated, and a dated document is never wrong, only old. **A record is closed** — once its date +passes it is not edited, only superseded by a newer one or moved to an archive directory. The +moment you start amending a record it becomes a stale body with a patch stack on top. + +Never write, in a state doc: + +- **Corrections addressed to the doc** — "corrected 2026-08-04", "an earlier revision quoted…", + "this section used to say the field is empty". State the fact; `git log -p` explains why it + changed. +- **Before/after narration of our own changes** — "before #290 the read ordered only by id", + "which cost 224 records their warning until #288 fixed it". State the invariant instead. +- **One-off migration steps** — "rows synced before #290 pick it up on the next run", plus the + verification query. That belongs in the PR body or the spec. +- **A ticket id on every sentence.** Cite a task only when it is the doc's own subject or an + open item. +- **A strikethrough of a superseded passage**, or a diagram plus a note explaining that the + diagram is wrong. Delete the passage; redraw the diagram. + +Do keep the *reason* behind a counter-intuitive rule, in a sentence — a field whose name lies +about its contents, a column that holds staff-written text despite being called a customer +comment. That is current-state knowledge a future reader needs in order not to "fix" the +mapping wrongly. + +**Budget.** A bug fix should move a state doc's line count by roughly zero, not add a section. +When a finding needs forty lines of measurements to justify itself, the measurements go in the +spec (dated, historical by design) and the doc gets the one-line conclusion plus a link. One +integration-contract doc nearly doubled — 2,800 → 5,000 words — across two bug-fix PRs exactly +this way. + +**A spec is both — keep the seam sharp.** The **body** is state: it describes the design as it +now stands, and you *rewrite* it when the design changes. The **Changelog** is the record: one +line per milestone, newest first, not one per edit. The failure mode is amending the body from +the changelog — a row reading "supersedes this spec's earlier dataset language" means the body +still says the wrong thing and the reader is expected to apply the patch themselves. If a +change invalidates a paragraph, **edit the paragraph** and leave one line in the changelog. + +**Code comments follow the same rule**: a comment explains the code as it is and why a +non-obvious choice is right — never the version it replaced, never the review thread it came +from. `git blame` covers both. A `// TICKET-N — what and why` header on a non-obvious block is +the one place a ticket id earns its keep. A comment block longer than the function it +describes is a spec section in the wrong file. + +--- + +## 2. The agent-instruction layer + +### 2.1 One canonical file + +`AGENTS.md` is canonical. `CLAUDE.md` is a pointer next to it containing exactly: + +``` +@AGENTS.md +``` + +Why this and not the alternatives: + +- **A fork** (two independent files) drifts, silently, and the drift is invisible until an + agent behaves differently depending on which tool was used. +- **A symlink** works, but breaks on Windows checkouts and on export/archive paths, and it + inverts in repos where someone symlinked the wrong direction — leaving the canonical name + as the pointer. +- **A `CLAUDE.md`-only repo** is invisible to every non-Claude agent. Codex, Cursor, and + most CI review bots read `AGENTS.md` and nothing else. + +### 2.2 The instruction budget + +Coding agents load the agent docs from the repository root down to the working directory and +stop once the **combined** size reaches their project-instruction budget. Codex's default +`project_doc_max_bytes` is **32,768 bytes** — a byte budget, not a token budget. Everything +past that offset is silently dropped from the first-turn prompt, with no warning that it +happened. + +Two consequences: + +1. The root file must stay under the budget on its own, or its own tail never reaches any agent. +2. The root also spends the budget the nested files need. The more the root takes, the less of + `packages//AGENTS.md` survives when an agent starts inside that package. + +The working limits: **root ≤ 31,232 B** (the budget minus a 1.5 KB reserve so nested files get +some of it), and **every root-to-leaf chain ≤ 32,768 B**. + +This is not theoretical. Chains measured in the wild: a backend package doc at 33 KB that +exceeds the whole budget by itself, and a monorepo chain totalling 48 KB — a third of it +unreachable, including the "never do X" rules at the bottom of the file. + +**Rule of thumb when editing any agent doc:** hard rules, boundaries and routing stay in the +file; long-form procedure, tables of options and worked examples move into a referenced +document the agent reads on demand. + +### 2.3 Structure + +The root doc, in this order: + +1. **What this repo is** — two or three sentences, including what it is *not*. +2. **Repository structure** — the map, one line per package. +3. **Boundary labels** — `## Always`, `## Ask First`, `## Never`, `## Validation Commands`. + - `Always` — required defaults agents apply without asking. + - `Ask First` — decisions needing a human before changing behavior, scope, dependencies, + branch/deploy flow, or contract surfaces. + - `Never` — prohibited actions and unsafe shortcuts. + - `Validation Commands` — short, real commands that prove the relevant path. +4. **Task Router** — once there are three or more nested agent docs. A table: task shape on + the left, which guide to read on the right. Without it, nested docs are only found by + accident, and an agent that starts at the root never learns the package rules apply. +5. **Conventions** — naming, commit style, language policy. +6. **`## Skill profile`** — the machine-read block (§2.4). + +Nested docs cover local architecture, imports and validation for their subtree, and nothing +the root already says. + +### 2.4 Skill profile + +A `## Skill profile` section in the root doc carries the values repo-agnostic skills would +otherwise have to guess: base branch, per-package check commands, tracker and its status +vocabulary, spec location, reviewer bot, merge policy, status channel and audience. Its +knobs are listed in this repo's README. + +The test of a good profile: a skill invoked in a fresh session asks the user nothing. + +### 2.5 Staleness + +Every command an agent doc prints is a claim. A `yarn typecheck` that no longer exists in +`package.json` sends the agent down a diagnostic rabbit hole before it can start. Check the +referenced scripts, not just the prose. + +--- + +## 3. The doc system + +**One index per directory, and the index carries judgment.** A file list is what `ls` is for. +An index says what each doc is *good for*, so "look at how we did X" resolves in one step +instead of a filesystem safari. An index that needs a paragraph per entry is a sign the +paragraph belongs in the doc itself. + +**Every doc is reachable from an index.** A doc nothing links to is reachable only by someone +who already knows it exists — which means it wants an index row, or deletion. Raw corpora +(imported sources, `_archive/`) are the deliberate exception; say so in the directory's README. + +**Specs.** One directory (`docs/specs/` or `.ai/specs/`), and: + +- **Naming** — `SPEC-{NNN}-{YYYY-MM-DD}-{kebab-title}.md`. Sequential number, creation date, + descriptive title. The number is the citation handle; never renumber, never reuse. (Repos + that skipped the index have collided twice.) +- **A committed template** — `SPEC-000-template.md`, so a new spec starts from the house shape + rather than from whichever spec the author last read. Sections: TLDR, Problem Statement, + Proposed Solution, Data Models, API Contracts, Implementation Approach, Key Design + Decisions, Open Questions, Changelog. Plus a header block: Date, Status, Owner, tracker link. +- **An index** — `README.md` in the spec directory: when to write a spec, the naming rule, the + workflow, and a table of every spec with its status and a one-line description. +- **Status is a lifecycle**, not decoration: Draft → Approved → Implemented → Superseded. A + directory where everything is permanently "Draft" is telling you the status field is unused. + +**Records** get their own home — `docs/archive/`, or a dated filename prefix — so that the +state docs around them stay editable and the records stay closed. + +**Keep docs near execution.** Durable knowledge belongs in the repo that executes on it. +Reusable method belongs wherever the company keeps its handbook. Do not mirror one into the +other without a sync script — an unsynced copy is a second source of truth that ages. + +--- + +## 4. Prose rules + +- **Verdict first, evidence attached.** State the conclusion, then the measurement that earns + it: dates, numbers, PR numbers, file paths. A claim with no evidence reads as an opinion and + gets re-litigated. +- **One clause per row in a table.** If a row needs a paragraph, the paragraph belongs in the + document that row points at. +- **Label cautionary examples as such.** "Go here for X. Cautionary: its root doc is 26 KB, + half of it post-incident essays" is more useful than either praise or silence. +- **Self-limiting sections.** An index that explains how to keep itself honest ("adding a row + is cheap; adding a *section* means this file is turning into the thing it exists to avoid") + survives contact with a year of edits. +- **Language policy is explicit.** Where a repo mixes languages — client-facing writing in one, + engineering in another — state the rule and state which terms stay untranslated. + +--- + +## 5. Enforcement + +A rule nobody can fail is a preference. If a convention matters, gate it: + +- The instruction budget is CI-checkable in ~100 lines (`templates/check-agents-md-budget.mjs` + in this skill). +- A "did you bump the version / update the index" rule is a `git diff` check in a workflow. +- Prefer a **ratchet** to a hard failure on legacy debt: a chain already over budget may only + shrink, so the debt is frozen rather than hidden, and new files still get a hard limit. + +The corollary, and the reason this section exists: **do not add a manual "verify X" checklist +step.** Make the command that consumes X fail loudly instead. + +--- + +## 6. Fast checklist + +| Symptom | Rule | +|---|---| +| `CLAUDE.md` and `AGENTS.md` both have content | §2.1 — pointer, not fork | +| No `AGENTS.md` at all | §2.1 — non-Claude agents read nothing | +| Root doc > 31 KB, or a chain > 32 KB | §2.2 — the tail never arrives | +| Nested docs exist, no Task Router | §2.3 — they're found by accident | +| Skills keep asking which branch / what to run | §2.4 — no Skill profile | +| A documented command no longer exists | §2.5 | +| "corrected 2026-…", "previously this said", `~~struck through~~` | §1 — state doc narrating its past | +| A doc grew a section during a bug fix | §1 — budget | +| Specs with reused numbers, or no template | §3 | +| A doc nothing links to | §3 — index row or delete | +| No CI gate on any of this | §5 | diff --git a/skills/docs-audit/scripts/doc-audit.sh b/skills/docs-audit/scripts/doc-audit.sh new file mode 100755 index 0000000..a72c439 --- /dev/null +++ b/skills/docs-audit/scripts/doc-audit.sh @@ -0,0 +1,374 @@ +#!/usr/bin/env bash +# doc-audit.sh — deterministic documentation facts for the `docs-audit` skill. +# +# doc-audit.sh [repo-path] # defaults to the current directory +# +# Read-only. Reports facts, never verdicts: what the instruction chains weigh, which +# links are dead, which docs nothing points at. Judgment is the skill's job. +# +# Requires git, awk, sed, grep. Only tracked files are considered. +# Targets bash 3.2 (macOS default) — no nested `case` inside `$( )`, no associative arrays. +set -euo pipefail + +CODEX_BUDGET=32768 # Codex `project_doc_max_bytes` — the shared root-to-leaf budget +ROOT_RESERVE=1536 # held back so the root cannot spend the whole budget +ROOT_MAX=$((CODEX_BUDGET - ROOT_RESERVE)) + +case "${1:-}" in + -h|--help) sed -n '2,9p' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;; +esac + +cd "${1:-.}" +git rev-parse --is-inside-work-tree >/dev/null 2>&1 || { + echo "doc-audit: not a git repository — run it inside the repo you are auditing." >&2 + exit 2 +} +cd "$(git rev-parse --show-toplevel)" + +TMP=$(mktemp -d "${TMPDIR:-/tmp}/doc-audit.XXXXXX") +trap 'rm -rf "$TMP"' EXIT + +REPO=$(basename "$(pwd)") +git ls-files '*.md' '*.mdx' | grep -v '/node_modules/' > "$TMP/md" || true +grep -E '(^|/)(AGENTS|CLAUDE)\.md$' "$TMP/md" | sort > "$TMP/agentdocs" || true + +bytes() { if [ -f "$1" ]; then wc -c < "$1" | tr -d ' '; else echo 0; fi; } +dirof() { dirname "$1"; } +count() { wc -l < "$1" 2>/dev/null | tr -d ' ' || echo 0; } + +# The file an agent actually loads for a directory: AGENTS.md when present (a CLAUDE.md +# beside it is expected to be a pointer to it), otherwise CLAUDE.md. +effective_doc() { + if [ "$1" = "." ]; then p=""; else p="$1/"; fi + if [ -f "${p}AGENTS.md" ]; then echo "${p}AGENTS.md" + elif [ -f "${p}CLAUDE.md" ]; then echo "${p}CLAUDE.md" + fi +} + +# Resolve a relative link against the directory of the file containing it, +# collapsing "." and ".." segments. Prints nothing for an unresolvable path. +resolve_link() { + from_dir=$1; link=$2 + case "$link" in + /*) cand=".${link}" ;; + *) if [ "$from_dir" = "." ]; then cand="$link"; else cand="$from_dir/$link"; fi ;; + esac + out="" + oldifs=$IFS; IFS='/' + for seg in $cand; do + case "$seg" in + ""|".") ;; + "..") out=$(dirname "/$out"); out=${out#/} ;; + *) if [ -z "$out" ]; then out="$seg"; else out="$out/$seg"; fi ;; + esac + done + IFS=$oldifs + echo "$out" +} + +# Every relative markdown link in the repo, one per line: STATUS file raw resolved +scan_links() { + while read -r f; do + [ -f "$f" ] || continue + grep -oE '\]\([^)]+\)' "$f" 2>/dev/null | sed -E 's/^\]\(//; s/\)$//' > "$TMP/raw" || true + while read -r raw; do + case "$raw" in + ""|http*|mailto:*|tel:*|file:*|data:*|\#*|'<'*|*'{{'*|*'$'*|*'…'*) continue ;; + esac + target=${raw%%#*}; target=${target%% *} + [ -n "$target" ] || continue + r=$(resolve_link "$(dirof "$f")" "$target") + if [ -e "$r" ]; then printf 'OK\t%s\t%s\t%s\n' "$f" "$raw" "$r" + else printf 'DEAD\t%s\t%s\t%s\n' "$f" "$raw" "$r"; fi + done < "$TMP/raw" + done < "$TMP/md" +} +scan_links > "$TMP/links" || true + +echo "# doc-audit — $REPO ($(git rev-parse --abbrev-ref HEAD) @ $(git rev-parse --short HEAD))" +echo "# tracked markdown: $(count "$TMP/md") files" +echo + +# ------------------------------------------------------------- 1. instruction budget +echo "## 1. Agent-doc chains vs the instruction budget" +echo "# An agent loads the AGENTS.md files from the repo root down to its working directory" +echo "# and stops at $CODEX_BUDGET bytes (Codex's default). Everything past that offset is dropped" +echo "# silently — no warning, no truncation notice. Root hard limit here: ${ROOT_MAX}B." +echo +if [ "$(count "$TMP/agentdocs")" = "0" ]; then + echo "NONE — no AGENTS.md or CLAUDE.md anywhere. Every agent starts this repo cold." +else + while read -r f; do dirof "$f"; done < "$TMP/agentdocs" | sort -u > "$TMP/dirs" + ROOT_DOC=$(effective_doc ".") + if [ -z "$ROOT_DOC" ]; then + echo "NO ROOT DOC — nested agent docs exist but nothing at the repo root." + else + rb=$(bytes "$ROOT_DOC") + if [ "$rb" -gt "$ROOT_MAX" ]; then + echo "root: $ROOT_DOC — ${rb}B / ${ROOT_MAX}B [OVER by $((rb - ROOT_MAX))B]" + else + echo "root: $ROOT_DOC — ${rb}B / ${ROOT_MAX}B [ok, $((ROOT_MAX - rb))B spare for nested files]" + fi + fi + echo + ndirs=$(count "$TMP/dirs") + while read -r leaf; do + deeper=$(awk -v p="$leaf/" 'index($0,p)==1' "$TMP/dirs" | wc -l | tr -d ' ') + if [ "$leaf" != "." ] && [ "$deeper" -gt 0 ]; then continue; fi + if [ "$leaf" = "." ] && [ "$ndirs" -gt 1 ]; then continue; fi + + : > "$TMP/chain" + while read -r d; do + keep=0 + [ "$d" = "." ] && keep=1 + [ "$d" = "$leaf" ] && keep=1 + case "$leaf" in "$d"/*) keep=1 ;; esac + [ "$keep" = 1 ] || continue + doc=$(effective_doc "$d"); [ -n "$doc" ] || continue + printf '%s %s\n' "$(bytes "$doc")" "$doc" >> "$TMP/chain" + done < "$TMP/dirs" + + total=$(awk '{s+=$1} END{print s+0}' "$TMP/chain") + if [ "$total" -gt "$CODEX_BUDGET" ]; then + echo "CHAIN OVER BUDGET: ${total}B — $((total - CODEX_BUDGET))B never reaches an agent working in $leaf" + else + echo "chain ok: ${total}B / ${CODEX_BUDGET}B ($leaf)" + fi + awk '{printf " %8dB %s\n", $1, $2}' "$TMP/chain" + done < "$TMP/dirs" +fi +echo + +# ------------------------------------------------------------------- 2. doc wiring +echo "## 2. CLAUDE.md / AGENTS.md wiring" +echo "# House rule: AGENTS.md is canonical, CLAUDE.md is a pointer that imports it, so" +echo "# every agent reads one file and it cannot drift from itself." +echo +if [ "$(count "$TMP/agentdocs")" = "0" ]; then + echo "n/a" +else + while read -r f; do dirof "$f"; done < "$TMP/agentdocs" | sort -u > "$TMP/dirs2" + while read -r d; do + if [ "$d" = "." ]; then p=""; else p="$d/"; fi + a="${p}AGENTS.md"; c="${p}CLAUDE.md" + if [ -f "$a" ] && [ -f "$c" ]; then + if [ -L "$c" ]; then + echo "symlink $d (CLAUDE.md -> $(readlink "$c"))" + elif [ -L "$a" ]; then + echo "INVERTED $d (AGENTS.md -> $(readlink "$a") — the canonical name is the pointer)" + elif grep -qE '^@AGENTS\.md[[:space:]]*$' "$c" && [ "$(bytes "$c")" -lt 512 ]; then + echo "pointer $d (CLAUDE.md imports @AGENTS.md)" + else + echo "FORK $d (two independent files, $(bytes "$a")B / $(bytes "$c")B — they will drift)" + fi + elif [ -f "$c" ]; then + echo "claude-only $d (no AGENTS.md — Codex and every non-Claude agent read nothing here)" + elif [ -f "$a" ]; then + echo "agents-only $d" + fi + done < "$TMP/dirs2" +fi +echo + +# ----------------------------------------------------------------- 3. skill profile +echo "## 3. Skill profile" +PROFILE_FILE="" +if [ "$(count "$TMP/agentdocs")" != "0" ]; then + while read -r f; do + if grep -q '^## Skill profile' "$f" 2>/dev/null; then echo "$f"; fi + done < "$TMP/agentdocs" | head -1 > "$TMP/profile" || true + PROFILE_FILE=$(cat "$TMP/profile" 2>/dev/null || true) +fi +if [ -n "$PROFILE_FILE" ]; then + echo "present in $PROFILE_FILE" + { awk '/^## Skill profile/{f=1;next} /^## /{f=0} f' "$PROFILE_FILE" | sed 's/^/ /' | head -30; } || true +else + echo "ABSENT — every repo-agnostic skill has to ask for or guess the base branch, the" + echo "check commands, the tracker, the reviewer and the merge policy." +fi +echo + +# ------------------------------------------------------------- 4. command staleness +echo "## 4. Commands referenced in agent docs" +echo "# Only commands written as code (inline backticks or a fenced block) are checked, and" +echo "# each is resolved against the nearest package.json above the doc that names it." +echo +if [ "$(count "$TMP/agentdocs")" != "0" ]; then + : > "$TMP/unresolved"; found=0 + while read -r f; do + fdir=$(dirof "$f") + # code content only: fenced blocks verbatim, plus every inline `code` span + awk ' + /^[[:space:]]*```/ { inf = !inf; next } + inf { print; next } + { line=$0 + while (match(line, /`[^`]+`/)) { + print substr(line, RSTART+1, RLENGTH-2) + line=substr(line, RSTART+RLENGTH) + } } + ' "$f" 2>/dev/null | grep -E '^[[:space:]]*(\$ )?(yarn|pnpm|npm run|bun run) ' > "$TMP/cands" || true + while read -r cand; do + [ -n "$cand" ] || continue + found=1 + rest=$(echo "$cand" | sed -E 's/^[[:space:]]*(\$ )?(yarn|pnpm|npm run|bun run)[[:space:]]+//') + dir="$fdir" + case "$rest" in + "--dir "*|"--filter "*|"-C "*) + dir=$(echo "$rest" | awk '{print $2}') + rest=$(echo "$rest" | sed -E 's/^(--dir|--filter|-C)[[:space:]]+[^[:space:]]+[[:space:]]*//') ;; + esac + script=$(echo "$rest" | awk '{print $1}') + case "$script" in + ""|-*|install|add|remove|dlx|exec|why|run|create|init|link|up|outdated|publish|pack|audit|upgrade|global|workspace|workspaces) continue ;; + esac + pkg=""; d="$dir" + while : ; do + if [ "$d" = "." ] || [ -z "$d" ]; then + [ -f package.json ] && pkg=package.json + break + fi + if [ -f "$d/package.json" ]; then pkg="$d/package.json"; break; fi + d=$(dirname "$d") + done + if [ -z "$pkg" ]; then + echo "UNRESOLVED $f: $cand (no package.json above $dir)" >> "$TMP/unresolved" + else + pkgdir=$(dirof "$pkg") + # a package manager also runs binaries from node_modules/.bin, not just scripts + if grep -qE "\"$script\"[[:space:]]*:" "$pkg"; then : + elif [ -e "$pkgdir/node_modules/.bin/$script" ] || [ -e "node_modules/.bin/$script" ]; then : + else + echo "UNRESOLVED $f: $cand (no \"$script\" script in $pkg, no node_modules/.bin/$script)" >> "$TMP/unresolved" + fi + fi + done < "$TMP/cands" + done < "$TMP/agentdocs" + if [ -s "$TMP/unresolved" ]; then { sort -u "$TMP/unresolved" | head -30; } || true + elif [ "$found" = "1" ]; then echo "clean — every command an agent doc prints still resolves to a real script" + else echo "no package-manager commands found in the agent docs" + fi +else + echo "n/a — no agent docs" +fi +echo + +# ------------------------------------------------------------------- 5. dead links +echo "## 5. Dead relative links" +awk -F'\t' '$1=="DEAD" && $2 !~ /\.mdx$/ {printf "DEAD %s -> %s\n", $2, $3}' "$TMP/links" | sort -u > "$TMP/dead" || true +mdx_dead=$(awk -F'\t' '$1=="DEAD" && $2 ~ /\.mdx$/' "$TMP/links" | wc -l | tr -d ' ') +if [ -s "$TMP/dead" ]; then + head -30 "$TMP/dead" + n=$(count "$TMP/dead") + if [ "$n" -gt 30 ]; then + echo " ($n total — worst files:)" + { sed -E 's/^DEAD ([^ ]+) .*/\1/' "$TMP/dead" | sort | uniq -c | sort -rn | head -8 | sed 's/^/ /'; } || true + fi +else + echo "clean — every relative link resolves on disk" +fi +if [ "$mdx_dead" != "0" ]; then + echo "# plus $mdx_dead in .mdx files, not listed: a docs site resolves links by route, not by" + echo "# path, so check those with the site's own link checker instead." +fi +echo + +# ---------------------------------------------------------------------- 6. orphans +echo "## 6. Docs nothing links to" +awk -F'\t' '{print $4}' "$TMP/links" | sort -u > "$TMP/linked" || true +: > "$TMP/orphans" +while read -r f; do + case "$f" in + README.md|AGENTS.md|CLAUDE.md|CONTRIBUTING.md|CHANGELOG.md|LICENSE.md|SECURITY.md|SPEC.md) continue ;; + */AGENTS.md|*/CLAUDE.md|*/README.md|*/SKILL.md) continue ;; + esac + grep -qxF "$f" "$TMP/linked" || echo "ORPHAN $f ($(bytes "$f")B)" >> "$TMP/orphans" +done < "$TMP/md" +if [ -s "$TMP/orphans" ]; then + head -40 "$TMP/orphans" + echo " ($(count "$TMP/orphans") total)" +else + echo "clean — every doc is reachable from an index" +fi +echo "# an orphan is reachable only by already knowing it exists: it wants an index row, or deletion" +echo + +# ------------------------------------------------------------------------ 7. specs +echo "## 7. Specs" +SPEC_DIR="" +for d in docs/specs .ai/specs specs docs/rfcs docs/adr; do + if [ -d "$d" ]; then SPEC_DIR="$d"; break; fi +done +if [ -z "$SPEC_DIR" ]; then + echo "no spec directory (looked for docs/specs, .ai/specs, specs, docs/rfcs, docs/adr)" +else + git ls-files "$SPEC_DIR/*.md" | grep -vE '/(README|AGENTS|CLAUDE)\.md$' > "$TMP/specs" || true + echo "directory: $SPEC_DIR — $(count "$TMP/specs") specs" + if ls "$SPEC_DIR" 2>/dev/null | grep -qiE '^(SPEC-000|template|_template|000-)'; then + echo "template: present" + else + echo "TEMPLATE MISSING — each new spec starts from whatever the author last happened to read" + fi + if [ -f "$SPEC_DIR/README.md" ]; then + : > "$TMP/notindexed" + while read -r s; do + [ -n "$s" ] || continue + grep -qF "$(basename "$s")" "$SPEC_DIR/README.md" || echo "NOT IN INDEX $s" >> "$TMP/notindexed" + done < "$TMP/specs" + if [ -s "$TMP/notindexed" ]; then head -20 "$TMP/notindexed"; else echo "index: complete"; fi + else + echo "NO INDEX — $SPEC_DIR/README.md absent, so the spec set has no entry point" + fi + : > "$TMP/naming" + while read -r s; do + [ -n "$s" ] || continue + b=$(basename "$s") + if echo "$b" | grep -qE '^SPEC-[0-9]{3}[a-z]?-[0-9]{4}-[0-9]{2}-[0-9]{2}-.+\.md$'; then : + elif echo "$b" | grep -qE '^[0-9]{4}-[0-9]{2}-[0-9]{2}-.+\.md$'; then echo " date-first $b" >> "$TMP/naming" + else echo " OFF-PATTERN $b" >> "$TMP/naming" + fi + done < "$TMP/specs" + if [ -s "$TMP/naming" ]; then echo "naming:"; head -20 "$TMP/naming"; else echo "naming: consistent"; fi + while read -r s; do basename "$s"; done < "$TMP/specs" | grep -oE '^SPEC-[0-9]{3}' | sort | uniq -d > "$TMP/dupes" || true + if [ -s "$TMP/dupes" ]; then sed 's/^/ NUMBER REUSED /' "$TMP/dupes"; fi + echo "statuses:" + while read -r s; do + [ -n "$s" ] || continue + line=$(grep -m1 -iE '^\*\*status\*\*' "$s" 2>/dev/null || true) + if [ -n "$line" ]; then echo "$line" | sed -E 's/^\*\*[Ss]tatus\*\*:?[[:space:]]*//'; else echo "(no status field)"; fi + done < "$TMP/specs" | sort | uniq -c | sort -rn | sed 's/^/ /' +fi +echo + +# ---------------------------------------------------------- 8. state-vs-record smells +echo "## 8. State-vs-record candidates" +echo "# Heuristic, for the reviewer to judge. A state doc (contract, runbook, README, spec" +echo "# body, code comment) describes the system as it is now; git holds the history. These" +echo "# lines look like a doc narrating its own revisions." +echo +git grep -n -iE '(corrected [0-9]{4}-[0-9]{2}-[0-9]{2}|an earlier (revision|draft|version)|this (section|doc|file|page) used to|previously (this|it) (said|was)|used to say|before (PR )?#[0-9]+|correction to earlier|~~[^~]{12,}~~)' -- '*.md' 2>/dev/null \ + | grep -viE '(^|/)(CHANGELOG|HISTORY)' \ + | grep -viE '/(archive|runs|analysis|postmortem)/' \ + | grep -viE '/(rca|postmortem)-' \ + | grep -viE '/(sources|vendor|_archive)/' \ + | grep -vE ':[[:space:]]*[-*0-9.]+[[:space:]]*\[[x ]\]' \ + | head -30 > "$TMP/smells" || true +if [ -s "$TMP/smells" ]; then cat "$TMP/smells"; else echo "clean — no state doc is narrating its own past"; fi +echo + +# --------------------------------------------------------------------- 9. outliers +echo "## 9. Largest docs" +while read -r f; do + [ -f "$f" ] && printf '%s %s\n' "$(bytes "$f")" "$f" +done < "$TMP/md" > "$TMP/sizes" || true +{ sort -rn "$TMP/sizes" | head -10 | awk '{printf " %8dB %s\n", $1, $2}'; } || true +echo + +# ------------------------------------------------------------------ 10. enforcement +echo "## 10. Enforcement" +git ls-files | grep -iE 'check-agents(-md)?-budget|agents-budget|docs?-lint|markdown-?lint|check-lessons' > "$TMP/gates" || true +if [ -s "$TMP/gates" ]; then { sed 's/^/ gate: /' "$TMP/gates" | head -10; } || true; else + echo " NONE — no documentation or agent-instruction rule is enforced by CI in this repo," + echo " so every rule here holds only as long as someone re-reads it." +fi +echo +echo "# end of doc-audit" diff --git a/skills/docs-audit/templates/check-agents-md-budget.mjs b/skills/docs-audit/templates/check-agents-md-budget.mjs new file mode 100644 index 0000000..e427781 --- /dev/null +++ b/skills/docs-audit/templates/check-agents-md-budget.mjs @@ -0,0 +1,136 @@ +#!/usr/bin/env node +/** + * check-agents-md-budget — keep agent instructions inside the budget agents actually read. + * + * An agent loads the AGENTS.md files from the repo root down to its working directory and + * stops once the COMBINED size reaches its project-instruction budget (Codex's default + * `project_doc_max_bytes` is 32,768 bytes). Everything past that offset is dropped from the + * prompt silently — no warning, no truncation notice. So the rules at the bottom of a long + * file are not "lower priority", they are absent. + * + * Two limits: + * 1. Root hard limit — the root file alone, so its own tail always arrives and nested + * files still get a share of the budget. + * 2. Chain ratchet — a root-to-leaf chain inside the budget may grow freely. Once it is + * over, its NESTED files may only shrink. This freezes existing debt + * instead of hiding it, and blocks new debt outright. + * + * Usage: + * node scripts/check-agents-md-budget.mjs # check (exit 1 on violation) + * node scripts/check-agents-md-budget.mjs --update-baseline + * + * Wire it into the CI job that already runs lint/typecheck. Re-record the baseline + * deliberately, in a PR that explains why. + */ +import { execFileSync } from 'node:child_process' +import { existsSync, readFileSync, writeFileSync, statSync } from 'node:fs' +import { dirname, join, resolve } from 'node:path' + +const DEFAULTS = { rootMaxBytes: 31232, chainBudgetBytes: 32768 } +const UPDATE = process.argv.includes('--update-baseline') + +const repoRoot = execFileSync('git', ['rev-parse', '--show-toplevel'], { encoding: 'utf8' }).trim() +process.chdir(repoRoot) +const BASELINE_PATH = resolve(repoRoot, 'scripts/agents-md-budget.baseline.json') + +const baseline = existsSync(BASELINE_PATH) + ? { ...DEFAULTS, chains: {}, ...JSON.parse(readFileSync(BASELINE_PATH, 'utf8')) } + : { ...DEFAULTS, chains: {} } + +const tracked = execFileSync('git', ['ls-files'], { encoding: 'utf8' }) + .split('\n') + .filter((f) => /(^|\/)(AGENTS|CLAUDE)\.md$/.test(f)) + +/** The file an agent actually loads for a directory: AGENTS.md, else CLAUDE.md. */ +function effectiveDoc(dir) { + for (const name of ['AGENTS.md', 'CLAUDE.md']) { + const p = dir === '.' ? name : join(dir, name) + if (existsSync(p)) return p + } + return null +} + +// statSync, not lstatSync: a CLAUDE.md/AGENTS.md symlink must weigh what the agent reads. +const sizeOf = (p) => (existsSync(p) ? statSync(p).size : 0) + +const dirs = [...new Set(tracked.map((f) => dirname(f)))].sort() +if (dirs.length === 0) { + console.log('check-agents-md-budget: no AGENTS.md or CLAUDE.md found — nothing to check.') + process.exit(0) +} + +const rootDoc = effectiveDoc('.') +const rootBytes = rootDoc ? sizeOf(rootDoc) : 0 +const leaves = dirs.filter((d) => !dirs.some((o) => o !== d && o.startsWith(d === '.' ? '' : `${d}/`))) + +const chains = leaves.map((leaf) => { + const parts = dirs + .filter((d) => d === '.' || d === leaf || leaf.startsWith(`${d}/`)) + .map(effectiveDoc) + .filter(Boolean) + .map((doc) => ({ doc, bytes: sizeOf(doc) })) + const total = parts.reduce((s, p) => s + p.bytes, 0) + return { leaf, parts, total, nested: total - rootBytes } +}) + +if (UPDATE) { + const next = { + rootMaxBytes: baseline.rootMaxBytes, + chainBudgetBytes: baseline.chainBudgetBytes, + chains: Object.fromEntries( + chains + .filter((c) => c.total > baseline.chainBudgetBytes) + .map((c) => [c.leaf, { nestedBytes: c.nested }]), + ), + } + writeFileSync(BASELINE_PATH, `${JSON.stringify(next, null, 2)}\n`) + console.log(`check-agents-md-budget: baseline written to ${BASELINE_PATH}`) + console.log(` over-budget chains recorded: ${Object.keys(next.chains).length}`) + process.exit(0) +} + +const failures = [] + +if (!rootDoc) { + failures.push('No root AGENTS.md (or CLAUDE.md). Nested agent docs are never reached from the repo root.') +} else if (rootBytes > baseline.rootMaxBytes) { + failures.push( + `${rootDoc} is ${rootBytes}B, over the ${baseline.rootMaxBytes}B root limit by ${rootBytes - baseline.rootMaxBytes}B.\n` + + ' Move long-form procedure into a referenced doc; keep hard rules and routing here.', + ) +} + +for (const c of chains) { + if (c.total <= baseline.chainBudgetBytes) continue + const recorded = baseline.chains?.[c.leaf]?.nestedBytes + const lost = c.total - baseline.chainBudgetBytes + const shape = c.parts.map((p) => ` ${String(p.bytes).padStart(7)}B ${p.doc}`).join('\n') + if (recorded === undefined) { + failures.push( + `New over-budget chain at ${c.leaf}: ${c.total}B, ${lost}B past the ${baseline.chainBudgetBytes}B budget.\n${shape}`, + ) + } else if (c.nested > recorded) { + failures.push( + `Chain at ${c.leaf} grew: nested files ${recorded}B -> ${c.nested}B. Already over budget, so they may only shrink.\n` + + ` ${lost}B of this chain never reaches an agent working there.\n${shape}`, + ) + } +} + +const overBudget = chains.filter((c) => c.total > baseline.chainBudgetBytes) +console.log( + `check-agents-md-budget: root ${rootBytes}B/${baseline.rootMaxBytes}B, ` + + `${chains.length} chain(s), ${overBudget.length} over the ${baseline.chainBudgetBytes}B budget.`, +) +for (const c of overBudget) { + console.log(` over: ${c.leaf} — ${c.total}B (${c.total - baseline.chainBudgetBytes}B unreachable)`) +} + +if (failures.length) { + console.error('\ncheck-agents-md-budget FAILED:\n') + for (const f of failures) console.error(` - ${f}\n`) + console.error(' Shrink the file, or re-record deliberately: --update-baseline (explain it in the PR).') + process.exit(1) +} + +console.log('check-agents-md-budget: ok') From e0f70b71ffa82dab43d9336f12efceee0be94973 Mon Sep 17 00:00:00 2001 From: Jacek Tomaszewski Date: Mon, 24 Aug 2026 18:48:14 +0200 Subject: [PATCH 2/2] fix(docs-audit): raise execFileSync maxBuffer, correct a hard-rule cross-reference MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit git ls-files exceeds execFileSync's 1 MB default on a large repo — 40k tracked files is ~2 MB of paths — and the gate died with ENOBUFS before it could measure anything. Reproduced, then fixed with an explicit 64 MB buffer. Also points the client-repo bullet at Hard rule 4 (deliberate conventions beat the rubric) instead of 5. Co-Authored-By: Claude Opus 5 (1M context) --- skills/docs-audit/SKILL.md | 2 +- skills/docs-audit/templates/check-agents-md-budget.mjs | 9 +++++++-- 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/skills/docs-audit/SKILL.md b/skills/docs-audit/SKILL.md index 5399193..7af5a5d 100644 --- a/skills/docs-audit/SKILL.md +++ b/skills/docs-audit/SKILL.md @@ -19,7 +19,7 @@ The rubric is [`references/conventions.md`](./references/conventions.md). **Read - **Mode** — write by default: mechanical fixes applied (Phase 4), judgment calls proposed (Phase 5). `--audit-only` writes nothing at all. - **Ownership decides how far you go.** `git remote -v` plus the repo's own docs: - **Ours** — audit and fix. - - **A client's, that we work in** — audit and fix, but their stated conventions win over this rubric (see Hard rule 5). + - **A client's, that we work in** — audit and fix, but their stated conventions win over this rubric (see Hard rule 4). - **Upstream we don't own** — **propose only**, however wrong it looks. Reformatting a maintainer's `AGENTS.md` in a drive-by PR is how a contribution gets closed unread. Report, and offer to open one narrow PR for the single highest-value fix. - **Read the repo's own rules first.** A root `AGENTS.md`/`CLAUDE.md`, a `docs/README.md`, a spec-directory `README.md`, `CONTRIBUTING.md`. A repo that has *deliberately* chosen differently is not in violation. diff --git a/skills/docs-audit/templates/check-agents-md-budget.mjs b/skills/docs-audit/templates/check-agents-md-budget.mjs index e427781..1e4df4a 100644 --- a/skills/docs-audit/templates/check-agents-md-budget.mjs +++ b/skills/docs-audit/templates/check-agents-md-budget.mjs @@ -29,7 +29,12 @@ import { dirname, join, resolve } from 'node:path' const DEFAULTS = { rootMaxBytes: 31232, chainBudgetBytes: 32768 } const UPDATE = process.argv.includes('--update-baseline') -const repoRoot = execFileSync('git', ['rev-parse', '--show-toplevel'], { encoding: 'utf8' }).trim() +// execFileSync's default maxBuffer is 1 MB, and `git ls-files` blows past it on a large +// repo — 40k tracked files is ~2 MB of paths, and the gate dies with ENOBUFS for a reason +// that has nothing to do with the budget it is meant to check. +const EXEC = { encoding: 'utf8', maxBuffer: 64 * 1024 * 1024 } + +const repoRoot = execFileSync('git', ['rev-parse', '--show-toplevel'], EXEC).trim() process.chdir(repoRoot) const BASELINE_PATH = resolve(repoRoot, 'scripts/agents-md-budget.baseline.json') @@ -37,7 +42,7 @@ const baseline = existsSync(BASELINE_PATH) ? { ...DEFAULTS, chains: {}, ...JSON.parse(readFileSync(BASELINE_PATH, 'utf8')) } : { ...DEFAULTS, chains: {} } -const tracked = execFileSync('git', ['ls-files'], { encoding: 'utf8' }) +const tracked = execFileSync('git', ['ls-files'], EXEC) .split('\n') .filter((f) => /(^|\/)(AGENTS|CLAUDE)\.md$/.test(f))