diff --git a/.claude-plugin/plugin.json b/.claude-plugin/plugin.json index c411c11..2e98ad8 100644 --- a/.claude-plugin/plugin.json +++ b/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "throughline", - "version": "3.54.0", + "version": "3.55.0", "description": "Throughline plugin: a PRD/TDD/ADR design-doc pipeline with phase-gate PRs, layered as a thin governance overlay on top of the official superpowers/pr-review-toolkit plugins (it owns governance; they own discovery + engineering). /prd-author requires an observable acceptance criterion per new requirement. /tdd-author decides how many TDDs a PRD change needs (git-diff + coverage), recommends ADR actions, requires an alternatives analysis for every new dependency, requires a verification plan (observable surface → observation point(s) → expected observations) per TDD, reads BLOCKERS.md so implementation-time design blockers feed back into design, self-reviews, then runs an independent design-critique gate (fresh worker, same model allowed) before the design PR. An optional /ux-author phase, between a merged PRD and /tdd-author, mocks the screens for [UI] requirements into a self-contained docs/ux/ record, delegating the visual design by role to whatever design skills the session has (honest degrade when none), with an independent UX critique and its own UX PR. /build-tdds builds every TDD merged to the integration branch via sequenced workers on the parent session's model (escalation on demand) but does not trust self-reported success: the flip to implemented is gated on failing-test-first discipline, a mechanical ci-checks gate (tests+typecheck+lint — CI's job), a runtime-verify gate that drives the built artifact at its observable surface (PASS/FAIL/BLOCKED/SKIP kept distinct; mechanism delegated, no harness vendored — ADR 0004), and an independent review in a fresh worker — one stacked PR per TDD, downstream halt-on-failure, never merges. Includes a toolchain bootstrap skill and a format+lint hook for JS/TS, Python, Rust, and Go.", "author": { "name": "Chris Henesy" diff --git a/.claude-plugin/releases.json b/.claude-plugin/releases.json index ea353b1..caae692 100644 --- a/.claude-plugin/releases.json +++ b/.claude-plugin/releases.json @@ -1,4 +1,5 @@ [ + { "version": "3.55.0", "local_impacting": false }, { "version": "3.54.0", "local_impacting": false }, { "version": "3.53.0", "local_impacting": false }, { "version": "3.52.0", "local_impacting": false }, diff --git a/agents/requirements-reviewer.md b/agents/requirements-reviewer.md new file mode 100644 index 0000000..8c0b075 --- /dev/null +++ b/agents/requirements-reviewer.md @@ -0,0 +1,106 @@ +--- +name: requirements-reviewer +description: Independent review of a PRD change (the new and changed requirements in docs/PRD.md) BEFORE the PRD PR is opened. Checks each delta requirement for the requirement-quality characteristics, the delta against the whole PRD for contradiction, duplication and terminology drift, re-checks recorded overrides, and grades the PRD's evaluation rubric. Use at /prd-author close-out. +tools: Read, Grep, Glob, Bash +model: inherit +--- +You are a senior requirements engineer doing an INDEPENDENT review of a PRD +change. You did not author it. You run in a fresh context that is not the +author's session; that fresh context is the independence. You may be on the +same model as the author. Do not spawn children. + +Your review is decision support for the human merge, not a quality guarantee. +Spend your judgment where a model is needed: the mechanical pre-pass +(`prd-lint`) has already run and was clean of blocking findings. + +## Inputs (from the dispatch prompt) +- the repo root and the PRD path (`docs/PRD.md`); +- the `prd-lint: base ` line and any degrade line (`base is HEAD …`, + `no base …`, `base unparseable …`); with a degrade line, apply judgement + about which requirements are really new; +- the delta lines (`new`, `changed`, `removed`). + When the dispatch gives no delta lines (the pre-pass was unavailable), + derive the delta yourself from `git diff -- docs/PRD.md`, or treat + the whole PRD as new when there is no base; +- the prd-lint `vague.term` info lines; +- the report-file path. + +## Rules +1. **Scope.** Check only the `new` and `changed` ids, one by one. Never + re-litigate earlier requirements. Read the whole PRD for the set-level + checks. A `removed` id is in scope only as a possible lost requirement + (for example a rename that dropped content). +2. **Per-requirement checks.** For each delta requirement, check that it is: + - necessary: it traces to a stated user goal or problem; + - unambiguous: there is exactly one reasonable reading; + - complete: no missing actor, trigger or outcome; + - singular: one need per requirement; + - feasible: no conflict with the stated constraints; + - verifiable: the `— Acceptance:` clause names an observable surface + (what a user or program would see), never "a test exists" (FR-24); + - conforms to the template: `**- .** <text> — Acceptance: …` + under `## Requirements`. +3. **Set-level checks** of the delta against the whole PRD: + - contradiction with an existing requirement, constraint or non-goal; + - duplication of an existing requirement; + - terminology drift, where the same concept is named differently. +4. **Readers' perspectives.** Read the delta as the product owner, an + implementer, a tester and a stakeholder. Report any finding that only one + of them would raise. +5. **Rubric.** Grade the PRD's `## Evaluation rubric` rows (FR-77) as + High-quality, Acceptable or Failing, citing each row's criterion. A + `Failing` grade on any row is a BLOCK. +6. **Findings.** Each finding is one line in exactly the shared format: + + `R-n <severity> — docs/PRD.md:<line> — <finding>` + + Number findings `R-1, R-2 …` in report order. Severity is one of + blocker, major, minor or nit. `<line>` is the PRD line the finding is + about (the requirement's title line when it is about the whole + requirement). + `<finding>` is one sentence stating the defect. Why it matters follows on + the next line, indented by two spaces. + + The prd-lint `vague.term` info lines are inputs, not findings. Each is a + finding only if the term really leaves the requirement unverifiable. +7. **Overrides (FR-105).** Read the `## Open questions` lines of the form + + `- "<finding>" — overridden: <rationale>` + + The quote is the `<finding>` sentence only, with no `R-n`, severity or + citation, so renumbering and line shifts don't break it. + - When you raise an issue that an override covers (judged by meaning, not + by exact wording), repeat that override's quoted `<finding>` text word + for word and report it as `overridden`, non-blocking: + `R-n <severity> — docs/PRD.md:<line> — overridden: "<finding>"`. + - That doesn't apply when the rationale is boilerplate: it restates the + decision, or gives no reason specific to the finding. Report the finding + as `R-n blocker — docs/PRD.md:<line> — override rejected: "<finding>"`, + with `<why>` on the indented second line like every finding. +8. **Non-findings (FR-110).** You may set a candidate finding aside only by + quoting it, under `### Non-findings`, one per line in the shared format: + + `<label> — "<quoted candidate finding>" — <why set aside>` + + `<label>` is `pedantic escalation` (a style preference raised as a defect) + or `imaginary architecture` (a requirement nobody stated, such as + multi-tenancy). Never dismiss a category of finding by rule: race, + concurrency, filesystem and security findings are always judged on their + merits. +9. **Calibration.** Block only on a blocker or major finding that would make + an implementer build the wrong thing, or leave a tester unable to tell + whether the requirement is met. Approve otherwise; list minor findings + and nits without blocking. Do not invent issues to look thorough; "no + material findings" is a valid result. +10. **Verdict.** Write your full report to the + report-file path given in the dispatch prompt, overwriting it. Its + sections, in order: `### Findings` (or "no material findings"), + `### Overrides` (each override and whether it was accepted), + `### Rubric`, `### Non-findings`. The LAST line must be + exactly one of: + - `REQUIREMENTS_REVIEW: PASS` + - `REQUIREMENTS_REVIEW: BLOCK <one-line reason>` + + Also return the same text. The report file, not your returned text, is + what the skill reads; a missing file or any other last line counts as + BLOCK. diff --git a/docs/tdd/0073-requirements-review-gate.md b/docs/tdd/0073-requirements-review-gate.md index 32d78f9..54e1674 100644 --- a/docs/tdd/0073-requirements-review-gate.md +++ b/docs/tdd/0073-requirements-review-gate.md @@ -1,6 +1,6 @@ # TDD 0073: Requirements review gate — coverage scan, PRD pre-pass, independent reviewer, overrides -Status: draft +Status: implemented PRD refs: FR-102, FR-103, FR-104, FR-105, FR-110, FR-87, NFR-3 PRD-rev: d526c55 ADR constraints: 0006, 0010, 0011, 0015 diff --git a/scripts/lib/prd-lint.sh b/scripts/lib/prd-lint.sh new file mode 100644 index 0000000..c594196 --- /dev/null +++ b/scripts/lib/prd-lint.sh @@ -0,0 +1,94 @@ +#!/usr/bin/env bash +# prd-lint.sh — the mechanical PRD pre-pass (TDD 0073; FR-103, FR-105). +# Blocks on structural defects in docs/PRD.md without spending model time, and +# computes the delta (the requirement ids this change adds or alters) that the +# requirements reviewer (agents/requirements-reviewer.md) is scoped to. +# +# tl_prd_base <repo-root> <out-file> base docs/PRD.md → <out-file>; rc 0 base, +# rc 1 no base (degrade line on stdout), +# rc 2 a real git failure +# tl_prd_lint <repo-root> `prd-lint: base <ref|none>`, findings, +# `prd-lint: delta <n> requirements, <m> major, <k> info` +# rc 0 clean | 1 major | 2 error | 3 no python3 +# tl_prd_delta <repo-root> new/changed/removed<TAB><id>; same rc rules +# +# The python half (prd_lint.py) is stdlib-only and runs as `python3 -I -B`; it +# shares ux_record.py's requirement grammar. Unlike ux.sh there is no +# short-circuit: every PRD needs the parser, so no python3 is rc 3. Sourced, +# never executed: functions only, no top-level side effects beyond its deps. + +# Absolute at source time: helpers later `cd` into the linted repo, so a +# relative dir would resolve to that repo's own scripts/lib/prd_lint.py. +_TL_PRD_LINT_DIR="$(cd "${BASH_SOURCE[0]%/*}" 2>/dev/null && pwd -P)" || { + echo "prd-lint: cannot resolve its own directory" >&2; return 2 2>/dev/null || exit 2; } +for _pl_dep in plugin-root.sh verdicts.sh; do + # shellcheck source=/dev/null + { [ -r "$_TL_PRD_LINT_DIR/$_pl_dep" ] && . "$_TL_PRD_LINT_DIR/$_pl_dep"; } || { + echo "prd-lint: cannot source $_TL_PRD_LINT_DIR/$_pl_dep" >&2 + unset _pl_dep + return 2 2>/dev/null || exit 2 + } +done +unset _pl_dep + +# tl_prd_base <repo-root> <out-file>. Sets _TL_PRD_BASE_REF (the ref, or empty) +# and _TL_PRD_BASE_DEGRADED (1 only when the base is missing because the repo +# is outside git). Degrade lines go to stdout so the skill can copy them into +# the PR body; it never stops the skill except on a real git failure (rc 2). +tl_prd_base() { + local root="${1:-}" out="${2:-}" ref fallback=0 tree rc + _TL_PRD_BASE_REF=""; _TL_PRD_BASE_DEGRADED=0 + [ -n "$root" ] && [ -n "$out" ] || { echo "prd-lint: usage: tl_prd_base <repo-root> <out-file>" >&2; return 2; } + if ! git -C "$root" rev-parse --is-inside-work-tree >/dev/null 2>&1; then + echo "prd-lint: no base (not a git work tree); every requirement is new" + _TL_PRD_BASE_DEGRADED=1 + return 1 + fi + ref="$(_tl_integration_ref "$root")" || { ref=HEAD; fallback=1; } + if ! git -C "$root" rev-parse -q --verify "${ref}^{commit}" >/dev/null 2>&1; then + echo "prd-lint: no base (no commits); every requirement is new" + return 1 + fi + [ "$fallback" -eq 1 ] && echo "prd-lint: base is HEAD (no integration branch)" + _TL_PRD_BASE_REF="$ref" + # ls-tree paths are relative to the current directory: -C is required. + tree="$(git -C "$root" ls-tree "$ref" -- docs/PRD.md)"; rc=$? + [ "$rc" -eq 0 ] || { echo "prd-lint: git ls-tree failed (rc $rc)" >&2; return 2; } + [ -n "$tree" ] || return 1 + git -C "$root" show "${ref}:docs/PRD.md" >"$out"; rc=$? + [ "$rc" -eq 0 ] || { echo "prd-lint: git show failed (rc $rc)" >&2; return 2; } + return 0 +} + +# _tl_prd_run <check|delta> <repo-root> — shared body of tl_prd_lint/tl_prd_delta. +_tl_prd_run() { + local mode="$1" root="${2:-}" py tmp msgs brc rc base label + [ -n "$root" ] || { echo "prd-lint: usage: tl_prd_$([ "$mode" = check ] && echo lint || echo delta) <repo-root>" >&2; return 2; } + command -v python3 >/dev/null 2>&1 || { echo "prd-lint: python3 required" >&2; return 3; } + py="$_TL_PRD_LINT_DIR/prd_lint.py" + [ -r "$py" ] || { echo "prd-lint: cannot read $py" >&2; return 2; } + [ -f "$root/docs/PRD.md" ] && [ -r "$root/docs/PRD.md" ] \ + || { echo "prd-lint: cannot read $root/docs/PRD.md" >&2; return 2; } + tmp="$(mktemp)" && msgs="$(mktemp)" || { echo "prd-lint: mktemp failed" >&2; rm -f "${tmp:-}"; return 2; } + tl_prd_base "$root" "$tmp" >"$msgs"; brc=$? + case "$brc" in + 0) base="$tmp"; label="$_TL_PRD_BASE_REF" ;; + 1) base=none; label=none ;; + *) rm -f "$tmp" "$msgs"; return 2 ;; + esac + if [ "$mode" = check ]; then + echo "prd-lint: base $label" + cat "$msgs" + set -- check docs/PRD.md "$base" + [ "${_TL_PRD_BASE_DEGRADED:-0}" = 1 ] && set -- "$@" --degraded + else + cat "$msgs" >&2 + set -- delta docs/PRD.md "$base" + fi + ( cd "$root" && python3 -I -B "$py" "$@" ); rc=$? + rm -f "$tmp" "$msgs" + return "$rc" +} + +tl_prd_lint() { _tl_prd_run check "$@"; } +tl_prd_delta() { _tl_prd_run delta "$@"; } diff --git a/scripts/lib/prd_lint.py b/scripts/lib/prd_lint.py new file mode 100644 index 0000000..df031c0 --- /dev/null +++ b/scripts/lib/prd_lint.py @@ -0,0 +1,313 @@ +"""prd_lint.py — the mechanical PRD pre-pass (TDD 0073; FR-103, FR-105). + +Run as `python3 -I -B prd_lint.py <subcommand> …` from scripts/lib/prd-lint.sh. + + delta <prd> <base-file|none> new / changed / removed requirement ids + check <prd> <base-file|none> [--degraded] findings, one per line, then a summary + +`none` means there is no base: every requirement is new. Stdin is never read. +`--degraded` (passed by prd-lint.sh when the base is missing only because the +repo is outside git) reports acceptance.missing and placeholder as info. + +Findings print as `<path>:<line> <severity> <code>: <message>` (tdd-lint's +format). rc 0 = no major finding, rc 1 = at least one major, rc 2 = a parse or +internal error. The requirement grammar is ux_record.parse_requirements, so +this lint and tl_ux_ui_reqs cannot disagree. stdlib only. +""" +import difflib +import os +import re +import sys + +# -I drops the script dir from sys.path; load the sibling module explicitly, +# exactly as ux_render.py does. +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from ux_record import (CODE_RE, FENCE_RE, REQ_RE, RETIRED_RE, UxError, # noqa: E402 + _delist, _read_lines, parse_requirements) + +# The placeholder tokens: to-be-decided, to-do, triple question mark. +PLACEHOLDER_RE = re.compile(r"(?<![A-Za-z0-9_])(?:TBD|TODO)(?![A-Za-z0-9_])|\?\?\?") +ACCEPT_RE = re.compile(r"(?:—|--)\s+Acceptance:") +MALFORMED_RE = re.compile(r"^\*\*[A-Za-z]+ ?- ?[0-9]") +OVERRIDE_RE = re.compile(r'^- "(?P<finding>.+)" (?:—|--) overridden: (?P<rationale>.*)$') +HEADING_RE = re.compile(r"^(#{1,6})\s") +REQ_SECTION_RE = re.compile(r"^##\s+Requirements\b") +OQ_SECTION_RE = re.compile(r"^##\s+Open questions\b") +# A sentence ends at . ! or ? followed (after any closing emphasis/quote) by +# whitespace or the end; a requirement id's digits are not a quantity. +SENT_END_RE = re.compile(r"[.!?](?=[*_)\"'\]]*(?:\s|$))") +ID_TOKEN_RE = re.compile(r"\b[A-Z][A-Z0-9]*-[0-9]+\b") +DIGIT_RE = re.compile(r"[0-9]") +VAGUE_TERMS = ["fast", "quick", "quickly", "slow", "scalable", "robust", "secure", + "user-friendly", "intuitive", "easy", "simple", "efficient", "flexible", + "seamless", "reliable", "performant", "appropriate", "adequate", + "sufficient", "reasonable", "minimal", "as needed", "as appropriate"] +VAGUE_RE = re.compile(r"(?<![A-Za-z0-9_-])(%s)(?![A-Za-z0-9_-])" + % "|".join(re.escape(t).replace(r"\ ", r"\s+") + for t in sorted(VAGUE_TERMS, key=len, reverse=True)), re.I) +RATIONALE_MIN, RATIONALE_MAX = 20, 400 + + +class LintError(Exception): + pass + + +def _strip_ux(msg): + return msg[4:] if msg.startswith("ux: ") else msg + + +class Doc: + """A parsed PRD: lines, fence mask, headings, sections, requirement blocks.""" + + def __init__(self, path, lines): + self.path = path + self.lines = lines + # Raises UxError (malformed [UI], unterminated fence) before anything else. + allreqs = parse_requirements(path, retired_ends=True, lines=lines) + self.fenced = [] + fence = None + for line in lines: + m = FENCE_RE.match(line) + if fence is None and m: + fence = m.group(1)[0] + self.fenced.append(True) + elif fence is not None: + if m and m.group(1)[0] == fence: + fence = None + self.fenced.append(True) + else: + self.fenced.append(False) + self.headings = [] # (index, level, text) + for i, line in enumerate(lines): + m = HEADING_RE.match(line) + if m and not self.fenced[i]: + self.headings.append((i, len(m.group(1)), line.rstrip())) + self.req_range = self._section(REQ_SECTION_RE) + self.oq_range = self._section(OQ_SECTION_RE) + lo, hi = self.req_range or (0, 0) + # (id, lineno, block, title) inside ## Requirements only. + self.reqs = [r for r in allreqs if lo <= r[1] - 1 < hi] + self.blocks = {} + for rid, ln, block, _ in self.reqs: + self.blocks.setdefault(rid, (ln, block)) + + def _section(self, rx): + """[start, end) line indexes of the first `## <name>` section's body.""" + for n, (i, level, text) in enumerate(self.headings): + if level == 2 and rx.match(text): + end = len(self.lines) + for j, lv, _ in self.headings[n + 1:]: + if lv <= 2: + end = j + break + return (i + 1, end) + return None + + def in_range(self, rng, i): + return rng is not None and rng[0] <= i < rng[1] + + def empty_sections(self): + """[(index, heading text)] for each ##/### heading with no content + (not even a subsection) before the next heading of the same or higher level.""" + out = [] + hidx = {i: lv for i, lv, _ in self.headings} + for i, level, text in self.headings: + if level not in (2, 3): + continue + empty = True + for j in range(i + 1, len(self.lines)): + if j in hidx and hidx[j] <= level: + break + if self.lines[j].strip(): + empty = False + break + if empty: + out.append((i, text)) + return out + + +def load(path, label=None): + return Doc(label or path, _read_lines(path)) + + +def load_base(base, unparseable_stream): + """(Doc|None, degraded). `none` or a missing file is no base (not degraded); + a base that fails to parse is no base, with a degrade line (never rc 2).""" + if base == "none" or not os.path.exists(base): + return None, False + try: + return load(base), False + except UxError as e: + print("prd-lint: base unparseable (%s); every requirement is new" % _strip_ux(str(e)), + file=unparseable_stream) + return None, True + + +def compute_delta(cur, base): + """[(kind, id, lineno)]: new/changed in PRD order, then removed in base order.""" + rows, seen = [], set() + have = base.blocks if base is not None else {} + for rid, ln, _, _ in cur.reqs: + if rid in seen: + continue + seen.add(rid) + block = cur.blocks[rid][1].rstrip() + if rid not in have: + rows.append(("new", rid, ln)) + elif have[rid][1].rstrip() != block: + rows.append(("changed", rid, ln)) + if base is not None: + done = set() + for rid, ln, _, _ in base.reqs: + if rid not in cur.blocks and rid not in done: + done.add(rid) + rows.append(("removed", rid, ln)) + return rows + + +def new_lines(cur, base): + """Indexes of cur lines that are new compared with the base (line-level diff).""" + if base is None: + return set(range(len(cur.lines))) + a = [l.rstrip() for l in base.lines] + b = [l.rstrip() for l in cur.lines] + out = set() + for tag, _, _, j1, j2 in difflib.SequenceMatcher(None, a, b, autojunk=False).get_opcodes(): + if tag in ("replace", "insert"): + out.update(range(j1, j2)) + return out + + +def _code_masked(line): + """The line with inline code removed; a span holding a digit leaves `0` + behind so a number written in code still quantifies a term.""" + return CODE_RE.sub(lambda m: " 0 " if DIGIT_RE.search(m.group(2)) else " ", line) + + +def vague_findings(cur, rid, ln, block): + nlines = block.count("\n") + 1 + idx = range(ln - 1, ln - 1 + nlines) + text = "\n".join("" if cur.fenced[i] else _code_masked(cur.lines[i]) for i in idx) + out, seen = [], set() + for m in VAGUE_RE.finditer(text): + term = re.sub(r"\s+", " ", m.group(1).lower()) + if term in seen: + continue + end = SENT_END_RE.search(text, m.end()) + rest = text[m.end():end.start() if end else len(text)] + if DIGIT_RE.search(ID_TOKEN_RE.sub("", rest)): + continue + seen.add(term) + line = ln + text.count("\n", 0, m.start()) + out.append((line, "info", "vague.term", + '%s uses "%s" with no number in the same sentence' % (rid, term))) + return out + + +def cmd_delta(args): + if len(args) != 2: + raise LintError("prd-lint: usage: delta <prd> <base-file|none>") + cur = load(args[0]) + base, _ = load_base(args[1], sys.stderr) + for kind, rid, _ in compute_delta(cur, base): + print("%s\t%s" % (kind, rid)) + return 0 + + +def cmd_check(args): + degraded = False + if args and args[-1] == "--degraded": + degraded, args = True, args[:-1] + if len(args) != 2: + raise LintError("prd-lint: usage: check <prd> <base-file|none> [--degraded]") + cur = load(args[0]) + base, unparseable = load_base(args[1], sys.stdout) + degraded = degraded or unparseable + soft = "info" if degraded else "major" + fresh = new_lines(cur, base) + rows = compute_delta(cur, base) + delta = [(rid, ln) for kind, rid, ln in rows if kind in ("new", "changed")] + f = [] # (line, severity, code, message) + + if cur.req_range is None: + f.append((1, "major", "placeholder", "no ## Requirements section")) + + for i, line in enumerate(cur.lines): + if cur.fenced[i] or i not in fresh: + continue + m = PLACEHOLDER_RE.search(CODE_RE.sub("", line)) + if m: + f.append((i + 1, soft, "placeholder", 'placeholder text "%s"' % m.group(0))) + base_empty = {t for _, t in base.empty_sections()} if base is not None else set() + for i, text in cur.empty_sections(): + if text not in base_empty: + f.append((i + 1, soft, "placeholder", 'empty section "%s"' % text)) + + for rid, ln in delta: + block = cur.blocks[rid][1] + if not ACCEPT_RE.search(block): + f.append((ln, soft, "acceptance.missing", "%s has no — Acceptance: clause" % rid)) + f.extend(vague_findings(cur, rid, ln, block)) + + first = {} + for rid, ln, _, _ in cur.reqs: + if rid in first: + f.append((ln, "major", "id.duplicate", + "%s is already the title at line %d" % (rid, first[rid]))) + else: + first[rid] = ln + + for i, line in enumerate(cur.lines): + if cur.fenced[i] or i not in fresh or not cur.in_range(cur.req_range, i): + continue + body = _delist(line) + if MALFORMED_RE.match(body) and not REQ_RE.match(body) and not RETIRED_RE.match(body): + tok = body[2:].split("**", 1)[0][:40] + f.append((i + 1, "major", "id.malformed", + '"%s" is not a requirement id (expected **<ID>-<n> <title>**)' % tok)) + + for i, line in enumerate(cur.lines): + if cur.fenced[i] or not cur.in_range(cur.oq_range, i): + continue + m = OVERRIDE_RE.match(line.rstrip()) + if m: + n = len(m.group("rationale").strip()) + if not RATIONALE_MIN <= n <= RATIONALE_MAX: + f.append((i + 1, "major", "override.rationale", + 'override of "%s" has a %d-character rationale (needs %d-%d)' + % (m.group("finding")[:60], n, RATIONALE_MIN, RATIONALE_MAX))) + + f.sort(key=lambda x: (x[0], x[2])) + for line, sev, code, msg in f: + print("%s:%d %s %s: %s" % (cur.path, line, sev, code, msg)) + major = sum(1 for x in f if x[1] == "major") + info = len(f) - major + print("prd-lint: delta %d requirements, %d major, %d info" % (len(delta), major, info)) + return 1 if major else 0 + + +CMDS = {"delta": cmd_delta, "check": cmd_check} + + +def main(argv): + try: + if len(argv) < 2 or argv[1] not in CMDS: + raise LintError("prd-lint: usage: %s {%s} …" % (os.path.basename(argv[0]), "|".join(CMDS))) + rc = CMDS[argv[1]](argv[2:]) + sys.stdout.flush() + return rc + except LintError as e: + print(str(e), file=sys.stderr) + return 2 + except UxError as e: + print("prd-lint: %s" % _strip_ux(str(e)), file=sys.stderr) + return 2 + except SystemExit: + raise + except BaseException as e: # noqa: BLE001 — the fail-loud top-level handler + print("prd-lint: internal error: %s: %s" % (type(e).__name__, e), file=sys.stderr) + return 2 + + +if __name__ == "__main__": + sys.exit(main(sys.argv)) diff --git a/scripts/lib/ux_record.py b/scripts/lib/ux_record.py index 3df803d..1336b85 100644 --- a/scripts/lib/ux_record.py +++ b/scripts/lib/ux_record.py @@ -16,6 +16,8 @@ UI_RE = re.compile(r"^\*\*(?P<id>[A-Z][A-Z0-9]*-[0-9]+) \[UI\] (?P<title>[^*]+?)\.?\*\*") REQ_RE = re.compile(r"^\*\*(?P<id>[A-Z][A-Z0-9]*-[0-9]+) ") +# A retired-id marker: the id closed by ** with no title (`**FR-56** *Retired.*`). +RETIRED_RE = re.compile(r"^\*\*(?P<id>[A-Z][A-Z0-9]*-[0-9]+)\*\*\s+\*Retired\b") ID_RE = re.compile(r"[A-Z][A-Z0-9]*-[0-9]+") CODE_RE = re.compile(r"(`+)(.+?)\1") FENCE_RE = re.compile(r"^ {0,3}(`{3,}|~{3,})") @@ -47,9 +49,7 @@ def parse_prd_strict(path): raise UxError(str(e), 2) -def parse_prd(path): - """Return (ui, titles): ui = [(id, hash, title, lineno)] in file order; - titles = {id: lineno} for every requirement title (with or without [UI]).""" +def _read_lines(path): try: with open(path, "rb") as fh: raw = fh.read() @@ -62,9 +62,30 @@ def parse_prd(path): lines = text.split("\n") if lines and lines[-1] == "": lines.pop() + return lines + + +def _ui_scan(line): + """The [UI]-relevant view of a non-heading line: (body, code-stripped body).""" + body = _delist(line) + return body, CODE_RE.sub("", body) + + +def parse_requirements(path, retired_ends=False, lines=None): + """Every requirement title (REQ_RE, with or without [UI]) in file order, as + [(id, lineno, block_text, title_text)]: lineno is the 1-based title line, + title_text that line's text, block_text the block (title line up to the + next title or heading, each line right-stripped). Fences and inline code + are skipped. A malformed [UI] marker or an unterminated fence raises. + + retired_ends=True (prd_lint only, TDD 0073) also ends a block at a + retired-id marker line (`**FR-56** *Retired.*`); the marker itself is + never a requirement. parse_prd never sets it, so [UI] hashes don't move.""" + if lines is None: + lines = _read_lines(path) fence = None fence_line = 0 - heads = [] # (index, kind, id, title) — kind: 'ui' | 'req' | 'heading' + heads = [] # (index, is_requirement, id) for i, line in enumerate(lines): m = FENCE_RE.match(line) if fence is None and m: @@ -79,34 +100,46 @@ def parse_prd(path): # off-grammar and must fail loudly, not read as "no UI requirements". if "[UI]" in CODE_RE.sub("", line): raise UxError("ux: malformed [UI] marker at %s:%d: %s" % (path, i + 1, line[:120])) - heads.append((i, "heading", None, None)) + heads.append((i, False, None)) continue - body = _delist(line) - scan = CODE_RE.sub("", body) + body, scan = _ui_scan(line) if "[UI]" in scan: if not UI_RE.match(scan): raise UxError("ux: malformed [UI] marker at %s:%d: %s" % (path, i + 1, line[:120])) m2 = UI_RE.match(body) or UI_RE.match(scan) - heads.append((i, "ui", m2.group("id"), m2.group("title"))) + heads.append((i, True, m2.group("id"))) continue m3 = REQ_RE.match(body) if m3: - heads.append((i, "req", m3.group("id"), None)) + heads.append((i, True, m3.group("id"))) + elif retired_ends and RETIRED_RE.match(body): + heads.append((i, False, None)) if fence is not None: raise UxError("ux: unterminated code fence at %s:%d" % (path, fence_line)) - ui, seen, titles = [], {}, {} - for n, (i, kind, rid, title) in enumerate(heads): - if kind == "heading": - continue - titles.setdefault(rid, i + 1) - if kind != "ui": + reqs = [] + for n, (i, is_req, rid) in enumerate(heads): + if not is_req: continue - if rid in seen: - raise UxError("ux: duplicate [UI] id %s at %s:%d,%d" % (rid, path, seen[rid], i + 1)) - seen[rid] = i + 1 end = heads[n + 1][0] if n + 1 < len(heads) else len(lines) block = "\n".join(l.rstrip() for l in lines[i:end]) - ui.append((rid, hashlib.sha256(block.encode("utf-8")).hexdigest(), title, i + 1)) + reqs.append((rid, i + 1, block, lines[i])) + return reqs + + +def parse_prd(path): + """Return (ui, titles): ui = [(id, hash, title, lineno)] in file order; + titles = {id: lineno} for every requirement title (with or without [UI]).""" + ui, seen, titles = [], {}, {} + for rid, lineno, block, title_text in parse_requirements(path): + titles.setdefault(rid, lineno) + body, scan = _ui_scan(title_text) + if "[UI]" not in scan: + continue + if rid in seen: + raise UxError("ux: duplicate [UI] id %s at %s:%d,%d" % (rid, path, seen[rid], lineno)) + seen[rid] = lineno + m2 = UI_RE.match(body) or UI_RE.match(scan) + ui.append((rid, hashlib.sha256(block.encode("utf-8")).hexdigest(), m2.group("title"), lineno)) return ui, titles diff --git a/skills/prd-author/SKILL.md b/skills/prd-author/SKILL.md index 14b4bcc..fcd11eb 100644 --- a/skills/prd-author/SKILL.md +++ b/skills/prd-author/SKILL.md @@ -159,6 +159,31 @@ a collaborative scribe. Apply this discipline throughout that interview: step 3). The marker is the only signal `/ux-author` and `/tdd-author` read, so a wrong answer here silently skips or forces the UX phase. +### Coverage scan (FR-102) + +Before the first substantive question of step 3, mark each of these nine +categories `Clear`, `Partial` or `Missing` for this change: +- functional scope; +- domain and data; +- interaction and flow; +- non-functional qualities; +- integrations and dependencies; +- edge cases and failure handling; +- constraints and trade-offs; +- terminology; +- completion signals. + +Ask first about the `Partial`/`Missing` categories with the highest +(impact on the design × uncertainty): where the answer would most change the +design and is least settled. Persist each category with the step-3 helper and +its quoting and STOP-on-non-zero rules, header `coverage: <category>`, answer +`initial <status>; final <status>; <resolution or open question>`. Append it +again when its status or resolution changes; the latest entry per header is +authoritative. Before writing the PRD, every `Partial`/`Missing` category has +a resolution (the requirement or answer that settled it) or an Open questions +entry. The `## Coverage scan` table in the commit body (Git, below) shows both +statuses, so a `Partial`/`Missing` row names what resolved it. + 3. Interview the user with structured multiple-choice questions. Surface scope, non-goals, constraints, and edge cases the user hasn't stated. Skip obvious questions; dig into ambiguity and conflicting goals. Prefer multiple-choice options; don't @@ -248,7 +273,8 @@ out of scope here — enforce it for new requirements. ## Self-review (before the PR) After writing the PRD, reread it with fresh eyes and fix issues inline: - **Placeholder scan** — any "TBD"/"TODO"/empty section/vague requirement? Resolve - it or move it to Open questions. + it or move it to Open questions. (The `tl:prd-lint` pre-pass below checks + this mechanically too.) - **Consistency** — do any requirements or goals contradict each other? - **Scope** — still one coherent product, or did it sprawl into several? - **Ambiguity** — could a requirement be read two different ways? Pick one and make @@ -295,6 +321,78 @@ answers. On non-zero, show its stderr (a `malformed [UI] marker` line names the PRD line), fix the PRD, and re-run the block. Never commit a PRD for which this block fails. +### Requirements review (FR-103, FR-104, FR-105) + +After the `[UI]` marker check and BEFORE the commit: + +1. **Pre-pass.** Run this block as one shell command with `TL_REPO` set to + the absolute repo root. It sources its own helpers, lints `docs/PRD.md` + against the integration branch's copy, then prints the delta (the + requirement ids this change adds, changes or removes). + +<!-- tl:prd-lint --> +```bash +: "${TL_REPO:?TL_REPO required (absolute repo root)}" +_tl_src="${CLAUDE_PLUGIN_ROOT:-${GROK_PLUGIN_ROOT:-}}" +. "${_tl_src}/scripts/lib/plugin-root.sh" || { echo "throughline: cannot source plugin-root.sh" >&2; exit 2; } +. "$(tl_plugin_root)/scripts/lib/prd-lint.sh" || { echo "throughline: cannot source prd-lint.sh" >&2; exit 2; } +tl_prd_lint "$TL_REPO"; _lint_rc=$? +tl_prd_delta "$TL_REPO"; _delta_rc=$? +[ "$_lint_rc" -ne 0 ] && exit "$_lint_rc" +exit "$_delta_rc" +``` + + The lint prints `prd-lint: base <ref>` (or `prd-lint: base none`), then + one `docs/PRD.md:<line> <severity> <code>: <message>` line per finding, + then `prd-lint: delta <n> requirements, <m> major, <k> info`. The delta + lines that follow are `new<TAB><id>`, `changed<TAB><id>` or + `removed<TAB><id>`. + - rc 0: continue to the review. `info` lines (for example `vague.term`) + do not block; they go to the reviewer. + - rc 1: a major finding (`placeholder`, `acceptance.missing`, + `id.duplicate`, `id.malformed`, `override.rationale`). Fix the PRD and + re-run the block. Do not dispatch the reviewer while it returns rc 1. + - rc 3: no python3. Record `Pre-pass unavailable: python3 required` for + the PR body; the review still runs (step 2). + - rc 2: a real git failure or a parse error. Show stderr and stop. + - Copy any `prd-lint: base is HEAD …`, `prd-lint: no base …` or + `prd-lint: base unparseable …` line into the PR body under + `## Requirements review`. + +2. **Independent review.** Dispatch **one** fresh worker that is not this + session, whose instructions are the file + `$(tl_plugin_root)/agents/requirements-reviewer.md` (dispatch by prompt + file, so it works on any harness): + - no model parameter: the worker inherits this session's model (FR-87); + - it must not spawn children; + - give it the repo root, the PRD path, the `prd-lint: base <ref>` line + and any degrade line, the delta lines, the prd-lint `vague.term` info + lines, and a report file created by a fresh `mktemp` for this dispatch. + The report path is never recorded in the draft. A resumed session always + makes a new one and dispatches a new reviewer, so an earlier verdict is + never reused (FR-104); + - when the pre-pass returned rc 3, give it no delta lines. Tell it instead + to derive the delta itself from `git diff <base> -- docs/PRD.md`, or + from the whole PRD when there is no base. + + Read the report file. Its last line is authoritative: exactly + `REQUIREMENTS_REVIEW: PASS` continues to Git. A missing file, or any other + last line, counts as BLOCK. + +3. **On BLOCK,** for each blocking `R-n` finding, ask a structured question + with three options: + - **Fix** the requirement (edit the PRD); + - **Override** with a rationale of 20–400 characters specific to the + finding. Write it under the PRD's `## Open questions` as exactly + `- "<finding>" — overridden: <rationale>`, where `<finding>` is the + finding's one-sentence text only (no `R-n`, severity or citation); + - **Stop** the skill. + + Then re-run the pre-pass block and dispatch a **new** reviewer with a new + `mktemp` report file. A verdict is never reused (as FR-50) or written to + the draft. The re-run reviewer re-checks each override and may BLOCK + again on a boilerplate rationale. + ## Template ``` @@ -302,7 +400,10 @@ this block fails. ## Problem & context ## Users & goals -## Requirements (numbered, each independently testable) +## Requirements (numbered, each independently testable; each title is + **<ID>-<n> <title>.** with an uppercase id prefix, e.g. + **FR-12 Saved searches.** … — Acceptance: …; a retired + id is **FR-12** *Retired.*) ## Non-goals ## Constraints & assumptions ## Open questions @@ -338,6 +439,19 @@ Unless the user says "skip git": `/tdd-author` session and makes the design intent of the PRD change auditable. For purely additive PRD changes with no downstream cascade, write "Cascade: none" so the absence is intentional, not an oversight. +- **Coverage scan (in the commit message body, FR-102).** Add a + `## Coverage scan` section: a table `| Category | Initial | Final | + Resolution |` with exactly the nine categories, from the latest + `coverage:` draft entries. Each `Partial`/`Missing` row names the + requirement or open question that resolved it. +- **Requirements review (in the commit message body, FR-104, FR-105).** Add + a `## Requirements review` section with: the reviewer's verdict line; the + findings summary, one `R-n <severity> — <finding>` line each; each override + as `"<finding>" — overridden: <rationale>`, the same form as its Open + questions line; any prd-lint degrade line; and + `Pre-pass unavailable: python3 required` when the pre-pass returned rc 3. + The review is decision support for the human merge, not a quality + guarantee. - **UI requirements (in the commit message body, FR-89).** When the delta adds or changes any `[UI]` requirement, add a `## UI requirements` section listing each one (`- <ID> <title>`, from the `tl:ui-markers` output), followed by the diff --git a/tests/implement-gate.test.sh b/tests/implement-gate.test.sh index d65911b..4ab02ef 100755 --- a/tests/implement-gate.test.sh +++ b/tests/implement-gate.test.sh @@ -207,13 +207,16 @@ echo "=== 0063 observations: $PASS passed, $FAIL failed ===" # ux-author-skill (FR-90..FR-100: the extracted /ux-author tl: blocks run # under env -i in temp repos; the ux-reviewer contract) + 0072 ux-pipeline # (FR-89/FR-101/FR-10: the extracted tl:ui-markers / tl:ux-coverage blocks and -# the tdd-lint ux.citation check on fixture TDDs). A registered eval that is missing is a failure, +# the tdd-lint ux.citation check on fixture TDDs) + 0073 prd-lint / +# requirements-review (FR-102..FR-105, FR-110: the PRD pre-pass in temp git +# repos, the extracted tl:prd-lint block, the requirements-reviewer contract). +# A registered eval that is missing is a failure, # never a silent skip (L-011). EVAL_FAIL=0 for ev in plugin-root verdicts run-record build-tdds-skill model-roles \ parent-session-check model-dispatch escalation \ format-and-lint-hook build-hardening ux-record ux-render \ - ux-author-skill ux-pipeline; do + ux-author-skill ux-pipeline prd-lint requirements-review; do f="$(dirname "$0")/${ev}.test.sh" echo if [ -f "$f" ]; then diff --git a/tests/prd-lint.test.sh b/tests/prd-lint.test.sh new file mode 100644 index 0000000..bbdb3c9 --- /dev/null +++ b/tests/prd-lint.test.sh @@ -0,0 +1,413 @@ +#!/usr/bin/env bash +# prd-lint.test.sh — eval for TDD 0073 / FR-103 (and FR-105's mechanical half): +# the PRD pre-pass scripts/lib/prd-lint.sh + prd_lint.py, and the +# parse_requirements refactor of ux_record.py (no [UI] regression). +# +# Observation points 1–7, 9, 9b, 9c, 14–17 and 19 of the TDD's Verification +# plan. Every function runs through the sourced prd-lint.sh in a clean +# `env -i` shell against temp git repos whose `master` holds the base +# docs/PRD.md and whose working tree holds the edit. A missing file is a FAIL +# (L-001), every negated grep first asserts its input is readable/non-empty +# (L-001/L-011), every temp dir is trap-cleaned (L-004). +# +# Written red-first. Run: bash tests/prd-lint.test.sh +set -uo pipefail +REPO="$(cd "$(dirname "$0")/.." && pwd)" +PL="$REPO/scripts/lib/prd-lint.sh" +UXR="$REPO/scripts/lib/ux_record.py" +# The pre-refactor ux_record.py lives at this commit (TDD 0073's base). +PRE_REFACTOR=4f217b3 +RESULTS=""; ROOT="" +cleanup() { [ -n "$ROOT" ] && rm -rf "$ROOT"; [ -n "$RESULTS" ] && rm -f "$RESULTS"; } +trap cleanup EXIT +RESULTS="$(mktemp)"; export RESULTS +ROOT="$(mktemp -d)" +ok() { printf 'ok\n' >>"$RESULTS"; printf ' ok — %s\n' "$1"; } +bad() { printf 'fail\n' >>"$RESULTS"; printf ' FAIL — %s\n' "$1"; } +q() { printf '%q' "$1"; } +H="$ROOT/home"; mkdir -p "$H" +BASH_BIN="$(command -v bash)" +TAB="$(printf '\t')" +command -v python3 >/dev/null 2>&1 || bad "infra: python3 not found" +[ -r "$PL" ] || bad "infra: $PL missing/unreadable" +G() { git -c user.name=t -c user.email=t@t -c init.defaultBranch=master -c commit.gpgsign=false "$@"; } + +# pl <dir> '<cmds>' [VAR=val …] — source prd-lint.sh in a clean shell in <dir>. +# Sets OUT ERR RC; ALL = stdout+stderr. +pl() { + local d="$1" c="$2"; shift 2 + ( cd "$d" && exec env -i HOME="$H" PATH="${PLPATH:-$PATH}" CLAUDE_PLUGIN_ROOT="$REPO" \ + GIT_CEILING_DIRECTORIES="$ROOT" "$@" \ + "$BASH_BIN" -c ". $(q "$PL") || exit 98; $c" ) >"$ROOT/out" 2>"$ROOT/err" + RC=$?; OUT="$(cat "$ROOT/out")"; ERR="$(cat "$ROOT/err")"; ALL="$OUT"$'\n'"$ERR" +} +lint() { pl "$1" "tl_prd_lint $(q "$1")"; } +delta() { pl "$1" "tl_prd_delta $(q "$1")"; } +# has <text> <fixed-string> — non-empty input that contains the string. +has() { [ -n "$1" ] && printf '%s\n' "$1" | grep -qF -- "$2"; } +# hasnt <text> <fixed-string> — non-empty (readable) input without it. +hasnt() { [ -n "$1" ] && ! printf '%s\n' "$1" | grep -qF -- "$2"; } +lineof() { grep -nF -- "$2" "$1" | head -1 | cut -d: -f1; } + +# mkrepo <dir> <base-prd-file|-> — git repo, `master` holds the base PRD +# (or, with `-`, only a README: no PRD on the base). The working tree's +# docs/PRD.md starts as a copy of the base. +mkrepo() { + local d="$1" b="$2" + mkdir -p "$d/docs" + G -C "$d" init -q + if [ "$b" = - ]; then echo readme >"$d/README.md"; else cp "$b" "$d/docs/PRD.md"; fi + G -C "$d" add -A && G -C "$d" commit -qm base +} + +BASE="$ROOT/base.md" +cat >"$BASE" <<'EOF' +# Product Requirements: Demo + +## Problem & context +People need a demo. + +## Requirements +### Core +- **FR-1 Login.** The user logs in. — Acceptance: `login` prints `ok`. +- **FR-2 Export.** The user exports a CSV. — Acceptance: `export` writes + `out.csv`. + +## Non-goals +- Nothing else. + +## Constraints & assumptions +- Linux only. + +## Open questions +- None yet. +EOF +# add_after <file> <anchor-fixed-string> <text…> — insert lines after anchor. +add_after() { + local f="$1" a="$2"; shift 2 + python3 -I -c 'import sys +p, a = sys.argv[1], sys.argv[2]; new = sys.argv[3:] +ls = open(p).read().split("\n"); i = next(n for n, l in enumerate(ls) if a in l) +ls[i+1:i+1] = new; open(p, "w").write("\n".join(ls))' "$f" "$a" "$@" +} +FR3_OK='- **FR-3 Import.** The user imports a CSV. — Acceptance: `import a.csv` prints `3 rows`.' +FR3_NOACC='- **FR-3 Import.** The user imports a CSV file from disk.' + +echo "[1] delta: changed / new / removed, PRD order" +R1="$ROOT/r1"; mkrepo "$R1" "$BASE" +sed -i 's/exports a CSV/exports a TSV/' "$R1/docs/PRD.md" +add_after "$R1/docs/PRD.md" '`out.csv`.' "$FR3_OK" +delta "$R1" +[ "$RC" -eq 0 ] && [ "$OUT" = "changed${TAB}FR-2"$'\n'"new${TAB}FR-3" ] \ + && ok "[1] edit FR-2 + add FR-3 → exactly changed FR-2, new FR-3 (rc 0)" \ + || bad "[1] rc=$RC out='$OUT' err='$ERR'" +sed -i '/FR-1 Login/d' "$R1/docs/PRD.md" +delta "$R1" +[ "$RC" -eq 0 ] && has "$OUT" "removed${TAB}FR-1" \ + && ok "[1] removing FR-1 → removed FR-1" || bad "[1] removed: rc=$RC out='$OUT' err='$ERR'" +R1b="$ROOT/r1b"; mkrepo "$R1b" -; cp "$BASE" "$R1b/docs/PRD.md" +delta "$R1b" +[ "$RC" -eq 0 ] && [ "$OUT" = "new${TAB}FR-1"$'\n'"new${TAB}FR-2" ] \ + && ok "[1] no PRD on master → every id new" || bad "[1] no base: rc=$RC out='$OUT' err='$ERR'" +R1c="$ROOT/r1c"; mkrepo "$R1c" "$BASE" +delta "$R1c" +[ "$RC" -eq 0 ] && [ -z "$OUT" ] && ok "[1] unchanged PRD → empty delta, rc 0" \ + || bad "[1] unchanged: rc=$RC out='$OUT' err='$ERR'" + +echo "[2] acceptance.missing — delta only" +B2="$ROOT/base2.md"; sed 's/ — Acceptance: `login` prints `ok`\.//' "$BASE" >"$B2" +R2="$ROOT/r2"; mkrepo "$R2" "$B2" +add_after "$R2/docs/PRD.md" '`out.csv`.' "$FR3_NOACC" +lint "$R2" +n3="$(lineof "$R2/docs/PRD.md" 'FR-3 Import')" +[ "$RC" -eq 1 ] && printf '%s\n' "$OUT" | grep -qE "^docs/PRD\.md:${n3} major acceptance\.missing: .*FR-3" \ + && ok "[2] new FR-3 without acceptance → rc 1, docs/PRD.md:$n3 major acceptance.missing naming FR-3" \ + || bad "[2] rc=$RC out='$OUT' err='$ERR'" +hasnt "$OUT" "FR-1 " && hasnt "$OUT" "acceptance.missing: FR-1" \ + && ok "[2] unchanged FR-1 lacking acceptance → no finding" || bad "[2] FR-1 flagged: '$OUT'" +has "$OUT" "prd-lint: base master" && has "$OUT" "prd-lint: delta 1 requirements, 1 major, 0 info" \ + && [ "$(printf '%s\n' "$OUT" | head -1)" = "prd-lint: base master" ] \ + && ok "[2] first line 'prd-lint: base master', summary 'delta 1 requirements, 1 major, 0 info'" \ + || bad "[2] header/summary: '$OUT'" +# A wrapped acceptance clause (line break between dash and Acceptance:) counts. +R2b="$ROOT/r2b"; mkrepo "$R2b" "$BASE" +add_after "$R2b/docs/PRD.md" '`out.csv`.' '- **FR-3 Import.** The user imports a CSV file from disk. —' ' Acceptance: `import a.csv` prints `3 rows`.' +lint "$R2b" +[ "$RC" -eq 0 ] && hasnt "$ALL" "acceptance.missing" \ + && ok "[2] '—' then newline then 'Acceptance:' → accepted, rc 0" || bad "[2] wrapped: rc=$RC out='$OUT' err='$ERR'" + +echo "[3] placeholder tokens and empty sections" +R3="$ROOT/r3"; mkrepo "$R3" "$BASE" +add_after "$R3/docs/PRD.md" 'People need a demo.' 'The pricing model is TODO for now.' +lint "$R3" +n="$(lineof "$R3/docs/PRD.md" 'is TODO for')" +[ "$RC" -eq 1 ] && printf '%s\n' "$OUT" | grep -qE "^docs/PRD\.md:${n} major placeholder: " \ + && ok "[3] TODO in prose → major placeholder at line $n, rc 1" || bad "[3] prose: rc=$RC out='$OUT' err='$ERR'" +R3b="$ROOT/r3b"; mkrepo "$R3b" "$BASE" +add_after "$R3b/docs/PRD.md" 'People need a demo.' 'Write `TODO` and ``a TBD b`` in code.' '```' 'TODO: fenced ???' '```' +lint "$R3b" +[ "$RC" -eq 0 ] && hasnt "$OUT" "placeholder" \ + && ok "[3] tokens in backticks / a fence → no finding, rc 0" || bad "[3] code: rc=$RC out='$OUT' err='$ERR'" +R3c="$ROOT/r3c"; mkrepo "$R3c" "$BASE" +add_after "$R3c/docs/PRD.md" 'People need a demo.' '' '### Empty' '' '### Next' 'Some text.' +lint "$R3c" +n="$(lineof "$R3c/docs/PRD.md" '### Empty')" +[ "$RC" -eq 1 ] && printf '%s\n' "$OUT" | grep -qE "^docs/PRD\.md:${n} major placeholder: .*empty section" \ + && ok "[3] empty ### section → major placeholder (empty section)" || bad "[3] empty: rc=$RC out='$OUT' err='$ERR'" +R3d="$ROOT/r3d"; B3d="$ROOT/base3d.md"; sed 's/People need a demo\./The TODO stays here./' "$BASE" >"$B3d" +mkrepo "$R3d" "$B3d"; add_after "$R3d/docs/PRD.md" '`out.csv`.' "$FR3_OK" +lint "$R3d" +[ "$RC" -eq 0 ] && hasnt "$OUT" "placeholder" \ + && ok "[3] an old TODO on the base never blocks a later change" || bad "[3] old token: rc=$RC out='$OUT' err='$ERR'" + +echo "[4] id.duplicate and id.malformed" +R4="$ROOT/r4"; mkrepo "$R4" "$BASE" +add_after "$R4/docs/PRD.md" '`out.csv`.' '- **FR-2 Again.** Dup. — Acceptance: `x` prints `y`.' +lint "$R4" +[ "$RC" -eq 1 ] && printf '%s\n' "$OUT" | grep -qE "major id\.duplicate: .*FR-2" \ + && ok "[4] two **FR-2 …** titles → id.duplicate, rc 1" || bad "[4] dup: rc=$RC out='$OUT' err='$ERR'" +R4b="$ROOT/r4b"; mkrepo "$R4b" "$BASE" +add_after "$R4b/docs/PRD.md" '`out.csv`.' '- **FR-2a Variant.** x. — Acceptance: `x` prints `y`.' +lint "$R4b" +[ "$RC" -eq 1 ] && printf '%s\n' "$OUT" | grep -qE "major id\.malformed: .*FR-2a" \ + && ok "[4] **FR-2a …** → id.malformed, rc 1" || bad "[4] malformed: rc=$RC out='$OUT' err='$ERR'" + +echo "[5] vague.term is info only" +R5="$ROOT/r5"; mkrepo "$R5" "$BASE" +add_after "$R5/docs/PRD.md" '`out.csv`.' '- **FR-3 Import.** The import is fast for the user. — Acceptance: `import a.csv` prints `3 rows`.' +lint "$R5" +[ "$RC" -eq 0 ] && printf '%s\n' "$OUT" | grep -qE "info vague\.term: .*fast" \ + && has "$OUT" "prd-lint: delta 1 requirements, 0 major, 1 info" \ + && ok "[5] 'fast' with no number → info vague.term naming it, rc 0" || bad "[5] rc=$RC out='$OUT' err='$ERR'" +R5b="$ROOT/r5b"; mkrepo "$R5b" "$BASE" +add_after "$R5b/docs/PRD.md" '`out.csv`.' '- **FR-3 Import.** The import is fast, within 2 seconds. — Acceptance: `import a.csv` prints `3 rows`.' +lint "$R5b" +[ "$RC" -eq 0 ] && hasnt "$OUT" "vague.term" \ + && ok "[5] 'fast, within 2 seconds' → no vague.term" || bad "[5] quantified: rc=$RC out='$OUT' err='$ERR'" + +echo "[6] override.rationale length" +R6="$ROOT/r6"; mkrepo "$R6" "$BASE" +add_after "$R6/docs/PRD.md" '- None yet.' '- "FR-2 has two readings" — overridden: ok' +lint "$R6" +[ "$RC" -eq 1 ] && printf '%s\n' "$OUT" | grep -qE "major override\.rationale: " \ + && ok "[6] rationale 'ok' → major override.rationale, rc 1" || bad "[6] short: rc=$RC out='$OUT' err='$ERR'" +R6b="$ROOT/r6b"; mkrepo "$R6b" "$BASE" +add_after "$R6b/docs/PRD.md" '- None yet.' '- "FR-2 has two readings" — overridden: CSV is the only export format.' +lint "$R6b" +[ "$RC" -eq 0 ] && hasnt "$OUT" "override.rationale" \ + && ok "[6] a 30-character rationale → no finding, rc 0" || bad "[6] ok-length: rc=$RC out='$OUT' err='$ERR'" + +echo "[7] no python3 → rc 3" +NP="$ROOT/nopy"; mkdir -p "$NP" +for t in git mktemp rm cat dirname grep sed head tr wc cut printf env; do + p="$(command -v "$t" 2>/dev/null)" && [ -x "$p" ] && ln -sf "$p" "$NP/$t" +done +PLPATH="$NP" lint "$R2" +[ "$RC" -eq 3 ] && has "$ALL" "prd-lint: python3 required" \ + && ok "[7] PATH without python3 → 'prd-lint: python3 required', rc 3" || bad "[7] rc=$RC out='$OUT' err='$ERR'" +PLPATH="$NP" delta "$R2" +[ "$RC" -eq 3 ] && has "$ALL" "prd-lint: python3 required" \ + && ok "[7] tl_prd_delta without python3 → rc 3" || bad "[7] delta rc=$RC out='$OUT' err='$ERR'" + +echo "[9] parse_requirements refactor: tl_ux_ui_reqs byte-identical" +OLD="$ROOT/old/ux_record.py"; mkdir -p "$ROOT/old" +if git -C "$REPO" show "$PRE_REFACTOR:scripts/lib/ux_record.py" >"$OLD" 2>/dev/null && [ -s "$OLD" ]; then + FX="$ROOT/fx"; mkdir -p "$FX" + cat >"$FX/a.md" <<'EOF' +# T +- **FR-1 [UI] Login.** body +Prose mentions the `[UI]` marker and ``a [UI] b`` too. +``` +**FR-9 [UI] Fenced.** +``` +**R-7 [UI] Cart** body +EOF + printf '%s\n' '- **FR-3 [UI] Use `x` here.** b' ' more ' '## H' 'tail' >"$FX/b.md" + cat >"$FX/c.md" <<'EOF' +# P +## Requirements +- **FR-1 [UI] Login.** The user logs in. + Details here. +- **FR-2 API.** Not UI. +- **FR-55 [UI] Panel.** body +- **FR-56** *Retired.* gone +- **FR-57 [UI] Other.** x +## Constraints +- **COVID-19 [UI] policy.** odd but parsed +EOF + cp "$REPO/docs/PRD.md" "$FX/repo-prd.md" + allsame=1 + for f in "$FX"/*.md; do + o="$(python3 -I -B "$OLD" ui-reqs "$f" 2>&1; echo "rc=$?")" + n="$(python3 -I -B "$UXR" ui-reqs "$f" 2>&1; echo "rc=$?")" + if [ "$o" = "$n" ] && [ -n "$o" ]; then :; else allsame=0; bad "[9] ui-reqs differs on $(basename "$f"): old='$o' new='$n'"; fi + o="$(python3 -I -B "$OLD" delta "$f" "$ROOT/none.json" 2>&1; echo "rc=$?")" + n="$(python3 -I -B "$UXR" delta "$f" "$ROOT/none.json" 2>&1; echo "rc=$?")" + [ "$o" = "$n" ] || { allsame=0; bad "[9] delta differs on $(basename "$f")"; } + done + printf '%s\n' '- **FR-1 [UI Login.** x' >"$FX/bad.md" + o="$(python3 -I -B "$OLD" ui-reqs "$FX/bad.md" 2>&1; echo "rc=$?")" + n="$(python3 -I -B "$UXR" ui-reqs "$FX/bad.md" 2>&1; echo "rc=$?")" + [ "$o" = "$n" ] || { allsame=0; bad "[9] malformed-marker error differs: old='$o' new='$n'"; } + [ "$allsame" -eq 1 ] && ok "[9] ui-reqs + delta output byte-identical to $PRE_REFACTOR on fixtures and this repo's PRD" + grep -q '^def parse_requirements(' "$UXR" && ok "[9] ux_record.py exposes parse_requirements" \ + || bad "[9] ux_record.py has no parse_requirements" + pr="$(python3 -I -B -c 'import sys; sys.path.insert(0, sys.argv[1]); import ux_record as u +for r in u.parse_requirements(sys.argv[2]): print("%s|%d|%s" % (r[0], r[1], r[3]))' "$REPO/scripts/lib" "$FX/c.md" 2>&1)" + [ "$pr" = "FR-1|3|- **FR-1 [UI] Login.** The user logs in. +FR-2|5|- **FR-2 API.** Not UI. +FR-55|6|- **FR-55 [UI] Panel.** body +FR-57|8|- **FR-57 [UI] Other.** x +COVID-19|10|- **COVID-19 [UI] policy.** odd but parsed" ] \ + && ok "[9] parse_requirements → (id, title line, block, title text) for every title, file order" \ + || bad "[9] parse_requirements: '$pr'" +else + bad "infra: cannot read $PRE_REFACTOR:scripts/lib/ux_record.py (need full history)" +fi + +echo "[9b] hyphenated bold bullets are not ids" +R9="$ROOT/r9"; mkrepo "$R9" "$BASE" +add_after "$R9/docs/PRD.md" '### Core' '- **Auto-merging** is never done.' '- **Cross-vendor review** is optional.' +lint "$R9" +[ "$RC" -eq 0 ] && hasnt "$OUT" "id.malformed" \ + && ok "[9b] **Auto-merging** / **Cross-vendor review** → no id.malformed" || bad "[9b] rc=$RC out='$OUT' err='$ERR'" +R9b="$ROOT/r9b"; mkrepo "$R9b" "$BASE" +add_after "$R9b/docs/PRD.md" '### Core' '- **FR-1a Variant.** x — Acceptance: `x` prints `y`.' +lint "$R9b" +[ "$RC" -eq 1 ] && has "$OUT" "id.malformed" \ + && ok "[9b] **FR-1a …** → id.malformed" || bad "[9b] FR-1a: rc=$RC out='$OUT' err='$ERR'" +# Retired markers: never malformed, and editing one never marks its neighbour changed. +B9r="$ROOT/base9r.md"; cp "$BASE" "$B9r" +add_after "$B9r" '`out.csv`.' '- **FR-56** *Retired.* Old text.' +R9r="$ROOT/r9r"; mkrepo "$R9r" "$B9r" +sed -i 's/Old text\./Older text, reworded./' "$R9r/docs/PRD.md" +add_after "$R9r/docs/PRD.md" 'Older text' '- **FR-57** *Retired.*' +lint "$R9r"; lo="$OUT"; lrc="$RC" +delta "$R9r" +[ "$lrc" -eq 0 ] && hasnt "$lo" "id.malformed" && [ -z "$OUT" ] && [ "$RC" -eq 0 ] \ + && ok "[9b] retired-marker lines: no id.malformed; editing one leaves FR-2 unchanged" \ + || bad "[9b] retired: lint rc=$lrc '$lo'; delta rc=$RC '$OUT' err='$ERR'" + +echo "[9c] unparseable base → treated as none, never rc 2" +B9c="$ROOT/base9c.md"; cp "$BASE" "$B9c"; printf '%s\n' '```' 'unterminated' >>"$B9c" +R9c="$ROOT/r9c"; mkrepo "$R9c" "$B9c"; cp "$BASE" "$R9c/docs/PRD.md" +lint "$R9c" +{ [ "$RC" -eq 0 ] || [ "$RC" -eq 1 ]; } && has "$OUT" "prd-lint: base unparseable (" \ + && has "$OUT" "); every requirement is new" && has "$OUT" "prd-lint: delta 2 requirements" \ + && ok "[9c] 'prd-lint: base unparseable (…); every requirement is new', rc $RC" \ + || bad "[9c] rc=$RC out='$OUT' err='$ERR'" +delta "$R9c" +[ "$RC" -eq 0 ] && [ "$OUT" = "new${TAB}FR-1"$'\n'"new${TAB}FR-2" ] \ + && ok "[9c] delta treats the unparseable base as none" || bad "[9c] delta rc=$RC out='$OUT' err='$ERR'" + +echo "[14] this repo's PRD lints rc 0 against its own integration base" +if [ -r "$REPO/docs/PRD.md" ] && git -C "$REPO" rev-parse -q --verify HEAD >/dev/null 2>&1; then + pl "$REPO" "tl_prd_lint $(q "$REPO")" + [ "$RC" -eq 0 ] && has "$OUT" "prd-lint: base " && hasnt "$OUT" " major " \ + && ok "[14] docs/PRD.md → rc 0, no major finding" || bad "[14] rc=$RC out='$OUT' err='$ERR'" + # The PRD as a whole (no base, not degraded) still holds no duplicate or + # malformed id and no empty section: retired-marker lines are recognised. + ( cd "$REPO" && python3 -I -B scripts/lib/prd_lint.py check docs/PRD.md none ) >"$ROOT/o14" 2>&1 + grep -qF 'prd-lint: delta ' "$ROOT/o14" && ! grep -qE 'id\.(duplicate|malformed)|empty section' "$ROOT/o14" \ + && ok "[14] whole PRD: no id.duplicate / id.malformed / empty section (retired lines recognised)" \ + || bad "[14] whole-PRD check: $(head -5 "$ROOT/o14" 2>/dev/null)" +else + bad "infra: $REPO/docs/PRD.md unreadable or $REPO is not a git repo" +fi + +echo "[15] degrade paths: outside git, no integration branch, no commits" +R15="$ROOT/r15"; mkdir -p "$R15/docs"; cp "$BASE" "$R15/docs/PRD.md" +lint "$R15" +[ "$RC" -ne 2 ] && has "$OUT" "prd-lint: no base (not a git work tree); every requirement is new" \ + && has "$OUT" "prd-lint: base none" && has "$OUT" "prd-lint: delta 2 requirements" \ + && ok "[15] outside git → no-base line, still lints (rc $RC)" || bad "[15] no git: rc=$RC out='$OUT' err='$ERR'" +R15b="$ROOT/r15b"; mkdir -p "$R15b/docs"; G -C "$R15b" init -q -b feature +cp "$BASE" "$R15b/docs/PRD.md"; G -C "$R15b" add -A; G -C "$R15b" commit -qm f +add_after "$R15b/docs/PRD.md" '`out.csv`.' "$FR3_OK" +lint "$R15b" +[ "$RC" -eq 0 ] && has "$OUT" "prd-lint: base is HEAD (no integration branch)" \ + && has "$OUT" "prd-lint: delta 1 requirements" \ + && ok "[15] only branch 'feature' → 'base is HEAD (no integration branch)', delta 1" \ + || bad "[15] HEAD: rc=$RC out='$OUT' err='$ERR'" +R15c="$ROOT/r15c"; mkdir -p "$R15c/docs"; G -C "$R15c" init -q; cp "$BASE" "$R15c/docs/PRD.md" +lint "$R15c" +[ "$RC" -ne 2 ] && has "$OUT" "prd-lint: no base (no commits); every requirement is new" \ + && hasnt "$OUT" "base is HEAD" \ + && ok "[15] fresh git init → no-commits line (rc $RC)" || bad "[15] no commits: rc=$RC out='$OUT' err='$ERR'" + +echo "[16] a parent heading is not an empty section" +R16="$ROOT/r16"; mkrepo "$R16" - +cat >"$R16/docs/PRD.md" <<'EOF' +# P + +## Requirements +### Setup +- **FR-1 Login.** x — Acceptance: `login` prints `ok`. +### Empty +### Next +- **FR-2 Out.** y — Acceptance: `out` prints `ok`. +EOF +lint "$R16" +c="$(printf '%s\n' "$OUT" | grep -c ' placeholder: ')" +n="$(lineof "$R16/docs/PRD.md" '### Empty')" +[ "$c" -eq 1 ] && printf '%s\n' "$OUT" | grep -qE "^docs/PRD\.md:${n} major placeholder: " \ + && ok "[16] ## Requirements → ### Setup: none; ### Empty → ### Next: exactly one" \ + || bad "[16] count=$c out='$OUT' err='$ERR'" + +echo "[17] a real git failure is rc 2" +GS="$ROOT/gitstub"; mkdir -p "$GS"; REALGIT="$(command -v git)" +cat >"$GS/git" <<EOF +#!$BASH_BIN +for a in "\$@"; do [ "\$a" = ls-tree ] && { echo "fatal: stub" >&2; exit 128; }; done +exec $(q "$REALGIT") "\$@" +EOF +chmod +x "$GS/git" +PLPATH="$GS:$PATH" lint "$R2" +[ "$RC" -eq 2 ] && has "$ALL" "prd-lint: git ls-tree failed (rc 128)" \ + && ok "[17] ls-tree exits 128 → 'prd-lint: git ls-tree failed (rc 128)', rc 2" || bad "[17] rc=$RC out='$OUT' err='$ERR'" + +echo "[19] requirements-section scoping and degraded strictness" +R19="$ROOT/r19"; mkrepo "$R19" "$BASE" +add_after "$R19/docs/PRD.md" '- Linux only.' '- **COVID-19 policy.** Offices may close.' +lint "$R19" +[ "$RC" -eq 0 ] && hasnt "$OUT" "acceptance.missing" && hasnt "$OUT" "COVID-19" \ + && ok "[19] new **COVID-19 policy.** under Constraints → not a requirement" || bad "[19] covid: rc=$RC out='$OUT' err='$ERR'" +delta "$R19" +[ "$RC" -eq 0 ] && [ -z "$OUT" ] && ok "[19] … and not in the delta" || bad "[19] covid delta: '$OUT'" +R19b="$ROOT/r19b"; mkrepo "$R19b" "$BASE"; sed -i 's/^## Requirements$/## Wants/' "$R19b/docs/PRD.md" +lint "$R19b" +c="$(printf '%s\n' "$OUT" | grep -c 'major placeholder: no ## Requirements section')" +[ "$RC" -eq 1 ] && [ "$c" -eq 1 ] && ok "[19] no ## Requirements → exactly one major placeholder" \ + || bad "[19] no-req: rc=$RC out='$OUT' err='$ERR'" +R19c="$ROOT/r19c"; mkdir -p "$R19c/docs"; cp "$B2" "$R19c/docs/PRD.md" +lint "$R19c" +[ "$RC" -eq 0 ] && printf '%s\n' "$OUT" | grep -qE "info acceptance\.missing: .*FR-1" \ + && hasnt "$OUT" "major acceptance.missing" \ + && ok "[19] outside git, legacy FR-1 without acceptance → info acceptance.missing, rc 0" \ + || bad "[19] degraded: rc=$RC out='$OUT' err='$ERR'" + +echo "[rc2] internal and parse errors" +R20="$ROOT/r20"; mkrepo "$R20" "$BASE"; printf '%s\n' '```' >>"$R20/docs/PRD.md" +lint "$R20" +[ "$RC" -eq 2 ] && has "$ERR" "prd-lint: " && ok "[rc2] unterminated fence in the PRD itself → rc 2 with a prd-lint: line" \ + || bad "[rc2] rc=$RC out='$OUT' err='$ERR'" +pl "$ROOT" "tl_prd_lint" +[ "$RC" -eq 2 ] && has "$ERR" "usage" && ok "[rc2] tl_prd_lint without a repo root → usage, rc 2" \ + || bad "[rc2] usage: rc=$RC err='$ERR'" + +echo "[S1] relative sourcing never runs the linted repo's own prd_lint.py" +RS="$ROOT/rs1"; mkdir -p "$RS/docs" "$RS/scripts/lib"; git -C "$RS" init -q -b master +printf '# P\n\n## Requirements\n- **FR-1 A.** x. — Acceptance: `a` prints `b`.\n' > "$RS/docs/PRD.md" +git -C "$RS" add -A; git -C "$RS" -c user.email=t@t.t -c user.name=t commit -qm base +printf 'import sys\nopen(sys.argv[0]+".pwned","w").write("x")\nprint("PLANTED")\n' > "$RS/scripts/lib/prd_lint.py" +LIBREL="$(python3 -c 'import os,sys; print(os.path.relpath(sys.argv[1]))' "$REPO/scripts/lib")" +out="$(cd "$REPO" && env -i HOME="$ROOT" PATH="$PATH" bash -c ". $LIBREL/prd-lint.sh && tl_prd_lint $(printf '%q' "$RS")" 2>&1)" +if [ ! -e "$RS/scripts/lib/prd_lint.py.pwned" ] && ! printf '%s' "$out" | grep -q PLANTED; then + ok "[S1] relative source path: plugin's prd_lint.py used, planted one never runs" +else bad "[S1] planted prd_lint.py ran: out='$out'"; fi + +echo +PASS="$(grep -c '^ok$' "$RESULTS" 2>/dev/null)"; PASS="${PASS:-0}" +FAIL="$(grep -c '^fail$' "$RESULTS" 2>/dev/null)"; FAIL="${FAIL:-0}" +echo "=== prd-lint (TDD 0073): $PASS passed, $FAIL failed ===" +[ "$FAIL" -eq 0 ] && [ "$PASS" -gt 0 ] diff --git a/tests/requirements-review.test.sh b/tests/requirements-review.test.sh new file mode 100644 index 0000000..c89b5cd --- /dev/null +++ b/tests/requirements-review.test.sh @@ -0,0 +1,173 @@ +#!/usr/bin/env bash +# requirements-review.test.sh — eval for TDD 0073 / FR-102, FR-103, FR-104, +# FR-105, FR-110, FR-87: the /prd-author requirements-review step. +# +# Observation 8: EXTRACTS the first fenced bash block after +# `<!-- tl:prd-lint -->` (skills/prd-author/SKILL.md) and RUNS it the way the +# harness does — a fresh shell, nothing pre-sourced, inputs from env only: +# env -i HOME=<tmp> PATH=<path> CLAUDE_PLUGIN_ROOT=<repo> TL_REPO=<fixture> bash <block> +# against a temp git repo (`master` holds the base docs/PRD.md). +# Observations 13 and 18 are text checks by necessity: the reviewer and the +# interview are model prose, so the contract is the prompt / skill text (the +# reviewer's behaviour is observed by parent-dispatched fixtures, obs 10–12). +# A missing marker, block or file is a FAIL (L-001), never a skip; every +# negated grep first asserts its file is readable (L-001/L-011); every temp dir +# is trap-cleaned (L-004). +# +# Written red-first. Run: bash tests/requirements-review.test.sh +set -uo pipefail +REPO="$(cd "$(dirname "$0")/.." && pwd)" +SKILL="$REPO/skills/prd-author/SKILL.md" +AGENT="$REPO/agents/requirements-reviewer.md" +RESULTS=""; ROOT="" +cleanup() { [ -n "$ROOT" ] && rm -rf "$ROOT"; [ -n "$RESULTS" ] && rm -f "$RESULTS"; } +trap cleanup EXIT +RESULTS="$(mktemp)"; export RESULTS +ROOT="$(mktemp -d)" +ok() { printf 'ok\n' >>"$RESULTS"; printf ' ok — %s\n' "$1"; } +bad() { printf 'fail\n' >>"$RESULTS"; printf ' FAIL — %s\n' "$1"; } +readable() { [ -r "$1" ] && [ -s "$1" ]; } +H="$ROOT/home"; mkdir -p "$H" +BASH_BIN="$(command -v bash)" +TAB="$(printf '\t')" +command -v python3 >/dev/null 2>&1 || bad "infra: python3 not found" +G() { git -c user.name=t -c user.email=t@t -c init.defaultBranch=master -c commit.gpgsign=false "$@"; } + +# extract <file> <marker-name> <out> — first ```bash block after `<!-- tl:<name> -->`. +# rc 0 ok | 10 file unreadable | 3 no marker | 4 no block | 5 unclosed | 6 empty. +extract() { + readable "$1" || return 10 + awk -v M="<!-- tl:$2 -->" ' + !seen && $0 == M { seen = 1; next } + seen && !open && /^```bash[ \t]*$/ { open = 1; next } + open && /^```[ \t]*$/ { closed = 1; exit } + open { print } + END { if (!seen) exit 3; if (!open) exit 4; if (!closed) exit 5 } + ' "$1" >"$3" || return $? + [ -s "$3" ] || return 6 +} + +echo "[8] tl:prd-lint block" +BLK="$ROOT/prd-lint.sh"; extract "$SKILL" prd-lint "$BLK"; rc=$? +case "$rc" in + 0) ok "[8] tl:prd-lint extracted" ;; + 10) bad "infra: $SKILL missing/unreadable/empty"; BLK="" ;; + 3) bad "[8] marker <!-- tl:prd-lint --> not found in $SKILL"; BLK="" ;; + 4) bad "[8] no fenced bash block after tl:prd-lint"; BLK="" ;; + 5) bad "[8] the block after tl:prd-lint is never closed"; BLK="" ;; + *) bad "[8] extractor rc=$rc"; BLK="" ;; +esac + +# runb <cwd> [VAR=val …] — run the extracted block under env -i; sets OUT ERR RC. +runb() { + local cwd="$1"; shift + if [ -z "$BLK" ]; then OUT=""; ERR="no block"; RC=99; return; fi + ( cd "$cwd" && exec env -i HOME="$H" PATH="$PATH" CLAUDE_PLUGIN_ROOT="$REPO" "$@" \ + "$BASH_BIN" "$BLK" ) >"$ROOT/out" 2>"$ROOT/err" + RC=$?; OUT="$(cat "$ROOT/out")"; ERR="$(cat "$ROOT/err")" +} + +# The observation-2 fixture: master holds FR-1 (no acceptance clause, legacy), +# the working tree adds FR-3 with no acceptance clause. +R="$ROOT/r"; mkdir -p "$R/docs" +cat >"$R/docs/PRD.md" <<'EOF' +# Product Requirements: Demo + +## Problem & context +People need a demo. + +## Requirements +- **FR-1 Login.** The user logs in. +- **FR-2 Export.** The user exports a CSV. — Acceptance: `export` writes `out.csv`. + +## Open questions +- None yet. +EOF +G -C "$R" init -q && G -C "$R" add -A && G -C "$R" commit -qm base +printf '%s\n' '- **FR-3 Import.** The user imports a CSV file from disk.' >"$ROOT/fr3" +python3 -I -c 'import sys +p = sys.argv[1]; ls = open(p).read().split("\n") +i = next(n for n, l in enumerate(ls) if "FR-2 Export" in l); ls.insert(i + 1, open(sys.argv[2]).read().rstrip("\n")) +open(p, "w").write("\n".join(ls))' "$R/docs/PRD.md" "$ROOT/fr3" + +runb "$R" TL_REPO="$R" +[ "$RC" -eq 1 ] && printf '%s\n' "$OUT" | grep -qE '^docs/PRD\.md:[0-9]+ major acceptance\.missing: .*FR-3' \ + && printf '%s\n' "$OUT" | grep -qF 'prd-lint: base master' \ + && printf '%s\n' "$OUT" | grep -qx "new${TAB}FR-3" \ + && ok "[8] obs-2 fixture under env -i → rc 1, the lint findings AND the delta line printed" \ + || bad "[8] rc=$RC out='$OUT' err='$ERR'" +runb "$R" +[ "$RC" -ne 0 ] && printf '%s' "$ERR" | grep -qF 'TL_REPO' \ + && ok "[8] TL_REPO unset → rc $RC naming TL_REPO" || bad "[8] unset: rc=$RC err='$ERR'" +runb "$R" TL_REPO="$R" CLAUDE_PLUGIN_ROOT="$ROOT/nope" +[ -n "$BLK" ] && [ "$RC" -ne 0 ] && [ -n "$ERR" ] && ok "[8] unresolvable plugin root → fails loud (rc $RC)" \ + || bad "[8] bad root: rc=$RC err='$ERR'" +# Clean fixture: FR-3 gains its acceptance clause → rc 0, delta still printed. +sed -i 's/^- \*\*FR-3 Import\.\*\* The user imports a CSV file from disk\.$/& — Acceptance: `import a.csv` prints `3 rows`./' "$R/docs/PRD.md" +runb "$R" TL_REPO="$R" +[ "$RC" -eq 0 ] && printf '%s\n' "$OUT" | grep -qx "new${TAB}FR-3" \ + && printf '%s\n' "$OUT" | grep -qF 'prd-lint: delta 1 requirements, 0 major' \ + && ok "[8] clean delta → rc 0 with the delta line" || bad "[8] clean: rc=$RC out='$OUT' err='$ERR'" + +echo "[13] requirements-reviewer contract" +if readable "$AGENT"; then + fm="$(awk 'NR==1 && $0=="---"{f=1; next} f && $0=="---"{exit} f' "$AGENT")" + printf '%s\n' "$fm" | grep -qx 'name: requirements-reviewer' && ok "[13] name: requirements-reviewer" || bad "[13] name missing" + printf '%s\n' "$fm" | grep -qx 'model: inherit' && ok "[13] model: inherit (FR-87)" || bad "[13] model: inherit missing" + printf '%s\n' "$fm" | grep -qx 'tools: Read, Grep, Glob, Bash' && ok "[13] tools: Read, Grep, Glob, Bash" || bad "[13] tools line" + for s in 'REQUIREMENTS_REVIEW: PASS' 'REQUIREMENTS_REVIEW: BLOCK <one-line reason>' \ + 'R-n <severity> — docs/PRD.md:<line> — <finding>' \ + '- "<finding>" — overridden: <rationale>' \ + 'R-n blocker — docs/PRD.md:<line> — override rejected: "<finding>"' \ + '### Non-findings' '<label> — "<quoted candidate finding>" — <why set aside>' \ + 'pedantic escalation' 'imaginary architecture' \ + 'report-file path given in the dispatch' 'overridden' \ + 'necessary' 'unambiguous' 'complete' 'singular' 'feasible' 'verifiable' \ + 'product owner' 'implementer' 'tester' 'stakeholder' \ + 'Evaluation rubric' 'terminology drift' 'contradiction' 'duplication' \ + 'blocker, major, minor or nit' 'Do not spawn'; do + grep -qF -- "$s" "$AGENT" && ok "[13] contains: $s" || bad "[13] missing: $s" + done + last="$(grep -nF 'REQUIREMENTS_REVIEW:' "$AGENT" | wc -l)" + [ "$last" -ge 2 ] && ok "[13] both verdict tokens present" || bad "[13] verdict tokens: $last" + # FR-110: no rule dismisses a category of finding. + if grep -qiE 'synchronous hand-wringing|harmless idempotence|ignore races?|dismiss race|not a real race|single-threaded so no race|idempotent so safe' "$AGENT"; then + bad "[13] a category-dismissal phrase is present" + else + ok "[13] no category-dismissal phrase (file readable)" + fi + grep -qiF 'never dismiss' "$AGENT" && ok "[13] states it never dismisses a category by rule" \ + || bad "[13] no-category-dismissal sentence missing" +else + bad "infra: $AGENT missing/unreadable/empty" +fi + +echo "[18] /prd-author skill text" +if readable "$SKILL"; then + for s in 'functional scope' 'domain and data' 'interaction and flow' 'non-functional qualities' \ + 'integrations and dependencies' 'edge cases and failure handling' \ + 'constraints and trade-offs' 'terminology' 'completion signals' \ + 'coverage: <category>' 'initial <status>; final <status>;' \ + '## Coverage scan' '## Requirements review' \ + '"<finding>" — overridden: <rationale>' \ + 'agents/requirements-reviewer.md' 'mktemp' 'never recorded' \ + 'Pre-pass unavailable: python3 required' 'REQUIREMENTS_REVIEW: PASS' \ + 'no model parameter' 'last line'; do + grep -qF -- "$s" "$SKILL" && ok "[18] contains: $s" || bad "[18] missing: $s" + done + # The review step sits after tl:ui-markers and before the Git phase gate. + a="$(grep -nxF '<!-- tl:ui-markers -->' "$SKILL" | head -1 | cut -d: -f1)" + b="$(grep -nxF '<!-- tl:prd-lint -->' "$SKILL" | head -1 | cut -d: -f1)" + c="$(grep -nx '## Git (phase gate)' "$SKILL" | head -1 | cut -d: -f1)" + [ -n "$a" ] && [ -n "$b" ] && [ -n "$c" ] && [ "$a" -lt "$b" ] && [ "$b" -lt "$c" ] \ + && ok "[18] tl:prd-lint sits after tl:ui-markers and before the Git phase gate" \ + || bad "[18] order: ui-markers=$a prd-lint=$b git=$c" +else + bad "infra: $SKILL missing/unreadable/empty" +fi + +echo +PASS="$(grep -c '^ok$' "$RESULTS" 2>/dev/null)"; PASS="${PASS:-0}" +FAIL="$(grep -c '^fail$' "$RESULTS" 2>/dev/null)"; FAIL="${FAIL:-0}" +echo "=== requirements-review (TDD 0073): $PASS passed, $FAIL failed ===" +[ "$FAIL" -eq 0 ] && [ "$PASS" -gt 0 ]