Skip to content

PRD: review rigor — one review contract across every phase - #198

Merged
cahenesy merged 1 commit into
masterfrom
docs/prd/review-rigor
Oct 8, 2026
Merged

cahenesy merged 1 commit into
masterfrom
docs/prd/review-rigor

Conversation

@cahenesy

@cahenesy cahenesy commented Oct 8, 2026

Copy link
Copy Markdown
Owner

Each authoring phase already ends in a review by a fresh worker, but the
depth varied. Requirements were self-reviewed only, design had a fixed
reviewer, and the /build-tdds review prompt was improvised on every run.
This update gives every phase the same review contract (FR-102–FR-110):

  • FR-102: a coverage scan in the /prd-author interview. Nine categories
    are marked Clear / Partial / Missing in a PR-body table, and the most
    consequential, least settled questions are asked first.
  • FR-103: a mechanical PRD pre-pass that blocks on placeholders, missing
    acceptance clauses, and duplicate or malformed ids, and flags
    unquantified terms.
  • FR-104 and FR-105: an independent requirements review that blocks the
    PRD PR (REQUIREMENTS_REVIEW: PASS|BLOCK). It checks per-requirement
    quality and set-level consistency of the delta against the whole PRD,
    from product, implementer, tester and stakeholder viewpoints. A finding
    can be overridden with a recorded rationale that a re-run can reject.
  • FR-106 and FR-107: the TDD pre-pass checks that Touched-files paths
    exist or are marked (new), going forward only, and blocks phrases
    that defer codebase discovery to the build.
  • FR-108: the design review adds a cited simulation trace, an
    alternatives table for every blocker or major finding,
    terminology/duplication checks, and a metrics line.
  • FR-109: the build review runs from a fixed reviewer prompt shipped with
    the plugin. The security checklist always applies, the trace must cite
    its steps, and findings on code the diff did not change are never a
    FAIL.
  • FR-110: a reviewer may set a finding aside only by quoting it.
    Race, concurrency, filesystem and security findings are never
    dismissed by rule.

Sources: research of 2026-10-07 (spec-kit /analyze and /clarify, the
Superpowers spec reviewer, TestAny's PRD Reviewer, two 2026 arXiv
studies), and issue #195, folded in with its stale references corrected.

Coverage scan (FR-102, applied to this change)

Category Status
Functional scope Clear
Domain & data Clear (reviewer reports, overrides, (new) marker)
Interaction & flow Clear (pre-pass, then review, then override, then PR)
Non-functional Partial: cost not bounded; resolved as no new bound (NFR-3)
Integrations Clear (FR-51, FR-10, FR-15(d), FR-87)
Edge cases & failure Clear (resume freshness, uncited trace steps, pre-existing debt)
Constraints & trade-offs Clear (benchmark recall recorded as an assumption)
Terminology Clear
Completion signals Clear (every new FR has an observable acceptance)

Open assumptions & waivers

  • Reviewer cost — resolved: no new bound; reviewers inherit the parent model (FR-87, NFR-3), and cost accounting stays a non-goal.
  • Overrides as a rubber stamp — resolved: each override records the finding and a 20–400 character rationale in the PR body and Open questions; a re-run sees it and may block again.
  • Grounding's new-file marker — resolved: an explicit (new) marker, applied only to TDDs written after this change.
  • Simulation trace as theater — resolved: every step cites a source, and an uncited step is unverified and cannot alone support a PASS.
  • Eval-aggregator check is throughline-specific — resolved: not a requirement; it goes in throughline's repo-level instructions, so the plugin stays generic.
  • Scope of the anti-theater rule — resolved: a named non-finding must quote what it dismisses, and no category is blanket-dismissed.
  • Whole PRD or delta — resolved: per-requirement checks cover the delta only; set-level checks compare the delta against the whole PRD.

UI requirements: none

Cascade

Additive: new FRs only, so no existing TDD is stale in its own scope.
Design work lands in the existing surfaces:

  • /prd-author skill: FR-102, FR-103, FR-104, FR-105, plus a new
    requirements-reviewer prompt;
  • tdd-lint (the FR-51 pre-pass): FR-106, FR-107;
  • agents/design-reviewer.md (FR-10): FR-108;
  • the /build-tdds review dispatch plus a new build-reviewer prompt
    (FR-15(d)): FR-109;
  • every reviewer prompt, including ux-reviewer: FR-110.
    Enumerations amended here: FR-87 and NFR-3 now list the requirements
    reviewer.
    ADR 0015 is only touched in Open questions (cross-model-family review);
    it is not superseded.
    Follow-up outside the PRD: add throughline's eval-aggregator convention
    to the repo's own instructions.

Refs #195

🤖 Generated with Claude Code

https://claude.ai/code/session_01NpGG21Y2VGGhTQaFr4oshB

Each authoring phase already ends in a review by a fresh worker, but the
depth varied. Requirements were self-reviewed only, design had a fixed
reviewer, and the /build-tdds review prompt was improvised on every run.
This update gives every phase the same review contract (FR-102–FR-110):
- FR-102: a coverage scan in the /prd-author interview. Nine categories
  are marked Clear / Partial / Missing in a PR-body table, and the most
  consequential, least settled questions are asked first.
- FR-103: a mechanical PRD pre-pass that blocks on placeholders, missing
  acceptance clauses, and duplicate or malformed ids, and flags
  unquantified terms.
- FR-104 and FR-105: an independent requirements review that blocks the
  PRD PR (`REQUIREMENTS_REVIEW: PASS|BLOCK`). It checks per-requirement
  quality and set-level consistency of the delta against the whole PRD,
  from product, implementer, tester and stakeholder viewpoints. A finding
  can be overridden with a recorded rationale that a re-run can reject.
- FR-106 and FR-107: the TDD pre-pass checks that Touched-files paths
  exist or are marked `(new)`, going forward only, and blocks phrases
  that defer codebase discovery to the build.
- FR-108: the design review adds a cited simulation trace, an
  alternatives table for every blocker or major finding,
  terminology/duplication checks, and a metrics line.
- FR-109: the build review runs from a fixed reviewer prompt shipped with
  the plugin. The security checklist always applies, the trace must cite
  its steps, and findings on code the diff did not change are never a
  FAIL.
- FR-110: a reviewer may set a finding aside only by quoting it.
  Race, concurrency, filesystem and security findings are never
  dismissed by rule.

Sources: research of 2026-10-07 (spec-kit /analyze and /clarify, the
Superpowers spec reviewer, TestAny's PRD Reviewer, two 2026 arXiv
studies), and issue #195, folded in with its stale references corrected.

## Coverage scan (FR-102, applied to this change)
| Category | Status |
|---|---|
| Functional scope | Clear |
| Domain & data | Clear (reviewer reports, overrides, (new) marker) |
| Interaction & flow | Clear (pre-pass, then review, then override, then PR) |
| Non-functional | Partial: cost not bounded; resolved as no new bound (NFR-3) |
| Integrations | Clear (FR-51, FR-10, FR-15(d), FR-87) |
| Edge cases & failure | Clear (resume freshness, uncited trace steps, pre-existing debt) |
| Constraints & trade-offs | Clear (benchmark recall recorded as an assumption) |
| Terminology | Clear |
| Completion signals | Clear (every new FR has an observable acceptance) |

## Open assumptions & waivers
- Reviewer cost — resolved: no new bound; reviewers inherit the parent model (FR-87, NFR-3), and cost accounting stays a non-goal.
- Overrides as a rubber stamp — resolved: each override records the finding and a 20–400 character rationale in the PR body and Open questions; a re-run sees it and may block again.
- Grounding's new-file marker — resolved: an explicit `(new)` marker, applied only to TDDs written after this change.
- Simulation trace as theater — resolved: every step cites a source, and an uncited step is `unverified` and cannot alone support a PASS.
- Eval-aggregator check is throughline-specific — resolved: not a requirement; it goes in throughline's repo-level instructions, so the plugin stays generic.
- Scope of the anti-theater rule — resolved: a named non-finding must quote what it dismisses, and no category is blanket-dismissed.
- Whole PRD or delta — resolved: per-requirement checks cover the delta only; set-level checks compare the delta against the whole PRD.

UI requirements: none

## Cascade
Additive: new FRs only, so no existing TDD is stale in its own scope.
Design work lands in the existing surfaces:
- /prd-author skill: FR-102, FR-103, FR-104, FR-105, plus a new
  requirements-reviewer prompt;
- tdd-lint (the FR-51 pre-pass): FR-106, FR-107;
- agents/design-reviewer.md (FR-10): FR-108;
- the /build-tdds review dispatch plus a new build-reviewer prompt
  (FR-15(d)): FR-109;
- every reviewer prompt, including ux-reviewer: FR-110.
Enumerations amended here: FR-87 and NFR-3 now list the requirements
reviewer.
ADR 0015 is only touched in Open questions (cross-model-family review);
it is not superseded.
Follow-up outside the PRD: add throughline's eval-aggregator convention
to the repo's own instructions.

Refs #195

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01NpGG21Y2VGGhTQaFr4oshB
@cahenesy
cahenesy merged commit a690c02 into master Oct 8, 2026
1 check passed
@cahenesy
cahenesy deleted the docs/prd/review-rigor branch October 8, 2026 12:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant