Repository navigation
PRD: review rigor — one review contract across every phase - #198
Merged
Merged
Conversation
Each authoring phase already ends in a review by a fresh worker, but the depth varied. Requirements were self-reviewed only, design had a fixed reviewer, and the /build-tdds review prompt was improvised on every run. This update gives every phase the same review contract (FR-102–FR-110): - FR-102: a coverage scan in the /prd-author interview. Nine categories are marked Clear / Partial / Missing in a PR-body table, and the most consequential, least settled questions are asked first. - FR-103: a mechanical PRD pre-pass that blocks on placeholders, missing acceptance clauses, and duplicate or malformed ids, and flags unquantified terms. - FR-104 and FR-105: an independent requirements review that blocks the PRD PR (`REQUIREMENTS_REVIEW: PASS|BLOCK`). It checks per-requirement quality and set-level consistency of the delta against the whole PRD, from product, implementer, tester and stakeholder viewpoints. A finding can be overridden with a recorded rationale that a re-run can reject. - FR-106 and FR-107: the TDD pre-pass checks that Touched-files paths exist or are marked `(new)`, going forward only, and blocks phrases that defer codebase discovery to the build. - FR-108: the design review adds a cited simulation trace, an alternatives table for every blocker or major finding, terminology/duplication checks, and a metrics line. - FR-109: the build review runs from a fixed reviewer prompt shipped with the plugin. The security checklist always applies, the trace must cite its steps, and findings on code the diff did not change are never a FAIL. - FR-110: a reviewer may set a finding aside only by quoting it. Race, concurrency, filesystem and security findings are never dismissed by rule. Sources: research of 2026-10-07 (spec-kit /analyze and /clarify, the Superpowers spec reviewer, TestAny's PRD Reviewer, two 2026 arXiv studies), and issue #195, folded in with its stale references corrected. ## Coverage scan (FR-102, applied to this change) | Category | Status | |---|---| | Functional scope | Clear | | Domain & data | Clear (reviewer reports, overrides, (new) marker) | | Interaction & flow | Clear (pre-pass, then review, then override, then PR) | | Non-functional | Partial: cost not bounded; resolved as no new bound (NFR-3) | | Integrations | Clear (FR-51, FR-10, FR-15(d), FR-87) | | Edge cases & failure | Clear (resume freshness, uncited trace steps, pre-existing debt) | | Constraints & trade-offs | Clear (benchmark recall recorded as an assumption) | | Terminology | Clear | | Completion signals | Clear (every new FR has an observable acceptance) | ## Open assumptions & waivers - Reviewer cost — resolved: no new bound; reviewers inherit the parent model (FR-87, NFR-3), and cost accounting stays a non-goal. - Overrides as a rubber stamp — resolved: each override records the finding and a 20–400 character rationale in the PR body and Open questions; a re-run sees it and may block again. - Grounding's new-file marker — resolved: an explicit `(new)` marker, applied only to TDDs written after this change. - Simulation trace as theater — resolved: every step cites a source, and an uncited step is `unverified` and cannot alone support a PASS. - Eval-aggregator check is throughline-specific — resolved: not a requirement; it goes in throughline's repo-level instructions, so the plugin stays generic. - Scope of the anti-theater rule — resolved: a named non-finding must quote what it dismisses, and no category is blanket-dismissed. - Whole PRD or delta — resolved: per-requirement checks cover the delta only; set-level checks compare the delta against the whole PRD. UI requirements: none ## Cascade Additive: new FRs only, so no existing TDD is stale in its own scope. Design work lands in the existing surfaces: - /prd-author skill: FR-102, FR-103, FR-104, FR-105, plus a new requirements-reviewer prompt; - tdd-lint (the FR-51 pre-pass): FR-106, FR-107; - agents/design-reviewer.md (FR-10): FR-108; - the /build-tdds review dispatch plus a new build-reviewer prompt (FR-15(d)): FR-109; - every reviewer prompt, including ux-reviewer: FR-110. Enumerations amended here: FR-87 and NFR-3 now list the requirements reviewer. ADR 0015 is only touched in Open questions (cross-model-family review); it is not superseded. Follow-up outside the PRD: add throughline's eval-aggregator convention to the repo's own instructions. Refs #195 Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01NpGG21Y2VGGhTQaFr4oshB
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Each authoring phase already ends in a review by a fresh worker, but the
depth varied. Requirements were self-reviewed only, design had a fixed
reviewer, and the /build-tdds review prompt was improvised on every run.
This update gives every phase the same review contract (FR-102–FR-110):
are marked Clear / Partial / Missing in a PR-body table, and the most
consequential, least settled questions are asked first.
acceptance clauses, and duplicate or malformed ids, and flags
unquantified terms.
PRD PR (
REQUIREMENTS_REVIEW: PASS|BLOCK). It checks per-requirementquality and set-level consistency of the delta against the whole PRD,
from product, implementer, tester and stakeholder viewpoints. A finding
can be overridden with a recorded rationale that a re-run can reject.
exist or are marked
(new), going forward only, and blocks phrasesthat defer codebase discovery to the build.
alternatives table for every blocker or major finding,
terminology/duplication checks, and a metrics line.
the plugin. The security checklist always applies, the trace must cite
its steps, and findings on code the diff did not change are never a
FAIL.
Race, concurrency, filesystem and security findings are never
dismissed by rule.
Sources: research of 2026-10-07 (spec-kit /analyze and /clarify, the
Superpowers spec reviewer, TestAny's PRD Reviewer, two 2026 arXiv
studies), and issue #195, folded in with its stale references corrected.
Coverage scan (FR-102, applied to this change)
Open assumptions & waivers
(new)marker, applied only to TDDs written after this change.unverifiedand cannot alone support a PASS.UI requirements: none
Cascade
Additive: new FRs only, so no existing TDD is stale in its own scope.
Design work lands in the existing surfaces:
requirements-reviewer prompt;
(FR-15(d)): FR-109;
Enumerations amended here: FR-87 and NFR-3 now list the requirements
reviewer.
ADR 0015 is only touched in Open questions (cross-model-family review);
it is not superseded.
Follow-up outside the PRD: add throughline's eval-aggregator convention
to the repo's own instructions.
Refs #195
🤖 Generated with Claude Code
https://claude.ai/code/session_01NpGG21Y2VGGhTQaFr4oshB