[Change] Calibrate Code Review into a truthful blocking gate #417

Scope rescope — 2026-09-20

Remove the prerequisite on optional preflight hardening #432. Preserve C14, profile, policy, exception authority, calibrated blocking findings, meaningful dogfood measurement, and signature checks. Current-run evidence suffices; historical RED and seals are not required. The proposal was located as uncommitted planning-only files on feature/code-review-15-signal-calibrated-blocking-gate and recovered into local dev and the R09 planning worktree on 2026-09-20. Remote integration remains pending; implementation has not started. Modules C14 #416 shipped but its proposal remains unarchived; protected core C14 adoption is still a separate prerequisite.

Owner: nold-ai/specfact-cli#740; paired product contract: #481. This section supersedes conflicting default-workflow and dependency statements below; other acceptance criteria remain in force.

Why

specfact code review can currently emit error findings and still return zero because severity, score, fixability, enforcement mode, and core fallback behavior are independent. C15 makes the signed policy/profile the sole severity-calibration authority and derives truthful assurance and exit behavior from effective severity, lifecycle, waiver, differential state, and enforcement mode without changing C14 scope or differential contracts.

Parent and coordination

• Parent Feature: Validation Evidence And Governance Runtime Follow-Ups #163
• Parent Epic: specfact code / AI-bloat defense #162
• OpenSpec change: code-review-15-signal-calibrated-blocking-gate
• Prior module change: [Change] Implement truthful Code Review scope and differential enforcement #416 / code-review-14-scope-truth-and-differential-enforcement
• Issue type: User Story
• Project: SpecFact CLI
• Project status: Todo
• Assignee: djm81
• Paired core adoption: [Change] Adopt the signal-calibrated Code Review gate specfact-cli#679 / cli-val-07-code-review-gate-adoption
• Rollout: shadow-first; blocking activation only after a public calibration method is reviewed, its finalized acceptance criteria pass, and both dogfood repositories are green

Blockers and dependency order

Current open prerequisites (verified 2026-09-20):

1. Policy Packs & Enforcement Modes (Advisory/Mixed/Hard) #158 — authoritative policy packs and enforcement modes.
2. [Change] Runtime Exception Management And Enforcement Follow-Up #167 — modules-side exception/suppression runtime.
3. Exception Management — Time-Bound, Tracked Policy Exceptions specfact-cli#248 — trusted exception authority.
4. [Change] Adopt C14 protected PR-range assurance specfact-cli#680 — released protected core C14 adoption.

Closed #416 (signed schema 1.6 producer) and core #237 (profile/config layering) remain historical baseline relationships whose compatible release identities must be verified. Native relationships now include those two closed baselines plus the four open prerequisites. Optional preflight #432 is not a prerequisite. Reverify issue ownership/status and current blockers at implementation start; this dated readback is not continuing readiness approval.

What changes

• Upgrade ReviewReport to schema 1.7 and preserve raw analyzer severity separately from effective severity.
• Block non-shadow changed or full enforcement on every open, applicable, unwaived effective error.
• Make required analyzer/policy uncertainty UNKNOWN and fail non-shadow enforcement closed.
• Keep warning/info advisory, keep info score-neutral, and prohibit score or autofix availability from determining verdict or exit code.
• Preserve truthful assurance_status in shadow while returning zero.
• Reuse C14 identity, source-span, multiset, scope, immutable snapshot, and differential contracts.
• Implement D1–D4: scoped MISSING_ICONTRACT, hierarchy-aware optional-param analysis, AST-backed attribution/exact deduplication, and deterministic calibrated contexts.
• Add audited suppression evidence authenticated against trusted, active, scope-matched exception records.
• Freeze and sign the initial rule/severity mapping and schema compatibility fixtures.
• Dogfood in shadow, remediate both repositories, run the reviewed calibration method, then explicitly activate blocking mode only if the precision target passes.

Acceptance criteria

• Open, applicable, unwaived effective errors yield FAIL and exit 1 in non-shadow changed/full enforcement.
• Required uncertainty yields UNKNOWN and exit 1 in non-shadow enforcement.
• Warning/info-only and waived/advisory reports do not block; info is score-neutral.
• Score and fixable never determine verdict or exit.
• Shadow preserves truthful assurance while exiting zero.
• Suppression directives never delete findings and cannot self-authorize candidate-added exceptions.
• Attribution ambiguity that could affect blocking becomes UNKNOWN; multiplicity is never silently collapsed.
• Schema 1.7 compatibility, signed module identity, dogfood remediation, and the finalized public calibration criteria all pass before blocking activation. The method must be reviewed before sampling; until then, keep shadow.

Measurement and rollback

No reproducible frozen audit method or corpus has been recovered. The earlier 40-PR replay, n=150 stratified sample, seed 20260819, minimum 40 adjudicated errors, weighted precision >=80% and Wilson 95% lower bound >=70% are proposed design targets, not an operative activation rule.

Before sampling, publish and review the corpus references, sampling unit/frame/strata/quotas and selection procedure, adjudication/disagreement rules, precision estimand/weights and statistically justified interval calculation in the public change. No weighted-Wilson construction is implicitly approved. Use one sample and existing review outputs; no per-PR ledger, duplicate suite or evidence service is required.

Keep shadow until the reviewed method's finalized criteria pass; repeat calibration only for relevant rule/policy changes. Retain zero silently collapsed multiplicity and no unresolved blocking attribution conflicts. Rollback restores shadow while retaining schema 1.7 evidence. This bounded C15 activation work does not block ordinary PRs or the independent R09 migration.

Non-goals

• No new analyzer family or AI reviewer.
• No blanket blocking of warning/info findings.
• No candidate-controlled self-waiver.
• No change to C14 Git scope, immutable snapshots, differential matching, or protected-envelope ownership.
• No generalization claim beyond the measured Python repositories.

OpenSpec Change Proposal: code-review-15-signal-calibrated-blocking-gate