Repository navigation
Conversation
Every run now either PASSES or is BLOCKED at the 5% exception-rate
gate -- never silently reported as clean, and never tuned to pass:
dataset A blocks (47.2%), dataset B passes (0%), both verified end to
end and visible via GET /collections/summary.
- control/gate.py: evaluate_gate, a pure function. QA_PREP.md Q8's
denominator ambiguity ("exception records over invoice records")
resolved by reporting both exception_row_rate (drives the gate) and
distinct_invoice_rate, rather than picking one silently.
- ai/fallback.py: render_summary, a deterministic Jinja narrative --
no LLM, no network, cannot fail short of a programming bug. Built
first, as the last rung of Phase 6's LLM fallback chain.
- service.py: wires control + summarise into execute_run between
calculate and persist.
- models.py: Run gains gate_threshold/exception_row_rate/
distinct_invoices_affected/distinct_invoice_rate/narrative.
RunStatus retires the placeholder COMPLETED for the real
PASSED/BLOCKED outcomes -- no schema change needed, since status was
already a plain indexed string, not a native enum.
- scripts/make_fixtures.py: generates fixtures/dataset_b_clean.xlsx by
removing every row dataset A's own rules touch -- passes by
construction, not hand-picked to look clean.
- api/summary.py: SummaryOut now carries the full "blocked payload"
QA_PREP.md Q7 describes -- the rate, both denominators, the
threshold, and the deterministic narrative.
Refs: docs/ARCHITECTURE.md's control gate section, docs/API.md
withrvr
merged commit Aug 31, 2026
6377c5a
into
claude/erp-collection-reporting-workflow-iycqto
3 of 7 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 5: Control gate and deterministic summary
What this adds
The 5% exception-rate control gate (
control/gate.py), never tuned to pass, reporting both denominators per QA_PREP.md Q8. A deterministic Jinja narrative (ai/fallback.py), no LLM, built first as the fallback chain's last rung.Run.statusis nowPASSED/BLOCKED/FAILED(retiring the Phase 3 placeholderCOMPLETED).GET /collections/summarycarries the full "blocked payload."Assessment requirement served
MASTER_PLAN.md Phase 5 done-when: "dataset A blocks, dataset B passes, both visible via API."
scripts/make_fixtures.pygeneratesdataset_b_clean.xlsx(every row touched by any exception rule removed -- passes by construction), and both outcomes are asserted directly against realexecute_runcalls intests/persistence/test_service.pyand through the actual HTTP API intests/api/test_api.py.Docs updated
Verification
Known gaps carried to next phase
dataset_b_clean.xlsxhas 0 exceptions (not just under 5%) -- functionally sufficient for this phase's done-when; Phase 9's fixture pass can hand-tune it to a nonzero-but-passing rate if a more realistic demo number is wanted.