Skip to content

Collections Workflow: ERP collection reporting service - #20

Merged
withrvr merged 58 commits into
masterfrom
claude/erp-collection-reporting-workflow-iycqto
Sep 1, 2026
Merged

withrvr merged 58 commits into
masterfrom
claude/erp-collection-reporting-workflow-iycqto

Conversation

@withrvr

@withrvr withrvr commented Sep 1, 2026

Copy link
Copy Markdown
Owner

Summary

Full build of the Collections Workflow service on top of the FastAPI full-stack template: upload an ERP export (Customers, Invoices, Payments, Region_Mapping), get back an overdue collections position, a full data-quality/exception report, and an AI-written summary — nothing silently dropped along the way.

  • Domain core: ingest, validation (14-rule exception catalogue E001-E014), overdue/ageing calculation, control gate
  • Persistence: SQLModel schema (collections_run + 4 child tables), Alembic migrations
  • API: run creation (with dynamic report_date), timeline, overdue, exceptions, regions, summary, file download, email
  • AI layer: Ollama (phi4-mini) via LiteLLM, three-rung fallback (Ollama → cloud → deterministic template), numeric/ID-containment guards against hallucination — used for the run summary narrative and per-exception explanations, never for auto-correcting data
  • Frontend: one-screen run flow, drag-and-drop upload, paginated runs list, public dashboard with charts, full rebrand off the FastAPI template identity
  • Postman collection (verified via Newman against the live backend) and a local gitignored test-fixtures generator for manual testing
  • 168 backend tests passing; full docs in backend/app/collections/README.md, MASTER_PLAN.md, docs/CHANGELOG.md, docs/RULES.md, ARCHITECTURE.md, API.md

Test plan

  • uv run pytest app/collections/tests -v — 168/168 passing
  • Postman collection via Newman — 25/25 assertions passing
  • Frontend bun run build / bun run lint — clean
  • Verified live via headless Chromium: dashboard, upload (drag-and-drop), run-detail, exceptions (no overlap/scroll), runs pagination
  • Verified live: file download, dynamic report_date, emailed summary (Mailpit)

claude and others added 30 commits August 31, 2026 10:45
The full engagement plan and the twenty-questions QA prep, plus the
standing build instructions in AGENTS.md that every subsequent phase
inherits. Every stub added from here on references these by name.
Structurally correct skeleton for the entire assessment service in one
folder (ingest, validate, calculate, control, ai/roles, export,
observability, api, docs, fixtures, tests), per MASTER_PLAN.md section
4. Every module is a stub naming the phase that populates it; no
business logic yet.

config.py wires REPORT_DATE as a real config value now, since the
"never datetime.now()" rule needs to hold from the first commit.

The seven collections API routers are aggregated behind one router and
wired into app/api/main.py with a single include_router line, the only
template file this service touches.
Route file stubs for the assessment UI (runs, run-detail, exceptions,
summary, upload, mapping), per MASTER_PLAN.md section 4. Each renders
a placeholder naming the phase that builds it for real; no nav wiring
yet since the real frontend work starts at Phase 8.
Keep a Changelog at the repo root, seeded with the v0.1.0 Foundation
entry, and the MR description template from MASTER_PLAN.md section 8
for every phase going forward.
…dependency

The given assessment workbook, committed as a reproducible fixture per
MASTER_PLAN.md section 3. openpyxl is the only new runtime dependency
Phase 1 needs, to read typed cells (Decimal/date) directly rather than
via pandas.
Frozen dataclasses (CanonicalCustomer/Invoice/Payment/RegionMap,
CanonicalDataset) plus tier-1 exact-match column resolution and an
openpyxl-based loader. Every row that can be structurally parsed loads
as-is; business-rule exclusions (unknown customer, bad status, etc.)
are left to calculate/, not filtered here. load_warnings only covers
rows missing their own primary identifier, never a business exclusion.
Pure functions implementing the workbook's rules: outstanding excludes
non-positive/late/customer-mismatched payments but keeps payments dated
before their own invoice's invoice date (a Phase 2 exception-report
flag, not a Phase 1 exclusion — see README.md Assumptions); overdue is
due-date-strictly-before-report-date AND outstanding > 0; region falls
back to Region_Mapping-by-State when Customer Region is blank; ageing
buckets are a documented default since the workbook doesn't specify
them.

Includes boundary tests: an invoice due exactly on the report date is
not overdue, and a payment landing exactly on the report date counts.
…cript

Regression test asserting the hand-derived reference numbers against
dataset A: 15 overdue invoices, Rs 12,02,000 total outstanding, West
the heaviest region, plus the payment-before-invoice and
due-exactly-on-report-date regressions. reference_summary.py is a thin
printer over the same calculate/ functions for a quick sanity check.
README.md: business rules, how to run, and the two judgement calls that
matter for the reference numbers (payment-before-invoice still counts;
ageing bucket boundaries are a standard-practice default).
DEVELOPMENT.md: pytest and reference-script commands.
CHANGELOG.md: 0.2.0 entry.
Phase 1: Domain core
One dataclass for every data-quality/business-rule finding: rule code,
category, plain-English message, severity (business weight, distinct
from run_events' pipeline-stage level in Phase 3), and the natural
keys of the record it concerns. detail stays natively typed
(Decimal/date), since JSON serialization is Phase 3's concern.
…decision

SheetSchema constants re-export ingest/resolver.py's header dicts as a
single source of truth, plus sheet_row_counts(). Deliberately not
Pandera: by the time a rule sees a CanonicalDataset there is no
dtype-mismatch state left to catch, since Phase 1's loader already
coerces every cell to Decimal/date at load time via openpyxl, never a
DataFrame. Reasoning documented in ARCHITECTURE.md; QA_PREP.md's Q20
answer corrected to match (it previously claimed Pandera was already
in use).
RULE_REGISTRY starts empty; each rule PR adds its own entries in the
same commit as the rule function, so tests/validate/test_coverage.py
(added once rules exist) can iterate the registry instead of
hardcoding rule names and can never drift from the catalogue.
run_all_rules/exceptions_by_rule are registry-size-agnostic by
design, verified here against an empty dataset and an empty registry.
Phase 2.1: Validation scaffold
An invoice with no Due Date cannot be classified overdue or current,
so it is excluded from the overdue report and flagged here rather
than guessed at or silently dropped. Fires on INV-1027 in dataset A.

Refs: docs/RULES.md
A negative or zero invoice amount is either a data-entry error or an
unreclassified credit; excluded from the overdue report rather than
silently netted in. Fires on INV-1028 (-45,000) in dataset A.

Refs: docs/RULES.md
Cancelled invoices are correctly excluded from the overdue report by
calculate/overdue.py's ELIGIBLE_STATUS; this rule surfaces that
exclusion instead of letting the invoice silently disappear. Fires on
INV-1029 in dataset A.

Refs: docs/RULES.md
Same treatment as E011: correctly excluded from the overdue report by
status, surfaced here rather than silently vanishing. Fires on
INV-1030 in dataset A.

Refs: docs/RULES.md
Two or more invoices sharing a non-blank Source_System_Ref could be a
duplicate entry or a legitimately split sales order -- flagged for
investigation rather than guessed at. Emits one row per affected
invoice, naming its sibling(s), so the finding is visible regardless
of which record a reviewer looks at first. Fires on INV-1011 and
INV-1031 (both 'EMP-SO-0011') in dataset A.

Refs: docs/RULES.md
Phase 2.2: Invoice-quality rules (E001, E004, E011, E012, E013)
Format-only check against the standard 15-character GSTIN pattern, no
checksum validation -- a deliberate, documented limitation. Fires on
C023 ('INVALIDGSTIN') in dataset A. Doesn't feed outstanding/overdue
math, so nothing downstream is excluded; flags master-data quality
only.

Refs: docs/RULES.md
Same master-data-quality rationale as E006, kept as a separate code
because "missing" and "malformed" call for different remediation.
Fires on C025 in dataset A.

Refs: docs/RULES.md
Phase 2.3: Customer-quality rules (E006, E008)
A zero or negative payment amount cannot reduce an outstanding
balance; excluded from the valid-payments total (mirrors
outstanding.py's is_valid_payment > 0 check) rather than silently
counted. Fires on PAY-2018 (0) and PAY-2029 (-5,000) in dataset A.

Refs: docs/RULES.md
Excluded from this period's valid-payments total per the workbook's
own rule, but still flagged (warning, not error -- it's expected
timing, not bad data) since the workbook's exception categories
explicitly list it and a reviewer should see why a position didn't
move rather than infer it. Fires on PAY-2020 (2026-08-02) in dataset A.

Refs: docs/RULES.md
Phase 2.4: Payment-quality rules (E005, E009)
Checked against both invoices and payments, since either kind of
record can carry a dangling Customer_ID. Fires independently of E003
when both a payment's customer and invoice reference are unknown
(PAY-2025) -- two true, independently actionable facts, not one
combined flag. Fires on INV-1026 and PAY-2025 (both C999) in
dataset A.

Refs: docs/RULES.md
A payment referencing an invoice that doesn't exist in the Invoices
sheet is never applied to anything, since outstanding.py's payment
index is keyed by real invoice IDs. Fires on PAY-2025 (Invoice_ID
INV-9999) in dataset A -- the same row that independently fires E002
for its unknown customer reference.

Refs: docs/RULES.md
Payments can reference an invoice belonging to a different customer.
PAY-2026 pays INV-1002 under C003 while the invoice belongs to C002.
The payment is excluded from the paid total and flagged rather than
reassigned, since reassignment would be a silent data correction.

Refs: docs/RULES.md
Flags a payment dated before its own invoice's invoice date as a
data-quality item, without excluding it from compute_outstanding --
the workbook doesn't authorize that exclusion, only this rule's
documented assumption does (see README.md). Fires on PAY-2027 against
INV-1004 in dataset A.

Refs: docs/RULES.md
claude and others added 28 commits August 31, 2026 11:44
Not on the assessment's required list -- added because the data
called for it (QA_PREP.md Q5). Reuses outstanding.py's own
valid_payments_total so this rule can never disagree with the real
calculation about what counts as a valid payment, and floors nothing
itself since compute_outstanding already floors at zero -- this rule
only explains why. Fires on INV-1007 (88,000 invoiced, 110,000 paid,
22,000 overpaid) in dataset A.

All 14 rules now implemented and registered; RULES.md's catalogue is
complete and matches the code.

Refs: docs/RULES.md
Phase 2.5: Cross-reference rules (E002, E003, E007, E010, E014)
Asserts every rule in RULE_REGISTRY fires at least once on dataset A
(the Phase 2 done-when criterion), that the registry covers exactly
E001-E014, and that the total row count is 17. Iterates the registry
rather than hardcoding rule names, so it can never drift from the
catalogue as rules are added or changed.
Two internal-consistency identities (not the independently-derived SQL
recompute -- that's Phase 3+): every invoice's outstanding + valid_paid
equals invoice_amount + overpaid, summed across the full 36-invoice
portfolio; and every payment rupee is either applied or explicitly
unapplied, with every unapplied payment traceable to an exception row.

Corrects a flawed assumption in an earlier design draft, which claimed
"overpaid" is nonzero only for INV-1007 (E014). It's also nonzero for
INV-1028: a negative invoice_amount with zero payments produces
max(0, 0 - (-45000)) = 45000 under the same formula, since
compute_outstanding's floor doesn't distinguish "genuine overpayment"
from "floored negative amount." Verified directly against dataset A
rather than assumed; both identities hold with total_overpaid =
67,000 (22,000 + 45,000), not 22,000.
README.md: "Validation performed" filled in (rule coverage,
reconciliation identities, exact-ID assertions), status updated to
Phase 2 complete. DEVELOPMENT.md: status and testing sections updated
for the full 107-test suite. RULES.md: reordered into E001-E014
numeric order for the final catalogue (previously in PR-landing
order), status line updated to 14/14 complete, dataset-A row-count
cross-check noted against QA_PREP.md Q7. CHANGELOG.md: 0.3.0 entry.
Phase 2.6: Rule coverage, reconciliation tests, docs close-out
Three issues hit bringing the stack up on a fresh machine, unrelated to
any phase's business logic:

- frontend/src/routeTree.gen.ts was stale: the collections/*.tsx route
  files existed but the generated route tree was never refreshed after
  they were added, breaking `tsc` in the backend Docker build (the
  frontend is built into the backend image). Regenerated via `bun run
  build`.
- compose.override.yml published the dev Postgres container on host
  port 5432, which collides with a native postgresql service already
  running on many dev machines. Remapped to 5433; internal
  container-to-container traffic (db:5432) is unaffected.
- scripts/dev-tmux.sh: reusable tmux session (compose/backend/frontend/
  git/ngrok windows) for local development.
Every uploaded workbook becomes a real Run: PENDING -> RUNNING ->
COMPLETED or FAILED, persisted alongside its RunEvent timeline,
RunException rows, and RunInvoicePosition rows. A deliberately broken
file produces a FAILED run with a readable, traceback-free error
message rather than a raised exception or a leaked KeyError/ValueError.

- errors.py: PipelineError, the single error contract every run
  failure maps to (code, stage, user_message, detail).
- models.py: Run/RunEvent/RunException/RunInvoicePosition, prefixed
  collections_*. status/stage/level are plain indexed strings rather
  than native Postgres enums, since a later phase (Phase 5 adds
  PASSED/BLOCKED) growing the catalogue would otherwise need a
  migration every time.
- observability/events.py: emit(), writing the run_events timeline,
  plus to_json_detail() to make Decimal/date values JSON-column safe.
  observability/logging.py: the separate developer-facing channel
  (stdlib logging for now -- Phase 12 upgrades it to structlog once
  there's a real consumer for JSON logs).
- service.py: execute_run, the orchestrator (load -> validate ->
  calculate -> persist). Never raises; always returns a Run.
- ingest/resolver.py: SheetMissingError, split out from
  ColumnResolutionError so a wholly absent sheet ("The workbook has no
  'Payments' sheet.") reads differently from a present sheet missing
  columns.
- Alembic migration for the four collections_* tables.
  compose.override.yml's dev backend command now runs
  `alembic upgrade head` before `fastapi dev`, verified against a
  fresh Docker volume end to end.

Refs: docs/ARCHITECTURE.md, docs/RULES.md
15 new tests (107 -> 122):

- tests/persistence/test_service.py: the happy path against dataset A
  (matches the Phase 1 reference numbers via a real Run/RunEvent/
  RunException/RunInvoicePosition round trip), event ordering, and
  five deliberately broken workbooks (missing sheet, missing column,
  non-xlsx file, an unparseable cell) each asserted to a specific
  error_code and a message containing neither "Traceback" nor "Error"
  -- not just that some message exists.
- tests/persistence/test_sql_crosscheck.py: the independently-derived
  SQL recompute MASTER_PLAN.md section 10 asks for and
  test_reconcile.py's docstring deferred here. A SQL SUM/GROUP BY over
  persisted RunInvoicePosition rows -- a genuinely different code path
  from scripts/reference_summary.py's in-memory loop -- agrees with
  the same reference numbers.
- tests/persistence/conftest.py: in-memory SQLite session fixture, not
  Postgres -- fast, no docker compose needed to run pytest. The real
  schema is exercised via `alembic upgrade head` and guarded against
  drift by `alembic check`.
- tests/ingest/workbook_builder.py: minimal in-memory workbook builder
  shared by the failure-path tests above and a new SheetMissingError
  test in tests/ingest/test_loader.py.
- tests/test_errors.py: PipelineError's own contract.

Refs: docs/DEVELOPMENT.md
README.md: status, updated test count (122), the SQL crosscheck and
error-contract test entries in Validation performed, and how to run
execute_run against the real Postgres.

ARCHITECTURE.md: Components filled in for real (was a placeholder
since Phase 0), Data flow (the four-stage orchestrator, and which
stages are named but not yet called), The error contract
(PipelineError, and exactly which exception maps to which code), and
why run status/event stage/level are plain strings rather than native
Postgres enums.

DEVELOPMENT.md, API.md: status bumped, new commands, and a pointer
from API.md to where the error contract actually lives so Phase 4
doesn't have to re-explain it.

CHANGELOG.md: [0.4.0] entry.

Refs: MASTER_PLAN.md section 5 (doc ownership)
Caught in self-review before merge: an earlier attempt to work around a
ty (not mypy) type-checker quirk on the sa_type=DateTime(...) overload
made created_at Optional, which drifted from the already-generated
migration (NOT NULL) without changing ty's diagnostic count at all.
Reverted to the non-Optional annotation the migration was actually
generated against -- `alembic check` now reports no drift.
Phase 3: Persistence and run events
This is no longer presented as a bare fastapi/full-stack-fastapi-template
checkout -- it's the ERP Collection Reporting Workflow, several phases
in. Root-level docs still carried the template's placeholder branding.

- README.md: was the template's own; renamed to FASTAPI.md (kept
  verbatim as reference -- MASTER_PLAN.md section 4 explicitly commits
  the base template unchanged so the diff stays legible). New
  README.md written as this project's actual entry point: what it is,
  a doc map into backend/app/collections/README.md (the real source of
  truth), and a quickstart that matches what's actually been verified
  end to end.
- release-notes.md: was the template's own upstream history (132K,
  hundreds of entries, not ours); renamed to FASTAPI-release-notes.md
  with a pointer to docs/CHANGELOG.md for this project's own history.
  Dropped the now-dead add-release-date pre-commit hook and fixed the
  one live reference to the old filename (deploy.yml's paths-ignore).
- LICENSE: kept the original MIT notice (required by its own terms),
  added this project's own copyright line alongside it.
- CONTRIBUTING.md: was the template's open-source community-PR policy
  (Discord links, "we'll close AI-generated PRs"); replaced with this
  repo's actual solo-assessment workflow (branch/PR/tag per phase,
  AGENTS.md, ponytail-review before merge).
- development.md, deployment.md, deployment-docker-compose.md,
  backend/README.md, frontend/README.md: retitled off the "FastAPI
  Project" placeholder; deployment-docker-compose.md's example
  domain/PROJECT_NAME updated to match.
- .env: PROJECT_NAME updated (flows into Swagger title and email
  sender name).
A reviewer can now drive the whole flow from /docs: upload a workbook,
watch it become a Run, read its overdue/exceptions/regions/summary,
and see exactly why it failed if it did.

- api/schemas.py: Pydantic response models, separate from models.py's
  SQLModel tables.
- api/deps.py: get_run_or_404, shared by every report endpoint.
- POST /collections/runs/: upload + synchronous execute_run. Always
  200 with a Run -- a bad file produces a FAILED run in the body,
  never a 500.
- GET /collections/runs/, /runs/{id}, /overdue/, /exceptions/
  (rule_code/severity filters), /regions/, /summary/,
  /run-log/{id}/events.
- No auth: this is an internal ops tool, not a multi-tenant product,
  and nothing in MASTER_PLAN.md asks for one.
- summary.py returns the numeric summary + region breakdown now;
  Phase 5 layers the control gate on the same response.

Refs: docs/API.md, docs/ARCHITECTURE.md
Every run now either PASSES or is BLOCKED at the 5% exception-rate
gate -- never silently reported as clean, and never tuned to pass:
dataset A blocks (47.2%), dataset B passes (0%), both verified end to
end and visible via GET /collections/summary.

- control/gate.py: evaluate_gate, a pure function. QA_PREP.md Q8's
  denominator ambiguity ("exception records over invoice records")
  resolved by reporting both exception_row_rate (drives the gate) and
  distinct_invoice_rate, rather than picking one silently.
- ai/fallback.py: render_summary, a deterministic Jinja narrative --
  no LLM, no network, cannot fail short of a programming bug. Built
  first, as the last rung of Phase 6's LLM fallback chain.
- service.py: wires control + summarise into execute_run between
  calculate and persist.
- models.py: Run gains gate_threshold/exception_row_rate/
  distinct_invoices_affected/distinct_invoice_rate/narrative.
  RunStatus retires the placeholder COMPLETED for the real
  PASSED/BLOCKED outcomes -- no schema change needed, since status was
  already a plain indexed string, not a native enum.
- scripts/make_fixtures.py: generates fixtures/dataset_b_clean.xlsx by
  removing every row dataset A's own rules touch -- passes by
  construction, not hand-picked to look clean.
- api/summary.py: SummaryOut now carries the full "blocked payload"
  QA_PREP.md Q7 describes -- the rate, both denominators, the
  threshold, and the deterministic narrative.

Refs: docs/ARCHITECTURE.md's control gate section, docs/API.md
Phase 5: Control gate and deterministic summary
…rung fallback

Every run's narrative can now come from a real local LLM (phi4-mini via
Ollama) -- but never has to, and never gets the final word: a numeric
guard rejects any output that invents a number, falling through to the
deterministic template built in Phase 5. Verified end to end against a
real, unmocked Ollama instance, not just unit-tested in isolation.

- ai/provider.py: call_ollama/call_cloud, the LiteLLM seam. Cloud is
  opt-in only (COLLECTIONS_CLOUD_LLM_MODEL), never required.
- ai/guard.py: numbers_are_contained -- every number in an LLM output
  must appear in its input, or the output is rejected outright.
- ai/roles/summary_narrator.py: narrate(), the three-rung chain
  (Ollama -> cloud -> ai/fallback.render_summary), numeric-guarded at
  every rung. Never raises -- the deterministic third rung always
  succeeds.
- ai/context.py: anydoc workbook-to-Markdown for LLM context only --
  not consumed by any Phase 6 role (summary_narrator narrates the
  already-computed metrics dict, never raw sheets); ready for Phase
  7's exception_explainer and Phase 10's schema_mapper.
- observability/metrics.py: llm_calls_total, a plain in-process
  counter for now -- Phase 6's own done-when, verified directly
  against real local Ollama calls.
- models.py: Run.summary_source records which rung actually produced
  a run's narrative, surfaced via GET /collections/summary.
- compose.override.yml: dev backend wired toward Ollama at
  host.docker.internal:11434, with the binding caveat documented
  (Ollama must listen on 0.0.0.0, not loopback-only, for a container
  to reach it -- verified the fallback chain degrades correctly
  either way).

New dependencies: litellm, firecrawl-anydoc.

Refs: docs/ARCHITECTURE.md's LLM layer section, docs/API.md
Phase 6: LLM layer (local Ollama, guarded, three-rung fallback)
…ever auto-fixes

Every one of the 14 rules that fires in a run now carries a real
cause/impact/suggested-fix/owner explanation -- one per rule code, not
per row, since "why did E001 fire" has a single answer regardless of
how many invoices it hit. Guarded against invented record IDs the same
way Phase 6 guards invented numbers. auto_fixable is hardcoded False
everywhere: the LLM explains, it does not get to decide something is
safe to auto-correct.

- ai/guard.py: ids_are_contained -- an explanation naming a record ID
  outside its own batch is rejected outright.
- ai/roles/exception_explainer.py: explain_rules, the same three-rung
  fallback chain as summary_narrator. RULE_METADATA is the
  deterministic third rung, in the same words docs/RULES.md's own
  "Why" prose already establishes for each code -- never improvised.
  Cached per (rule_code, category, sorted affected IDs), so re-running
  against the same dataset never re-hits the LLM.
- models.py: RunRuleExplanation, one row per (run, rule_code).
- service.py: validate stage also explains and persists.
- api/exceptions.py: ExceptionOut carries the joined explanation on
  every row (cause/impact/suggested_fix/owner/auto_fixable/
  explanation_source).

Verified against a real, unmocked local Ollama: all 14 rules on
dataset A explained, zero invented IDs.

Refs: docs/ARCHITECTURE.md's Exception Explainer section, docs/API.md
A reviewer who has never seen a terminal can now upload a workbook and
read every output: five routes (upload, runs, run-detail, exceptions,
summary), no auth (matching the API), the template's own shadcn/ui
components throughout, no new UI library.

- run-detail's timeline groups run_events by stage, colors each group
  green/amber/red by the worst level inside it -- MASTER_PLAN.md
  section 7's exact design ("Green through calculate, amber at control
  when the gate blocks, red on failure") -- expandable per stage to
  the raw events, never a JSON dump.
- exceptions joins each row's Phase 7 explanation inline
  (cause/impact/suggested-fix/owner), filterable by severity.
- summary renders the full "blocked payload" plus a badge naming
  which of the three LLM rungs produced the narrative.
- runId is a TanStack Router search param (zod-validated), not a path
  param -- route files stay flat, matching the Phase 0 scaffold's own
  names exactly.
- Client regenerated (scripts/generate-client.sh) against the current
  OpenAPI schema: CollectionsService now exists.

Verified end to end in a real, unmocked headless Chromium against the
actual Docker-served app -- upload dataset_a_original.xlsx, land on
its BLOCKED run-detail page, expand the timeline, click through to
exceptions and summary, zero console errors -- not just tsc/vite build
passing. Also verified a PASSED run (dataset B) renders correctly on
the runs list.

This closes out MASTER_PLAN.md's submission scope, Phases 0-8. Phases
9-13 are presentation-scope extras for the later live demo.

Refs: docs/ARCHITECTURE.md's frontend section, docs/DEVELOPMENT.md
Phase 8: Frontend -- submission scope complete
Two independent asks, done together since both touch the same surface:

Postman (postman/): full collection covering the whole API -- upload,
runs list/detail, overdue, exceptions (with filters), regions, summary,
run-log, plus a dedicated error-handling folder (a real corrupt-file
fixture proving FAILED-never-500, an unknown-id 404). Test scripts
chain run_id automatically between requests. Verified against the live
backend via Newman before committing: 13 requests, 25 assertions, 0
failures -- not a hand-written guess at response shapes. Two
environments (local Docker, the ngrok tunnel).

Frontend: run-detail is now the one-screen automated view MASTER_PLAN.md
Phase 8 was aiming at -- upload a file, land here, and see the timeline,
the block/pass banner, the AI narrative, the region breakdown, and the
first 10 exceptions (cause/impact/fix/owner, expandable inline) all on
one page, no further clicks required. Extracted ExceptionsPanel and
SummaryPanel as shared components so the standalone /exceptions and
/summary pages (still there, for filtering and direct links) render off
the exact same code, not a second copy that can drift.

Verified in a real, unmocked headless Chromium against the actual
Docker-served app: upload dataset_a_original.xlsx, everything renders
inline on run-detail with zero clicks, expand a row inline to see its
fix, confirm the "open full page" deep links still work. Zero console
errors.

Refs: postman/README.md, backend/app/collections/README.md's frontend section
…shboard, full rebrand

- Persist uploads to disk, add per-run download endpoint/link.
- report_date becomes a user-overridable form field (was fixed setting).
- POST /collections/runs/{id}/send-email for run summaries via SMTP/Mailpit.
- Pagination on the runs list; new public dashboard at / with charts.
- ExceptionsPanel rebuilt table->cards to fix overlapping/scrolling text;
  hover tooltips added throughout.
- AI narrative (LLM prompt + deterministic fallback) rewritten longer/richer:
  region, ageing, and top-rule breakdowns.
- Drag-and-drop on the upload page.
- Full rebrand off the FastAPI template identity (logo, titles, favicon,
  footer); FASTAPI.md/FASTAPI-release-notes.md removed.
- Local gitignored test-files/ generator script for manual Postman/UI testing.

168/168 backend tests pass; Postman collection re-verified via Newman
(25/25 assertions); frontend build+lint clean; verified live via headless
Chromium screenshots (dashboard, run-detail, expanded exception card, upload).
User asked for a way back to / from upload/runs pages.
README.md rewritten around a "why it's built this way" narrative, a
mermaid flow diagram of the six pipeline stages, an honest walkthrough
of how the two LLM roles and anydoc are actually used (and what anydoc
is reserved for, not overstated as active), and a UI section covering
drag-and-drop and step-by-step failure visibility explicitly. Moved the
agent-tooling (ponytail/caveman) and quality-gate content out of the
README and into DEVELOPMENT.md's new "Quality gates and agent-assisted
development" section, since that's dev-guide material, not product
material -- no duplication across files. Filled in ARCHITECTURE.md's
previously-stub trade-offs section. Added an empty images/ folder with
a filename guide so screenshots can be dropped in and render in the
README automatically. Corrected the stale 122/162 test counts to 168
throughout.
Renamed to consistent screenshot-*.png/jpg naming, added a
Screenshots section covering dashboard, upload, run detail (desktop
and mobile), runs list, and the emailed summary.
@withrvr
withrvr merged commit 7a3c6c2 into master Sep 1, 2026
4 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants