Skip to content

fix(ui): show only what the server confirmed, and say when the worker is down - #623

Open
privatejoel wants to merge 2 commits into
mainfrom
fix/honest-send-status
Open

privatejoel wants to merge 2 commits into
mainfrom
fix/honest-send-status

Conversation

@privatejoel

Copy link
Copy Markdown
Collaborator

What

Several steps told the operator that something happened when the server had not done it. Line numbers are on main before this PR.

  1. Teams outbound answered ok: true with status 202 and sent nothing (src/api/_lib/comms-send.js:80). The copilot confirm path could reach it (src/api/copilot/confirm.js:52), and Ask Anvil said "Done".
  2. WhatsApp with no provider returned ok: true (src/api/whatsapp/send.js:121). The row defaulted to the email channel with a phone number as the address.
  3. Quote send inserted the email as queued and flipped the quote to SENT, while the drawer said "Quote sent to customer" (src/api/quotes/send.js:455, :530). Queued email drains only through the agents/run reaper, so it left days late or never.
  4. Logistics and inventory alert emails were queued with no recipient, so the reaper could only fail them (src/api/_lib/logistics/notifications.js:192, src/api/_lib/inventory/notifications.js:113).
  5. Bell mark-read errors were swallowed (src/v3-app/components/Shell.tsx:135).
  6. Communications list fetched without auth and with wrong action names, so it was always empty (src/v3-app/screens/comms.tsx:84).
  7. Email Triage Promote and Attach toasted success and did nothing useful (src/v3-app/screens/email.tsx:128, :144).
  8. Extraction watch said "waiting for the background worker" but not whether the worker had run (src/v3-app/components/BackgroundExtractionWatch.tsx:155). A stale tick wrote only a console.warn (src/api/_lib/heartbeat-check.js:97).

Why

docs/SO_EXECUTION_COHESION.md, plan PR C-5, recommendation R-U6 ("Show only what the server confirmed"). When a screen reports a success that did not happen, people re-check every step by hand. C-5 is one of the decision-free Phase 0 PRs.

How

Commit 1 (0ca8299c, the build):

  • Teams. sendViaChat fails a Teams row with not_configured before the config lookup, so the row never falls through to email. Copilot confirm returns sent plus send_error (new notSentReason). Ask Anvil says "Draft saved, but the message was not sent: reason".
  • WhatsApp. The endpoint returns ok: false, sent: false and code not_configured or provider_error. The row has channel: "whatsapp" and is stored failed, so the email reaper never takes it.
  • Quote send. The email is sent in the request through sendCommunication, as invoices/send does. The quote moves to SENT, learns customer parts and arms its goals only when a provider accepted the email. Otherwise the quote keeps its status and the row is failed, not queued. The drawer says "sent" only on sent: true.
  • Alert emails. New alertRecipientsForRoles (_lib/internal-recipients.js) goes through resolveInternalRecipients. Recipients are the bell's audience: the rule's escalate_roles for logistics, and admin for inventory. To is the first address and CC is the rest. Recipients are resolved once per run. When nobody resolves, the dispatcher writes no row and stamps email_skipped: "no_recipients".
  • Bell. A refused mark-read or mark-all-read is shown with notifyWarn.
  • Communications list. Reads AnvilBackend.communications.list (GET /api/communications) and shows each row's stored status. It does not use the comm_draft and comm_send audit names: comm_send is also written for failed sends.
  • Email Triage. Promote and Attach are disabled, and the screen shows why.
  • Extraction watch. The strip shows when the worker last ran, from the cron_health rows that the public /api/health probe already returns, and says when the run is late. The new bellStaleTick, called from cron/daily, writes one admin bell row per tenant when cron/tick is stale.

Commit 2 (80f9435e, the review fixes):

  • Alert email volume (review 1). The logistics digest now stands in for the email too: each digested exception is stamped email_at with email_skipped: "digested". New _lib/alert-email-budget.js caps the logistics_alert and inventory_alert emails. The per-run cap defaults to 10, and an email past it waits for the next run. The per-day cap defaults to 30 per tenant, counted from today's rows of that template. An exception past it is stamped email_skipped: "daily_cap"; its bell still rings and the screen still lists it. Env overrides: LOGISTICS_MAX_ALERT_EMAILS_PER_RUN / _PER_DAY and INVENTORY_MAX_ALERT_EMAILS_PER_RUN / _PER_DAY.
  • Quote SENT update after a real send (review 2). The update is retried once. If it still fails, the handler does not answer 500. It returns sent: true with a warning, writes a quote_send_status_update_failed audit row, and does not arm goals. A guard at the top refuses a later send of the same quote with 409 QUOTE_ALREADY_EMAILED when a sent or replied quote_email row already exists. The drawer shows the warning next to the success.
  • Copilot confirm (review 5). A row left queued that Ask Anvil reported as not sent is now marked failed with not_sent_reason, so the reaper cannot deliver it later. The shared helper failUnsentRow in comms-send.js is used by quote send too. If that update fails, the message says the row is still queued and may be sent later.
  • Wrong reason (review 6). sendCommunication now returns lastError. notSentReason reports a provider that threw (for example an Outlook token refresh) as "The send failed: ...", and no longer says "No provider is configured".
  • Undelivered accept token (review 7). On the not-sent branch, the portal token minted for that attempt is revoked (revoked_at), and portal_url is null in the response.
  • History and metrics (review 8). QuoteHistoryTab labels quote_send_failed "Send failed" (bad) and quote_send_status_update_failed "Sent, status not updated" (warn). isIssued in _lib/metrics/catalog.js now excludes failed, so a failed attempt followed by a successful retry counts once. Failed rows still count in comms_delivery_rate.
  • Tick that never ran (review 9). bellStaleTick also rings when cron_health has no cron/tick row but has a cron/daily row. The wording is "There is no record of the 5-minute worker (cron/tick) running". When the cron/daily row is missing too, it does not ring, because nothing is recording heartbeats.

Behaviour change for review: when no mail provider is configured, a quote now stays DRAFT, as invoice send already does. Before, it became SENT with an email that never left.

No migrations. No new requireAction names.

Tests

New or extended (all behavioural: they call the handler or the function through the existing in-memory mocks):

  • src/v3-app/api-comms-send-teams.test.js. A Teams row is failed with not_configured and is never emailed. Copilot confirm returns sent: false for Teams and for email with no provider, and the row is failed with not_sent_reason, with nothing left queued. Confirm returns sent: true when a provider accepts.
  • src/v3-app/api-whatsapp-send-no-provider.test.js. No provider returns not_configured, with channel whatsapp and status failed. A refusal returns provider_error. An accepted send returns sent: true.
  • src/v3-app/api-quotes-send-status.test.js
    • A direct send marks the quote SENT and arms its goals.
    • With no provider, the quote stays DRAFT and the row is failed.
    • A refusal is reported, and a retry reaches the customer once.
    • New: the below-floor override is recorded only when the email left.
    • New: the accept token of a not-sent attempt is revoked, and that of a sent quote stays live.
    • New: an Outlook failure is named.
    • New: one failed SENT update is retried.
    • New: two failed SENT updates give sent: true with a warning, then 409 on a retry with no second email.
  • src/v3-app/api-alert-email-recipients.test.js
    • Logistics goes To and CC to the bell's roles and follows escalate_roles.
    • When no recipient resolves, no row is written and the exception records why.
    • Recipients resolve once per run.
    • Inventory goes to the admins.
    • New: two admins give To plus CC.
    • New: a digested backlog sends no per-exception email.
    • New: the logistics per-run cap defers the rest, which the next run sends.
    • New: the logistics per-day cap stamps daily_cap and counts only this tenant's rows of this template.
    • New: the inventory run cap, then the day cap.
  • src/v3-app/api-heartbeat-stale-bell.test.js (through the cron/daily handler)
    • A stale tick writes one bell row per tenant that has an admin, and no email.
    • A fresh tick rings nothing, and another stale worker rings nothing.
    • New: the dedup key carries the day.
    • New: no tick row with a daily row rings "no record".
    • New: no daily row either rings nothing.
  • src/v3-app/api-metric-catalog.test.js. comms_sent excludes failed attempts. New: a failed quote email plus its retry counts as one, and still shows in the delivery rate.
  • src/v3-app/components/QuoteDetailDrawer.test.tsx. sent: false shows "Quote not sent: reason" with no success. sent: true toasts sent. New: a warning is shown with notifyWarn.
  • src/v3-app/components/QuoteHistoryTab.test.tsx. New: "Send failed" (bad) and "Sent, status not updated" (warn).
  • src/v3-app/components/Shell.test.tsx, screens/ask-anvil.test.tsx, screens/comms.test.tsx, screens/email.test.tsx, components/BackgroundExtractionWatch.test.tsx. These cover the bell refusal, the not-sent copilot message, the list read and the status tabs, the disabled triage actions, and the worker-last-ran line, including the late wording.

Mutation checks. Each check broke the fix, saw a named test fail, and restored the file byte for byte.

  • Build: 25 of 25 killed.
    • M1 to M3: Teams answers ok: true; Teams falls through to email; confirm always sent: true.
    • M4: the ask-anvil not-sent branch is removed.
    • M5 to M6: WhatsApp answers ok; the WhatsApp channel is dropped.
    • M7 to M10: quote sent is forced; the row is left queued; sendCommunication is skipped; the drawer ignores sent.
    • M11 to M15: the logistics and inventory recipients, the no-row rule and the cache are each broken.
    • M16 to M17: the bell errors are swallowed.
    • M18 to M19: the comms list is broken; the Sent tab filter is broken.
    • M20 to M21: triage Promote and Attach are re-enabled.
    • M22 to M25: the watch line and its stale flag are removed; cron/daily skips the bell; the bell rings for any stale worker.
  • Review fixes: 21 of 21 killed.
    • N1: the digest does not stamp email_skipped.
    • N2: the logistics day cap is ignored.
    • N3: the run cap is ignored.
    • N4: inventory ignores the budget.
    • N5: confirm leaves the row queued.
    • N6: notSentReason ignores lastError.
    • N7: the token is not revoked.
    • N8: the SENT update is not retried.
    • N9: the already-emailed guard is removed.
    • N10: a double failure throws 500.
    • N11: the history label is removed.
    • N12: isIssued counts failed rows again.
    • N13: there is no never-ran bell.
    • N14: the never-ran bell rings without a daily row.
    • N15: the drawer ignores the warning.
    • N16 (review MK): the dedup key has no day.
    • N17 (review MG): the margin override is recorded before the send.
    • N18 (review MJ): inventory drops cc_addrs.
    • N19: lastError is not returned.
    • N20: quote send leaves the row queued.
    • N21: goals are armed after a failed flip.

Gates

On head 80f9435e, with local Node 25.8.1 and npm 11.11.0. npm ci ran from the existing lockfile, and the lockfile is unchanged.

  • npm run check: pass. This includes tsc and the cold import of api/dispatch.js.
  • npm run lint: 0 errors and 511 warnings, the same count as before the review fixes.
  • npm run build: pass. Build output is not committed.
  • npm run verify: pass. docs/RBAC_AUDIT.md and docs/WRITE_PATH_AUDIT.md were reverted after the run.
  • npm run eval:golden: pass, with an average score of 1.000.
  • Full NODE_OPTIONS=--no-experimental-webstorage npx vitest run: 624 of 624 files and 7313 of 7313 tests passed. The known flake did not fire.

Main has moved since the branch was cut (C-4 #612 and docs). The branch was not rebased.

Left out

Declined review findings:

  • Review 3: hide Email Triage from pilot roles in this PR. Declined. Nav visibility has no code-level default. Every item is visible until a tenant admin disables it for a role (src/v3-app/lib/nav-settings.ts, Admin > Navigation). A default "hidden for pilot roles" is the pilot nav preset that plan PR C-8 builds (nav-visibility.pilot-preset.test.ts). The user-facing defect, a false success, is fixed here: Promote and Attach are disabled and the screen says why. An admin can hide the item today through Admin > Navigation. For that reason the plan's email-triage.hidden.test.tsx is not in this PR. screens/email.test.tsx covers the disabled actions instead.

Deferred items:

  • Before logistics_monitor_enabled or inventory_planning_enabled is switched on: the C-8 lane-readiness card should check the alert email caps and the backlog size (review 1). The caps here bound the volume. They do not decide whether a backlog of wrong criticals should exist.
  • Invoice send (src/api/invoices/send.js) still leaves a not-sent invoice email queued when no provider is configured. If the operator retries after a provider is set up, the reaper could send it twice. The fix is one failUnsentRow call, but the change is outside the C-5 list.
  • WhatsApp and Slack rows with no chat config still fall through to the email providers in the send core, with a phone number or channel id as the address. Only Teams is closed off here. The WhatsApp endpoint itself now stores its rows as failed.
  • Who may mark a notification read. Operators see the bell, but the server allows admins only. The refusal is now shown. Changing the rule is a later PR.
  • Real Email Triage Promote and Attach need a new server path for one chosen email (plan C-32, decision C-D28).
  • Stale tick. The bell fires from the daily sweep only, at most once a day, because cron/daily is the only path that Vercel guarantees. As instructed, no email is sent for it. The plan's "emails admins" part is not built.
  • QUOTE_ALREADY_EMAILED does not heal the status. After a double SENT failure, the operator marks the quote SENT through the status transition. That path does not save payload_hash. portal/accept_quote needs only SENT and records a null hash.
  • Day boundary. The per-day alert cap counts from 00:00 UTC, not tenant local time.
  • R-U6 CI gates. The plan's per-SO-transition contract tests and the route-deadlinks and column-drift CI gates are not in this PR.

🤖 Generated with Claude Code

privatejoel and others added 2 commits October 17, 2026 08:44
… is down

Plan C-5 (R-U6). Each step below reported a success that did not happen.

Teams outbound: the shared send core answered a Teams row with ok: true and
status 202, so the row was marked sent and nothing left. A Teams row is now
failed with a not_configured reason, before the config lookup, so it never
falls through to the email providers with a Teams address. The copilot
confirm result carries sent and the reason, and Ask Anvil says "not sent"
instead of "Done".

WhatsApp: with no Twilio or Meta provider the endpoint answered ok: true. It
now answers sent: false, code not_configured, with the reason. The row is
stored on the whatsapp channel (it defaulted to email with a phone number as
the address) and as failed, so the queued-comms reaper never picks it up.

Quote email: quotes/send queued the email and the drawer said "sent". It now
sends in the same request through the shared send core, as invoices/send
does. The quote moves to SENT, learns customer part numbers and arms its
goals only when a provider accepted the email. Otherwise the quote keeps its
status (Send stays on offer), the drawer shows the reason, and the email row
is marked failed, not left queued, so the reaper cannot send a stale quote
later or send it twice after a retry.

Alert emails: logistics and inventory alerts were queued with no recipient,
so the reaper could only fail them. Recipients now come from the bell's roles
through resolveInternalRecipients. When nobody resolves, no row is written
and the exception records email_skipped: no_recipients.

Bell: an operator's mark-read (and mark-all-read) is refused by the server's
admin gate, and the error was swallowed. The refusal is now shown. Who may
mark read is unchanged.

Communications list: it read the audit log with a bare fetch (no session)
and action names nothing writes, so it was always empty. It now reads GET
/api/communications through the API client and shows each row's stored
status. Sent shows only sent and replied rows.

Email Triage: Promote made a blank DRAFT and Attach sent a field the orders
endpoint drops, both with a success toast. No server path takes one chosen
email to an order, so both are disabled with the reason on screen. Nav
visibility is unchanged.

Extraction watch: it now says when the background worker last ran, from
cron_health through the existing public /api/health probe, and says when that
run is late. The daily sweep now writes one admin bell row per tenant through
notifyAdmins when cron/tick is stale. No email is sent for it.

Tests (behavioural, through the handlers and screens): api-comms-send-teams,
api-whatsapp-send-no-provider, api-quotes-send-status,
api-alert-email-recipients, api-heartbeat-stale-bell, plus new cases in the
Ask Anvil, QuoteDetailDrawer, Shell, comms, email and BackgroundExtractionWatch
tests. 25 mutations, one per fix, each make a new test fail.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Alert email volume. Addressed alert emails now have a cap: a logistics
digest stamps each exception email_skipped "digested", and a new
alert-email-budget caps logistics_alert and inventory_alert emails per run
(default 10, deferred to the next run) and per tenant per day (default 30,
stamped email_skipped "daily_cap"; the bell still rings).

Quote send. The SENT update after a real send is retried once; if it
still fails the response is sent: true with a warning and a
quote_send_status_update_failed audit row, goals are not armed, and a
later send of the same quote is refused with 409 QUOTE_ALREADY_EMAILED.
The accept token of an attempt that was not sent is revoked. The drawer
shows the warning. History names quote_send_failed and the new action.

Copilot confirm marks a not-sent queued row failed (shared failUnsentRow,
also used by quote send), so the reaper cannot deliver a message the
user was told did not go.

notSentReason reports a provider that threw (sendCommunication now
returns lastError) instead of "no provider is configured".

comms_sent and the other issued counts exclude failed attempts.

bellStaleTick also rings when cron/tick has no row while cron/daily has
one ("no record of the 5-minute worker running").

Tests cover each fix, plus the three mutation gaps the review found
(margin override only after a real send, inventory To plus CC, the
stale-bell key carries the day). 21 mutations, all killed.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@vercel

vercel Bot commented Oct 19, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
anvil Ready Ready Preview Oct 19, 2026 3:10am UTC

This branch was successfully deployed

1 active deployment
Preview — 80f9435e Deployed Oct 19, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant