Skip to content

docs(so): audit of the SO agent infrastructure - #645

Open
privatejoel wants to merge 1 commit into
docs/so-agent-infra-audit-planfrom
docs/so-agent-infra-audit
Open

privatejoel wants to merge 1 commit into
docs/so-agent-infra-audit-planfrom
docs/so-agent-infra-audit

Conversation

@privatejoel

Copy link
Copy Markdown
Collaborator

Summary

This PR adds the audit that the plan in #643 describes. It is stacked on #643. Merge #643 first. Docs only.

  • docs/SO_AGENT_INFRA_AUDIT.md is the audit. It is read only, at 931eb143.
  • docs/SO_AGENT_INFRA_AUDIT_CARDS.md is the evidence: one Q1 to Q18 card for each mechanism, and the five layer reports.

Verdict. The owner is right. SO agent work runs on 84 mechanisms (49 from the plan, 35 found by the audit). They share almost nothing. On main there are 6 trust models, 11 queue shapes, 2 staleness maps, 8 recipient resolvers and 8 paths that create an order.

Findings that need attention now

  • Hold fix(tally): the reconciler re-queues a voucher missing in Tally #632. The reconciler reads three columns that tally_voucher_state does not have (_lib/tally-reconciler.js:325-329). It flags every exported voucher as missing. With fix(tally): the reconciler re-queues a voucher missing in Tally #632 and auto-fix on, every recent voucher is pushed to Tally again as a new voucher.
  • Tenant isolation. There are 16 cross-tenant findings. fix(drains): a manual reply or voice drain reads only the caller's tenant #644 fixes 2 of them. The worst open one: any approver of any tenant can run the LLM classifier on every tenant's inbound mail and read the results (inbound/email/parse.js:148).
  • One voucher can be posted twice by four live paths (X-12).
  • No shared order write gate. Only the PATCH applies the transition table. Sixteen other mechanisms write an order around it.
  • Nobody is told. On main, five of six SO events reach no person. When a Tally push gives up, nobody is told.
  • The clock. Nothing in the repo calls the 5-minute tick. The docs say that a cron-job.org job does. Only the owner's query can confirm it.

Outputs

  • O-1 inventory (section 2).
  • O-2 overlap and conflict matrix: 24 conflicts and an event-by-channel table (section 4).
  • O-3 target model: consolidate in place onto 9 shared primitives, each grown from a part that already works (section 5).
  • O-4 migration path: 30 small PRs. The first 10 are safety fixes that need no decision (section 6).
  • O-5: 12 owner decisions, each with a default (section 7).
  • O-6: 122 defects (24 high), each with path:line and a fix PR (section 8).

Method. Five read-only layer auditors worked in parallel. The lead re-read every high claim in the code. Then a fresh verifier checked 123 claims against the code. It found 14 that were wrong or overstated, and all 14 are corrected (section 9). #643 also gets one correction: only orders/[id].js imports the order write gate.

Deploy notes

None. This PR changes docs only. It has no migration and no env change. The migration numbers that the U-PRs name (271 to 274, plus 266 from #611) are proposals. Nothing reserves them until a PR builds them.

Tests

None needed, because no code changed. Every high defect, every conflict and every signal verdict was checked against 931eb143 twice: once by the lead and once by the verifier. Section 9 lists what changed. Both files have no em or en dashes and no customer, person or part names. One code identifier contains a customer name, so the cards file calls it "a second legacy tenant header".

Not covered

🤖 Generated with Claude Code

The audit that the plan in #643 describes. Read only, at 931eb14.
84 mechanisms in five layers, 24 conflicts, a target model of 9
shared primitives grown from parts that already work, 30 small PRs
in merge order, 12 owner decisions, and 122 defects (24 high). A
verifier re-read 123 claims against the code; its corrections are
in. The cards file holds the five layer reports as evidence.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@vercel

vercel Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
anvil Ready Ready Preview Oct 28, 2026 2:56am UTC

This branch was successfully deployed

1 active deployment
Preview — fe05fa5d Deployed Oct 28, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant