Skip to content

feat(wallet-toolbox): add prepared BEEF fast path - #506

Merged
BraydenLangley merged 19 commits into
mainfrom
codex/prepared-beef-cook
Sep 2, 2026
Merged

feat(wallet-toolbox): add prepared BEEF fast path#506
BraydenLangley merged 19 commits into
mainfrom
codex/prepared-beef-cook

Conversation

@BraydenLangley

@BraydenLangley BraydenLangley commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

Program and scope

  • Tracker or issue: operator-requested createAction latency optimization; no public issue was provided.
  • Goal: move normal managed-change calls such as GlobalKVStore pushdrop token creation toward a 50 ms service target by reusing prepared input-proof material.
  • Explicitly out of scope: enabling the feature in a deployment, changing funding selection, changing BRC-100 or Wallet Wire, publishing npm packages, or claiming this cache alone guarantees a 50 ms end-to-end SLO.
  • Exact head SHA: 0cc390359dc193b79b704ac3f2a3be2894a53d19.

Impact

  • Public package source or manifest changed
  • Infrastructure source, dependency, image, or deployment configuration changed
  • Public API, exports, types, runtime targets, or browser/mobile behavior changed
  • Security-sensitive boundary changed
  • Documentation or examples changed

Affected packages remain in the existing unpublished 2.11.0 minor release:

  • @bsv/wallet-toolbox: prepared-BEEF options, Knex persistence, coordinator, read path, migration, and security controls.
  • @bsv/wallet-toolbox-client and @bsv/wallet-toolbox-mobile: shared sync hardening and platform-budget accounting; IndexedDB/mobile storage remains canonical-only.

What changed

  • Adds COOK (Create Once, Output Kept) as the project name for a Knex-only prepared-BEEF lifecycle. Production identifiers use preparedBeef; a prepared BEEF is already reduced, verified, serialized, checksummed, stored, and ready to merge.
  • Adds separate default-off read, write, and backfill controls. createAction and processAction finish foreground result assembly before queuing identifiers and never await proof reload, graph traversal, verification, serialization, or persistence.
  • Preserves a one-query prepared hit for the normal one-root funding case. Fragmented calls first run a metadata-only aggregate over actual SQL blob lengths and bypass prepared storage when matching artifacts exceed maxLookupBytes (2 MiB by default). Calls above 32 roots also bypass.
  • Enforces the per-artifact blob limit in SQL before Knex transfers a row, then retains checksum, metadata, format, root, and parse validation in process.
  • Uses canonical BEEF for every miss or invalid, stale, corrupt, oversized, unsupported, conflicting, or resource-bypassed artifact.
  • Adds additive prepared_beefs and prepared_beef_metadata tables. A transactionally advanced proof epoch fences in-flight worker writes across processes.
  • Immediately closes local prepared reads on receipt of a reorg, before the aged replacement-proof task or any network I/O. The shared epoch is then advanced and ready artifacts are marked stale atomically.
  • Keeps derived artifacts out of wallet sync and purges them when no corresponding unspent managed output remains.

Security and correctness hardening

  • Replaced interpolated raw SQL in Wallet Storage transaction slicing with Knex query-builder predicates and bound substring operands. Slice requests require a 64-hex txid and non-negative safe integer bounds with a safe sum.
  • Applied the same parameterization and bounds checks to the adjacent Chaintracks bulk-file slice path.
  • Authenticated RPC sync cannot establish global proven_txs authority without server verification. The server parses the raw transaction and Merkle path, recomputes txid/root, checks height and leaf position, and matches the active header and chain tracker.
  • Direct backup/conflict sync now canonicalizes txid, block hash, and Merkle root identifiers before lookup/persistence. A differing existing proof must pass the same active-chain validation before replacement.
  • Proof replacement authorization is rechecked inside EntityProvenTx.mergeExisting. A candidate preflighted as a new insert cannot become an unvalidated replacement if another process inserts the txid before the merge transaction.
  • A stale/orphaned backup proof therefore fails closed before opening the merge transaction and cannot overwrite a repaired active proof. Validated repairs continue to propagate to backups.
  • Every inserted or replaced proof invalidates prepared artifacts in the same storage transaction.
  • The worker source-size preflight now exactly matches canonical-load statuses, including unconfirmed; an oversized no-send inputBEEF is rejected before load or parse.
  • The worker queue retains only bounded {userId, rootTxids} identifiers, enforces global and per-user admission (32/4 by default), and rejects source graphs over byte/transaction limits before verification or serialization.

Proof epoch and read isolation

The proof epoch fences background writes, not ordinary reads. A lookup begun after invalidation commits cannot read an old ready artifact. A lookup overlapping that transaction may observe the preceding database snapshot and is ordered before invalidation, as the canonical proof builder may be. Copying the epoch into each artifact does not strengthen a same-snapshot read where both metadata and artifact belong to the previous snapshot. Reorg notification adds the separate synchronous local read gate so new requests do not wait for database invalidation to acquire locks.

Verification

Local validation across the final change series (the final helper-only refactor was rerun through build, lint, and its 36 focused tests):

  • Wallet Toolbox build and root/package lint: passed with no warnings.
  • Full Wallet Toolbox suite: 222/222 suites passed; 2,146 tests passed and one expected test skipped.
  • Focused storage/RPC/portable/IndexedDB compatibility run after the final proof architecture: 98/98 passed.
  • Browser/mobile typechecks and package documentation consistency: passed.
  • git diff --check: passed.
  • Browser platform contract: Vite 1,669,441 raw / 392,278 gzip / 306,936 brotli; esbuild 1,303,413 raw / 356,486 gzip / 286,484 brotli.
  • Mobile platform contract: Metro 1,717,313 raw / 432,501 gzip / 335,933 brotli; Hermes 3,491,425 raw / 1,393,939 gzip / 1,096,540 brotli.
  • Hosted CI run 33647784347: completed successfully for this head, including build/lint/typecheck, full affected tests and coverage, wallet browser/mobile platform contracts, exact-head Sonar zero-findings, CodeQL, dependency review, conformance, Socket, Codecov patch coverage at 90.49%, and merge-gate checks.

Coverage includes prepared hit/fallback/corruption, async foreground timing, real processAction warming, root-count and aggregate-byte lookup bypass, identifier-only queue retention, per-user fairness, unconfirmed source rejection before BEEF load, authenticated proof rejection/acceptance/replacement, stale direct-sync rejection, lowercase canonicalization, replacement authorization at the entity boundary, immediate reorg fencing/failure recovery, parameterized slice behavior, migration, and purge behavior.

  • No dependency or lockfile change
  • Self-review of correctness, security, compatibility, API, docs, and operations completed
  • Exact-head hosted checks are terminal and successful

Release and operations

All COOK flags default off. MySQL/PXC evidence remains a hard rollout gate because this host does not expose a deployment-equivalent MySQL/PXC test environment:

  1. Before release: run the additive migration against disposable MySQL 8, confirm prepared_beefs.beef is LONGBLOB, and exercise insert/upsert, lookup, aggregate byte preflight, invalidation, and rebuild.
  2. Before enabling writes on PXC: run two storage processes against a non-production schema. Pause one worker after reading the epoch, invalidate from the other, release it, and prove the stale artifact cannot commit. Exercise concurrent same-root upserts and proof-sync replacement/retry behavior.
  3. Deploy with all flags off, canary write-only, enable reads only after warming and metric review, and enable bounded backfill only when database pressure permits.

This is not a merge blocker while the feature remains default-off, but it is a documented hard gate before production activation. Rollback is to disable read/write/backfill. Derived rows may remain; before downgrading to code that cannot advance the epoch, remove the prepared-BEEF tables or data.

@BraydenLangley
BraydenLangley force-pushed the codex/prepared-beef-cook branch from d913d59 to 090c92f Compare September 1, 2026 01:30
@BraydenLangley
BraydenLangley force-pushed the codex/prepared-beef-cook branch 6 times, most recently from 68ddfca to 178727b Compare September 1, 2026 02:28
@BraydenLangley
BraydenLangley force-pushed the codex/prepared-beef-cook branch from 178727b to 64857f4 Compare September 1, 2026 03:11
@BraydenLangley
BraydenLangley marked this pull request as ready for review September 1, 2026 03:29
…o codex/prepared-beef-cook

# Conflicts:
#	docs/packages/wallet/wallet-toolbox-mobile.md
#	docs/reference/package-api-migrations.md
#	governance/package-release-notes.json
#	packages/wallet/wallet-toolbox/CHANGELOG.md
#	packages/wallet/wallet-toolbox/client/README.md
#	packages/wallet/wallet-toolbox/client/platform-budget.json
#	packages/wallet/wallet-toolbox/mobile/platform-budget.json
#	packages/wallet/wallet-toolbox/src/storage/StorageKnex.ts
#	packages/wallet/wallet-toolbox/src/storage/methods/createAction.ts
#	packages/wallet/wallet-toolbox/src/storage/schema/KnexMigrations.ts
@sonarqubecloud

sonarqubecloud Bot commented Sep 2, 2026

Copy link
Copy Markdown

@BraydenLangley
BraydenLangley merged commit 98734b0 into main Sep 2, 2026
40 checks passed
@BraydenLangley
BraydenLangley deleted the codex/prepared-beef-cook branch September 2, 2026 15:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants