Repository navigation
averin export perf: linear-time verifier D6/M4/anchor checks and AVERIN_EXPORT_TIMEOUT 504 (feir-ops#26) - #58
Merged
Merged
Conversation
The offline verifier rebuilt every checkpoint's grant log from the whole history (committed_set walk, record_trust scan, sort, hash fold), so a history with C checkpoints and N records cost O(C x N). /v2/export runs the same verifier before its first byte, so the export inherited the superlinear time. LogCursor (core/src/verify/committed.rs) gives the same (count, gapless, root) per frontier while doing work proportional to what is new in each frontier. State from the previous frontier is reused only after the walk from the new frontier has reached every known member of the previous one (the committed set is closed under parents, so the previous closure is then contained); otherwise the cursor resets and recomputes in full. New entries are merged at their sorted (broker_seq, record position) position and the fold resumes from the stored accumulator there, never from a bundle value. Also: a reach DP for the "headless checkpoint commits grants" checks (D6 and M4), a cursor per broker for the federation heads, and one backward pass for the anchor times plus one committed_set over the union of anchored frontiers (replacing one committed set per anchored checkpoint held in memory). No check is skipped; the report is byte-identical (golden comparison on prefixes of the 8 h soak export). The old per-frontier functions stay as the fallback and as the test oracle. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Member
Author
|
Gate output (host: WSL, old glibc; cargo and go under
|
Deploying averin-verify with
|
| Latest commit: |
56dd780
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://f7191356.averin-verify.pages.dev |
| Branch Preview URL: | https://wave1-averin-export-perf.averin-verify.pages.dev |
…-path guards (feir-ops#26) Co-Authored-By: Claude Opus 5.5 <[email protected]>
…k nine verify_bundle_with claims; docs: state measured ratios The nine relocked claims (temporal-revocation-ordering-tests, pop-intent-and-disclosure-regressions, production-call-path-gate, approver-identity-in-signed-evidence, month-segment-verifier, gateway-refusal-counted-when-proven, gateway-refusal-shape-fail-closed, gateway-refusal-never-widens-surface, oldest-uncovered-received-at) still hold: verify_bundle_with changed only in how anchored_before and the closed set are computed (exactly equal, golden and mutant checked) and in passing topo_order through. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
…laim Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
…lbacks, harden DagIndex The per-broker LogIndex and reach vectors are built inside the per-broker loop and dropped at its end, and the headless M4 check finds every committing broker with one walk per distinct headless frontier, so peak extra memory is O(N) however many brokers a crafted bundle has. DagIndex refuses a topo_order that lists a child before its parent. Per-thread counters (slow_path_probe) record per-frontier fallbacks, cursor advances and anchor fallbacks so a silent revert of the speed-up fails a test. Adds the equal-boundary anchored-before test and non-canonical stamps that sort before canonical ones. Co-Authored-By: Claude Opus 5.5 <[email protected]>
…y_bundle_with and advance covers The relocked statements still hold: the edits to verify_bundle_with and advance add only slow-path counters and the lazy per-broker build; every report is byte-identical and the detectors pass. Co-Authored-By: Claude Opus 5.5 <[email protected]>
…bundle_with and build_inner in the incremental-log claim Co-Authored-By: Claude Opus 5.5 <[email protected]>
…te, worker-held slot
A whole-history export or verify that has not finished by the deadline (default 10m, range 1m to 60m, invalid value stops startup) is answered 504 with a generic body before any body byte. The export build runs in a worker; once the handler gives up the worker owns the concurrency slot until the build ends. A verify waiter gets the same 504 while the shared flight continues for others. Counter averin_bundle_deadline_exceeded_total{route} and one timing log line per whole-history export. Docs: CONFIGURATION, API, LIMITATIONS, TESTING.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The posture-report-live and durable-records-need-durable-content statements still hold: the edit only reads one more environment variable and sets the build deadline; posture and content-store wiring are untouched, and the full server test suite passes. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Post-release 1.
Problem
feir-ops#26: whole-history
GET /v2/exportandaverin-verify bundlewere superlinear (28.5 MiB 86 s, 41.7 MiB 177 s, 110.6 MiB 1,208 s), so the export reached the fixed 10 minute deadline at about 110 to 130 MiB and the connection was cut. The export runs the same verifier before its first byte.Verifier (core)
LogCursorgives the same(count, gapless, root)ascommitted_broker_log+grant_head_rootfor a sequence of frontiers. State is reused only after the walk from the new frontier has reached every known member of the previous frontier; otherwise it resets and recomputes in full. The old per-frontier functions stay as the fallback and the test oracle.DagIndexrefuses a topo order that lists a child before its parent.issuesand their order are byte-identical.Export deadline (server)
AVERIN_EXPORT_TIMEOUT: Go duration, default 10m, range 1m to 60m; unparsable or out of range stops startup. Logged at startup.504with a generic body (duration and the month hint) before any body byte; counted inaverin_bundle_deadline_exceeded_total{route}./v2/verifywaiters get the same 504; the shared flight keeps running for other waiters. The deadline context bounds only the bundle-view build (and on Postgres the 45 s read session cuts first); the wait for a bundle slot and the Rust verify are not bounded by it./v2/exporttime to first byte at 10k/40k/80k records, a same-database export bytes golden between origin/main and this build, and the anchored-history (TSA pinned) performance measurement from design section 6. The deadline and 504 are covered by unit tests with a slowed store. feir-ops#26 should stay open for those measurements, and for the follow-ups in feir-os and govder (pass the setting, map the 504).Tests and mutants
cargo test -p averin-decision-core --lib committed: seeded random differential tests against the old code, plus (new) counters proving the cursor ran for every head and no frontier took the per-frontier code on an honest history, headless checks answered through the index, 300 brokers equal to the old code, and the topo-order refusal.tests/adversarial.rs:d6_cursor_*, and a new equal-boundary test (key compromised exactly at the anchor time stays trusted, anchored_before answered by the anchor index). Non-canonical stamps that sort before canonical ones were added to the anchor-time test.export_deadline_internal_test.go(parse table, 504 before any byte, slot held after 504 and released at the end, verify 504).formal/kit-claims.json(three original, eight new: log index, cursor, reach, federation partition, anchor index each forced unused; canonical-guard removal; exclusive boundary; parent-order check removal). All killed with the claim's detector, and each patch changes a symbol the claim covers.Measurements (same host, dev profile, user CPU of
averin-verify, prefixes of the proof run 6 export)Reviewer rerun, unloaded: 4.57 s, 9.97 s, 24.23 s user at 10k, 20k, 42.7k records. t(20k)/t(10k) = 2.18 (target <= 2.6). t(42.7k)/t(10k) = 5.30 on the unloaded rerun (5.74 on the loaded run); the pre-registered limit of 5.5 was missed on the loaded run and met on the unloaded one. Full 116 MB export: report JSON sha256 identical to origin/main, user CPU 529.2 s to 24.2 s, RSS 684 MB to 698 MB.
Gates (after merging origin/main of 2026-10-28, including #57 and #59)
cargo fmt --check,cargo clippy --all-targets -- -D warnings,make test-corepass.make test-server: vet passes;go test ./...withCGO_LDFLAGS='-ldl -lpthread -lm'passes on all packages (the host's old glibc needs the flags).make test-verifier: 53 pass, 0 fail (the pin files it rewrites locally were restored, not committed).python3 scripts/formal/check_claims.py --claims formal/kit-claims.json: 789 checks, 0 failures;python3.12 scripts/check-claims.py: 53 claims;formal/production/check-production.pyOK.verify_bundle_withclaims andadvance(only counters and the lazy build were added; reports unchanged), and twomain.goclaims (one more env read; posture and content wiring untouched).Risks
verifier/averin_decision_core.wasm.sha256andverifier/index.htmlare not repinned (a local build does not reproduce the committed pin; a repin must come from the canonical build).🤖 Generated with Claude Code