Repository navigation
Rate-limit the scheduled checkpoint shadow to once per hour per project - #52
Merged
Merged
Conversation
The shadow check is O(history) and runs under the project write lock. At a cut every 3 s, 1 in 100 came due every ~5 minutes and stalled writes for up to the shadow budget. A scheduled shadow now also needs the project's last shadow attempt to be at least DefaultCheckpointShadowMinInterval (1 h) old. The first cut after a start and every cut after a mismatch are not limited. The builder knob can only tighten the interval. Docs and claims state the weaker cadence. Co-Authored-By: Claude Opus 5.5 <[email protected]>
…orced Co-Authored-By: Claude Opus 5.5 <[email protected]>
…adow Add a rotation test (two projects, 180 minutes) that requires each project to keep getting a scheduled shadow about once an hour, and two kit mutants (timer reset on a skipped cut, server-wide timer). Cover the new helpers in the claim. Remove the stray checkpoint_cut.go.orig. Co-Authored-By: Claude Opus 5.5 <[email protected]>
…exemption Review findings left after the fix round: - The claim and LIMITATIONS said an incorrect cut 'can stay signed for up to an hour of cuts'. The gap is at least an hour: until the first cut whose seq is a multiple of 100 after the hour, longer under a backoff. Wording corrected (weaker, as docs must only get); FORMAL.md re-rendered. - New test: an unverified project (first shadow could not run) retries after the 10 min backoff, not after the hour. Dropping 'verified &&' from the rate-limit condition now fails it. - createCheckpointTx comment no longer says every 100th cut is checked. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The checkpoint shadow check (full O(history) computation) runs under the project write lock, inside a 5 s budget. For a verified project it runs on every cut with seq % 100 == 0. At a cut every 3 s that is about every 5 minutes, and in the 2026-10-21 proof run every metered call slower than 300 ms began on that grid (stall 0.35 s at 1 h of history, 1.4 s at 2.9 h).
Change
DefaultCheckpointShadowMinInterval = 1hand a per-project last-attempt time (any outcome).WithCheckpointShadowMinInterval(d): d < 0 is 0 (no limit), d above the default is clamped to it, so the knob only tightens. No env var, no flag.Formal
formal/kit-claims.jsonandformal/claims.jsonweakened to the new cadence. In claimcheckpoint-cut-incremental-equals-full, relockedcheckpointBodyCheckedand newly covered and lockedshadowTooSoon,shadowAttempted,shadowMinIntervalandWithCheckpointShadowMinInterval. FORMAL.md re-rendered. Itsdoes_not_establishstates the real gap (lead follow-up e4bc93a): an incorrect cut can stay signed for AT LEAST an hour of cuts, until the first due seq after the hour, and longer under a backoff.surface-shadow-min-interval-not-enforced,surface-shadow-min-interval-timer-reset-on-skipandsurface-shadow-min-interval-server-wide-timer, all killed (check_mutants.py --only). Three existing shadow mutant patches regenerated (context shift only, same edits).Tests
New
checkpoint_cut_mininterval_test.go(limit within the hour, shadow after the interval, per-project first cut, latched, interval 0, clamp, failed attempt counts toward both backoff and interval). Existing tests that need a due shadow inside the same hour now set the interval to 0.Gates
make test-server, go vet, gofmt, both claim checks (753 checks, 0 failures; legacy 52 claims) all green. The lead added
TestCheckpointCutShadowMinIntervalUnverifiedIsExempt, which kills the 'verified && dropped' mutation. The fix round addedTestCheckpointCutShadowMinIntervalEachProjectStillShadowsHourly(rotation of two projects for 180 min, exact count 6).🤖 Generated with Claude Code