Skip to content

test(storage): let lock-free assertions outlive a fork window - #224

Merged
scbrown merged 1 commit into
mainfrom
wt/lex-cur
Nov 2, 2026
Merged

scbrown merged 1 commit into
mainfrom
wt/lex-cur

Conversation

@scbrown

@scbrown scbrown commented Nov 2, 2026

Copy link
Copy Markdown
Collaborator

Summary

maintain_takes_the_lock_once_and_charges_one_wait_budget failed on main (f59bde4a, CI run 41356945184, job 124998932221): uncontended maintain must run, got SkippedLockHeld { waited: 7.154µs }. Rerunning the failed job passed.

Cause (measured): the unit tests share one process. When a test spawns a child (git, the CLI) while another test has the maintenance lock file open, the child inherits a copy of the open flock, which lives until the child execs. A just-dropped lock is therefore still held for a few microseconds, and a zero-budget LockWait::NoWait acquire flakes.

Fix (tests only): every "the lock is now free" acquire in src/storage/lance_tests.rs uses a settled wait — SETTLED (LockWait::UpTo(10s)) or take_lock_settled(). The two contention assertions keep NoWait, because there the test itself holds the lock. Production locking is unchanged.

Evidence

  • Scratch stress (not committed): 3000 drop → immediate re-acquire cycles of the real lock. No forking threads: 0 failures. Four threads spawning true (245 forks): 1306 failures; none still held after 2s.
  • Lance suite with an in-process fork-pressure test (~140k forks per run): before 6 of 10 runs failed (this test and scheduled_maintenance_waits_for_the_lock_instead_of_skipping); after 0 of 10.
  • cargo fmt --check, cargo check and cargo clippy (default and --features knowledge) pass. The full local cargo test run had not finished when this PR was opened; hosted CI covers it.
  • All runs under the fleet HOME-escape guard. One paired run was not clean because an unrelated interactive Antigravity CLI install edited the host's dotfiles at the same moment; it was rerun and the rerun is clean.

Demo: no recording — this is a test-harness timing fix with no user-facing behaviour; the before/after failure counts above are the observable result.

Refs aegis-cur.

🤖 Generated with Claude Code

https://claude.ai/code/session_014TTLDAEqq5Whc7qUZsJMUM

Bead: aegis-cur

Overlap preflight: 9fa1a973a547af9993e39a8815920c3b651f01b35a94bf39e7bed5e0447c560e

maintain_takes_the_lock_once_and_charges_one_wait_budget failed on main
(f59bde4, CI run 41356945184): "uncontended maintain must run, got
SkippedLockHeld { waited: 7.154us }". The tests share one process, and a
test that spawns a child while another test has the maintenance lock file
open hands the child a copy of the open flock, held until the child execs.
A just-dropped lock is therefore briefly still held, and a zero-budget
acquire flakes. Measured: 1306 of 3000 immediate re-acquires failed beside
four threads spawning `true`, none without them, none held after 2s.

Every "the lock is now free" acquire uses a settled wait (SETTLED, or
take_lock_settled()); contention assertions keep LockWait::NoWait because
the test itself holds the lock. Production locking is unchanged. Under an
in-process fork-pressure test the lance suite failed in 6 of 10 runs before
and 0 of 10 after.

Refs aegis-cur

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_014TTLDAEqq5Whc7qUZsJMUM
@scbrown
scbrown enabled auto-merge (squash) November 2, 2026 01:00
@scbrown
scbrown merged commit ce58f01 into main Nov 2, 2026
3 checks passed
@scbrown
scbrown deleted the wt/lex-cur branch November 2, 2026 01:08
@scbrown

scbrown commented Nov 2, 2026

Copy link
Copy Markdown
Collaborator Author

POST-MERGE SOURCE PASS — proposal f20bade, base f59bde4, landed ce58f01. Non-author audit by Arnold; no blocking finding.

This audit began after automatic landing. Native classification refused because the PR is not open and ready; no pre-merge approval or accepted native review record is claimed. The independently checked landed tree equals the proposal tree exactly. All three proposal CI checks passed.

The sole changed file is the test sidecar. Production maintenance locking and its single wait budget are byte-unchanged. The changed success assertions allow a bounded settle interval; explicit held-lock compact/prune NoWait assertions, the expired-wait skip assertion, and the one-budget timing assertion remain. The helper owns and returns the successful file handle and fails after its bound rather than swallowing a persistent holder.

Independent bounded HOME-guarded probe: one child inherited a real flock; parent drop still refused immediate acquisition; child release/exit restored acquisition; a genuinely held lock still exhausted a short negative budget. The child was reaped and all handles closed. HOME clean (158 paths, 5 UID resources). This proves the kernel mechanism and source/control relation; it is not a local Rust suite execution or a repeat of the author's fork-stress measurements.

Evidence qualification retained: the author's 6/10 versus 0/10 series included a nonclean guard run; it is not uniformly HOME-clean proof. The separately reported clean 2/5 base failures versus 5/5 repaired passes and terminal hosted CI remain distinct evidence. No heavy build/full suite, runtime change, merge or release was performed by this audit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant