Skip to content

fix(init): recover a lockless index root instead of dead-ending - #204

Merged
sunerpy merged 1 commit into
mainfrom
fix/lockless-root-recovery
Aug 16, 2026
Merged

sunerpy merged 1 commit into
mainfrom
fix/lockless-root-recovery

Conversation

@sunerpy

@sunerpy sunerpy commented Aug 16, 2026

Copy link
Copy Markdown
Owner

The dead end

A user's codegraph init failed permanently on a project that had never been indexed:

Error: permanent index lock does not exist: .../AgentLens/.codegraph/index.lock

.codegraph/ held exactly one file — daemon.log — with four identical entries over two days saying the daemon refused to start because index state was missing, and advising codegraph init. The daemon had already made that impossible.

Measured on the reproduction: init, index --force, sync, and uninit --force all exit 1. Only status runs, reporting State: missing with no remedy. The single way out was rm -rf .codegraph.

The shape is trivial to create: the parent process called create_dir_all on the index root purely to place the detached child's stdio, so a failed background start left a directory containing a log and nothing else.

Fixes

A — init takes over a lockless, state-less root. The recovery predicate is now bare ExtractionStatus::Missing, which holds only when neither state slot exists: any unreadable slot classifies Corrupt earlier, so widening the predicate cannot reach authoritative data. Verified: a lockless root with a malformed index-state.0.json still exits 1, keeps the slot, and creates no lock.

Creating the lock is not possessing it. Between create_new and the first try_lock the file exists unlocked, so a competing writer can acquire it and publish state; final_path_matches corroborates inode identity, not namespace state. So the recovery arm re-asserts Missing while holding the lease and refuses with LocklessTakeoverStateChanged otherwise, dropping the lease before any authorization, phase=building publication, or database mutation.

B — the spawn no longer creates an index root to hold diagnostics. The log redirect falls back to the null sink when the root is absent, and the daemon refusal stops claiming no log was published when the parent had written one.

C — diagnostics name a remedy that works. LockNotFound and lockless Missing now point at codegraph status and say plainly that only State: missing is safe to recover with codegraph init, instead of naming an internal artifact.

Why the race test is trustworthy

A test-hooks-only checkpoint parks a real init child in the pre-lock window. The test then acquires the still-unlocked permanent lock itself, publishes Building, and releases the child — proving the window exists rather than asserting it. No sleeps; the rendezvous is a TCP handshake and the timeouts are hang-guards only.

Reverting just the re-assertion makes that test fail because the takeover succeeds — Indexed 3 files — while the competitor's Building state is live. That is the hazard: destroying another process's in-progress work. The test was also run 20× consecutively without a flake.

Verification

  • make ci green: 127 suites / 3156 passed / 0 failed (baseline 3146).
  • Hands-on reproduction of the user's dead end: init now succeeds, the sentinel daemon.log is byte-identical afterwards, and index.lock + codegraph.db appear.
  • Safety boundary held: no --force state override, no widened lock creation for Building/Corrupt/Future, no takeover of a namespace with a valid slot, and the global FullRebuild gate is unchanged.
  • reference/ untouched, CURRENT_EXTRACTION_VERSION still 5, guardrail passes.

Third instance of one family: v0.42.5 taught init to take over a legacy .codegraph classifying Missing with a database present, v0.42.9 when every slot was OwnerMismatch after a project move. Each time a state init could not recover from, plus an error naming an internal artifact instead of a remedy.

A failed background daemon start could leave `.codegraph/` holding nothing
but `daemon.log`, because the parent created the index root solely to place
the child's stdio. That shape is unrecoverable: `init`, `index --force`,
`sync` and `uninit --force` all refuse it, and the log itself keeps advising
`codegraph init` — the very command the daemon had just made impossible. Only
`rm -rf .codegraph` got out.

Teach explicit init to take over such a root. The recovery predicate is now
bare `ExtractionStatus::Missing`, which holds only when neither state slot
exists: any unreadable slot classifies `Corrupt` earlier, so widening the
predicate cannot reach authoritative data.

Creating the permanent lock is not possessing it. Between `create_new` and
the first `try_lock` the file exists unlocked, so a competing writer can
acquire it and publish state; `final_path_matches` corroborates inode
identity, not namespace state. Re-assert `Missing` while holding the lease
and refuse with `LocklessTakeoverStateChanged` otherwise, dropping the lease
before any authorization, `phase=building` publication or database mutation.
A `test-hooks` checkpoint parks a real `init` child in that pre-lock window so
the race is covered deterministically, without sleeps.

Stop the spawn from creating an index root just to hold diagnostics: the log
redirect now falls back to the null sink when the root is absent, and the
daemon refusal no longer claims no log was published when the parent wrote one.
`LockNotFound` and lockless `Missing` now name a remedy that works instead of
an internal artifact.
@codecov

codecov Bot commented Aug 16, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.61404% with 5 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
crates/codegraph-cli/src/main.rs 88.37% 5 Missing ⚠️

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #204      +/-   ##
==========================================
+ Coverage   95.16%   95.18%   +0.02%     
==========================================
  Files         136      136              
  Lines       72320    73024     +704     
==========================================
+ Hits        68822    69510     +688     
- Misses       3498     3514      +16     
Files with missing lines Coverage Δ
crates/codegraph-daemon/src/spawn.rs 95.15% <100.00%> (+0.99%) ⬆️
crates/codegraph-store/src/index_lease.rs 88.57% <100.00%> (+0.39%) ⬆️
crates/codegraph-store/src/rebuild.rs 92.87% <100.00%> (+0.26%) ⬆️
crates/codegraph-cli/src/main.rs 90.55% <88.37%> (-0.01%) ⬇️

... and 4 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@sunerpy
sunerpy merged commit 5175f2a into main Aug 16, 2026
7 checks passed
@sunerpy
sunerpy deleted the fix/lockless-root-recovery branch August 16, 2026 11:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant