Index store: one shared store replaces the trigram, SQLite call-graph and views indexes (0.64)

.alfonso/release-notes/next/allocation-tracing.md

@@ -0,0 +1,2 @@
+- The `aft` binary now names the operation behind large heap use in its log, for diagnosing short multi-gigabyte memory bursts. Any single allocation or reallocation of 64 MiB or more writes one WARN line (`large allocation 512 MiB realloc from 256 MiB thread=aft-subc-runtime op=frame_encode root=<repo> build_id=<uuid> frames=aft+0x…`). When one operation's live heap bytes cross 512 MiB through many smaller allocations, it writes `allocation burst: op=<label> …`, and when they fall back below half of that, `allocation burst ended: op=<label> peak … over …s`. The labels are each tool call (`tool:<name>`), `index_build`, `callgraph_start`, `callgraph_refresh`, `segment_fold`, `meaning_fill`, `symbol_prewarm`, `remote_input_capture`, `frame_encode` and `census`. Call-graph resolver pool work carries the label of the operation that started it. The uncapped memory census adds `allocation_labels`: the five operations with the highest live-byte marks since the previous census. The tracer's logger writes at most 20 lines a minute. Memory that C libraries take from `malloc` directly (SQLite, tree-sitter, ONNX Runtime) is not seen.
+- Each frame is printed as an image and an offset because the shipped binary is stripped. To name the frames, take the binary's dSYM (macOS) or debug file (Linux) with the `build_id` on the line: `aft profile` keeps them under the AFT storage root's `dsym/<build_id>`. Then run `atos -o <aft.dSYM>/Contents/Resources/DWARF/aft -arch arm64 -offset 0x1a2b3c …` on macOS, or `addr2line -C -f -p -e <aft.debug> 0x1a2b3c …` on Linux, with the `aft+0x…` offsets from the line.

.alfonso/release-notes/next/allocator-walk-cost-backoff.md

@@ -0,0 +1 @@
+- The health rollup and the module loop's allocator-slack sampler now wait longer between allocator statistics walks when a walk is expensive. The wait is at least 200 times the last walk's duration, never less than before (30 s for the rollup, 60 s for the sampler) and never more than 10 minutes. On Linux, glibc's `mallinfo2` visits every free chunk while holding that arena's lock. On a fragmented heap with 5-7 GiB freed in small blocks it took 0.65-6.5 s per walk, so walking every 30 s cost up to 20% of a core and stalled allocation in other threads. On macOS a walk on a similar heap takes 16-27 ms, so the interval there stays as it was. Memory replies are unchanged: `allocator_observation_age_ms` still reports how old the reused reading is.

.alfonso/release-notes/next/bash-wait-wording.md

@@ -0,0 +1 @@
+- The bash and bash_watch descriptions now tell agents to finish any work that doesn't need a running command's result before waiting on it or ending the turn.

.alfonso/release-notes/next/callgraph-edit-memory.md

@@ -0,0 +1 @@
+- After a broad edit, AFT's index store keeps the call-graph files it re-resolved encoded until the changes are published, so a busy worktree's call graph holds about a tenth of the memory it did (on AFT's own repository, typically about 120 MB instead of 1 GB).

.alfonso/release-notes/next/callgraph-engine-survives-publish.md

@@ -0,0 +1 @@
+- A checkout's call-graph engine now carries over each publication of the checkout's changes. Before, the engine was dropped and the next edit's refresh built a new one from the publication. The engine keeps every file it has bound and is moved onto the new publication, but only when it resolved exactly what the publication holds; otherwise it starts again as before. On a replay of 430 real edits to AFT's repository, published as AFT publishes them (88 publications), refreshes started an engine once instead of 88 times. They used about a third less CPU (1,291 s against 868 s), took about the same wall time (+2%), and the process peaked at 3.8-4.2 GB instead of 5.4-5.5 GB. An engine is released after 10 minutes without an edit, when the engines of all checkouts together hold more than 1 GiB (the least recently used goes first), or when its checkout is unbound. Each release is logged with counts by reason. The first refresh after a bind, a restart or a release still takes 5-15% more CPU than before the change that cut its memory.

.alfonso/release-notes/next/callgraph-refresh-transient-reads.md

@@ -0,0 +1 @@
+- The first call-graph refresh after AFT loads or publishes a checkout's call graph no longer keeps a decoded copy of every file it only checks. That covers a new worktree's first edit, the first edit after a restart, and the first edit after each publication. That refresh used to hold every stored parse, plus the resolutions of every file that reads the edited one, until it finished. On AFT's own repository, one edit to the crate root held about 1.5 GB, and three new worktrees refreshing at once took the process to about 6 GB. The same edit now holds about 360 MB, three worktrees at once peak at about 2 GB, and a refresh takes about as long as before. It returns the same answers as before.

.alfonso/release-notes/next/compaction-folds-segments.md

@@ -0,0 +1 @@
+- A busy worktree that piles up index changes no longer rebuilds its whole index to tidy them: AFT now combines the stored pieces into one without rereading or reparsing any file. On a 7,000-file repository that takes under a second and a few tens of MB, where the rebuild took about 12 seconds, 40 seconds of CPU and over 1 GB of memory.

.alfonso/release-notes/next/configure-warnings-store-list.md

@@ -0,0 +1 @@
+- Configure's missing-tool and missing-language-server warnings no longer walk the whole project on each configure once the root's index-store checkout has published. The pass takes the source file list from the checkout, as the background TODO refresh already does, and needs only the files' paths. It still walks when the store's list can't stand in for a walk: no checkout is bound or published yet (usually on a new root's first configure), a rescan against the disk is pending, the folder is a folder of repositories or one the index store never indexes, or confinement or a Jujutsu-only repository applies different ignore rules. Each such walk logs `configure warning files walked: reason=<reason> files=<n> ms=<ms>`.

.alfonso/release-notes/next/edit-backup-flush-and-checkpoint.md

@@ -0,0 +1 @@
+- Edits, writes and patches on macOS return sooner, most of all while the disk is busy. Their undo snapshot now does one fewer device-cache flush. Its undo-history write no longer waits for an aft.db WAL checkpoint, and waits at most 50 ms for the daemon's shared database connection and other aft.db writers. Past that, the next edit of the same file catches the history up. Undo reads the snapshot files on disk, so it is unaffected.

.alfonso/release-notes/next/gh-speech-authority-retention.md

@@ -0,0 +1,3 @@
+- AFT now deletes GitHub bot-write authorities from `aft.db` once they have been revoked for 24 hours. Every shell launch in a bot-enabled session stores one row of about 84 KB in `gh_speech_authorities`, and nothing deleted revoked rows, so on a busy machine the table reached 7 GB and grew by about 600 MB a day. The once-a-minute retention worker now deletes at most 64 such rows per transaction, for at most 100 ms per minute, and logs `speech authority retention: examined=… deleted=… batches=… exhausted=… stopped=… vacuum_pages=… elapsed_us=…` (INFO when it deleted something, DEBUG otherwise). On a synthetic table of 100,000 revoked 84 KB rows it deleted about 2,300 rows a minute, 5 ms per transaction, with the data still in the operating system's file cache. Active and candidate authorities are never deleted. The revocation records stay, so a deleted authority's ticket is still refused, now as `ticket_unknown` instead of as revoked. Authorities that a write queued by an older AFT build still refers to are kept.
+- Revoking an authority now also drops its copy of the signed routing manifest, which was most of the row. A revoked row takes about 1 KB until it is deleted. The manifest itself stays in `aft.db`'s policy ledger.
+- Deleting rows leaves free pages inside `aft.db`. The file does not shrink by itself; SQLite reuses the free pages for later writes, so the file stops growing. AFT databases use SQLite's default `auto_vacuum=NONE`. A database converted to `auto_vacuum=INCREMENTAL` returns up to 4,096 free pages (16 MiB) to the file system each minute.

.alfonso/release-notes/next/health-names-failed-index-stores.md

@@ -0,0 +1 @@
+- When roots' index stores fail to build or refresh, the daemon's health report now names them instead of giving only a count. `metrics.data_store_failed` lists up to five roots in path order, each with its reason code (for example `bind_failed` or `build_failed`), and `metrics.data_store_failed_omitted` counts the rest. The `detail` sentence names the same roots, for example `1 root(s) could not build or refresh their index store: /Users/me/Documents (bind_failed); see roots[].data_store`. The names add at most 1 KiB to the detail, and the metrics stay within their 12 KiB budget.

.alfonso/release-notes/next/import-old-embeddings.md

@@ -0,0 +1 @@
+- Meaning search's first fill after upgrading reuses the vectors the previous meaning index already computed for unchanged files with the same embedding model, instead of embedding them again.

.alfonso/release-notes/next/index-store-bounded-segments.md

@@ -0,0 +1 @@
+- The index store no longer piles up files while worktrees keep editing, and AFT holds at most 1,024 index files open, so busy daemons stay well below macOS's open-file limit.

.alfonso/release-notes/next/index-store-threads-and-edits.md

@@ -0,0 +1 @@
+- AFT's index store resolves every edited worktree's call graph on one shared set of threads instead of a set per worktree (a daemon with dozens of edited worktrees ran hundreds of resolver threads), and write, edit and apply_patch no longer parse the edited file before returning: search sees the edit at once, and the call graph parses it when it next refreshes.

.alfonso/release-notes/next/index-store-worktree-start.md

@@ -0,0 +1 @@
+- A new worktree of a repository AFT already indexes is ready in seconds instead of re-reading and re-indexing every file, which removes the CPU spikes when agents open worktrees.

.alfonso/release-notes/next/inspect-on-demand.md

@@ -0,0 +1 @@
+- Dead code, unused exports, duplicates, cycles and complexity now run only when `aft_inspect` is called, never in the background; the status bar shows errors, warnings and TODOs (`[AFT E1 W3 | T5]`), the dead-code graph is released after 10 idle minutes, and `inspect.tier2_idle_minutes` is retired.

.alfonso/release-notes/next/opencode2-shell-refusal-permission-rejected.md

@@ -0,0 +1 @@
+- On OpenCode 2, a shell command refused by a permission rule or by a prompt rejected with feedback is reported as a permission rejection, not a tool failure (broken since 0.63.1).

.alfonso/release-notes/next/outline-read-search-cpu.md

@@ -0,0 +1,3 @@
+- `aft_outline` builds the method previews for Rust traits with a single pass over each file's syntax tree, and skips files that never mention `trait`. The first outline of a large Rust file uses less CPU, and the outline is unchanged.
+- `aft_read` with line tags, and edits that refer to those tags, no longer hash a file's whole content each time they record a tagged version of the file. The hash is computed only in the rare case where an earlier version of the same file with the same tag has been dropped. Results are unchanged.
+- When nothing new has been published, a meaning-search refresh now lists the store's folder and stops there. It no longer opens and checks the newest manifest.

.alfonso/release-notes/next/outline-url-fetch-times.md

@@ -0,0 +1 @@
+- A slow `aft_outline` or `aft_zoom` of a web page now says in the daemon log where the time went: the slow-call line carries a `url_fetch=` field with the cache hit count and the milliseconds spent on the address check, connecting, waiting for the server's first byte, reading the body and converting HTML, plus body bytes, redirects, retries and any timeout phase. The URL itself is never logged.

.alfonso/release-notes/next/permissions-saved-folder-access.md

@@ -0,0 +1,13 @@
+## Saved folder access on the permissions pages
+
+The desktop and phone permissions pages now list the folders you let an agent
+or its worker use from AFT's folder-access cards, and you can revoke each one
+there. AFT serves the fleet's `permissions/v1` capability through Cingulate.
+After a revoke, the agent's next call into that folder asks again. A shell
+command that is still running with the folder in its sandbox stops, and the
+agent sees it as "killed (folder access revoked)". Commands whose sandbox
+never included the folder keep running.
+
+Each saved grant now records when it was granted and which card granted it
+(aft.db schema 27). Grants saved before this version show the upgrade time
+as their grant time and name no card.

.alfonso/release-notes/next/prune-legacy.md

@@ -0,0 +1 @@
+- `npx @cortexkit/aft cache prune-legacy` lists the index folders AFT used before 0.64 and their sizes; add `--yes` to delete them and free the disk space. AFT does not delete them on its own in this release.

.alfonso/release-notes/next/refuse-old-config-names.md

@@ -0,0 +1 @@
+- AFT now refuses configuration names retired in 0.62 or earlier, with `removed_config_key`: `tool_surface`, `hoist_builtin_tools`, top-level `enabled`, `search_index`, `semantic_search`, `callgraph_store`, `github.enabled`, `gh_read`, `gh_shim.enabled`, `sandbox.enabled`, `idle.lsp_ttl_minutes`, `inspect.tier2_soft_deadline_ms`, `inspect.max_drill_down_items`, the flat `experimental_*` keys, the `experimental.bash` block, and `aft_`-prefixed names in `disabled_tools`. Run `npx @cortexkit/aft doctor --fix` to rewrite them; AFT still rewrites your user config automatically when it can.

.alfonso/release-notes/next/runon-fewer-record-writes.md

@@ -0,0 +1 @@
+`runon` jobs write to disk less often while waiting on a draining build server, and look up the build server's capabilities once per job instead of once per check.

.alfonso/release-notes/next/search-reads-each-file-once.md

@@ -0,0 +1 @@
+- `aft_search` reads each candidate file from disk once per search instead of once per step, and works out the line that best matches the query only once per file, so searches over many matching files take less time and use less CPU. Results are unchanged.

.alfonso/release-notes/next/search-wait-fixes.md

@@ -0,0 +1 @@
+- `aft_search` no longer makes concurrent searches of one project embed their queries one at a time: a query whose embedding is already cached answers at once instead of waiting behind another search's embedding call, and two searches with the same query text share one embedding call. Searches for a file name such as `subc_format.rs` read the index's path list instead of walking the project, while AFT's file watcher keeps that list current. The daemon's slow-call log line for `aft_search` now names what the search waited on (`waits=` and `search_cpu=`).

.alfonso/release-notes/next/store-growth-folds-and-vector-merges.md

@@ -0,0 +1 @@
+- AFT's index store stays small in long use. A worktree that stopped publishing its changes used to keep every segment it read, so idle worktrees could pin gigabytes; now, when another checkout of the repository publishes, it copies what the idle one still reads into one new segment and drops the rest, with no rebuild. Meaning-search vectors, which each batch of embeddings stored in a segment of its own, are now combined once a repository has more than eight such segments, and searches answer exactly as before. On copies of two real stores, segments went from 213 to 30 (5.2 GB to 1.9 GB) and from 335 to 20 (1.6 GB to 0.6 GB).

.alfonso/release-notes/next/symbol-prewarm-shared-tables.md

@@ -0,0 +1 @@
+- A new worktree whose files match another checkout of the repository now fills its symbol cache in about a fifth of the time (AFT's repository: 988 ms to 175 ms per bind on Linux), allocating about 16 MiB instead of about 1 GiB and holding about 1.4 MiB instead of 20 MiB. Checkouts share the symbol tables decoded from content records instead of each decoding its own copy, and a file whose recorded stat evidence still proves its content is not read again. The memory census counts the shared tables once, with the data layer's shared caches.

.alfonso/release-notes/next/todo-refresh-store-list.md

@@ -0,0 +1 @@
+- The status bar's background TODO refresh no longer walks the whole project to list its files. It takes the list from the root's index-store checkout, the same list `aft_inspect` already used, and the disk is no longer stat'ed once per file. It still walks the folder in four cases: no checkout is bound or published yet, a rescan of the checkout against the disk is pending, the folder is a folder of repositories or one the index store never indexes, or a request fenced to part of the root (confinement) or a Jujutsu-only repository applies different ignore rules. Each such walk logs `inspect scope files walked: reason=<reason> files=<n> ms=<ms>`. In a replay of 430 edits on a copy of AFT's repository (6,003 source files, debug build), the 43 listings walked 0 times instead of 43. They used 45 ms of CPU each instead of 130 ms, and every list matched the walk's.

.alfonso/release-notes/next/todos-from-index-store.md

@@ -0,0 +1 @@
+- TODO counts (the status bar's `T` and `aft_inspect`'s TODO section) now come from the index store, which records each file's TODO markers when it indexes the file, so a TODO refresh no longer re-parses every file; with the call-graph index switched off, or for a folder that is not indexed, the TODO count reports itself unavailable instead.

.github/workflows/beta-pin-gate.yml

@@ -1,12 +1,17 @@
 name: Beta-pin gate
 
 # Path-filtered gate: runs the full OpenCode V2 load and acceptance matrix
-# when matrix paths or version pins change.
+# when matrix paths, version pins, or the plugin and bridge code the real host
+# exercises change. A plugin change once broke how OpenCode 2 files a refused
+# shell command and went unnoticed for two weeks because only the matrix's own
+# files triggered this gate.
 # Per branch-protection rules: paths-filtered jobs must never be a required check on main.
 
 on:
   pull_request:
     paths:
+      - "packages/opencode-plugin/src/**"
+      - "packages/aft-bridge/src/**"
       - "packages/opencode-plugin/test/matrix/**"
       - "packages/opencode-plugin/test/load-matrix/**"
       - ".github/opencode-version.txt"

.github/workflows/release.yml

@@ -177,10 +177,10 @@ jobs:
         # fallback silently masks the version-mismatch bug (we try to publish
         # vN, it already exists, we say success even though we wanted vN+1).
         #
-        # `agent-file-tools` depends on `aft-tokenizer` and `aft-secret-rules`
-        # by path; cargo refuses to publish a crate whose path-only dep has no
+        # `agent-file-tools` depends on `aft-tokenizer`, `aft-secret-rules` and
+        # `aft-store` by path; cargo refuses to publish a crate whose path-only dep has no
         # `version =` or isn't on crates.io. All three crates therefore declare
-        # versions (synced by version-sync.mjs) and we publish the two leaf
+        # versions (synced by version-sync.mjs) and we publish the leaf
         # crates first so `agent-file-tools` can resolve them on crates.io.
         # scripts/lib/test_native_release_workflow.py fails when a new path
         # dependency is added without a publish line here. The `--no-verify` flag on the
@@ -204,6 +204,8 @@ jobs:
             || { ec=$?; cargo publish --package aft-tokenizer --dry-run 2>&1 | grep -q "already exists" && true || exit $ec; }
           cargo publish --package aft-secret-rules \
             || { ec=$?; cargo publish --package aft-secret-rules --dry-run 2>&1 | grep -q "already exists" && true || exit $ec; }
+          cargo publish --package aft-store \
+            || { ec=$?; cargo publish --package aft-store --dry-run 2>&1 | grep -q "already exists" && true || exit $ec; }
           # Wait briefly for the new leaf crate versions to become visible on
           # crates.io's sparse index before publishing `agent-file-tools` —
           # otherwise the verify step in the second publish can race against
@@ -821,7 +823,7 @@ jobs:
       # Cargo's local registry overlay, without needing unpublished siblings on
       # crates.io. Stable Cargo >=1.90 supports this selection mode.
       - name: Rehearse crates.io publishing
-        run: cargo publish --dry-run --locked --allow-dirty -p aft-tokenizer -p aft-secret-rules -p agent-file-tools
+        run: cargo publish --dry-run --locked --allow-dirty -p aft-tokenizer -p aft-secret-rules -p aft-store -p agent-file-tools
 
       # Mirrors the inline 10 MiB check in scripts/release.sh (crates.io package
       # sizes). A publish dry run does not enforce the registry's upload limit.
@@ -839,7 +841,7 @@ jobs:
           # Multi-package publish stages archives in tmp-crate and tmp-registry;
           # single-package packaging leaves them directly in the package folder.
           crates = sorted(crate_dir.rglob("*.crate"))
-          expected = {"aft-tokenizer", "aft-secret-rules", "agent-file-tools"}
+          expected = {"aft-tokenizer", "aft-secret-rules", "aft-store", "agent-file-tools"}
           filenames = {crate.name for crate in crates}
           found = {
               package["name"] for package in metadata["packages"]