Give tool catalogs a real sync lifecycle - #1612
Draft
RhysSullivan wants to merge 8 commits into
Draft
Conversation
Replace connection.tools_stale_at (bigint ms) with tools_stale_token (nullable text). The column answers "was this catalog invalidated since the listing now finishing began" — a version question, which a wall clock answers only as precisely as it ticks. markToolsStale writes a fresh token; an authoritative listing captures the token it observed at its start and clears it by compare-and-set inside replaceCatalog's existing transaction, so a mark that landed mid-listing carries a different token and survives. A missed clear fails closed and converges on the next re-list. This restores the read scan's zero-rows steady state: nothing ever cleared the timestamp column, so the prefilter's drift arm re-selected every once-drifted connection on every read forever. Contract change from a1fdc72: the same-millisecond stale/synced tie is gone rather than resolved either way, so the schedule tests that pinned inclusive comparison now pin token presence instead. Deliberate. Also: - produceConnectionTools returns outcome synced/incomplete/lost_claim instead of a boolean, with its own counter and span attribute. A plugin reporting bad news correctly is no longer counted as a sync, nor as a refresh that crashed. writeSyncOutcome reports whether it wrote, so an incomplete listing that had also lost its lease reports lost_claim. - The auth park gates expired only. A never-synced parked connection has no catalog to serve, so it walks the retry ladder instead, which caps the waste and restores recovery for credentials repaired upstream. - markToolsStale runs under the catalog persist lock; it was the one lifecycle write left exposed to enrollment in another fiber's open transaction. Migration 0016 and its snapshot are edited in place: unpushed, never run.
…k and atomicity notes
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
executor-cloud | 2fb74bd | Aug 16 2026, 04:08 PM |
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
executor-marketing | 2fb74bd | Commit Preview URL Branch Preview URL |
Aug 16 2026, 04:07 PM |
RhysSullivan
changed the title
Contributor
Cloudflare preview
Sign-in is Cloudflare Access (one-time PIN to an allowed email). The preview has its own database and encryption key; it is destroyed when this PR closes. |
@executor-js/cli
@executor-js/config
@executor-js/execution
@executor-js/sdk
@executor-js/codemode-core
@executor-js/runtime-quickjs
@executor-js/plugin-file-secrets
@executor-js/plugin-graphql
@executor-js/plugin-keychain
@executor-js/plugin-mcp
@executor-js/plugin-onepassword
@executor-js/plugin-openapi
executor
commit: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #1605.
Tool catalogs previously had one timestamp doing every job:
tools_synced_atwas stamped even when a listing failed (a month-dead MCP server read as freshly synced and was re-dialled every 15 minutes forever — 443 handshakes/day fleet-wide fail with 401 alone),markToolsStaledestroyed the stamp to signal drift, and concurrent reads across isolates duplicated refresh work with no coordination.This gives connections an explicit lifecycle, derived from columns (no state enum):
tools_stale_token— drift is an opaque nonce, not a timestamp.markToolsStalewrites a fresh token; an authoritative listing clears it with a compare-and-set on the value it observed, inside the persist transaction. A drift signal landing mid-listing carries a different token and survives. No clock comparisons anywhere.tools_sync_claim_id/_claim_at— a leased (60s) compare-and-set claim dedupes refreshes across isolates; fumadb has no row counts, so claim = conditional update + read-back.tools_sync_failures/_retry_at— failed or incomplete listings walk a jittered exponential ladder (TTL × 2^(n−1), 6h cap) instead of retrying on every read.tools_synced_atis only ever stamped by an authoritative listing, with the listing's start instant.tools_sync_error_kind— plugins classify incomplete listings (auth/unreachable/protocol/config); auth-dead connections with a catalog to serve park (expiredonly) until any explicit signal (refresh, config revision, credential change, healthy probe) or ladder-driven retry for cold ones.synced/incomplete/lost_claim— a broken fleet no longer reports as healthy syncs.One migration (0016), additive nullable columns only. TTL-expired refresh still runs inline (filter-scoped, claimed); moving it off the read path is the next PR.