[antiguru]

4 of 14 in the interactive-runtime stack. Stacks on #38386, followed by #38388. Tracked by CPU-215. The design document is doc/developer/design/20261107_interactive_runtime.md in #40559.

A reader on another thread needs a way to find a published arrangement. This adds ArrangementSharingRegistry, one per local worker ordinal, shared by worker i of one runtime and worker i of the other. Pairing by local ordinal is sound only because both runtimes run the same workers per process at the same process ordinal, so both shard keys by key.hashed() % peers.

• The registry holds one slot per GlobalId, and the slot records the as_of of the dataflow that exports the index. get_or_create returns the slot when its as_of matches and it is not frozen, and otherwise creates a fresh slot and replaces the map entry. So an index recreated below the old incarnation's since never reads the old, compacted chain. Two incarnations of one id wanted at once evict each other's entry, which is safe only under the reconcile barrier compute: a reconcile barrier between the two compute runtimes #40558 adds.
• Whichever side touches an incarnation first creates the slot, backed by unbacked publication points, and the other adopts it in place.
• A slot whose traces closed or dropped is frozen and replaced by a fresh one, because its points accept no second trace. Readers of the frozen slot keep its final view.
• The registry holds slots weakly: a slot lives as long as its publisher's PublishToken or a reader holds it, so dropping a publication needs no removal call.
• publish is the one entry point for a publisher. It panics if the slot it adopts is live, not frozen, and already backed, and soft-panics if the same trace backs it. A NOTE records why a re-export's slot, which does not freeze when the re-exporting index drops, is gone by the time reconciliation recreates it.
• Every publication, seal, and unpublication unparks the reading worker.
• attach_publisher and attach_reader panic on a second thread of either kind. publish and notify do not check their caller.

Inert: nothing constructs a registry outside tests.

Tests in sharing/tests.rs cover slot convergence and lifetime, frozen-slot replacement, the thread pairing, cross-thread reads and joins, and re-exports: a_recreated_reexport_publishes_into_a_fresh_slot_once_its_holders_dropped, and a_held_reexport_slot_is_not_replaced, which pins the limit.

A reviewer should check the key choice against the design document's argument, and the re-export NOTE on publish.

Posted by Claude Code.

🤖 Generated with Claude Code

[def-]

QA LLM Review

1. HIGH -- reexport overwrites a reader's placeholder slot, stranding its import forever

src/compute/src/sharing.rs:161

reexport writes slots[worker_index] = Some(arr) unconditionally, so when a reader has already created a placeholder for to and built an import over it, that point is dropped from the registry and never adopted. The import stays pinned at the minimum frontier for the life of the dataflow, which is a silent permanent stall, and it breaks exactly the convergence contract get_or_create documents two functions above.

Details

The placeholder path exists because the reader may touch an id before any publisher does. That applies to a re-exported id just as much as to a normally published one: from the interactive runtime's side, to is an ordinary index id, and it has no way to know the maintenance runtime will satisfy it by aliasing from. So the sequence is reachable whenever the interactive runtime processes a CreateDataflow importing to ahead of the maintenance runtime rendering the dataflow that exports it — the same command-arrival-order hazard join_over_point_adopted_late_matches_direct covers for the normal path.

The re-export producer is Context::export_index's ArrangementFlavor::Trace(gid, ..) arm (src/compute/src/render.rs:751), which today just clones the TraceBundle under the new id. Note that arm cannot be fixed by calling adopt instead: the flavor carries Arranged<RowRowEnter<..>>, not Arranged<TraceAgent<Tr>>, so registry-level aliasing really is the only mechanism available and the clobber has to be handled here.

Fix: reexport must not drop an occupied slot on the floor. Either back the reader's existing point in place — a Published-to-Published binding, the analogue of adopt, so handles already captured by value follow the alias — or, if that is too invasive for now, have reexport detect the occupied slot and surface it (return it to the caller, or assert) rather than swap the Arc. Worth a regression test mirroring get_or_create_converges_on_one_slot: mint a handle off get_or_create(to, ..), then reexport(&from, to, ..), and assert the handle observes from's rows.

2. MEDIUM -- Two aliased ids share one publication point's per-id compaction state

src/compute/src/sharing.rs:281

After reexport, from and to name the same SharedIndexArrangement, but each id has its own TraceAgent in the TraceManager and its own controller compaction frontier. note_allow_compaction assigns into the single writer_logical (last writer wins) and note_standing_hold joins into the single standing_hold, so the two ids' independent frontiers collide, and reads at an as_of the trace can still serve get refused.

Details

TraceBundle::clone at src/compute/src/render.rs:753 gives to an independent agent, so handle_allow_compaction drives allow_compaction(from, f1) and allow_compaction(to, f2) separately and the trace's real since is the meet of both. The published since is meet(publisher_after, writer_logical) with writer_logical holding only whichever of f1/f2 landed last, so it can sit strictly above the real since, and Published::handle_at then returns Err for an as_of the arrangement still supports. That Err is documented as a protocol-ordering failure callers report loudly, so the visible effect is a hard failure on a legitimate read.

The standing_hold join is the sharper half. Its whole purpose is to keep the publisher's agent at the frontier the importing runtime has applied for this id. Joining the two ids' applied frontiers lifts it to the higher one, so if the maintenance runtime has already advanced from's own agent, nothing holds the trace at the as_of the interactive runtime may still present for from, and it compacts past it. Both need per-id state: either keep writer_logical/standing_hold keyed by id inside the point, or give the alias its own point rather than sharing one.