[antiguru]

10 of 14 in the interactive-runtime stack. Stacks on #40558, followed by #38676. Tracked by CPU-216. The design document is doc/developer/design/20261107_interactive_runtime.md in #40559.

enable_compute_interactive_runtime moves out of UNINTERESTING_SYSTEM_PARAMETERS, where #38391 parked it, and becomes a variable system parameter defaulting to on. That is what makes every preceding piece reachable: sqllogictest, testdrive, and the mzcompose suites provision replicas with two runtimes. Parallel-workload's FlipFlagsAction is unpinned to both values. Production keeps the dyncfg's own default, off.

Four clusterd-test-driver specs join the two-runtime workflow #40558 added, which runs every spec with one and two workers, so the registry's per-worker pairing is exercised rather than assumed: an index read, a query dataflow scheduled before the index it imports has produced anything, a read through an index that re-exports another's arrangement under a publication point of its own, and a query dataflow over such a re-export. create-dataflow takes a single-read flag, which sets until one past the as_of and the class to OneShotRead, as the adapter does.

divergent-dataflow-cancellation.td keeps the flag off, because its INSERT's read would render on the interactive runtime, whose work is not yet visible in introspection (CPU-222).

read_at_max_timestamp.slt covers reads at Timestamp::MAX, the one read whose until is empty, so the adapter leaves its dataflow a maintained one, which the maintenance runtime renders. Such a read is answerable only against a collection that has sealed, since no finite upper passes MAX, so the file reads off constant-backed indexes, whose dataflows have no inputs and drop their capability after emitting once. Both shapes are covered: a fast-path index peek, served by the interactive runtime over shared indexes, and a join that cannot take the fast path and so builds a dataflow. The reads double as the assertion that those indexes sealed, since a peek finding a live frontier would never return.

With this PR, ReadIsolationUnderHydration (#38542) measures peek latency on a two-runtime replica while hydration saturates the maintenance workers.

A reviewer should check that the test suites need no expectation changes beyond the two files above.

Posted by Claude Code.

🤖 Generated with Claude Code

[linear-code]

CPU-215

[def-]

QA LLM Review

1. MEDIUM -- divergent-dataflow-cancellation.td can no longer see the peek dataflow it cancels

test/testdrive/divergent-dataflow-cancellation.td:107

With the flag now on in testdrive, this test's INSERT ... SELECT over a divergent WMR renders its read as a peek dataflow on the interactive runtime. That runtime installs no logging, so the expect-empty introspection queries at the end of the file cannot observe it. If a cancelled peek dataflow were left running on the interactive runtime, its WMR loop would spin forever on the runtime that now serves every peek of the replica, and this test would still pass.

Details

A non-constant INSERT becomes a read-then-write (src/adapter/src/coord/sequencer/inner.rs:2743), whose read goes through sequence_peek (inner.rs:2908). optimize::peek gives that read until = as_of + 1 (src/adapter/src/optimize/peek.rs:318), so the dataflow is transient and single-time. is_peek_dataflow therefore holds, and Multiplexer::send routes the dataflow, its CancelPeek and its drop to interactive (src/compute-client/src/multiplex.rs:170).

initialize_logging returns early on the interactive role (src/compute/src/compute_state.rs:1204), so none of that dataflow's operators, exports, frontiers or arrangements reach mz_introspection. Every query from line 124 onward reads the maintenance runtime's logs, which never contained it. The index, materialized view and SUBSCRIBE cases still render on maintenance and are still covered. Before this PR the flag sat in UNINTERESTING_SYSTEM_PARAMETERS, testdrive ran a single runtime, and the INSERT case was visible.

The interactive runtime also has a drop path of its own that this check would otherwise guard. handle_allow_compaction treats any id missing from collections as a peer's publication and only records a standing hold, never dropping anything (compute_state.rs:1009). Its soft assert passes precisely on the interactive role. So if a peek dataflow's collection ever went missing there, its empty-frontier AllowCompaction would be silently discarded and the dataflow would keep running.

The PR discussion already attributes the missing interactive introspection to CPU-222. Until that lands, two options keep the test honest. One is to pin enable_compute_interactive_runtime = false beside the existing ALTER SYSTEM at the top of the file, before CREATE CLUSTER test, which works because the flag is read at replica provisioning. The other is to assert the INSERT case's teardown through something the interactive runtime does report.

[antiguru]

Re the QA review on divergent-dataflow-cancellation.td: the file now sets enable_compute_interactive_runtime = false before CREATE CLUSTER test, with TODO(CPU-222), so its introspection checks see the INSERT case again. The silent-discard path is gone too. The interactive runtime now knows its peer collections explicitly, and an AllowCompaction for an id that is neither a collection nor a peer soft-asserts on every runtime. Interactive introspection is planned: interactive workers forward their log events to the paired maintenance worker, with interactive operator and dataflow ids offset by 2^63.

Posted by Claude Code

[bosconi]

The flag flip itself is scoped correctly; two coverage gaps and a CI timeout question inline.

[bosconi]

Like enable_upsert_chunked_stash above, say why this is on in CI and off in production, and that it only affects newly provisioned replicas.

Suggested change

	VariableSystemParameter(
	"enable_compute_interactive_runtime",
	"true",
	["true", "false"],
	),
	# On by default so CI exercises the interactive runtime, which is off in
	# production while it earns trust. Read when a replica is provisioned, so
	# only replicas created after the setting lands are affected.
	VariableSystemParameter(
	"enable_compute_interactive_runtime",
	"true",
	["true", "false"],
	),

[bosconi]

Unmanaged replicas ignore the flag (clusters.rs#L467-L479), and only test/clusterd-test-driver passes interactive_compute=True, so platform-checks (UseClusterdCompute), test/cluster, test/cluster-isolation, test/replica-isolation and test/bounded-memory stay single-runtime. Please state that scope in the description, and consider opting at least one compute-focused composition in.

[bosconi]

The default workflow now adds ten clusterd runs, but the nightly step still has timeout_in_minutes: 20. Has a nightly run on this head confirmed headroom, or should the timeout go up here?

[bosconi]

Nothing asserts single-read parses to true, and the specs would still pass if it were dropped, because the maintenance runtime would serve the same rows. Add a parse check like the optimize one in parses_create_dataflow_with_mir:

        // The `single-read` flag on the directive line is picked up.
        let single_read = parse_command(
            "create-dataflow name=r as-of=0 single-read\n  import index=1001\n  build id=2000\n    Get u1000\n  export index=2001 on=2000 key=[0]",
        )
        .unwrap();
        assert!(matches!(
            single_read,
            Command::CreateDataflow { single_read: true, .. }
        ));