[antiguru]

12 of 14 in the interactive-runtime stack. Stacks on #38676, followed by #38736. Tracked by CPU-216. The design document is doc/developer/design/20261107_interactive_runtime.md in #40559.

Two questions #38676's benchmarks do not ask.

What a saturated interactive runtime does to maintenance. Two runtimes are twice the worker threads on the same cores. MaintenanceUnderPeekSaturation loads a replica with join peeks issued open-loop at one per worker per second and two hydration churn loops, while a writer advances a small materialized view. The replica takes the largest power of two of workers up to the host's core count, capped at 32, because that is what the size map offers. mzcompose sets no CPU limit, so the host is fully oversubscribed only when its core count is a power of two of at most 32. On other hosts the second runtime gets idle cores of its own and can only report two runtimes as equal or better. The measured query reads the view strict serializable from a separate one-worker cluster, so it cannot answer before the view's write frontier passes the write and nothing else competes with it: its latency is the view's maintenance lag under the peek load. Read on the loaded replica it would queue behind the joins on the runtime that serves peeks instead, which is what the join loops' own latency reports.

What publishing costs at scale with nobody reading. ManyIndexesIdle and ManyIndexesRestart publish two hundred indexed views over one table, each a distinct arrangement. The first times a count over one index, a peek dataflow rather than a fast-path lookup, and reports, through the clusterd memory column, the resident cost of two hundred publications. The second restarts the replica and waits for a read that imports every index, so every publication is rendered and bound again on the way up. ManyReexportsIdle is the other shape: two hundred indexes on one relation and key share one arrangement, and each publishes it under a publication point of its own.

Benchmarks only, no product code changes. A reviewer should check that each scenario measures what its docstring claims, in particular the worker-count rounding.

Posted by Claude Code.

🤖 Generated with Claude Code

[def-]

QA LLM Review

1. MEDIUM -- ManyReexportsIdle samples clusterd memory without waiting for the 200 re-exports to hydrate

misc/python/materialize/feature_benchmark/scenarios/interactive_runtime.py:202

The scenario's stated purpose is that "the clusterd memory measurement is the resident cost of INDEXES publications of the same arrangement", but nothing in init() waits for those 200 index dataflows to become resident, so the reported figure can be a snapshot taken while they are still building. Its sibling ManyIndexesIdle closes exactly this gap and this one does not, which makes the number understate precisely on the builds where re-exports are expensive.

Details

CREATE INDEX returns once the catalog transaction commits; hydration proceeds in the background. The only gate at the end of init() is SELECT count(*) FROM t (line 202). import_into_dataflow imports every index on t, but prune_and_annotate_dataflow_index_imports then drops the ones the optimized plan does not need, so that read binds a single index and waits for a single frontier. The measured SELECT f1 FROM t WHERE f1 = 1 in benchmark() is a fast-path peek on one index and does not gate either. Contrast ManyIndexes.init(), which ends on SELECT c FROM all_v: that read imports all 200 indexes, so under strict serializable it cannot answer until every one has hydrated.

What that costs: Benchmark.run() terminates after at least 6 iterations, each roughly a testdrive docker exec plus two docker stats calls, and _memory_clusterd_filter = FilterFirst(3) discards the first three samples, so the first recorded clusterd sample lands ~10s after init(). MinAggregation then reports the minimum over the recorded samples, i.e. deliberately the earliest, least-hydrated one. When re-exports share the arrangement the 200 dataflows hydrate ~instantly and this is harmless. When they do not — 200 re-imports each arranging a 100k-row collection on the one-worker cluster_default, tens of millions of record insertions on a single worker — hydration outlasts that window, and the min sample is taken mid-build. That is the regression the scenario exists to price, and it is the case in which the number is attenuated.

A gate in the same shape as the sibling's, appended to init():

> SELECT bool_and(hydrated) FROM mz_internal.mz_hydration_statuses
  WHERE object_id IN (SELECT id FROM mz_catalog.mz_indexes WHERE on_id = (SELECT id FROM mz_catalog.mz_tables WHERE name = 't'))
true

mz_internal.mz_hydration_statuses covers indexes via mz_catalog.mz_indexes LEFT JOIN mz_internal.mz_compute_hydration_statuses, with COALESCE(hydrated, false), so a not-yet-reporting index reads as false rather than dropping out of the aggregate. Worth confirming an aliased re-export does report hydrated there before relying on it; if it does not, the same effect can be had by making the last statement a read that the optimizer cannot satisfy from a single index.

[bosconi]

Adds three feature-benchmark scenarios for the resident cost of 200 publications and one parallel-benchmark scenario that measures maintenance under a saturating peek load. I re-checked the wiring at this head: ManyIndexes is still a group parent so only its three leaves run, MaintenanceUnderPeekSaturation is picked up by all_subclasses(Scenario) in both parallel-benchmark steps, and the derived worker count always lands on a size cluster_replica_size_map defines (scale=1,workers=2^i for i in 0..5, matching the min(32, ...) cap).

Reviewed at cc05e9c.

Addressed since f63ecbb: the worker count is now derived from the host's cores, which fixes the 8-core long-parallel-benchmark agent oversubscribing the single-runtime side, and it is one local feeding both the size and the rate. ManyIndexesIdle's docstring now says it times a count rather than a lookup. The PeekIsolationUnderExpensivePeeks retune moved down into #38676, so my version comment moved with it.

Nits

• The description has fallen behind the code in two places. It says MaintenanceUnderPeekSaturation runs "one join peek in flight per worker", but the joins are now an open loop at one per worker per second, for the reason the docstring gives. And it says ManyIndexesIdle "measures one lookup", where the scenario now says, correctly, that it times a count over one index, which is a peek dataflow and not a fast-path lookup.