Skip to content

feat(query-engine): computed top-k in range queries + stage-E equivalence tests (#581 stage E prep) - #629

Merged
milindsrivastava1997 merged 4 commits into
mainfrom
581-unify-instant-range-stage-c
Aug 26, 2026
Merged

feat(query-engine): computed top-k in range queries + stage-E equivalence tests (#581 stage E prep)#629
milindsrivastava1997 merged 4 commits into
mainfrom
581-unify-instant-range-stage-c

Conversation

@milindsrivastava1997

Copy link
Copy Markdown
Contributor

Summary

  • Range queries previously had zero top-k support at all (execute_range_query_pipeline had no limiting/formatting params). Adds step-major top-k ranking/truncation (apply_range_topk) — ranks and truncates independently at each output timestamp, since a range query's surviving key set can differ step to step, unlike instant's single-value-per-group case. Wired through all three PromQL range call sites.
  • Adds an instant/range equivalence test matrix ahead of stage E's full pipeline collapse: 4 window shapes (Tumbling ratio 1/2/3, one genuinely-overlapping Sliding case) × {no keys, SetAggregator keys, DeltaSetAggregator keys} × {instant, range}, plus 3 dedicated tests for the new topk plumbing (step-major re-ranking across steps, single-step-vs-instant equivalence, non-interference with fix(query-engine): range queries expand keys_query per output step #595's self-keyed sketch expansion).
  • Two corrections to the originally-scoped test matrix, discovered while writing it (documented in the new test file's header): a Sliding window's lookback_ms must equal window_size_ms (active assert! in execute_range_query_pipeline, tracked separately as Sliding window execution: support multi-window merge/subtract in query engine #554), so "Sliding at ratio 2/3" isn't constructible; and SetAggregator/DeltaSetAggregator are keys-side types whose window_type is independent of the paired value aggregation's, so no cells needed excluding.
  • Rebased onto main post-feat(store): added store interface for batched exact-window queries #627/refactor(query-engine): remove is_exact_query as a threaded bool #628 merge; no functional overlap (different regions of mod.rs/promql.rs).

Note: "computed top-k" over an arbitrary non-self-keyed expression (e.g. topk(5, rate(foo[5m]))) turns out to be unsupported everywhere in this codebase today, not just in range (SumAccumulator::query hard-errors on Statistic::Topk). Out of scope here, left as-is.

Test plan

  • cargo test -p query_engine_rust --lib: 520/520 passing
  • cargo clippy -p query_engine_rust --lib --tests -- -D warnings: clean
  • Roborev review (job Re-architect code to use Redis for control messages #23): 1 Low finding fixed (dead-code retain), 1 Low finding (tie-break nondeterminism) left as-is per reviewer's own note — pre-existing instant-side behavior, not introduced here

🤖 Generated with Claude Code

milindsrivastava1997 and others added 2 commits August 26, 2026 00:07
…ence tests (#581 stage E prep)

Adds step-major topk ranking/truncation to execute_range_query_pipeline
(previously range had no top-k support at all), wires it through PromQL's
range call sites, and adds an instant/range equivalence test matrix across
Tumbling/Sliding window shapes and SetAgg/DeltaSetAgg keys configs, ahead
of stage E's full pipeline collapse.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
…23)

kept_timestamps_by_key's entries are only ever inserted alongside a
timestamp drawn from that same element's own samples, so the per-sample
filter can never leave a surviving key's samples empty -- the trailing
retain was unreachable dead code.

Co-Authored-By: Claude Sonnet 5 <[email protected]>

@milindsrivastava1997 milindsrivastava1997 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review findings.

Comment thread asap-query-engine/src/engines/simple_engine/promql.rs
Comment thread asap-query-engine/src/engines/simple_engine/mod.rs
Comment thread asap-query-engine/src/engines/simple_engine/mod.rs
Comment thread asap-query-engine/src/engines/simple_engine/mod.rs
milindsrivastava1997 and others added 2 commits August 26, 2026 08:55
…Finding 1

topk(...) as one arm of a binary expression is broken two different
ways, neither of which is what Finding 1's review comment described nor
fixable as part of #629 -- both are pre-existing/orthogonal and tracked
in #631 instead. One test pins the current (surprising) None-return
behavior; the other reproduces the join-corruption bug Finding 1
actually describes, and is #[ignore]d since it isn't fixed here.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
…ones

Addresses PR #629 review findings 2-4 (mod.rs::apply_range_topk):

- Finding 2: candidates.sort_by was value-only with no tiebreak, so
  groups tied at the k-th value boundary kept a different survivor
  run to run (HashMap iteration order is randomized per-process).
  Confirmed via a flaky RED test (4/5 pass rate) before adding a
  label-values tiebreak; stable across 20+ runs after.
- Finding 3: folded into the same restructure -- index each group once
  instead of cloning its label vector per (group, timestamp) sample
  (G clones instead of G*T).
- Finding 4: documented why range has no observable (false, true)
  case for enable_topk_limiting/enable_topk_formatting, unlike
  instant's always-sort-when-Topk behavior.

Finding 1 is not addressed here -- tracked separately in #631.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
@milindsrivastava1997
milindsrivastava1997 marked this pull request as ready for review August 26, 2026 13:14
@milindsrivastava1997
milindsrivastava1997 merged commit 1fc18be into main Aug 26, 2026
16 checks passed
@milindsrivastava1997
milindsrivastava1997 deleted the 581-unify-instant-range-stage-c branch August 26, 2026 13:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant