feat(store): added store interface for batched exact-window queries - #627
Conversation
Checkpoint before merging main (legacy stores gated behind feature flag in #624). Not yet wired into scan_windows_via_exact; no tests yet.
| /// `current_epoch` / `sealed_epochs` in a single pass. Otherwise identical semantics | ||
| /// to calling `query_precomputed_output_exact` once per window and merging the | ||
| /// results (a window with no exact match simply contributes nothing). | ||
| fn query_precomputed_output_exact_batch( |
There was a problem hiding this comment.
The new batched exact-query methods drop the lock_profiling instrumentation (wait/hold time logging) present on the single-window query_precomputed_output_exact.
Building with --features lock_profiling to diagnose lock contention on range/instant queries: query_precomputed_output_exact_batch (here and in global.rs:692) never emits the lock wait/hold time logs that the single-window path emits, so exactly the code path most affected by the increased lock-hold-duration (see other comment) is invisible to that diagnostic tooling.
| /// whole `windows` slice instead of once per window. Otherwise identical semantics to | ||
| /// calling `query_precomputed_output_exact` once per window and merging the results | ||
| /// (a window with no exact match simply contributes nothing). | ||
| fn query_precomputed_output_exact_batch( |
There was a problem hiding this comment.
query_precomputed_output_exact_batch is duplicated almost line-for-line between this file and per_key.rs (epoch lookup loop, read-count update loop, debug logging), adding ~90 more lines of near-identical logic that must be kept in sync by hand.
A future fix to the exact-match resolution order (current_epoch then sealed_epochs newest-to-oldest) or to the read-count bookkeeping could get applied to one copy (per_key.rs:801-831) and forgotten in the other (here, 722-753), silently reintroducing a bug in one of the two lock strategies — the contract test suite may not catch a strategy-specific regression.
milindsrivastava1997
changed the title
…e per_key/global Addresses review feedback on #609's batch method: - per_key.rs and global.rs had dropped the lock_profiling wait/hold-time instrumentation that the single-window path has, on exactly the path now most affected by longer lock hold times. - The per-window epoch-resolution loop (current_epoch/sealed_epochs lookup, read-count bookkeeping) was copy-pasted between the two backends. Extracted into common::resolve_exact_windows, shared by both.
1 participant
Adds
Store::query_precomputed_output_exact_batch: resolves N windows against one (metric, aggregation_id) shard in a single lock acquisition instead of N.per_key.rs: one read-lock for the whole batch.global.rs: one process-wide mutex for the whole batch.scan_windows_via_exact: builds its grid upfront, one batched call instead of a per-window loop — this loop fed range queries, instant tumbling queries, and keys queries alike after refactor(query-engine): fetch non-exact store queries via a window-grid walk of exact lookups #616/fix(query-engine): Sliding range queries no longer sum overlapping windows #621.Closes #609.