Repository navigation
Conversation
Measure completed queries with a delayed light task, a delayed heavy task, and no setup delay in static and adaptive execution. Environment: Datadog workspace Co-Authored-By: Codex GPT-6 <noreply@localhost>
joybestourous
marked this pull request as ready for review
November 10, 2026 16:14
gabotechs
approved these changes
Nov 10, 2026
gabotechs
left a comment
Collaborator
There was a problem hiding this comment.
Nice! just a small comment about doc placement, but otherwise looks good. Thanks @joybestourous!
Move the microbenchmark instructions from the README into the benchmark module documentation. Environment: Datadog workspace Co-Authored-By: Codex GPT-6 <noreply@localhost>
gabotechs
approved these changes
Nov 10, 2026
joybestourous
added a commit
to joybestourous/datafusion-distributed
that referenced
this pull request
Nov 10, 2026
Keep the benchmark documentation beside its source, as merged in datafusion-contrib#873, and remove the duplicate README section from this branch. Environment: Datadog workspace Co-Authored-By: Codex GPT-6 <noreply@localhost>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Task setup skew can postpone useful work on a ready worker. Add a Criterion benchmark for that scheduling case before the execution changes in #852. This PR changes no library behavior and adds no dependencies.
The benchmark runs one CPU-heavy task and one light task over localhost gRPC, in both static and AQE modes. Each mode has three cases: no routing delay, a 250 ms routing delay on the light task, and the same delay on the heavy task. It measures execution through complete output collection, excluding worker startup and planning. Every iteration checks output
[0, 1]and exactly one heavy remote execution. An untimed preflight reports whether CPU work starts and finishes before delayed sibling routing returns.Applied unchanged to #852, the benchmark demonstrates the conditional benefit:
In both delayed-light preflights, heavy work started and finished before sibling routing returned with #852; neither witness occurred on the baseline. The controls stayed essentially flat. This demonstrates value when ready useful work can overlap a sibling's setup delay; it does not predict representative S3 or multi-host speedups.
Validation: optimized builds and all six cases passed on baseline and candidate. Formatting and Criterion-target Clippy with warnings denied passed.
Measurement scope and reproduction
Measurements used baseline
0291e08(maine5d6ea3plus this benchmark) and candidate source matching #852 at354d386. The benchmark and lockfile were identical. The candidate binary was built before the final commit; source hashes were verified against that commit.This was a controlled synthetic experiment on a shared host, with the same four-CPU affinity for both binaries, four Tokio threads, ten Criterion samples per case, one-second warmup and a five-second target measurement window. Three outer pairs were attempted in baseline/candidate, candidate/baseline, baseline/candidate order. Two opposite-order pairs passed the predefined resource gate. The entire third pair was excluded because baseline selected-CPU average utilization p95 was 85.31%, exceeding the 85% threshold; the maximum-utilization threshold was 98%. Raw results for both sides of the excluded pair were retained. Results were not filtered by measured latency.
Times above are medians of the accepted per-run Criterion medians; changes are medians of the paired percentage changes. Two outer pairs illustrate this scheduling case, rather than establishing production performance or precise population estimates.
cargo bench -p datafusion-distributed-benchmarks --bench startup_overlap -- --save-baseline main # Run the same benchmark on the candidate revision: cargo bench -p datafusion-distributed-benchmarks --bench startup_overlap -- --baseline mainUse separate build directories or rebuild both local packages when switching revisions. Confirm the candidate's delayed-light preflight shows early work and the baseline's does not before interpreting timings.