Repository navigation
perf(tdigest): scan centroids for dense CDF and PMF batches - #305
Merged
tisonkun merged 3 commits intoOct 27, 2026
Merged
Conversation
tisonkun
approved these changes
Oct 27, 2026
tisonkun
left a comment
Member
There was a problem hiding this comment.
Self-proven improvement.
But you may follow #307 (comment) as well.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CDF and PMF currently repeat a centroid binary search for every ordered split point. Share a forward cursor for dense batches, while retaining binary search for sparse batches. Both use the existing tail, equal-mean, weight, and interpolation logic; the CDF loop and weight accumulation remain shared.
Benchmark
Measured
aa23deaa051e71c18beed66907f86ca23f35199eagainst675e3e123fda6da19657d345038252b16adc2ebbon an isolated EC2 c7i.2xlarge (Xeon Platinum 8488C), Rust 1.99.1 release defaults, CPU 2. Baseline calibration selects fixed work near 300 ms; both variants perform exactly the same count in seven alternating pairs. These are measured whole-batch totals.Before/after are median batch times. Speedup is the median of seven paired before/after ratios; above 1× is faster, below 1× is slower. Unrounded ratios determine the result; integer milliseconds are only for display. Medians of paired ratios need not equal the ratio of median times. These totals do not describe a single request or application latency.
Prebuilt rows count complete CDF or PMF calls on prepared digests. The lifecycle row counts complete construct/update/CDF/destroy workflows, each ingesting 100,000 items; it includes pending compression. Its paired before/after ratios range from 0.992 to 1.020, so this run does not establish a complete-lifecycle gain. The benefit is concentrated in dense queries on existing digests; the tail PMF control is slightly slower. Input generation is outside timing in both variants.
Across all 41 measured cases, the largest median latency regression is 1.21%, below the 5% limit used for this comparison. The limit applies separately to each case, including controls; it is not an average across workloads or a guarantee for every individual sample or environment. Targeted baseline/candidate correctness fingerprints match exactly.