Union with serialized bitmap, continued - #361
Draft
harry-hao wants to merge 17 commits into
Draft
Conversation
union reads all containers sequentially, so the offsets table is redundant. Replace the parse-and-seek path with a single reader.seek call and drop the with_offsets impl.
harry-hao
force-pushed
the
union-patched
branch
from
August 11, 2026 01:59
8a53288 to
ccb87b6
Compare
…lized_impl Tight while loop is what 5c60dd40 used; extend_from_slice does not show a measurable benefit on top of with_capacity (capacity is already reserved). Co-Authored-By: Claude Opus 4.7 <[email protected]>
Three scenarios track who gets the "larger side as rhs" treatment: both_swap — baseline and compare both swap none_swap — neither swaps (original order) ws_only_swap — only compare swaps when USE_WITH_SERIALIZED is set Setup is unconditional; swap is decided per-bench, independent of env var, so baseline and compare see consistent inputs in both_swap / none_swap. Co-Authored-By: Claude Opus 4.7 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a follow-up to #343.
Regarding the benchmark diffs, I made some changes to
pairwise_ops_with_serialized:.cloned()fromdataset.bitmaps.iter().cloned().tuple_windows(), so that setup clones references to RoaringBitmaps instead of owned instances.iter_batched_refinstead ofiter_batched, which passes input by reference.With this fix applied, no regression is observed from introducing
union_with_serialized_unchecked:The fixed benchmark also runs faster compared to before:
(imp = improved, no = no significant change)
Next steps
Unlike intersection, union cannot skip containers — both sides must be fully merged. Whether
union_with_serializedcan deliver net performance gains, and if so how, remains an open question. I'll keep investigating and post findings to this PR.