Skip to content

Size the algorithms built on the scan partitioner - #7967

Draft
hkaiser wants to merge 1 commit into
task-size-predicatesfrom
task-size-scans
Draft

hkaiser wants to merge 1 commit into
task-size-predicatesfrom
task-size-scans

Conversation

@hkaiser

@hkaiser hkaiser commented Nov 1, 2026

Copy link
Copy Markdown
Contributor

This is the fourth part of the task-size series (#7926, #7946, #7947), it applies on top of #7947 and is independent of the other open parts.

The scan partitioner runs each chunk of the first and third steps as a task of its own. These tasks now have the size of the iterations of their chunk: count calls of the step's function.

inclusive_scan, exclusive_scan and their transform variants size their steps by op called with the result so far and an element, the transform variants size the first step by the transformation. copy_if, partition_copy and unique_copy size the first step by the predicate or the comparison of two elements, their third step only copies the selected elements and stays unsized.

Tests: the new task_sizes_scans test and the existing tests of the changed algorithms pass on Windows.

The scan partitioner runs each chunk of the first and third steps as a task
of its own, which now has the size of count iterations of the step's
function. inclusive_scan and exclusive_scan size both steps by op called
with the result so far and an element, the transform scans size the first
step by the transformation. copy_if, partition_copy and unique_copy size
the first step by the predicate or the comparison, their third step only
copies elements and has no size.

Signed-off-by: Hartmut Kaiser <[email protected]>
@coderabbitai

coderabbitai Bot commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 27eb0c3b-93e3-4256-8487-eeb43ca1ee91
Commits

Reviewing files that changed from the base of the PR and between d5720bb and 2fa18c7.

Files selected for processing: 10
  • libs/core/algorithms/include/hpx/parallel/algorithms/copy.hpp
  • libs/core/algorithms/include/hpx/parallel/algorithms/exclusive_scan.hpp
  • libs/core/algorithms/include/hpx/parallel/algorithms/inclusive_scan.hpp
  • libs/core/algorithms/include/hpx/parallel/algorithms/partition.hpp
  • libs/core/algorithms/include/hpx/parallel/algorithms/transform_exclusive_scan.hpp
  • libs/core/algorithms/include/hpx/parallel/algorithms/transform_inclusive_scan.hpp
  • libs/core/algorithms/include/hpx/parallel/algorithms/unique.hpp
  • libs/core/algorithms/include/hpx/parallel/util/scan_partitioner.hpp
  • libs/core/algorithms/tests/unit/task_sizes/CMakeLists.txt
  • libs/core/algorithms/tests/unit/task_sizes/task_sizes_scans.cpp

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 2 remain after this review.

Summary

Summary by CodeRabbit

  • Performance
    • Improved workload sizing for parallel scan, copy, partition, and unique-copy operations. This helps distribute work more appropriately across tasks while keeping existing algorithm behavior and public interfaces unchanged.

Parallel scan and filtering algorithms now attach task-size estimates to selected partition steps. The scan partitioner scales those estimates by chunk length. A new unit test covers task sizes across scan and related algorithms.

Changes: Scan task sizing

Layer / File(s) Summary
Algorithm task-size estimates
libs/core/algorithms/include/hpx/parallel/algorithms/{copy,partition,unique}.hpp, libs/core/algorithms/include/hpx/parallel/algorithms/*scan.hpp
The algorithms estimate task sizes from operation, predicate, projection, and iterator types. They attach estimates to selected scan-partitioner steps.
Chunk-based task sizing
libs/core/algorithms/include/hpx/parallel/util/scan_partitioner.hpp
The partitioner adds sized_chunk and uses it to scale task sizes by each chunk’s iteration count.
Task-size test coverage
libs/core/algorithms/tests/unit/task_sizes/CMakeLists.txt, libs/core/algorithms/tests/unit/task_sizes/task_sizes_scans.cpp
The test target checks task sizes for scans, transform scans, copy_if, partition_copy, and unique_copy, including cases that expect an unknown size.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Feature

Possibly related issues

  • TheHPXProject/hpx issue 7837: This issue defines the task-size mechanism for scheduler decisions. This change applies task-size metadata to scan-partitioner tasks in the listed algorithms.

Suggested reviewers: dimitraka, v4xsh

Merge Risk: Minimal · up to 2fa18

No merge-blocking issue was identified in the scan task-sizing changes.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentat… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Title check Passed The title clearly and concisely identifies the main change: adding task sizing to algorithms built on the scan partitioner.
Description check Passed The description directly explains the task-sizing changes, affected algorithms, test coverage, and relationship to the task-size series.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 12 functions across 9 files. (1 skipped: 1 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

Finishing Touches
Generate docstrings
  • Commit to this branch
  • Create a new PR
Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@codacy-production

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant