Skip to content

Allow any site to be the root of a hierarchical communicator - #7829

Merged
hkaiser merged 1 commit into
TheHPXProject:masterfrom
constracktor:rotating_root_leader
Oct 23, 2026
Merged

hkaiser merged 1 commit into
TheHPXProject:masterfrom
constracktor:rotating_root_leader

Conversation

@constracktor

Copy link
Copy Markdown
Contributor

The tree of a hierarchical communicator was always rooted at site 0. Make root_site work for any site: the leader of a range of sites is the root if the range contains it and its leftmost site otherwise. Leaders nest, so the root takes part in every level of the tree while the ranges stay contiguous in site order. Gather, scatter and the scans keep their site order without any rotation, and reductions need no commutativity.

The communicator names of a non-zero root get a root/ prefix, so different roots can share a base name; the names of root 0 are unchanged. The root-side operations (broadcast_to, gather_here, reduce_here, scatter_to) must be called from the root site, and the composite operations accept only the default or the root of the communicator as root_site.

Every hierarchical collective test now also runs with other roots, including across localities, and hierarchical_helpers checks which sites lead which ranges for every root.

@constracktor
constracktor requested a review from hkaiser as a code owner October 22, 2026 16:47
@coderabbitai

coderabbitai Bot commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: bc91dca2-1de2-4b4c-9e47-b2e2ac5562d1
📥 Commits

Reviewing files that changed from the base of the PR and between a8ef320 and 49aef0f.

📒 Files selected for processing: 2
  • libs/full/collectives/include/hpx/collectives/create_communicator.hpp
  • libs/full/collectives/src/detail/hierarchical_helpers.cpp

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.

📝 Summary

Summary by CodeRabbit

  • New Features
    • Hierarchical communicators now support selecting a root site other than site 0, and provide a way to query the configured root.
    • Hierarchical collective operations support root-aware group leadership.
  • Bug Fixes
    • Corrected hierarchical collectives to use each communicator’s configured site, improving behavior with nonzero roots.
  • Tests
    • Expanded coverage for nonzero roots across hierarchical collectives, including mismatched and out-of-range root validation.

Hierarchical communicators now support a configured root site other than site 0. Communicator construction and topology helpers carry that root through the tree, and collective operations use stored sub-communicator site identifiers. Unit tests cover nonzero roots across collective families and topology configurations.

Changes: Configurable hierarchical roots

Layer / File(s) Summary
Root contract and topology rules
libs/full/collectives/include/hpx/collectives/create_communicator.hpp, libs/full/collectives/include/hpx/collectives/detail/hierarchical_helpers.hpp, libs/full/collectives/src/detail/hierarchical_helpers.cpp
The communicator exposes its configured root. Validation and representative selection use that root and its position in each range.
Rooted communicator construction
libs/full/collectives/src/create_communicator.cpp
Recursive construction carries the selected root into leaf and inter-group communicators. Nonzero roots are accepted and included in communicator names.
Collective operations use configured roots
libs/full/collectives/include/hpx/collectives/*.hpp, libs/full/collectives/include/hpx/collectives/detail/hierarchical_*.hpp
Hierarchical collectives validate callers against the communicator root and use recorded site identifiers for sub-communicator operations.
Root-parametrized collective coverage
libs/full/collectives/tests/unit/CMakeLists.txt, libs/full/collectives/tests/unit/*hierarchical.cpp, libs/full/collectives/tests/unit/hierarchical_helpers.cpp, libs/full/collectives/tests/unit/hierarchical_roots.hpp, libs/full/collectives/tests/unit/subtree_gather_scatter.cpp
Tests exercise configured roots across collective families, local and distributed setups, and representative-selection cases. Listed networking tests use four localities.

Priority: ➖ Normal

Estimated code review effort: 4 (Complex) | ~50 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant Caller
  participant create_communicator
  participant recursively_fill_communicators
  participant hierarchical_collective
  Caller->>create_communicator: provide root_site
  create_communicator->>recursively_fill_communicators: construct tree with root_site
  recursively_fill_communicators-->>Caller: return rooted communicator set
  Caller->>hierarchical_collective: invoke operation with communicator
  hierarchical_collective->>hierarchical_collective: validate root and use recorded site identifiers
Loading

Suggested reviewers: iemanshuman

Merge Risk: ⚪ Minimal · up to 49aef

The reviewed changes are consistent across communicator construction, collective operations, and test coverage, with no merge-blocking risk identified.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 30.61% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 98 functions across 29 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Title check Passed The title "Allow any site to be the root of a hierarchical communicator" matches the main change. The raw summary and PR description both show that hierarchical communicators now accept any site as ro…
Description check Passed The description states that the hierarchical communicator tree was always rooted at site 0 and that root_site now works for any site. It explains the leader selection rule for ranges, the root/ com…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title "Allow any site to be the root of a hierarchical communicator" matches the main change. The raw summary and PR description both show that hierarchical communicators now accept any site as root_site, where previously the tree was always rooted at site 0. The title is specific, concise, and free of file lists or vague wording.

Full details: Description check

Explanation

The description states that the hierarchical communicator tree was always rooted at site 0 and that root_site now works for any site. It explains the leader selection rule for ranges, the root<N>/ communicator name prefix for non-zero roots, the call-site restrictions for root-side operations, and the expanded test coverage. This matches the changeset in the summaries.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@StellarBot

Copy link
Copy Markdown
Collaborator

Can one of the admins verify this patch?

@codacy-production

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@hkaiser hkaiser left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this is a nice generalization. I went through the tree construction and the walks: since leaders nest, the root leads every range containing it, each sub-communicator is created with the leader's rank as its root on all sites, and communicators.site(i) is exactly that rank wherever the code used to pass this_site_arg(0). Gather, scatter and the scans keep the site order. I built the PR locally (Windows, Release, TCP parcelport) and all 17 affected collectives tests pass.

The DCO check fails: the commit has no Signed-off-by line. Please amend it with git commit --amend -s and force-push.

Two small things about the documentation of root_site, see the inline comments.

Comment thread libs/full/collectives/include/hpx/collectives/create_communicator.hpp Outdated
Comment thread libs/full/collectives/src/detail/hierarchical_helpers.cpp Outdated
The tree of a hierarchical communicator was always rooted at site 0.
Make root_site work for any site: the leader of a range of sites is the
root if the range contains it and its leftmost site otherwise. Leaders
nest, so the root takes part in every level of the tree while the ranges
stay contiguous in site order. Gather, scatter and the scans keep their
site order without any rotation, and reductions need no commutativity.

The communicator names of a non-zero root get a root<N>/ prefix, so
different roots can share a base name; the names of root 0 are unchanged.
The root-side operations (broadcast_to, gather_here, reduce_here,
scatter_to) must be called from the root site, and the composite
operations accept only the default or the root of the communicator as
root_site.

Every hierarchical collective test now also runs with other roots,
including across localities, and hierarchical_helpers checks which sites
lead which ranges for every root.

Signed-off-by: constracktor <[email protected]>
@hkaiser

hkaiser commented Oct 23, 2026

Copy link
Copy Markdown
Contributor

The remaining CI problem is unrelated.

@hkaiser
hkaiser merged commit cacea4d into TheHPXProject:master Oct 23, 2026
80 of 81 checks passed
@hkaiser hkaiser mentioned this pull request Oct 23, 2026
1 of 2 tasks
hkaiser added a commit that referenced this pull request Oct 26, 2026
Describe the PRs merged since release-2.0.X was last rebuilt on master
(#7769, #7784, #7787, #7829, #7832, #7833, #7835, #7836, #7838, #7839, #7845,
#7855, #7858, #7860, #7864, #7865, #7867), and state in the C++20 modules
section and under Breaking changes that applications must include only the
public headers and the generated hpx/modules/<module>.hpp headers, not the
headers inside an HPX module. Regenerate the closed issue and pull request
lists for release-2.0.X at b059b41 against the v1.11.0 tag (795 PRs,
178 issues).

Signed-off-by: Hartmut Kaiser <[email protected]>
hkaiser added a commit that referenced this pull request Oct 27, 2026
Describe the PRs merged since release-2.0.X was last rebuilt on master
(#7769, #7784, #7787, #7829, #7832, #7833, #7835, #7836, #7838, #7839, #7845,
#7855, #7858, #7860, #7864, #7865, #7867), and state in the C++20 modules
section and under Breaking changes that applications must include only the
public headers and the generated hpx/modules/<module>.hpp headers, not the
headers inside an HPX module. Regenerate the closed issue and pull request
lists for release-2.0.X at b059b41 against the v1.11.0 tag (795 PRs,
178 issues).

Signed-off-by: Hartmut Kaiser <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants