Skip to content

Support checked native collectives with local payloads - #1589

Draft
tpn wants to merge 88 commits into
NVIDIA:mainfrom
tpn:codex/coop-interop-payload-collectives
Draft

tpn wants to merge 88 commits into
NVIDIA:mainfrom
tpn:codex/coop-interop-payload-collectives

Conversation

@tpn

@tpn tpn commented Oct 21, 2026 •

Copy link
Copy Markdown

Summary

Native collectives such as scans consume per-thread inputs and produce fresh results, while existing checked calls require a global-memory resource. Add checked Block and Warp payload calls so these operations can use the compiler's participation checks and shared-storage plan.

Changes

  • Accept input/output pointers and optional uniform scalar controls. Block calls support existing storage descriptors; Warp calls use per-group scratch.
  • Reject legacy untracked calls inside planned-storage roots, including roots without allocation-resource witnesses.
  • Exercise both payload forms in the existing native-library example and document their unsafe contract.

Depends on #1586. This draft contains the cumulative integration stack; its owning change is the payload-call support and accompanying checks/docs.

Testing

The corrected backend passes 20 focused compiler controls, all 156 existing Load/Store compiler controls, 41 backend unit tests, formatting and strict Clippy. Frontend unit tests, rustdoc and the book build also pass.

A normal Git installation of this head builds and runs a private CUB Scan probe using the real cuda.coop payload and Load/Store: five cases and 1,792 values pass. The preceding candidate also passes the native-library example's 2,304 values, three probe sanitizers and an expected invalid-input device error. The final probe preserves that candidate's device instructions and resources.

Checklist

  • All commits signed off.
  • Existing source files only; license headers preserved.

fxeqxmulfx and others added 30 commits October 12, 2026 02:14
With `lto = "fat"` or `lto = "thin"`, the build of a binary that uses a
library with kernels stopped with an error: "failed to get bitcode from
object file for LTO (Can't find section .llvmbc)". rustc keeps a list of
the Rust objects in each rlib. This list contains the objects of all
modules. With LTO, rustc reads bitcode from these objects and removes
them from the link. The artifact objects have no bitcode.

If a session writes an rlib, the backend now gives each artifact object
to rustc as the DWARF object of its module. This object is in the rlib,
but it is not in the list. LTO does not read this object, and the linker
gets it from the rlib.

The system linker does not get the DWARF objects. If the session also
writes a binary or a dynamic library, the backend gives the artifact
objects to this linker as link arguments. With
`-Zlink-native-libraries=no`, rustc does not give link arguments to the
linker. Then the artifact objects of this session are Rust objects, as
before.

The book said that `join_codegen()` and `link()` only call the LLVM
backend. Now it tells what they add to the outputs.

CI builds `cuda_module_in_lib` with fat LTO. The library is an rlib, and
then an rlib and a cdylib. After each build, CI makes sure that the
binary has the PTX bundles of the library and of the binary. It also
makes sure that the cdylib has the PTX bundle of the library.

Fixes NVIDIA#1495

Signed-off-by: Mikhail Borisov <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Use Cargo target names for standalone emit-ltoir artifacts. Bind the
compiler and reader to the same intermediate directory when invoked
from a Cargo workspace member.

Signed-off-by: Trent Nelson <[email protected]>
Accept ordered external LTO files when materializing embedded cubins.
Include their content in the codegen fingerprint and verify owned bytes
before linking through the existing finalizer. Preserve inputs that
overlap generated output paths and diagnose unresolved device symbols.

Add a standalone CUDA FFI example and a native regression script.

Signed-off-by: Trent Nelson <[email protected]>
Reuse existing example dependency pins and classify external LTO as
an FFI fixture. Run its linking regression in native sweeps and verify
relocatable debug output in the existing full-debug route.

Signed-off-by: Trent Nelson <[email protected]>
Preserve exact geometry and fixed dynamic shared-memory requirements
through artifact loading so preparation can select and validate the
compiled allocation. Keep artifact writers and readers on one in-tree
implementation while preserving legacy bundles without launch facts.

Signed-off-by: Trent Nelson <[email protected]>
Combine the reviewed compiler prerequisites for native library calls.
Keep the external LTO fixture on the shared in-tree artifact implementation.

Signed-off-by: Trent Nelson <[email protected]>
Use the reviewed Cargo target and output-directory correction in the
combined compiler prerequisite tree.

Signed-off-by: Trent Nelson <[email protected]>
Resolve concrete registration functions and scalar argument types during
device compilation, then invoke the native provider and link its owned
LTO output through the ordinary artifact finalizer.

Signed-off-by: Trent Nelson <[email protected]>
Carry the convergence marker through private dependency metadata and
emit it on each marked LLVM declaration before native optimization.
Deduplicate repeated extern symbols while rejecting incompatible
signatures and convergence requirements.

Signed-off-by: Trent Nelson <[email protected]>
Signed-off-by: Trent Nelson <[email protected]>

# Conflicts:
#	cuda-oxide/crates/rustc-codegen-cuda/src/library.rs
Keep original MIR and static identities while emitting checked helper

bodies for each root. Route typed calls and native plans through those

bodies so shared helpers can use distinct block shapes.

Signed-off-by: Trent Nelson <[email protected]>
Treat each block as one cluster when a known earlier target is selected.
Bind a device hint only after this fallback is used, so later feature
selection cannot silently change its semantics. Keep raw intrinsic and
cluster operation target requirements intact.

Signed-off-by: Trent Nelson <[email protected]>
tpn added 20 commits October 20, 2026 11:14
…op-interop-storage-publication

Signed-off-by: Trent Nelson <[email protected]>
…op-interop-callback-publication

Signed-off-by: Trent Nelson <[email protected]>
…op-interop-storage-publication

Signed-off-by: Trent Nelson <[email protected]>
…op-interop-callback-publication

Signed-off-by: Trent Nelson <[email protected]>
Recognize canonical group queries and checked scratch-free warp calls.
Track participation and memory access by group scope while preserving
block synchronization and existing native-call contracts.

Signed-off-by: Trent Nelson <[email protected]>
Carry inline helper scopes through MIR lowering whenever line tables
are enabled, so device instructions keep their source file and callsite.
Keep local variable metadata limited to full debug information.

Signed-off-by: Trent Nelson <[email protected]>
Carry logical widths through checked call lowering and preserve the
completion scope of canonical WarpTile synchronization.

Signed-off-by: Trent Nelson <[email protected]>
Preserve the first matching call edge and library record while avoiding
repeated linear scans during collective validation.

Signed-off-by: Trent Nelson <[email protected]>
Keep active control and memory completion checks while allowing wider
operations after finite sibling work. Remove the persistent cycle
restriction and its helper propagation.

Signed-off-by: Trent Nelson <[email protected]>
Add Block and Warp payload call markers with optional uniform controls.
Reuse participation checks and storage planning for local input/output.

Exercise both roles in the native library smoke fixture.

Signed-off-by: Trent Nelson <[email protected]>
Apply the existing checked-call admission guard when storage is planned,
even when the kernel has no allocation-resource witness. This prevents
legacy calls from using scratch outside the storage plan.

Signed-off-by: Trent Nelson <[email protected]>
@copy-pr-bot

copy-pr-bot Bot commented Oct 21, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@nihalpasham nihalpasham added cuda-oxide SIMT programming model: rustc backend, cargo oxide, cuda-device, cuda-host, book codegen Device code-generation pipeline (Rust MIR to IR to PTX) device-apis User-facing device-side and kernel-authoring APIs (cuda-device, cuda-macros) safety Memory safety, soundness, or undefined behavior interop Interop with external compilers / frontends enhancement New feature or request Depends Blocked on another PR/issue or an upstream dependency labels Oct 21, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

codegen Device code-generation pipeline (Rust MIR to IR to PTX) cuda-oxide SIMT programming model: rustc backend, cargo oxide, cuda-device, cuda-host, book Depends Blocked on another PR/issue or an upstream dependency device-apis User-facing device-side and kernel-authoring APIs (cuda-device, cuda-macros) enhancement New feature or request interop Interop with external compilers / frontends safety Memory safety, soundness, or undefined behavior

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants