[Rust] Add Vectorize and warp-transpose Block Load/Store by tpn · Pull Request #12106 · NVIDIA/cccl

tpn · 2026-10-20T21:10:34Z

Superseded by #12088, which now owns this implementation within its complete feature family. The original commits, branch and validation evidence remain available. This draft is closed unmerged.

Historical scope and validation below refer to the original source heads.

Add the remaining Block Load/Store algorithm options. Vectorize uses wide memory accesses when type, extent and adjusted alignment permit, with Direct fallback for misaligned or counted accesses. WarpTranspose and WarpTransposeTimesliced use warp exchanges with compiler-planned scratch; both require full-block participation and a thread count divisible by 32. Vectorize Store preserves its payload. Transpose Store permutes values and validity, including zero-count writes.

Review after Block Transpose/Striped #12101. The combined owning diff has 16 files (+1,450/-136); GitHub's main-based diff includes prerequisites. Uses the pinned compiler owned by cuda-rust #1589.

Consolidates #12112 into this draft, retaining separate Vectorize and warp-transpose commits. The accepted source and pins are unchanged.

Original qualification covers host/docs/MSRV checks, native requests, compiler controls and ordinary Cargo consumers. GPU checks cover all Block algorithms, the example, sanitizers and expected traps with health replays. Artifact inspection verifies wide accesses, fallback, typed validity, launch shape and scratch. The minimal example has no stack/local traffic; the observation fixture retains such traffic. Whole CUBINs differ from earlier builds, and no timing claim is made. The accepted endpoint passed host CI; checks for this updated draft are tracked separately.

Create a host-only Cargo workspace that specializes selected CUB forms through NVRTC and reports exact scratch layouts with their LTO output. Bind specialization identities to source, headers and toolkit bytes. Include a runnable provider example, Rust guide and CPU checks. Signed-off-by: Trent Nelson <[email protected]>

Install the pinned toolchain without updating rustup itself. Signed-off-by: Trent Nelson <[email protected]>

Serve native specialization batches through the versioned Oxide boundary and exercise Load and Store through a standalone Cargo consumer with a generic helper dependency. Signed-off-by: Trent Nelson <[email protected]>

Signed-off-by: Trent Nelson <[email protected]>

Reuse the same helper sites with 64- and 128-thread kernels and retain one device static across their emitted bodies. Check each launch plan and preserve fixed external-symbol diagnostics. Signed-off-by: Trent Nelson <[email protected]>

Select fixture output locations from raw CUDA coordinates and exercise payload initialization, access and copying at both item counts. Annotate compile-fail examples and clarify hierarchy ordering in the guide. Signed-off-by: Trent Nelson <[email protected]>

Signed-off-by: Trent Nelson <[email protected]> # Conflicts: # rust/Cargo.lock

Signed-off-by: Trent Nelson <[email protected]>