Skip to content

Unify the async host runtime and remove cuda_async::simt - #1604

Open
elibol wants to merge 3 commits into
NVIDIA:pull-request/1603from
elibol:refactor/shared-cuda-async
Open

elibol wants to merge 3 commits into
NVIDIA:pull-request/1603from
elibol:refactor/shared-cuda-async

Conversation

@elibol

@elibol elibol commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

Tile operations and SIMT launches currently use separate async runtimes and cannot compose through the same operation types. This change removes cuda_async::simt and routes both programming models through the shared DeviceOp, DeviceFuture, execution context, scheduler, errors and launch API.

Closes #1461. Stacked on #1603, which supplies the shared core types and storage. The base is its existing pull-request/1603 mirror so this diff contains only the async migration; retarget after the prerequisite merges. Both feature branches are hosted on the contributor's fork.

  • One KernelArgument blanket implementation uses DeviceCopy. Generic Tile scalars require DType + DeviceCopy; tensor-only type parameters retain their independent DType contract. The shared argument arena preserves inline storage, padded values, over-alignment and typed destruction for the explicit unsafe boxed-argument escape hatch.
  • An inert AsyncKernelLaunchBuilder requires unsafe finalization before becoming a schedulable, immutable launch. Generated typed launchers retain their checked boundaries. Tile PDL and SIMT cluster/cooperative launch paths are preserved. Raw async macros marshal slice pointers and lengths as direct scalar values; optional CPU-reference examples use the corresponding scalar bounds.
  • Launches retain their function/module, allocation owners and access leases before enqueueing work. Early future drop retires owners on completion without waiting on the GPU. Partial errors and panics retain submitted resources; unknown completion deliberately leaks owners and exclusion. Forgotten futures also retain their submissions indefinitely. Outputs can contain ordinary Rust borrows and must be valid to drop during submission, as required by host combinators.
  • Graphs retain captured resources and reacquire allocation access on every replay. DeviceBox transfers to/from shared typed buffers without copying and preserves ordinary read permissions. Existing DeviceOperation, and_then, apply and arc spellings delegate to the shared implementations.
  • cutile_inter_kernel now uses the normal shared launch API to update the Tile-produced allocation in place, checks its pointer and allocation identity, and verifies results. Consumers, generated-project templates, documentation, UI fixtures and GPU test entry points are migrated. AGENTS.md records that future branches must originate from the user's fork.

Validation: shared async tests and doctests; Tile unit, aggregate GPU, submission-lifetime, interop and UI tests; cuda-host/cuda-macros suites; 268 cargo-oxide tests; four explicit SIMT lifetime/concurrency GPU tests. Both cargo oxide run cutile_inter_kernel and cargo oxide run async_mlp pass. Miri passes all seven argument-storage tests, and Loom passes the completion-handoff model. Host/Tile/affected SIMT clippy checks, formatting, strict async rustdoc, SPDX/API-path guards and Tile dependency checks pass.

GPU execution was validated on one RTX 5090. This does not add multi-GPU/NVLink hardware coverage. API migration and custom-operation retention obligations are documented in cuda-async/README.md.

Additional validation: the full CPU runner passes (1,381 tests; 94 intentionally ignored), all top-level SIMT example workspaces pass Clippy with warnings denied, and all 144 macro unit tests pass. The host_closure GPU example also passes all 13 scenarios, covering the corrected async slice argument marshalling.

@elibol
elibol force-pushed the refactor/shared-cuda-async branch from e97d214 to fcfe932 Compare October 21, 2026 23:13

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant