Skip to content

Make copy engine initialization optional - #579

Closed
efric wants to merge 1 commit into
ROCm:mainfrom
efric:efric/sdma-optional
Closed

efric wants to merge 1 commit into
ROCm:mainfrom
efric:efric/sdma-optional

Conversation

@efric

@efric efric commented Oct 9, 2026

Copy link
Copy Markdown
Collaborator

Motivation

Iris currently initializes SDMA and creates its host/device queues eagerly when constructing a context, even for workloads that never use SDMA. The issue is that each queue reserves 8 MiB of host/device side memory, with the total allocation growing with the number of local ranks.

Make SDMA opt-in through a new copy_engine parameter on Iris and iris.iris(), defaulting to None. The default skips SDMA initialization, queue creation, and queue-context tensor allocation. SDMA users must explicitly pass copy_engine="sdma" to enable the existing host- and device-initiated operations.

When disabled, get_copy_engine_ctx() returns None, and host SDMA methods raise an error explaining how to enable SDMA. Update the SDMA examples and tests to opt in, and document the new behavior.

Validation: Ruff lint and formatting passed. Targeted tests on two AMD GPUs covered context initialization, symmetric allocation, barriers, SDMA operations, and message passing: 231 passed and 36 existing ROCm graph-capture skips per rank.

Submission Checklist

@efric

efric commented Oct 13, 2026

Copy link
Copy Markdown
Collaborator Author

subsumed by #568

@efric efric closed this Oct 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant