Triggered SDMA Plans

[ariscript]

Adds a way for a running kernel to hand data to the copy engines (SDMA) without spending compute
units on the transfer. The host pre-arms copy-engine chains that wait on gates; a kernel opens a
gate once its data is written, and the copy engine copies that data to every peer and signals its
arrival there.

kernel writes a batch ──► publish(batch) opens the gate
          copy engine:  POLL_REGMEM(ready >= epoch) ─► COPY × segments ─► ATOMIC(peer arrival += 1)
peer kernel:  wait(batch) ◄── arrival

This is the plan layer only implemented with a Triton API so far. Gluon support and triggered collectives
can be added on top of this functionality.

API

ctx = iris.iris(heap_size, allocator_type="vmem_chunked")
schedules = [chunks(out[r * M : (r + 1) * M], num_batches, base=out) for r in range(world_size)]
plan = TriggeredPlan(ctx, out, schedules)      # collective: allocates gates, arms epoch 1

view = plan.begin_device_epoch()
producer[grid](..., view)                      # TriggeredView.initialize(view).publish(batch)
plan.end_device_epoch()                        # every batch delivered everywhere
plan.next_epoch()                              # barrier, re-arm, barrier

• Schedules list each rank's batches (ordered groups of 1D or 2D segments behind one gate) in
  the order its producer completes them. Builders: whole, chunks, waves (one batch per wave of
  a persistent tiled kernel). A batch can have several contributors; the last publish opens it.
• Device side (TriggeredView): publish, arrived, wait, reusable.

Design notes

• Arming uses rocm-xio's host-initiated queue to each peer: POLL_REGMEM (func 5, >=), one
  COPY_LINEAR or sub-window copy per segment, then ATOMIC ADD_RTN_64 on the peer's arrival gate.
• Ordering. Chains on one queue run strictly in arming order, so batches should be listed in
  the order they complete (a persistent kernel's waves complete in order by construction). We keep
  one queue per peer, matching rocm-xio's design. Plans sharing queues must begin and abort in
  arming order, and while chains are armed on a peer's queue, Iris.put / put_tile / put_tiles /
  quiet to that peer raise instead of queueing behind them.
• Memory ordering. publish is s_waitcnt vmcnt(0) + CTA barrier + a release atomic at system
  scope; consumers use an acquire at system scope. Measured on MI300X and MI350X; see
  docs/conceptual/triggered-sdma.md.
• Completion. end_device_epoch() checks completion with a small kernel that reads peers'
  arrival gates, rather than quiet(), which waits for everything on the queue (including other
  plans' parked chains).
• Gates are 64-byte rows (ready, arrival, completion, error, contributor counter),
  polled with >= on the low 32 bits and never reset; a plan is limited to 2³² − 1 epochs.
• Plans require allocator_type="vmem_chunked" (or "vmem"). The torch allocator re-imports
  peer heaps at new addresses on every symmetric allocation (establish_peer_access destroys the old
  imports), so a chain armed before an allocation would hold stale peer addresses. VMem keeps peer
  addresses fixed, including as the chunked heap grows. The torch allocator is rejected.

Testing

On 8× MI300X (gfx942):

• New triggered tests at 2, 4 and 8 ranks: 1D batches with several segments each, 2D waves with
  clipped edge tiles, 8 contributors releasing one 4 MiB gate over 10 epochs (stale-data check),
  head-of-line ordering, misuse cases, heap growth while a plan is armed, a buffer spanning a heap
  chunk boundary, and the host put(wait_flag=, signal_flag=) path.
• GPU-only tests of the device view, and byte-level tests of the copy, poll and NOP emitters.
• Full unit suite at 2 ranks: 3119 passed, 46 skipped.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[mawad-amd]

We need to think a bit more about the naming/new concepts introduced here.

[ariscript]

Yes, this was mostly intended as a prototype to check if this is going in the right direction and pivot if necessary.