[FEA] cuda.core: adopt nvFatbin for multi-arch fat binary assembly

[leofang]

Summary

cuda.bindings.nvfatbin (available since cuda-bindings 12.9.x) is not used anywhere in cuda.core
product code today — it only appears as a test helper that builds a two-arch fatbin to exercise
ObjectCode.from_fatbin:

cuda-python/cuda_core/tests/test_module.py

Lines 144 to 175 in c000331

	from cuda.bindings import nvfatbin
	dev = Device()
	arch = dev.arch
	# Pick a second arch different from the current device
	second_arch = "75" if arch != "75" else "80"
	# Compile to cubin for current device arch
	prog = Program(SAXPY_KERNEL, code_type="c++", options=ProgramOptions(arch=f"sm_{arch}"))
	mod = prog.compile(
	"cubin",
	name_expressions=("saxpy<float>", "saxpy<double>"),
	)
	cubin = mod.code
	sym_map = mod.symbol_mapping
	# Compile to PTX targeting the second arch
	ptx_mod = Program(SAXPY_KERNEL, code_type="c++", options=ProgramOptions(arch=f"sm_{second_arch}")).compile(
	"ptx",
	name_expressions=("saxpy<float>", "saxpy<double>"),
	)
	ptx = ptx_mod.code
	# Create fatbin with both cubin + PTX
	handle = nvfatbin.create([], 0)
	nvfatbin.add_cubin(handle, cubin, len(cubin), arch, "saxpy")
	nvfatbin.add_ptx(handle, ptx, len(ptx), second_arch, "saxpy", f"-arch=sm_{second_arch}")
	fatbin_size = nvfatbin.size(handle)
	fatbin = bytearray(fatbin_size)
	nvfatbin.get(handle, fatbin)
	nvfatbin.destroy(handle)

cuda.core covers the consume side of fat binaries
(ObjectCode.from_fatbin)
but has no produce side: to target several architectures, users must juggle N per-arch
ObjectCodes or shell out to the fatbinary CLI. nvFatbin closes this gap in-process.

Use cases:

• compile per arch (CUBIN/PTX/LTO-IR) and ship one artifact; PTX/LTO-IR entries keep JIT
  forward-compatibility on newer GPUs
• AOT kernel caches / packaging: one fatbin per kernel suite instead of per-arch files
• bundling Tile IR (nvFatbinAddTileIR) once cuda.core.Program: Support Tile IR #1322 / cuda.core.ObjectCode: Support Tile IR #1434 land

Underlying C APIs to cover

The entire libnvfatbin surface (pythonic names from
cuda.bindings.nvfatbin @ v13.3.1):

C API	cuda.bindings.nvfatbin	Purpose
nvFatbinCreate	create(options, options_count)	new fatbin handle; options include -compress=<bool>, -compress-all, -compress-mode=<mode>, -host=<name>, -32/-64
nvFatbinAddCubin	add_cubin(handle, code, size, arch, identifier)	add SASS for one arch
nvFatbinAddPTX	add_ptx(handle, code, size, arch, identifier, options_cmd_line)	add PTX (JIT-compatible entry)
nvFatbinAddLTOIR	add_ltoir(handle, code, size, arch, identifier, options_cmd_line)	add LTO-IR
nvFatbinAddReloc	add_reloc(handle, code, size)	add relocatable PTX from a host object
nvFatbinAddTileIR	add_tile_ir(handle, code, size, identifier, options_cmd_line)	add Tile IR (documented since CUDA 13.1; pairs with #1434)
nvFatbinAddIndex	add_index(handle, code, size, identifier)	new in cuda-bindings 13.3.x (not yet in the public nvFatbin docs)
nvFatbinSize / nvFatbinGet	size(handle) / get(handle, buffer)	serialize the finished fatbin
nvFatbinDestroy	destroy(handle)	release the handle
nvFatbinVersion / nvFatbinGetErrorString	version() / get_error_string(result)	introspection / error text

The bindings load libnvfatbin lazily, so each entry point also requires a new-enough runtime
library — option/entry-point support must be version-gated with actionable error messages,
in the spirit of #337.

Design sketch (draft — needs design-meeting review)

Important

The sketch below is a starting point only, not a settled design. API shape, names, and
scope should all be reviewed in the cuda.core design meeting before implementation.

Option A — one-shot constructor, no intermediate handle exposed:

code: ObjectCode = ObjectCode.from_parts(     # all names TBD
    [objcode_sm100a,                          # ObjectCode in, kind/arch from metadata...
     (ptx, "ptx", "sm_120")],                 # ...or raw bytes + explicit kind/arch
    options=...,
)                                             # ObjectCode with code_type="fatbin"

Option B — a small builder object (fb = FatbinBundler(...); fb.add(...); fb.finalize()),
only if Option A's argument shape gets unwieldy — cuda.core uses the builder pattern sparingly
(CUDA graph construction is the only case today).

Option C — Program-level sugar that compiles for multiple archs and bundles the results
(e.g. Program.compile("fatbin", archs=[...])) — likely a follow-up on top of A/B rather than
initial scope, since it multiplies compilation work behind one call.

Open questions for the meeting:

1. Can kind/arch be inferred when the input is an ObjectCode (PTX carries its arch in
   text; cubin would need parsing or an explicit argument)?
2. Options surface: a FatbinOptions dataclass mirroring ProgramOptions/LinkerOptions
   style, vs. plain kwargs?
3. Handle lifetime: route through the _cpp resource-handle registry like other native handles?
4. Version gating & diagnostics for older libnvfatbin (no add_tile_ir/add_index), per Add descriptive exception handling to linker and program options which are not supported on a CTK version basis + handle in tests #337.
5. Is Option C in scope at all initially?

References

• nvFatbin docs: https://docs.nvidia.com/cuda/nvfatbin/index.html
• Found during the CUDA 12.8 → 13.3 bindings vs. cuda.core gap sweep (2026-07-14)

-- Leo's bot