Skip to content

fix(intrinsics): lower fast float ops - #187

Merged
nihalpasham merged 2 commits into
NVIDIA:mainfrom
niklebedenko:prepare-fast-float-intrinsics
Jun 16, 2026
Merged

nihalpasham merged 2 commits into
NVIDIA:mainfrom
niklebedenko:prepare-fast-float-intrinsics

Conversation

@niklebedenko

Copy link
Copy Markdown
Contributor

Closes #186.

Related: #62, #142, #148, #161.

Problem

core::intrinsics::{fadd_fast,fsub_fast,fmul_fast,fdiv_fast,frem_fast} are generic compiler intrinsics with no MIR body. When the importer leaves them as ordinary calls, unresolved cuda-oxide placeholder callees can survive into LLVM lowering and fail verification or symbol resolution.

The initial lowering used FastmathFlagsAttr::default() while claiming to emit fast-math binops. In pliron-llvm, FastmathFlagsAttr::default() is FastmathFlags::empty(), so that did not express Rust's finite/non-NaN f*_fast contract.

Fix

The importer recognizes the core::intrinsics and std::intrinsics spellings for all five f*_fast operations and emits the same centralized Rust-intrinsic placeholder calls as the other target-specific Rust intrinsics.

mir-lower now routes those placeholders before the libdevice path and replaces them with the matching LLVM float binop: llvm.fadd, llvm.fsub, llvm.fmul, llvm.fdiv, or llvm.frem. The fast-intrinsic path uses FastmathFlags::FAST.into() explicitly instead of the empty default.

Diligence

Sibling float intrinsic paths checked:

  • fixed-width core::intrinsics math such as sqrtf32, powif64, fmaf32, round_ties_even_f64, maximum_number_nsz_f32, and cbrtf32;
  • std::intrinsics aliases for the same fixed-width math intrinsics;
  • libm path matching, including re-exported names such as libm::sqrtf and nested names such as libm::math::sqrt::sqrtf;
  • std::sys::cmath names for atan, atan2, and cbrt;
  • ordinary MIR float arithmetic lowering, which still intentionally uses the existing empty/default fast-math attribute behavior and is not conflated with Rust's explicit f*_fast intrinsics;
  • fcmp lowering, whose regression coverage depends on empty fast-math flags so NaN checks do not become poison.

The generic f*_fast operations are the only sibling path with no type suffix and no libdevice entry point, so they are routed to LLVM binops rather than the libdevice declaration/call path.

Tests

Added a focused mir-lower regression that constructs placeholder calls for all five f*_fast intrinsics for both f32 and f64. The test asserts:

  • FastmathFlagsAttr::default() is distinct from explicit FastmathFlags::FAST;
  • each placeholder lowers to the expected LLVM float binop;
  • both f32 and f64 monomorphizations are covered for every operation;
  • every lowered binop carries explicit LLVM fast flags;
  • no unresolved mir.call or placeholder-named LLVM call survives.

Validation

Targeted local validation:

cargo test -p mir-lower test_fast_float_intrinsics_lower_to_explicit_fast_binops

niklebedenko and others added 2 commits June 16, 2026 11:28
The fast-float intrinsic lowering set FastmathFlags::FAST on the llvm
float binops, but the textual exporter never read the attribute, so the
flags were dropped at .ll emission and the fast-math contract was inert
end to end. Read the fast-math attribute in export_binop and print the
matching LLVM keywords (the all-bits set as the 'fast' shorthand, any
subset as individual keywords) between the opcode and the type.

This is a no-op for integer binops (no attribute) and for float ops
lowered from ordinary Rust arithmetic (empty flags), so existing IR is
unchanged; it only fires for the f*_fast intrinsics. Verified end to end:
a kernel calling core::intrinsics::f{add,sub,mul,div,rem}_fast on f32 and
f64 now exports 'f.. fast float'/'f.. fast double', while vecadd's plain
'fadd float' is unchanged. Adds a focused export regression test.
@nihalpasham
nihalpasham force-pushed the prepare-fast-float-intrinsics branch from 563f12a to 71680ac Compare June 16, 2026 06:20
@nihalpasham nihalpasham added bug Something isn't working intrinsics Device intrinsics and libdevice math mappings codegen Device code-generation pipeline (Rust MIR to IR to PTX) IR-lowering Lowering between dialects (dialect-mir to LLVM dialect) labels Jun 16, 2026
@nihalpasham

Copy link
Copy Markdown
Collaborator

Lowers the f*_fast compile gap correctly and matches rustc's flag set.

One gap: the textual exporter was dropping the fast-math flags, so they never reached the PTX (the optimization was inert).

  • Layered: taught export_binop to emit fast-math flags (no-op for ordinary float math) + an export regression test.
  • .ll now shows f.. fast float/double for all five ops;

@nihalpasham
nihalpasham merged commit ab610cf into NVIDIA:main Jun 16, 2026
24 checks passed
@niklebedenko
niklebedenko deleted the prepare-fast-float-intrinsics branch June 24, 2026 11:40
meefs pushed a commit to meefs/cuda-oxide that referenced this pull request Sep 30, 2026
Transitive dependency (rayon via candle/criterion/gemm). Affected
versions dereference the underlying pointer in the fmt::Pointer impl
for Atomic and Shared even when it is invalid (e.g. Atomic::null).
Lockfile-only change; cargo deny check advisories passes.

Signed-off-by: Melih Elibol <[email protected]>
meefs pushed a commit to meefs/cuda-oxide that referenced this pull request Sep 30, 2026
…NVIDIA#225, NVIDIA#185) (NVIDIA#256)

Three squash merges landed from branches that had resolved a merge against
main by keeping pre-merge copies of files, so each one quietly reverted an
earlier, unrelated merge:

- NVIDIA#239 (2482cb9) reverted NVIDIA#240 (1216aef): the const raw-pointer surface
  (as_ptr, program_id/num_programs, the cast_* and get_tensor_base helpers,
  unsafe on load_ptr_tko/store_ptr_tko), the compiler's const-pointer
  typing, the two GPU test files and the matching book and tutorial text.
- NVIDIA#247 (5f5f4c5) reverted NVIDIA#225 (a2843e7): integer division and remainder
  serialization (rounding<zero> and inferred signedness on divi/remi) and
  its test.
- NVIDIA#187 (35a9cc2) reverted NVIDIA#185 (cbf52c0): the flake.nix toolkit-assembly
  hardening and host-arch driver directory derivation.

This re-applies all three verbatim as cherry-picks. The only overlapping
file, cutile-book/guide/performance.md, merges cleanly with NVIDIA#239's own
edits. A per-file scan of the last 60 merges finds no other reversal still
missing at HEAD: NVIDIA#249's removal of the About page was deliberate, and the
two setup-python flips in .github/workflows/pages.yml were re-applied by
the later dependabot bump.

Verified: cargo fmt, cargo build, and the restored GPU tests
(const_pointers, program_id) plus integer_ops, memory_and_atomic_ops and
optimization_hints pass on an RTX 5090 with CUDA 13.3.

Signed-off-by: Melih Elibol <[email protected]>
Co-authored-by: Gabriel Wu <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working codegen Device code-generation pipeline (Rust MIR to IR to PTX) intrinsics Device intrinsics and libdevice math mappings IR-lowering Lowering between dialects (dialect-mir to LLVM dialect)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Lower Rust fast floating-point intrinsics with explicit fast flags

2 participants