Skip to content

fix(intrinsics): pin the probe toolchain by rustc commit, not llc bytes - #1227

Merged
nihalpasham merged 1 commit into
NVIDIA:mainfrom
almightychang:probe-llc-identity
Sep 6, 2026
Merged

nihalpasham merged 1 commit into
NVIDIA:mainfrom
almightychang:probe-llc-identity

Conversation

@almightychang

@almightychang almightychang commented Sep 3, 2026 •

Copy link
Copy Markdown
Contributor

Closes #1211

Summary

Validate the selected llc by version plus the rustc -vV commit instead of its file hash. Keep SHA-256 as provenance. Explicit --llc comparison probes remain unchecked.

repository source identity                 per-evidence provenance
┌──────────────────────────────┐           ┌──────────────────────┐
│ upstream.lock                │           │ evidence/catalog     │
│ rustc commit (enforced)      │──────────▶│ llc SHA-256 (printed)│
└──────────────┬───────────────┘           └──────────────────────┘
               │
               ▼
      llc version + commit
          selected probe

Chose upstream.lock because one pinned Rust toolchain probes all records, and the lock's [llvm] provenance line already named this commit in prose. A per-evidence field would duplicate the commit 1025 times, back-fill existing evidence from that prose line, and churn evidence, catalog.json, and generated stamps.

The commit check sits in validate_backend_identity next to the version check, as the issue comment specified, so the listed tests exercise one function; the same expected commit is passed for every record.

Changes

  • Add required [rust_toolchain] commit_hash = "e457a7b0d326d67b4322ef0d11bd715cfaeda48f".
  • Parse lock and rustc -vV commits as exactly 40 lowercase hex characters.
  • Check version and commit for selected probes; print the rustc commit next to the llc SHA-256, which is now provenance only.
  • Fail loudly for missing or malformed toolchain metadata.
  • Update the Justfile, workflow, and book upstream.lock row.
  • Add no Nix CI lane, per fix(nix): advance the rust-overlay pin to the toolchain's nightly, and gate the flake in CI #1210.

Testing

Before, origin/main 596a6353d failed under Nix:

error: rust-toolchain llc SHA-256 mismatch: selected evidence records 2e5bc688625ca7d98bca9b5f0f0d14525d2eb59e8a9842964e480df3ffd1f921, found ca7af83e237af640fe337faff2eff49055988921933186ab5535b38addeb7b49; use an explicit `--llc` only for a comparison probe

After, the first selected-backend lines show the same source identity with different provenance bytes:

rustup: selected evidence backend LLVM version 23.1.0-rust-1.100.0-nightly (rustc commit e457a7b0d326d67b4322ef0d11bd715cfaeda48f; llc SHA-256 2e5bc688625ca7d98bca9b5f0f0d14525d2eb59e8a9842964e480df3ffd1f921) lowered abs_bf16x2 to `abs.bf16x2 <register>, <register>;` for selected evidence target sm_80 +ptx70
nix:   selected evidence backend LLVM version 23.1.0-rust-1.100.0-nightly (rustc commit e457a7b0d326d67b4322ef0d11bd715cfaeda48f; llc SHA-256 ca7af83e237af640fe337faff2eff49055988921933186ab5535b38addeb7b49) lowered abs_bf16x2 to `abs.bf16x2 <register>, <register>;` for selected evidence target sm_80 +ptx70

The full gate passed under rustup and nix develop:

$ cargo run -p cuda-intrinsics-gen -- probe --all --skip-terminal --per-target
probed 2374 target configurations across all 1025 generated intrinsic routes

Negative controls rejected a zeroed commit and a missing table. An explicit --llc still passed with the zeroed lock in comparison mode.

$ cargo fmt --all -- --check
pass
$ cargo clippy -p cuda-intrinsics-gen --all-targets -- -D warnings
pass
$ cargo test -p cuda-intrinsics-gen
314 passed, 0 failed
$ cargo run -p cuda-intrinsics-gen -- check
generated intrinsic outputs are current
$ sha256sum intrinsics/catalog.json
20bc41c2ab93828b7f97b6ee90efccfc118317a590803216a3a045b1c2879889  intrinsics/catalog.json

Tests cover different bytes with the same identity, commit and version mismatches, comparison mode, missing and malformed lock metadata, parse_rustc_verbose, and preflight intrinsic context.

The catalog and evidence files did not change, so the catalog stamp script was not needed. aarch64 was not tested because no machine was available.

  • just check passes, run as one command inside nix develop (the shell the issue is about): exit 0 in 4m26s, all recipes including check-intrinsics and doc-check
  • cargo oxide run <example> passes — not applicable to this host-side generator change

Checklist

  • All commits signed off (git commit -s)
  • SPDX headers on new source files — no new files

The evidence probe refused to run unless the rust-toolchain `llc` hashed
to the SHA-256 recorded in the catalog. That hash names the wrong thing:
`llc` is a 281 KB driver, the NVPTX lowering lives in libLLVM.so, and
the bytes change whenever the loader path or RPATH does. nixpkgs rewrites
both, so every `nix develop` shell failed `just check`; rustup on aarch64
fails the same way. The compiler was identical in every case.

The identity the probe needs is the compiler source, which the rustc
commit-hash names on every platform and which tells nightlies apart
where the `LLVM version` line alone does not. The probe now compares the
`llc --version` line per catalog record plus the `rustc -vV` commit-hash
against a new required `[rust_toolchain] commit_hash` in
intrinsics/upstream.lock. The lock is the one place that already pins
what the evidence was built from, and rust-toolchain.toml pins one
channel for the whole repository, so the commit is a repository-level
fact rather than a per-evidence field; catalog.json and the evidence
files are unchanged.

Preserved: `backend_sha256` is still recorded and printed as provenance
on every probe line, now next to the rustc commit; `--llc PATH` comparison probes stay unchecked; the
emitted PTX still has to match the catalog's expected instruction on
every probe. A lock without the table, or with a commit that is not 40
lowercase hex digits, fails at load.

Verified with `probe --all --skip-terminal --per-target` under rustup
nightly-2026-08-28 (llc 2e5bc688...) and under `nix develop`
(llc ca7af83e..., same commit e457a7b0d): both probe 2374 target
configurations across all 1025 routes. A wrong lock commit and a
missing table both fail before the first probe. aarch64 was not run.

Closes NVIDIA#1211

Signed-off-by: Sam, Joochul Chang <[email protected]>
@nihalpasham nihalpasham added bug Something isn't working build-related Build scripts, toolchain discovery, or compilation setup labels Sep 6, 2026
@nihalpasham

Copy link
Copy Markdown
Collaborator

Reviewed and validated:

Merging.

@nihalpasham
nihalpasham merged commit ef48b17 into NVIDIA:main Sep 6, 2026
52 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working build-related Build scripts, toolchain discovery, or compilation setup

Projects

None yet

Development

Successfully merging this pull request may close these issues.

just check cannot pass inside the recommended nix develop shell: the probe pins llc by SHA-256, nixpkgs rewrites it

2 participants