Skip to content

fix(nix): advance the rust-overlay pin to the toolchain's nightly, and gate the flake in CI - #1210

Closed
almightychang wants to merge 2 commits into
NVIDIA:mainfrom
almightychang:flake-lock-nightly
Closed

almightychang wants to merge 2 commits into
NVIDIA:mainfrom
almightychang:flake-lock-nightly

Conversation

@almightychang

@almightychang almightychang commented Aug 30, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Fixes #1209. Every flake output fails to evaluate on main: flake.nix derives its toolchain from rust-toolchain.toml, but fe616639d (#1201) moved the pin to nightly-2026-08-28 without updating the 2026-05-09 rust-overlay lock revision. That revision predates the nightly, so both the dev shell CONTRIBUTING.md:123 recommends and the cargo-oxide package fail with error: Nightly 2026-08-28 is not available.

This advances only rust-overlay and adds the gate that would have caught the drift.

Changes

  • flake.lock: advance rust-overlay from 1d634a90 (2026-05-09) to 89e26eea (2026-08-30). No other node moves; nixpkgs, CUDA, LLVM, and support packages remain pinned. Advancing that one input is sufficient; a whole-lock bump is not needed.
  • .github/workflows/flake.yml: evaluate both public outputs without building the toolchain. No existing workflow evaluated the flake.
  • Justfile: mirror the gate as check-flake, outside check and check-guards so contributors without Nix can still run them.

Why a flake evaluation and not an extension of check-toolchain-parity.sh

check-toolchain-parity.sh compares copies of the pin text: the nested rustc-codegen-cuda pin, the cargo oxide new scaffold, the devcontainer, and the book's quoted [toolchain] blocks. flake.lock is not another copy; its overlay must resolve the pin. Comparing lastModified with the nightly date could pass when the dates align even if the flake does not work. Evaluating both output derivations tests the property that broke and builds no toolchain; the cost is installing Nix and fetching inputs, with evaluation taking about a second once cached.

Testing

Before, on 8b41ac1b5:

$ nix eval .#devShells.x86_64-linux.default.drvPath
error: Nightly 2026-08-28 is not available
$ nix eval .#packages.x86_64-linux.cargo-oxide.drvPath
error: Nightly 2026-08-28 is not available

After, on this branch (just check-flake; the store hashes depend on the source tree, the point is that both resolve):

$ nix eval .#devShells.x86_64-linux.default.drvPath
"/nix/store/0jbdhnhg28fmaa7x0l5gf5mly8n7yra4-nix-shell.drv"
$ nix eval .#packages.x86_64-linux.cargo-oxide.drvPath
"/nix/store/b9xq9d7w8yqd9cshkc9xkl8j8risqam9-cargo-oxide-0.2.1.drv"

The shell also supplies the pinned rustc 1.100 / LLVM 23 toolchain and CUDA, and completes a workspace check:

$ nix develop --command rustc --version --verbose
rustc 1.100.0-nightly (e457a7b0d 2026-08-27)
commit-date: 2026-08-27
release: 1.100.0-nightly
LLVM version: 23.1.0

$ nix develop --command cargo --version
cargo 1.100.0-nightly (e8cb624d5 2026-08-22)

$ nix develop --command bash -c 'nvcc --version | tail -2'
Cuda compilation tools, release 13.2, V13.2.51

$ nix develop --command cargo check --workspace --all-targets
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 21.00s

just check runs in the restored shell and gets through fmt-check, clippy, test, test-cuda, and all of check-guards. It stops in check-intrinsics:

$ nix develop --command just check
error: validate probe backend for abs_bf16x2: rust-toolchain llc SHA-256 mismatch:
selected evidence records 2e5bc688...f921, found ca7af83e...7b49

That is not this change and not a missing llc. intrinsics/catalog.json pins the probe backend by binary hash, and the recorded 2e5bc688... is byte-for-byte the llc that rustup ships for nightly-2026-08-28. The shell's llc is the same nightly and reports the same LLVM version 23.1.0-rust-1.100.0-nightly, but nixpkgs rewrites its ELF interpreter and RPATH into the store, so its hash cannot match. Any nix develop shell hits this, before and after this PR. It does mean the environment CONTRIBUTING.md recommends cannot run the repo's own probe validation; that is a separate problem and I have not touched it here.

One standard example end to end in the same shell, on an RTX 5080:

$ nix develop --command cargo oxide run vecadd
Detected GPU arch: sm_120a (via nvidia-smi)
    Finished `release` profile [optimized] target(s) in 10.79s
     Running `target/release/vecadd`
✓ SUCCESS: All 1024 elements correct!

The new workflow has never run on GitHub Actions; this PR's runs are action_required pending maintainer approval. Locally, its two commands pass through just check-flake. cachix/install-nix-action enables nix-command flakes by default, so no extra_nix_config is needed. The action uses the exact v31.11.1 release because no floating v31 tag exists, matching the Jimver/[email protected] precedent.

  • just check passes (the local mirror of CI: fmt, clippy, tests, guards, docs) (fmt, clippy, tests, CUDA tests and guards pass; check-intrinsics cannot pass inside any nix develop shell, for the pre-existing reason above)
  • cargo oxide run <example> passes, or scripts/smoketest.sh -o '^<example>$' (vecadd, in the restored shell, on hardware)
  • New example added (not applicable)

Checklist

  • All commits signed off (git commit -s)
  • SPDX headers on new source files (the new file is a workflow; workflows here carry none)

Out of scope, noted for the record

flake.nix:85 still selects pkgs.llvmPackages_22 while the pin uses rustc 1.100 with LLVM 23. It does not block evaluation and remains separate from this lock fix; #1209 records it.

`flake.nix` derives the toolchain from the checked-in pin with
`fromRustupToolchainFile ./rust-toolchain.toml`, so the lock's
`rust-overlay` revision has to know every nightly that file names.
`fe616639d` moved the pin to `nightly-2026-08-28` without touching the
lock, leaving `rust-overlay` at a 2026-05-09 revision that predates it.
Every flake output then failed to evaluate with `Nightly 2026-08-28 is
not available` -- the dev shell `CONTRIBUTING.md` recommends and the
`cargo-oxide` package alike.

Only the `rust-overlay` node moves. `nixpkgs` and every other input stay
at their current revisions, so the shell's CUDA, LLVM, and support
packages are unchanged and the fix is confined to the input that went
stale.

Verified on the updated lock: both `devShells.x86_64-linux.default` and
`packages.x86_64-linux.cargo-oxide` evaluate, and `nix develop` reports
rustc 1.100.0-nightly (e457a7b0d 2026-08-27) with LLVM 23.1.0 alongside
CUDA 13.2.51 -- the toolchain `fe616639d` intended to pin.

Signed-off-by: Sam <[email protected]>
The preceding commit fixed a lock that had fallen four months behind the
nightly `rust-toolchain.toml` names. Nothing would have caught it: no
workflow mentions Nix, so no job evaluates the flake.

`scripts/check-toolchain-parity.sh` is the closest existing guard, and it
is the wrong place for this. It compares copies of the pin's *text* --
the nested pin, the scaffold, the devcontainer, the book's quoted
blocks -- while `flake.lock` instead has to be able to *resolve* the pin.
The only comparison a text guard could make is the overlay's
`lastModified` against the nightly's date, a proxy that passes whenever
the dates happen to line up regardless of whether the flake works.

Evaluating the flake checks the property that actually broke. It is also
cheap: forcing the two output derivations builds no toolchain and takes
about a second once the inputs are fetched, so the job's cost is
installing Nix and fetching inputs. Both public outputs are covered,
because the failure hit the dev shell `CONTRIBUTING.md` recommends and
the `cargo-oxide` package alike.

`check-flake` mirrors the gate locally but stays out of `check` and
`check-guards`, which must keep running for contributors without Nix.
The `check-guards` comment records that so the omission reads as a
decision rather than the drift that comment exists to prevent.

Signed-off-by: Sam <[email protected]>
@nihalpasham

Copy link
Copy Markdown
Collaborator

Right diagnosis in #1209, and half of this is already in: main landed the same rust-overlay advance as 5944e11 shortly before this PR went up, which is why the branch now conflicts. #1209 is closed against that commit.

Closing the PR because the remaining half, the CI gate, is a trade we don't want:

what can break the flake:  rust-toolchain.toml moves    (a few times a year, by us)
what the gate costs:       Nix install + input fetch    (every PR, every main push;
                                                         flake.yml has no path filter)
  • the only writer of the risk is the maintainer doing a toolchain bump, so the guard belongs in the bump procedure (re-pin rust-overlay, evaluate both outputs), not in every contributor's CI run
  • even path-filtered to rust-toolchain.toml/flake.*, it would put a new toolchain and a network dependency on nixpkgs/rust-overlay into the CI fleet for what is a contributor-convenience surface, and a flaky fetch there fails PRs that never touched Nix
  • your "why not extend check-toolchain-parity" argument is correct as far as it goes (the lock isn't a text copy, it has to resolve), which is why we're not doing a date-comparison guard either; the eval you wrote is the right check, just run at bump time, and we're adding exactly that to the toolchain-bump checklist

Appreciate the tight scoping of the lock bump itself (one input, no whole-lock churn); that matched what we landed independently.

@nihalpasham nihalpasham added already-fixed Resolved on a newer commit or branch; no action needed declined PR intentionally not merged (closed without merging) build-related Build scripts, toolchain discovery, or compilation setup labels Aug 30, 2026
@almightychang

almightychang commented Aug 30, 2026 •

Copy link
Copy Markdown
Contributor Author

@nihalpasham
Agreed that bump time is the right place. Since #1201 was itself a bump that missed the lock, how would you make that check durable beyond a checklist entry?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

already-fixed Resolved on a newer commit or branch; no action needed build-related Build scripts, toolchain discovery, or compilation setup declined PR intentionally not merged (closed without merging)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

nix develop and nix build fail on main: flake.lock's rust-overlay predates the pinned nightly

2 participants