[praveenperera]

The problem

Several unit tests write a small shell script as a stand-in for a tool (llc, opt, clang, tileiras, ptxas) and run it right away. Under parallel test threads they sometimes fail with Text file busy (os error 26). just check hit this twice in a row for me, once in ptx::tests::debug_builds_fail_when_opt_drops_the_debug_graph and once in tileiras_capability_follows_the_listed_bytecode_versions. Both passed on a rerun.

The test process holds the script open for writing while fs::write runs. If another test thread spawns a process in that window, the child inherits a copy of that file handle. O_CLOEXEC only closes it when the child execs, and until then the kernel refuses to execute a file that has a writer. Writing a staging file and renaming it, as write_debug_dropping_tool already does, does not help: the inherited handle refers to the same file, whatever its name.

The fix

The test writes the script to a staging file, and a cp child creates the tool from it. The only process that ever opens the tool for writing is cp, and the helper waits for it to exit before returning. Then it sets the mode on the tool, because cp applies the umask, and a chmod opens no handle.

• cuda-oxide-codegen/src/ptx.rs: new test helper write_tool. write_debug_dropping_tool and the llc stand-in in legacy_tool_warnings_survive_a_later_llc_failure now use it, replacing the staging-and-rename version.
• cargo-oxide/src/commands/tests.rs: the same helper, used by doctor_clang_fixture and the tileiras stand-ins.
• cuda-intrinsics-gen/src/probe.rs: write_fake_tool creates the tool the same way.
• ptx-schedule/src/campaign.rs: run_binary_drains_output_larger_than_a_pipe_buffer creates spam.sh the same way.

The helper is copied rather than shared because there is no common test-support crate for these four.

I left out three other tests that write and then execute a script. In cuda-oxide-codegen/tests/concurrent_compile.rs, the binary has a single test, which writes its tool before starting any threads. cuda-oxide-codegen/tests/compile_to_ptx.rs already retries until its tool starts. cuda-artifact-finalizer already retries ETXTBSY when it runs the tool.

Testing

All runs were on a 24-core machine with Ubuntu 26.04, Linux 7.0 and GNU coreutils 9.7.

• Standalone repro: 16 threads each wrote and executed 2,000 scripts, while 8 more threads spawned true in a loop. That is 32,000 executions per method:

  How the tool is created	ETXTBSY
  fs::write + chmod (most tests before this change)	475
  staging file + rename (write_debug_dropping_tool before this change)	447
  staging file + cp (this change)	0

  On one failure, the repro scanned /proc and caught the writer: the repro's own forked child, not yet exec'd, holding the script open O_WRONLY|O_CLOEXEC.

• The four test binaries: I ran each binary directly in 4 parallel copies with every core busy. Each failure listed below is a test whose fake tool could not run.

  Test binary	main	This branch
  cuda-oxide-codegen lib	1 of 200 runs failed (debug_builds_fail_when_opt_drops_the_debug_graph)	0 of 600
  cargo-oxide bin	1 of 400 (doctor_clang_resource_prefers_the_first_successful_candidate)	0 of 600
  cuda-intrinsics-gen bin	1 of 200 (fake_non_nvidia_ptxas_banner_is_rejected)	0 of 600
  ptx-schedule lib	0 of 200	0 of 600

  The cargo-oxide runs skip three tests that drive cargo and need the environment that cargo test provides; the doctor test there falls through to its second fake clang when the first cannot run. At idle, 40 sequential cargo test runs per binary did not fail on main.

• just check passes.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.