Skip to content

Prepare sm_86 Ampere encoder, cubin and discrete-memory runtime - #170

Merged
aien-spark merged 25 commits into
mainfrom
feat/ampere-sm86-port
Oct 31, 2026
Merged

aien-spark merged 25 commits into
mainfrom
feat/ampere-sm86-port

Conversation

@aien-spark

@aien-spark aien-spark commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

Recovered host-only Ampere port

Final source: 3b6ca6fda726a6b2eb01100ae155fbcb890d00af. Draft, not merged or deployed. Recovery found the __truncsfhf2 source repair already complete at 592cee255eaba4b86c5a823ed071c2773a3d4f13; it did not replace that Omega implementation with libgcc. This follow-on fixes CodeQL62 in the offline cubin fixture writer by creating new files with mode0600, closes the descriptor if fdopen fails, and reconciles the documentation's stale acceptance inventory. Existing file permissions are intentionally unchanged; this is a new-file creation repair, not a broader overwrite/symlink policy change.

Dependencies and production boundary

  • Exact omega-bridge36 source: ead98302ff0a649c49c3f2c7568af5e1b3da33eb, pinned by om/omega-bridge.lock. All40 roots:37 bounded hosted compositions and import-free revision/epoch/f16 scalar units. No executable-code trampoline.
  • Binary32-to-binary16 RNE implementation is om/numeric/f16.om, emitted through bridge-freest; the C caller bitcasts and uses its checked scalar ABI. The hosted _Float16 cast is a verification oracle only. Production scalar object has no undefined symbols; no __truncsfhf2 or libgcc production dependency was added.
  • Compiler 2c82480596bbdc50cda9626866209ef662d5834f278f839ff4ce0250c495029f at omega-mv e5b32567a7aff3ce594e3254a622a846a6cd190b; baseline 191b776ef98821597c3e30655da6d680a9b66edbc9ceaca3c39c61ea6df098e9 remains required. Converter/frontend hashes are checked before installation. All5 independently corrupted-pin controls refused exit3 with zero installed outputs.
  • Bridge36 behavior CI40758894095 remains PASS at its exact source. Native x86 Omega compiler923 and bridge37 are separate, not prerequisites. Guide114 is separately owned; no guide source or pin was changed by this recovery.

Final host behavior and controls

Clean detached source3b6ca6fd, native AArch64 host:
OMEGA_SEED=<191b baseline> OMEGA_OMC=<2c824 seeded compiler> OMEGA_MV=<e5b32567 tree> OMEGA_BRIDGE=<ead98302 tree> OMEGA_BRIDGE_DIR=<ead98302 tree> make -j2 all test test-native-artifact zero-cuda-gate ampere-device-build test-ampere-cubin-disasm

Exit0; CUDA++ HOST TESTS PASS (58 targets). Byte-identical40-root regeneration; F16 boundaries/native stepped sweep/six mutant controls; unchanged freestanding import allowlist and canary refusal; Ampere cubin/numeric, simulated RM/session ordering/VRAM/target refusals; native artifacts, zero-CUDA probe rejection and offline cubin decoding passed. Device-build is build-only, not device execution.

Fixture security controls: fresh output under umask000 is0600; historical592cee writer produces0666 with identical cubin bytes and fails the private-creation check; preexisting0666 output stays0666 with identical bytes; nonexistent-parent output refuses exit1 without creating files; no-output invocation succeeds without creating a fixture. Current-head CodeQL check123126643103 SUCCESS: no new alerts in code changed by this pull request.

An initial attempt in the recovered x86pre tree failed because preexisting x86 build/test_ampere_runtime was reused by a hardcoded build/ target despite OUT_DIR; on ARM its dynamic loader was absent. Those artifacts were preserved. The clean detached final tree above passed; the contaminated attempt is not reported as PASS.

Native x86 consumer acceptance

Current-head native x86 workflow40776306246 SUCCESS, completed2026-10-28T22:33:33Z: https://github.com/aien-dev/CUDApp/actions/runs/40776306246. Candidate source 3b6ca6fda726a6b2eb01100ae155fbcb890d00af; actual checked-out PR merge cf65a61c8b95f7d0d7b51c5618a9dc87bcbb8cf4 onto base c6977b4906229fe83f41103b7eff65b64a0986d6. Both provenance-baseline and native-x86-host jobs passed. Three consumer ELF headers are x86-64;58 host targets,40-root regeneration, F16 boundary/sweep/mutant controls, native artifacts and zero-CUDA passed. The unchanged17-file freestanding link exposes only memcmp/memcpy/memset/strcmp/strlen, not a half-conversion helper. Final offline oracle:2169 kernels, every opcode/operand matched, mutated operand refused;73 native-FP8 kernels refused. Compiler/converter QEMU execution remains EMULATED provenance; the x86 compiled-C consumers actually ran natively. Handler teardown passed. Physical RTX qualification remains NOT_RUN.
Prior source592cee run40762515409 SUCCESS checked out PR merge 6a2b759333ce791663d0107949659c7a3cd887bb: three x86-64 ELF headers,58 host targets,40-root regeneration and2169-kernel exact SM86 offline oracle. Current-head outcome is not inferred from that earlier run. AArch64 compiler/converter execution under pinned QEMU is EMULATED provenance; compiled-C x86 consumers are NATIVE. Neither establishes a native x86 Omega compiler result.

Scope and physical NOT_RUN

Separate SM121 MMIO work remains separate; this recovery did not touch176/173/174/175, bridge37 or CodexIntegration. No gate was weakened and historical receipts remain unchanged.

RTX3090 and RTX3060Ti execution, loader acceptance, BAR/coherence, full device numerical math and safe VRAM-refusal device proof remain NOT_RUN. They require the actual card host, pinned580.173.02 open module, unambiguous PCI BDF and current explicit GPU authorization. Follow tests/ampere/README.md and doctor/run.py; cached mappings refuse. Host simulation and offline disassembly are not physical qualification. No GPU, driver, host binfmt configuration or live production service was touched.

Independent review (Main desk, 2026-10-31)

Adversarial review by Cudapp170b and C170c (agent lanes), plus a read-only second-agent review by Rev1001. Rev1001 gave NO_WEAKENING on efaf8d7. It also gave NO_WEAKENING on the final diff 570b5d6 (fixes for its notes a and b, plus the loop-scan follow-up).

Decisions taken while merging main:

Fixes from Rev1001's review (commit 570b5d6):

  • (b) Uniform destination rule. The encoder writes a uniform register only for LDCU/LDCU64; on IMAD/ISETP, is_uniform marks a uniform source. Three places still read any is_uniform dst as uniform, and now all three use the encoder's predicate:

    • omega_bw_ir_append, the allocator's live intervals;
    • check_loop_invariant in omega_gpu_attention_api.c;
    • check_loop_invariant in omega_gpu_elementwise_api.c.

    Regression test test_uniform_source_dst (tests/bw_hazard_test.c) builds a uniform-source IMAD whose GPR destination is read later.

    • On efaf8d7 source: bw_hazard_test: 264 checks, 3 failed: FAIL.
      • The dst interval starts at IR 4, not 2.
      • The dst reuses R2, the GPR of a value still live at IR 3.
      • The uniform interval of the same index is polluted, with first def 2 instead of 5.
    • With the fix: 264 checks, 0 failed: PASS.
    • Apart from those lines, the sweep output is identical before and after (2360 distinct kernels, 189 findings). Only LDCU64 builders set is_uniform in src/, so production kernels are unchanged.
  • (a) tools/test_bw_spec_encoder.sh: a skipped AArch64 native probe now exits 3 (NOT_RUN, the script's existing code) instead of 0, unless BW_SPEC_NATIVE_PROBE_NOT_RUN_OK=1. make test-om-spec-encoder sets that variable only on non-AArch64 hosts, and the log then names the accepted NOT_RUN row. Exercised runs:

    • AArch64: full probe, rc 0.
    • Fake x86 CC, no acknowledgement: rc 3.
    • Fake x86 CC with acknowledgement: rc 0, NOT_RUN line printed.

Local checks (AArch64, cores 6-9,15-19 at nice 19):

  • On 570b5d6: make -j2 test rc 0, CUDA++ HOST TESTS PASS (74 targets); spec encoder full PASS; crumb verify OK, 35 crumbs.
  • After merging main 105d65d (head 19d7dc8): tests/gates/cudapp-run.sh runtime-api rc 0, with both RESULT: PASS lines and every negative control caught; make test-runtime-api-semver PASS (0.8.0 -> 0.9.0); crumb verify OK.

Trailer disclosure: this PR has 9 Signed-off-by: Drake Stapleton <[email protected]> trailers, on commits a80179b, 8cce4a6, 2e6bf8c, b026505, 1147b0c, fd9f23b, ae65cd6, 83c2a8a and dbb1b50. Their author and committer are the Drake identity, which was not verified. [INFERENCE] The AmperePort agent lane wrote them. The PR merges with a merge commit, so the trailers stay as written.

NOT_RUN:

  • GB10 silicon for the sm_121 encoder/allocator uniform-destination change. Production bytes are expected to be unchanged, but that is not chip evidence. The command is queued as item G in the batched window queue: make hello-gpu && ./build/gpu_elementwise_test --out ... && ./build/gpu_attention_test --out ....
  • RTX3090/3060Ti device execution, as above.
  • The native AArch64 function-byte probe on x86 CI, which is reported as NOT_RUN.

Drift fix (99ec266): CI drift went red against omega 2638a49b because tests/bw_hazard_test.c (a byte copy of omega 6d7b8838, no baseline row) now carries the sm_86 cases. Added baseline row = omega blob e052b090 plus comment; self-test PASS, a scratch omega edit to the file still FAILs. Rev1001 NO_WEAKENING. Note: commit 99ec266 carries author identity "x " (pushed; not rewritten, no force-push).

Comment thread tests/ampere_cubin_test.c Fixed
Comment thread tools/ampere_corpus.c Fixed
Comment thread tools/ampere_corpus.c Fixed
# Conflicts:
#	Makefile
#	mk/om-encoder.mk
#	mk/omega-gpu-engine.mk
#	om/omega-bridge.lock
#	om/units.sh
#	src/cudapp_native.h
#	src/cudapp_om_units.c
#	src/cudapp_om_units.ident
#	src/omega_gpu_session.c
Reproduces merged166/169 legal staged U3 trap before source fix; preserve capacity refusal and immutable historical receipts. Add independent guard-removal mutants, final-slot/HMMA/root controls, and the actual nv_staged recomposition seam. NativeIntegrate actual generated-caller regression precedes this commit. All first-party production fix code remains seeded Omega; full37-root bindings regenerated with existing pinned compiler and bridge.

Signed-off-by: Drake Stapleton <[email protected]>
… behavior fixtures

- om/numeric/f16.om: native_f16(bits ulong) to ulong implementing IEEE 754-2008
  binary32 to binary16 RNE scalar integer conversion. Preserves signed zero,
  IEEE subnormals with RNE tie-breaking, overflow to signed infinity, and
  quiet/signaling NaNs with payload preservation. No dynamic shifts or div/rem.
- tests/ampere/f16_fixture.om: pure Omega behavior fixture covering all boundary
  conditions and tie-breaking rules.
- tests/ampere/test_f16_behavior.c: C behavior fixture running boundary table,
  6 negative mutant controls, and 44M sweep points against native _Float16
  reference on ARM host and x86_64 under QEMU.
- src/cudapp_freest_f16.c: generated freestanding C via bridge-freest emitting
  omega_native_f16(uint64_t *result, uint64_t p0).

@drakestapleton drakestapleton left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Classification: blocked (needs on-hardware evidence).

  • RTX 3090 / 3060 Ti execution, loader acceptance, BAR/coherence, device numerics and VRAM-refusal proof are all NOT_RUN. Host simulation and offline nvdisasm are not physical qualification.
  • CodeQL comments on tools/ampere_corpus.c:17 (file created 0666; fopen path derived from an environment variable) still sit on head 3b6ca6f. The CodeQL check is green, so either confirm they are fixed or dismiss them with a reason.
  • GitHub has not computed mergeability for this PR (state unknown).
  • The body says "Draft, not merged", but the PR is not marked draft; please align.
  • Very large (180 files, +69k). Consider splitting encoder / cubin / runtime.
  • To become merge-ready: physical RTX receipts under evidence/, the CodeQL items resolved, mergeability confirmed.

— Grokbot

@drakestapleton drakestapleton left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up: ADR 0043 (now on aien-architecture-mv main, docs/adr/0043-gpu-stack-portable-top-thin-backend.md, Status: Accepted 2026-10-30, merged via #343) affects this PR.

What the ADR says:

  • Decision 1: thin backends (driver client, command encoding/QMD, code-object writer, per-target lowering and machine-code encoder) are ".om for every new backend" and live in CUDApp backend/<target>/. Only the existing GB10 C files stay where they are in src/, nvrm/, m16/ as migration.
  • Context / PLAN rule: "any new C needs a missing-feature reason with an extension proposal filed, behind a migration-pending build switch."
  • Decision 0: no P2–P4 code until P0's head-to-head is published and C2–C5 are QUALIFIED on the GB10; until then only design docs, extension proposals and host-only experiments "that touch no shared file".

Where #170 conflicts (sm_86 is a new backend):

  • New hand-written C for the sm_86 target: src/omega_ampere_codegen.{c,h}, src/omega_ampere_qmd.{c,h}, tools/ampere_corpus.c, tools/ampere_families.c, plus new C tests (tests/ampere/*.c, tests/test_ampere_*.c, tests/ampere_cubin_test.c). None of it is in backend/sm86/, and I don't see a migration-pending switch or extension-proposal reference for it.
  • The sm_86 encoder/runtime .om units are in om/encoder/ (ampere_enc_sm86.om, amp_runtime.om) rather than backend/<target>/.
  • Sm_86 code-object writing is added to shared GB10 files (src/omega_cubin.c +67, src/omega_cubin.h +21, om/cubin/cubin_elf.om +48), along with shared runtime/driver files (src/omega_gpu_session.c, src/omega_gpu_platform_linux.c, nvrm/nvrm.c +170). Decision 0 allows only experiments that touch no shared file before the gate.

Suggestion: rebase onto the ADR layout: sm_86 encoder, QMD and code-object writer as .om under backend/sm86/ (or the target name the ADR's ABI uses), keep shared GB10 files unchanged, and put any remaining C behind migration-pending with its missing-feature reason. Alternatively, state how this PR fits the Decision 0 gate.

Classification unchanged: blocked (now also on ADR 0043 conformance, in addition to RTX hardware evidence).

— Grokbot

@drakestapleton drakestapleton left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up: ADR 0044 is now on aien-architecture-mv main at docs/adr/0044-one-cubin-writer-per-nvidia-target-family.md (Status: "Accepted (Primate, 2026-10-30).", merged via #357, e5262c3). It names this PR directly: "CUDApp#170 rebases onto this layout." (The ADR 0043 points are in my earlier review.)

What ADR 0044 says:

  • Ownership: "Option B. One cubin writer per NVIDIA target family, owned by CUDApp." / "CUDApp owns all cubin writing, the sm_80 family included, under backend/<target>/. The NVIDIA backend/<target>/ directory does not exist on CUDApp yet. It is created when the work moves. New writer work goes only into .om."
  • om/cubin/cubin_elf.om: "B. cubin_elf.om is canonical." Context: "The hosted build packs through om/cubin/cubin_elf.om."
  • src/omega_cubin.c: "omega_cubin.c is frozen, and it is retired only after the .om writer produces byte-identical sm_121 cubins and loads on GB10. That removal is a later, separate PR." Context: "The C body is slated for deletion."

Mapping to this diff (head 3b6ca6f):

  • src/omega_cubin.c (+67) and src/omega_cubin.h (+21): this is new writer work in the frozen C body. To comply, drop these changes and leave both files exactly as on main.
  • om/cubin/cubin_elf.om (+48): .om is the right language and this is the canonical writer. sm_86 is in the sm_80 family, though, and the ADR puts cubin writing for that family under backend/<target>/. Either move the sm_86 writer work into the new backend/<target>/ directory or explain why it extends om/cubin/ in place. In both cases, keep the sm_121 output byte-identical.
  • tests/ampere_cubin_test.c and any other C that writes or patches sm_86 cubins: per "New writer work goes only into .om", the writing logic must be .om. A C test harness that only checks the output can stay if it writes nothing itself.
  • If this PR ports anything from omega-mv#864 (ABI 8 or register-reserve findings), the ADR says it comes into CUDApp "as new work". Please note the provenance in the PR body.

Hardware: the ADR keeps hardware tiers "NOT_RUN until an authorized window". The RTX 3090/3060Ti execution here is still NOT_RUN. Classification unchanged: blocked.

— Grokbot

aien-spark and others added 6 commits October 31, 2026 07:27
# Conflicts:
#	.github/workflows/ci.yml
#	Makefile
#	docs/runtime/API.md
#	docs/runtime/PORTABILITY.md
#	m16/m16_concurrent.c
#	m16/m16_native.c
#	mk/omega-gpu-engine.mk
#	nvrm/nvrm.c
#	om/units.sh
#	src/cudapp_om_units.c
#	src/cudapp_om_units.ident
#	src/omega_blackwell_engine.c
#	src/omega_blackwell_matmul.c
#	src/omega_blackwell_submit.c
#	src/omega_gpu_elementwise_api.c
#	src/omega_gpu_platform_linux.c
#	tests/moe/.crumb
#	tests/moe/run.sh
#	tests/test_omega_gpu_platform_linux.c
#	tools/extract/drift-baseline.tsv
#	tools/extract/drift.sh
#	tools/gpu_engine_backend_mutations.sh
# Conflicts:
#	docs/runtime/API.md
#	tools/extract/.crumb
Main's #984 added bw_hazard_test to sm121dis-corpus-bins, so the Ampere
oracle corpus grew from 773 to 4679 pairs (3131 distinct) and the serial
two-nvdisasm-calls-per-pair loop overran native-x86-host's 40-minute
budget (run 41202617380 cancelled inside tools/ampere_nvdisasm.py).
compare() is pure in (reference, code, ops): identical triples are
decoded once and distinct ones run on a thread pool sized to the CPUs.
Every pair is still checked and reported by name; the mutation controls
are unchanged.
… NOT_RUN exits 3

The encoder writes a uniform register only for LDCU/LDCU64 (is_uniform on
IMAD/ISETP marks a uniform source). omega_bw_ir_append and the attention and
elementwise loop-invariant scans still read any is_uniform dst as uniform.
They now use the encoder's rule. tests/bw_hazard_test.c
test_uniform_source_dst fails 3 checks without the allocator change (GPR dst
interval started at its first read, reused a live GPR, polluted the uniform
interval of the same index) and passes with it. No production builder sets
is_uniform on a non-LDCU op, so production kernels are unchanged (hazard sweep
output identical).

tools/test_bw_spec_encoder.sh: a skipped AArch64 native probe exits 3
(NOT_RUN) unless BW_SPEC_NATIVE_PROBE_NOT_RUN_OK=1; make test-om-spec-encoder
sets it only on non-AArch64 hosts.
…++ is ahead with the sm_86 cases

The file was a byte copy of omega with no baseline row; #170's sm_86
uniform-destination cases made it differ, which drift read as omega drift.
Self-test vs omega 2638a49b PASS; a scratch omega edit to the file still fails.
@aien-spark
aien-spark merged commit be8aea4 into main Oct 31, 2026
20 checks passed
aien-spark added a commit that referenced this pull request Oct 31, 2026
Clears the CUDA++ drift gate after omega-mv #1006 (main a1667d594929).

- `src/omega_blackwell_codegen.c`: omega delta applied (OMEGA_KERNEL_DUMP_DIR copied through `omega_path_copy`; refusal prints `omega_kernel_dump: ... refused`, which `tools/sm121dis/corpus.sh` fails on).
- `src/omega_path_check.h`: byte copy of omega's.
- `tests/gpu_attention_test.c` (already 0600 via `test_out_file.h`) and `tests/gpu_tern_test.c` (omega-only): baseline rows only.

Evidence (Spark host, no GPU):
- before: `tools/extract/drift.sh --self-test <omega@a1667d59> a1667d59` → DRIFT-GATE FAIL (3 paths), SELF-TEST FAIL.
- after: DRIFT-GATE PASS (790 paths), SELF-TEST PASS (both controls red).
- `build/gpu_attention_test --sim`: dump dir `/tmp/cq39kd` → rc 0, 219 checks 0 failed, 45 kernels dumped; negative control dir `/tmp/cq39 kd` (space) → 79 `omega_kernel_dump:` refusal lines, 0 files written (sim checks still pass; corpus.sh is the gate that fails on the diagnostic).

NOT_RUN: chip runs; full host gate suite.

Update: merged CUDApp main be8aea4 (#170); Cq340 added f13d736 (baseline rows for tests/gpu_bf16_gemv_test.c and tests/gpu_decode_attn_test.c at omega c77d3ddf, omega-mv #1010; carries CUDApp#243). Drift `--self-test` at head f13d736 vs omega main 57a4135d: DRIFT-GATE PASS (891 paths), SELF-TEST PASS (all controls red).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants