Skip to content

27B: exact GPU head and reusable command seals - #251

Open
aien-spark wants to merge 4 commits into
mainfrom
perf/q38-head-bounds
Open

aien-spark wants to merge 4 commits into
mainfrom
perf/q38-head-bounds

Conversation

@aien-spark

@aien-spark aien-spark commented Nov 1, 2026 •

Copy link
Copy Markdown
Collaborator

The direct compact 27B output head changes a greedy choice in the longer reference. This change uses compact scores only to screen candidates, then recomputes every possible winner with the original BF16 head. Omega owns the conservative bounds, fixed normalization envelope, memory layout, and GPU schedule. The exact head remains opt-in through Core #895; sampled requests use all original scores.

The current native FP8 path also recomputes a SHA256 of the command image on every launch. An Omega memo reuses that digest only after the argument count and every normalized image byte match. Changed bytes get the original hash. The C adapter stores the optional per-slot memo and preserves ownership, epoch, revision, completion and intent checks. Allocation failure uses the uncached path. The narrow GB10 admission repair accepts the observed raw identity only with the existing qualified PCI identity and class set.

Validation:

  • Exact-head bounds, schedule, emitted code, memory reserve, 1309 envelope cases and five envelope mutants pass; 90 normalization stress cases and eight real-state GPU replays pass. Rechecked scores are bit-identical. The underestimated-bound and zero-coefficient controls fail as intended.
  • Seal reuse passes 6304 independent SHA256 cases, changes to every input byte, and five compiled Omega mutants. All 27 native ownership/lifetime host cases pass. Portable bindings reproduce byte for byte.
  • The combined private server passes conversation continuation, three 256-token reference runs, exact streaming text and token IDs, repeated coding/prose checks, and seeded sampling equality with the full head. The coding answer passes seven cases. The private server exits cleanly.

Shared-machine diagnostic: recording falls from 75.606 to 9.925 ms/token in the direct full-head probe. The repeated 256-token serving requests fall from about 48.6 to 32.8 seconds after seal reuse, with identical outputs. First visible streaming output remains about 0.5 seconds. These are not five alternating quiet-machine pairs and do not establish an improvement against the older frozen server. No accepted TTFT improvement or deployment is claimed.

The generated integer seal unit now builds at O3; the archive comparison confirms every other object is unchanged. Its 6304-case oracle, five mutants and 27 native-runtime cases pass at that setting. An additional server check passes exact128, sampled32 and exact128 after sampling. The paired host measurements and separate serving screen are recorded in the build receipt. The server screen is shared-machine diagnostic data, with no accepted TTFT gain.

The full local O2 host suite passes all 76 targets on the exact-head/seal-memo source (code39160652, receipt009ada76). The later isolated O3 setting has the separate oracle, native-runtime and serving coverage above. All 20 current-head hosted checks now pass, including the full host suites. The later adaptive descendant also passes all 81 local host targets; the exact-head, seal, and admission source directories/files are unchanged between these revisions. This is ready for code review. Controlled performance qualification remains NOT_RUN; these measurements do not authorize a default change or establish a qualified TTFT gain. The scoped receipt is here.

The direct compact 27B output head changes a greedy choice in the longer reference. This change uses compact scores only to screen candidates, then recomputes every possible winner with the original BF16 head. Omega owns the conservative bounds, fixed normalization envelope, memory layout, and GPU schedule. The exact head remains opt-in through Core #895. Sampled requests must call omega_gpu_q38_graph_set_sampling(g, 1), which refuses while the exact head is on. omega_gpu_q38_graph_read refuses a logits pointer when sampling is on and the last token wrote the masked buffer.

The current native FP8 path also recomputes a SHA256 of the command image on every launch. An Omega memo reuses that digest only after the argument count and every normalized image byte match. Changed bytes get the original hash. The C adapter stores the optional per-slot memo and preserves ownership, epoch, revision, completion and intent checks. Allocation failure uses the uncached path. make test-seal-reuse requires omega_native_mcreseal to match omega_native_mcseal on the same bytes, on a miss and on a hit, in addition to the Python SHA256 oracle. The image layout in seal_reuse.om is still a copy of mcc_seal; the differential test is the gate. Sharing the normalizer in one Omega function is not in this PR.

origin/main is merged (3676340). nvrm/nvrm.c, om/encoder/amp_runtime.om, src/cudapp_om_amp_runtime.c, src/cudapp_om_amp_runtime.ident, and tests/test_ampere_runtime.c match main and are no longer in the diff. SM 0xa04 admission is #246, already on main.

Validation:

  • Exact-head bounds, schedule, emitted code, memory reserve, 1309 envelope cases and five envelope mutants pass; 90 normalization stress cases and eight real-state GPU replays pass. Rechecked scores are bit-identical. The underestimated-bound and zero-coefficient controls fail as intended.
  • Seal reuse passes 6304 independent SHA256 cases, and the same 6304 cases match omega_native_mcseal (local make test-seal-reuse on 6e2adc7). Five compiled Omega mutants still run where the pinned compiler is present. All 27 native ownership/lifetime host cases pass. Portable bindings reproduce byte for byte.
  • The combined private server passes conversation continuation, three 256-token reference runs, exact streaming text and token IDs, repeated coding/prose checks, and seeded sampling equality with the full head. The coding answer passes seven cases. The private server exits cleanly. Those serving runs are not a run of this head.

Shared-machine diagnostic: the 75.606 to 9.925 ms/token recording claim is NOT RUN / unverified. No receipt in this PR backs that pair. The repeated 256-token serving requests were described as falling from about 48.6 to 32.8 seconds after seal reuse, with identical outputs, and first visible streaming output as about 0.5 seconds. Those sentences are the same shared-machine notes: they are not five alternating quiet-machine pairs and do not establish an improvement against the older frozen server. No accepted TTFT improvement or deployment is claimed.

The generated integer seal unit builds at O3. An additional server check was reported for exact128, sampled32 and exact128 after sampling on an earlier head. The paired host measurements and separate serving screen were recorded in the build receipt at 6a93cb5, not at this head.

Head is f8b0a3e038fc956939c2717723750bc6b2a22fa6. 6e2adc7 is the sampling guard and the seal differential, on top of the main merge. f8b0a3e only adds the fresh-clone host receipt: evidence/Q38-FRESH-20261102/RECEIPT.md. Those gates ran on 6e2adc7: make test PASS (73 targets), make test-native-artifact PASS (47 checks), zero-cuda-gate PASS, hello_gpu --sim PASS, make doctor NOT READY (expected here, exit 2). GB10 chip and serving on this head are NOT RUN. om/q38-head/serving-validation.json is still the earlier O2 screen.

Controlled performance qualification remains NOT_RUN. These measurements do not authorize a default change or establish a qualified TTFT gain.

Open in Web Open in Cursor 

Sweep re-check (2026-11-04, DRA-130)

CI: only drift fails (https://github.com/aien-dev/CUDApp/actions/runs/41910510525/job/126856498671). The review's file-drop point is resolved: nvrm/nvrm.c, om/encoder/amp_runtime.om, src/cudapp_om_amp_runtime.{c,ident} and tests/test_ampere_runtime.c are identical to main. The diff is 10 files. Chip/perf at this head: NOT_RUN.

The drift FAIL (45 of ~920 GPU paths vs omega-mv d3d1b1bc7) is inherited from main. It clears when #266 merges.

@aien-spark aien-spark changed the title Add Omega GPU score bounds and repair GB10 identity admission Experiment: exact 27B head with Omega bounds and GPU schedule Nov 1, 2026
@aien-spark aien-spark changed the title Experiment: exact 27B head with Omega bounds and GPU schedule 27B experiment: exact GPU head and reusable command seals Nov 1, 2026
@aien-spark aien-spark changed the title 27B experiment: exact GPU head and reusable command seals 27B: exact GPU head and reusable command seals Nov 1, 2026
@cursor cursor Bot mentioned this pull request Nov 1, 2026
5 of 6 tasks
cursor Bot pushed a commit that referenced this pull request Nov 1, 2026
The launch is cb5bd70, the merge of 03cdcc5 and PR #251 6a93cb5.
raw.json, the filled table, and sm121dis --pairs of each threshold
and the stall under it are the chip output. UNITS.md compares that
stall field with NAK's deps.delay at Mesa 01bab930.

Co-authored-by: aien-spark <[email protected]>
cursor Bot pushed a commit that referenced this pull request Nov 1, 2026
c94c2c9 is the merge of 4f3a4d0 and PR #251. stall_required matches
the earlier receipt. raw.json still has one lane-0 word and a null
reference, so the file does not itself show the 32-lane or d0..d3 check.

Co-authored-by: aien-spark <[email protected]>
cursor Bot pushed a commit that referenced this pull request Nov 1, 2026
fb91560 is 7ec0184 merged with PR #251. Passing stalls have
lanes_matched_min 32. Every recorded first_mismatch is word 2, so the
file does not by itself show a later word failing.

Co-authored-by: aien-spark <[email protected]>
aien-spark added a commit that referenced this pull request Nov 1, 2026
…l-B class set

At driver 580.173.02 the GB10 reports the GR SM-version field 10_04 (0xa04).
ctrl2080gr.h:323-328 keeps 10_04 only until its 12_* names are validated, so
10_04 alone does not identify sm_121. Seeded Omega op 1 now admits sm_121 for
12_01, or for 10_04 only with PCI 0x10de:0x2e12, and still requires the full
Blackwell-B class set. nvrm.c only marshals the vendor constant and PCI ids.

Rule taken from #251 (same four source files); this branch is today's main plus
that rule, with om/units.sh regenerated (pinned omc 2c824805, omega-bridge
3dc4a00e). Generated amp_runtime.c is byte-identical to #251's.

Host: make test-ampere-runtime PASS (GB10 admitted; wrong device, wrong vendor,
partial class set and Ampere classes refused).
GB10: build/hello_gpu PASS, 1048576/1048576 elements correct. Negative control:
origin/main hello_gpu refuses "unsupported SM 0xa04 or missing target classes".

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@drakestapleton
drakestapleton marked this pull request as ready for review November 1, 2026 21:04
aien-spark added a commit that referenced this pull request Nov 1, 2026
…kwell-B class set (#246)

* Admit the GB10 (SM 0xa04) only with its PCI identity and the Blackwell-B class set

At driver 580.173.02 the GB10 reports the GR SM-version field 10_04 (0xa04).
ctrl2080gr.h:323-328 keeps 10_04 only until its 12_* names are validated, so
10_04 alone does not identify sm_121. Seeded Omega op 1 now admits sm_121 for
12_01, or for 10_04 only with PCI 0x10de:0x2e12, and still requires the full
Blackwell-B class set. nvrm.c only marshals the vendor constant and PCI ids.

Rule taken from #251 (same four source files); this branch is today's main plus
that rule, with om/units.sh regenerated (pinned omc 2c824805, omega-bridge
3dc4a00e). Generated amp_runtime.c is byte-identical to #251's.

Host: make test-ampere-runtime PASS (GB10 admitted; wrong device, wrong vendor,
partial class set and Ampere classes refused).
GB10: build/hello_gpu PASS, 1048576/1048576 elements correct. Negative control:
origin/main hello_gpu refuses "unsupported SM 0xa04 or missing target classes".

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* docs: point PORT-AMPERE at nvrm_copy_to/from and the Ampere refusal line

Independent review of #246 found :749/:755 and :1058 off by a few lines.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>
@aien-spark

Copy link
Copy Markdown
Collaborator Author

PR desk (2026-11-01): the GB10 admission rule from this PR landed on main separately through #246 (merge dba5821). SM 0xa04 is admitted only with PCI 0x10de:0x2e12 and the full Blackwell-B class set. src/cudapp_om_amp_runtime.c is byte-identical to this PR's copy. Its evidence: host make test-ampere-runtime PASS; on the GB10, build/hello_gpu gave 1,048,576 of 1,048,576 correct; the negative control is the old main refusing "unsupported SM 0xa04". When you next refresh this branch, take main's version of om/encoder/amp_runtime.om, nvrm/nvrm.c, tests/test_ampere_runtime.c, the generated runtime C and src/*.ident, so this PR keeps only the 27B head and seal-reuse work.

@aien-spark aien-spark left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reviewed at 6a93cb5. ci 20/20 green, that's host-only.

  • the admission half is dead weight. nvrm/nvrm.c, om/encoder/amp_runtime.om, src/cudapp_om_amp_runtime.{c,ident} and tests/test_ampere_runtime.c are byte-identical to main since #246 (dba5821). rebase and drop them so the diff is only exact head + seal reuse.
  • evidence isn't at the head. the serving receipt is at CUDApp 3916065 and seal-build at base 009ada7. mk/seal-reuse.mk (-O3) changed after both. REVIEWING.md §1 wants a fresh-clone run at the exact head, with the receipt path posted before review. there isn't one.
  • the body's "75.606 → 9.925 ms/token recording" isn't in any receipt in this diff. cite the file or drop the number.
  • receipts sit under om/pipeline and om/q38-head, not evidence// (AGENTS.md, CONTRIBUTING step 4).
  • nothing in aien-dev/benchmarks-mv covers 27B decode, so the 48.6 → 32.8 s number has no baseline to check against. it's shared-machine anyway, as the body says.
  • not checked, because it needs running code: exact head + seal reuse on chip at this head.

fix first

Comment thread om/pipeline/seal_reuse.om
Comment on lines +11 to +27
b = new uchar 16384 0
off = add ulong 896 mul n 4
for i ulong off
put b i get bank i
end
for i ulong 384
v = get qmd i
if or and ge i 40 lt i 44 and ge i 48 lt i 52
v = 0
end
put b add off i v
end
off = add off 384
for i ulong 384
put b add off i get qmd add 512 i
end
size = add off 384

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this hand-copies mcc_seal's normalization (mccover.om ~146-172) instead of calling it. if mcc_seal's zeroed ranges ever change, reseal keeps hashing the old contract and its digests stop matching mcseal. pull the normalizer into one place used by both, or at minimum gate it with the differential test on seal-reuse-check.py.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The normalizer is still the copy in om/pipeline/seal_reuse.om. Sharing it with mcc_seal would regenerate the sealed C unit, so this change takes the other option from the review: make test-seal-reuse now links omega_native_mcseal and requires the same digest as omega_native_mcreseal on every case, on a miss and on a hit. Local run on 6e2adc7: 6304 cases, pass. SEAL_REUSE.md says the copy remains and the differential test is the gate.

def digest(n):
v=bytearray(q[:384]);v[40:44]=bytes(4);v[48:52]=bytes(4)
return hashlib.sha256(bytes(b[:896+n*4])+v+bytes(q[512:896])).digest()
cases=0

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the oracle is python hashlib over a third copy of the same contract. nothing compares omega_native_mcreseal to omega_native_mcseal on the same (bank, qmd, n). link both, run the 6304 cases through each, require equal digests on miss and on hit. that's the check that actually covers the copy-paste.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seal-reuse.so now links src/cudapp_om_closure_reseal.c and src/cudapp_om_closure_seal.c. om/pipeline/seal-reuse-check.py calls omega_native_mcseal on the same bank, QMD, and argument count as omega_native_mcreseal and requires equal digests, including the repeated call that hits the memo and the invalid counts that both refuse. The mutant driver links omega_native_mcseal the same way. Local make test-seal-reuse on 6e2adc7 printed oracle_cases 6304 and pass. The five Omega source mutants still run only where the pinned compiler is present (CUDAPP_OMC_HOST).

Comment thread src/omega_gpu_q38_graph.c
if(g->exact_bound.code.cpu)omega_gpu_session_free(&g->exact_bound.code);
memset(&g->exact_mem,0,sizeof g->exact_mem);
memset(&g->exact_scale,0,sizeof g->exact_scale);memset(&g->exact_bound,0,sizeof g->exact_bound);
} else {g->exact8=head;g->exact_on=1;}

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

installing the head sets exact_on=1, so from then on every record_final goes through rec_exact_head (L483), and only the candidate rows get the bf16 recompute. omega_gpu_q38_graph_read (L535) still copies the logits buffer back as if it were the full logits. the only thing keeping sampling safe is the header comment plus Core flipping set_exact_head_on(0) for sampled requests. default it off, or have graph_read refuse or flag logits when the last token ran exact.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Masked logits are refused when sampling is on. omega_gpu_q38_graph_set_sampling(g, 1) returns OMEGA_GPU_DGRAPH_BAD_ARGS while exact_on is set. omega_gpu_q38_graph_set_exact_head_on now returns that same code instead of enabling the head when sampling is on, and set_exact_head refuses to install while sampling is on (the check is before shape_is, so it does not touch the device). record_final refuses the pair before any launch. graph_read refuses a logits pointer when sampling is on and the last record_final wrote the masked buffer (exact_logits). omega_gpu_q38_graph_test_sampling_guard covers that on a stack object with no device, and tests/gpu_q38_graph_test --host-only calls it. Local result: gpu_q38_graph_test (host-only): 5 checks, 0 failed: PASS. 6e2adc7.

{
"status": "PASS_SCOPED_FUNCTIONAL_GATE",
"code": {
"CUDApp": "39160652aa4b4224d64c1fa9b7c012416efa1cd0",

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3916065 is 3 commits behind the head. one of those commits moves the reseal object to -O3, so this served binary is the O2 build. needs a run at 6a93cb5.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No new serving run is in this push. om/q38-head/serving-validation.json is still the earlier O2 screen, not a receipt for 6e2adc7 or for the O3 reseal. The GB10 serving gate on this head is NOT RUN. The host fresh-clone receipt (make test, make test-native-artifact, and the other host gates) will be committed under evidence/ when it finishes; that is not the serving screen.

cursoragent and others added 2 commits November 2, 2026 00:13
set_sampling and set_exact_head_on refuse that pair, record_final refuses
it before any device work, and graph_read refuses a logits pointer when
sampling is on and the last token wrote the mask. The host check covers
that without opening a device.

The seal reuse oracle also requires omega_native_mcseal to return the
same digest on all 6304 cases, on a miss and on a hit.

Co-authored-by: aien-spark <[email protected]>
@cursor
cursor Bot requested a review from drakestapleton November 2, 2026 00:23

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Risk: high. I left a non-blocking comment and did not approve: Security Agent passed with no findings, Bugbot was not running, and this exact-head plus command-seal change is above the medium approval threshold. Human review is needed; drakestapleton was assigned.

Open in Web View Automation 

Sent by Cursor Approval Agent: Pull Request Router and Approver

aien-spark added a commit that referenced this pull request Nov 2, 2026
#259 #260)

native-x86-host on this head hit its 40 min timeout (cancelled, not failed); the same code at 9ade9f5 (diff since: evidence JSON + .crumb only) passed it in 34 min, and main e6598c7 passes it. All 10 other checks green on the merge with main. Independent review APPROVE-WITH-NOTES (see PR comment).
The gates ran on 6e2adc7. make test and make test-native-artifact
passed. make doctor is NOT READY on this VM. The GB10 serving run
is NOT RUN.

Co-authored-by: aien-spark <[email protected]>
@cursor

cursor Bot commented Nov 2, 2026

Copy link
Copy Markdown

Fresh-clone host gates, committed at evidence/Q38-FRESH-20261102/RECEIPT.md on f8b0a3e038fc956939c2717723750bc6b2a22fa6. The gates themselves ran on the parent 6e2adc77fef9ff1af8257fe525f37deacc42f59e (sampling guard and seal differential, after the main merge). This commit only adds the receipt. That moves the head, as asked.

Cloned from github.com/aien-dev/CUDApp, detached checkout, git status --porcelain empty. 2026-11-02T00:39:39Z. Linux 6.12.94+ x86_64, gcc 13.3.0, GNU Make 4.3. CUDAPP_MEM_RESERVE_MIB=0 on this VM only, because MemAvailable under-reports. The platform test still checks the 4096 default. No GPU was opened.

Gate Result Exit
make -j4 test PASS. Last line: CUDA++ HOST TESTS PASS (73 targets) 0
make -j4 test-native-artifact PASS. test_gpu_artifact_export: 47 checks, 0 failed 0
make -j4 zero-cuda-gate PASS 0
./build/hello_gpu --sim PASS. 1048576 elements, 0 wrong 0
make doctor NOT READY, expected on this VM (no NVIDIA module, no /dev/nvidia*). GNU make exits 2 2
GB10 chip and serving NOT RUN

om/q38-head/serving-validation.json is still the earlier screen. It is not a receipt for this head. The 75.606 to 9.925 ms/token sentence is NOT RUN / unverified.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Risk: high. I left a non-blocking comment and did not approve: Security Agent passed with no findings, Bugbot was not present, and this exact-head plus command-seal change remains above the medium approval threshold. Human review is needed and no reviewers were assigned.

Open in Web View Automation 

Sent by Cursor Approval Agent: Pull Request Router and Approver

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants