Skip to content

perf(prefill): select scoped Omega reading profiles and reuse safe storage - #259

Closed
drakestapleton wants to merge 8 commits into
perf/q38-qg-orderfrom
perf/q38-scoped-reading
Closed

drakestapleton wants to merge 8 commits into
perf/q38-qg-orderfrom
perf/q38-scoped-reading

Conversation

@drakestapleton

@drakestapleton drakestapleton commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

All 81 local host targets, API/version gates and all 16 hosted checks pass. The final integration preserves every score bit and next-token choice in six full-model cases and fourteen boundary cases. Its immutable server also passes short serving, streaming, quality and saved/fresh continuation checks through 128,000 tokens; full Core preflight passes.

The 27B reader needs faster long reads without imposing a large matrix tile on short requests or changing another caller's settings. This adds per-call matrix options and checked Omega policies for request-size selection and allocation reuse. Requests through 64 rows use 64-row/group0; larger requests use 128-row/group4. Omega admits reuse only when row, padded-row and tile capacities all fit. Existing C and Rust adapters transport these decisions. Original defaults are unchanged. Runtime API 0.12.0 records the additive entries. Stacked on #258; Core integration is aien-sovereign-core-mv#898.

At native 8837939 and Core f440df0e, the allocation oracle covers 1,575 cases and catches five source faults. Real-dispatch isolation and refusal tests pass. The repeated 5,017-token read took 14.497 s versus 18.901 s with fast mode off; the repeated 30-token case took 0.290 s versus 0.274 s. These are shared-machine screens. Quiet alternating performance qualification remains NOT_RUN and is required before a qualified speed claim or default promotion.

Native evidence is in om/prefill-shape/adaptive-validation.json. Its then-pending serving status is superseded by the completed integration receipt, which retains exact source/archive/binary identities and the earlier preflight failure alongside the passing unchanged final suite. Apache-2.0 WITH LLVM-exception notices retained; diff and pinned Crumb verification pass. No new foreign dependency, shared-service configuration change or default promotion.

The 27B reader needs faster long reads without imposing a large matrix tile on short requests or changing another caller's settings. This adds per-call matrix options and checked Omega policies for request-size selection and allocation reuse. Requests through 64 rows use 64-row/group0; larger requests use 128-row/group4. Omega admits reuse only when row, padded-row and tile capacities all fit. Existing C and Rust adapters transport these decisions. Original defaults are unchanged. Runtime API 0.12.0 records the additive entries. Stacked on #258; the base stays perf/q38-qg-order. Core integration is aien-sovereign-core-mv#898.

At native 88379395 and Core f440df0e, the allocation oracle covers 1,575 cases and catches five source faults. Real-dispatch isolation and refusal tests pass. The repeated 5,017-token read took 14.497 s versus 18.901 s with the setting off. The same screen's two 30-token reads were 0.288 s versus 0.298 s and 0.290 s versus 0.274 s. One candidate was faster and one was slower. GEMM time was 0.1641 s versus 0.1645 s and 0.1622 s versus 0.1622 s. Requests through 64 rows use the same profile as the baseline, so that gap is outside the matrix kernel and is not a significant effect. These are shared-machine screens. Quiet alternating performance qualification remains NOT_RUN and is required before a qualified speed claim or default promotion.

Native evidence for that screen is in om/prefill-shape/adaptive-validation.json. Its then-pending serving status is superseded by the completed integration receipt. That receipt's serving.long128.native_archive_sha256 is 752564da817349a38ced23a8beda735f221afce78b1dce0a2e20ea09be1d5ce0 at native_source 88379395. Reviewed head 6820ff0f does not change src/, m16/, nvrm/, third_party/ or mk/ relative to that source. make -j4 all on this x86_64 gcc 13.3.0 produced build/libomega_gpu.a sha256 c6b7052b8eb84e8fbe9b4bc9f87a6216af4ff55cc0d94073c28ac188d135be5b, which is not the GB10 archive. The new head changes src/omega_gpu_pfgemm.c, so its archive does not match either: fresh-clone build/libomega_gpu.a sha256 5677d44e1f494089ac62ec3d14ee1c01f988a8b1847f38f22757016ebf26a0a2. The 128K serving result stays on 88379395. Apache-2.0 WITH LLVM-exception notices retained. No new foreign dependency, shared-service configuration change or default promotion.

A scoped qgemm call with omega_gpu_pf_qgemm_test_set_mutant set now returns OMEGA_GPU_PF_BAD_ARGS before device access. docs/REVIEWING.md section 2.1 asks for that refusal in the change's own test (make test-prefill-scoped), not a full-model score-compare red run. No model-level control was added: that run is a GB10 result in another repository, and this session has no authorized GPU window.

Evidence on 932d9548d3ca8584316c9f2cd3307d2ec32535d7

Receipt: evidence/PREFILL-SCOPED-MUTANT-20261102/. The receipt commit is 1ddff2f and only adds that directory. Linux 6.12.94+ x86_64, gcc 13.3.0.

  • CUDAPP_MEM_RESERVE_MIB=0 make -j4 test PASS, exit 0, CUDA++ HOST TESTS PASS (78 targets). Log sha256 f32d60d0017ab4c467a39fdd7e45e63b59d8efcb659900aa25e535611ba9cfde.
  • make test-native-artifact PASS inside that suite (test_gpu_artifact_export: 47 checks, 0 failed).
  • make zero-cuda-gate PASS inside that suite (ZERO_CUDA PASS for build/hello_gpu and build/gpu_matmul_api_test; self-test rejected a CUDA probe).
  • build/hello_gpu --sim PASS inside that suite (hello --sim tests: 12 passed, 0 failed).
  • make doctor NOT READY, recipe exit 1 (make exit 2). No NVIDIA module and no /dev/nvidia*. Expected on this machine. The suite's fake-machine doctor printed doctor tests: 9 passed, 0 failed.
  • GB10 hardware tests NOT RUN. No authorized GPU window.

make -j4 test with the reserve unset refused a 741.9 MiB upload (CUDAPP_MEM_RESERVE_MIB=4096, 4.6 GiB free). That run was stopped while make was still waiting, so it has no completed exit code. See evidence/PREFILL-SCOPED-MUTANT-20261102/default-reserve.txt. The 4096 default is still checked by the platform test.

Stack

This PR was not retargeted. It merges into perf/q38-qg-order (#258). #258 is still a draft and has to leave draft and merge first. #257 is still a draft on #252, which is a draft on #251. Nothing here reaches main until that chain lands.

Open in Web Open in Cursor 

@drakestapleton drakestapleton changed the title feat(prefill): scope reading options to each matrix call perf(prefill): select scoped Omega reading profiles and reuse safe storage Nov 1, 2026
@drakestapleton
drakestapleton marked this pull request as ready for review November 1, 2026 21:35

@aien-spark aien-spark left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reviewed at 6820ff0. the scoped spec, reading.om and workspace.om read right, the legacy path is unchanged (qg_core with NULL builds the same spec as before), and all 16 checks are green.

  • the scoped path drops mutant, so the kernel fault controls can't reach the code that's now selected for every read. details inline at pfgemm.c:200.
  • stacked on #258, which is still a draft (its base is #257). this merges into perf/q38-qg-order, not main, so nothing ships until the stack lands. don't merge this ahead of #258 leaving draft.
  • the 30-token "penalty" doesn't hold up. adaptive-validation.json has two 30-token runs: 0.298 -> 0.288 (faster) and 0.274 -> 0.290 (slower). the body and README quote only the slower one. gemm time is identical in both (0.1622 vs 0.1622), which fits, because <= 64 rows picks the same 64/group0 profile as the baseline. the gap is outside the GEMM, so it's noise, not a penalty. say that instead.
  • the 128k serving claim lives in the Core receipt (sc e9366221), not in this PR's json, which still says serving pending. fine, but the native archive there (7525...) should match what this head builds. i can't check that without building.
  • that Core receipt has no model-level kernel-mutant red control either, so "every score bit preserved" has no run showing the compare can fail on the scoped path.

verdict: fix first (mutant guard on the scoped path, then wait for #258)

Comment thread src/omega_gpu_pfgemm.c
if (ntiles > 65535u) return OMEGA_GPU_PF_TOO_LARGE;
OmegaQgSpec s; memset(&s, 0, sizeof s);
s.k = k; s.rt = QRT(); s.warps = QWARPS(); s.wait = QWAIT(); s.n = n; s.mutant = q.mutant; s.group = q.group;
if (scoped) s = *scoped;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s = *scoped comes from qg_scoped_spec, which memsets, so mutant is always 0 here. omega_gpu_pf_qgemm_test_set_mutant is silently ignored on scoped calls, and scoped-check.c even asserts mutant 0 while the global mutant is 1. the adaptive path is now the one Core uses for every read, so any kernel-mutant negative control run through it goes green without mutating anything. keep isolation but fail loud: if (scoped && q.mutant) return OMEGA_GPU_PF_BAD_ARGS;, or carry the mutant in on purpose for tests. right now a red control can't fail.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A scoped call with omega_gpu_pf_qgemm_test_set_mutant set now returns OMEGA_GPU_PF_BAD_ARGS before omega_gpu_pf_open_locked. om/prefill-shape/scoped-check.c requires return code -1 with the device path unopened. Before the guard, that check printed scoped mutant rc -2 opened 1 keyed 1 and scoped qgemm isolation: FAIL. drop_scoped_mutant_guard is caught by scoped-mutants.py.

docs/REVIEWING.md section 2.1, followed by convention until the proposed CODEOWNERS file is activated, requires a deliberately broken kernel or input to be refused or detected by the change's own test. That test is make test-prefill-scoped. It does not require a full-model score-compare red run. The Core compare harness is in another repository and needs a GB10 window this session does not have, so no model-level control was added. A model run on the scoped path can no longer pass by dropping the mutant: the call fails before a kernel exists.


`workspace.om` admits existing storage only when its row, padded-row and tile capacities all cover the newly selected profile. This lets the adapter rebuild each tile table without reallocating otherwise sufficient storage. The independent oracle passes 1,575 cases and catches five source faults. Incompatible storage is refused. Revised full-model reuse measurements and the complete host suite remain pending.

The revised adaptive profile and capacity reuse pass the complete 81-target host suite and version gate at native source `88379395296357571618ebf7f701a6e2cdf8be31`. Core source `f440df0e3f9fd4c6b5d37deb1b635037f140bb32` preserves all score bits and next-token choices in six model cases and 14 boundary cases. The repeated 5,017-token read took 14.497 seconds versus 18.901 seconds with the setting off. The repeated 30-token case took 0.290 versus 0.274 seconds. These shared-machine observations do not establish quiet timing or eliminate the small-request penalty. Exact commands, identities and raw observations are in `adaptive-validation.json`. Serving and long-context validation remain separate.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"small-request penalty": the json shows one 30-token run faster and one slower, with identical gemm time and the same profile as baseline. that's noise outside the GEMM. don't write it up as a penalty of the adaptive policy.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The penalty sentence is removed. The paragraph now reports both 30-token reads from adaptive-validation.json: 0.288 s versus 0.298 s, and 0.290 s versus 0.274 s. GEMM time was 0.1641 s versus 0.1645 s and 0.1622 s versus 0.1622 s. One candidate was faster and one was slower. Requests through 64 rows use the same 64-row/group0 profile as the baseline, so the wall-time gap is outside the matrix kernel and is not a significant effect.

Comment thread src/omega_gpu_prefill.h
uint64_t rowidx_va, uint64_t xq_va, uint64_t xs_va, uint64_t y_va);
/* Explicit options belong to this call only. bm returns 0 for refused options;
* a real call also validates its projection shape. Tile buffers must use this bm. */
/* Omega-owned reading options (rt, warps, wait, group), UINT32_MAX on refusal. */

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: both comments sit above workspace_fits. "bm returns 0" belongs to scoped_bm and "reading options" belongs to reading_option. move them next to the decls they describe.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tile-height comment now sits on omega_gpu_pf_qgemm_scoped_bm. The reading-option comment now sits on omega_gpu_pf_qgemm_reading_option.

aien-spark added a commit that referenced this pull request Nov 2, 2026
#259 #260)

native-x86-host on this head hit its 40 min timeout (cancelled, not failed); the same code at 9ade9f5 (diff since: evidence JSON + .crumb only) passed it in 34 min, and main e6598c7 passes it. All 10 other checks green on the merge with main. Independent review APPROVE-WITH-NOTES (see PR comment).
@aien-spark

Copy link
Copy Markdown
Collaborator

Merge desk (AI), 2026-11-02. Landed on main as part of the reviewed Ribbon stack through #260 (merge 183fa4f). This head (6820ff0) is contained in main. Closing as landed.

@aien-spark aien-spark closed this Nov 2, 2026
@cursor

cursor Bot commented Nov 2, 2026

Copy link
Copy Markdown

Fresh-clone host receipt for the suite that ran on 932d9548d3ca8584316c9f2cd3307d2ec32535d7: evidence/PREFILL-SCOPED-MUTANT-20261102/. Head is now 1ddff2f, which only adds that directory.

CUDAPP_MEM_RESERVE_MIB=0 make -j4 test exited 0. Last line: CUDA++ HOST TESTS PASS (78 targets). Log sha256 f32d60d0017ab4c467a39fdd7e45e63b59d8efcb659900aa25e535611ba9cfde.

The 128K serving archive 752564da817349a38ced23a8beda735f221afce78b1dce0a2e20ea09be1d5ce0 is recorded at native source 88379395. Reviewed head 6820ff0f has the same library inputs. make -j4 all here is x86-64 and hashed c6b7052b8eb84e8fbe9b4bc9f87a6216af4ff55cc0d94073c28ac188d135be5b, so this machine does not reproduce the GB10 bytes. The new head changes src/omega_gpu_pfgemm.c; its fresh-clone archive is 5677d44e1f494089ac62ec3d14ee1c01f988a8b1847f38f22757016ebf26a0a2 and does not match the 128K archive.

Stack, not retargeted. This PR still merges into perf/q38-qg-order.

#258 has to leave draft and merge before this PR. #257 has to merge before #258. The chain does not reach main until #252 and #251 land as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants