Repository navigation
perf(prefill): select scoped Omega reading profiles and reuse safe storage - #259
drakestapleton wants to merge 8 commits into
Conversation
aien-spark
left a comment
There was a problem hiding this comment.
reviewed at 6820ff0. the scoped spec, reading.om and workspace.om read right, the legacy path is unchanged (qg_core with NULL builds the same spec as before), and all 16 checks are green.
- the scoped path drops
mutant, so the kernel fault controls can't reach the code that's now selected for every read. details inline at pfgemm.c:200. - stacked on #258, which is still a draft (its base is #257). this merges into perf/q38-qg-order, not main, so nothing ships until the stack lands. don't merge this ahead of #258 leaving draft.
- the 30-token "penalty" doesn't hold up. adaptive-validation.json has two 30-token runs: 0.298 -> 0.288 (faster) and 0.274 -> 0.290 (slower). the body and README quote only the slower one. gemm time is identical in both (0.1622 vs 0.1622), which fits, because <= 64 rows picks the same 64/group0 profile as the baseline. the gap is outside the GEMM, so it's noise, not a penalty. say that instead.
- the 128k serving claim lives in the Core receipt (sc e9366221), not in this PR's json, which still says serving pending. fine, but the native archive there (7525...) should match what this head builds. i can't check that without building.
- that Core receipt has no model-level kernel-mutant red control either, so "every score bit preserved" has no run showing the compare can fail on the scoped path.
verdict: fix first (mutant guard on the scoped path, then wait for #258)
| if (ntiles > 65535u) return OMEGA_GPU_PF_TOO_LARGE; | ||
| OmegaQgSpec s; memset(&s, 0, sizeof s); | ||
| s.k = k; s.rt = QRT(); s.warps = QWARPS(); s.wait = QWAIT(); s.n = n; s.mutant = q.mutant; s.group = q.group; | ||
| if (scoped) s = *scoped; |
There was a problem hiding this comment.
s = *scoped comes from qg_scoped_spec, which memsets, so mutant is always 0 here. omega_gpu_pf_qgemm_test_set_mutant is silently ignored on scoped calls, and scoped-check.c even asserts mutant 0 while the global mutant is 1. the adaptive path is now the one Core uses for every read, so any kernel-mutant negative control run through it goes green without mutating anything. keep isolation but fail loud: if (scoped && q.mutant) return OMEGA_GPU_PF_BAD_ARGS;, or carry the mutant in on purpose for tests. right now a red control can't fail.
There was a problem hiding this comment.
A scoped call with omega_gpu_pf_qgemm_test_set_mutant set now returns OMEGA_GPU_PF_BAD_ARGS before omega_gpu_pf_open_locked. om/prefill-shape/scoped-check.c requires return code -1 with the device path unopened. Before the guard, that check printed scoped mutant rc -2 opened 1 keyed 1 and scoped qgemm isolation: FAIL. drop_scoped_mutant_guard is caught by scoped-mutants.py.
docs/REVIEWING.md section 2.1, followed by convention until the proposed CODEOWNERS file is activated, requires a deliberately broken kernel or input to be refused or detected by the change's own test. That test is make test-prefill-scoped. It does not require a full-model score-compare red run. The Core compare harness is in another repository and needs a GB10 window this session does not have, so no model-level control was added. A model run on the scoped path can no longer pass by dropping the mutant: the call fails before a kernel exists.
|
|
||
| `workspace.om` admits existing storage only when its row, padded-row and tile capacities all cover the newly selected profile. This lets the adapter rebuild each tile table without reallocating otherwise sufficient storage. The independent oracle passes 1,575 cases and catches five source faults. Incompatible storage is refused. Revised full-model reuse measurements and the complete host suite remain pending. | ||
|
|
||
| The revised adaptive profile and capacity reuse pass the complete 81-target host suite and version gate at native source `88379395296357571618ebf7f701a6e2cdf8be31`. Core source `f440df0e3f9fd4c6b5d37deb1b635037f140bb32` preserves all score bits and next-token choices in six model cases and 14 boundary cases. The repeated 5,017-token read took 14.497 seconds versus 18.901 seconds with the setting off. The repeated 30-token case took 0.290 versus 0.274 seconds. These shared-machine observations do not establish quiet timing or eliminate the small-request penalty. Exact commands, identities and raw observations are in `adaptive-validation.json`. Serving and long-context validation remain separate. |
There was a problem hiding this comment.
"small-request penalty": the json shows one 30-token run faster and one slower, with identical gemm time and the same profile as baseline. that's noise outside the GEMM. don't write it up as a penalty of the adaptive policy.
There was a problem hiding this comment.
The penalty sentence is removed. The paragraph now reports both 30-token reads from adaptive-validation.json: 0.288 s versus 0.298 s, and 0.290 s versus 0.274 s. GEMM time was 0.1641 s versus 0.1645 s and 0.1622 s versus 0.1622 s. One candidate was faster and one was slower. Requests through 64 rows use the same 64-row/group0 profile as the baseline, so the wall-time gap is outside the matrix kernel and is not a significant effect.
| uint64_t rowidx_va, uint64_t xq_va, uint64_t xs_va, uint64_t y_va); | ||
| /* Explicit options belong to this call only. bm returns 0 for refused options; | ||
| * a real call also validates its projection shape. Tile buffers must use this bm. */ | ||
| /* Omega-owned reading options (rt, warps, wait, group), UINT32_MAX on refusal. */ |
There was a problem hiding this comment.
nit: both comments sit above workspace_fits. "bm returns 0" belongs to scoped_bm and "reading options" belongs to reading_option. move them next to the decls they describe.
There was a problem hiding this comment.
The tile-height comment now sits on omega_gpu_pf_qgemm_scoped_bm. The reading-option comment now sits on omega_gpu_pf_qgemm_reading_option.
#259 #260) native-x86-host on this head hit its 40 min timeout (cancelled, not failed); the same code at 9ade9f5 (diff since: evidence JSON + .crumb only) passed it in 34 min, and main e6598c7 passes it. All 10 other checks green on the merge with main. Independent review APPROVE-WITH-NOTES (see PR comment).
|
Fresh-clone host receipt for the suite that ran on
The 128K serving archive Stack, not retargeted. This PR still merges into
#258 has to leave draft and merge before this PR. #257 has to merge before #258. The chain does not reach |
All 81 local host targets, API/version gates and all 16 hosted checks pass. The final integration preserves every score bit and next-token choice in six full-model cases and fourteen boundary cases. Its immutable server also passes short serving, streaming, quality and saved/fresh continuation checks through 128,000 tokens; full Core preflight passes.
The 27B reader needs faster long reads without imposing a large matrix tile on short requests or changing another caller's settings. This adds per-call matrix options and checked Omega policies for request-size selection and allocation reuse. Requests through 64 rows use 64-row/group0; larger requests use 128-row/group4. Omega admits reuse only when row, padded-row and tile capacities all fit. Existing C and Rust adapters transport these decisions. Original defaults are unchanged. Runtime API 0.12.0 records the additive entries. Stacked on #258; Core integration is aien-sovereign-core-mv#898.
At native 8837939 and Core f440df0e, the allocation oracle covers 1,575 cases and catches five source faults. Real-dispatch isolation and refusal tests pass. The repeated 5,017-token read took 14.497 s versus 18.901 s with fast mode off; the repeated 30-token case took 0.290 s versus 0.274 s. These are shared-machine screens. Quiet alternating performance qualification remains NOT_RUN and is required before a qualified speed claim or default promotion.
Native evidence is in om/prefill-shape/adaptive-validation.json. Its then-pending serving status is superseded by the completed integration receipt, which retains exact source/archive/binary identities and the earlier preflight failure alongside the passing unchanged final suite. Apache-2.0 WITH LLVM-exception notices retained; diff and pinned Crumb verification pass. No new foreign dependency, shared-service configuration change or default promotion.
The 27B reader needs faster long reads without imposing a large matrix tile on short requests or changing another caller's settings. This adds per-call matrix options and checked Omega policies for request-size selection and allocation reuse. Requests through 64 rows use 64-row/group0; larger requests use 128-row/group4. Omega admits reuse only when row, padded-row and tile capacities all fit. Existing C and Rust adapters transport these decisions. Original defaults are unchanged. Runtime API 0.12.0 records the additive entries. Stacked on #258; the base stays
perf/q38-qg-order. Core integration is aien-sovereign-core-mv#898.At native
88379395and Coref440df0e, the allocation oracle covers 1,575 cases and catches five source faults. Real-dispatch isolation and refusal tests pass. The repeated 5,017-token read took 14.497 s versus 18.901 s with the setting off. The same screen's two 30-token reads were 0.288 s versus 0.298 s and 0.290 s versus 0.274 s. One candidate was faster and one was slower. GEMM time was 0.1641 s versus 0.1645 s and 0.1622 s versus 0.1622 s. Requests through 64 rows use the same profile as the baseline, so that gap is outside the matrix kernel and is not a significant effect. These are shared-machine screens. Quiet alternating performance qualification remains NOT_RUN and is required before a qualified speed claim or default promotion.Native evidence for that screen is in
om/prefill-shape/adaptive-validation.json. Its then-pending serving status is superseded by the completed integration receipt. That receipt'sserving.long128.native_archive_sha256is752564da817349a38ced23a8beda735f221afce78b1dce0a2e20ea09be1d5ce0atnative_source88379395. Reviewed head6820ff0fdoes not changesrc/,m16/,nvrm/,third_party/ormk/relative to that source.make -j4 allon this x86_64 gcc 13.3.0 producedbuild/libomega_gpu.asha256c6b7052b8eb84e8fbe9b4bc9f87a6216af4ff55cc0d94073c28ac188d135be5b, which is not the GB10 archive. The new head changessrc/omega_gpu_pfgemm.c, so its archive does not match either: fresh-clonebuild/libomega_gpu.asha2565677d44e1f494089ac62ec3d14ee1c01f988a8b1847f38f22757016ebf26a0a2. The 128K serving result stays on88379395. Apache-2.0 WITH LLVM-exception notices retained. No new foreign dependency, shared-service configuration change or default promotion.A scoped qgemm call with
omega_gpu_pf_qgemm_test_set_mutantset now returnsOMEGA_GPU_PF_BAD_ARGSbefore device access.docs/REVIEWING.mdsection 2.1 asks for that refusal in the change's own test (make test-prefill-scoped), not a full-model score-compare red run. No model-level control was added: that run is a GB10 result in another repository, and this session has no authorized GPU window.Evidence on
932d9548d3ca8584316c9f2cd3307d2ec32535d7Receipt:
evidence/PREFILL-SCOPED-MUTANT-20261102/. The receipt commit is1ddff2fand only adds that directory. Linux 6.12.94+ x86_64, gcc 13.3.0.CUDAPP_MEM_RESERVE_MIB=0 make -j4 testPASS, exit 0,CUDA++ HOST TESTS PASS (78 targets). Log sha256f32d60d0017ab4c467a39fdd7e45e63b59d8efcb659900aa25e535611ba9cfde.make test-native-artifactPASS inside that suite (test_gpu_artifact_export: 47 checks, 0 failed).make zero-cuda-gatePASS inside that suite (ZERO_CUDA PASSforbuild/hello_gpuandbuild/gpu_matmul_api_test; self-test rejected a CUDA probe).build/hello_gpu --simPASS inside that suite (hello --sim tests: 12 passed, 0 failed).make doctorNOT READY, recipe exit 1 (makeexit 2). No NVIDIA module and no/dev/nvidia*. Expected on this machine. The suite's fake-machine doctor printeddoctor tests: 9 passed, 0 failed.make -j4 testwith the reserve unset refused a 741.9 MiB upload (CUDAPP_MEM_RESERVE_MIB=4096, 4.6 GiB free). That run was stopped whilemakewas still waiting, so it has no completed exit code. Seeevidence/PREFILL-SCOPED-MUTANT-20261102/default-reserve.txt. The 4096 default is still checked by the platform test.Stack
This PR was not retargeted. It merges into
perf/q38-qg-order(#258). #258 is still a draft and has to leave draft and merge first. #257 is still a draft on #252, which is a draft on #251. Nothing here reachesmainuntil that chain lands.