Skip to content

perf(attention): add scoped Omega-controlled cached history - #260

Merged
aien-spark merged 36 commits into
mainfrom
perf/q38-attention-mirror
Nov 2, 2026
Merged

aien-spark merged 36 commits into
mainfrom
perf/q38-attention-mirror

Conversation

@drakestapleton

@drakestapleton drakestapleton commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

The same immutable server delivered the first visible response for 32,768 input tokens in 104.728 seconds with cached history versus 108.372 seconds before; fresh follow-up was 103.074 versus 108.918 seconds. Full-model reading measured 109.500 versus 103.451 seconds with identical scores. These are single shared-machine observations; repeated 5,017-token total was essentially unchanged.

This carries the 30B cached-history mechanism into 27B attention. Omega validates requests and directs allocation, preparation, completed full-history copy and attention through one reusable buffer. Original history and arithmetic remain authoritative. The per-call API has no global selector: mode 0 preserves ordinary behavior, mode 1 mirrors supported calls, and mode 2 applies a provisional 1,024-row threshold. Failures cannot advance to output publication.

Completed verification includes the complete 83-target native host suite, full Core preflight, 63 physical attention checks, six complete-score cases, seven threshold/tail cases, long32 plus short-after-long, and deliberate corruption controls. Every compared score bit matches. Short serving covers streaming, seeded sampling, seven coding cases and prose; paired32K serving matches answers and token IDs across settings. The 65,536-token candidate run passes retrieval and saved/fresh continuation parity. Its first visible response was 237.355 seconds, saved follow-up 0.531 seconds and fresh follow-up 237.970 seconds. This candidate-only 64K run is a correctness observation, not a paired speed claim.

The host-suite registration defect is fixed: carried-tests.mk appends instead of replacing targets registered by earlier fragments. The actual graph retains all 81 previous targets plus both new attention targets. The full 83-target run exits 0 at 9ade9f5; production source and the tested binary remain unchanged. Earlier failures and historical receipts remain preserved.

Integration evidence and host83/64K follow-up pin sources, binaries and original logs. The 128,000-token serving test now passes retrieval and saved/fresh continuation parity; first visible response was 570.312 seconds, saved follow-up 0.778 seconds and fresh follow-up 570.967 seconds. This is a candidate-only correctness run, not a paired speed claim. Five alternating quiet timing pairs and independent review remain open; no quiet qualification or default promotion is claimed.

Stacked on #259; tracked by aien-dev/aien-sovereign-core-mv#710. Native/stub compatibility, API 0.13, pinned Bridge regeneration and Crumb checks pass. No new kernel, production dependency, default change or deployment. Apache-2.0 WITH LLVM-exception notices retained.

Completed 128K evidence preserves the immutable server/source identity and original event hashes. All planned short/32K/64K/128K serving correctness gates now pass; quiet alternating timing and independent review remain open.

aien-spark and others added 5 commits November 1, 2026 15:45
Host oracle passes 1679 requests and 1008 transitions; nine policy and four actual-dispatch mutants are caught. Omega owns bounded plan and completed-operation order. Reuse existing GPU-private allocation and copy transport with one session-owned buffer. API 0.13 adds the explicit scoped entry; legacy entries and defaults remain unchanged. Full suite and chip qualification remain pending.

Co-authored-by: Drake Stapleton <[email protected]>
The full host suite caught a missing inventory row. The portability gate and its negative controls pass with this explicit chip-specific classification; native source is unchanged.

Co-authored-by: Drake Stapleton <[email protected]>
@drakestapleton
drakestapleton marked this pull request as ready for review November 1, 2026 23:13
@aien-spark
aien-spark changed the base branch from perf/q38-scoped-reading to main November 1, 2026 23:22
@aien-spark

Copy link
Copy Markdown
Collaborator

Merge desk note (AI), 2026-11-01.

Independent review of the whole stack (#251, #252, #257, #258, #259, #260): APPROVE-WITH-NOTES, no blockers. The stack merges cleanly onto main. The #246 GB10 admission files are byte-identical on this head and on main, so there is no duplicate rule.

Default-path changes (switches off), each output-equivalent or a hardening:

  1. omega_gpu_cmdlist.c (27B: exact GPU head and reusable command seals #251): every command list now routes launch seals through omega_native_mcreseal. It uses the same SHA-256 over the same normalized bytes as mcc_seal, plus a byte-compare memo, and every other gate still runs.
  2. omega_blackwell_codegen_qg.c (27B: test an Omega 32-row prompt kernel #252): the store-offset bound is now the exact immediate limit from admit.om. Default rt=4 F32 widths from 32768 to 37449 were refused and are now admitted. The 27B uses widths of at most 17408, so it is unaffected.
  3. omega_gpu_prefill_attn256_api.c (perf(attention): add scoped Omega-controlled cached history #260): a failed session_on_close registration now returns CHIP_FAIL. It used to be ignored.

Hashes check out: for all nine units, the src= and c= values in cudapp_om_units.ident match the .om parts and the committed C. A byte-for-byte regeneration with units.sh --check needs the pinned omc and was not rerun in this review.

Follow-up, not in this stack: set_exact_head copied its coefficient table to the chip without checking it, so a NaN, negative or infinite entry could pick a wrong greedy token. A fix with a host test and a mutant is on branch fix/q38-exact-coef-check and will be opened against main after this merges.

@aien-spark aien-spark closed this Nov 1, 2026
@aien-spark aien-spark reopened this Nov 1, 2026
@cursor
cursor Bot requested a review from aien-spark November 1, 2026 23:47

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Risk: high. I left a non-blocking comment and did not approve: this stacked attention/runtime/nvrm change is above the medium approval threshold. Cursor Security Agent completed with no findings, Cursor Bugbot was not present, and aien-spark was assigned for human review.

Open in Web View Automation 

Sent by Cursor Approval Agent: Pull Request Router and Approver

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants