Skip to content

Keep response-format grammars applied through a budgeted thought on WebGPU (#1954) - #1964

Merged
leehack merged 3 commits into
mainfrom
lane-1954-webgpu
Nov 12, 2026
Merged

leehack merged 3 commits into
mainfrom
lane-1954-webgpu

Conversation

@leehack

@leehack leehack commented Nov 12, 2026 •

Copy link
Copy Markdown
Owner

Fixes #1954

Summary

  • Repin WebGPU bridge assets v0.1.83 to v0.1.84 (release 443341300, tag commit 0a3c45f2, bridge source dc359398, manifest SHA-256 55abeb37...04e, native v0.9.0), which adds the thinkingBudgetPauseGrammar capability and the thinkingBudget.pauseGrammar request flag (llama-web-bridge#213).
  • WebGpuLlamaBackend sends pauseGrammar: false exactly where native passes pauseGrammarDuringReasoning: false: an eager grammar for which ToolCallGrammarUtils.keepsAppliedThroughThought holds (Qwen3.5 responseFormat, Seed-OSS required calls). Every other request is unchanged on the wire.
  • On assets without the capability, that combination throws LlamaUnsupportedException naming thinkingBudgetPauseGrammar and v0.1.84+, instead of returning an unconstrained reply.
  • Pin sites updated per native-and-web-sync.md: README, chat app index.html, fetch script, web_bridge_setup.dart, check_webgpu_bridge_tag.dart (release id, tag commit, source commit), tag-check fixtures, docs (doc/webgpu_bridge.md, website page, multimodal embeddings note), changelogs.

Production-readiness scope

  • User-facing scope: On WebGPU, responseFormat (and required Seed-OSS calls) with a ThinkingBudget stays constrained after the budget closes the thought, matching native (#1956, #1962).
  • Supported platforms/paths: WebGPU/Wasm GGUF via bridge assets v0.1.84+, direct and worker.
  • Unsupported or intentionally unavailable paths: Older assets throw LlamaUnsupportedException (names thinkingBudgetPauseGrammar, v0.1.84+) for that combination; other budget requests on older assets work as before.
  • Out of scope / follow-ups: None.

Completeness checklist

  • Declared scope is fully implemented, or explicitly reduced above.
  • Unsupported platform/option combinations fail loudly with actionable diagnostics or are clearly documented as unavailable.
  • Public API docs, README/website docs, examples, support matrices, and changelog entries are updated where relevant (thinkingBudget Dartdoc kept minimal because #1961 also edits it; skills/llamadart-web).
  • Regression coverage covers the original issue plus key negative/version-skew paths where relevant.
  • Public capability changes update the validation coverage registry with a pack or a reasoned, issue-linked exemption; inventory checks pass (or N/A with scope). N/A: no new public capability flag; validation_coverage_test passes.
  • Security/privacy review completed: no secrets, tokens, signed URLs or paths added.
  • Follow-up work tracked: None.

PR type guidance

  • Bugfix PR: Root cause: the bridge paused every grammar while a budgeted thought was open, so a grammar that takes the template-opened thought resumed inside it and stopped constraining the reply. Fix: bridge flag adopted via the repin, sent under the same condition as native. Regression tests below; affected platform: WebGPU only.

Test Plan

  • dart run tool/prepare_workspace.dart
  • dart format --output=none --set-exit-if-changed .
  • dart analyze
  • dart test -p vm -j 1 --exclude-tags local-only N/A as a whole: only tooling/web_setup/structure VM suites are touched; they pass (test/unit/tooling/ test/unit/test_structure test/unit/web_setup)
  • dart test -p chrome on test/unit/backends/webgpu (540 tests pass)
  • coverage-lib: N/A, changed lib/ code is browser-only (webgpu_backend.dart, interop.dart) and is not in VM coverage; covered by the Chrome suite above
  • Other: check_webgpu_bridge_tag.dart --verify-manifest, verify_release_docs_versions.dart, web-lazy-grammar scenario (--list, --dry-run, real run), mutation run below

Matrix Evidence

Matrix row Scope covered Platform / model / backend Result Evidence / notes
static-format-analyze format, analyzer, import boundaries Dart/Flutter 3.47.1 PASS local
root-vm tooling, structure, web_setup suites VM PASS local; whole-suite run left to CI
root-chrome test/unit/backends/webgpu incl. new webgpu_thinking_budget_grammar_test.dart Chrome PASS 540 tests
coverage-lib - - N/A browser-only lib code, see Test Plan
webgpu-bridge-tag-consistency 16 pins agree on v0.1.84; release 443341300 immutable, commit and manifest bytes match GitHub + CDN PASS --verify-manifest
release-doc-version-consistency install snippets and pins - PASS local
docs-site website build - N/A docs text edits only; left to CI docs lane
web-lazy-grammar new thinking_budget_grammar_e2e_test.dart: Qwen3.5-0.8B Q4_K_M responseFormat JSON with budgets 0, 1, 16, direct and worker, CPU Wasm, pinned v0.1.84; control on v0.1.83 asserts unconstrained no-budget JSON and the typed rejection chrome-webgpu, CPU Wasm PASS 17/17 incl. lazy and prompt-penalty suites
structured-output-adversarial native parser/grammar families - N/A no parser, grammar or template change; grammar selection reuses keepsAppliedThroughThought from #1962
high-risk-exact-head-independent-qa - - pending see below

Mutation check on the new unit test (webgpu_thinking_budget_grammar_test.dart, Chrome):

Variant Result
origin/main lib (no fix) 2 of 3 fail
Fixed, unmutated 3/3 pass
M1: drop the pauseGrammar: false flag fails (keeps-applied)
M2: drop the capability rejection fails (version-skew)
M3: takesForcedOpenThought instead of keepsAppliedThroughThought (misses Seed-OSS) fails
M4: ignore grammarLazy fails (stays-paused)
M5: send the flag on every budget fails

Review Notes

High-risk regression review

  • Classification: high-risk (artifactConsumer, backendRuntime, regressionPolicy)
  • Implementation task: lane-1954-webgpu
  • Independent blocking QA task: fresh auditor-deep, PASS. It checked the wire contract against bridge dc359398, native parity, the capability probe and the repin (manifest SHA re-downloaded), and simulated 0.19.8 release prep with no drift. 12/12 backend and 10 of 12 tooling mutants killed (T7/T7b shard-weight entries survive, P3). The e2e caught M1 with the WebGPU: thinking budget pauses forced-open-thought grammars (same as #1953) #1954 symptom. Evaluator exit 2.
  • Exact head / current base: 65c62f433b / 523ab63a3b
  • Production-branch deletion, bypass, or miswire proof: M1-M12 each fail the Chrome unit test; M1 also fails the real-model e2e (direct and worker)
  • Affected-family real-model/artifact evidence: Qwen3.5-0.8B Q4_K_M on pinned v0.1.84 assets, direct and worker; v0.1.83 control
  • Explicit unavailable-family or other N/A evidence: Seed-OSS has no local weights; covered by the unit test with the Seed-OSS grammar root only. Hardware WebGPU not qualified (CPU Wasm).
  • Known PR-caused P1 regressions: 0
  • Unresolved review threads: 0

…ebGPU (#1954)

Repin bridge assets v0.1.83 -> v0.1.84, which reports
thinkingBudgetPauseGrammar. Send thinkingBudget.pauseGrammar: false only for an
eager grammar that keepsAppliedThroughThought, as native does, and throw
LlamaUnsupportedException on assets without the capability.
@leehack
leehack marked this pull request as ready for review November 12, 2026 19:32
@leehack leehack mentioned this pull request Nov 12, 2026
17 of 20 tasks
@leehack
leehack merged commit c2eb1a8 into main Nov 12, 2026
45 checks passed
@leehack
leehack deleted the lane-1954-webgpu branch November 12, 2026 19:33
@github-actions

Copy link
Copy Markdown
Contributor

Chat app preview removed for leehack/llamadart-chat-pr-1964.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

WebGPU: thinking budget pauses forced-open-thought grammars (same as #1953)

1 participant