Skip to content

Keep a grammar applied inside a thinking-budget block on request - #213

Merged
leehack merged 2 commits into
mainfrom
fix/thinking-budget-grammar-pause
Nov 12, 2026
Merged

leehack merged 2 commits into
mainfrom
fix/thinking-budget-grammar-pause

Conversation

@leehack

@leehack leehack commented Nov 12, 2026

Copy link
Copy Markdown
Owner

Bridge half of leehack/llamadart#1954.

Problem

grammar_is_active() pauses every grammar while the thinking budget is inside a reasoning block. A grammar that takes the block itself (root think-body "</think>" ..., written for a prompt that ends inside the block) never sees the forced </think>, resumes at its start and takes the reply as reasoning.

Change (additive; default behaviour and wire payloads unchanged)

  • CompletionOptions.thinkingBudget.pauseGrammar?: boolean | null; omitted, null or true pauses as before. Boolean-validated (TypeError: CompletionOptions.thinkingBudget.pauseGrammar must be a boolean; got X.).
  • New capability thinkingBudgetPauseGrammar; pauseGrammar: false is rejected without it.
  • New export llamadart_webgpu_set_next_grammar_reasoning_pause(int32_t), called by runtime.ts after _setNextGrammar and before begin, only when pauseGrammar === false; in EXPORTED_FUNCTIONS.
  • grammar_is_active() returns true when the budget is null or the flag is false; the flag is reset in end_generation_state.
  • Generated js/ outputs regenerated by npm run check:js; docs/api.md and CHANGELOG.md (Unreleased) updated.

Validation

Check Result
npm run check:js (603 tests) pass
node scripts/ci/verify_ci_reliability.mjs pass
tests/bridge/thinking_budget_grammar_native_test.mjs on cb6383f sources fails (assertion in compiled predicate)
same test, mutations: pause every grammar, invert the flag, drop the begin assignment, drop the end reset each fails
scripts/smoke/grammar.mjs open-budget-taking-grammar (pauseGrammar: false, expects </think>yes|no) real wasm, CI lanes

Local emcc is 6.0.13 (pin 6.0.8), so the wasm build and grammar smoke rely on this PR's CI.

No candidate, qualification or publish workflow is dispatched; publishing assets needs maintainer approval.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant