Skip to content

Add ChatTemplateEngine.resolveThinkingBudget (Refs #1929) - #1943

Merged
leehack merged 2 commits into
mainfrom
lane-1929-core-api
Nov 11, 2026
Merged

leehack merged 2 commits into
mainfrom
lane-1929-core-api

Conversation

@leehack

@leehack leehack commented Nov 11, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • Add public ChatTemplateEngine.resolveThinkingBudget(int formatIndex, ThinkingBudget budget, {bool rendersPrompt = true}), which fills a budget's omitted start/end tags and the Qwen3-Coder XML line-break close prefix, exactly as LlamaEngine does.
  • ChatCompletionRequestPlanner now calls the same static, so the logic lives in one place; planner behavior is unchanged.
  • Lets genkit_llamadart (which renders the prompt itself) match core. Genkit adoption follows the 0.19.7 release in a separate PR.

Refs #1929; plan #1939.

Production-readiness scope

  • User-facing scope: Integrations that call ChatTemplateEngine.thinkingTagsFor and run runtime.generate themselves can get the same resolved ThinkingBudget as core, including the \n close prefix on Qwen3-Coder XML templates.
  • Supported platforms/paths: Pure Dart template logic; all platforms.
  • Unsupported or intentionally unavailable paths: rendersPrompt: false (backends that take chat messages and close the thought themselves) adds no prefix, as before.
  • Out of scope / follow-ups: genkit_llamadart adoption (separate PR after the core 0.19.7 release, #1929).

Completeness checklist

  • Declared scope is fully implemented, or explicitly reduced above.
  • Unsupported platform/option combinations fail loudly with actionable diagnostics or are clearly documented as unavailable. (N/A: no platform dependence.)
  • Public API docs, README/website docs, examples, support matrices, and changelog entries are updated where relevant. (/// Dartdoc, both changelogs; no README/skill/example mentions thinkingTagsFor.)
  • Regression coverage covers the original issue plus key negative/version-skew paths where relevant.
  • Public capability changes update the validation coverage registry: N/A, a helper on an existing public class, not a new capability; validation_coverage_test.dart passes.
  • Security/privacy review completed: no secrets or paths involved.
  • Follow-up work tracked: genkit adoption above.

PR type guidance

  • Feature PR: public static with Dartdoc and tests for the happy path (Qwen3-Coder XML, forced-message append), no-prefix format, caller end tag and rendersPrompt: false.

Test Plan

  • dart run tool/prepare_workspace.dart
  • dart format --output=none --set-exit-if-changed .
  • dart analyze (no issues)
  • dart test -p vm -j 1 --exclude-tags local-only: macOS, 14086 passed; only the 6 windows_registry_qualification_test.dart cases failed locally (no pwsh)
  • dart test -p chrome --exclude-tags local-only: left to CI; the change is platform-neutral Dart
  • coverage-lib: 93.79% (threshold 70)
  • Targeted: test/unit/core/template, test/unit/core/engine, test_structure, validation_coverage_test.dart, check_platform_boundaries all pass.

New test test/unit/core/template/thinking_budget_resolution_test.dart failed to load before the change. Mutations, each caught:

Mutation Failing test
drop rendersPrompt && no prefix when backend does not render
drop budget.endTag == null caller end tag left as given
condition forced false Qwen3-Coder XML newline
replace append with closePrefix appends to forced message
ignore caller startTag caller end tag/start tag as given
ignore caller endTag caller end tag/start tag as given

Matrix Evidence

Matrix row Scope covered Platform / model / backend Result Evidence / notes
coverage-lib >=70% line coverage for lib/ macOS VM PASS 93.79% line coverage, local
high-risk-exact-head-independent-qa independent blocking-only review of exact head b103a1c PASS evidence llamadart-evidence/pr-1943/ (evaluator exit 2); 11/11 change mutants killed
structured-output-adversarial thinking/budget parity N/A N/A Planner output unchanged; the existing thinking_budget_close_parity_test.dart (#1924 upstream fixtures) passes unchanged
release-metadata-verification, litert-lm-*, chat-app-litert-json-schema N/A N/A No release metadata, LiteRT-LM or device change

Review Notes

  • Independent review status: independent audit PASS at b103a1cb. Planner output matched main's inline logic in all 1176 cases tried.
  • CI status / head SHA: green (32 pass, 9 skipping), b103a1c

High-risk regression review

  • Classification: high-risk (backendRuntime, structuredOutput; chat_completion_request_planner.dart)
  • Implementation task: lane-1929-core-api
  • Independent blocking QA task: a fresh auditor session (not the author) passed it, with zero PR-caused P1s and zero unresolved threads. The Dartdoc P3s are on 0.19.7 audit follow-ups #1938.
  • Exact head / current base: b103a1c / 81ba323
  • Production-branch deletion, bypass, or miswire proof: planner call site now delegates to the static; mutations above and the unchanged Close a Qwen3-Coder XML thinking budget with a line break (#1914) #1924 parity test cover it
  • Affected-family real-model/artifact evidence: Qwen3-Coder XML upstream fixtures (thinking_budget_close_upstream.json) via the unchanged parity test; no model run
  • Explicit unavailable-family or other N/A evidence: none
  • Known PR-caused P1 regressions: 0
  • Unresolved review threads: 0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant