Skip to content

genkit_llamadart: resolve the thinking budget through core (#1929) - #1952

Merged
leehack merged 5 commits into
mainfrom
lane-1929-genkit
Nov 12, 2026
Merged

leehack merged 5 commits into
mainfrom
lane-1929-genkit

Conversation

@leehack

@leehack leehack commented Nov 12, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • genkit_llamadart builds its ThinkingBudget with ChatTemplateEngine.resolveThinkingBudget(template.format, budget) instead of hand-built tags, so Qwen3.5 / Qwen3-Coder XML closes a spent budget with \n</think> rather than a bare </think>.
  • Raises the package floor to llamadart: ^0.19.7 and the matching lower-bound pins (ci.yml, publish_genkit_pubdev.yml, genkit-package.md, ci_workflow_selection_test.dart); adds an Unreleased package changelog entry. No genkit version bump (separate release-prep PR).

Fixes #1929

Production-readiness scope

  • User-facing scope: A spent thinking budget on Qwen3-Coder XML templates (Qwen3.5) closes the thought with the template's \n</think> instead of a bare </think>. This does not make Qwen3.5 constrained-JSON output with a budget work end to end: a separate pre-existing bug, Qwen3.5: constrained JSON with a thinking budget ignores the grammar after the forced </think> #1953, still breaks that.
  • Supported platforms/paths: Genkit structured-output path (_runStructuredModel), where Genkit renders the prompt itself (rendersPrompt: true).
  • Unsupported or intentionally unavailable paths: None added. A caller-set endTag gets no prefix.
  • Out of scope / follow-ups: Genkit release (needs separate maintainer approval; next version: minor, 3.9.0, since the llamadart floor moves, as in 3.6.0 to 3.8.0).

Completeness checklist

  • Declared scope is fully implemented, or explicitly reduced above.
  • Unsupported platform/option combinations fail loudly or are documented as unavailable (N/A: no new options).
  • Changelog updated (package CHANGELOG.md). The genkit docs usage.md, migration.md and tool-input-schemas.md are updated for the \n close prefix and the ^0.19.7 floor. No public API change, so no skills change.
  • Regression coverage covers the original issue plus the caller-endTag path.
  • Validation coverage registry: N/A, no public capability change.
  • Security/privacy review: no secrets or paths touched.
  • Follow-ups: None.

PR type guidance

  • Bugfix PR: root cause is the hand-built budget in model_action.dart duplicating, and missing, core's \n close prefix for Qwen3-Coder XML. Regression test: inference_controls_test.dart, "Qwen3-Coder XML thinking budget". Platforms: all that run the structured path.

Test Plan

  • dart run tool/prepare_workspace.dart (N/A: only the independently resolved adapter changed)
  • dart format --output=none --set-exit-if-changed . (root: 0 changed)
  • dart analyze (adapter: dart analyze --fatal-infos, no issues)
  • dart test -p vm -j 1 --exclude-tags local-only N/A: no root code changed; root tests skip packages/
  • dart test -p chrome --exclude-tags local-only N/A: same
  • coverage-lib N/A: no lib/ change
  • Other: adapter dart format, python3 .github/check_docs.py, dart test -x real-model (474 passed)

Matrix Evidence

Matrix row Scope covered Platform / model / backend Result Evidence / notes
genkit-contracts adapter analyze, docs, tests against published llamadart 0.19.7 macOS, fake runtime PASS local run; hosted CI adds lower-bound and local-core legs
structured-output-adversarial thinking-budget close prefix on Qwen3-Coder XML fake runtime capturing params.thinkingBudget PASS forced message + \n, \n alone, none with caller endTag; no real model
genkit-native-cpu, genkit-release not run N/A N/A no native or release-routing change

Mutation check: reverting to the old construction fails the two prefix tests; rendersPrompt: false fails the same two.

Review Notes

  • Independent review status: deep audit PASS at f46824cc; delta re-audit PASS at d8b49b28
  • CI status / head SHA: all checks green at d8b49b2

High-risk regression review

  • Classification: high-risk (surfaces: artifactConsumer, backendRuntime, regressionPolicy, releaseAutomation, structuredOutput; the workflow pin touches the genkit publisher)
  • Implementation task: this PR (lane-1929-genkit)
  • Independent blocking QA task: fresh auditor sessions (not the author). Genkit and core agreed in all 576 parity cases; zero PR-caused P1s and zero unresolved threads. P3s go to 0.19.8 audit follow-ups #1950.
  • Exact head / current base: d8b49b2 / 4b96fde
  • Production-branch deletion, bypass, or miswire proof: mutation check above (old construction and rendersPrompt: false both fail the test)
  • Affected-family real-model/artifact evidence: N/A: Qwen3.5 weights not run; template-level fake-runtime evidence only (core Add ChatTemplateEngine.resolveThinkingBudget (Refs #1929) #1943 carries the parity evidence)
  • Explicit unavailable-family or other N/A evidence: the Qwen3 and Qwen3.5 real-model tests skip in hosted CI. The audit's local real Qwen3.5-0.8B probe showed genkit and core giving byte-identical output (llamadart-evidence/pr-1952/prior-f46824cc/qwen35-raw-probe.log). End-to-end constrained JSON with a budget still fails until Qwen3.5: constrained JSON with a thinking budget ignores the grammar after the forced </think> #1953 is fixed.
  • Known PR-caused P1 regressions: 0
  • Unresolved review threads: 0

@leehack
leehack marked this pull request as ready for review November 12, 2026 14:01
@leehack
leehack merged commit 9aa0837 into main Nov 12, 2026
41 checks passed
@leehack
leehack deleted the lane-1929-genkit branch November 12, 2026 14:01
@github-actions

Copy link
Copy Markdown
Contributor

Chat app preview removed for leehack/llamadart-chat-pr-1952.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

genkit_llamadart: thinking budget forces a bare </think> on Qwen3-Coder XML templates

1 participant