Repository navigation
Close genkit 0.17 audit follow-ups - #1587
Merged
Merged
Conversation
Pin that structured output without declared tools parses no tool call, so a user schema with a top-level response or tool_call field is returned as the model wrote it. Note the Qwen3.5 thinking-off parse change in the migration guide, and move the lower-bound and LiteRT-LM references to llamadart 0.17.0.
Contributor
|
Chat app preview removed for |
This was referenced Oct 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
responseortool_callfield gets the reply exactly as the model wrote it, as the engine's final parse does (parseToolCallsonly with tools). The grammar can produce this output, so the contract is reachable. It kills audit mutant m3 (parseToolCalls: true), which unwraps{"response":…}or turns{"tool_call":…}into a dropped call.<think>…</think>{...}as content, which fails JSON validation, as Seed-OSS already does.genkit-package.mdlower-bound lane: published llamadart 0.17.0 (the CI pin since Parse genkit structured output with ChatTemplateEngine.parseRendered #1574).CONTRIBUTING.md: llamadart 0.17.0 useslitert-lm-nativev0.18.1-4(liteRtLmReleaseTagatv0.17.0).No new tests for mutants m6 and m8. Under the inputs
model_actioncan reach they change nothing observable (probe ofparseRenderedon hosted 0.17.0 with the Qwen3.5, Seed-OSS, Qwen3-4B and generic renders, thinking on and off, with and without tools):tools: null): tools reach the parse only on the generic format (the action rejects any other format with tools). There, a parse with and withouttoolsgave identical results for every output tried, including undeclared names and mistyped or string-encoded arguments.isPartial: true): with tools, the partial parse differs only on output that is not one complete JSON envelope. The unmutated envelope parse then rejects it withfailedPreconditionanyway. Without tools, it differs only by trimming trailing whitespace from the visible JSON, and pinning that would assert nothing users rely on.Production-readiness scope
website/docs/integrations/genkit.mdandchoosing-an-api.md(3.5.0 / llamadart 0.16.1 today), as #1368 did for 3.5.0. They must name published versions, so the update cannot land before publication.Completeness checklist
CHANGELOG.md.PR type guidance
Test Plan
Pinned Flutter 3.47.1, macOS arm64, head
ad876415a927f4688617c6b3a940109478e6d09e.dart run tool/prepare_workspace.dart: no tracked changes.dart format --output=none --set-exit-if-changed .: 0 changed.dart analyze: no issues.dart test -p vm -j 1 --exclude-tags local-only: N/A, no core change;dart test -p vm test/unit/tooling/+1240 passed (~5 skipped).dart test -p chrome --exclude-tags local-only: N/A, no core change.packages/genkit_llamadartagainst hosted llamadart 0.17.0:dart format,dart analyze --fatal-infos,python3 .github/check_docs.py,.githubunittest: pass.dart test -x real-model: +468 passed.stage_genkit_package.pyto a fresh directory, thendart pub publish --dry-run: 0 warnings.model_action.dart: the new test fails; restored: passes../tool/docs/build_site.sh: pass.verify_release_docs_versions.dartin a full clone: pass.Matrix Evidence
genkit-contractsdocs-sitebuild_site.shexit 0release-doc-version-consistencygenkit-native-cpuReview Notes
ad876415a927f4688617c6b3a940109478e6d09e.High-risk regression review
git diff --name-only --no-renames origin/main...HEAD | dart run tool/testing/classify_high_risk_changes.dart:regressionPolicycomes frompackages/genkit_llamadart/CONTRIBUTING.md; this PR changes one version reference in it.