Repository navigation
Add ChatTemplateEngine.parseRendered to parse a reply as the engine does - #1573
Merged
Merged
Conversation
The public parse takes only a format and flags, so it cannot see how a render left the prompt's thought: after the stock Seed-OSS template it returns reasoning Plan for Plan</seed:think>Sunny., where llama.cpp and the engine return content. parseRendered takes the render result and parses through the engine's own parseRenderedOutput with its thought marks. parse is unchanged.
10 of 13 tasks
Owner
Author
|
Stacked genkit half: #1574 (blocked on the core 0.17.0 publish). |
Contributor
|
Chat app preview removed for |
This was referenced Oct 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ChatTemplateEngine.parseRendered(LlamaChatTemplateResult rendered, String output, {isPartial, cancelled, parseToolCalls, tools})parses a reply from itsrender/engine.chatTemplateresult through the engine's ownparseRenderedOutput, reading the render's internal thought marks. Public callers now get the engine's final-message parse: after the stock Seed-OSS template,Plan</seed:think>Sunny.is content, as in llama.cpp 620d2225, whereChatTemplateEngine.parsestill returns reasoningPlan.ChatTemplateEngine.parseis unchanged. The marks stay internal; the result type is unchanged.ChatTemplateEngine.parseon the stock Seed-OSS template still returns reasoningPlan" (core half).genkit_llamadartswitches in a stacked PR that ships after the core 0.17.0 publish (see Out of scope).API shape: a static on
ChatTemplateEngine, besiderenderandparse, taking the render result. Not a method onLlamaChatTemplateResult, which would make the model type depend on the engine. Format, parser andthinkingForcedOpencome from the result, so callers cannot pass flags that disagree with it.Production-readiness scope
ChatTemplateEngine.renderorengine.chatTemplateand generates itself (e.g.genkit_llamadart's structured path) can parse exactly asengine.createdoes.fromJson) loses the marks and parses as an unmarked render; documented in the Dartdoc.genkit_llamadartswitch: stacked draft PR (link in comments) that raises its floor tollamadart ^0.17.0; its published-dependency and lower-bound lanes can pass only after 0.17.0 is on pub.dev, then a genkit release prep. The chat app's format-onlyparsecalls have no render result and stay.Completeness checklist
chat-template-and-parsing.md,template-engine-internals.md; both changelogs. README, support matrix andskills/do not describe template parsing.)ChatTemplateEnginestatics;validation_coverage_test.dartpasses)PR type guidance
Test Plan
dart run tool/prepare_workspace.dartdart format --output=none --set-exit-if-changed .dart analyzedart test -p vm -j 1 --exclude-tags local-onlydart test -p chrome --exclude-tags local-only— template/engine subset (test/unit/core/template test/unit/core/engine); pure Dart parsing.dart run tool/testing/check_platform_boundaries.dart;dart test -p vm test/unit/tooling/;./tool/docs/build_site.sh; Match llama.cpp's Seed-OSS and Cohere2 MoE parse when no thought is open #1569 oracle harness withparseRenderedin the public slot.Matrix Evidence
static-format-analyzeroot-vm-j 1 --exclude-tags local-onlyroot-chromeparseRenderedunit groupstructured-output-adversarialparseRenderedequals the engine-derived parse on every row; engine rows unchanged vs #1569 head; public slot vs oldparse: 0 OK->BAD, 2,208 BAD->OK (Seed-OSS 2,144, Cohere2 MoE 64)high-risk-exact-head-independent-qa/opt/UnitySrc/personal/llama/llamadart-evidence/pr-1573/evidence-1573-1510fc2.jsonrelease-metadata-verificationlitert-lm-json-schemaTests and mutation:
test/integration/core/engine/public_rendered_parse_test.dart(VM, fixture-driven):parseRenderedafterChatTemplateEngine.renderand afterChatTemplateRenderer.render(whatengine.chatTemplatereturns) matches llama.cpp 620d2225 (closed_thought_upstream.json) and the engine's stream for every closed-thought, open-thought and no-thought case, plus stock vs custom open-thought Seed-OSS with thinking on and off.chat_template_engine_test.dartgroupChatTemplateEngine.parseRendered(VM and Chrome): Seed-OSS stock vs opened (andparsestill returningPlan),isPartial,cancelled,parseToolCalls,tools, PEG parser.parseRendereddelegating toparseas callers can today (format,thinkingForcedOpen, parser), 9 tests fail, including the stock Seed-OSS case, closed-thought DeepSeek V3/Qwen (thinking tags became reasoning) and marked Cohere2 MoE.parse; drop the closed-for-thinking-off mark; drop the template-closed mark; dropthinkingForcedOpen; drop parser; ignoreisPartial,cancelled,parseToolCalls,tools).Review Notes
High-risk regression review