Skip to content

Add ChatTemplateEngine.parseRendered to parse a reply as the engine does - #1573

Merged
leehack merged 1 commit into
mainfrom
feat/public-rendered-parse
Oct 29, 2026
Merged

leehack merged 1 commit into
mainfrom
feat/public-rendered-parse

Conversation

@leehack

@leehack leehack commented Oct 29, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • New ChatTemplateEngine.parseRendered(LlamaChatTemplateResult rendered, String output, {isPartial, cancelled, parseToolCalls, tools}) parses a reply from its render / engine.chatTemplate result through the engine's own parseRenderedOutput, reading the render's internal thought marks. Public callers now get the engine's final-message parse: after the stock Seed-OSS template, Plan</seed:think>Sunny. is content, as in llama.cpp 620d2225, where ChatTemplateEngine.parse still returns reasoning Plan.
  • ChatTemplateEngine.parse is unchanged. The marks stay internal; the result type is unchanged.
  • Clears the #1565 item "public ChatTemplateEngine.parse on the stock Seed-OSS template still returns reasoning Plan" (core half). genkit_llamadart switches in a stacked PR that ships after the core 0.17.0 publish (see Out of scope).

API shape: a static on ChatTemplateEngine, beside render and parse, taking the render result. Not a method on LlamaChatTemplateResult, which would make the model type depend on the engine. Format, parser and thinkingForcedOpen come from the result, so callers cannot pass flags that disagree with it.

Production-readiness scope

  • User-facing scope: Code that renders with ChatTemplateEngine.render or engine.chatTemplate and generates itself (e.g. genkit_llamadart's structured path) can parse exactly as engine.create does.
  • Supported platforms/paths: All (pure Dart; native and Web).
  • Unsupported or intentionally unavailable paths: A copy of the result (constructor, fromJson) loses the marks and parses as an unmarked render; documented in the Dartdoc.
  • Out of scope / follow-ups: genkit_llamadart switch: stacked draft PR (link in comments) that raises its floor to llamadart ^0.17.0; its published-dependency and lower-bound lanes can pass only after 0.17.0 is on pub.dev, then a genkit release prep. The chat app's format-only parse calls have no render result and stay.

Completeness checklist

  • Declared scope is fully implemented, or explicitly reduced above.
  • Unsupported platform/option combinations fail loudly with actionable diagnostics or are clearly documented as unavailable. (copies documented above)
  • Public API docs, README/website docs, examples, support matrices, and changelog entries are updated where relevant. (Dartdoc and class example; chat-template-and-parsing.md, template-engine-internals.md; both changelogs. README, support matrix and skills/ do not describe template parsing.)
  • Regression coverage covers the original issue plus key negative/version-skew paths where relevant.
  • Public capability changes update the validation coverage registry with a pack or a reasoned, issue-linked exemption; inventory checks pass (or N/A with scope). (N/A: parse helper, not an engine capability; the inventory does not list ChatTemplateEngine statics; validation_coverage_test.dart passes)
  • Security/privacy review completed: no secrets, bearer tokens, signed URLs, or raw secret-bearing paths leak through logs, cache keys, metadata, errors, or snapshots.
  • Follow-up work that is useful but not required for this PR is tracked in GitHub Issues, or explicitly marked "None" above.

PR type guidance

  • Feature PR: additive public API; tests through the public API on both happy and negative paths (stock vs custom open-thought Seed-OSS, Cohere2 MoE marked and unmarked, closed-thought DeepSeek V3/Qwen/Qwen3.5, PEG parser rejection, option forwarding).
  • Bugfix PR: N/A. Docs-only PR: N/A.

Test Plan

  • dart run tool/prepare_workspace.dart
  • dart format --output=none --set-exit-if-changed .
  • dart analyze
  • dart test -p vm -j 1 --exclude-tags local-only
  • dart test -p chrome --exclude-tags local-only — template/engine subset (test/unit/core/template test/unit/core/engine); pure Dart parsing.
  • Other: dart run tool/testing/check_platform_boundaries.dart; dart test -p vm test/unit/tooling/; ./tool/docs/build_site.sh; Match llama.cpp's Seed-OSS and Cohere2 MoE parse when no thought is open #1569 oracle harness with parseRendered in the public slot.

Matrix Evidence

Matrix row Scope covered Platform / model / backend Result Evidence / notes
static-format-analyze format, analyzer, boundaries macOS, Flutter 3.47.1 PASS 0 changed, no issues, boundaries OK
root-vm VM unit/integration macOS arm64 PASS (+9654 ~126) -j 1 --exclude-tags local-only
root-chrome browser template/engine subset Chrome PASS (+2139 ~1) includes the new parseRendered unit group
structured-output-adversarial final parse vs upstream after every render kind, tools on/off, thinking on/off llama.cpp 620d2225 oracle (#1569 harness, main corpus) PASS 185,160 rows: parseRendered equals the engine-derived parse on every row; engine rows unchanged vs #1569 head; public slot vs old parse: 0 OK->BAD, 2,208 BAD->OK (Seed-OSS 2,144, Cohere2 MoE 64)
high-risk-exact-head-independent-qa independent blocking review of exact head fresh auditor session, exact head 1510fc2 PASS CLEAR: CLEAR at 1510fc2, 0 P1/P2; Dartdoc P3s on #1565; evidence /opt/UnitySrc/personal/llama/llamadart-evidence/pr-1573/evidence-1573-1510fc2.json
release-metadata-verification core-patch metadata — N/A no release metadata change
litert-lm-json-schema LiteRT-LM JSON Schema — N/A no LiteRT-LM path touched

Tests and mutation:

  • test/integration/core/engine/public_rendered_parse_test.dart (VM, fixture-driven): parseRendered after ChatTemplateEngine.render and after ChatTemplateRenderer.render (what engine.chatTemplate returns) matches llama.cpp 620d2225 (closed_thought_upstream.json) and the engine's stream for every closed-thought, open-thought and no-thought case, plus stock vs custom open-thought Seed-OSS with thinking on and off.
  • chat_template_engine_test.dart group ChatTemplateEngine.parseRendered (VM and Chrome): Seed-OSS stock vs opened (and parse still returning Plan), isPartial, cancelled, parseToolCalls, tools, PEG parser.
  • Failing-first: with parseRendered delegating to parse as callers can today (format, thinkingForcedOpen, parser), 9 tests fail, including the stock Seed-OSS case, closed-thought DeepSeek V3/Qwen (thinking tags became reasoning) and marked Cohere2 MoE.
  • Mutation: 9 of 9 killed (delegate to parse; drop the closed-for-thinking-off mark; drop the template-closed mark; drop thinkingForcedOpen; drop parser; ignore isPartial, cancelled, parseToolCalls, tools).

Review Notes

  • Independent review status: CLEAR at exact head 1510fc2 by a fresh auditor session (0 P1, 0 P2).
  • CI status / head SHA: 1510fc2

High-risk regression review

  • Classification: high-risk (surfaces: structuredOutput)
  • Implementation task: llamadart 0.17.0 lane for the #1565 genkit-parse item
  • Independent blocking QA task: fresh auditor session, CLEAR, 0 known PR-caused P1, 0 unresolved threads
  • Exact head / current base: 1510fc2 / 04821e0 (merge-tree clean)
  • Production-branch deletion, bypass, or miswire proof: mutation above (9/9 killed); failing-first 9 failures
  • Affected-family real-model/artifact evidence: unavailable (Seed-OSS, Cohere2 MoE weights); primary upstream emissions via llama.cpp 620d2225 plus durable fixtures
  • Explicit unavailable-family or other N/A evidence: Seed-OSS, Cohere2 MoE GGUF weights unavailable
  • Known PR-caused P1 regressions: 0
  • Unresolved review threads: 0

The public parse takes only a format and flags, so it cannot see how a render
left the prompt's thought: after the stock Seed-OSS template it returns
reasoning Plan for Plan</seed:think>Sunny., where llama.cpp and the engine
return content. parseRendered takes the render result and parses through the
engine's own parseRenderedOutput with its thought marks. parse is unchanged.
@leehack

leehack commented Oct 29, 2026

Copy link
Copy Markdown
Owner Author

Stacked genkit half: #1574 (blocked on the core 0.17.0 publish).

@leehack
leehack marked this pull request as ready for review October 29, 2026 01:24
@leehack
leehack merged commit 176e12e into main Oct 29, 2026
38 checks passed
@leehack
leehack deleted the feat/public-rendered-parse branch October 29, 2026 01:24
@leehack leehack mentioned this pull request Oct 29, 2026
29 of 32 tasks
@github-actions

Copy link
Copy Markdown
Contributor

Chat app preview removed for leehack/llamadart-chat-pr-1573.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant