Skip to content

Document LiteRT-LM Web generate() prompt wrapping - #1876

Closed
leehack wants to merge 1 commit into
mainfrom
fix/1863-web-generate
Closed

leehack wants to merge 1 commit into
mainfrom
fix/1863-web-generate

Conversation

@leehack

@leehack leehack commented Nov 10, 2026

Copy link
Copy Markdown
Owner

Summary

  • Docs only: state precisely that LiteRT-LM Web generate(prompt) wraps the prompt in one user turn of the bundle's chat template, and why no workaround ships.
  • Runtime finding: @litert-lm/[email protected] (and 0.20.0-nightly.20261108) offers Engine.createSession() with applyPromptTemplateInSession, but Session only has runPrefill(string[]) and runDecode(), which resolves one whole reply: no streaming, no stop cause, no per-token cancel. Using it would change generate() streaming semantics, so the conversation route stays. No runtime/Dart behavior change.
  • Docs touched: generation guide, backend selection, support matrix note, llamadart-chat-streaming SKILL.md, both changelogs.

Refs #1863 (stays open: runtime-limited), #1874.

Production-readiness scope

  • User-facing scope: accurate documentation of the Web generate() contract.
  • Supported platforms/paths: LiteRT-LM Web docs; no code change.
  • Unsupported or intentionally unavailable paths: raw generate() on LiteRT-LM Web stays templated until @litert-lm/core offers a streaming raw route or template replacement.
  • Out of scope / follow-ups: #1863 stays open.

Completeness checklist

  • Declared scope is fully implemented, or explicitly reduced above.
  • Unsupported combinations: documented as unavailable.
  • Docs, support matrix, skill and changelog updated.
  • Regression coverage: N/A, docs only (validation_coverage_test, package_skills_test pass).
  • Validation registry: N/A, no capability change (a table row was avoided because it would need a mapping).
  • Security/privacy: N/A.
  • Follow-ups tracked: LiteRT-LM Web generate() wraps the raw prompt in the bundle's user turn #1863.

PR type guidance

  • Docs-only PR: no runtime behavior changes. Validated with the markdown test suites in tool/ci/markdown_test_suites.txt; the ./tool/docs/build_site.sh link check could not start its server locally (port in use), so it is left to CI.

Test Plan

  • dart format --output=none --set-exit-if-changed .
  • dart test -p vm -j 1 on tool/ci/markdown_test_suites.txt (after fixing the validation-coverage failure)
  • Remaining lanes: CI. N/A locally: no code changed.

Matrix Evidence

Matrix row Scope covered Platform / model / backend Result Evidence / notes
docs-site docs build - CI local build_site.sh server failed to start
root-vm markdown suites macOS VM PASS

Review Notes

  • Independent review status: N/A, docs-only.
  • CI status / head SHA: pending.

High-risk regression review

  • Classification: standard
  • Implementation task: N/A
  • Independent blocking QA task: N/A

@litert-lm/core 0.19.0 and the 0.20 nightly expose no raw streaming route:
the Session API decodes a whole reply per call, with no streaming or stop
cause, so generate() keeps the conversation route. State the Web behaviour
in the generation guide, backend selection, support matrix and chat
streaming skill.

Refs #1863, #1874
@github-actions

Copy link
Copy Markdown
Contributor

@leehack

leehack commented Nov 10, 2026

Copy link
Copy Markdown
Owner Author

Landed on main via batch integration PR #1895 (rebase merge, main ba4e0e1b; tree identical to the audited integration head 5ffd1224).

@leehack leehack closed this Nov 10, 2026
@leehack
leehack deleted the fix/1863-web-generate branch November 10, 2026 13:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant