Skip to content

Name the bundle template in a failed C API 1.0 LiteRT-LM chat send - #1875

Closed
leehack wants to merge 2 commits into
mainfrom
fix/1868-v1-send-note
Closed

leehack wants to merge 2 commits into
mainfrom
fix/1868-v1-send-note

Conversation

@leehack

@leehack leehack commented Nov 10, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • A failed native LiteRT-LM chat send on the C API 1.0 runtime now gets the bundle-template / ModelParams.chatTemplate note, as the legacy runtime already did. _generateChat caught only StateError.
  • The note is added by where the failure is thrown, not by message text: the v1 conversation marks a failed send call (conversation_send_message_stream, where template failures surface) as LiteRtLmSendFailureException (a LlamaModelException), and only that type gets the note, keeping its type and details. Failures the stream reports later (C API 1.0 stream failed, such as a context overflow) and LiteRtLmV1CopyLossException pass through unchanged, as do other LlamaExceptions.

Fixes #1868. Refs #1874.

Production-readiness scope

  • User-facing scope: the explanatory note on a failed chat send on the default (C API 1.0) native runtime. Diagnostic only; the request fails with the native error either way.
  • Supported platforms/paths: native LiteRT-LM, both clients.
  • Unsupported or intentionally unavailable paths: N/A.
  • Out of scope / follow-ups: None.

Completeness checklist

  • Declared scope is fully implemented.
  • Unsupported platform/option combinations: N/A, no change.
  • Changelog updated (both files); no public API or docs change.
  • Regression coverage: legacy and v1 fakes throw what each client throws, for both notes; a v1 stream failure (context-overflow shape) gets no note, with and without a custom template; details preserved; another LlamaException unchanged; v1 conversation tests assert the send-call failure type and that stream failures are not marked.
  • Validation registry: N/A, no capability change.
  • Security/privacy: no new data in messages; the note text is unchanged.
  • Follow-ups: None.

PR type guidance

  • Bugfix PR: root cause above; tests in litert_lm_service_test.dart; native platforms only.

Test Plan

  • dart run tool/prepare_workspace.dart
  • dart format --output=none --set-exit-if-changed .
  • dart analyze
  • dart test -p vm -j 1 --exclude-tags local-only (CI)
  • dart test -p chrome --exclude-tags local-only (CI)
  • coverage-lib N/A: the new branch is covered by the unit tests above; main enforces the threshold.
  • TMPDIR=$(mktemp -d) dart test -p vm -j 1 test/unit/backends/litert_lm/litert_lm_service_test.dart

Mutations (each fails the tests): retyping the v1 note as StateError; catching any LlamaModelException (note on stream failures); dropping details; retyping to plain LlamaModelException; not marking the send failure in the v1 conversation; dropping the catch.

Matrix Evidence

Matrix row Scope covered Platform / model / backend Result Evidence / notes
static-format-analyze format, analyze macOS PASS local
root-vm litert_lm service unit tests macOS VM PASS test/unit/backends/litert_lm 1542 tests
high-risk-exact-head-independent-qa exact-head audit - PENDING classified high-risk (backendRuntime); draft until audited

Review Notes

  • Independent review status: audit of df7cf70e2 passed with 3 findings, fixed in 03d50e48e.
  • CI status / head SHA: pending.

High-risk regression review

  • Classification: high-risk (backendRuntime, from classify_high_risk_changes.dart; the change is a diagnostic-only catch clause)
  • Implementation task: this PR
  • Independent blocking QA task: pending

Independent audit

Exact head 03d50e48e8f9c40bf1bb56f0116513049a8a26d7 against main f78cafa97: PASS (delta re-audit after the first pass), 0 blocking, 0 PR-caused P1, 0 threads. Real model (Qwen3-0.6B.litertlm, CPU): a custom template failing at send keeps the note; context overflow gets no note; cancel and normal requests unchanged. 11/12 mutants killed (survivor is unobservable today). Readiness evidence v2: local run exits 2. Non-blocking items tracked on #1861.

LiteRtLmService._generateChat added the bundle-template note only to the
legacy client's StateError. The C API 1.0 client reports a failed send as
LlamaModelException, so it lost the note. Route it through the same path,
keeping its exception type.

Refs #1868, #1874
…ures

The note went to every LlamaModelException before output, so a context
overflow reported later by the stream ("C API 1.0 stream failed") got it
too. The v1 conversation now marks a failed send call with
LiteRtLmSendFailureException and only that type gets the note; stream
failures and LiteRtLmV1CopyLossException pass through unchanged.

Refs #1868, #1874
@leehack
leehack marked this pull request as ready for review November 10, 2026 02:05
@github-actions

Copy link
Copy Markdown
Contributor

@leehack

leehack commented Nov 10, 2026

Copy link
Copy Markdown
Owner Author

Landed on main via batch integration PR #1895 (rebase merge, main ba4e0e1b; tree identical to the audited integration head 5ffd1224).

@leehack leehack closed this Nov 10, 2026
@leehack
leehack deleted the fix/1868-v1-send-note branch November 10, 2026 13:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

LiteRT-LM C API 1.0 chat send failures miss the bundle template note

1 participant