[leehack]

Summary

• Closes the remaining LiteRT-LM auto CPU fallback: Galaxy S24 device evidence (#1456 audit follow-up) #1458 decision: a LiteRT-LM auto engine failure on both GPU and CPU stays unremembered. The next call that needs an engine retries both, so a cause the caller can clear (an unwritable or full cache directory) recovers without reloading the model.
• Pins that contract with a model-free unit test and documents it in backend-selection.md.
• No lib/ change. Items 1, 2 and the item 4 harness mapping already landed in #1658; evidence below.

Refs #1458
Refs #1845

#1458 item	Status	Evidence
1. SPI context re-create after fallback under a metal hint	Already fixed (#1658, 0cf17cf94)	lib/src/backends/litert_lm/litert_lm_service.dart:1932-1938 exempts an unchanged automatic hint naming the load-time backend (_loadedBackend); test recreates a CPU fallback context with unchanged metal hints (test/unit/backends/litert_lm/litert_lm_service_test.dart:3152). Mutants re-run on this head: guard disabled -> killed with the issue's exact cannot change the loaded backend from cpu to gpu; device == auto guard dropped -> killed (explicit GPU context no longer refused).
2. Docs: CPU-resolved auto failure has no retry	Already fixed (#1658)	website/docs/guides/backend-selection.md ("When auto already resolves to CPU ... without a second attempt"). Code: _mayRetryAutoGpuOnCpu requires backend == gpu (litert_lm_service.dart:1693), default is CPU off Android/iOS/macOS arm64 (litert_lm_platform.dart:36), and a non-GPU/NPU device maps LiteRtLmEngineCreateError to LlamaModelException. Test: a CPU engine that fails to start under ComputeDevice.auto is not retried.
3. Repeated GPU+CPU failures retry both engines	Decided: do not remember; contract pinned here	See Review Notes. New test kills both memo mutants.
4a. Harness maps LiteRT auto to GpuBackend.cpu	Already fixed (#1658)	packages/llamadart_validation/lib/src/manifest.dart:766 maps only cpu to GpuBackend.cpu; test LiteRT profiles preserve automatic and explicit device selection. Mutant (auto -> cpu) killed.
4b. Galaxy S24 Firebase Test Lab run	Deferred: paid/device run needs maintainer authorization	Command below.

Production-readiness scope

• User-facing scope: Docs now state that a failed GPU+CPU pair under auto is retried on the next call, with another warning, and recovers without a reload.
• Supported platforms/paths: Native LiteRT-LM where auto resolves to GPU (Android, iOS, macOS arm64, or a GPU preferredBackend hint).
• Unsupported or intentionally unavailable paths: Unchanged. Explicit gpu/npu still throw LlamaUnsupportedException without a CPU retry; CPU-resolved auto throws LlamaModelException without a second attempt.
• Out of scope / follow-ups: Galaxy S24 auto device run (LiteRT-LM auto CPU fallback: Galaxy S24 device evidence (#1456 audit follow-up) #1458 item 4b), which needs a maintainer-authorized Firebase run.

Completeness checklist

• Declared scope is fully implemented, or explicitly reduced above.
• Unsupported platform/option combinations fail loudly with actionable diagnostics or are clearly documented as unavailable. (Unchanged.)
• Public API docs, README/website docs, examples, support matrices, and changelog entries are updated where relevant. (Website guide updated; no changelog entry because behavior is unchanged; no API, support-matrix, example or skill change.)
• Regression coverage covers the original issue plus key negative/version-skew paths where relevant.
• Public capability changes update the validation coverage registry with a pack or a reasoned, issue-linked exemption; inventory checks pass (or N/A with scope). N/A: no capability change.
• Security/privacy review completed: no secrets, bearer tokens, signed URLs, or raw secret-bearing paths leak through logs, cache keys, metadata, errors, or snapshots. (Synthetic test data only.)
• Follow-up work that is useful but not required for this PR is tracked in GitHub Issues, or explicitly marked "None" above. (LiteRT-LM auto CPU fallback: Galaxy S24 device evidence (#1456 audit follow-up) #1458 stays open for the S24 run.)

PR type guidance

• Feature PR: N/A, no new capability.
• Bugfix PR: N/A, no defect fixed here; the item 1 bug was fixed in fix: preserve automatic LiteRT backend on context recreation #1658.
• Docs-only PR: Docs plus one test; no runtime behavior changes.

Test Plan

• dart run tool/prepare_workspace.dart (clean on base 87aaec616)
• dart format --output=none --set-exit-if-changed . (0 changed)
• dart analyze (no issues)
• dart test -p vm -j 1 --exclude-tags local-only: affected litert_lm_service_test.dart 332 passed; Markdown-selected suites (tool/ci/markdown_test_suites.txt) 729 passed, 4 skipped; full suite in CI.
• dart test -p chrome --exclude-tags local-only: N/A, the changed test is @TestOn('vm') and no web code changed; CI runs the lane.
• coverage-lib matrix row: N/A, no lib/ change.
• Other: dart run tool/testing/check_platform_boundaries.dart OK; packages/llamadart_validation dart test -p vm 638 passed, 4 CLI tests timed out under parallel host load and passed on a serial rerun (110 passed, -j 1 --timeout 3x). Mutation checks listed in Review Notes.

Matrix Evidence

Matrix row	Scope covered	Platform / model / backend	Result	Evidence / notes
static-format-analyze	format, analyzer, web/native import boundaries	macOS arm64, Flutter 3.47.1	PASS	0 formatted, no analyzer issues, boundaries OK
root-vm	native-safe unit/integration coverage on the Dart VM	macOS arm64, fake LiteRT-LM runtime clients	PASS	affected suite 332 passed; full suite in CI
root-chrome	browser-compatible unit/integration coverage	N/A	N/A	VM-only test and docs; CI runs the lane
coverage-lib	>=70% line coverage for maintainable lib/ code	N/A	N/A	no lib/ change
validation-harness	private runner, planned model/backend coverage	macOS arm64, model-free	PASS	package unchanged; 638 + 4 serial-rerun tests pass; auto mapping mutant killed
docs-site	website build, docs metadata, generated docs routes	CI docs lane	PASS	CI Docs Build Check passed; local build_site.sh could not bind port 8080 (in use by another worktree)
high-risk-exact-head-independent-qa	independent review	N/A	N/A	classifier: standard

Review Notes

• Item 3 decision. Remembering the failure per loaded model is not simple and safe:

  • Engine options vary per request (_ensureClient rebuilds the engine for vision, audio, maxNumImages and speculative decoding), so a failed image request must not poison a later text request. A correct memo would need a key over those options plus both errors to keep the "throws the GPU error" contract.
  • Causes are recoverable without a reload today. The documented remedy for a v0.18 engine-create failure is a writable cache directory, and the directory is fixed at load, so a sticky memo would force a reload after the caller frees space or fixes permissions. Mobile GPU memory pressure is also transient.
  • A memo does not shorten the 5 s dispose window the audit flagged. The first failing request still tries GPU then CPU, and only later calls would be saved, and only when the model cannot run on that device at all.

• Mutation results (production restored after each; git status clean):

  Mutant	Test	Result
  Disable the automatic-hint recreate exemption (false && _automaticBackend)	recreates a CPU fallback context with unchanged metal hints	Killed: cannot change the loaded backend from cpu to gpu
  Drop params.device == ComputeDevice.auto from that exemption	same	Killed: explicit ComputeDevice.gpu context no longer throws
  Harness maps auto to GpuBackend.cpu	LiteRT profiles preserve automatic and explicit device selection	Killed: expected GpuBackend.auto, got GpuBackend.cpu
  After GPU+CPU fail, remember the CPU backend (_activeBackend = cpu)	new the next call after a failed CPU retry tries the GPU and the CPU again	Killed: second call skipped the GPU
  Memoize the GPU error per loaded model and rethrow it	same	Killed: second call rethrew the stored error

• S24 run (item 4b), not run here. It is a device/paid Firebase run. With a Galaxy S24 (SC-51E, API 36) row in an ignored firebase.json (see doc/cross_platform_validation.md, "Firebase setup"):

  # Local, uncommitted profile, as the 2026-10-08 `auto` row used.
  jq '.id = "chat-litert-auto" | .backend = "auto"' \
    packages/llamadart_validation/assets/profiles/chat-litert-gpu.json \
    > packages/llamadart_validation/assets/profiles/chat-litert-auto.json
  dart run tool/testing/validation.dart build --target android --profile chat-litert-auto --out .dart_tool/validation/bundles/android-litert-auto
  dart run tool/testing/validation.dart plan --target firebase-android --config .dart_tool/validation/firebase.json --bundle .dart_tool/validation/bundles/android-litert-auto --out .dart_tool/validation/plan-s24-litert-auto.json
  dart run tool/testing/validation.dart run --plan .dart_tool/validation/plan-s24-litert-auto.json

  Expected: Qwen3 0.6B fails GPU engine creation on Adreno 750 (binding-size limit), the cases then pass on the CPU, and logcat shows one fallback warning.

• Coordination: lib/src/backends/litert_lm/litert_lm_service.dart is not touched. The test is one added block in the ComputeDevice.auto GPU engine fallback group.

• Independent review status: N/A, standard risk.

• CI status / head SHA: all checks green (pass or intentionally skipped), 0 failures, head 43877dd11583cc3cfd98defdbbb45de821528386.

High-risk regression review

• Classification: standard
• Implementation task: N/A
• Independent blocking QA task: N/A
• Exact head / current base: N/A
• Production-branch deletion, bypass, or miswire proof: N/A
• Affected-family real-model/artifact evidence: N/A
• Explicit unavailable-family or other N/A evidence: N/A
• Known PR-caused P1 regressions: 0
• Unresolved review threads: 0

[leehack]

Landed through the 0.19.2 batch integration PR #1870 (rebase merge, one commit per constituent; this PR's pinned head was audited/CI-gated there).

[github-actions]

Chat app preview removed for leehack/llamadart-chat-pr-1856.