Skip to content

Pin extraction marker failure paths, the not-published list and Qwen3.5 structured parse - #1605

Merged
leehack merged 1 commit into
mainfrom
fix/1565-carry-overs
Oct 30, 2026
Merged

leehack merged 1 commit into
mainfrom
fix/1565-carry-overs

Conversation

@leehack

@leehack leehack commented Oct 30, 2026 •

Copy link
Copy Markdown
Owner

Summary

Tests only, for the #1565 carry-overs in the 0.17.1 plan #922:

  • Marker gaps A3, A5, A8 (#1584 audit): a marker write that fails after creating .tmp removes it and rethrows the original error, including when the cleanup fails too. IOOverrides supplies the failure; no real directory permission can fail a write after the file exists.
  • Gap A9: the "not published" list must equal the stable_diffusion bundles resolveNativeBundleSpec and stableDiffusionBundleForNativeBundle produce, so it follows the hook rather than a copied string.
  • Durable Qwen3.5 test (claim in doc/migration.md, from #1587): through the Genkit model action and the vendored template, thinking off keeps <think>…</think>{...} as content and fails with failedPrecondition, thinking off returns plain JSON, and thinking on returns reasoning and JSON. Upstream values for this template are in test/fixtures/closed_thought_upstream.json (llama.cpp 620d2225).

The LFM2 grammar carry-over is a separate PR because it is high-risk.

Production-readiness scope

  • User-facing scope: None, tests only.
  • Supported platforms/paths: VM unit tests; the Genkit test reads ../../test/fixtures/templates/Qwen3_5-0_8B.jinja, which every CI lane has because each runs inside the checkout.
  • Unsupported or intentionally unavailable paths: N/A, no runtime change.
  • Out of scope / follow-ups: None.

Completeness checklist

  • Declared scope is fully implemented, or explicitly reduced above.
  • Unsupported platform/option combinations: N/A, no runtime change.
  • Public API docs, README/website docs, examples, support matrices, and changelog entries: N/A, nothing users can observe.
  • Regression coverage: see the mutation table.
  • Public capability changes: N/A, no capability change.
  • Security/privacy review: tests hold no secrets.
  • Follow-up work: None.

PR type guidance

  • Feature PR: N/A.
  • Bugfix PR: N/A, closes test gaps only.
  • Docs-only PR: N/A.

Test Plan

  • dart run tool/prepare_workspace.dart
  • dart format --output=none --set-exit-if-changed on the changed files
  • dart analyze on the changed files; dart analyze --fatal-infos on the Genkit test
  • dart test -p vm: native_extraction_marker_test.dart, stable_diffusion_runtime_io_test.dart, mirrored_unit_structure_test.dart, hook_cache_isolation_test.dart
  • Genkit against checkout core: parses_structured_output_from_its_render_test.dart
  • Full VM, Chrome and Genkit lanes: CI pass

Matrix Evidence

Matrix row Scope covered Platform / model / backend Result Evidence / notes
genkit-contracts local-core resolution for the new Qwen3.5 test macOS arm64, VM PASS (targeted), CI for the rest no model; fake runtime with the vendored template
Mutant Result Killed by
A3 write outside try killed marker write that fails leaves no .tmp
A5 rethrow replaced by a new exception killed both new marker tests (same(failure))
A8 inner cleanup catch removed killed .tmp that cannot be deleted does not hide the write error
A9 old wording killed list equals the hook's bundles
A9b list drops linux-arm64 killed list equals the hook's bundles
Q1 model_action parses by format instead of by render killed Qwen3.5 thinking off keeps a thought in the content

Review Notes

  • Independent review status: N/A, standard risk, tests only.
  • CI status / head SHA: 16 pass, 14 skipped by selection, at 3a7ed8bae.

High-risk regression review

  • Classification: standard
  • Implementation task: N/A
  • Independent blocking QA task: N/A
  • Exact head / current base: N/A
  • Production-branch deletion, bypass, or miswire proof: N/A
  • Affected-family real-model/artifact evidence: N/A
  • Explicit unavailable-family or other N/A evidence: N/A
  • Known PR-caused P1 regressions: 0
  • Unresolved review threads: 0

….5 structured parse

Marker: a failed write removes its .tmp and rethrows the original error, even
when the cleanup itself fails (#1584 audit A3, A5, A8). The not-published list
names the hook's stable_diffusion bundles (A9). Genkit: Qwen3.5 with thinking
off keeps a thought in the content, as the migration guide says.
@leehack
leehack merged commit f662682 into main Oct 30, 2026
30 checks passed
@leehack
leehack deleted the fix/1565-carry-overs branch October 30, 2026 11:39
@github-actions

Copy link
Copy Markdown
Contributor

Chat app preview removed for leehack/llamadart-chat-pr-1605.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant