Skip to content

Sweep 1565: forced-open Seed-OSS verbatim thought, hermetic SD hook test, extraction marker follow-ups - #1575

Merged
leehack merged 4 commits into
mainfrom
sweep/1565-last
Oct 29, 2026
Merged

leehack merged 4 commits into
mainfrom
sweep/1565-last

Conversation

@leehack

@leehack leehack commented Oct 29, 2026 •

Copy link
Copy Markdown
Owner

Summary

Clears these items on tracker #1565 (not the stable genkit item #1574):

  • parseRendered Dartdoc (#1573 audit): engine equivalence needs parseToolCalls set like the engine, the engine's partial reparse after a PEG failure, copies (constructor, fromJson, isolate transfer) losing the marks, and reasoning chunks dropped with thinking off. Class-doc example fixed.
  • Hermetic build_hook_stable_diffusion_integration_test.dart ("the stable_diffusion companion selects and links it in process"): seeds the LiteRT-LM macOS arm64 cache from fake libraries. Passes with outbound network denied (sandbox-exec); on main it fails there downloading litert-lm-native-runtime-macos-arm64-v0.18.1-4.tar.gz.
  • Forced-open Seed-OSS (#1572 lane): a thought the prompt opened is now the reply up to the first </seed:think> as generated (keepsForcedThoughtVerbatim, as DeepSeek V3, Cohere2 MoE, Qwen3-Coder XML), so a repeated <seed:think> and text before it stay in the reasoning, as llama.cpp 620d2225 keeps them. Stream and parse agree; a thought closed for thinking off and later reached by a thinking tag streams like Cohere2 MoE.
  • Reuse a cached native bundle only while it matches its archive; pin Seed-OSS ASCII trim #1572 audit items: the marker takes the new archive mtime after a successful rehash (no rehash every build); a cache with no marker logs an info migration message, a real mismatch still warns; tests kill mutants A1, A6, A9, A13, A14 and "no ASCII trim at all" in _endsUnopenedSeedOssThought (B12 too: a NBSP-led public parse now pins its reasoning).

Production-readiness scope

  • User-facing scope: forced-open Seed-OSS reasoning matches llama.cpp; quieter hook logs on cache upgrade; Dartdoc.
  • Supported platforms/paths: all targets (parser); native hook (all bundles).
  • Unsupported or intentionally unavailable paths: llamadart trims the reasoning, so a leading NBSP that llama.cpp keeps in <seed:think>Plan is trimmed (llama.cpp keeps leading space too); content, tags and later text are identical.
  • Out of scope / follow-ups: public ChatTemplateEngine.parse on the stock Seed-OSS template (#1565, 0.18 candidate).

Completeness checklist

  • Declared scope is fully implemented, or explicitly reduced above.
  • Unsupported platform/option combinations fail loudly or are documented: N/A, no new option.
  • Changelog updated (both files); no API change.
  • Regression coverage covers the original issue plus negative paths.
  • Validation coverage registry: N/A, no public capability change.
  • Security/privacy: no secrets, tokens or paths added to logs.
  • Follow-ups tracked: None beyond the above.

PR type guidance

  • Bugfix PR: forced-open Seed-OSS used the generic thinking extractor, which dropped a repeated start tag (and merged text before it into content); it now uses the verbatim forced-thought path. Tests: closed_thought_upstream_test.dart (values from the llama.cpp 620d2225 oracle), hook and marker tests. Platforms: all.
  • Feature PR / Docs-only PR: N/A.

Test Plan

  • dart run tool/prepare_workspace.dart
  • dart format --output=none --set-exit-if-changed .
  • dart analyze; dart run tool/testing/check_platform_boundaries.dart
  • TMPDIR=$(mktemp -d) dart test -p vm -j 1 --exclude-tags local-only: +9799 ~8, all passed (re-run at the audit-fix head)
  • dart test -p chrome --exclude-tags local-only test/unit/core/template test/unit/core/engine: +2139 ~1, all passed
  • tool/testing/run_template_parity_suites.sh exit 0; test/unit/tooling +1240 ~5
  • Hermetic check: sandbox-exec deny network-outbound, whole build_hook_stable_diffusion_integration_test.dart +34 passed (fails on main with the LiteRT-LM download)
  • Failing-first: the new Seed-OSS test fails on main's parser; each new hook test fails against its mutant below

Matrix Evidence

Matrix row Scope covered Platform / model / backend Result Evidence / notes
static-format-analyze format, analyzer, boundaries macOS arm64 PASS 0 changed, no issues, OK
root-vm VM suite macOS arm64 PASS +9679 ~126
root-chrome template/engine subset Chrome PASS +2139 ~1
oracle llama.cpp 620d2225 vs engine path (whole, bytewise, every 2-split, final parse) #1569 audit harness, base 176e12e4c vs head PASS main 185,160 rows: 0 changed; X 37,056: BAD->OK 234; Y 7,440: BAD->OK 336; whitespace 480: BAD->OK 120; OK->BAD 0 and new stream!=parse 0 everywhere; fixed stream!=parse 19/44/18 (X/Y/Z). Seed-OSS stock rows unchanged; later first-content than upstream: 9 rows of X ("Hello seed:thinkthink</seed:think> world", 1 char)
high-risk-exact-head-independent-qa independent blocking review of exact head fresh auditor session, exact head 0629111 PASS CLEAR: CLEAR delta re-audit at 0629111; evidence /opt/UnitySrc/personal/llama/llamadart-evidence/pr-1575/evidence-1575-0629111.json

Mutants (all killed, incl. audit H3 wrong refresh digest and uncaught refresh failure): A1, A6, A9, A13, A14, A15 (no marker refresh), A16 (always warn); Seed-OSS: old forced-open parse, trait off, stream marker branch off, no ASCII trim, Unicode trimLeft() in _endsUnopenedSeedOssThought.

Review Notes

  • Independent review status: CLEAR at exact head 0629111 by a fresh auditor session (0 P1, 0 P2).
  • CI status / head SHA: fill after CI runs

High-risk regression review

  • Classification: high-risk (artifactConsumer, backendRuntime, structuredOutput)
  • Implementation task: this lane
  • Independent blocking QA task: fresh auditor session, CLEAR, 0 known PR-caused P1, 0 unresolved threads
  • Exact head / current base: 0629111 / 0bddcde (merge-tree clean)
  • Production-branch deletion, bypass, or miswire proof: N/A (trait wired in chat_format_registry.dart; mutant with the trait off is killed)
  • Affected-family real-model/artifact evidence: llama.cpp 620d2225 oracle rows above; no real Seed-OSS weights run
  • Explicit unavailable-family or other N/A evidence: Seed-OSS weights unavailable locally
  • Known PR-caused P1 regressions: 0
  • Unresolved review threads: 0

@leehack
leehack marked this pull request as ready for review October 29, 2026 03:12
@leehack
leehack merged commit d6f96cc into main Oct 29, 2026
40 of 41 checks passed
@leehack
leehack deleted the sweep/1565-last branch October 29, 2026 03:12
@leehack leehack mentioned this pull request Oct 29, 2026
29 of 32 tasks
@github-actions

Copy link
Copy Markdown
Contributor

Chat app preview removed for leehack/llamadart-chat-pr-1575.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant