Skip to content

Run the chat-app Web smoke in Safari and Firefox and add a WebGPU generation E2E (#1025) - #2000

Closed
leehack wants to merge 1 commit into
mainfrom
lane-1025-browser-harness
Closed

leehack wants to merge 1 commit into
mainfrom
lane-1025-browser-harness

Conversation

@leehack

@leehack leehack commented Nov 14, 2026

Copy link
Copy Markdown
Owner

Summary

  • run_local_e2e.dart --scenario chat-app-web-real-model-smoke --browser safari|firefox serves the repo with --coep require-corp and drives the smoke through safaridriver or geckodriver (FIREFOX_EXECUTABLE, GECKODRIVER).
  • --backend webgpu passes --backend-index 0 --gpu-layers 99 (default stays CPU Wasm).
  • The Dart web scenarios reject --browser safari|firefox: package:test pages are never cross-origin isolated.
  • New web-webgpu-generation scenario and test/e2e/webgpu/webgpu_generation_e2e_test.dart (local-only): full-offload WebGPU text vs CPU Wasm, direct and worker.
  • Firefox injects the init script as a page <script> because geckodriver's execute sandbox hides page globals.
  • Part of #1025, plan #1995. Support wording lands with the repin PR.

Production-readiness scope

  • User-facing scope: maintainer test harness only; no lib/ change.
  • Supported platforms/paths: macOS Safari 27, Firefox via geckodriver, Chrome WebGPU.
  • Unsupported or intentionally unavailable paths: Safari/Firefox on Dart web scenarios exit 64 with the isolation reason; tool-call, speech and mmproj checks in Safari/Firefox exit 2.
  • Out of scope / follow-ups: Firefox real run (see Matrix Evidence); coordinates with #1999, which edits the same four files in separate hunks.

Completeness checklist

  • Declared scope is fully implemented, or explicitly reduced above.
  • Unsupported platform/option combinations fail loudly with actionable diagnostics or are clearly documented as unavailable.
  • Public API docs, README/website docs, examples, support matrices, and changelog entries are updated where relevant. (N/A: no public API; matrix doc updated; changelog and support docs belong to the repin PR)
  • Regression coverage covers the original issue plus key negative/version-skew paths where relevant.
  • Public capability changes update the validation coverage registry. (N/A: none)
  • Security/privacy review completed: no secrets leak.
  • Follow-up work tracked, or "None" above.

PR type guidance

  • Feature PR: N/A, test tooling only.

Test Plan

  • dart run tool/prepare_workspace.dart
  • dart format --output=none --set-exit-if-changed .
  • dart analyze
  • dart test -p vm test/unit/tooling/ (1830 passed); full -j 1 VM and Chrome suites left to CI
  • coverage-lib: N/A, no lib/ change
  • Other: mutation check (dropping --coep require-corp fails the dry-run and matrix tests); real runs below

Matrix Evidence

Matrix row Scope covered Platform / model / backend Result Evidence / notes
web-real-model-smoke-safari chat app, plain chat, require-corp Safari 27.0, macOS, Qwen3.5-0.8B-Q4_K_M, CPU Wasm PASS 20.1 s, reply "2+2 equals 4."
web-real-model-smoke-safari same, --backend webgpu Safari 27.0, Apple WebGPU, same model PASS 21.5 s, runtime shown as WEBGPU
web-real-model-smoke-firefox chat app, plain chat Firefox 160 N/A (not run) scratch Firefox 160 and geckodriver were not on this machine and the download was not permitted; covered by the scripted-driver unit test only
web-webgpu-generation direct and worker, 8 and 32 tokens Chrome WebGPU (Apple metal-3), stories15M PASS WebGPU text equals CPU text in both runtimes

Review Notes

  • Independent review status: pending (high-risk)
  • CI status / head SHA: pending, 3cd3457

High-risk regression review

  • Classification: high-risk (regressionPolicy)
  • Implementation task: this PR
  • Independent blocking QA task: pending
  • Exact head / current base: pending
  • Production-branch deletion, bypass, or miswire proof: N/A
  • Affected-family real-model/artifact evidence: Qwen3.5-0.8B (Safari), stories15M (Chrome generation)
  • Explicit unavailable-family or other N/A evidence: Firefox real run
  • Known PR-caused P1 regressions: 0
  • Unresolved review threads: 0

…eration E2E (#1025)

--browser safari|firefox serves require-corp and drives the real-model smoke
through safaridriver or geckodriver; --backend webgpu offloads all layers.
The Dart web scenarios reject both browsers: package:test pages are never
cross-origin isolated. web-webgpu-generation compares full-offload WebGPU
text with CPU Wasm in direct and worker runtimes.
@leehack

leehack commented Nov 14, 2026

Copy link
Copy Markdown
Owner Author

Superseded by #2002 (rebased on #1999 with the audit P2/P3 fixes).

@leehack leehack closed this Nov 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant