Skip to content

fix(ci): retain workspace screenshot proof artifacts - #2435

Merged
schlessera merged 9 commits into
mainfrom
fix/1903-proof-capture-retention-release044-r1
Oct 19, 2026
Merged

schlessera merged 9 commits into
mainfrom
fix/1903-proof-capture-retention-release044-r1

Conversation

@schlessera

@schlessera schlessera commented Oct 19, 2026 •

Copy link
Copy Markdown
Owner

Refs #1903

What changed

Retain workspace capture diagnostics through both upload and unit-worker cleanup. All seven existing proof uploads collect kit, consumer and repository-root attachments, six bounded proof directories and numeric-counter Vitest failure PNGs. Before removing each stopped unit worktree, copy its diagnostics into tmp/unit-worker-diagnostics/<invocation>/worker-N/<original-relative-path>; the unit upload now includes that directory. The contribution is four files: the workflow, scripts/unit-workers.ts, and its existing fixture/test pair. It advances the workspace-resident slice of #1903 and leaves the issue open.

Why this shape

The upload paths alone cannot reach sibling unit worktrees after cleanup. The handoff covers exactly nine workspace-relative roots: .vitest-attachments/, packages/ui-kit/.vitest-attachments/, packages/ui-react/.vitest-attachments/, and tmp/{diagram-renderer-proof,mermaid-renderer-proof,diagram-export-proof,cold-capture,share-review,svg-proof}/. It also selects numeric-counter PNGs beneath __screenshots__ in the UI-kit/UI-react source and test trees, without traversing dependency/build directories. Hidden receipts and associated PDF/JSON files inside the bounded roots survive with their original relative paths.

One immutable, bounded Git tree lookup supplies tracked blob identities. Unchanged tracked bytes are filtered from worker diagnostics; changed tracked counters remain evidence and still fail the existing changed-source guard. The root collectors exclude 391 tracked Linux references; 18 tracked numeric overlaps may still appear there and must be identified by exact path and hash, never counted as new failure images.

Each invocation and worker has a separate destination. Files are created exclusively; source directories are pinned through no-follow descriptors and regular-file checks. Copy readback verifies size and SHA-256. Only newly generated automatic counter PNGs absent from immutable worker HEAD may be unlinked, after successful verified copying and same-source identity checks. Tracked files are never removed or restored by the handoff. Copy, corruption or replacement failures retain the source and expose ordinary worktree-removal refusal. Report copying, diagnostic copying and normal cleanup are attempted independently for every worker; an AggregateError preserves the primary failure and cleanup errors. No force removal or arbitrary external glob is added.

Existing always-upload behavior, hidden-file inclusion, seven-day retention, kit outputs, receipts and probes remain. The contribution changes no product capture writer, screenshot configuration, baseline, scenario selection, assertion or deadline. It contains no package shipping or machine-contract change, so no changeset is needed.

Proof

Current local candidate: head 939562a0d53edc265aa1ebefed5587d74b746863, tree 9fc417f72bdda9225e73b0aef1bc33f2da171e44, base/merge base 8ac0fc6cfd7f4142f961deac260e6a1b98f6c415; parents are a69fb21dbfc978b23b758d8645536c5d746a1328 and that base. The normal history-preserving refresh adds the already-landed #2197 integration protocol (12 files) and #2425's three-line held-root fixture TMPDIR fix. Earlier main merges brought module-settings/overlay test fixes, the #2437 scope-observation fixture repair, Storage #2172 and Git-maintenance fixture policy #2448. Those changes retain their upstream attribution. All four R4-reviewed contribution files and the full contribution patch relative to current main remain byte-identical: 399 additions and 25 deletions across four files, patch SHA-256 75c2c5dc0bb9cbe4f0f801da298b4c20d1843b86d2c997c540abcf070205fc50.

  • Fresh bun run check:pr --base origin/main on this exact combined candidate: exit 0; 959 tests / 6,361 assertions across 48 files in 83.79 seconds. Its automatic import-graph feedback additionally passes 10 worker tests / 597 assertions in 4.75 seconds. The one-shot command used the normal UID/HOME, original /tmp, private TMP/cache/ledger and offline/output-lease guards with a 600-second cap and 30-second joined cleanup allowance. It completed within that bound; all owned processes and worker checkouts drained. No separate install/build/typecheck, focused, browser or native-prover run was launched.
  • The one actual plan is unchanged in its entire parsed output: global=true, 19 affected packages, 48 fast files, five ordered ordinary commands, typecheck=true, pack=true and packedConsumerChecks=false; all hosted categories remain selected. Selection SHA-256: eec4050a57b9755430d27b3f1b65bb85fe68138b1ca810c0461eb6aa79a7d7d7. Plan inspection is not runtime proof. Current Storage/Vitest/preload inputs do not establish equivalence with old native/browser/cache receipts. packages/core/dist/storage.js remains absent; the actual default succeeded without an automatic build, which makes no claim about a fresh complete build or strict types.
  • Historical published a69 default: head a69fb21dbfc978b23b758d8645536c5d746a1328, tree 100daef44d51ff1cc51ee58aeeb83216dcefbd38, base/merge base cea53611ddea4916ac323060bfef42ec18118fea, parents b618c3c4a5a48277dc9545c98107727229c24ffa and that base. Exit 0; 959 tests / 6,357 assertions across 48 files in 135.00 seconds, plus automatic feedback 10 tests / 597 assertions in 6.19 seconds. This remains a69/CEA proof, separately from the fresh combined candidate.
  • Historical published bf172 default: head bf17214e9a1311e8d0241a1e69c7ef3f55c4804a, tree 55790439c27abbb8f878590c9da3aa9c93d9b124, base/merge base e16b8407f81da4071a3a1bf1cc04299acb1a2522, with parents b300377edf953f87661f36685e5c8d41b242e43b and that base. Exit 0; 959 tests / 6,354 assertions across 48 files in 118.29 seconds, plus automatic import-graph feedback 10 tests / 597 assertions in 8.42 seconds. This remains bf172/e16b proof, separate from the current candidate.
  • R4 worker-handoff source: head b300377edf953f87661f36685e5c8d41b242e43b, tree 7b507be84589783bbb47867c0841a2456aef5663, parent 663a67718b72eee5d45b03aafb17a79886b4c7c8, base 09eb1f78e99555110f4ef1c8b78e23d188ec77e5. Independent R4 review reported no findings. Focused four-file qualification passes 69 tests / 1,029 assertions; strict typecheck and cheap gates pass. Real child writers cover all nine roots, hidden receipts/PDF, unignored generated counters, worker/repeated-invocation collisions, unchanged tracked references, parent artifact preservation, deliberate child failure, symlink refusal, and report/diagnostic copy, corruption and replacement errors. These are path/byte controls, not raster PNG proof.
  • Meaningful omission mutation: omit only the closed-root handoff while retaining automatic-counter handoff/removal. Both real child tests pass and normal worktree removal completes; WORKER_DIAGNOSTIC_SURVIVES_REMOVAL then fails on missing .vitest-attachments/worker/.receipt.json. Exact source restoration passes. The restored source is the R4 source retained in the current candidate.
  • Historical b300 default: exit 0, 959 tests / 6,354 assertions across 48 files in 88.10 seconds, plus automatic feedback 10 tests / 597 assertions in 5.23 seconds. This receipt remains attributed to b300/base09eb, separately from the fresh combined candidate.
  • Collector controls: use the actual actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 bundle. The precise patch replaces the upload entrypoint with an export of the unmodified collector; no network upload is performed by the harness. The real module-settings writer produces the omitted file under the old arrangement; restored collection passes 22 assertions. Expanded controls cover nine destinations across all seven uploads with 3,454 passing assertions; the previous collector fails the named cold-capture omission assertion, and exact restoration passes. Existing receipts retain upstream source, bundle hash and patch identities.
  • Original native PNG execution only: head 69c5b3ee383797fca7cc26ae90f05e742d88732b, tree 41947c7346ae416874c0f83da831a2bef1f1f0a8, base 8cb0fdbc59a5b2d2d9ca9120895bbf6d71a7e18a. The bounded pinned offline run passes 129 cases and produces 57 nonempty PNGs: four durable-action headers, 17 keyboard fine/coarse/mixed/live captures and 36 module-settings captures. Actual runner cwd is /repo/packages/ui-kit. Dimensions, sizes and hashes were verified; all 57 hashes remain unchanged. These receipts are not new native execution on b300 or bf172.

Historical hosted artifact audit, not current-head confirmation: run 39225826747, attempt 1 executed actual checkout 4dad8ab924b05410a1750194793f0845dacfc31e, tree ddda919e06cae34eae50c623f6fab60049aff9fb, with immutable parents 09eb1f78e99555110f4ef1c8b78e23d188ec77e5 and 663a67718b72eee5d45b03aafb17a79886b4c7c8. All 13 completed proof ZIPs were downloaded; archive digests match the API, and job checkout logs and artifact receipts match the actual checkout. Safe ZIP-path inspection, nonzero PNG dimensions/sizes, hashes and chunk CRCs were verified. Archive paths remain repository-rooted and preserve hidden attachments, receipts/probes and existing outputs.

That audit contains 1,302 PNG instances. Of these, 234 are repeated instances of the same 18 tracked numeric files with exact path/hash matches; none is new failure evidence. All 391 Linux references are excluded. Required generated classes actually present include headers (4), keyboard fine/coarse/mixed (5 each) and live fine/coarse (1 each), module-settings (36), diagram-renderer (170), diagram-export (148), Mermaid (5), cold-capture (2) and share-review (1).

SVG PNG evidence is absent from those ZIPs despite both real SVG producer cases passing in unit4, worker1. artifact-recovery-runtime.test.ts resolves the worker checkout as its repository root and writes persistent tmp/svg-proof/svg-print.html.{png,pdf} and svg-share.html.{png,pdf} there. The old cleanup copied only reports/timings/build probes, then normally removed the sibling checkout; the root upload path could not reach those worker files. This is distinct from temporary fixture artifact-*.png files removed by their own teardown. The new handoff repairs that source arrangement locally, but no newly downloaded hosted SVG PNG/PDF proves it yet.

The hosted run concluded failure: unit2's Pi dictation second-receipt wait is tracked in #2436, with both worker reports preserved in artifact 12174009122 and no merged root unit report. That historical failure was repaired upstream by #2437, delivered on main 0b46b9db7925ce2e52c85a56c6200a4e2f330bd7, and #2436 is closed in 0.44.0. The current candidate incorporates that repair through main; the old failed run is not relabeled as successful and the new candidate still requires its own complete selected hosted proof. No newly generated automatic Vitest failure PNG appears in the 13 ZIPs; tracked overlaps cannot substitute for it. Earlier independent source rounds 1 and 3 reported no findings; round 2's omitted cold/share/SVG roots were repaired before R3. R4 additionally reviewed the worker handoff without findings.

Checklist

  • Focused local behavior, mutation/restoration and current-base default receipts are recorded above.
  • Complete selected current-head hosted proof and required downloaded artifact evidence passed.
  • Unmet ci: retain consumer UI screenshots in hosted proof artifacts #1903 acceptance criteria are explicit below; this PR uses Refs only.
  • PR and advanced issue milestone recorded as 0.44.0.
  • No changeset: test-infrastructure retention changes nothing a package consumer sees.
  • Local lint/leakage gates pass; no personal data in the contribution.
  • Machine contract, documented product behavior and provider-facing history remain unchanged.

Anything left open

Keep #1903 open with its full original criteria. For this workspace-collector and worker-handoff delivery, complete selected current-candidate confirmation and actual downloaded worker SVG coverage are required; job success alone does not prove that the uploaded ZIP contains those bytes. Existing kit/consumer/root outputs, metadata, hidden files, safe archive paths and retention must also be reconciled. Root uses a separately acquired exact-candidate integration lease, minute renewal and confirmation watch, audits artifacts before the explicit lease-checked merge, and retains the sixty-minute hard cap.

Genuinely generated automatic-failure PNG evidence is still pending, distinct from passing counter-path controls. This body explicitly reports that gap under #1903; it does not declare that criterion passed or silently waive it. The 18 unchanged tracked overlaps cannot fill it. The full issue also retains the following unfinished work and verification:

  • Persistence of hygiene server/loop captures under container tmpdir(): c5-<theme>-<width>-{failure,complete}.png, replication-wait-<width>.png, completion-<completion>-<width>.png, c5-local-receipt-<theme>-<width>.png, c5-native-<theme>-<width>.png, and c6-{stalled,complete,invalidation}-<width>.png. Their ephemeral files are not retained by adding workspace collectors.
  • Persistence of runtime recovery fixture artifact-*.png outputs deleted in teardown, and root /tmp/brain-kit-software-*.png outputs. Optional externally directed capture destinations and standalone navigation probes remain inventoried; no arbitrary external paths are swept into public artifacts.
  • Complete selected proof for the actual published candidate, with real downloaded worker-namespaced SVG PNG/PDF evidence tied to worker checkout receipts and nonzero PNG dimensions, size and hashes. Other required classes must retain their actual producer/path inventory.
  • Genuine generated automatic-failure PNG evidence in downloaded hosted artifacts; all 18 unchanged tracked numeric overlaps remain excluded from failure-evidence claims. Passing path controls or old native PNGs do not satisfy this requirement.
  • Complete selected current-candidate verification after incorporating the upstream test(ui-server): await scope observation before continuation #2437 repair of ui-server: diagnose Pi scope grant wait in hosted unit proof #2436. The historical failed unit artifact remains a failed receipt; upstream issue closure does not supply this candidate's hosted confirmation.

This is artifact retention work. It claims no #1229 timeout diagnosis or repair; unrelated test fixes arriving through main retain their own attribution. The historical failed run and the current candidate's missing hosted confirmation remain explicit. The separate main #2413 failure remains unknown; this retention change and upstream #2437 do not claim to repair it. No closure or merge eligibility is claimed.

@schlessera schlessera added this to the 0.44.0 milestone Oct 19, 2026
@schlessera schlessera added type: fix Something is broken and must behave the way it is documented. area: ci CI workflows, the gates they run, release scripts, packaging. labels Oct 19, 2026
@schlessera
schlessera marked this pull request as ready for review October 19, 2026 08:52
@schlessera

Copy link
Copy Markdown
Owner Author

Confirmation run 39225826747 attempt 1 is complete and non-green. Unit shard 2 job 117972100537 failed the Pi scope-grant fixture; aggregate job 117979768022 correctly refuses unit: selected proof did not pass (failure). Every other selected execution job passed. Aggregate candidate metadata retains head 663a677, base 09eb1f7 and actual checkout 4dad8ab/tree ddda919e06cae34eae50c623f6fab60049aff9fb.

#2436 records the unit failure. Retained frames show both first-grant submissions refused while continuation headroom was unconfirmed; there was no accepted first grant or second pause. A controlled real-worker fixture correction is being prepared separately, preserving the refusal guard and deadlines. No retry or waiver is requested.

Downloaded browser/layout artifacts confirm real retained capture classes, with tracked numeric screenshots classified separately. The audit also found generated SVG proof lost during unit-worker checkout cleanup before upload; that boundary is recorded under #1903 and remains unfinished. No claim of complete hosted capture retention or issue completion follows. This PR is not mergeable on the current evidence.

@schlessera
schlessera marked this pull request as draft October 19, 2026 16:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: ci CI workflows, the gates they run, release scripts, packaging. type: fix Something is broken and must behave the way it is documented.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant