Skip to content

fix(ci): keep Docker proof bind sources stable across runs - #2385

Merged
schlessera merged 7 commits into
mainfrom
fix/2376-stable-docker-mounts
Oct 20, 2026
Merged

schlessera merged 7 commits into
mainfrom
fix/2376-stable-docker-mounts

Conversation

@schlessera

@schlessera schlessera commented Oct 17, 2026 •

Copy link
Copy Markdown
Owner

Refs #2376

What changed

Docker-based local proofs now bind stable checkout-owned directories and select invocation files inside the container. Native-host, identity, lease and capture callers keep their existing assertions and security flags. Captures publish atomically on the destination filesystem. Native diagram export transports private source and captured runtime bytes through tar-filled named volumes and copies results out, preserving host-state/canary absence checks.

Why this shape

Docker Desktop retains shares keyed by host bind sources. Reusing stable sources bounds further growth. Diagram source isolation needs private inputs, so it uses the copy/volume form permitted by #2376. The implementation changes local proof transport; no runtime qualification pins, product isolation settings or timing budgets are changed.

Proof

Historical draft checkpoint: 6cc530a7acb8abd3cfec796a28ee72a3e6d56f34; integrated main: c4d31c6bd1aeed60fd3cda02f9c8586fe8a509ae; tree: 27d9b66065f9fd0d53ade446d18b3c7dea5982cf.

Refreshed real Docker receipts below executed source 16c9ed01b32c2ec7630111ec079a6c1669703b16, based on delivered #2375. The subsequent main advance changes CI headroom/sharding metadata; native/capture implementations, package sources and their runtime inputs are unchanged. These remain receipts for their actual executed source, not executions on the later SHA.

Caller Desktop shares before / first / second Result
Real pinned lease 15 / 15 / 15 both pass
Complete native host 15 / 18 / 18 both pass
Complete default identity 18 / 19 / 19 both pass
Private diagram export, both engines 19 / 19 / 19 both pass

Native-host includes Claude/Pi × UID 1000/0, the intended retained-index negative/recovery, unchanged namespace quota and full owned loop/mount/backing cleanup. Both identity runs use complete scope, all four default app/control phases, existing SDK/prepared-consumer/Btrfs controls, no skipped categories and confirmed container/resource cleanup. Optional projection byte/write controls and five named bind-source failing-first/mutation receipts are retained in #2376.

Complete capture verification passes twice (362.08 / 368.87 s), each with five recipes, 13 exact files and ten paint-history pairs. The raw global counter increases 19 / 26 / 39 during concurrent foreign-workspace proof activity; that aggregate measurement remains failed evidence. Mount-root inspection finds one share per stable source in this checkout, with no duplicates. A bounded diagnostic repeats complete verification with transparent Docker argv logging and before/after mount IDs: pass in 375.12 s, 39 / 39 total shares, eight / eight owned checkout shares, zero new mounts. Seven Docker run calls use five stable bind sources and no fresh staging/runner bind paths. This attributes the global interference without changing comparisons or guards.

  • Historical build and strict types pass at the recorded draft checkpoint. Its six focused suites pass 73 tests / 229 assertions; lint/leakage and citation checks pass.
  • All original containment, source/canary absence, read-only inputs, network, cleanup and atomic-output assertions remain intact. AppArmor is unavailable locally; no enforced AppArmor qualification is claimed.
  • Initial refreshed diagram copy failed ENOSPC with only 1.3 GiB available. Failed evidence is retained. Removing only regenerable installs in this campaign's completed worktrees restored capacity; both unchanged diagram runs then pass.
  • Historical preflight at e6f1716 passes 956 fast tests and 176 affected tests, with one frozen-native-positive failure owned by ci: native diagram qualification self-test rejects legitimate source drift #2365. At that historical checkpoint, complete preflight and hosted candidate confirmation remained pending; no known-failed guard or qualification hash was waived/refreshed.

Historical local candidate — 2026-10-19

Head 5da13f8c7dc1e02281a37d7e8f577632786a26e2, integrated base and preflight merge base 90a95123cfceb169bd2ecf7c213544db742eaed7, tree b088ad82d16c481c3bcfd021f6c2135fcfb57584. This conflict-free, nonrewriting integration preserves the original implementation and historical receipts. The diff remains 12 files, 251 insertions and 51 deletions. Main's frozen-positive correction and draft-proof gate are integrated; measured native/capture implementations and their runtime inputs are unchanged from 16c9ed01b32c2ec7630111ec079a6c1669703b16. Those earlier measurements remain historical executions, not runs on this head.

  • The actual bun run check:pr --base origin/main exits 0 in 203.15 seconds: all lint/leakage, environment and changeset gates pass, 959 fast tests / 6,354 assertions pass, and 189 import-graph feedback tests / 53,369 assertions pass. Feedback includes the real pinned lease-container test and complete corrected native frozen-positive/current-source controls. No test selection, offline guard, deadline, image or assertion changes were made.
  • The pinned browser image was already installed at the declared digest; no pull occurred. The lease fixture uses its existing network-disabled container and cleanup. No loop/Btrfs/identity container or repeated five-caller mount campaign ran in this continuation. The earlier resource restriction was resolved before execution; it was not a test failure.
  • Own frozen copyfile dependencies use a fresh private cache. All 59,516 regular installed files have link count 1, and no dependency symlink escapes the owned checkout. Installation identity is 0c2758c046f2b69d5f33b66521baf3098db5f44256b73ec96961c98d381e6245. Lockfile SHA-256 remains 5bbd5f6651f111db3fcfabeda512d44a21163cba54c58993f8878012d548cc14.
  • Earlier checks on this same unchanged candidate pass strict types, lint/leakage, and 19 focused route-free offline tests / 51,830 assertions. The actual complete preflight above is a separate receipt. No consumer-visible package change or changeset is required.
  • The recorded planner selects all hosted proof categories. Complete selected hosted proof is still required. Current local success does not qualify a later base or substitute for actual selected-job checkout, resource and aggregate evidence.

Historical candidate 5030576 — local pass followed by hosted failure

Head 50305761ecc341b11da7142ab72e4e5f2baa15a6, resolved base and preflight merge base 8474b80c1c83e5f11de87011de14087131188203, tree 23738204039da4add6788c43286d3afc2175c9d2. History-preserving refresh retains parents c8274839cfae3eaa9122fee90bd1414b4d72c143 and that base. Contribution remains 12 files, 251 insertions and 51 deletions.

Independent R2 source review reported no findings across the full contribution and resolved fixture at c8274839cfae3eaa9122fee90bd1414b4d72c143, tree 279bfa9691414af2362b27817c3615f0098e56c0. The subsequent seven-file candidate-admission integration is conflict-free; eleven contribution blobs, including the fixture, are identical to that reviewed source. Only testing documentation inherits the new admission wording. The fixture preserves the publication boolean third and stable parent fourth. Incoming Storage/prune diagnostics, Git maintenance suppression, child temporary-directory correction, worker diagnostic collection, early admission, final aggregate and reservation controls are preserved.

  • The ONE actual default bun run check:pr --base origin/main exits 0 in 161.97 seconds. All lint/leakage, environment and contribution changeset gates pass; 960 fast tests across 48 files / 6,364 assertions pass; required import-graph feedback passes 193 tests across 20 files / 53,450 assertions. No selection skip, guard change, timeout change, retry or source fix was used.
  • Actual feedback includes all five Docker bind-stability cases, all ten native frozen-positive/current-source/pin/importer/installed-origin controls, complete PID-publication readiness, both SIGTERM/SIGKILL owner-lifetime cases, and the real pinned lease-container assertion. These requested focused controls are covered by this run, so no duplicate focused run is proposed.
  • Execution uses ordinary UID 1000 and its real HOME, all capability sets zero, a route-free loopback-only namespace, private temporary/cache/maintenance-ledger paths, original temporary/repository visibility and normal project output leases. No outer Python repository flock is held. Resource floors and zero Btrfs device-error counters pass before and after; bounded owned write probes are removed. The nonstealing native-capacity lock is held only for this admitted command and released on drain.
  • The exact pinned Playwright image is already local, matching installed Playwright 1.63.0. No image pull occurs. The only real Docker fixture executed is the automatically selected pinned lease container with the unchanged network-disabled, pull-never, ordinary-user flags and stable source/working-directory selection. Its owned container, fixture directory and processes are absent afterward; ledger and output/capacity locks are drained. No loop/Btrfs native-host matrix, identity container, explicit browser/diagram producer or five-caller benchmark is executed locally in this round.
  • Reused private copyfile dependencies retain installation identity 0c2758c046f2b69d5f33b66521baf3098db5f44256b73ec96961c98d381e6245, 843 packages / 59,516 files / 1,833,832,361 bytes, single-link regular files and no escaping dependency symlinks. Lockfile SHA-256 remains 5bbd5f6651f111db3fcfabeda512d44a21163cba54c58993f8878012d548cc14; no reinstall, standalone types or build command runs in this round. Source is clean at exit.

That local success did not establish complete selected hosted proof. The subsequent selected confirmation failed as recorded below. Strict types, packaging, full affected unit discovery, browser/layout/endurance/captures and runtime/native-host categories require actual selected-job identities and results. The native-host default must qualify the complete matrix plus retained-index negative/recovery and owned cleanup, default identity must retain every phase, and native diagram proof must execute both engines. New source/preload/caller/proof inputs cannot inherit the old native qualification. Frozen positive ownership, inventory, qualification hashes and current-source refusal remain unchanged. No local preflight or older native receipt substitutes for these fresh selected-job identities and results.

Earlier 5da13f8c/90a95123 preflight and 16c9ed01 runtime/mount evidence above retain their original identities, missing-raw-log limits and failed global-growth measurement. None is relabelled as a run on this refreshed tree. Per-new-worktree stable-path mounts remain out of scope. No consumer-visible package change or changeset is introduced by this tooling contribution. Keep Refs #2376 and draft until the remaining acceptance and selected proof are satisfied.

Failed hosted candidate and fixture correction

Confirmation 39355998063 ran source head 50305761ecc341b11da7142ab72e4e5f2baa15a6, base 8474b80c1c83e5f11de87011de14087131188203, actual synthetic checkout df04979178341b27f7edb6a8c586f20f304acb35, tree 23738204039da4add6788c43286d3afc2175c9d2, immutable parents [8474b80c1c83e5f11de87011de14087131188203, 50305761ecc341b11da7142ab72e4e5f2baa15a6]. Unit shard 2, job 118411159633, had two actual failures in tests/visual-runner.test.ts: local explicit parallel-file override expected runner exit 7 and received 1; cold execution expected 0 and received 1. The old Docker fake accepted only the earlier run/host-bind form and returned 91 for the new volume operation. Aggregate job 118417050431 failed. The failed run remains failed; its reservation is released. It was not waived or rerun to obtain green.

The independent failed-candidate audit matched all 17 actual checkouts, 13 resource pairs and 14 artifact digests to that 50305761/df049791 identity. Only unit shard 2 and the aggregate failed; no additional reported assertion failures were found. Unit shard 2 is nevertheless incomplete: its sibling terminated with exit 143, reports are missing, and 143 selected files lack complete case evidence. Fixing the two fixture assertions cannot establish complete unit proof; fresh selected hosted full-unit coverage remains mandatory.

For that failed candidate only, the audit verified four healthy native-host backend/UID cells, retained-index negative/recovery, quotas and six cleanups; every default-identity phase passed. Both diagram engines passed 124 cases each with nonempty retained outputs. Browser/layout/captures/endurance and the other selected jobs passed for their actual checkout. These successes do not turn the failed candidate green and are not current 972e0004 native proof or a native-cache claim. The default-identity CLI-version probe warning is retained as a limitation; it also occurs on green main 8474 and is not identified as a new root cause.

The scoped correction is only tests/visual-runner.test.ts, 94 insertions and 10 deletions. Its lowest Docker adapter now models named-volume creation, real tar input transfer, container create/start/inspect, output copy and owned container/volume removal. It checks private tracked source bytes, captured Bun bytes and filtered module input while retaining image bootstrap, drift refusal, readonly/offline flags and the original browser scheduling/filter/exit assertions. New cases check all three copied formats and producer hashes for both engines, and a nonzero native exit blocking browser execution without copying output while draining modeled resources. It does not bypass the native preparation step or replace every command with success.

The first local focused file run after editing passed 14 cases and failed four: a newly added fake assertion incorrectly demanded a renderer probe omitted from the deliberately tiny source fixture. That failure is retained. The assertion was corrected to compare the fixture's actual archived wrapper source bytes; only the four affected cases reran and passed. This is not described as a final whole-file pass. Suppressing the production native exit-code rejection then made the new failure case fail specifically at NATIVE_FAILURE_MUST_BLOCK_BROWSER; restoring the producer byte-for-byte made the same case pass. Production producer SHA-256 is restored to 38f7f3e7e3b9a9b2588a3b8b5cf1d343b8a4dc322bd6aaeb4284d0cb7597aade. No production source, qualification hash, frozen pin or current-source refusal is changed by this correction.

Independent R3 review reported no findings on exact head 972e0004d5a0b3d08ebc4b8d61431adc120927e5, tree c7a8d221c1d905b1a8475d98f9fad1e4030cd04b, base 8474b80c1c83e5f11de87011de14087131188203, reviewing the full brief, prior R2/source, fix and actual producer. These fake cases prove routing/output handoff, producer-failure propagation and modeled cleanup. They do not prove native rendering, container isolation, cleanup-command failure behavior or preservation of unrelated Docker resources.

Current local candidate — 2026-10-20

Head 972e0004d5a0b3d08ebc4b8d61431adc120927e5, tree c7a8d221c1d905b1a8475d98f9fad1e4030cd04b, parent 50305761ecc341b11da7142ab72e4e5f2baa15a6, resolved base 8474b80c1c83e5f11de87011de14087131188203. Full contribution: 13 files, 345 insertions and 61 deletions. The original twelve-file contribution remains byte-identical to the published parent.

  • The ONE actual normal bun run check:pr --base origin/main exits 0 in 171.18 seconds on this exact clean candidate/base. Lint/leakage, environment documentation and changeset gates pass. Fast invariants pass 960 tests across 48 files / 6,364 assertions in 75.91 seconds; actual --changed=8474b80c1c83e5f11de87011de14087131188203 feedback passes 211 tests across 21 files / 53,520 assertions in 81.87 seconds. No retry, source change, selector skip or assertion/guard/deadline weakening was used in this default run.

  • Executed feedback includes the complete current tests/visual-runner.test.ts: all 18 cases pass, including both original hosted failures and both new volume-handoff/failure-cleanup cases. This whole-file evidence comes from the actual default feedback, not a cosmetic standalone rerun. All ten frozen-positive/current-source controls, all five Docker bind-stability cases and all thirteen lease cases also pass, including PID publication/lifetime controls and the real pinned lease container. No additional focused run is proposed because these requested controls are covered.

  • Fresh entry checks verify the owned clean branch/head/tree/base and source manifest, unchanged lockfile/private frozen copyfile install, Bun 1.4.3, Node v26.7.0, ordinary UID 1000 and HOME, all capability sets zero, loopback-only namespace without a default route, private temporary/cache/maintenance-ledger paths, original temporary/repository visibility and normal output-lease inheritance. No outer Python repository flock is used. RAM, repository disk and temporary-space floors and Btrfs device counters are checked before and after; bounded owned write probes are removed.

  • The pinned image is already local at mcr.microsoft.com/playwright@sha256:eff16c30e6f3f4af0a03fa4b706120d5e9b0891c344a27d64559aff5900a4a27, matching installed Playwright 1.63.0. No image pull occurs. The only real Docker test admitted locally is the automatically selected lease fixture with its unchanged flags and stable bind/cwd. The nonstealing native-capacity lock protects this admitted command; owned fixtures/processes/containers and locks drain at exit. No explicit native-host matrix, identity, diagram renderer, browser, Btrfs/loop campaign, install, build or standalone typecheck executes in this round.

  • The accepted actual plan selects 19 affected packages globally, 48 fast invariant files and five ordinary commands: lint/leakage, environment documentation, pending changeset package names, affected fast invariants and import-graph feedback. Its typecheck/pack and all hosted category flags describe selection requirements, not local typecheck/pack execution. Complete fresh selected hosted qualification for this head remains pending.

Any successful native jobs from failed candidate 50305761 remain evidence for their actual source/checkout; this artifact makes no claim that they executed 972e0004. Fresh current selected native-host proof must cover the complete matrix, retained-index negative/recovery and owned cleanup; default identity must retain every phase; native diagram proof must execute both engines. Earlier local preflight or historical native measurements do not substitute for those identities/results. The current hosted checklist remains open.

Checklist

  • Exact current candidate passed the actual normal local preflight recorded above.
  • R3 independent source review and scoped mutation/restoration receipts recorded with their limits.
  • Complete selected hosted proof passed for the current head/base with actual checkout identities verified.
  • Historical native/capture receipts retain original identities, missing-log limits and failed global-growth evidence.
  • PR and issue release milestone: 0.44.0; parent claim and assignment preserved.
  • Tooling/test-only contribution; no consumer-visible release change or changeset.
  • Local lint/leakage and citation checks passed; existing machine-surface and append-only provider-history constraints preserved.

Anything left open

Refs #2376 is intentionally partial. The parent owns returning the remote candidate to draft, publication of this correction/body, new selected hosted confirmation, all remaining qualification and merge windows. This artifact does not change the existing PR's state. The prior failed confirmation cannot merge. #2376 remains open until acceptance and final proof are satisfied. Per-new-worktree stable-path mounts remain explicitly out of scope. Historical raw runtime logs were not located during bounded recovery; all original 16c9ed01/5da13f8c narratives and failed global-growth limits remain attributed to those executions. #2365 remains delivered through PR #2374/squash 90a95123cfceb169bd2ecf7c213544db742eaed7.

@schlessera schlessera added this to the 0.44.0 milestone Oct 17, 2026
@schlessera schlessera added type: fix Something is broken and must behave the way it is documented. area: ci CI workflows, the gates they run, release scripts, packaging. labels Oct 17, 2026
Preserve PID publication controls as the fixture's third argument and move the stable Docker parent to its fourth argument. Retain both incoming main and original transport behavior; source-only integration, with fresh qualification pending.
@schlessera
schlessera marked this pull request as ready for review October 20, 2026 01:54
@schlessera
schlessera marked this pull request as draft October 20, 2026 02:25
@schlessera
schlessera marked this pull request as ready for review October 20, 2026 02:26
@schlessera
schlessera merged commit 9a72682 into main Oct 20, 2026
29 of 31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: ci CI workflows, the gates they run, release scripts, packaging. type: fix Something is broken and must behave the way it is documented.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant