Skip to content

ui-server: attribute the full-suite boot-prune timeout and mount timer residue #3020

Closed

Description

@schlessera

Context

Found while running the complete ui-server suite for #3002 on head 21f820efc6ebac1fad92082db8cd8a636ebd2c6b, based on main fd51cadf42918202fe3a68e0333ffdcecbdbcb8d, with Bun 1.4.2 and frozen dependencies through the shared heavy-command wrapper.

BrainUiApp.close > resolves only once a scratch prune in flight has been killed and has exited timed out at the unchanged 30,000 ms limit (packages/ui-server/tests/cron-scratch-prune.test.ts:244-276). The test awaits app creation, then polls hostPidsRunning without its own failure observation before calling app.close; the receipt does not establish which await was pending when the overall timeout fired. The PID matcher is hostPidsRunning at packages/ui-server/tests/cron-scratch-prune.test.ts:31-36.

Later in the same process, a dirty failed mount discards its resources before the next independent case failed at packages/ui-server/tests/http-supported-mounts.test.ts:244: expected zero pending timers, received one after failed-startup disposal. The observer is observeMountTimers, packages/ui-server/tests/helpers/http-mount-timers.ts:5-50. A causal relation to the earlier timeout has not been established.

Both cases pass together when filtered in a fresh test process on the base (2 tests, 122 assertions, 11.35 s) and on the unchanged head (2 tests, 122 assertions, 7.65 s). Those are isolated controls, not full-suite repairs or proof of the original cause. The complete first head run finished with 4,144 passes, four failures and 28,460 assertions in 3,585.36 s. The other two failures are the known budget race #3009 and the recovery cleanup observation #3021. The old-base and unchanged-head package controls were stopped through their owned unit-test wrappers after the coordinator requested a refresh onto main 64af9b5a0. Both exited 143 and are superseded partial runs, not passes. The refreshed complete suite on merged head 9b64841d58fb02d23292d1e864fbfc90bd226e97 subsequently passed: 4,168 pass, zero failures, 28,751 assertions across 351 files (2,502.89 s); exit 0.

Related to #736's keyless verification outcome. This issue stays standalone by reference because that epic currently has 100 direct children.

Scope

Attribute the natural full-suite timeout and the timer residue. Capture the boot-prune outcome, exact awaited boundary, real owned-child command/PID identity, app-close progress and the identity/origin of the pending timer. Determine whether the two observations share an unwind failure or need separate follow-ups.

Out of scope

The account/security write migration in #3002; changing scratch retention semantics, production deadlines, durability, host state or unrelated tests; treating an unchanged passing retry as a correction.

Acceptance criteria

  • The failing boundary is identified from a retained natural failure, or a bounded non-reproduction records its exact source/order/environment limits without claiming a repair.
  • A fixture whose boot prune cannot start or whose child cannot be observed terminates with a named failure and attempts all of its owned teardown; it does not leave an unbounded polling task or contaminate a later timer observer.
  • Any correction retains the real scratch-prune child/exit and mount cleanup assertions, with a mutation that fails the intended assertion and restored proof.
  • Independent causes are split into scoped follow-ups rather than inferred from temporal proximity.

Verification

Run bun run test packages/ui-server through the shared heavy-command wrapper, retaining the complete result. Compare the same sources with bun run test ./packages/ui-server/tests/cron-scratch-prune.test.ts ./packages/ui-server/tests/http-supported-mounts.test.ts -t 'resolves only once a scratch prune|a dirty failed mount'. Preserve the 30,000 ms timeout, real runtime, offline guards, complete descriptor restoration and owned-process cleanup.

Contract impact

None expected; this is a verification attribution and fixture-lifetime follow-up. No common cause or product repair is claimed.

2026-10-26: recorded the completed full-suite result and corrected the two helper-anchor line starts after source verification.

2026-10-26: recorded the coordinator-requested base refresh and the superseded partial comparisons without reclassifying them as passing proof.

2026-10-26 refreshed control: the complete suite on head 9b64841d58fb02d23292d1e864fbfc90bd226e97, after merging main 64af9b5a0803f6c890c9660d180a3c46a9a00ac0, reports 4,168 pass, zero failures, 28,751 assertions across 351 files (2,502.89 s); exit 0. All four earlier failing cases pass. The base changed, so this is bounded non-reproduction with different source/order/environment; it neither identifies the earlier cause nor claims a repair. The original failure receipts and superseded controls remain retained.

2026-10-26 triage at main 2f10e0407769b685083c17a1c90bb43a060b94a0: Classified as priority: p2 (wanted, unscheduled work; no release commitment inferred). The recorded controls do not establish the missing cause; retain needs: repro. The closest general verification epic #736 is at its direct-child limit, so this investigation remains standalone by reference.

Later main-CI recurrence — 2026-10-26

At main ab617740e8f6ba98681cd53348a20efd32eb7e3f, attempt 1, unit (3) fails the same dirty-mount case at a different cleanup boundary, packages/ui-server/tests/http-supported-mounts.test.ts:290: expected zero observed pending timers, received six, after closeHttpMountFixture(fresh.t) returns. The case took 3703.45 ms; the complete shard reports 2117 pass / 1 fail, 13738 assertions, 158 files, 700.98 s, with Bun 1.4.2 (744846f84). This pass read the failed job log and verified the source; it did not execute a new local reproduction. The earlier residue at line 244 was one timer after injected startup failure; neither a shared owner nor a causal link to the earlier boot-prune timeout is established.

Source context: the later branch creates a fresh fixture, opens six real native HTTP sockets, explicitly re-arms their existing idle timeouts, verifies six additional observed handles, then closes the fixture (packages/ui-server/tests/http-supported-mounts.test.ts:272-290). closeHttpMountFixture snapshots that fixture’s agent sockets and joins their close events with a bounded five-second barrier (packages/ui-server/tests/helpers/http-mount-pool.ts:10-28). The timer observer records global set/clear calls and fired callbacks; it deliberately excludes keepSocketAlive calls because a native clear can bypass that global observer (observeMountTimers, packages/ui-server/tests/helpers/http-mount-timers.ts:5-50). Six entries in the observer are not, by themselves, attribution of six still-live socket timers. Capture the retained handle identities, their socket owners and actual firing/cancellation path to distinguish unfinished resource cleanup from an observer that missed cancellation. Do not exclude more timers or widen deadlines without that evidence.

The reporter’s control records eight local focused passes on the same main. Attempt 2, unit (3) also succeeds on the unchanged source, which the run/job API confirms. Preserve attempt 1 as a failure; these controls establish variability, not a cause, repair or replacement proof.

2026-10-26 triage: Folded the new CI recurrence into the active brief, distinguished the fresh-socket cleanup stage from the earlier startup-failure stage, and named the missing owner/observer evidence. Keep p2 and needs: repro; no common mechanism or correction is selected.

Release preflight recurrence — 2026-10-26

The 0.44.0 release preparation (#3030) reproduced timer residue in the complete pre-version bun run check:pr --all on main ab617740e8f6ba98681cd53348a20efd32eb7e3f, tree b68bf8fc4b55b464ae202d0197ae1a845457b6a2, Bun 1.4.2, frozen lock SHA-256 5bbd5f6651f111db3fcfabeda512d44a21163cba54c58993f8878012d548cc14. The sole tracked difference is the Claude pricing documentation check date; no product/test/lock inputs changed. Installed dependencies use the copyfile backend.

a dirty failed mount discards its resources before the next independent case failed at line 244, expected 0 pending timer handles and received 6, after the injected startup failure was disposed (2581.67 ms). This is the earlier startup-failure boundary, not the later fresh-socket boundary at line 290. The preceding assertions establish removal of the failed fixture root and socket and closure of its DB. The observer records global timers; handle count alone does not establish live resource leakage or identify their owner. No cause, relationship to boot pruning, or correction is claimed. The full run is retained and continuing to collect its complete result; no retry replaces this failure.

This blocks the complete release proof required by #3030. Promote to p1 and schedule into 0.44.0 for attribution and a reviewed correction or an explicit maintainer disposition that preserves complete required proof; no deadline widening, timer exclusion, test skipping or retry-until-green is authorized.

Final release-preflight result: 16,367 pass, 28 skip, 1 fail, 607,520 assertions, 1,265 files, 4,360.32 seconds; the unit wrapper and check:pr --all both exited 1. The only failed assertion was the line-244 recurrence above. The 28 skips comprise 24 opt-in paid image tests and four disabled inverse sqlite-vec placeholder tests (test.if(!vecAvailable)); the actual vector cases ran and passed. Strict typecheck passed. The preflight stopped at the failing unit tier, so its subsequent pinned visual/layout/endurance, editorial verification, native host/runtime probes and packed consumer stages are unverified, not passing. No rerun, versioning, release PR, publication or merge was performed.

Corrected-source release preflight recurrence — 2026-10-27, round 4

Fresh complete pre-version bun run check:pr --all on reviewed main 1fde631f6c3e505c3a774d08c986ce637373b233, tree 02b7b2dca0b633847e91769bde72c5b9dee210fa, naturally failed the same dirty-mount case at a third cleanup boundary, packages/ui-server/tests/http-supported-mounts.test.ts:269: expected zero observed pending timers, received six, after the rebuilt shared pool completed all sharedOperations and await pool.close() returned. The case took 3453.78 ms. Earlier zero assertions after injected startup failure (line 244) and injected dirty-case disposal (line 254) passed. The later fresh native-socket rearming control at lines 272–293 was not reached.

This run includes PR #3031's reviewed native _destroyed observer correction and removal of the keepalive exclusion (observeMountTimers, packages/ui-server/tests/helpers/http-mount-timers.ts:4-34). The actual pending handles, owners and cancellation states are not printed by the current assertion; six entries alone do not establish live socket leakage, a timer-observer defect or a relationship to the historical boot-prune timeout. The scratch-prune close/child-exit case passed in 457.45 ms in this invocation.

Environment: Bun 1.4.2, Google Chrome 140.0.7339.127, independent frozen copyfile dependencies, SDK 0.3.293 and unchanged lock SHA-256 5bbd5f6651f111db3fcfabeda512d44a21163cba54c58993f8878012d548cc14. The only tracked difference is the Claude pricing verification date. The full invocation is still collecting its final result. No unchanged retry, deadline widening, exclusion, skipped assertion or proposed correction replaces this failure. Versioning and the release PR remain blocked under #3030.

Next required investigation: retain handle identity, native cancellation/liveness and owner evidence at the line-269 boundary during a bounded reproduction, preserving real mounted handlers, all existing zero assertions, six fresh-socket rearm assertions and teardown. Parent coordination owns any separately reviewed correction and integration; release preparation makes no fixture/product change.

2026-10-27 round-4 final retained result: complete pre-version bun run check:pr --all finished with exit 1, 16,386 pass / 28 expected skip / 1 fail, two snapshots, 607,677 assertions, 16,415 tests across 1,265 files in 3,922.76 seconds. The line-269 dirty-mount timer count is the sole actual outer-suite failure. Intentional failed diagnostic worker children passed their parent test. The expected skips are 24 opt-in paid image cases and four inverse sqlite-vec placeholders; real vector cases passed. Full log SHA-256: ecc2cfe0e04451c9b188c265a57e41550662585ed699de7db2f05ca6dcae7342.

Strict typecheck, setup gates and first real Chrome launch/render passed. The two-tab answer-delivery case (2799.73 ms), Liveness B (1488.48 ms), scratch-prune close/child-exit case (457.45 ms), and corrected #2482 two-client convergence case (1412.86 ms) passed. These observations do not attribute #3020's historical failures or close #3034. Later pinned visual/layout/endurance, editorial reproducibility, standalone native runtime/host probes and packed consumers did not execute after the failed unit tier. No versioning, lock refresh, template/floor edit, post-version proof, experiment receipt or release PR exists. Main was read back unchanged at 1fde631f6c3e505c3a774d08c986ce637373b233 after the run.

A separate instrumented external copy retained every existing assertion and the original timer predicate. Its first three invocations failed at earlier seed/disposal boundaries because the external path lacked the test-stack shape used by the fixture-write guard. Nested logging identified APP_CONNECTION_WRITE_WHILE_SERVING for direct fixture seeding; these are diagnostic setup failures, not reproduction or proof of a new product bug. One corrected-path diagnostic control then passed (1 test, 122 assertions, 4.26 seconds; exit 0), with zero pending handles at all existing post-disposal boundaries. Corrected diagnostic log SHA-256: b88a1020dbb08b9385557d22d9fe79f0a9605f36c7f3c31f81130a098cfd0b1e. Its focused order, copied test location and stack logging differ from the natural full run; this is bounded non-reproduction, not attribution, a repair or replacement release proof. No repository fixture/product source was edited and no unchanged full-suite retry was performed.

The remaining required action is parent-coordinated attribution of the natural line-269 failure, with retained handle/owner/native-state evidence in a failing invocation and independently reviewed resolution preserving all existing assertions; then fresh complete pre-version release proof on the integrated corrected source. Historical #3020 remains open. The later release sequence is still blocked; no human publication handoff is requested.

Bounded instrumented attribution controls — 2026-10-27 continuation

Continuation token 3020-round2-native-attribution-20261027, branch as/http-timer-attribution, fast-forwarded the independent worktree to verified main 1fde631f6c3e505c3a774d08c986ce637373b233. No competing open #3020 PR appeared at either recheck. Main remained unchanged after both complete controls. Bun 1.4.2, SDK 0.3.293, Chrome 140.0.7339.127, independent frozen copyfile dependencies and lock SHA-256 5bbd5f6651f111db3fcfabeda512d44a21163cba54c58993f8878012d548cc14 were retained. The native binary matched the earlier release hash and had link count 1.

The diagnostics stayed in the actual repository test/helper paths, forwarding original timer/socket calls without wrapping callbacks. They retained creation stacks, handle identity/type/prototype/ref state/native _destroyed, socket identity/agent ownership/request paths, explicit clear-call stacks, socket timeout/close events and cleanup phases. The expanded control also recorded boot-prune create, child-observation and close phases with the actual owned-child command/PID. No zero assertion, six-native-socket rearm assertion, dirty/stale-identity check, five-second close barrier, 30-second test timeout, offline guard or unknown-handle observation was removed or relaxed. Foreign resources and the global agent were preserved.

  • Complete package control, diagnostic tree 74dc11c9908a2cc4c5539a749607be74b8903289: bun run test packages/ui-server completed 4,206 pass / zero fail, 28,958 assertions, 354 files, 1,661.79 seconds, exit 0. Boot-prune close/exit passed in 434.70 ms; dirty-mount passed in 4415.88 ms. Log SHA-256: 8e5bb0aeb19e20363de41a5cd2901b6adb114ec6b423e5c8ee9be03d100fa488.
  • Complete default-discovery control, expanded diagnostic tree cdf2836e3e27df992323f0ae80d2ba59cd42ef54: the normal bun run test wrapper, requiring Chrome and isolating only the standalone maintenance ledger in a private mount namespace, completed 16,385 pass / 28 expected skip / 2 fail, two snapshots, 607,678 assertions, 16,415 tests across 1,265 files, 3,904.24 seconds, exit 1. The skips are 24 opt-in paid image cases and four inverse sqlite-vec placeholders; actual vector cases passed. Both actual failures are existing ci: sandbox-note fixture depends on ambient provider credentials #2769's PI_INIT_NOTE and GOVERNED_PI_INIT_RESOLVED_NOTE: the actual note names the admitted ANTHROPIC_API_KEY variable while the expected note assumes no provider keys. No credential values were printed. Planned failed diagnostic children passed their parent tests. Log SHA-256: 4636d87df154d7fe71cc118d9b8e49c042f4fc663aa77bd05034370b458ceaa8.

In the wider control, the boot-prune case passed in 454.81 ms: its real scratch-prune child was observed and none of its captured PIDs answered after app close. The dirty-mount case passed in 4509.04 ms, reaching all four cleanup zero assertions including the previously failing rebuilt-pool boundary. The test-file header order matched the retained release invocation through the target file. All four disposal-phase snapshots reached zero live handles after the existing barrier. The fresh six rearmed native socket timers were live when fixture close returned, then cancelled as their owned native sockets closed. These observations establish only the successful controls' ownership and cancellation paths.

Attribution remains incomplete. Neither complete instrumented control reproduced #3020. The original failed process ended without printing its six handle identities/states/owners, so those facts cannot be reconstructed from its count-only log. The controls differ in instrumentation overhead, invocation timing/resources, ambient provider credentials and, for complete default discovery, standalone-ledger namespace isolation. No common cause, timer-observer defect or cleanup repair is selected. The historical boot-prune timeout and the explicit unbounded-poll/named-failure/owned-teardown criterion remain open.

Every outcome and both diagnostic patches were retained locally. Temporary instrumentation was restored byte-for-byte; the worktree is clean. No implementation commit, PR, new repair mutation claim, unchanged complete retry, release proof, versioning, integration reservation, hosted confirmation, merge, publication or tag operation occurred. Preserve needs: repro, priority p1, milestone 0.44.0 and the #3030 release-proof blocker. #2769 retains its separate unscheduled/not-in-0.44.0 disposition. The remaining prerequisite is actual failure-time handle/owner/native-state evidence from a natural recurrence, followed by a separately reviewed supported correction or explicit maintainer disposition preserving complete required proof.

Draft fixture correction and failure-time evidence — 2026-10-27 round 3

Draft PR #3037, head 31d6c76414ab78952380d02cc49426efc258d83c on main 1fde631f6c3e505c3a774d08c986ce637373b233, implements the deterministic fixture criterion: a five-second boot-child observation maximum, named refusal/deadline/observation failures, actual owned scratch-prune command/PID matching, and app close before state deletion on failure. The same-file equivalent polls are bounded too. Real refusal, unobservable live-child and observation-error controls retain child reaping, DB closure and state-removal assertions. A deliberate guard bypass fails the named failure assertions; restored proof passes. Production scratch retention/deadlines and the 30-second test timeout are unchanged.

A separate controlled lifecycle defect is demonstrated with six real owned Unix HTTP connections created during fixture close. The original disposal helper snapshots no sockets before close and returns with six unref'ed native _destroyed=false timers at its zero-count boundary. Awaiting the actual server-stop promise also leaves those client timers live. The correction joins newly present owned sockets after fixture close under the existing five-second native close-event barrier, including when fixture close rejects. It does not poll timers, widen deadlines or destroy the global agent. Installed HTTP-server types and runtime confirm Bun.Server.stop returns a promise; the void signature belongs to SocketListener.stop. Neither server-side completion nor destroyed substitutes for the owned client close event.

All four original mount-zero assertions now retain failure-time phase, handle/callback identity, native destruction/ref state, socket state and owned-agent evidence. The ordinary path retains references and synchronous socket context without eager stacks, native callback wrapping, diagnostic async work or pass-time JSON. The liveness predicate is unchanged: only _destroyed === true is excluded; unknown, unref'ed and live keepalive timers count. Full descriptors and the fresh six-socket exact replacement/barrier assertions remain. Removing the diagnostic evidence fails FAILURE_RETAINS_NAMED_ORACLE; restored real-native and unknown-state controls pass.

Complete bun run test packages/ui-server: 4,213 pass / zero fail / zero skip, 29,004 assertions, 354 files, 1,667.95 seconds; exit 0. Boot close/exit passed in 466.05 ms; dirty mount passed in 4587.53 ms, reaching all four zero checks and the six-socket control. Complete package log SHA-256: d405c83b960c27d48571f1c04ebbdaa038f91ded8b62900b9dc32a2b6df94d2c. bun run check:pr --base origin/main exits 0, including 209 fast-invariant tests / 1,666 assertions across seven files; final strict typecheck exits 0. This is one purposeful package run for the proven correction, not another unchanged default-discovery or release retry.

Package input tree ce543ad34f8782c7dc3410ad932e84dc207fe0d5 and final tree f75ef9d88ea7f47293a9618ccce2dee3f9df4200 differ only by a diagnostic-helper lint comment and TypeScript annotations added during the invocation. Bun-transpiled minified JavaScript is byte-identical before/after (SHA-256 be717d343a03bda2c9bf8d7839706327ed29d0728d2904380e915981c88b3aad); final strict typecheck covers those annotations. Bun 1.4.2, Chrome 140.0.7339.127, SDK 0.3.293, independent frozen copyfile install and unchanged lock SHA-256 5bbd5f6651f111db3fcfabeda512d44a21163cba54c58993f8878012d548cc14 are retained. Main remains 1fde631f6c3e505c3a774d08c986ce637373b233.

Historical attribution remains open. The controlled late-connection defect does not identify the original failed processes' six handles or prove a common boot-prune cause. The draft delivers independently useful fixture corrections and durable future evidence. Keep needs: repro, p1 and milestone 0.44.0; no release resolution or replacement complete release proof is claimed. Parent review owns hosted proof/integration, the next natural failure collection and release-blocker disposition. No integration lease/confirmation, CI watching, merge, versioning, publishing or tags were performed.

Final committed-head preflight at 31d6c76414ab78952380d02cc49426efc258d83c also exits 0: 209 fast-invariant tests, 1,666 assertions, seven files, 12.73 seconds. Log SHA-256 02513d732a4c4783f65f897a58b15bf3a3adeda92a92ae1e8c014bceddc638be. The repeated short preflight covers final type-only lint inputs; no complete package/default-discovery/release run was repeated.

Current-main release proof — 2026-10-27, round 5

Complete pre-version bun run check:pr --all is collecting results on reviewed main 5eebfc6289f07f11106de5c6aec9b5abd9355358, including integrated PR #3037 and its parent-reviewed rejection cleanup. The boot-prune close/child-exit case now fails with the bounded named outcome BOOT_PRUNE_CHILD_NOT_OBSERVED phase=observe-child outcome=prune-failed elapsedMs=18 iterations=2 (observeBootPrune, packages/ui-server/tests/cron-scratch-prune.test.ts:275-275), called at line 316; the whole case took 272.24 ms. This is a reported prune failure before observing the actual child, not the historical unbounded hang or evidence that the five-second bound expired. The observer-error negative control separately fails BOOT_OBSERVATION_HAS_NAMED_FAILURE at line 358: it expected BOOT_PRUNE_OBSERVATION_FAILED, but received the same prune-failed outcome after 16 ms/two iterations (270.50 ms whole case), before its intended observation-error branch. The refusal and deadline controls pass.

The full invocation has additional failures retained under #3030: eight default-ledger setup guards, #3038's geo cancellation assertion, #3010's strict scratch-writability assertion and #2866's three app-close command-start waits. The isolated default ledger was incorrectly created mode 0755 rather than required 0700; separate same-source native API controls reject 0755 and admit/release at 0700. The same process also logs worker qualification/version-probe EINVAL: invalid argument, fchown. No per-case failing-prune trace currently attributes these boot-prune failures to either setup condition. Do not infer a common cause or claim a production defect from the named outcome alone.

Bun 1.4.2, frozen copyfile dependencies, SDK 0.3.293 and lock SHA-256 5bbd5f6651f111db3fcfabeda512d44a21163cba54c58993f8878012d548cc14 are retained. Only the pricing verification date differs from main; no active source/environment change or unchanged retry was performed. The full run and eventual exact counts/log digest will be retained. Native mount-zero/fresh-socket results are still pending. Historical six-handle attribution remains open; no issue completion, release resolution, versioning or publication is claimed. Parent coordination owns attribution and any independently reviewed correction before another complete proof.

Failure-time dirty-mount diagnostic — round 5

The same complete invocation now fails a dirty failed mount discards its resources before the next independent case at packages/ui-server/tests/http-supported-mounts.test.ts:255, phase dirty-case-disposed: expected zero, received six (2259.01 ms). The earlier startup-failure zero check passed. The rebuilt shared-pool zero and fresh six-socket rearm control were not reached.

The integrated lazy observer retained the actual failed boundary before teardown. All six handles are native live (nativeDestroyed: false), referenced (hasRef: true) Timeout objects, delay 30000, anonymous distinct callback identities, with no recorded socket association. The two recorded owned Unix HTTP sockets are already destroyed and closed; their owned agents have no active/free sockets or queued requests. socket: null means association was not recorded, not proof of an origin. Do not assign these timers to the controlled late-connection defect, the historical processes, test deadlines or another subsystem without attribution. No new instrumentation, callback wrapping, timer filtering, polling or timeout change was added.

Exact extracted JSON SHA-256: f23b34f19d04741775a1de8dc57fd5522d2227d9812574ec193fb67ae3548c04. Complete payload follows; all paths are fictional temporary fixture sockets.

[
  {
    "phase": "dirty-case-disposed",
    "pending": 6,
    "handles": [
      {
        "id": 1,
        "type": "object",
        "prototype": "Timeout",
        "nativeDestroyed": false,
        "hasRef": true,
        "ownProperties": [],
        "kind": "timeout",
        "delay": 30000,
        "callback": {
          "id": 2,
          "name": ""
        },
        "socket": null
      },
      {
        "id": 3,
        "type": "object",
        "prototype": "Timeout",
        "nativeDestroyed": false,
        "hasRef": true,
        "ownProperties": [],
        "kind": "timeout",
        "delay": 30000,
        "callback": {
          "id": 4,
          "name": ""
        },
        "socket": null
      },
      {
        "id": 5,
        "type": "object",
        "prototype": "Timeout",
        "nativeDestroyed": false,
        "hasRef": true,
        "ownProperties": [],
        "kind": "timeout",
        "delay": 30000,
        "callback": {
          "id": 6,
          "name": ""
        },
        "socket": null
      },
      {
        "id": 7,
        "type": "object",
        "prototype": "Timeout",
        "nativeDestroyed": false,
        "hasRef": true,
        "ownProperties": [],
        "kind": "timeout",
        "delay": 30000,
        "callback": {
          "id": 8,
          "name": ""
        },
        "socket": null
      },
      {
        "id": 9,
        "type": "object",
        "prototype": "Timeout",
        "nativeDestroyed": false,
        "hasRef": true,
        "ownProperties": [],
        "kind": "timeout",
        "delay": 30000,
        "callback": {
          "id": 10,
          "name": ""
        },
        "socket": null
      },
      {
        "id": 11,
        "type": "object",
        "prototype": "Timeout",
        "nativeDestroyed": false,
        "hasRef": true,
        "ownProperties": [],
        "kind": "timeout",
        "delay": 30000,
        "callback": {
          "id": 12,
          "name": ""
        },
        "socket": null
      }
    ],
    "sockets": [
      {
        "id": 13,
        "ownedAgentKeys": [
          "localhost:80::/tmp/odysseus-native-app-Hks5rC/native-test-manager.sock"
        ],
        "destroyed": true,
        "closed": true,
        "connecting": false,
        "pending": true,
        "timeout": 5000,
        "readable": false,
        "writable": false
      },
      {
        "id": 14,
        "ownedAgentKeys": [
          "localhost:80::/tmp/odysseus-native-app-0exWJ4/native-test-manager.sock"
        ],
        "destroyed": true,
        "closed": true,
        "connecting": false,
        "pending": true,
        "timeout": 5000,
        "readable": false,
        "writable": false
      }
    ],
    "ownedAgents": [
      {
        "key": "localhost:80::/tmp/odysseus-native-app-Hks5rC/native-test-manager.sock",
        "active": [],
        "free": [],
        "queuedRequests": 0
      },
      {
        "key": "localhost:80::/tmp/odysseus-native-app-0exWJ4/native-test-manager.sock",
        "active": [],
        "free": [],
        "queuedRequests": 0
      }
    ]
  }
]

The independently recorded ledger-mode setup error and process qualification errors remain caveats for this invocation. Six live handles here do not establish their cause. This retained failing oracle is not replaced by another run; parent coordination owns bounded attribution and reviewed correction.

Final retained round-5 invocation — 2026-10-27

The complete pre-version bun run check:pr --all finished with exit 1: 16,368 pass / 28 expected skip / 27 fail, two snapshots, 607,298 assertions, 16,423 tests across 1,265 files in 3,918.90 seconds. Head and fetched main both remain 5eebfc6289f07f11106de5c6aec9b5abd9355358; only the Claude pricing check date is edited. Bun 1.4.2, SDK 0.3.293, independently copied frozen dependencies and lock SHA-256 5bbd5f6651f111db3fcfabeda512d44a21163cba54c58993f8878012d548cc14 are unchanged. Complete log SHA-256: b894c1b32b57d47530eb2cdc1d2d6cc84d671679ffbdd2dc10ce0406708a0d59.

The 28 skips are 24 opt-in paid image cases and four inverse sqlite-vec placeholders; real vector cases passed. Two planned failed diagnostic workers passed their parent evidence-handoff test and are not additional outer failures. No paid inference call or registry mutation was made. Setup/lint/leakage, metadata, pinned fonts and the first real Chrome launch/render passed. The standalone pinned browser/layout/endurance, editorial reproducibility, native probes and packed consumer stages did not execute after the failed unit tier. No versioning, post-version proof, experiment receipt or release PR exists.

The isolated ledger's original mode 0755 is retained as a preparation error. Native controls establish its exact guard refusal at 0755 and successful admission/release at 0700; they do not constitute a passing release gate or attribute unrelated symptoms. Full failure contexts and source/environment identities are retained. No unchanged full retry, assertion/deadline change or waiver replaced the failure. Parent coordination must reconcile the setup error and actual failures, preserve current scheduling rulings, and select any supported separately reviewed correction before authorizing new complete proof. Historical attribution remains open.

Final explicit bun run typecheck on the unchanged failed-proof source also exits 0 (strict tsc --noEmit); this focused result does not replace the failed complete gate. Log SHA-256: 8366207267355d3e3d5bf3bf6e8c94c5f93f6078c34f08973fa2b38cdda6cc92.

Bounded callback attribution and diagnostic slice — 2026-10-27

Draft #3041 on c220802b58891a7acab222c338f5d61f3cf2b1f6, based on reviewed main 5eebfc6289f07f11106de5c6aec9b5abd9355358, retains callback function source in the existing failure-time mount snapshot and the real recorded prune-error attributes in a named boot-observation failure. This changes test diagnostics only; it does not identify the historical callbacks or claim a runtime repair. No creation-stack capture or callback wrapping is added to the normal registration path. Every original zero assertion, liveness predicate, deadline and real child/socket control remains.

A fresh dirty-mount control passed 122 assertions. A bounded combined run of the two complete relevant test files passed 123 tests / 1,036 assertions in 106.69 seconds; a temporary trace limited to 30-second registrations recorded none. These narrower source/order/environment controls do not reproduce or replace the retained complete failure. An attempted DOM-signal preload comparison retained no effective signal identity and remains inconclusive.

The anonymous real 30-second ref'ed timer regression failed specifically at ANONYMOUS_TIMER_FAILURE_RETAINS_CALLBACK_SOURCE on the original helper. The real exit-23 boot-prune control failed at BOOT_REFUSAL_RETAINS_PRUNE_ERROR. Removing the new diagnostic values makes both assertions fail again. Restored focused proof passes 15 tests / 205 assertions across four files, including the original dirty-mount case, native sockets, descriptor restoration and actual prune child/exit checks (8.33 seconds, exit 0). Final restored log SHA-256: 6c513e56bd3001dee3dee881874f41c7c9300f70e6013ca6c4469d1ee8e0a9ab.

Strict typecheck, lint, leakage and required bun run check:pr --base origin/main pass; preflight includes 209 fast tests / 1,666 assertions. Bun 1.4.2, SDK 0.3.293 and frozen copyfile lock SHA-256 5bbd5f6651f111db3fcfabeda512d44a21163cba54c58993f8878012d548cc14 match. Every namespace command used its own explicit 0700 ledger.

Historical #3020 remains open. The failed round-5 receipt contains no callback source/creation stack and no underlying prune-error attributes. Its six non-socket-associated live callback registrations remain unattributed, and its two boot-prune outcomes cannot yet be classified as setup propagation or independent defects. Draft #3041 prepares those missing failure details for independent review; parent coordination owns attribution, complete required proof and any release-blocker disposition. The original failed full run is preserved. No full-suite retry, versioning, release PR, experiment tier, lease, confirmation, merge, publication, tags or CI polling/watch was performed.

Activity

  1. added
    type: testCoverage, harness or proof work with no behaviour change.
    area: ui-serverHono app factory: turn coordinator, auth, session catalog, routes, cron.
    needs: reproBlocked on a reproduction.
    on Oct 26, 2026
  2. added this to the 0.45.0 milestone on Oct 26, 2026
  3. added
    priority: p2Wanted, not scheduled. Pulled in when a milestone has room.
    on Oct 26, 2026
  4. schlessera commented on Oct 26, 2026

    @schlessera
    OwnerAuthor

    Seen again in main CI on ab61774 (#3029, run 40385356890, unit (3)). This time it failed at packages/ui-server/tests/http-supported-mounts.test.ts:290: six socket idle timers were still pending after the fresh fixture closed, where the earlier failure at line 244 had one. On the same main, -t 'a dirty failed mount' passed 8 of 8 locally, and the failed-job rerun of that CI run passed. It is the same mount-timer-residue class; the cause is still unattributed.

  5. added
    priority: p1Scheduled into the current or next milestone.
    and removed
    priority: p2Wanted, not scheduled. Pulled in when a milestone has room.
    on Oct 26, 2026
  6. modified the milestones: 0.45.0, 0.44.0 on Oct 26, 2026
  7. self-assigned this
    on Oct 26, 2026
  8. schlessera commented on Oct 26, 2026

    @schlessera
    OwnerAuthor

    Assigned investigation for the 0.44.0 release blocker. No open PR or competing assignee was present at claim time. Retaining the original complete release failure; bounded controls will record observed handle identity, origin, runtime destruction state and native socket ownership before selecting a correction. The earlier scratch-prune timeout remains unattributed and is not assumed to share the timer mechanism. No merge or publication in this investigation.

  9. schlessera commented on Oct 27, 2026

    @schlessera
    OwnerAuthor

    Continuing the existing team investigation by explicit handoff on branch as/http-timer-attribution, token 3020-round2-native-attribution-20261027, at verified main 1fde631f6c3e505c3a774d08c986ce637373b233. No open matching PR was present on recheck. The existing assignee is retained. This continuation targets handle identity, creation origin, native state and socket ownership at the newly retained shared-pool cleanup failure. Historical boot-prune and other cleanup boundaries remain distinct and unattributed. Existing assertions, native deadlines, offline guards and foreign resources remain intact. Any supported correction will be handed back as a draft for parent review; no merge or publication is part of this continuation.

  10. schlessera commented on Oct 27, 2026

    @schlessera
    OwnerAuthor

    The continuation control on reviewed main 1fde631f6c3e505c3a774d08c986ce637373b233 completed bun run test packages/ui-server: 4,206 pass, zero failures, 28,958 assertions, 354 files, 1,661.79 seconds, exit 0. Diagnostic tree 74dc11c9908a2cc4c5539a749607be74b8903289 adds registration stacks, native handle state, socket identities/events and cleanup-phase snapshots in the actual repository helper paths. The timer predicate, callbacks, all existing mount assertions, native sockets, five-second close barrier and 30-second test timeout remain intact. Bun 1.4.2, SDK 0.3.293 and frozen copyfile lock SHA-256 5bbd5f6651f111db3fcfabeda512d44a21163cba54c58993f8878012d548cc14 match the retained release run. Full package log SHA-256: 8e5bb0aeb19e20363de41a5cd2901b6adb114ec6b423e5c8ee9be03d100fa488.

    The boot-prune close/exit case passed (434.70 ms). The dirty-mount case passed (4415.88 ms): all four post-disposal zero boundaries were reached. In this invocation, application close returned before native keepalive cancellation; the existing owned-socket barrier then joined the close events and all retained handles reported destroyed. The six fresh, explicitly rearmed native socket timers were live before that barrier and cancelled afterwards. This confirms the existing barrier is necessary in this control; it does not attribute any historical six-handle failure or support a new cleanup change.

    One complete default-discovery control is now running to test the wider cross-package order absent from the package control. Diagnostic tree cdf2836e3e27df992323f0ae80d2ba59cd42ef54 additionally records clear-call stacks, socket request paths and boot-prune phase/owned-child command/PID markers. The expanded focused pair passed (2 tests, 124 assertions, 6.29 seconds); all five existing timer/socket helper controls passed (29 assertions). Real Chrome is required, and only the standalone temporary-maintenance ledger is isolated in a private mount namespace, preserving UID/GID, zero capability sets and other checkouts' resources. Every outcome of the complete control will be retained. No unchanged full retry, correction, release proof, versioning or publication is claimed. #3020 and the #3030 release blocker remain open.

  11. schlessera commented on Oct 27, 2026

    @schlessera
    OwnerAuthor

    Round 6 complete pre-version proof naturally reproduced the zero-timer failure on reviewed main 3fb5199797c7ef3c9b864b4f9b79146a5b5cc267, before any versioning. The existing assertion failed at packages/ui-server/tests/http-supported-mounts.test.ts:245, phase failed-startup-disposed: expected 0, received 6 (1109.77 ms). The later dirty-disposal, rebuilt-pool and fresh-native-socket controls were not reached in that test. Named boot-prune refusal/deadline/observer-error cases passed separately; no boot-error JSON was emitted for this zero-oracle failure.

    The new failure-only callback-source diagnostic was captured synchronously by the original assertion before cleanup, from this same complete invocation. All six handles are native-live, referenced anonymous 30000-ms timeouts. Every callback source is () => {\n this.reconnectTimer = null;\n this.connect();\n }. This source matches scheduleReconnect in packages/ui-sdk/src/client/ws-client.ts:544-551; that is a source match, not yet attribution of the six client instances or their creation/closure path. The one observed fixture socket is closed/destroyed; its owned agent active/free/request tables are empty. This does not retroactively identify earlier rounds’ handles.

    Environment: NEW owned 0700 default ledger; official user/mount namespace launcher preserving UID/GID 1000, one-entry maps and five zero capability sets; Bun 1.4.2; frozen copyfile dependencies; SDK-relative 0.3.293 binary SHA-256 8968405e26db478af44eabc4635ab5ca557057b702a54460a59c13e1b253e978; unchanged lock SHA-256 5bbd5f6651f111db3fcfabeda512d44a21163cba54c58993f8878012d548cc14. No source, environment or deadline changed during proof. The invocation is continuing to retain its final result; no full retry, versioning or release PR will substitute for this failure.

    Exact failure JSON (retained standalone file SHA-256 ce87b51745687af74c0f815792026ed460892c134033f39c084aa0ac4ee9e67a):

    {
      "phase": "failed-startup-disposed",
      "pending": 6,
      "handles": [
        {
          "id": 1,
          "type": "object",
          "prototype": "Timeout",
          "nativeDestroyed": false,
          "hasRef": true,
          "ownProperties": [],
          "kind": "timeout",
          "delay": 30000,
          "callback": {
            "id": 2,
            "name": "",
            "source": "() => {\n      this.reconnectTimer = null;\n      this.connect();\n    }"
          },
          "socket": null
        },
        {
          "id": 3,
          "type": "object",
          "prototype": "Timeout",
          "nativeDestroyed": false,
          "hasRef": true,
          "ownProperties": [],
          "kind": "timeout",
          "delay": 30000,
          "callback": {
            "id": 4,
            "name": "",
            "source": "() => {\n      this.reconnectTimer = null;\n      this.connect();\n    }"
          },
          "socket": null
        },
        {
          "id": 5,
          "type": "object",
          "prototype": "Timeout",
          "nativeDestroyed": false,
          "hasRef": true,
          "ownProperties": [],
          "kind": "timeout",
          "delay": 30000,
          "callback": {
            "id": 6,
            "name": "",
            "source": "() => {\n      this.reconnectTimer = null;\n      this.connect();\n    }"
          },
          "socket": null
        },
        {
          "id": 7,
          "type": "object",
          "prototype": "Timeout",
          "nativeDestroyed": false,
          "hasRef": true,
          "ownProperties": [],
          "kind": "timeout",
          "delay": 30000,
          "callback": {
            "id": 8,
            "name": "",
            "source": "() => {\n      this.reconnectTimer = null;\n      this.connect();\n    }"
          },
          "socket": null
        },
        {
          "id": 9,
          "type": "object",
          "prototype": "Timeout",
          "nativeDestroyed": false,
          "hasRef": true,
          "ownProperties": [],
          "kind": "timeout",
          "delay": 30000,
          "callback": {
            "id": 10,
            "name": "",
            "source": "() => {\n      this.reconnectTimer = null;\n      this.connect();\n    }"
          },
          "socket": null
        },
        {
          "id": 11,
          "type": "object",
          "prototype": "Timeout",
          "nativeDestroyed": false,
          "hasRef": true,
          "ownProperties": [],
          "kind": "timeout",
          "delay": 30000,
          "callback": {
            "id": 12,
            "name": "",
            "source": "() => {\n      this.reconnectTimer = null;\n      this.connect();\n    }"
          },
          "socket": null
        }
      ],
      "sockets": [
        {
          "id": 13,
          "ownedAgentKeys": [
            "localhost:80::/tmp/odysseus-native-app-0QIRQ0/native-test-manager.sock"
          ],
          "destroyed": true,
          "closed": true,
          "connecting": false,
          "pending": true,
          "timeout": 5000,
          "readable": false,
          "writable": false
        }
      ],
      "ownedAgents": [
        {
          "key": "localhost:80::/tmp/odysseus-native-app-0QIRQ0/native-test-manager.sock",
          "active": [],
          "free": [],
          "queuedRequests": 0
        }
      ]
    }

    Parent next action: coordinate bounded attribution of the observed reconnect callbacks to their actual client/fixture lifecycle, preserving all zero oracles, original deadlines and liveness behavior, then independently review any supported correction. This release implementation performs no unrelated fix or scheduling change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area: ui-serverHono app factory: turn coordinator, auth, session catalog, routes, cron.needs: reproBlocked on a reproduction.priority: p1Scheduled into the current or next milestone.type: testCoverage, harness or proof work with no behaviour change.

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions