test(runtime-host): attribute Windows startup stalls

[liugddx]

Summary

Adds bounded structured evidence to Runtime Host election timeout failures and a repeatable Windows Node Runtime Host fresh-root startup stress entry point.

The diagnostic distinguishes the observed startup cut with the exact Candidate PID/attempt identity and exit state, the final election elapsed time, endpoint-connected observation, fixed observation counters, and the last safe registration summary. Desktop and CLI startup errors include the same structured JSON.

This is diagnostic-only with respect to startup policy. It does not change the 45-second election deadline, retry/respawn policy, candidate backoff, endpoint behavior, public wire protocol, or existing failure classification.

Refs #3279.

Diagnostic boundary

• Candidate launches are counted by launch-object identity, so a coalesced owned launch remains one process attempt even when election asks repeatedly.
• Endpoint evidence is written at the connection phase: registration/read/connect failures record false; draining, incompatible, and post-connect handshake failures record true. The internal phase field is stripped from the existing public connection result.
• Quick exits retain exact code/signal evidence; running attempts remain distinguishable from exited/unknown attempts.
• The diagnostic does not add the runtime root path or endpoint value to startup errors.
• The detached Candidate still uses stdio: 'ignore'. This PR does not claim to capture child stderr.
• sawEndpointConnected and the observation counters are election observations plus final elapsed time, not exact endpoint publication/phase timestamps.

Capturing child stderr, adding detailed phase timestamps, or changing startup policy remains deferred until this evidence classifies another real failure.

Node stress harness

npm run stress:windows-runtime-host-node-startup -- <iterations> <parallel> <electionDeadlineMs> <settleTimeoutMs> [keep-failures]

The public command performs the repository root build from a checkout without workspace dist, then delegates to the explicit stress:windows-runtime-host-node-startup:dist seam. The runner creates an isolated root per iteration, emits candidateRuntime: "node" in every NDJSON iteration and summary, coalesces ownership of the exact Candidate attempt, and settles that attempt before removing the root. A rejected spawn is reported as election failure evidence rather than overriding the iteration in cleanup; resolved-attempt settlement errors still propagate.

This harness does not represent packaged Electron startup. The independent packaged boundary remains Release Windows check, which packages Maka and runs packaged release smoke, upgrade/uninstall, and automatic update end to end.

Verification

L1/L2 on exact head 8ab1ce2cd78f4f5fe7eadc5bd981b1aa4f65e00b rebased onto main 2e8cd03babbcccc007b9723300c4b363b21f0e02:

• npm run build passed
• npm run typecheck passed across all workspaces
• npm run lint: 2,517 files, 0 errors
• npm run format:check: 1,577 files, 0 changes
• checkout-only CI contracts: 63 passed, 0 failed, 1 privilege-dependent symlink skip
• election endpoint/coalescing diagnostics: 8 passed
• real pre-connect/post-handshake root phase writers: 2 passed
• startup-error + Node stress tests: 10 passed
• owned Candidate quick-exit: 1 passed
• CLI Runtime Host context: 9 passed
• Desktop Runtime Host manager: 22 passed
• git diff --check passed
• independent author-gate review: no unresolved P0-P2 findings

Windows Node dynamic evidence on this exact head:

• fresh-root dist stress: 1/1 connected in 7.761 seconds
• output identifies candidateRuntime: "node"
• exact Candidate exited with code 0 / no signal
• Candidate PID exited and the new root was removed

The self-contained public wrapper was also executed after removing all workspace dist; it completed the root build and connected 1/1 rather than failing before the runner.

Known local noise: an unrelated Runtime Host full-suite process from an earlier run remains stalled on this machine. Two pre-existing forged-handshake tests can also observe the Windows endpoint ACL probe's extra empty connection as read_eof; the new writer tests isolate that fixture connection and pass. Neither process was terminated or used as passing evidence.

L3 on the pushed exact head:

• core CI 32567384326 passed in 14m55s, including the full Runtime Host suite, Desktop E2E, Storybook, and installed CLI validation
• Dependency audit 32567384309 passed in 44s
• Release Windows check 32567384310 reached the final automatic-update startup and then reproduced runtime-host: intermittent 'stopped responding during startup' after a Windows upgrade over an existing profile #3279. The packaged app reported a 45.174s election with 69 Candidate launches, one connected/failed-ready observation, a final recovering registration, and a still-running latest Candidate. Packaging, packaged release smoke, and pinned upgrade/uninstall passed before this product failure.

This failed packaged run is the first classified occurrence produced by this PR; it is not evidence that the startup behavior is fixed, and it should not be hidden by verifier retries. The behavior fix remains a separate #3279 change.

Remaining work

• Use a future real occurrence to classify Candidate exit, endpoint observation, or stale registration before proposing a behavior fix.
• Decide separately whether a bounded retry/respawn, endpoint hygiene, or deadline policy change is justified.

AI use

Select exactly one:

• No generative tool made a substantive contribution
• Generative tooling made a substantive contribution

Tool(s) and scope: Codex implemented the bounded diagnostic and Node stress harness, addressed review findings, added focused regression coverage, ran local verification, and used an independent review agent for the author gate. Both commits include a Generated-by: Codex trailer.

Checklist

• Tests cover the change and fail without it
• Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

• Yes — startup error diagnostics and the new developer stress command are described above
• No

[Astro-Han]

Thanks for strengthening the Runtime Host startup diagnostics and keeping the wire/lifecycle behavior unchanged. The exact-head checks are green, but three parts of the new diagnostic/stress contract still lose or overstate the evidence they are meant to provide. I left focused P2 comments inline; none require a parallel startup path.\n\nAI-assisted review disclosure: Codex performed an independent exact-head review and reproduction; I verified the reported paths and live PR state before posting.

[Astro-Han]

[P2] draining is returned only after a Host accepted the connection and completed a handshake, but this branch records endpointConnected: false. A timeout then reports sawEndpointConnected:false, removing the key distinction this PR is trying to diagnose. Please derive this flag from whether a handshake was reached (including draining, and any equivalent incompatible-handshake result), and add a regression assertion for the resulting diagnostic.

[liugddx]

Fixed in 8ab1ce2. Election-only unavailable results now carry a required internal �ndpointConnected phase fact written by connectResolvedRuntimeHost, rather than deriving it from the ambiguous reason. Registration/read/connect failures write alse; draining, non-blocking incompatible, and post-connect handshake failures write rue. inalizeConnectRuntimeHostResult strips the field so the public result contract is unchanged. The consumer table covers all six diagnostic variants, and real forged-peer tests distinguish registration root mismatch (alse) from an accepted wrong-root handshake ( rue).

[Astro-Han]

[P2] This new public stress entry cannot run from a clean checkout because building only @maka/runtime-host leaves referenced workspace dist artifacts absent and tsc fails before the runner starts. Please make the command self-contained using the existing root/reference build seam (as smoke:windows does), or explicitly split it into a documented :dist command whose prerequisite is enforced by the caller.

[liugddx]

Fixed in 8ab1ce2. The public command is now stress:windows-runtime-host-node-startup and reuses the root
pm run build seam before delegating to an explicit stress:windows-runtime-host-node-startup:dist command. The contract test pins both scripts and rejects the old ambiguous command. I also removed all workspace dist locally and ran the public wrapper successfully, so it no longer depends on stale referenced outputs.

[Astro-Han]

[P2] This stress path launches the candidate with the default process.execPath; under npm run, that is Node, not the packaged Electron executable implicated by the Windows startup stall. The result therefore demonstrates a Node fresh-root launch, not the advertised packaged path. Please either pass the packaged executable and run this in the Windows/package lane, or name/document it as Node-only and add a separate packaged smoke boundary. That keeps the evidence claim aligned with what the harness actually exercises.

[liugddx]

Fixed by narrowing the evidence contract in 8ab1ce2, without adding a parallel packaged startup path. The file, command, exported functions, error text, and PR wording now identify this as Node-only; every iteration and summary emits candidateRuntime: \node. Packaged Electron remains a separate boundary owned by Release Windows check, which is triggered on this PR and will be treated as the authoritative packaged smoke rather than inferred from the Node stress result.

[Astro-Han]

Thanks for this — and worth saying up front, because the CI state makes it look worse than it is: the failing package job is this PR working, not this PR breaking.

Verify automatic update end to end failed because the packaged app genuinely stalled during Runtime Host startup. Before this PR that failure would have read Runtime Host did not become ready and nothing else. Instead it produced:

{"deadlineMs":45000,"elapsedMs":45174,"candidateLaunches":69,
 "sawEndpointConnected":true,
 "observations":{"notRegistered":73,"connectFailed":0,"handshakeFailed":0,
                 "connected":1,"readyWaitFailed":1,"deadlineElapsed":0},
 "lastRegistration":{"pid":8268,"state":"recovering","lifecycleMode":"ephemeral"}}

That is exactly the evidence the PR set out to capture: 69 candidate launches in 45s, zero connect/handshake failures, one successful connection followed by a ready-wait failure, and a timeout that missed by 174ms. It says the problem is retry convergence, not a deadline that's slightly too short — which is a materially different repair than the one you'd guess from the old error message.

Four questions: the problem is that startup stalls were unattributable; the fix instruments the existing election loop with counters and a candidate/registration snapshot and appends a bounded JSON diagnostic on failure. Right layer — evidence before repair, and the instrumentation is pure observation with no behavioural branch. Counters reuse the existing connection/election authority rather than standing up a second view of startup state.

We checked the counters rather than trusting them, since the conclusions above depend on them: candidateLaunches de-duplicates via a Set (so coalesced launches count once), and notRegistered counts poll observations rather than launches — so 73 > 69 is definitional, not double-counting. Two reviewers verified this independently.

On the CI gate: we can't approve while package is red — approval means "safe to merge," and merging red turns main red, regardless of whose fault it is. This isn't a mark against the PR. Once the underlying stall is fixed (or the job is green on this head for another reason), the review conclusion is otherwise clean: the three earlier P2s are all addressed at this head.

One finding inline.

---

AI-assisted review. Two reviewers worked this independently and converged on the same finding and the same counter verification.

[Astro-Han]

[P3] Four unavailable reasons fall through the switch without landing in any bucket, and there's no other or total to reveal it.

recordElectionResult counts not_registered, connect_failed, and handshake_failed. RuntimeHostUnavailableReason also includes invalid_registration, root_mismatch, composition_mismatch, and epoch_mismatch — those keep the election going but increment nothing. The early return above for result.kind !== 'unavailable' is silent in the same way.

Why it matters more than a missing counter usually would: this diagnostic exists to tell someone where to look, and the failure mode is that it points somewhere wrong. If a run stalls mainly on epoch_mismatch, the JSON reports notRegistered: 0, connectFailed: 0, handshakeFailed: 0 — and the person reading it concludes the connection layer is healthy and goes hunting elsewhere. A diagnostic that quietly omits the actual cause is worse than no diagnostic, because it carries authority.

The loop above already shows how easy the fix is: add the missing cases, or an other bucket plus a totalAttempts field so the buckets can be reconciled against it. The total is the part worth having — it makes any future omission visible instead of silent, including reasons added after this PR.