fix(runtime-host): escalate reconnect delay while a Host never stabilizes - #3462
fix(runtime-host): escalate reconnect delay while a Host never stabilizes#3462me2seeks wants to merge 1 commit into
Conversation
jackwener
left a comment
There was a problem hiding this comment.
Automated review of exact head 6e323981ddd625ca6eab0ee3aa70ad3be83ad677 against main@f1f4e71a9540a4da23159052c924fee72202e989.
Two actionable blockers:
-
Existing valid
maxMsconfigurations now throw.requireDelayallowsmaxMsthrough 120000, and before this change a caller could set (for example){ minMs: 100, maxMs: 90000 }. Lines 116–123 defaultunstableMaxMsto 60000 and then require it to be at leastmaxMs, so that previously valid call now rejects during construction even though the caller never opted into the new field. WhenunstableMaxMsis omitted, derive it asMath.max(DEFAULT_UNSTABLE_MAX_MS, this.#maxMs)(or otherwise preserve the old valid range), and add the missing compatibility test. -
This does not fix the issue it closes. #3458 requires that prolonged flapping not exhaust the main-process heap and calls for backoff escalation / circuit-breaking instead of unbounded regeneration. This patch only changes the unbounded loop from roughly once per 5 seconds to once per 60 seconds; if the observed cross-generation retention remains, it delays the same OOM rather than bounding it. The PR itself says the retention root still needs investigation. Either re-scope this as a mitigation (
Refs #3458, keep #3458 open) or add a real bounded/circuit-breaking ownership rule or fix the retained root before usingFixes #3458.
Required conclusions:
- Optimal for the actual problem: no for the OOM claim; it is a useful churn mitigation, not the root fix.
- Production code to delete: none identified.
- Tests to delete/replace: none; add the omitted-
unstableMaxMs/ large-maxMscompatibility case. - Deeper refactor: not required if the PR is honestly narrowed to mitigation; the retention/circuit-break owner still needs a separate completed fix.
- Ready to merge: no on this head; both points above need resolution, required
testhas not run, and the PR also omits the CONTRIBUTING-required AI-use declaration (state explicitly “none” if none was used). - Residual risks: the flapping root and retained heap owner remain unknown; the new cadence still retries forever.
This changes exported reconnect behavior and user-visible recovery timing. Independent human review is required; this automated review is not approval.
…izes A Host that dies immediately after spawn previously forced a regeneration attempt roughly every maxMs (5s by default) forever. Hours of this churn re-ran the full desktop registration stack thousands of times and ended in a main-process OOM (apache#3458). Once the failure streak persists - no connection stayed installed for stableConnectionMs - the jittered delay now keeps doubling past maxMs up to an absolute ceiling (unstableMaxMs, default 60s, must be >= maxMs), and the ladder restarts as soon as a connection stabilizes again. Callers that need the old flat ceiling can set unstableMaxMs equal to maxMs. Generated-by: Maka
6e32398 to
082085c
Compare
Fixes #3458
Problem
When a Runtime Host dies immediately after spawn (transport
read_eofon every attempt), the reconnect lifecycle retried roughly everymaxMs(5s by default) forever. Each retry re-ran the full desktop registration stack — new candidate process, IPC handlers, subscriptions, capability publisher bind. ~12 hours of this churn produced hundreds of candidate generations and ended in an Electron main-process OOM (heap pinned at ~3.6 GB), with the log dominated by repeatedMCP capability alignment failederrors.Change
RuntimeHostReconnectBackoffgains an optionalunstableMaxMsceiling (default 60s, validated>= maxMs). While the failure streak persists (no connection stays installed forstableConnectionMs, which already resets the streak), the jittered delay now keeps doubling pastmaxMsup to that ceiling instead of saturating atmaxMs. As soon as a connection stabilizes, the ladder restarts fromminMs.unstableMaxMsequal tomaxMs; callers withminMs: 0are unaffected.Testing
reconnecting-connection.test.ts: delay escalates pastmaxMsand saturates atunstableMaxMs; a stabilized connection restarts the ladder;unstableMaxMs < maxMsis rejected.npm run test --workspace @maka/runtime-host: 1065/1065 pass; typecheck + biome lint clean.Note: the full desktop workspace build could not run locally because this environment's
node_modulesis missing declared deps (@slack/*) — unrelated to this change; CI will cover it.Follow-ups (not in this PR)
read_eoforigin) is observable.AI use
Tool(s) and scope: Maka (AI coding agent) authored the implementation and tests; the diff was human-reviewed before push.
Generated-by: Makatrailers are present on the branch commits.