Repository navigation
Fix cluster service port allocation races - #155
Conversation
valthon
left a comment
There was a problem hiding this comment.
Approved at 737f4fe. No correctness findings.
Coverage: read every changed line in all 10 files and traced allocation, registry locking, persisted reclaim, and validator readiness. Compared all seven of my other open PRs and the relevant recently merged registry, locking, and readiness changes. The shared Node/Python registration protocol and persisted relaunch guarantees remain intact.
Merge simulations: #122 has one additive export conflict in packages/cluster-tool/src/utils/index.ts; retain both assetUtils.js and portRangeUtils.js. #79 already conflicts with the base in 146 files, with no additional conflicting paths introduced here. #157, #152, #149, #136, and #80 merge cleanly. No opposing behavioral intent found.
Validation: 168/168 focused tests passed locally, including Python interoperability and WebSocket readiness. The initial eight readiness failures reproduced on base 58facaa with identical failing test names; the reused installation lacked the declared ws dependency, and supplying ws 8 in the temporary checkout resolved them. Exact-head CI passed 466 suites / 4,692 Jest tests plus package unit lanes. No live E2E or native macOS/Windows execution performed locally.
Summary
Why
This PR was motivated by failed E2E run 40113142471. Both attempts finished 13/14: attempt 1 failed
emissions-soakwhen PubSub could not bind0.0.0.0:34502; attempt 2 failedbatch-operator-slashingon0.0.0.0:34662. Each reportedAddress already in usewhile the validator continued producing slots until its 480-second readiness timeout.The cross-language bind registry coordinates planned ports among cooperating processes, but
get-portcloses its probe sockets before returning. An unrelated outbound connection can then receive a planned service port as its automatic source port, causing the later daemon bind to fail withEADDRINUSE. The registry support now onmasterstill permits this kernel-selected fallback; reserving PubSub as RPC + 1 alone cannot prevent it.The historical runs did not capture the conflicting socket owner at bind time, so their particular holders are not proven. A separate local reproduction demonstrated kernel-assigned source-port reuse of a registry-reserved PubSub port and the resulting
EADDRINUSE. This PR prevents that reproduced collision class and captures prompt bind-time diagnostics for future failures.Automatic allocation now uses explicit candidates outside observed Linux, macOS, and Windows client-port ranges, with randomized fallback starting points. Unreadable host settings and exhausted safe pools fail automatic allocation. Fully pinned plans retain their availability checks without requiring host-range observations; operators must keep host settings stable while a cluster runs. Validator readiness continues to require both slot production and an acknowledged subscription.
Validation
fd576232, Sysiob537bb50, Ethereuma76e7a0b, and Solana2c7ba391. Resolved revisions, build artifacts and sibling SDK consumption were verified. This is not exact-head validation of the merge-resolution candidate.737f4fe8, Sysioe34ccf9a, Ethereuma76e7a0b, Solana2c7ba391and Libraries254badb6were verified, including sibling SDK consumption and complete flow coverage.