Redesign NUClearNet and rename to nuclearnet

[TrentHouliston]

Summary

Replaces the monolithic NUClearNetwork implementation in src/extension/network/ with a redesigned, modular src/nuclearnet/ library. The new implementation is built as a standalone library that can be used independently of the reactor framework.

Improvements over the old NUClearNetwork

• Modular architecture — The old implementation was a single ~1100-line class handling discovery, fragmentation, reliability, and routing all in one place. The new version separates each concern into its own class with clear interfaces.
• Standalone library — nuclearnet can now be built and linked independently of the NUClear reactor framework, enabling reuse in other projects.
• Selective ACK/NACK — The old implementation used simple timeout-based retransmission. The new version uses bitset-based selective acknowledgement so only missing fragments are retransmitted, not entire packets.
• Adaptive retransmission — Per-peer RTT estimation (Jacobson/Karels algorithm) replaces fixed timeouts, adapting to actual network conditions.
• Packet deduplication — Sliding-window deduplication prevents processing the same packet twice, which the old version did not handle.
• Scatter-gather IO — Fragment transmission uses vectored writes to avoid copying payload data into intermediate buffers.
• Testability — Each component has dedicated unit tests. The old implementation had no unit-level test coverage of its networking internals.

Architecture

The new NUClearNet is decomposed into focused components:

• Discovery — Multicast-based peer announce/leave with two-flag connection model
• Fragmentation — Splits large payloads into MTU-sized fragments and reassembles them with timeout-based cleanup
• Reliability — Selective ACK/NACK retransmission using bitset-based tracking
• RTTEstimator — Per-peer round-trip time estimation for adaptive retransmission timeouts
• Routing — Subscription-based message filtering per peer, with receiver-side filtering for multicast data
• PacketDeduplicator — Sliding-window deduplication to discard already-seen packets
• wire_protocol — Defines the on-wire packet format (Announce, Leave, Data, ACK, Connect)

NUClearNet ties these together to provide the public API (join, leave, send, process).

Connection establishment

Peers must satisfy two independent conditions before a connection is considered "up":

1. Announce path confirmed (announce_heard) — received the peer's announce on the multicast/broadcast channel, proving their data port can reach our announce address.
2. Data handshake confirmed (handshake == CONFIRMED) — a 3-way SYN/SYN+ACK/ACK handshake over the data ports proves bidirectional unicast connectivity.

The packet type encodes which path was used — ANNOUNCE packets always go to the multicast group, CONNECT packets always go to the peer's data port. This confirms all four communication paths without needing socket tracking.

When a new peer's announce is heard:

• We immediately re-announce to the multicast group (so they hear us)
• We send CONNECT(SYN) to their data port to begin the data handshake
• Connection fires only when both flags are satisfied (either order)

Multicast broadcast delivery

Unreliable broadcast sends (empty target, non-reliable) are sent once to the multicast/broadcast group rather than unicasting to each peer individually. Receivers filter by local subscription before fragmentation reassembly.

Reliable sends and targeted sends remain unicast for per-peer ACK tracking.

What Changed

• Removed: src/extension/network/NUClearNetwork.{cpp,hpp} and src/extension/network/wire_protocol.hpp (old monolithic implementation)
• Added: src/nuclearnet/ — all new source and headers with its own CMakeLists.txt
• Added: src/util/network/sock_t.hpp — cross-platform socket type alias
• Modified: src/extension/NetworkController.{cpp,hpp} — updated to use the new nuclearnet library API
• Added: tests/tests/nuclearnet/ — Catch2 BDD-style unit tests for each component (Discovery, Fragmentation, Integration, PacketDeduplicator, RTTEstimator, Reliability, Routing, wire_protocol)

Build

• Targets C++14 (no C++17 features used)
• nuclearnet library links against the platform socket library and can be consumed standalone via CMake
• Full test suite passes locally (68/68) and across Docker CI matrix (GCC 7–13, Clang 18)

Logging

NUClearNet has its own log level and log sink in src/nuclearnet/Log.hpp, defaulting to Off and writing to stderr. Because the library is usable without the reactor framework, the sink is a settable callback rather than a hard dependency on NUClear's logging system:

using LogHandler = std::function<void(LogLevel level, const char* component, const std::string& message)>;
NUClearNet::set_log_handler(handler);  // an empty handler restores the stderr default

NetworkController installs a handler that re-emits the messages as NUClear LogMessages, so the usual log handlers pick them up instead of the output bypassing them on stderr. NUClearNet.js installs one that hands them to its JavaScript logger.

message::NetworkConfiguration gains a log_level, which sets both the library level (deciding what it hands to the handler) and the NetworkController reactor level (deciding what it emits), so one setting controls the whole path. UNKNOWN leaves the logs off, which is the existing behaviour.

Note that NetworkController dispatches on the level with an explicit switch rather than calling Reactor::log(level, args...). That runtime-level overload loses overload resolution to the compile time one (whose Arguments... pack absorbs the LogLevel), so it logs everything at DEBUG with the level name stringified into the message. Worth fixing separately — it affects every caller of that overload, not just this one.

[Copilot]

Pull request overview

This PR replaces the legacy monolithic NUClearNetwork implementation under src/extension/network/ with a new modular, standalone src/nuclearnet/ library, and updates the reactor-facing NetworkController to use the new API. It also adds a comprehensive Catch2 unit/integration test suite for the new components and improves multicast capability detection for test gating.

Changes:

• Introduces the new src/nuclearnet/ standalone library (Discovery, Fragmentation, Reliability, RTT estimation, routing, deduplication, wire protocol, RAII FD wrapper).
• Migrates src/extension/NetworkController from NUClearNetwork to the new NUClearNet API, including subscription propagation.
• Adds/updates tests and test utilities (including multicast round-trip detection) to cover the new networking components.

Reviewed changes

Copilot reviewed 33 out of 33 changed files in this pull request and generated 11 comments.

Show a summary per file

File	Description
src/nuclearnet/CMakeLists.txt	Adds standalone CMake build for the new nuclearnet library.
src/nuclearnet/wire_protocol.hpp	Defines new on-wire structs and header validation helper.
src/nuclearnet/RTTEstimator.hpp	Declares RTT estimator API.
src/nuclearnet/RTTEstimator.cpp	Implements Jacobson/Karels-style RTT estimation.
src/nuclearnet/Routing.hpp	Declares peer/local subscription tracking and filtering API.
src/nuclearnet/Routing.cpp	Implements subscription-based routing decisions.
src/nuclearnet/Reliability.hpp	Declares ACK/NACK tracking and retransmission API.
src/nuclearnet/Reliability.cpp	Implements retransmission tracking and ACK/NACK processing.
src/nuclearnet/PacketDeduplicator.hpp	Declares sliding-window packet deduplication.
src/nuclearnet/PacketDeduplicator.cpp	Implements wraparound-safe sliding-window deduplication.
src/nuclearnet/Fragmentation.hpp	Declares fragmentation and reassembly API.
src/nuclearnet/Fragmentation.cpp	Implements MTU fragmentation, reassembly, and expiry cleanup.
src/nuclearnet/FileDescriptor.hpp	Adds RAII wrapper for sockets/file descriptors.
src/nuclearnet/Discovery.hpp	Declares announce/leave processing and peer tracking API.
src/nuclearnet/Discovery.cpp	Implements peer discovery, timeout handling, and callbacks.
src/nuclearnet/NUClearNet.hpp	Declares the public standalone networking façade and callbacks.
src/nuclearnet/NUClearNet.cpp	Implements socket setup, polling loop, packet IO, and module integration.
src/util/network/sock_t.hpp	Extends sock_t with comparison and stream operators for use in maps/logging.
src/extension/NetworkController.hpp	Switches controller to use network::NUClearNet.
src/extension/NetworkController.cpp	Adapts controller wiring to the new API and subscriptions model.
src/CMakeLists.txt	Uses CONFIGURE_DEPENDS for recursive globbing.
tests/test_util/has_multicast.cpp	Adds multicast round-trip probing for more reliable test gating.
tests/tests/nuclearnet/Discovery.cpp	Adds unit tests for discovery behaviors.
tests/tests/nuclearnet/Fragmentation.cpp	Adds unit tests for fragmentation/reassembly behaviors.
tests/tests/nuclearnet/Integration.cpp	Adds integration tests for two peers discovering/exchanging data.
tests/tests/nuclearnet/PacketDeduplicator.cpp	Adds unit tests for deduplication window/wraparound.
tests/tests/nuclearnet/Reliability.cpp	Adds unit tests for ACK/NACK and retransmission behavior.
tests/tests/nuclearnet/Routing.cpp	Adds unit tests for routing/subscription filtering.
tests/tests/nuclearnet/RTTEstimator.cpp	Adds unit tests for RTT estimator behavior.
tests/tests/nuclearnet/wire_protocol.cpp	Adds tests for packed sizes/layout and header validation.
src/extension/network/NUClearNetwork.hpp	Removes legacy network header.
src/extension/network/NUClearNetwork.cpp	Removes legacy network implementation.
src/extension/network/wire_protocol.hpp	Removes legacy wire protocol header.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 38 out of 38 changed files in this pull request and generated 7 comments.

[Copilot]

Pull request overview

Copilot reviewed 45 out of 45 changed files in this pull request and generated 5 comments.

Suppressed comments (7)

src/nuclearnet/NUClearNet.cpp:476

• get_peers() includes peers that have only announced or are still handshaking. Sending and tracking reliable data for those entries makes the receiver discard it as unconnected and starts retransmission before a connection exists. Filter this list with is_connected() as the API documentation requires.

            for (const auto& peer : peers) {
                const auto& addr = peer.first;
                if (routing.should_send(addr, hash)) {
                    targets.push_back(addr);

src/nuclearnet/NUClearNet.cpp:486

• Named sends also select peers that are known but not fully connected. This can send DATA before the handshake and, for reliable messages, leave retransmission state for packets the peer was required to reject. Require is_connected(addr) here too.

            for (const auto& peer : peers) {
                const auto& addr = peer.first;
                const auto& info = peer.second;
                if (info.name == target && routing.should_send(addr, hash)) {
                    targets.push_back(addr);

src/nuclearnet/NUClearNet.cpp:733

• A DATA datagram's fixed wire header is sizeof(DataPacket) - 1; the final byte in the struct is only a payload placeholder. send() emits exactly that 20-byte header for an empty payload, but this 21-byte check rejects it, so empty messages can never be delivered despite fragmentation explicitly supporting them.

        if (length < sizeof(DataPacket)) {

docs/how-to/networking.md:57

• These are not defaults of the documented NetworkConfiguration message: its address is empty and its port is 0, and NetworkController copies both values over the engine defaults. A default-constructed message therefore does not use this multicast endpoint. Mark these fields as required here, or change the public message defaults to match.

| `name`             | `string`   | —                   | Unique name for this node on the network        |
| `announce_address` | `string`   | `"239.226.152.162"` | Address for node discovery announcements        |
| `announce_port`    | `uint16_t` | `7447`              | Port for announce messages                      |

src/nuclearnet/wire_protocol.hpp:118

• The stated DATA wire size is inconsistent with the offsets below it: fields through hash occupy bytes 0–19, so the fixed header is 20 bytes before payload, not 18. Correct this so external protocol implementations do not use the wrong payload offset.

     * Wire layout (18+ bytes):

src/nuclearnet/Reliability.cpp:93

• Every partial or duplicate ACK before a retransmission records another RTT sample for the same transmission. A fragmented packet therefore contributes many progressively later samples, biasing the estimator toward transfer duration rather than round-trip time and inflating its RTO. Track whether this packet has already supplied an RTT sample and measure only the first valid ACK.

        if (tp.retransmit_count == 0) {
            auto rtt = now - tp.last_send;
            {
                const std::lock_guard<std::mutex> rtt_lock(rtt_mutex);
                rtt_estimators[source].measure(rtt);

tests/tests/dsl/UDP.cpp:378

• This skips the entire UDP suite on any Windows environment that defines CI, not only the unreliable GitHub loopback cases. Consequently even unicast and broadcast behavior receives no Windows CI coverage, allowing platform regressions to pass. Restrict the condition to the affected runner/cases rather than returning from the whole test.