Skip to content

feat(bindings): a Python gateway-daemon client behind the reticulum slot - #490

Merged
bahdotsh merged 6 commits into
mainfrom
feat/headless-python-gateway-client
Sep 30, 2026
Merged

bahdotsh merged 6 commits into
mainfrom
feat/headless-python-gateway-client

Conversation

@bahdotsh

@bahdotsh bahdotsh commented Sep 30, 2026 •

Copy link
Copy Markdown
Member

Summary

Python gets a client for the gateway-daemon contract, behind the slot the FFI names reticulum. ProtocolManager used to install a stub for that slot; it now builds GatewayManager as pm.gateway when reticulum_enabled=True, and the callback forwards the core's wake to it.

  • gateway_manager.py. TCP to a daemon on local IP over asyncio, newline-delimited JSON, a 1 MiB line cap. It attaches in the contract's order (Identify, Challenge, DeclareAddress, AddressDeclared, Capabilities, StatusUpdate), gets the declaration from the core (gateway_address_declaration), and announces the carrier from one place, on StatusUpdate(connected), only for a session the gateway echoed back under this device's own address. Every submitted frame is held until the gateway's verdict names it; the write is never the outcome. The silent-gateway sweep at 60 s stays under the core's 120 s pending-confirmation expiry, presence is asked about in one frame per tick from the core's watchlist, the reconnect ladder resets only in the attach completion, every close path retires the session and fails what it carried, and a session generation is checked after every suspension.
  • gateway_attach_policy.py, gateway_verdict_tracker.py, presence_watch_policy.py. Pure ports of the Swift files of the same names: no socket, no core, so they are unit-tested. The policy also holds the mapping from the wire's two optional verdict flags to the core's tokens, which is what makes this the first client on this carrier to read stored and pushed.
  • Rust guards. The three guards that read the Swift and Kotlin gateway sources now read the Python ones too: the attach constants and the verdict-timeout relationship against the transport crate's constant, the presence-watch defaults, and the ban on the signing domain in any manager or policy (negative-controlled: planting the domain in gateway_manager.py fails react_native_reticulum_attaches_before_it_reports_available naming the file).
  • Docs. P11 in docs/bridges/python.md; C5's seventh and eighth sets and the Python row of "What each binding owes" in docs/bridges/README.md; the Reticulum row in the Python README; a CHANGELOG entry.

What a verdict tells the core

Wire Reported to the core Recipient watched and fed as offline
MessageSent reticulum_confirm_sent no
MessageSent { pushed } relay_pushed yes
MessageSent { pushed, stored } relay_pushed_stored yes
DeliveryError { stored } relay_stored yes
DeliveryError { reason: "recipient_unreachable…" } the reason verbatim yes
any other DeliveryError the reason verbatim no
no verdict within 60 s gateway_silent: no verdict within 60s no
connection lost, or stop() Connection lost, Disconnected no

Both flags are read only when they are the JSON true; absent or anything else is false, never unknown. stored on a DeliveryError is applied to the one id the verdict names, as the relay client does. The core normalises relay_stored back onto the unreachable path with the probe suppressed and treats relay_pushed* as a park, so nothing typed (a connection request, a Welcome) changes behaviour on this carrier relative to the relay.

What closes the connection

An echo of an address we do not hold, an AddressError, a challenge that is not 32 bytes, a core that cannot sign, a StatusUpdate(connected) before the bind, ten seconds without it (bound or not), an over-long line, three consecutive write failures, EOF. Each retires the session (fails in flight, stops the watch, disarms the attach timeout) and climbs the ladder; the ladder resets only on a bound and announced session, never on the TCP open.

A malformed frame costs the frame, never the carrier

The first review found that one line from the gateway wedged the carrier for good: a handler that raised ended the receive task with no close path, so the session stayed bound, the poll loop kept submitting, every frame failed at 60 s as gateway_silent, and nothing addressed to this device arrived again until stop(). json.loads accepts a lone surrogate escape ("\ud800"), and encoding the string it yields raised outside every try in the delivery, echo and capability handlers and inside the FFI lowering of a verdict's reason or recipient. Reproduced against the fake daemon; fixed in the second commit:

  • Every remote string goes through utf8_bytes, which answers None for a value that cannot be encoded, and each parser treats that as the field's absence: an unencodable capability token is dropped, an unencodable presence peer or delivery sender is an invalid frame, an unencodable verdict recipient is no recipient (the id still settles, nobody is watched), and an unencodable verdict reason is reported as the bare DeliveryError, which the core reads as a retry. The session survives every one of these, and the three core calls in the verdict handler are additionally wrapped for UnicodeEncodeError only.
  • The receive loop has the net the relay client has. Any other exception from a handler emits Gateway frame handler raised, closes the session (failing every id in flight, so the core hears an outcome for each rather than waiting out its expiry) and lets the ladder replace it. A core that raises on a verdict report now costs the connection, not the transport.
  • Tests: the surrogate escape in every field a handler reads, followed by a submission whose verdict still reaches the core; a surrogate reason and recipient on a DeliveryError that still settles the id; a raising reticulum_confirm_sent that produces a close, status:False and a fresh attach; and the diagnostic on the net. Mutants for the net, the helper, the delivery encode and the reason sanitiser each fail a test.

Two hardening items from the same review: the Rust guard reads the Python constants as line-anchored module assignments (NAME = value at column zero, exactly once) rather than flattened text with docstrings kept, and parses VERDICT_TIMEOUT and the capability bounds out of the file for the comparisons against the core's constants, dropping the test-local spelling; and the signing-domain ban reads all six files raw, comments included, negative-controlled once per language (a planted comment in ReticulumManager.swift, in ReticulumManager.kt, and in gateway_manager.py each fails the guard naming that file).

Validation

  • Rust: cargo fmt --all -- --check, cargo clippy --workspace -- -D warnings, and the four guards that read the Python sources pass in the worktree. The domain ban was negative-controlled in each of the three languages.
  • Python: the four new test files (158 tests: policy, tracker, watch policy, and the manager against a fake daemon on a real loopback socket) pass under pytest --count 40 on 3.12, 3.13 and 3.14 with no flake. The whole suite passes once on all three. 3.10 is not installed here; CI runs it.
  • Mutation-checked: twenty-three mutants (announce unbound, confirm on write, reset the ladder on open, never reset it, settle duplicates, announce on degraded, no in-flight cap, resend an in-flight id, no sweep, keep in flight on retire, bind on a mismatch, stored and pushed mapping, count tokens before the size filter, watch on a plain error, unchecked challenge size, tracker begin and expiry, no net under a raising handler, a raising UTF-8 helper, the delivery encode without the helper, an unsanitised verdict reason, and two more). Twenty-two fail a test. The one survivor is equivalent: the pause gate in on_messages_available is checked again inside the drain, so removing the early return only costs a task.

Not in this PR

  • docs/reticulum.md still describes the mobile managers only; the sibling PR that restates the backbone owns that file.
  • The Kotlin and Swift managers do not read stored or pushed. Their policy files parse neither; this PR ports the shape and leaves the two bridges as they are.
  • A daemon_address in ProtocolConfig. The Python manager is configured in code, as the peer-stream manager is.

Notes for reviewers

  • No daemon has been run against this. The fake daemon in the tests speaks the contract as written; a real one is a deployment this repository does not ship.
  • The manager calls the core inline on the event loop, as the relay and peer-stream managers do. The Swift manager's "check the generation after every queue hop" becomes "after every await" here: after the open, after every write, per line in the receive loop, and in every timer callback.
  • GatewayManager.transport_id is "reticulum" because that is the slot's FFI name (H6 keeps those); the class and module are named for what they are.

`GatewayManager` speaks the gateway-daemon contract over TCP from Python:
it attaches with the core-built address declaration, offers the carrier to
the core only once the gateway has bound the session to this device's
address, settles every send on the gateway's verdict rather than on the
socket write, fails what a dead or silent connection owes, and asks about
the core's presence watchlist. It is the first client on this carrier to
read the `stored` and `pushed` verdict flags, reported under the relay
client's tokens.

The decisions and frame shapes are ports of the Swift policy files, with
no socket and no core, so they are unit-tested; the manager is driven
against a fake daemon on a loopback socket. `ProtocolManager` builds it as
`pm.gateway` when `reticulum_enabled` and stops it with the rest; the stub
callback for the slot forwards the core's wake to it.

The Rust guards that read the Swift and Kotlin gateway policies now read
the Python ones too: the constants and the verdict-timeout relationship,
the presence-watch defaults, and the ban on the signing domain in any
client. P11 in docs/bridges/python.md records what the client promises.
…arrier

A handler that raised ended the Python gateway client's receive task with
no close path: the session stayed bound, the poll loop kept submitting,
every frame failed at the verdict timeout, and nothing addressed to this
device arrived again until stop(). One line from the gateway could cause
it: json.loads accepts a lone surrogate escape, and encoding the string
it yields raised outside every try, in the delivery, echo, capability and
verdict handlers and inside the FFI lowering.

Every remote string now passes through one helper that answers None for
an unencodable value, and the parsers treat that as the field's absence:
the frame is skipped with a diagnostic and the session survives. The
receive loop gains the net the relay client has, so a handler that raises
for any other reason closes the session and the ladder replaces it, which
also fails every id in flight so the core hears an outcome for each.

The Rust guard reads the Python constants as line-anchored module
assignments rather than flattened text, and parses the verdict timeout out
of the file for the relationship against the core's expiry. The signing
domain ban reads all three languages raw, comments included, and was
negative-controlled once per language. docs/reticulum.md names the Python
client beside the two mobile ones.
# Conflicts:
#	bindings/python/README.md
@bahdotsh
bahdotsh merged commit 8f07ff4 into main Sep 30, 2026
22 checks passed
@github-actions github-actions Bot locked and limited conversation to collaborators Sep 30, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant