feat: every overlay peer gets PersistentKeepalive = 25 (keel#140)

debian/changelog

@@ -1,3 +1,27 @@
+keel (0.23.41) unstable; urgency=medium
+
+  * Every overlay peer gets PersistentKeepalive = 25 (keel#140). The
+    keepalive keeps the NAT and firewall state open between sites, so
+    a node behind a filter that drops unsolicited inbound UDP stays
+    reachable when the mesh is idle. The release test of LNMP 19.0-3
+    found it: the root holder could not send the cluster to the member
+    on BR2 after 290 s with no traffic. keel spec apply sets it live;
+    keel diff takes 25 as the default of a peer that gives none.
+
+ -- Marcos Méndez <[email protected]>  Fri, 16 Oct 2026 16:28:02 +0000
+
+keel (0.23.40) unstable; urgency=medium
+
+  * keel mesh etcd form sends the cluster again to a member of the
+    first cluster that never ran (keel#199). etcd lists such a member
+    with its name and no client URL; keel took the name as proof that
+    it ran, so the member stayed out, and etcd ran with 2 of 3 voters.
+  * The invite's listener and the members' channel free the slot of a
+    connection that failed before they close it (keel#177). A client
+    that asked again at once found its slot taken.
+
+ -- Marcos Méndez <[email protected]>  Fri, 16 Oct 2026 16:36:14 +0000
+
 keel (0.23.39) unstable; urgency=medium
 
   * A MariaDB holder whose lease is in doubt acknowledges no write

docs/apply.md

@@ -1439,7 +1439,11 @@ goes through the same window, marker, lock, timers and boot unit, with
   restart, and a kept file whose interface has another MTU gets the same
   action, said as drift (keel#139). The package's postinst runs `keel
   network mtu`, which does the same for each WireGuard interface that is
-  up, so an upgrade needs no reboot. A change
+  up, so an upgrade needs no reboot. Every peer gets
+  `PersistentKeepalive = 25` when the spec gives none (keel#140), set
+  live with `wg set`; a change that only sets a peer's keepalive cannot
+  cut the node off either, and keel mesh keeps it with no handshake, as
+  an addition. A change
   that only added peers live is marked so in the window's marker, and
   keel mesh keeps it with no handshake (mesh.md). A change is dated on the clock a session's
   start time is read on (the start of a thread, in seconds since the

docs/diff.md

@@ -225,6 +225,7 @@ listed after every other section:
 | `derived.monit.included` | `monitor.enabled` | whether `/etc/monit/conf.d/keel-manifest.conf` is keel's link to that file |
 | `derived.firewall` | with `firewall.enabled: true`, the ruleset apply renders; with `false`, nothing | `/etc/keel/firewall/keel-manifest.nft`, compared whole; with `false`, keel's file still there is drift |
 | `network.overlay.wireguard.peers.<key>.handshake` | a handshake within 180 s | on the live system, `wg show <interface> latest-handshakes`: a peer with none since the interface came up is drift, and so is a peer the interface does not hold, and one whose last handshake is older than 180 s (keel#140: WireGuard does a new handshake every 2 minutes while packets flow, so the member does not answer and is unreachable); a recent one is `same`, with its age (keel#117). Not compared under `--root` |
+| `network.overlay.wireguard.peers.<key>.persistent_keepalive` | the spec's, else 25, which keel renders for every peer | the peer's `PersistentKeepalive` in the file; a file written before keel rendered it is drift until apply writes it again |
 | `network.overlay.wireguard.mtu` | the MTU the interface's file sets | on the live system, the MTU `ip link show` gives for the interface (keel#139): another one is drift, which `keel network mtu` or `keel spec apply --system` sets live with no restart. Not compared under `--root`, nor for a file with no `MTU` line |
 | `derived.firewall.loaded` | the digest of that ruleset | on the live system, the digest in the comment of the `inet keel` table the kernel holds (`nft list table inet keel`), so a table flushed or edited by hand is drift; not compared under `--root` |
 

docs/spec.md

@@ -298,7 +298,7 @@ network:
 | `...peers[].public_key` | system | Required. The peer's key as `wg pubkey` prints it (44 characters of base64). Each key once, and never this node's own: with secret files checked, the public key of this node's private key file, when it exists, is refused as a peer's |
 | `...peers[].endpoint` | system | Optional: where to reach the peer, `host:port`. An IPv6 literal goes in brackets, `[2001:db8:2::20]:51820`, as wg writes it; a name or an IPv4 address without. Without it this node waits for the peer to reach it |
 | `...peers[].allowed_ips` | system | Required, at least one prefix: what is routed to this peer and accepted from it. For a node, its overlay address as a `/128` (and `/32`). A prefix with host bits set is an error, and so is a prefix given to two peers, since wg would silently keep it for the last one only. Each prefix must be inside the private ranges the overlay addresses use (`fc00::/7`; RFC 1918 or `100.64.0.0/10`): a `/0`, a public prefix (a `::/1` and `8000::/1` pair takes as much as `::/0`) and a prefix that overlaps the uplink or a peer's endpoint are refused (see "Routes" below). A prefix inside the overlay's VIP range is refused too: keel routes a VIP to its holder at runtime, never in the spec |
-| `...peers[].persistent_keepalive` | system | Optional, seconds between 1 and 65535: keeps a path through a NAT or a stateful firewall open. Leave it out for none |
+| `...peers[].persistent_keepalive` | system | Optional, seconds between 1 and 65535: keeps a path through a NAT or a stateful firewall open. Left out, keel writes 25 for every peer (keel#140): the state stays open between sites, and a last handshake older than 180 s means the member is unreachable. A file without it is drift in keel diff |
 
 **Joining with one command.** The peers and the address need not be
 typed: `keel mesh invite`, on a node already in the mesh, prints a `keel

keel/__init__.py

@@ -12,7 +12,7 @@
 # an image's tree read offline): debian/changelog's and pyproject.toml's,
 # which tests/test_version.py keeps in step. Installed, the metadata
 # dpkg put beside the package answers (keel#98).
-FALLBACK_VERSION = "0.23.39"
+FALLBACK_VERSION = "0.23.41"
 
 
 def version() -> str:

keel/diff/compare.py

@@ -390,15 +390,20 @@ def with_slaac_default(network: dict) -> dict:
 
 
 def with_overlay_defaults(network: dict) -> dict:
-    """The overlay's interface and port as apply renders them when the
-    spec leaves them out, so a machine carrying them is not drift. A copy.
+    """The overlay's interface and port, and each peer's keepalive, as
+    apply renders them when the spec leaves them out, so a machine
+    carrying them is not drift, and one without the keepalive is. A copy.
     """
     overlay = network.get("overlay")
     wg = overlay.get("wireguard") if isinstance(overlay, dict) else None
     if not isinstance(wg, dict):
         return network
     wg = {"interface": wireguard.DEFAULT_INTERFACE,
           "listen_port": wireguard.DEFAULT_PORT, **wg}
+    if isinstance(wg.get("peers"), list):
+        wg["peers"] = [
+            {"persistent_keepalive": wireguard.DEFAULT_KEEPALIVE, **peer}
+            if isinstance(peer, dict) else peer for peer in wg["peers"]]
     return {**network, "overlay": {**overlay, "wireguard": wg}}
 
 

keel/mesh/etcdclient.py

@@ -74,8 +74,11 @@ class Tls:
 
 @dataclass(frozen=True)
 class Member:
-    """One member as etcd lists it; `started` once it ran and named
-    itself, which a learner added for a join that never came has not"""
+    """One member as etcd lists it; `started` once it ran: it named
+    itself and published its client URLs. A learner added for a join
+    that never came has no name; a member of the first cluster that
+    never ran has the name of ETCD_INITIAL_CLUSTER and no client URL
+    (keel#199)"""
 
     id: str
     name: str
@@ -85,7 +88,7 @@ class Member:
 
     @property
     def started(self) -> bool:
-        return bool(self.name)
+        return bool(self.name) and bool(self.client_urls)
 
     @property
     def address(self) -> str | None:

keel/mesh/inviting.py

@@ -53,11 +53,11 @@
 from keel.mesh.signing import SigningError
 from keel.network import live, wgkeys, wireguard
 from keel.system import DEFAULT_WINDOW
+from keel.network.wireguard import DEFAULT_KEEPALIVE as KEEPALIVE
 
 UNIT = "keel-mesh-invite@{id}"
 # a join accepted just before the expiry is confirmed within its window
 LINGER = DEFAULT_WINDOW + 60
-KEEPALIVE = 25
 EVERY_ADDRESS = "::"
 
 

keel/mesh/listener.py

@@ -335,8 +335,11 @@ def serve_one(listener: Listener, context, raw: socket.socket, source: str,
         pass
     finally:
         read.cancel()
-        held[-1].close()
+        # free first: a client that sees the close may ask again at once,
+        # and it must find its slot free (keel#177), also when the
+        # handshake failed and the handler never ran
         release()
+        held[-1].close()
 
 
 def run(server: socket.socket, listener: Listener, context,

keel/mesh/memberlink.py

@@ -261,8 +261,10 @@ def serve_one(front: Front, raw: socket.socket, source: str,
         pass
     finally:
         read.cancel()
-        raw.close()
+        # free first: a source that sees the close may ask again at once
+        # and must find its slot free (keel#177)
         release()
+        raw.close()
 
 
 def bound(iface: str, address: str, port: int) -> socket.socket:

keel/network/switch.py

@@ -196,6 +196,13 @@ def bounce(root: str, iface: str, relative: str, text: str,
 CHANGED = "changed"
 
 
+def paths(one: dict | None) -> dict | None:
+    """A peer as far as the paths to it go: without its keepalive"""
+    if one is None:
+        return None
+    return {k: v for k, v in one.items() if k != "persistent_keepalive"}
+
+
 def live_peers(old: str, new: str, iface: str, dump: str,
                live_mtu: int | None = None) -> list[tuple[str, ...]] | None:
     """The `wg set` lines of `live_plan`, or None"""
@@ -237,8 +244,10 @@ def live_plan(old: str, new: str, iface: str, dump: str,
     `wg set` does not.
 
     With the lines, whether they only add peers the interface does not
-    hold (keel#117): such a change removes nothing and changes no peer
-    this node reaches, so it cannot cut this node off.
+    hold (keel#117), or set the keepalive of a peer it holds (the
+    keepalive every peer gets, keel#140): such a change removes nothing
+    and changes no path to a peer this node reaches, so it cannot cut
+    this node off.
     """
     before, after = wireguard.parse(old), wireguard.parse(new)
     if before.problems or after.problems or before.inline_key or \
@@ -264,7 +273,9 @@ def live_plan(old: str, new: str, iface: str, dump: str,
                 for name in ("address", "ipv4_address")
                 if before.section.get(name)]
     for key in set(was) | set(now):
-        if was.get(key) != now.get(key) and not all(
+        # a change of the keepalive alone changes no path: the upgrade
+        # writes it for every peer, an off-link one too (keel#140)
+        if paths(was.get(key)) != paths(now.get(key)) and not all(
                 on_link(one, networks) for one in (was.get(key),
                                                    now.get(key)) if one):
             return None
@@ -280,10 +291,17 @@ def live_plan(old: str, new: str, iface: str, dump: str,
         found = peer_fix(iface, now[key], held.get(key), old_ips)
         if found:
             commands.append(found)
-            added = added and key not in held
+            # a keepalive set on a peer it holds changes nothing it
+            # reaches, as an addition does not
+            added = added and (key not in held or keepalive_only(found))
     return commands, added and bool(commands)
 
 
+def keepalive_only(argv: tuple[str, ...]) -> bool:
+    """A `wg set` line that sets one peer's keepalive and nothing else"""
+    return len(argv) == 7 and argv[5] == "persistent-keepalive"
+
+
 def mtu_line(before: int | None, after: int | None, now: int | None,
              iface: str) -> tuple[str, ...] | None | bool:
     """The `ip link set` line that gives `iface` the new file's MTU, None

keel/network/wireguard.py

@@ -60,6 +60,11 @@
 # the minimum MTU of IPv6 (RFC 8200), with the IPv6, UDP and WireGuard
 # headers (80 bytes) under every path of 1360 bytes or more (keel#119)
 OVERLAY_MTU = 1280
+# every peer's keepalive when the spec gives none (the coordinator's
+# decision of 2026-10-10 on keel#140): a packet every 25 s keeps the
+# NAT and firewall state open between sites, and the handshake renewed,
+# so a last handshake older than 180 s means the member is unreachable
+DEFAULT_KEEPALIVE = 25
 PEER_KEYS = ("publickey", "endpoint", "allowedips", "persistentkeepalive")
 ULA_PREFIX = 0xFD
 GLOBAL_ID_BYTES = 5
@@ -187,12 +192,15 @@ def render(overlay: dict) -> str:
         if peer.get("endpoint"):
             lines.append(f"Endpoint = {peer['endpoint']}")
         lines.append(f"AllowedIPs = {', '.join(allowed(peer))}")
-        if peer.get("persistent_keepalive"):
-            lines.append(
-                f"PersistentKeepalive = {peer['persistent_keepalive']}")
+        lines.append(f"PersistentKeepalive = {keepalive(peer)}")
     return "\n".join(lines) + "\n"
 
 
+def keepalive(peer: dict) -> int:
+    """The peer's keepalive: the spec's, else DEFAULT_KEEPALIVE"""
+    return int(peer.get("persistent_keepalive") or DEFAULT_KEEPALIVE)
+
+
 @dataclass(frozen=True)
 class Parsed:
     """A wg-quick file read back into the spec's shape

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "keel"
-version = "0.23.39"
+version = "0.23.41"
 authors = [{name = "KeelLinux maintainers"}]
 description = "Declarative management of one appliance instance"
 readme = "README.md"

tests/test_inspect_overlay.py

@@ -15,6 +15,8 @@
 
 from helpers import spec  # noqa: F401
 
+from test_network_wireguard import with_keepalive
+
 from keel.diff import DRIFT, NOT_COMPARED, NOT_DECLARED, SAME, compare
 from keel.inspect import inspect_root
 from keel.inspect.collect import overlay_section
@@ -61,7 +63,8 @@ def test_the_rendered_file_reads_back(self):
         section, findings = probe_overlay(
             [conf(wireguard.render(OVERLAY))], public, NO_MODULE_INFO)
         self.assertEqual(section, {"wireguard": {
-            "interface": "wg0", **OVERLAY, "listen_port": 51820,
+            "interface": "wg0", **with_keepalive(OVERLAY),
+            "listen_port": 51820,
             "private_key": {"file": "/etc/wireguard/wg0.key"}}})
         found = by_field(findings)
         self.assertEqual(found["network.overlay.wireguard.public_key"].value,
@@ -174,7 +177,7 @@ def test_inspect_puts_the_overlay_in_the_network_section(self):
                         return_value=(PUBLIC, None)):
             inspection = inspect_root(self.root)
         self.assertEqual(inspection.spec["network"]["overlay"]["wireguard"]
-                         ["peers"], OVERLAY["peers"])
+                         ["peers"], with_keepalive(OVERLAY)["peers"])
 
 
 def declared(**wg):
@@ -206,6 +209,19 @@ def test_the_rendered_file_is_the_same_as_its_spec(self):
             found["network.overlay.wireguard.private_key.file"].status,
             NOT_DECLARED)
 
+    def test_a_file_without_the_keepalive_is_drift(self):
+        """the keepalive keel renders for every peer (keel#140): a file
+        written before it is drift until apply writes it"""
+        key = f"network.overlay.wireguard.peers.{OTHER_KEY}"
+        self.assertEqual(self.diff(declared())[
+            f"{key}.persistent_keepalive"].status, SAME)
+        self.put("etc/wireguard/wg0.conf", wireguard.render(OVERLAY).replace(
+            "AllowedIPs = fd00:1::3/128\nPersistentKeepalive = 25\n",
+            "AllowedIPs = fd00:1::3/128\n"))
+        found = self.diff(declared())[f"{key}.persistent_keepalive"]
+        self.assertEqual((found.status, found.declared, found.observed),
+                         (DRIFT, 25, None))
+
     def test_peers_compare_by_key_whatever_their_order(self):
         doc = declared(peers=[
             {"public_key": OTHER_KEY, "allowed_ips": ["fd00:1::3/128"]},

tests/test_mesh_etcd.py

@@ -363,8 +363,10 @@ def test_nothing_before_a_cluster(self):
     def test_learners_promoted_or_removed_and_leaves_renewed(self):
         etcdstate.save_cluster(self.a.root, Cluster("new", (
             Member(KEYS[0], address(0)),), MESH.hex()))
-        started = EtcdMember("5", "keel-x", ("https://[fd00::5]:2380",), (),
-                             True)
+        # a learner that started named itself and published its client
+        # URLs (keel#199)
+        started = EtcdMember("5", "keel-x", ("https://[fd00::5]:2380",),
+                             ("https://[fd00::5]:2379",), True)
         never = EtcdMember("6", "", ("https://[fd00::6]:2380",), (), True)
         self.fake.members_ = [voter(0), started, never]
         self.assertEqual(etcdcare.tend(self.a), exits.OK)

tests/test_mesh_etcd_form.py

@@ -3,6 +3,7 @@
 a mesh adopted under keel 0.18, with no CA, in one process"""
 
 import json
+from dataclasses import replace
 import os
 import unittest
 from datetime import timedelta
@@ -70,6 +71,14 @@ def test_run_again_on_a_member_of_the_cluster(self):
         client.members_ = [voter(0), voter(1), voter(2, name=False)]
         out = []
         self.assertEqual(etcdform.form(a, True, out.append), exits.OK)
+        self.assertEqual(out[0], "would send the first cluster again to"
+                         f" {address(2)}")
+        # keel#199: a member of the first cluster that never ran is
+        # listed with its name, and no client URL
+        named = replace(voter(2), client_urls=())
+        client.members_ = [voter(0), voter(1), named]
+        out = []
+        self.assertEqual(etcdform.form(a, True, out.append), exits.OK)
         self.assertEqual(out[0], "would send the first cluster again to"
                          f" {address(2)}")
         self.assertEqual(etcdform.form(a, False, out.append), exits.OK)

tests/test_mesh_etcdclient.py

@@ -165,6 +165,17 @@ def test_the_list_learners_and_unstarted(self):
         named = etcdclient.Member("1", "", ("https://host:2380",), (), True)
         self.assertIsNone(named.address)
 
+    def test_a_member_of_the_first_cluster_that_never_ran_is_unstarted(self):
+        """keel#199: etcd lists each member of ETCD_INITIAL_CLUSTER with
+        its name, also one that never ran; such a member has published
+        no client URL (seen on real nodes, 2026-10-16)"""
+        self.ctl.answer("member list", {"members": [
+            {"ID": 1, "name": "keel-fd00--3",
+             "peerURLs": ["https://[fd00::3]:2380"]}]})
+        (found,) = self.client.members()
+        self.assertEqual(found.name, "keel-fd00--3")
+        self.assertFalse(found.started)
+
     def test_add_a_learner_promote_remove(self):
         self.ctl.answer("member add", {"member": {
             "ID": 16334501404808820146,