From 590e56ddb14546631d6db8b1cd728a2251288cad Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Marcos=20M=C3=A9ndez?= Date: Thu, 8 Oct 2026 15:01:03 +0000 Subject: [PATCH] docs: 0049, precision of 2026-10-11: the rejoin leaves the spec alone The role is runtime state from the first claim on (0020); what 0049 says the rejoining old primary writes into its own spec is the emitted spec. With it, the three points approved the same day for keel#93: the root CA in every cloud mode and a database leaf per node, ip_nonlocal_bind on both nodes of a pair, and the standby's dumps as the next pull request. --- .../0049-webmin-per-role-and-the-mesh-vip.md | 24 +++++++++++++++++++ docs/decisions/README.md | 2 +- 2 files changed, 25 insertions(+), 1 deletion(-) diff --git a/docs/decisions/0049-webmin-per-role-and-the-mesh-vip.md b/docs/decisions/0049-webmin-per-role-and-the-mesh-vip.md index adadea3..6c56091 100644 --- a/docs/decisions/0049-webmin-per-role-and-the-mesh-vip.md +++ b/docs/decisions/0049-webmin-per-role-and-the-mesh-vip.md @@ -488,6 +488,30 @@ machine it runs on. "Who moves it" and "How it is carried out" above read `keel vip promote` where they say `keel database promote` for the VIP. +## Precision, 2026-10-11: the rejoin leaves the spec alone + +Approved by the maintainer on 2026-10-11, with the design of the MariaDB +pair in keel (Keel-Linux/keel#93). "The old primary coming back", +MariaDB, step 3, says the node "writes `role: replica` and the new +primary's address into its own spec". That is read as the **emitted** +spec, what `keel inspect` writes out: the rejoin does not rewrite +`/etc/keel/instance.yaml`. The role in the spec is the one chosen at +installation, and from the first claim of the VIP on it is runtime +state, the VIP's (0020, "One role"): `inspect` reports the role the VIP +gives the node, `diff` reports the difference from the declared role as +information and never as drift to repair, and `apply` never converges +it. Who writes the operator's file is Keel-Linux/keel#69's question. + +Three more points approved the same day for the same design, each as +recommended: the mesh root CA is made by `keel mesh create` in every +cloud mode, and the root issues a `database` leaf per node (a CN that is +no etcd user, its SANs the node's /128 and the pair's VIP, `REQUIRE X509` +on the replication account), so there is no database CA per pair; both +nodes of a pair bind the VIP through `net.ipv6.ip_nonlocal_bind`, so no +promotion or demotion restarts MariaDB; and the dumps on the standby +(the maintainer's interim backup of 2026-10-03) are the pull request +after the replication one. + ## Reconciled with 0029, 0041 and keel - **0029, "Resolved"**: "node-to-node and management traffic use real diff --git a/docs/decisions/README.md b/docs/decisions/README.md index 49ca429..e668ab4 100644 --- a/docs/decisions/README.md +++ b/docs/decisions/README.md @@ -36,7 +36,7 @@ that changes an earlier one says so under "What this amends". | [0043](0043-release-formats.md) | Release formats: only the ISO and the `.tar.zst` (Docker through `docker import`), AWS and OpenStack by local conversion later, names, layout, signed SHA512SUMS, GitHub Releases | decided 2026-10-01 | | [0047](0047-one-image-per-application.md) | One image per application: the installation mode decides what runs; WordPress on Core, Web and PHP with an embedded MariaDB; Keel Cloud in no application image | decided 2026-10-02 | | [0048](0048-joining-the-mesh-with-one-command.md) | Joining the mesh with one command: `keel mesh invite` prints `keel mesh join keel1:`, valid one hour and once; an HMAC authenticated HTTPS port with a pinned certificate, a fallback command; both sides under 0018's window; etcd forms at the third node, with its own TLS | decided 2026-10-03, third round 2026-10-04 (etcd: intermediate CAs, state not spec, `keel mesh etcd form`, removal rule); amendment (admission evidence) approved 2026-10-04 | -| [0049](0049-webmin-per-role-and-the-mesh-vip.md) | Webmin per mode and role, and the VIP on the WireGuard mesh: the primary's Webmin only on the VIP, a replica's (database or files) on its mesh and private LAN addresses with a warning; one VIP per replicated appliance pair, a `/128` the primary carries on `wg0` and peers route by `allowed_ips`, moved by promote before etcd and by the controller with it, fenced by an epoch; an old primary rejoins by itself when it holds nothing the new one lacks; root loses `READ_ONLY ADMIN` on a MariaDB replica; third round: the code in keel, `keel vip promote`, only the pair claims (a signed pair record), a 20 s etcd lease | decided 2026-10-03, third round 2026-10-04 | +| [0049](0049-webmin-per-role-and-the-mesh-vip.md) | Webmin per mode and role, and the VIP on the WireGuard mesh: the primary's Webmin only on the VIP, a replica's (database or files) on its mesh and private LAN addresses with a warning; one VIP per replicated appliance pair, a `/128` the primary carries on `wg0` and peers route by `allowed_ips`, moved by promote before etcd and by the controller with it, fenced by an epoch; an old primary rejoins by itself when it holds nothing the new one lacks; root loses `READ_ONLY ADMIN` on a MariaDB replica; third round: the code in keel, `keel vip promote`, only the pair claims (a signed pair record), a 20 s etcd lease; precision of 2026-10-11: the rejoin leaves the spec alone, the role is runtime state, the root CA in every cloud mode signs a `database` leaf per node, `ip_nonlocal_bind` binds the VIP on both nodes | decided 2026-10-03, third round 2026-10-04, precision 2026-10-11 | | [0050](0050-the-keel-poor-networks-and-file-replication.md) | The keel: everything that spans nodes is designed and tested for 250 ms with jitter and loss, under netem profiles; the hot path stays local and only changes replicate; application files on Syncthing v2, one writer per application, the folder flipped by keel's role, built into Keel Web; Garage a later regional step; JuiceFS and GlusterFS rejected; evidence in [docs/evidence/2026-10-03-storage](../evidence/2026-10-03-storage/) | decided 2026-10-04 | | [0051](0051-regions-and-federated-root-cas.md) | Regions and federated root CAs: a root CA per region on its holder, certifying its nodes locally, another region's root when it is down; etcd one cluster trusting a bundle of every root, the bundle changed only by a majority of the roots and approved on each holder; each root revokes what it issued, a whole root by a majority that counts it but needs not its signature (two regions need a third, or `distrust` on each member); CRLs merged per member; `installation.region` and a /112 per region; at most five voters, placed so losing any region keeps a quorum; the roots' status in confconsole, `keel mesh status` and `keel diff`, alerted by email; keel#78 the single region case with the bundle and region field, keel#79 the rest | decided 2026-10-04, two rounds (tracker#60) | | [0052](0052-every-release-ships-both-formats.md) | Every release ships both formats: each appliance release stages the `.tar.zst` and the ISO from one build and is published only when both pass; the container drops the boot set (kernel, initrd, firmware, GRUB, installer, about 192 MB of Core's 903.6 MB unpacked), which moves to a per appliance boot layer only the ISO uses; both scanned, signed and listed, the Proxmox index the `.tar.zst` only; the ISO boot tested under BIOS and UEFI in QEMU on the build host as a gate; amends 0043 | decided 2026-10-10, with the five points of the proposal approved as recommended: the boot layer, UEFI as a gate, the install to disk once automated, `.tar.zst` only testing pre-releases until the ISO builder exists, the 990 pin first |