Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions docs/decisions/0049-webmin-per-role-and-the-mesh-vip.md
Original file line number Diff line number Diff line change
Expand Up @@ -488,6 +488,30 @@ machine it runs on. "Who moves it" and "How it is carried out" above
read `keel vip promote` where they say `keel database promote` for the
VIP.

## Precision, 2026-10-11: the rejoin leaves the spec alone

Approved by the maintainer on 2026-10-11, with the design of the MariaDB
pair in keel (Keel-Linux/keel#93). "The old primary coming back",
MariaDB, step 3, says the node "writes `role: replica` and the new
primary's address into its own spec". That is read as the **emitted**
spec, what `keel inspect` writes out: the rejoin does not rewrite
`/etc/keel/instance.yaml`. The role in the spec is the one chosen at
installation, and from the first claim of the VIP on it is runtime
state, the VIP's (0020, "One role"): `inspect` reports the role the VIP
gives the node, `diff` reports the difference from the declared role as
information and never as drift to repair, and `apply` never converges
it. Who writes the operator's file is Keel-Linux/keel#69's question.

Three more points approved the same day for the same design, each as
recommended: the mesh root CA is made by `keel mesh create` in every
cloud mode, and the root issues a `database` leaf per node (a CN that is
no etcd user, its SANs the node's /128 and the pair's VIP, `REQUIRE X509`
on the replication account), so there is no database CA per pair; both
nodes of a pair bind the VIP through `net.ipv6.ip_nonlocal_bind`, so no
promotion or demotion restarts MariaDB; and the dumps on the standby
(the maintainer's interim backup of 2026-10-03) are the pull request
after the replication one.

## Reconciled with 0029, 0041 and keel

- **0029, "Resolved"**: "node-to-node and management traffic use real
Expand Down
2 changes: 1 addition & 1 deletion docs/decisions/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ that changes an earlier one says so under "What this amends".
| [0043](0043-release-formats.md) | Release formats: only the ISO and the `.tar.zst` (Docker through `docker import`), AWS and OpenStack by local conversion later, names, layout, signed SHA512SUMS, GitHub Releases | decided 2026-10-01 |
| [0047](0047-one-image-per-application.md) | One image per application: the installation mode decides what runs; WordPress on Core, Web and PHP with an embedded MariaDB; Keel Cloud in no application image | decided 2026-10-02 |
| [0048](0048-joining-the-mesh-with-one-command.md) | Joining the mesh with one command: `keel mesh invite` prints `keel mesh join keel1:<token>`, valid one hour and once; an HMAC authenticated HTTPS port with a pinned certificate, a fallback command; both sides under 0018's window; etcd forms at the third node, with its own TLS | decided 2026-10-03, third round 2026-10-04 (etcd: intermediate CAs, state not spec, `keel mesh etcd form`, removal rule); amendment (admission evidence) approved 2026-10-04 |
| [0049](0049-webmin-per-role-and-the-mesh-vip.md) | Webmin per mode and role, and the VIP on the WireGuard mesh: the primary's Webmin only on the VIP, a replica's (database or files) on its mesh and private LAN addresses with a warning; one VIP per replicated appliance pair, a `/128` the primary carries on `wg0` and peers route by `allowed_ips`, moved by promote before etcd and by the controller with it, fenced by an epoch; an old primary rejoins by itself when it holds nothing the new one lacks; root loses `READ_ONLY ADMIN` on a MariaDB replica; third round: the code in keel, `keel vip promote`, only the pair claims (a signed pair record), a 20 s etcd lease | decided 2026-10-03, third round 2026-10-04 |
| [0049](0049-webmin-per-role-and-the-mesh-vip.md) | Webmin per mode and role, and the VIP on the WireGuard mesh: the primary's Webmin only on the VIP, a replica's (database or files) on its mesh and private LAN addresses with a warning; one VIP per replicated appliance pair, a `/128` the primary carries on `wg0` and peers route by `allowed_ips`, moved by promote before etcd and by the controller with it, fenced by an epoch; an old primary rejoins by itself when it holds nothing the new one lacks; root loses `READ_ONLY ADMIN` on a MariaDB replica; third round: the code in keel, `keel vip promote`, only the pair claims (a signed pair record), a 20 s etcd lease; precision of 2026-10-11: the rejoin leaves the spec alone, the role is runtime state, the root CA in every cloud mode signs a `database` leaf per node, `ip_nonlocal_bind` binds the VIP on both nodes | decided 2026-10-03, third round 2026-10-04, precision 2026-10-11 |
| [0050](0050-the-keel-poor-networks-and-file-replication.md) | The keel: everything that spans nodes is designed and tested for 250 ms with jitter and loss, under netem profiles; the hot path stays local and only changes replicate; application files on Syncthing v2, one writer per application, the folder flipped by keel's role, built into Keel Web; Garage a later regional step; JuiceFS and GlusterFS rejected; evidence in [docs/evidence/2026-10-03-storage](../evidence/2026-10-03-storage/) | decided 2026-10-04 |
| [0051](0051-regions-and-federated-root-cas.md) | Regions and federated root CAs: a root CA per region on its holder, certifying its nodes locally, another region's root when it is down; etcd one cluster trusting a bundle of every root, the bundle changed only by a majority of the roots and approved on each holder; each root revokes what it issued, a whole root by a majority that counts it but needs not its signature (two regions need a third, or `distrust` on each member); CRLs merged per member; `installation.region` and a /112 per region; at most five voters, placed so losing any region keeps a quorum; the roots' status in confconsole, `keel mesh status` and `keel diff`, alerted by email; keel#78 the single region case with the bundle and region field, keel#79 the rest | decided 2026-10-04, two rounds (tracker#60) |
| [0052](0052-every-release-ships-both-formats.md) | Every release ships both formats: each appliance release stages the `.tar.zst` and the ISO from one build and is published only when both pass; the container drops the boot set (kernel, initrd, firmware, GRUB, installer, about 192 MB of Core's 903.6 MB unpacked), which moves to a per appliance boot layer only the ISO uses; both scanned, signed and listed, the Proxmox index the `.tar.zst` only; the ISO boot tested under BIOS and UEFI in QEMU on the build host as a gate; amends 0043 | decided 2026-10-10, with the five points of the proposal approved as recommended: the boot layer, UEFI as a gate, the install to disk once automated, `.tar.zst` only testing pre-releases until the ISO builder exists, the 990 pin first |
Expand Down
Loading