From 4b9373eb9deb21e801abad5aaf756d585d8e683f Mon Sep 17 00:00:00 2001 From: Your Name Date: Thu, 20 Aug 2026 00:57:21 +0000 Subject: [PATCH] feat(secrets): make verifying an age key backup a command MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The private key at ~/.config/sops/age/keys.txt has one copy. If that disk dies, what is lost is not really the secrets — a password can be reset and a community string can be changed — but every device visit needed to do it: Grafana, the Alertmanager webhook, and four SNMP communities re-entered by hand on pfSense, the APC NMC, iLO and the MokerLink switch. The switch is the one that hurts, because its firmware will not persist a deletion from the community table and drops its SNMP agent on each attempt, which is why SECURITY.md still records an accepted residual there from the last rotation. Doing that again, unplanned, is the scenario this is against. The backup itself is manual and stays out of the repository. What is added is the part that is easy to get wrong. The obvious verification does not verify anything. On the host that already holds the key, `SOPS_AGE_KEY_FILE= sops -d secrets/observability.sops.yaml` passes whatever is in the file you name: sops also consults SOPS_AGE_KEY and the default ~/.config/sops/age/keys.txt. This was measured, not assumed — pointed at a freshly generated, entirely unrelated keypair with SOPS_AGE_KEY set, it exits 0 and prints every secret. A backup that has never been verified and one that has been verified wrongly are the same backup; only one of them feels safe. So scripts/verify-key-backup.sh clears SOPS_AGE_KEY, redirects HOME and XDG_CONFIG_HOME at an empty directory so a fallback finds nothing rather than silently succeeding, and refuses to run against the live key at all. It compares device and inode rather than paths, because the first version compared with `stat -c` and no -L, which reports the symlink instead of its target — a symlink straight back to the live key passed as a good backup, and the test that caught it is the reason the guard is written this way. Two further guards, both from watching how this fails rather than how it works: a decrypt failure alone cannot distinguish "wrong keypair" from "not an age file" and those have different fixes, so the recipient is checked against .sops.yaml first; and the exit status is not trusted alone — the plaintext is asserted to contain all six keys render-config.sh requires, the same reasoning bootstrap.sh already applies to its own encrypt. No secret value reaches stdout, a temp file, or another process's argv; the here-string that would have been the obvious way to search the plaintext is avoided because bash may back one with a temp file. The target sits under Maintenance next to snmp-verify rather than Validation. Everything under Validation is offline and safe for CI; this needs a private key on disk and must never end up inside `make validate`. Deliberately not done: no CI check, since CI holds no key and a check that always skips is noise. No roadmap entry — #11 says explicitly it is filed separately. No second age recipient, which is the stronger arrangement but re-keys the committed secrets file and is worth doing when the lab stops being a one-person project. And nothing here records where the backup lives: a public pointer to the location of the only key is worth less than the reminder would be. The three checkboxes on #11 are still the operator's to tick — this makes the middle one provable rather than performing it. Refs #11 --- Makefile | 12 ++ README.md | 4 +- docs/runbooks/back-up-the-age-key.md | 167 +++++++++++++++++++ docs/security.md | 6 + scripts/bootstrap.sh | 9 + scripts/verify-key-backup.sh | 237 +++++++++++++++++++++++++++ secrets/README.md | 9 +- 7 files changed, 441 insertions(+), 3 deletions(-) create mode 100644 docs/runbooks/back-up-the-age-key.md create mode 100755 scripts/verify-key-backup.sh diff --git a/Makefile b/Makefile index f3834a2..195d09f 100644 --- a/Makefile +++ b/Makefile @@ -219,6 +219,18 @@ snmp-verify: ## Check each SNMP device answers to its community (ARGS=--old) @# it into `make validate`. ./scripts/snmp-verify.sh $(ARGS) +.PHONY: secrets-verify-backup +secrets-verify-backup: ## Check a backup age key decrypts the secrets (KEY=/path/to/keys.txt) + @# Under Maintenance rather than Validation for the same reason as + @# snmp-verify: everything under Validation is offline and safe for CI, + @# and this needs a private key on disk. It must never end up inside + @# `make validate`, where it would either always skip or ask CI for a key. + @# + @# KEY rather than ARGS because there is exactly one argument and it is + @# required — an empty ARGS would reach the script as no argument at all + @# and print usage, which reads like the target is broken. + ./scripts/verify-key-backup.sh "$(KEY)" $(STACK) + .PHONY: certs certs: ## Create the internal CA / issue a leaf (ARGS="--host x.matrix.elysium --ip 10.0.0.1") @# certificates/ is gitignored. Nothing in the stack terminates TLS yet — diff --git a/README.md b/README.md index bec58d9..fcbdbcb 100644 --- a/README.md +++ b/README.md @@ -58,7 +58,7 @@ incident. `sha256:` digest, so a moved tag cannot change what deploys. CI enforces it; `make pin-digests` re-resolves them from the registry. - **Documented decisions and runbooks.** Five ADRs covering what was chosen and - what was rejected; five runbooks for the operations that are easy to get wrong + what was rejected; six runbooks for the operations that are easy to get wrong at 1am. ## Architecture @@ -137,7 +137,7 @@ the internet and nothing more. Full topology and data flow in │ ├── architecture.md network.md hardware.md │ ├── observability.md security.md roadmap.md │ ├── adr/ # 5 architecture decision records -│ └── runbooks/ # deploy, add device, rotate creds, certs, purge history +│ └── runbooks/ # deploy, add device, rotate creds, certs, key backup, purge └── Makefile # make help ``` diff --git a/docs/runbooks/back-up-the-age-key.md b/docs/runbooks/back-up-the-age-key.md new file mode 100644 index 0000000..f120369 --- /dev/null +++ b/docs/runbooks/back-up-the-age-key.md @@ -0,0 +1,167 @@ +# Runbook: Back up the age private key + +**Target:** `~/.config/sops/age/keys.txt` on the monitoring host (10.0.99.20) +**Time:** five minutes, once +**You will need:** shell access to the monitoring host, and somewhere durable to +put a copy — a password manager, or paper and a place to keep it + +This is the highest-value five minutes in the repository, and the only one whose +downside is unbounded. + +## Why + +That file is the sole copy of the private key that decrypts +[`secrets/observability.sops.yaml`](../../secrets/observability.sops.yaml). The +public half in [`.sops.yaml`](../../.sops.yaml) is committed and can only +encrypt; nothing in this repository, and nothing in any backup of this +repository, can recover the private half. + +What is lost with that disk is not really the secrets — a password can be reset +and a community string can be changed. What is lost is every device visit needed +to do it: the Grafana admin password, the Alertmanager webhook, and four SNMP +communities re-entered by hand on pfSense, the APC NMC, HPE iLO and the +MokerLink switch. The switch is the one that hurts. Its firmware will not +persist a deletion from the community table and drops its SNMP agent on each +attempt, which is why [`SECURITY.md`](../../SECURITY.md) still records an +accepted residual there from the last rotation. Doing that a second time, +unplanned, because a 2012 MacBook Pro's disk failed, is the scenario this +avoids. + +Background on the scheme itself is in +[`secrets/README.md`](../../secrets/README.md). + +## 1. Copy the key + +```bash +cat ~/.config/sops/age/keys.txt +``` + +Two lines of plain ASCII: a `# public key: age1…` comment and one +`AGE-SECRET-KEY-1…` line. The secret line alone is sufficient — the comment is a +convenience, and `age-keygen -y` can regenerate it from the secret half at any +time. + +Put it somewhere that is not this machine: + +| Where | Good for | Watch out for | +| --- | --- | --- | +| Password manager, as a secure note | The default choice — encrypted, backed up, searchable | Losing the master password loses this too. Title it so you can find it in five years | +| Printed, in a safe or with documents | Survives every digital failure at once | A printer with a spool, and a photocopier's memory | +| Encrypted USB stick, stored elsewhere | Fast restore | Flash cells fade unpowered; re-verify yearly | + +Record *what it is and what it unlocks* alongside it. A bare +`AGE-SECRET-KEY-1…` string found in a password manager in three years is +indistinguishable from junk. + +The location itself is deliberately **not** written down in this repository. A +public pointer to where the only key lives is worth less than the reminder it +would provide. + +## 2. Verify it actually decrypts + +Not optional, and not the same as looking at it. From a checkout of this repo: + +```bash +make secrets-verify-backup KEY=/path/to/the/copy +``` + +Expect: + +```console +-- recipient matches .sops.yaml: age1yrdu996… +-- decrypting observability.sops.yaml with the backup key only +ok — keys.txt decrypts secrets/observability.sops.yaml (6/6 keys) +``` + +What it proves: the key parses, its public half is the recipient the file was +encrypted to, it decrypts the real ciphertext, and the result contains all six +keys `render-config.sh` requires. No secret value is printed, written to a +temporary file, or passed to another process. + +What it deliberately does not prove: that where you put the copy will still +exist after a fire, a theft, or a forgotten master password. That part is +judgement, not a check. + +> **Why not just run `sops -d` by hand.** Because on the host that already holds +> the key, it passes no matter what. `SOPS_AGE_KEY_FILE= sops -d …` also +> consults `SOPS_AGE_KEY` and the default `~/.config/sops/age/keys.txt`; pointed +> at a freshly generated, completely unrelated keypair it exits 0 and prints +> every secret. [`verify-key-backup.sh`](../../scripts/verify-key-backup.sh) +> clears `SOPS_AGE_KEY`, redirects `HOME` and `XDG_CONFIG_HOME` at an empty +> directory, and refuses to run against the live key file at all — so a pass +> means the backup did the work. + +### Verifying a paper backup + +Type it **back in** and verify that file. Verifying the copy you pasted from is +verifying your clipboard; verifying the copy you typed is verifying your +handwriting, which is the thing that will actually fail. + +```bash +umask 077 +vi /dev/shm/restore-test.txt # transcribe from the paper +make secrets-verify-backup KEY=/dev/shm/restore-test.txt +shred -u /dev/shm/restore-test.txt +``` + +age keys transcribe better than most secrets: uppercase Bech32, so the data part +contains no lowercase, and no `1`, `b`, `i` or `o` at all. The `0`/`O` and +`1`/`l` confusions that ruin handwritten passwords cannot occur here. A +mis-transcription is caught by the checksum in the key itself, and the script +reports it as "not a readable age identity file" rather than a decrypt failure. + +## 3. Confirm the copy is not republishing itself + +The third thing that goes wrong: a backup that is durable and also public. + +- **Not in a git repository.** `.gitignore` here matches `keys.txt` and + `age.key`, which protects nothing outside this tree and nothing under a + different filename. +- **Not in a synced folder.** Dropbox, OneDrive, Google Drive, iCloud, + Nextcloud, Syncthing — a copy there is a copy on someone else's disk. + `make secrets-verify-backup` warns when the path looks like one, but it can + only see the path, not what the folder actually syncs to. +- **Not on the monitoring host.** A second copy on the same disk is not a + backup. The script refuses to verify the live key for this reason. + +## Restoring on a new host + +```bash +mkdir -p ~/.config/sops/age +vi ~/.config/sops/age/keys.txt # paste the backup +chmod 600 ~/.config/sops/age/keys.txt +git clone https://github.com/Gerrrt/HomeLab.git && cd HomeLab +make render # decrypts; this is the proof it worked +make up +``` + +**Do not run `make secrets-init` to restore.** It generates a *new* keypair +whose public half does not match `.sops.yaml`, and then nothing decrypts. It +refuses to overwrite an existing key or an existing encrypted secrets file, so +the damage is recoverable — but the error it produces afterwards +(`sops metadata not found`, or a failed decrypt) sends you looking in the wrong +place. Restore is a file copy, nothing more. + +## What this still does not solve + +One key, one person, one copy plus the original. If the answer to "who else can +recover this" needs to be more than one, the mechanism already exists: generate a +second keypair that lives only offline, add its public half to `.sops.yaml` as an +additional recipient, and re-key with +`sops updatekeys secrets/observability.sops.yaml` from a host that can already +decrypt. That is the same procedure as bringing a second host in, described in +[`secrets/README.md`](../../secrets/README.md). It is worth doing on the day the +lab stops being a one-person project, and not before — every extra recipient is +another key that can leak. + +## If something goes wrong + +| Symptom | Cause | Fix | +| --- | --- | --- | +| `that is the live key on this host, not a backup of it` | `KEY=` points at `~/.config/sops/age/keys.txt`, or a symlink or hard link to it | Point it at the copy. This is the check working | +| `this is a valid age key, but not one the secrets are encrypted to` | A different keypair — usually one `age-keygen` made by accident | Find the right backup. If there is none, the secrets must be rotated on the devices | +| `is not a readable age identity file` | Truncated, wrapped, or mis-transcribed secret line | Re-copy. Check the whole `AGE-SECRET-KEY-1…` line landed on one line | +| `decryption failed with this key` after the recipient matched | Right public half, damaged secret half — a partial copy | Re-copy from the original while the host still exists | +| `the backup is inside this repository` | The copy was written into the working tree | Move it out, then `git log --all -- ` to confirm it was never committed | +| `no encrypted secrets at secrets/observability.sops.yaml` | Wrong working directory, or a fresh clone that never ran `make secrets-init` | Run from the repo root | +| `mode 644 — group or other can read this copy` | A copy written without `umask 077` | `chmod 600` it, and consider it seen by anything else on that machine | diff --git a/docs/security.md b/docs/security.md index c3e22d4..8242497 100644 --- a/docs/security.md +++ b/docs/security.md @@ -45,6 +45,12 @@ assumption consistent with what they are. and committed in encrypted form. See [`secrets/README.md`](../secrets/README.md). - The private key lives at `~/.config/sops/age/keys.txt` on the deployment host and is never in the repository. +- That key is the single point of failure for every encrypted secret here, so it + is copied off the host and the copy is proven to decrypt with + `make secrets-verify-backup KEY=` — which refuses to run against the live + key and blanks the environment first, because the obvious hand-typed + equivalent passes even for an unrelated keypair. See + [`runbooks/back-up-the-age-key.md`](runbooks/back-up-the-age-key.md). - `scripts/render-config.sh` decrypts at deploy time into gitignored files. Nothing writes a plaintext secret into a tracked path. - CI runs `gitleaks` with rules specifically for SNMP communities, inline diff --git a/scripts/bootstrap.sh b/scripts/bootstrap.sh index 0c576bb..a91c812 100755 --- a/scripts/bootstrap.sh +++ b/scripts/bootstrap.sh @@ -40,6 +40,8 @@ else chmod 600 "${KEY_FILE}" warn "BACK THIS FILE UP OFF THIS MACHINE. Without it the encrypted secrets" warn "in this repository are unrecoverable." + warn " docs/runbooks/back-up-the-age-key.md" + warn " make secrets-verify-backup KEY= # proves the copy decrypts" fi PUBLIC_KEY="$(grep -oE 'age1[a-z0-9]+' "${KEY_FILE}" | head -n1)" @@ -163,4 +165,11 @@ Next steps: 1. make secrets-edit # replace every change-me value 2. make validate # confirm the configs are sound 3. make up # render config and start the stack + +And the one that has no second chance — back up ${KEY_FILE} off this +machine, then prove the copy works: + + make secrets-verify-backup KEY=/path/to/the/copy + + docs/runbooks/back-up-the-age-key.md EOF diff --git a/scripts/verify-key-backup.sh b/scripts/verify-key-backup.sh new file mode 100755 index 0000000..fb61bd2 --- /dev/null +++ b/scripts/verify-key-backup.sh @@ -0,0 +1,237 @@ +#!/usr/bin/env bash +# +# Prove that a backup copy of the age private key can decrypt this stack's SOPS +# file — and that it decrypted because of *that* file rather than because the +# live key on this host was quietly picked up instead. +# +# The second half is the whole point. `SOPS_AGE_KEY_FILE= sops -d ...` +# typed by hand looks like a verification and is not one: sops consults +# SOPS_AGE_KEY and the default ~/.config/sops/age/keys.txt as well, so on the +# host that already holds the real key it prints the secrets whatever is in the +# file you named. Measured, not theorised — pointing that command at a freshly +# generated unrelated keypair with SOPS_AGE_KEY set exits 0 and prints every +# secret. A backup that has never been verified and a backup that has been +# verified wrongly are the same backup; only one of them feels safe. +# +# Nothing here writes a secret to disk, to a temp file, or to the terminal. +# +# Usage: scripts/verify-key-backup.sh [stack] +# make secrets-verify-backup KEY=/path/to/backup/keys.txt +# +# See docs/runbooks/back-up-the-age-key.md. + +set -euo pipefail + +REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +BACKUP="${1:-}" +STACK="${2:-observability}" +SECRETS_FILE="${REPO_ROOT}/secrets/${STACK}.sops.yaml" +SOPS_CONFIG="${REPO_ROOT}/.sops.yaml" + +# Where the *live* key is. Read before the decrypt below scrubs the environment, +# and only ever used to refuse to verify the original by mistake. +LIVE_KEY="${SOPS_AGE_KEY_FILE:-${HOME}/.config/sops/age/keys.txt}" + +die() { printf '\033[0;31merror:\033[0m %s\n' "$*" >&2; exit 1; } +info() { printf '\033[0;34m--\033[0m %s\n' "$*"; } +warn() { printf '\033[0;33m!!\033[0m %s\n' "$*"; } + +if [[ -z "${BACKUP}" ]]; then + cat >&2 < [stack] + or: make secrets-verify-backup KEY=/path/to/backup/keys.txt + +Checks that the age key in decrypts secrets/${STACK}.sops.yaml. +Procedure: docs/runbooks/back-up-the-age-key.md +EOF + exit 2 +fi + +for tool in sops age-keygen; do + command -v "${tool}" >/dev/null 2>&1 || die "${tool} not found. + age: https://github.com/FiloSottile/age/releases + sops: https://github.com/getsops/sops/releases" +done + +[[ -f "${SECRETS_FILE}" ]] || die "no encrypted secrets at ${SECRETS_FILE} +There is nothing to verify a key against. Run 'make secrets-init' first." +[[ -f "${BACKUP}" ]] || die "no such file: ${BACKUP}" +[[ -r "${BACKUP}" ]] || die "${BACKUP} is not readable by $(id -un)" + +# Directory symlinks are resolved with `pwd -P`; the leaf is left alone so the +# stat comparison below is what catches a symlink pointing back at the live key. +BACKUP_ABS="$(cd "$(dirname "${BACKUP}")" && pwd -P)/$(basename "${BACKUP}")" + +# --------------------------------------------------------------------------- +# 1. It has to be a different file from the live key +# --------------------------------------------------------------------------- +# Verifying the original proves nothing at all, and it is an easy mistake: the +# obvious way to test the command before pointing it at the real backup is to +# point it at the key you already have. Device + inode rather than path, so a +# symlink or a hard link back to the original is caught too. +# -L dereferences: without it GNU stat reports the symlink's own inode, and a +# symlink pointing straight back at the live key sails through as a "backup". +file_id() { stat -Lc '%d:%i' "$1" 2>/dev/null || stat -Lf '%d:%i' "$1" 2>/dev/null; } + +if [[ -e "${LIVE_KEY}" ]]; then + backup_id="$(file_id "${BACKUP}" || true)" + live_id="$(file_id "${LIVE_KEY}" || true)" + live_abs="$(cd "$(dirname "${LIVE_KEY}")" && pwd -P)/$(basename "${LIVE_KEY}")" + + if { [[ -n "${backup_id}" ]] && [[ "${backup_id}" == "${live_id}" ]]; } \ + || [[ "${BACKUP_ABS}" == "${live_abs}" ]]; then + die "that is the live key on this host, not a backup of it: + ${BACKUP_ABS} + +A copy that lives on the same disk as the original is not a backup, and +verifying the original proves only that the original works. Point this at the +copy — a mounted USB stick, or a file you pasted back out of your password +manager. See docs/runbooks/back-up-the-age-key.md." + fi +fi + +# --------------------------------------------------------------------------- +# 2. It must not be somewhere that republishes it +# --------------------------------------------------------------------------- +# Issue #11's third checkbox. The repo case is a hard failure because this tree +# is a public repository and .gitignore's `keys.txt` rule is a filename match, +# not a guarantee — a copy named anything else is committable. +if [[ "${BACKUP_ABS}" == "${REPO_ROOT}/"* ]]; then + die "the backup is inside this repository: + ${BACKUP_ABS} + +This tree is published. Move it out before verifying it, and check that it was +never committed: git log --all --oneline -- $(printf '%q' "${BACKUP_ABS#"${REPO_ROOT}/"}")" +fi + +# Advice, not a verdict: the script cannot know what a directory syncs to, and +# for some people a synced vault *is* the durable copy. Saying so out loud is +# the useful part. +for pattern in Dropbox OneDrive 'Google Drive' Nextcloud ownCloud Syncthing iCloud 'Mobile Documents'; do + shopt -s nocasematch + if [[ "${BACKUP_ABS}" == *"${pattern}"* ]]; then + warn "the path contains '${pattern}' — if that folder syncs to a third party," + warn "the private key is now wherever that service keeps it." + fi + shopt -u nocasematch +done + +# --------------------------------------------------------------------------- +# 3. Is it even the right key? (offline, no ciphertext involved) +# --------------------------------------------------------------------------- +# A decrypt failure alone cannot distinguish "this is not the key the file was +# encrypted to" from "this file is not an age identity at all", and those have +# completely different fixes. +if ! PUBLIC_KEYS="$(age-keygen -y "${BACKUP}" 2>&1)"; then + die "${BACKUP_ABS} +is not a readable age identity file. + +age-keygen said: ${PUBLIC_KEYS} + +An age key file holds a line beginning AGE-SECRET-KEY-1. If you transcribed +this from paper, check for a truncated or wrapped line." +fi + +matched="" +while IFS= read -r pub; do + [[ -n "${pub}" ]] || continue + if grep -qF "${pub}" "${SOPS_CONFIG}"; then + matched="${pub}" + break + fi +done <<< "${PUBLIC_KEYS}" + +if [[ -z "${matched}" ]]; then + die "this is a valid age key, but not one the secrets are encrypted to. + + backup holds: $(printf '%s' "${PUBLIC_KEYS}" | tr '\n' ' ') + .sops.yaml wants: $(grep -oE 'age1[a-z0-9]+' "${SOPS_CONFIG}" | tr '\n' ' ') + +A freshly generated keypair looks exactly like this. If you meant to add a new +recipient rather than restore an old one, that is 'sops updatekeys' — see +secrets/README.md." +fi +info "recipient matches .sops.yaml: ${matched}" + +# --------------------------------------------------------------------------- +# 4. The real test: decrypt with nothing but the backup available +# --------------------------------------------------------------------------- +# Every element of the env line below is load-bearing: +# +# -u SOPS_AGE_KEY takes precedence over SOPS_AGE_KEY_FILE, so an +# inherited value would make any file "pass" +# HOME / XDG_CONFIG_HOME pointed at an empty directory, so that if sops falls +# back to the default ~/.config/sops/age/keys.txt it +# finds nothing and fails — rather than decrypting with +# the live key and reporting the backup good +# +# Remove any one of them and this script becomes the hand-typed command it +# exists to replace. +TMP_HOME="$(mktemp -d)" +trap 'rm -rf "${TMP_HOME}"' EXIT INT TERM + +info "decrypting $(basename "${SECRETS_FILE}") with the backup key only" + +# Plaintext is captured into a shell variable and never redirected anywhere. A +# here-string is deliberately not used to search it further down: bash may +# implement one as a temp file, which is exactly what this is avoiding. +if ! plaintext="$(env -u SOPS_AGE_KEY -u XDG_CONFIG_HOME \ + HOME="${TMP_HOME}" SOPS_AGE_KEY_FILE="${BACKUP}" \ + sops --decrypt "${SECRETS_FILE}" 2>"${TMP_HOME}/sops.err")"; then + die "decryption failed with this key. + +sops said: +$(sed 's/^/ /' "${TMP_HOME}/sops.err") + +The public half matched .sops.yaml, so the secret half is damaged or truncated — +the usual cause is a partial copy or a mis-transcribed line." +fi + +# --------------------------------------------------------------------------- +# 5. Do not trust the exit status alone +# --------------------------------------------------------------------------- +# Same reasoning as bootstrap.sh: a zero exit and an empty or partial result are +# indistinguishable from the caller's side, and this is the one check standing +# between a lost key and a lost network. +REQUIRED=( + GRAFANA_ADMIN_PASSWORD + ALERTMANAGER_WEBHOOK_URL + SNMP_COMMUNITY_PFSENSE + SNMP_COMMUNITY_APC + SNMP_COMMUNITY_MOKERLINK + SNMP_COMMUNITY_ILO +) +found=0 +missing=() +for var in "${REQUIRED[@]}"; do + # Pure bash matching: no grep, no process substitution, nothing that could put + # a decrypted value into another process's argv or into a file. + if [[ $'\n'"${plaintext}" == *$'\n'"${var}":* ]]; then + found=$((found + 1)) + else + missing+=("${var}") + fi +done +unset plaintext + +if ((${#missing[@]} > 0)); then + die "the key decrypted the file, but the result is missing keys that +render-config.sh requires: ${missing[*]} + +The backup is fine; the encrypted file is incomplete. Fix it with +'make secrets-edit' on a working host, then verify the backup again." +fi + +# --------------------------------------------------------------------------- +# 6. Housekeeping notes on the copy itself +# --------------------------------------------------------------------------- +mode="$(stat -Lc '%a' "${BACKUP}" 2>/dev/null || stat -Lf '%Lp' "${BACKUP}" 2>/dev/null || true)" +if [[ -n "${mode}" && "${mode: -2}" != "00" ]]; then + warn "mode ${mode} — group or other can read this copy: chmod 600 $(printf '%q' "${BACKUP_ABS}")" +fi + +printf '\033[0;32mok\033[0m — %s decrypts secrets/%s.sops.yaml (%d/%d keys)\n' \ + "$(basename "${BACKUP}")" "${STACK}" "${found}" "${#REQUIRED[@]}" +printf ' Proven: this key, on its own, recovers every secret in the repo.\n' +printf ' Not proven: that where you keep it will still exist after a fire,\n' +printf ' a theft, or a forgotten password. That part is your judgement.\n' diff --git a/secrets/README.md b/secrets/README.md index 41a2bd9..e4f6336 100644 --- a/secrets/README.md +++ b/secrets/README.md @@ -49,7 +49,14 @@ public half into `.sops.yaml`, and encrypts `observability.example.yaml` into `observability.sops.yaml` for you to fill in. **Back up `~/.config/sops/age/keys.txt` somewhere outside this machine.** -Without it the encrypted file is unrecoverable. +Without it the encrypted file is unrecoverable — not "reset with some effort", +but gone, with every secret in it re-entered by hand on four devices. The +procedure, and the way to prove the backup actually decrypts, are in +[`docs/runbooks/back-up-the-age-key.md`](../docs/runbooks/back-up-the-age-key.md): + +```bash +make secrets-verify-backup KEY=/path/to/the/copy +``` ## Editing