From 76459cd0c04ed7b229d50d2ef6eec1e69d8be274 Mon Sep 17 00:00:00 2001 From: navigator Date: Thu, 1 Oct 2026 19:20:39 +0000 Subject: [PATCH 1/2] iso, rootfs-cleanup: ship the ISO and its images with an empty machine-id buildtasks#15 empties /etc/machine-id in every layer bt-layer exports, which covers the tar.zst templates. The ISO and every image made from it did not pass through that reset (Keel-Linux/tracker#24). fab squashes root.patched into /live/10root.squashfs with the bootstrap's id in it, and tkl-installer unpacks that squashfs onto the disk as it is: it has no machine-id step, so every machine installed from one ISO shared the id, and with it the DHCP DUID and IAID, the journal and networkd identity and keel's MariaDB server_id. bt-vm, bt-container, bt-xen, bt-openstack, bt-ec2, bt-otc and bt-docker extract that rootfs and then run apt and patches in a chroot, where systemd's postinst can write a fresh id of its own. - bin/reset-machine-id applies layer_reset_machine_id to any rootfs: an empty /etc/machine-id, /var/lib/dbus/machine-id the link to it when present. - bt-iso builds root.patched first, resets it, then lets fab squash it. The second make reuses root.patched, since nothing it depends on changed. - bin/iso-machine-id-check reads the squashfs out of the ISO with xorriso and the two files out of it with unsquashfs, and refuses an ISO that is not in that state. bt-iso runs it before screenshots and release, -u included, so an ISO made before this change is not released again. - bin/rootfs-cleanup, which every image made from the ISO's rootfs runs after its chroot work, resets the id last. - layer-lib gains layer_machine_id_is_reset, the predicate the check uses, and its comment no longer says a first boot disables units: preset-all on a first boot only enables, unless systemd was built with first-boot-full-preset, which Debian's is not (review of buildtasks#15). tests/machine-id covers the predicate, both scripts, rootfs-cleanup with a chroot stub that writes an id, and bt-iso against a fake product whose root.patched carries the 2026-09-30 id: the released ISO is empty, -u with an old ISO is refused before iso-release, a failed root.patched stops the build. Reverting the bt-iso or the rootfs-cleanup change each turns it red. CI installs squashfs-tools and xorriso for it. Coverage stays at 99. --- .github/workflows/tests.yml | 2 + COVERAGE.md | 16 ++ bin/iso-machine-id-check | 89 +++++++++++ bin/layer-lib | 23 ++- bin/reset-machine-id | 41 +++++ bin/rootfs-cleanup | 10 +- bt-iso | 16 +- tests/coverage.sh | 10 +- tests/layer | 14 ++ tests/machine-id | 298 ++++++++++++++++++++++++++++++++++++ 10 files changed, 510 insertions(+), 9 deletions(-) create mode 100755 bin/iso-machine-id-check create mode 100755 bin/reset-machine-id create mode 100755 tests/machine-id diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index 54caea5..f01a320 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -11,3 +11,5 @@ jobs: uses: keel-linux/.github/.github/workflows/test-shell.yml@main with: threshold: 99 + # tests/machine-id makes small ISOs, as fab does + apt-packages: squashfs-tools xorriso diff --git a/COVERAGE.md b/COVERAGE.md index 8a86eed..86c6556 100644 --- a/COVERAGE.md +++ b/COVERAGE.md @@ -59,6 +59,22 @@ pattern, the digest match, the grep status check, `-I` or the symlink rule each turns the suite red. Coverage: layer-lib and bt-layer unchanged at one uncovered line each, the same lines as on 19.x. +## 2026-10-01: the machine-id of the ISO and the images made from it + +`bin/reset-machine-id` applies `layer_reset_machine_id` to any rootfs, and +`bin/iso-machine-id-check` refuses an ISO whose live squashfs carries an id, +read with `layer_machine_id_is_reset` (Keel-Linux/tracker#24). `bt-iso` builds +`root.patched` first, resets it, then lets fab squash it, and checks the ISO +before screenshots and release, `-u` included. `bin/rootfs-cleanup`, which +every image made from the ISO's rootfs runs last (bt-vm, bt-container, +bt-xen, bt-openstack, bt-ec2, bt-otc, bt-docker and the others), resets the +id after the chroot work. The new suite `tests/machine-id` makes small ISOs +with mksquashfs and xorriso, so CI installs `squashfs-tools` and `xorriso`. +Reverting the `bt-iso` or the `rootfs-cleanup` change each turns it red. +Coverage: bin/reset-machine-id 9 of 9, bin/iso-machine-id-check 35 of 35, +bin/rootfs-cleanup 11 of 11, layer-lib 390 of 391 (the predicate covered in +`tests/layer`, the same one uncovered line as on 19.x), bt-layer unchanged. + ## 2026-09-30: the machine-id reset `layer_reset_machine_id` leaves every exported layer with an empty diff --git a/bin/iso-machine-id-check b/bin/iso-machine-id-check new file mode 100755 index 0000000..be5a237 --- /dev/null +++ b/bin/iso-machine-id-check @@ -0,0 +1,89 @@ +#!/bin/bash -e +# Copyright (c) 2026 KeelLinux +# +# This file is part of buildtasks. +# +# Buildtasks is free software; you can redistribute it and/or modify it +# under the terms of the GNU Affero General Public License as published by the +# Free Software Foundation; either version 3 of the License, or (at your +# option) any later version. + +# Refuses an ISO whose live root filesystem carries a machine-id. tkl-installer +# unpacks that squashfs onto the disk as it is and makes no id of its own, so +# an id in the ISO is the id of every machine installed from it, and of every +# image bt-vm, bt-container and the others derive from it. +# +# Reads the squashfs out of the ISO with xorriso, which fab already needs to +# write the ISO, and only the two files out of the squashfs. + +APP=$(basename "$0") +fatal() { echo "FATAL [$APP]: $*" 1>&2; exit 1; } +info() { echo "INFO [$APP]: $*"; } + +usage() { +cat</dev/null || fatal "$tool not found" +done + +# the library beside this script, whatever BT the caller exported +# shellcheck source=bin/layer-lib +source "$(dirname "$(readlink -f "$0")")/layer-lib" + +work=$(mktemp -d) +trap 'rm -rf "$work"' EXIT + +if ! xorriso -osirrox on -indev "$iso" -extract "$ISO_SQUASHFS" "$work/root.squashfs" \ + >"$work/xorriso.log" 2>&1 || [[ ! -f "$work/root.squashfs" ]]; then + cat "$work/xorriso.log" >&2 + fatal "cannot read $ISO_SQUASHFS out of $iso" +fi +# unsquashfs exits 0 when a named file is not in the image, so a missing +# /etc/machine-id is told apart by the check below, not here +if ! unsquashfs -q -n -d "$work/root" "$work/root.squashfs" \ + "$LAYER_MACHINE_ID" "$LAYER_DBUS_MACHINE_ID" >"$work/unsquashfs.log" 2>&1; then + cat "$work/unsquashfs.log" >&2 + fatal "cannot read the machine-id out of $ISO_SQUASHFS in $iso" +fi +if ! layer_machine_id_is_reset "$work/root"; then + id=$work/root/$LAYER_MACHINE_ID + dbus=$work/root/$LAYER_DBUS_MACHINE_ID + if [[ -L "$id" ]]; then + id_state="a link to $(readlink "$id")" + elif [[ -f "$id" ]]; then + id_state="'$(head -c 64 "$id")'" + else + id_state=missing + fi + if [[ -L "$dbus" ]]; then + dbus_state="a link to $(readlink "$dbus")" + elif [[ -e "$dbus" ]]; then + dbus_state="a file of its own, '$(head -c 64 "$dbus")'" + else + dbus_state=absent + fi + fatal "$iso does not ship an empty machine-id: every machine installed" \ + "from it would share one. /$LAYER_MACHINE_ID is $id_state," \ + "/$LAYER_DBUS_MACHINE_ID is $dbus_state" +fi +info "$iso: empty /$LAYER_MACHINE_ID, each machine makes its own" diff --git a/bin/layer-lib b/bin/layer-lib index dce462c..542c676 100755 --- a/bin/layer-lib +++ b/bin/layer-lib @@ -579,14 +579,31 @@ layer_audit_keys() { # # /etc/machine-id is left an empty file, as Debian's cloud images ship it, # and systemd makes the id at each machine's boot. Not missing and not -# "uninitialized": either makes that boot a first boot to systemd, which -# presets every unit and undoes what the build enabled and masked -# (machine-id(5), "First Boot Semantics"). D-Bus's copy becomes the link to +# "uninitialized": either makes that boot a first boot to systemd, which runs +# preset-all over every unit (machine-id(5), "First Boot Semantics"). Built +# without first-boot-full-preset, as Debian's is, that only enables: every +# unit a preset enables is enabled, including the ones the build disabled on +# purpose, and nothing is disabled. D-Bus's copy becomes the link to # it that tmpfiles.d/dbus.conf makes when it is absent, because systemd reads # that file when /etc/machine-id is empty and would bring the old id back. LAYER_MACHINE_ID=etc/machine-id LAYER_DBUS_MACHINE_ID=var/lib/dbus/machine-id +# layer_machine_id_is_reset ROOTFS: true when ROOTFS is in that state: an +# empty /etc/machine-id that is a file of its own, and no D-Bus copy or the +# link to it. What an image is checked against when it was not made here +# (bin/iso-machine-id-check). +layer_machine_id_is_reset() { + local rootfs=$1 + local id=$rootfs/$LAYER_MACHINE_ID + local dbus=$rootfs/$LAYER_DBUS_MACHINE_ID + [[ -f "$id" && ! -L "$id" && ! -s "$id" ]] || return 1 + if [[ -e "$dbus" || -L "$dbus" ]]; then + [[ -L "$dbus" && "$(readlink "$dbus")" == "/$LAYER_MACHINE_ID" ]] || return 1 + fi + return 0 +} + # A tree already in that state is not written, so a child layer does not # copy its parent's empty file up into its own tarball. layer_reset_machine_id() { diff --git a/bin/reset-machine-id b/bin/reset-machine-id new file mode 100755 index 0000000..51a2216 --- /dev/null +++ b/bin/reset-machine-id @@ -0,0 +1,41 @@ +#!/bin/bash -e +# Copyright (c) 2026 KeelLinux +# +# This file is part of buildtasks. +# +# Buildtasks is free software; you can redistribute it and/or modify it +# under the terms of the GNU Affero General Public License as published by the +# Free Software Foundation; either version 3 of the License, or (at your +# option) any later version. + +# Leaves a root filesystem with the machine identity every image must ship: +# an empty /etc/machine-id and /var/lib/dbus/machine-id, when present, the +# link to it, so that each machine makes its own id at boot. The same reset +# bt-layer applies to every layer (layer_reset_machine_id in bin/layer-lib), +# for the images that do not go through bt-layer: the ISO, which bt-iso +# squashes from root.patched, and every image made from the ISO's rootfs +# (bt-vm, bt-container, bt-xen, bt-openstack and the others, through +# bin/rootfs-cleanup). + +APP=$(basename "$0") +fatal() { echo "FATAL [$APP]: $*" 1>&2; exit 1; } + +usage() { +cat<&1 | tee build/log || true + mkdir -p build + # root.patched first, so its machine-id is emptied before fab squashes it + # into the ISO (bin/reset-machine-id); the rest of the build then reuses + # it, since nothing it depends on has changed + make build/stamps/root.patched 2>&1 | tee build/log || true + if [[ -f build/stamps/root.patched ]]; then + "$BT/bin/reset-machine-id" build/root.patched \ + || fatal "cannot reset the machine-id of $appname" + make 2>&1 | tee -a build/log || true + fi if [[ ! -e build/product.iso ]]; then if [[ -z "$BT_DEBUG" ]]; then @@ -410,6 +419,11 @@ else fi fi +# whether built here or reused with -u, an ISO that ships a machine-id is +# not released, nor screenshot +"$BT/bin/iso-machine-id-check" build/product.iso \ + || fatal "build/product.iso ships a machine-id - rebuild it without -u" + if [[ -z "$no_screens" ]]; then if [[ -n "$use_existing" ]]; then info "Product iso being loaded as docker container" diff --git a/tests/coverage.sh b/tests/coverage.sh index 173aa76..01fd357 100755 --- a/tests/coverage.sh +++ b/tests/coverage.sh @@ -6,13 +6,15 @@ # bin/layer-measure-lib, bin/layer-report-lib and bt-layer-measure under # tests/measure # bin/layer-compare-lib under tests/compare +# bin/reset-machine-id, bin/iso-machine-id-check and bin/rootfs-cleanup +# under tests/machine-id # Exits 1 when a measured file is below the threshold (default 95), 2 when a # tool is missing. # # tests/coverage.sh [THRESHOLD] (or COVERAGE_THRESHOLD in the environment) # # COVERAGE_DIR keeps the kcov report (default: a temporary directory). -# Needs the Debian packages kcov, zstd, git and gpg. +# Needs the Debian packages kcov, zstd, git, gpg, squashfs-tools and xorriso. # # The tests run the bt-* scripts from a scratch copy (they need their own # config directory), so kcov lists the copy next to the checkout: the two @@ -25,7 +27,7 @@ set -euo pipefail here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" threshold="${1:-${COVERAGE_THRESHOLD:-95}}" -for tool in kcov zstd git gpg; do +for tool in kcov zstd git gpg mksquashfs unsquashfs xorriso; do if ! command -v "$tool" >/dev/null; then echo "$tool not found (apt-get install $tool)" >&2 exit 2 @@ -33,9 +35,9 @@ for tool in kcov zstd git gpg; do done report="${COVERAGE_DIR:-$(mktemp -d)}" -measured=/bin/layer-lib,/bt-layer,/bin/aplinfo-lib,/bt-aplinfo,/bin/signature-lib,/bin/generate-signature,/bin/layer-measure-lib,/bin/layer-compare-lib,/bin/layer-report-lib,/bt-layer-measure +measured=/bin/layer-lib,/bt-layer,/bin/aplinfo-lib,/bt-aplinfo,/bin/signature-lib,/bin/generate-signature,/bin/layer-measure-lib,/bin/layer-compare-lib,/bin/layer-report-lib,/bt-layer-measure,/bin/reset-machine-id,/bin/iso-machine-id-check,/bin/rootfs-cleanup skipped=/tests/,/scratch-bt/bin/generate-signature -for suite in layer aplinfo signature measure compare; do +for suite in layer aplinfo signature measure compare machine-id; do kcov --include-pattern="$measured" --exclude-pattern="$skipped" \ "$report" "$here/$suite" done diff --git a/tests/layer b/tests/layer index 4e0e0d1..4cce080 100755 --- a/tests/layer +++ b/tests/layer @@ -33,6 +33,8 @@ # layer_reset_machine_id (an id and a D-Bus copy, already reset and # left untouched, missing, a symlink, a D-Bus link elsewhere, no D-Bus # copy, no /etc, a directory in the way of either); +# layer_machine_id_is_reset (reset, no D-Bus copy, populated, a D-Bus +# file of its own); # layer_check_build (ok, no stamp, not mounted, no upper, a private key, # no key list); # layer_hash_size (ok, missing); layer_write_sha256; layer_umount_targets @@ -519,6 +521,18 @@ rm "$mid_root/var/lib/dbus/machine-id" layer_reset_machine_id "$mid_root" [[ ! -e "$mid_root/var/lib/dbus/machine-id" && ! -L "$mid_root/var/lib/dbus/machine-id" ]] \ || fatal "mid-no-dbus: a D-Bus copy was made" +# the predicate the ISO check reads an image with agrees with the reset +layer_machine_id_is_reset "$mid_root" || fatal "mid-is-reset-no-dbus: refused" +ln -s /etc/machine-id "$mid_root/var/lib/dbus/machine-id" +layer_machine_id_is_reset "$mid_root" || fatal "mid-is-reset-link: refused" +rm "$mid_root/var/lib/dbus/machine-id" +echo f0e97605ab594989b4d78f4a126b5b37 > "$mid_root/var/lib/dbus/machine-id" +expect_fail mid-is-reset-dbus-file layer_machine_id_is_reset "$mid_root" +rm "$mid_root/var/lib/dbus/machine-id" +chmod 0644 "$mid_root/etc/machine-id" +echo f0e97605ab594989b4d78f4a126b5b37 > "$mid_root/etc/machine-id" +expect_fail mid-is-reset-populated layer_machine_id_is_reset "$mid_root" +layer_reset_machine_id "$mid_root" expect_fail mid-no-etc layer_reset_machine_id "$tmp/no-such-root" out=$(layer_reset_machine_id "$tmp/no-such-root" 2>&1) || true [[ "$out" == *"no /etc"* ]] || fatal "mid-no-etc: unexpected message: $out" diff --git a/tests/machine-id b/tests/machine-id new file mode 100755 index 0000000..b3c2539 --- /dev/null +++ b/tests/machine-id @@ -0,0 +1,298 @@ +#!/bin/bash -e +# Test that the images which do not go through bt-layer ship an empty +# machine-id too (Keel-Linux/tracker#24): the ISO bt-iso writes, and every +# image made from the ISO's rootfs through bin/rootfs-cleanup (bt-vm, +# bt-container, bt-xen, bt-openstack and the others). Nothing is built: +# products are fake Makefiles, and the ISOs are made from a few files with +# mksquashfs and xorriso. +# +# Code paths covered: +# layer_machine_id_is_reset: reset, populated, missing, a symlink, empty +# but a D-Bus copy of its own, a D-Bus link elsewhere, no D-Bus copy; +# bin/reset-machine-id: usage, no such directory, an id and a D-Bus copy +# reset, a tree already reset left untouched, an id that cannot be reset; +# bin/rootfs-cleanup: usage, no such directory, the chroot cleanup and the +# reset after it, a reset that fails stops it; +# bin/iso-machine-id-check: usage, no such file, a tool missing, a clean +# ISO, an ISO with an id, with a D-Bus copy of its own, with no +# machine-id at all, with a symlinked one, an ISO without the squashfs, +# a squashfs that cannot be read, ISO_SQUASHFS elsewhere; +# bt-iso: a build whose root.patched carries an id makes an ISO without +# one; -u with an ISO that carries one is refused before release; a +# build whose root.patched fails stops before the ISO. + +fatal() { echo "FATAL [$(basename "$0")]: $*" 1>&2; exit 1; } +info() { echo "INFO [$(basename "$0")]: $*"; } + +usage() { +cat</dev/null || fatal "$tool not found" +done + +BT=$(dirname "$(dirname "$(readlink -f "$0")")") +export BT +source "$BT/bin/layer-lib" + +tmp=$(mktemp -d) +trap 'rm -rf "$tmp"' EXIT + +expect_fail() { + local label=$1 + shift + if "$@" >/dev/null 2>&1; then + fatal "$label: should have failed" + fi +} + +assert_eq() { + local label=$1 + local got=$2 + local want=$3 + [[ "$got" == "$want" ]] || fatal "$label: got '$got', want '$want'" +} + +# the id every image carried on 2026-09-30 +OLD_ID=f0e97605ab594989b4d78f4a126b5b37 + +# populated_root DIR: a tree with an id and the D-Bus copy of it that +# dbus-daemon's postinst writes as a file of its own +populated_root() { + mkdir -p "$1/etc" "$1/var/lib/dbus" + echo "$OLD_ID" > "$1/etc/machine-id" + cp "$1/etc/machine-id" "$1/var/lib/dbus/machine-id" +} + +# assert_reset LABEL DIR +assert_reset() { + [[ -f "$2/etc/machine-id" && ! -L "$2/etc/machine-id" ]] || fatal "$1: not a regular file" + [[ ! -s "$2/etc/machine-id" ]] || fatal "$1: the id is still there" + assert_eq "$1-dbus" "$(readlink "$2/var/lib/dbus/machine-id")" /etc/machine-id +} + +# make_iso DIR ISO [PATH]: DIR squashed to PATH (default +# /live/10root.squashfs) inside ISO, the way fab lays it out +make_iso() { + local dir=$1 + local iso=$2 + local path=${3:-/live/10root.squashfs} + local cd=$tmp/cd.$RANDOM + mkdir -p "$cd$(dirname "$path")" + mksquashfs "$dir" "$cd$path" -quiet -no-progress -noappend >/dev/null + xorriso -as mkisofs -quiet -r -J -o "$iso" "$cd" >/dev/null 2>&1 + rm -rf "$cd" +} + +info "1: layer_machine_id_is_reset" +r=$tmp/pred +mkdir -p "$r/etc" "$r/var/lib/dbus" +: > "$r/etc/machine-id" +ln -s /etc/machine-id "$r/var/lib/dbus/machine-id" +layer_machine_id_is_reset "$r" || fatal "pred-reset: a reset tree is not reset" +rm "$r/var/lib/dbus/machine-id" +layer_machine_id_is_reset "$r" || fatal "pred-no-dbus: no D-Bus copy is reset" +echo "$OLD_ID" > "$r/var/lib/dbus/machine-id" +expect_fail pred-dbus-file layer_machine_id_is_reset "$r" +rm "$r/var/lib/dbus/machine-id" +ln -s /var/lib/dbus/other "$r/var/lib/dbus/machine-id" +expect_fail pred-dbus-elsewhere layer_machine_id_is_reset "$r" +rm "$r/var/lib/dbus/machine-id" +echo "$OLD_ID" > "$r/etc/machine-id" +expect_fail pred-populated layer_machine_id_is_reset "$r" +rm "$r/etc/machine-id" +expect_fail pred-missing layer_machine_id_is_reset "$r" +: > "$tmp/empty-elsewhere" +ln -s "$tmp/empty-elsewhere" "$r/etc/machine-id" +expect_fail pred-symlink layer_machine_id_is_reset "$r" + +info "2: bin/reset-machine-id" +msg=$("$BT/bin/reset-machine-id" 2>&1) && fatal "reset-usage: should have failed" +[[ "$msg" == *"Syntax:"* ]] || fatal "reset-usage: no syntax: $msg" +msg=$("$BT/bin/reset-machine-id" "$tmp/no-such-root" 2>&1) && fatal "reset-missing: should have failed" +[[ "$msg" == *"no such directory"* ]] || fatal "reset-missing: unexpected message: $msg" +r=$tmp/reset +populated_root "$r" +"$BT/bin/reset-machine-id" "$r" +assert_reset reset "$r" +assert_eq reset-mode "$(stat -c %a "$r/etc/machine-id")" 444 +# a tree already reset is left as it is +touch -d @1234567890 "$r/etc/machine-id" +"$BT/bin/reset-machine-id" "$r" +assert_eq reset-untouched "$(stat -c %Y "$r/etc/machine-id")" 1234567890 +# with BT pointing at another checkout, the library beside the script is used +env BT="$tmp/nowhere" "$BT/bin/reset-machine-id" "$r" +rm "$r/etc/machine-id" +mkdir "$r/etc/machine-id" +msg=$("$BT/bin/reset-machine-id" "$r" 2>&1) && fatal "reset-dir: should have failed" +[[ "$msg" == *"cannot reset the machine-id of $r"* ]] || fatal "reset-dir: unexpected message: $msg" + +info "3: bin/rootfs-cleanup" +mkdir -p "$tmp/stubs" +# tklpatch-chroot runs the cleanup in the chroot and, like any package +# configured in there, may write an id: the stub does both +cat > "$tmp/stubs/tklpatch-chroot" <> "$tmp/chroot.log" +rm -f "\$1/etc/machine-id" +echo $OLD_ID > "\$1/etc/machine-id" +EOF +printf '#!/bin/bash\necho "$*" >> "%s/fuser.log"\n' "$tmp" > "$tmp/stubs/fuser" +chmod 755 "$tmp/stubs"/* +export PATH=$tmp/stubs:$PATH +msg=$("$BT/bin/rootfs-cleanup" 2>&1) && fatal "cleanup-usage: should have failed" +[[ "$msg" == *"Syntax:"* ]] || fatal "cleanup-usage: no syntax: $msg" +msg=$("$BT/bin/rootfs-cleanup" "$tmp/no-such-root" 2>&1) && fatal "cleanup-missing: should have failed" +[[ "$msg" == *"no such directory"* ]] || fatal "cleanup-missing: unexpected message: $msg" +r=$tmp/cleanup +populated_root "$r" +"$BT/bin/rootfs-cleanup" "$r" +assert_reset cleanup "$r" +grep -q "^$r rm -rf .*/var/lib/apt/lists/\*" "$tmp/chroot.log" || fatal "cleanup: the chroot cleanup did not run" +grep -qx -- "-k $r" "$tmp/fuser.log" || fatal "cleanup: fuser -k did not run" +rm "$r/var/lib/dbus/machine-id" +mkdir -p "$r/var/lib/dbus/machine-id/x" +msg=$("$BT/bin/rootfs-cleanup" "$r" 2>&1) && fatal "cleanup-fail: should have failed" +[[ "$msg" == *"cannot reset the machine-id of $r"* ]] || fatal "cleanup-fail: unexpected message: $msg" + +info "4: bin/iso-machine-id-check" +check=$BT/bin/iso-machine-id-check +msg=$("$check" 2>&1) && fatal "check-usage: should have failed" +[[ "$msg" == *"Syntax:"* ]] || fatal "check-usage: no syntax: $msg" +msg=$("$check" "$tmp/no-such.iso" 2>&1) && fatal "check-missing: should have failed" +[[ "$msg" == *"no such file"* ]] || fatal "check-missing: unexpected message: $msg" +r=$tmp/isoroot +populated_root "$r" +make_iso "$r" "$tmp/dirty.iso" +msg=$("$check" "$tmp/dirty.iso" 2>&1) && fatal "check-dirty: an ISO with an id passed" +[[ "$msg" == *"does not ship an empty machine-id"* ]] || fatal "check-dirty: unexpected message: $msg" +[[ "$msg" == *"'$OLD_ID'"* ]] || fatal "check-dirty: the id is not named: $msg" +[[ "$msg" == *"a file of its own"* ]] || fatal "check-dirty: the D-Bus copy is not named: $msg" +rm "$r/var/lib/dbus/machine-id" +make_iso "$r" "$tmp/id-only.iso" +msg=$("$check" "$tmp/id-only.iso" 2>&1) && fatal "check-id-only: an ISO with an id passed" +[[ "$msg" == *"/var/lib/dbus/machine-id is absent"* ]] || fatal "check-id-only: unexpected message: $msg" +cp "$r/etc/machine-id" "$r/var/lib/dbus/machine-id" +layer_reset_machine_id "$r" +make_iso "$r" "$tmp/clean.iso" +msg=$("$check" "$tmp/clean.iso" 2>&1) || fatal "check-clean: a clean ISO was refused: $msg" +[[ "$msg" == *"empty /etc/machine-id"* ]] || fatal "check-clean: unexpected message: $msg" +rm "$r/var/lib/dbus/machine-id" +make_iso "$r" "$tmp/no-dbus.iso" +"$check" "$tmp/no-dbus.iso" >/dev/null 2>&1 || fatal "check-no-dbus: an ISO without a D-Bus copy was refused" +echo "$OLD_ID" > "$r/var/lib/dbus/machine-id" +make_iso "$r" "$tmp/dbus-file.iso" +msg=$("$check" "$tmp/dbus-file.iso" 2>&1) && fatal "check-dbus-file: should have failed" +[[ "$msg" == *"/etc/machine-id is '',"* ]] || fatal "check-dbus-file: unexpected message: $msg" +rm "$r/var/lib/dbus/machine-id" +ln -s /var/lib/dbus/other "$r/var/lib/dbus/machine-id" +rm "$r/etc/machine-id" +make_iso "$r" "$tmp/no-id.iso" +msg=$("$check" "$tmp/no-id.iso" 2>&1) && fatal "check-no-id: an ISO without a machine-id passed" +[[ "$msg" == *"/etc/machine-id is missing"* ]] || fatal "check-no-id: unexpected message: $msg" +[[ "$msg" == *"is a link to /var/lib/dbus/other"* ]] || fatal "check-no-id: the D-Bus link is not named: $msg" +ln -s /run/machine-id "$r/etc/machine-id" +make_iso "$r" "$tmp/link-id.iso" +msg=$("$check" "$tmp/link-id.iso" 2>&1) && fatal "check-link: a symlinked machine-id passed" +[[ "$msg" == *"/etc/machine-id is a link to /run/machine-id"* ]] || fatal "check-link: unexpected message: $msg" +# fab's layout is the default; another path is read when asked for +rm "$r/etc/machine-id" "$r/var/lib/dbus/machine-id" +layer_reset_machine_id "$r" +make_iso "$r" "$tmp/elsewhere.iso" /casper/filesystem.squashfs +msg=$("$check" "$tmp/elsewhere.iso" 2>&1) && fatal "check-no-squashfs: should have failed" +[[ "$msg" == *"cannot read /live/10root.squashfs"* ]] || fatal "check-no-squashfs: unexpected message: $msg" +ISO_SQUASHFS=/casper/filesystem.squashfs "$check" "$tmp/elsewhere.iso" >/dev/null 2>&1 \ + || fatal "check-iso-squashfs: ISO_SQUASHFS was not read" +# a file at the squashfs path that is not a squashfs +mkdir -p "$tmp/badcd/live" +echo "not a squashfs" > "$tmp/badcd/live/10root.squashfs" +xorriso -as mkisofs -quiet -r -J -o "$tmp/bad.iso" "$tmp/badcd" >/dev/null 2>&1 +msg=$("$check" "$tmp/bad.iso" 2>&1) && fatal "check-bad-squashfs: should have failed" +[[ "$msg" == *"cannot read the machine-id out of"* ]] || fatal "check-bad-squashfs: unexpected message: $msg" +# a tool missing: PATH with everything but unsquashfs +mkdir -p "$tmp/notools" +for t in bash basename dirname readlink xorriso; do ln -s "$(command -v "$t")" "$tmp/notools/$t"; done +msg=$(PATH=$tmp/notools "$check" "$tmp/clean.iso" 2>&1) && fatal "check-no-tool: should have failed" +[[ "$msg" == *"unsquashfs not found"* ]] || fatal "check-no-tool: unexpected message: $msg" + +info "5: bt-iso (scratch copy with stubs)" +sbt=$tmp/scratch-bt +mkdir -p "$sbt/bin" "$sbt/config" "$tmp/isos" +cp "$BT/bt-iso" "$sbt/bt-iso" +for f in layer-lib reset-machine-id iso-machine-id-check; do + cp "$BT/bin/$f" "$sbt/bin/$f" +done +# iso-release copies the ISO where BT_ISOS says; the stub records that +# shellcheck disable=SC2016 # the stub's own variables, expanded when it runs +printf '#!/bin/bash\necho "$*" > "%s/iso-release.log"\ncp build/product.iso "${!#}/released.iso"\n' "$tmp" \ + > "$sbt/bin/iso-release" +chmod 755 "$sbt/bin/iso-release" +cat > "$sbt/config/common.cfg" < "$dir/Makefile" < build/root.patched/etc/machine-id + cp build/root.patched/etc/machine-id build/root.patched/var/lib/dbus/machine-id + touch \$@ + +build/product.iso: build/stamps/root.patched + echo rebuilt >> build/root.patched.count + mkdir -p build/cdroot/live + mksquashfs build/root.patched build/cdroot/live/10root.squashfs -quiet -no-progress -noappend + xorriso -as mkisofs -quiet -r -J -o \$@ build/cdroot + +clean: + rm -rf build +EOF +} +write_iso_product "$tmp/products/core" +# no setup, no screenshots; DEBUG turns the missing TKLDev profile and +# preseed into warnings +export DEBUG=1 +unset BT_DEBUG +(cd "$tmp" && "$sbt/bt-iso" -s -n core) > "$tmp/bt-iso.log" 2>&1 \ + || { cat "$tmp/bt-iso.log"; fatal "bt-iso: the build failed"; } +grep -q "empty /etc/machine-id" "$tmp/bt-iso.log" || fatal "bt-iso: the ISO was not checked" +mkdir -p "$tmp/released" +xorriso -osirrox on -indev "$tmp/isos/released.iso" \ + -extract /live/10root.squashfs "$tmp/released/root.squashfs" >/dev/null 2>&1 +unsquashfs -q -n -d "$tmp/released/root" "$tmp/released/root.squashfs" >/dev/null +assert_reset bt-iso "$tmp/released/root" +# root.patched is built once, by the first make, and reused by the second +assert_eq bt-iso-once "$(wc -l < "$tmp/products/core/build/root.patched.count")" 1 +# -u releases nothing that ships an id: an ISO made before this fix +populated_root "$tmp/isoroot-old" +make_iso "$tmp/isoroot-old" "$tmp/products/core/build/product.iso" +rm -f "$tmp/iso-release.log" +msg=$(cd "$tmp" && "$sbt/bt-iso" -s -u core 2>&1) && fatal "bt-iso-u: an ISO with an id was released" +[[ "$msg" == *"ships a machine-id - rebuild it without -u"* ]] || fatal "bt-iso-u: unexpected message: $msg" +[[ ! -e "$tmp/iso-release.log" ]] || fatal "bt-iso-u: iso-release ran" +# a root.patched that fails stops the build before any ISO +write_iso_product "$tmp/products/broken" +touch "$tmp/products/broken/FAIL" +msg=$(cd "$tmp" && "$sbt/bt-iso" -s -n broken 2>&1) && fatal "bt-iso-broken: should have failed" +[[ "$msg" == *"Build failed..."* ]] || fatal "bt-iso-broken: unexpected message: $msg" +[[ ! -e "$tmp/products/broken/build/product.iso" ]] || fatal "bt-iso-broken: an ISO was made" + +info "all machine-id tests passed" From f2a79a3ecd351074e2e5d90ed86efbfa6e4e0219 Mon Sep 17 00:00:00 2001 From: navigator Date: Thu, 1 Oct 2026 19:29:08 +0000 Subject: [PATCH 2/2] iso-machine-id-check, bt-iso: work next to the ISO, clean up on failure From the review of the previous commit: - the squashfs is copied out next to the ISO, in build/ for bt-iso, not to /tmp, which can be a tmpfs smaller than a squashfs; a directory that cannot take it is refused by name; - the message says what is at each path, a directory included; - bt-iso detaches a root.sandbox left mounted by an earlier DEBUG run before the reset writes into root.patched under it, cleans the build when the reset or the check fails, and tells an ISO reused with -u apart from one it just built; - rootfs-cleanup no longer repeats reset-machine-id's own message; - the first-boot comment names Debian 13's systemd 257, since upstream means to turn first-boot-full-preset on later. --- COVERAGE.md | 2 +- bin/iso-machine-id-check | 40 ++++++++++++++++++++-------------------- bin/layer-lib | 4 +++- bin/rootfs-cleanup | 4 ++-- bt-iso | 19 +++++++++++++++---- tests/machine-id | 27 +++++++++++++++++++++------ 6 files changed, 62 insertions(+), 34 deletions(-) diff --git a/COVERAGE.md b/COVERAGE.md index 86c6556..79173c4 100644 --- a/COVERAGE.md +++ b/COVERAGE.md @@ -71,7 +71,7 @@ bt-xen, bt-openstack, bt-ec2, bt-otc, bt-docker and the others), resets the id after the chroot work. The new suite `tests/machine-id` makes small ISOs with mksquashfs and xorriso, so CI installs `squashfs-tools` and `xorriso`. Reverting the `bt-iso` or the `rootfs-cleanup` change each turns it red. -Coverage: bin/reset-machine-id 9 of 9, bin/iso-machine-id-check 35 of 35, +Coverage: bin/reset-machine-id 9 of 9, bin/iso-machine-id-check 30 of 30, bin/rootfs-cleanup 11 of 11, layer-lib 390 of 391 (the predicate covered in `tests/layer`, the same one uncovered line as on 19.x), bt-layer unchanged. diff --git a/bin/iso-machine-id-check b/bin/iso-machine-id-check index be5a237..ef335e4 100755 --- a/bin/iso-machine-id-check +++ b/bin/iso-machine-id-check @@ -24,7 +24,8 @@ usage() { cat<&1 | tee build/log || true if [[ -f build/stamps/root.patched ]]; then - "$BT/bin/reset-machine-id" build/root.patched \ - || fatal "cannot reset the machine-id of $appname" + # a root.sandbox left mounted on root.patched by an earlier run + # (DEBUG skips the clean) would be an overlay whose lower layer + # changes under it + deck -D build/root.sandbox 2>/dev/null || true + if ! "$BT/bin/reset-machine-id" build/root.patched; then + [[ -n "$BT_DEBUG" ]] || product_build_clean + fatal "cannot reset the machine-id of $appname" + fi make 2>&1 | tee -a build/log || true fi @@ -421,8 +427,13 @@ fi # whether built here or reused with -u, an ISO that ships a machine-id is # not released, nor screenshot -"$BT/bin/iso-machine-id-check" build/product.iso \ - || fatal "build/product.iso ships a machine-id - rebuild it without -u" +if ! "$BT/bin/iso-machine-id-check" build/product.iso; then + if [[ -n "$use_existing" ]]; then + fatal "build/product.iso ships a machine-id - rebuild it without -u" + fi + [[ -n "$BT_DEBUG" ]] || product_build_clean + fatal "build/product.iso ships a machine-id although root.patched was reset - a build step after it wrote one" +fi if [[ -z "$no_screens" ]]; then if [[ -n "$use_existing" ]]; then diff --git a/tests/machine-id b/tests/machine-id index b3c2539..07b8178 100755 --- a/tests/machine-id +++ b/tests/machine-id @@ -15,7 +15,8 @@ # reset after it, a reset that fails stops it; # bin/iso-machine-id-check: usage, no such file, a tool missing, a clean # ISO, an ISO with an id, with a D-Bus copy of its own, with no -# machine-id at all, with a symlinked one, an ISO without the squashfs, +# machine-id at all, with a symlinked one, with a directory, an ISO in a +# read-only directory, an ISO without the squashfs, # a squashfs that cannot be read, ISO_SQUASHFS elsewhere; # bt-iso: a build whose root.patched carries an id makes an ISO without # one; -u with an ISO that carries one is refused before release; a @@ -173,12 +174,12 @@ populated_root "$r" make_iso "$r" "$tmp/dirty.iso" msg=$("$check" "$tmp/dirty.iso" 2>&1) && fatal "check-dirty: an ISO with an id passed" [[ "$msg" == *"does not ship an empty machine-id"* ]] || fatal "check-dirty: unexpected message: $msg" -[[ "$msg" == *"'$OLD_ID'"* ]] || fatal "check-dirty: the id is not named: $msg" -[[ "$msg" == *"a file of its own"* ]] || fatal "check-dirty: the D-Bus copy is not named: $msg" +[[ "$msg" == *"/etc/machine-id is a file holding '$OLD_ID'"* ]] || fatal "check-dirty: the id is not named: $msg" +[[ "$msg" == *"/var/lib/dbus/machine-id is a file holding '$OLD_ID'"* ]] || fatal "check-dirty: the D-Bus copy is not named: $msg" rm "$r/var/lib/dbus/machine-id" make_iso "$r" "$tmp/id-only.iso" msg=$("$check" "$tmp/id-only.iso" 2>&1) && fatal "check-id-only: an ISO with an id passed" -[[ "$msg" == *"/var/lib/dbus/machine-id is absent"* ]] || fatal "check-id-only: unexpected message: $msg" +[[ "$msg" == *"/var/lib/dbus/machine-id is missing"* ]] || fatal "check-id-only: unexpected message: $msg" cp "$r/etc/machine-id" "$r/var/lib/dbus/machine-id" layer_reset_machine_id "$r" make_iso "$r" "$tmp/clean.iso" @@ -190,7 +191,7 @@ make_iso "$r" "$tmp/no-dbus.iso" echo "$OLD_ID" > "$r/var/lib/dbus/machine-id" make_iso "$r" "$tmp/dbus-file.iso" msg=$("$check" "$tmp/dbus-file.iso" 2>&1) && fatal "check-dbus-file: should have failed" -[[ "$msg" == *"/etc/machine-id is '',"* ]] || fatal "check-dbus-file: unexpected message: $msg" +[[ "$msg" == *"/etc/machine-id is a file holding '',"* ]] || fatal "check-dbus-file: unexpected message: $msg" rm "$r/var/lib/dbus/machine-id" ln -s /var/lib/dbus/other "$r/var/lib/dbus/machine-id" rm "$r/etc/machine-id" @@ -202,8 +203,22 @@ ln -s /run/machine-id "$r/etc/machine-id" make_iso "$r" "$tmp/link-id.iso" msg=$("$check" "$tmp/link-id.iso" 2>&1) && fatal "check-link: a symlinked machine-id passed" [[ "$msg" == *"/etc/machine-id is a link to /run/machine-id"* ]] || fatal "check-link: unexpected message: $msg" +rm "$r/etc/machine-id" +mkdir -p "$r/etc/machine-id/x" +make_iso "$r" "$tmp/dir-id.iso" +msg=$("$check" "$tmp/dir-id.iso" 2>&1) && fatal "check-dir: a directory at the machine-id passed" +[[ "$msg" == *"/etc/machine-id is a directory"* ]] || fatal "check-dir: unexpected message: $msg" +rm -r "$r/etc/machine-id" +# the squashfs is copied out next to the ISO, never to /tmp, and removed +! compgen -G "$tmp/.iso-machine-id-check.*" >/dev/null || fatal "check-work: a work directory was left next to the ISO" +mkdir -p "$tmp/ro" +cp "$tmp/clean.iso" "$tmp/ro/clean.iso" +chmod 555 "$tmp/ro" +msg=$("$check" "$tmp/ro/clean.iso" 2>&1) && fatal "check-ro: should have failed beside a read-only ISO" +[[ "$msg" == *"cannot make a work directory next to"* ]] || fatal "check-ro: unexpected message: $msg" +chmod 755 "$tmp/ro" # fab's layout is the default; another path is read when asked for -rm "$r/etc/machine-id" "$r/var/lib/dbus/machine-id" +rm -f "$r/etc/machine-id" "$r/var/lib/dbus/machine-id" layer_reset_machine_id "$r" make_iso "$r" "$tmp/elsewhere.iso" /casper/filesystem.squashfs msg=$("$check" "$tmp/elsewhere.iso" 2>&1) && fatal "check-no-squashfs: should have failed"