Repository navigation
Honour SOURCE_DATE_EPOCH in product.mk for reproducible squashfs and ISO images - #1
Merged
Merged
Conversation
With SOURCE_DATE_EPOCH exported, two builds of the same product currently still differ: the packers read the clock. Three changes, all no-ops when the variable is unset, so existing builds are untouched: - add SOURCE_DATE_EPOCH to CONF_VARS_BUILTIN, so it is exported to fab-chroot and fab-install (dpkg-deb, mkinitramfs and friends read it) - pass --set_all_file_dates to xorriso: it honours the variable for its own metadata, but not for the mtimes of the files it stores - pass --id to isohybrid, which otherwise writes a random MBR id mksquashfs needs nothing: it reads the variable itself, and fails with "SOURCE_DATE_EPOCH and command line options can't be used at the same time to set timestamp(s)" if given equivalent options as well. Measured on TKLDev 19.0 (Debian 13, mksquashfs 4.6.1, xorriso 1.5.6), building core from the same root.patched, rm -rf build/cdroot between runs: SOURCE_DATE_EPOCH=1700000000 squashfs 8c339c02.. iso 6fe6fdca.. SOURCE_DATE_EPOCH=1700000000 squashfs 8c339c02.. iso 6fe6fdca.. SOURCE_DATE_EPOCH=1800000000 squashfs b888b670.. iso 153097f3.. Without it, consecutive builds differ, as before.
tests/source-date-epoch.sh dry-runs (make -n) product.iso for a minimal fake product against share/product.mk and checks, with the variable unset and set: --set_all_file_dates on both xorriso recipes, the isohybrid --id (decimal, epoch & 0xffffffff, wrapping at 2^32), SOURCE_DATE_EPOCH in FAB_CHROOT_ENV and in fab-plan-resolve -D, and mksquashfs options unchanged. TAP output, 19 checks; 7 fail against the previous product.mk.
make has no line coverage tool, so the number is the share of passing checks in tests/source-date-epoch.sh, one per effect of the touched code. 19 of 19 on a TKLDev 19 host.
The reusable workflow test-shell.yml exports the caller's threshold as COVERAGE_THRESHOLD and runs the script without arguments, so the script enforced its own default (95) whatever the workflow said. The positional argument still wins, the environment variable comes next, 95 remains the fallback.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem: two builds of the same product from the same root.patched still produced different squashfs and ISO images even with SOURCE_DATE_EPOCH exported, so a layer cannot be rebuilt bit for bit and verified against its manifest.
Diagnosis: the packers read the clock: SOURCE_DATE_EPOCH was not exported to fab-chroot and fab-install (dpkg-deb, mkinitramfs and friends read it), xorriso honours the variable for its own metadata but not for the mtimes of the stored files, and isohybrid writes a random MBR id.
Fix, three changes in share/product.mk, all no-ops when the variable is unset: SOURCE_DATE_EPOCH added to CONF_VARS_BUILTIN, --set_all_file_dates passed to both xorriso recipes, --id passed to isohybrid (decimal, epoch masked to 32 bits). mksquashfs needs nothing and refuses equivalent options when the variable is set. The last commit makes tests/coverage.sh read COVERAGE_THRESHOLD, the variable the reusable workflow exports.
Tests: tests/source-date-epoch.sh dry-runs (make -n) product.iso for a minimal fake product against share/product.mk, with the variable unset and set, and emits TAP: 19 checks, one per effect of the touched code, 7 of which fail against the previous product.mk; tests/coverage.sh reports the share of passing checks (19/19). Measured on TKLDev 19.0 (mksquashfs 4.6.1, xorriso 1.5.6) building core twice with SOURCE_DATE_EPOCH=1700000000: identical squashfs and ISO hashes; a different epoch gives different hashes; unset, consecutive builds differ as before.
Serves BRIEF sections 5.1 and 5.4 (content-addressed layers, reproducible and verifiable).