Repository navigation
docs: Diátaxis site for eb-staged-rebuild - #17
Merged
Merged
Conversation
The site was a Sphinx build of Markdown pages under docs/, with the Python API from autodoc and nothing from the Zig build. Nothing checked its links. Its transcripts came from a walk-through script run in a directory under /tmp, and the ledger guide used /abs/... placeholders. Beside it, docs/orgmode/ held an audit of an old commit that cites a runbook which no longer exists, and two older pages: the how-to asked for eb-stack 0.5.2 or later, and the reference said one flock covers a run and left exit status 75 and the preparation program's status out. The documentation template builds the site now, as it does for eessi-arch-guard. Org pages under docs/orgmode/ are exported to reStructuredText and built by Sphinx with warnings and unresolved names as errors. docs/tools/zig_api.zig writes the API pages from the doc comments of zig/main.zig and the files it imports, sphinx-autoapi reads the Python package without importing it, and zig build docs installs the Zig autodoc site, from a compile of its own so that the binary is not built with -femit-docs. check_site.py checks the internal links and anchors, and that no page names a site; lychee checks the external links. pixi.lock pins the tools, so the docs extra leaves pyproject.toml. The first pages are the tutorial, "Rebuild an installation and put the old one back", run with Python 3.12.3 as the page says; the command reference, with the binary's help; the compatibility table; and the API page. docs/examples/tutorial.sh runs the tutorial's commands, and a new CI job runs it and shellchecks the example scripts. The contributing page says how to record a change, run the tests and write a page. The docs workflow builds the site as docbld does, checks the external links, also weekly, and publishes from main. test_cli_docs.py reads the Org pages instead of the Markdown ones. The walk-through script goes, with its two checks and their TESTMAP rows, and so do the unreleased fragments about it and about the old command page.
Half of the 144 pub declarations under zig/ had no doc comment, so their API entries were a signature alone. Three module comments were out of date: main.zig said that run, resume and rollback of a run journal were not in the binary yet, engine.zig that the build, the swap and resume would follow, and run.zig that a run had no ledger. Each of them now says what its file does, and every pub declaration says what it is, in the words of the Python function it matches where there is one. main.zig binds build_events.zig to a name, as it does the other files it imports, so docs/tools/zig_api.zig follows it and the build-events command gets an API page; builders.zig and publish.zig, which it imports first, are documented under it. The API page says which two files have no page of their own. The release binary built with -Dcommit=c44b8ad is the same file as before, sha256 78d154bfc09786f3a6a765da3587dab4323fa7b5192a3d1735f55a841346bff8.
57 of the package's 151 public modules, classes, functions and methods had no docstring, so their API entries were a signature alone: cli.main and the other entry points, the journal and outbox helpers, the probes, the command each backend runs, and the lock, among them. Each now says what it does or returns, and what it refuses. The comment on DEFAULT_GRACE_HOURS in constants.py and the module docstring of gc.py cited the staged-rebuild notes for the 144-hour grace period; they say what it assumes instead. Docstrings and comments only: with the docstrings left out, the AST of every module is the one before.
The Markdown guides showed a stand-in eb rebuilding demo/1 under /tmp/ebsr-docs-prefix, with <checkout> in place of the paths it printed, and the Org guide showed its commands without their output and asked for eb-stack 0.5.2, where the ledger tests pin 0.7.0. Seven guides replace them: install the tool and probe the software filesystem, rebuild an installed module, set permissions before the swap with a --prepare program, finish a run that failed or stopped, roll a rebuild back, remove old trees, and retire a module. Each runs on an example site under /opt/site, where EasyBuild 5.4.2 installs Example/1.0-GCCcore-15.2.0 from docs/examples/site/, and shows what the commands printed for alice on login1. docs/examples/site.sh runs the commands of every guide in order and stops at the first one that exits other than the guide says. The docs job of the tests workflow runs it with bubblewrap, Lmod, EasyBuild and the pinned eb-stack. Its ledger cases belong to the guides of the next commit; one run takes the site through the states that all the guides show.
The how-to guides run the eb-staged-rebuild binary that the install
guide puts on PATH, and their output was recorded with it. The ledger
guides use its build-events and ledger-maintenance commands, which
the Python package has only as the modules
eb_staged_rebuild.build_events and eb_staged_rebuild.ledger_maintenance.
The docs job ran docs/examples/site.sh with the Python package it
installs for the tutorial, so the script stopped at its backfill case:
eb-staged-rebuild: error: argument command: invalid choice: 'ledger-maintenance' (choose from 'probe', 'run', 'resume', 'rollback', 'gc', 'retire', 'status', 'resolve')
site.sh: eb-staged-rebuild exited 2, not 0
The job now builds the release binary and puts it first on PATH for
the script; the tutorial still runs with the Python package. The
comment at the top of the script says which eb-staged-rebuild it
needs.
The ledger guide gave its commands with /abs/eb-stack, /abs/ledger and /abs/keys, and showed as their output the refusal each printed because those paths did not exist. The build-events guide showed the refusal for a missing /etc/build-events/ledger.json and the --help text, and neither showed a record that was appended. Five guides replace them: record each rebuild in a signed ledger, record any build command with build-events, record software installed before the ledger, anchor the ledger once a day, and write the history of each package. They run on the example site of the operator's guides, with eb-stack 0.7.0 and keys made with openssl, and show the records, receipts, anchors and history pages the commands wrote, and what a rollback adds to the log. They run the binary's build-events and ledger-maintenance commands, and each guide that does names the Python module that runs the command with the package.
The topic pages described the staging, the isolation, the journal, the locks and the publication as lists of facts, and kept the limits on a page of their own, with the probe results of the walk-through host. The ledger had no page, and the figures were drawn by Graphviz at every docs build. Six pages replace them: why a rebuild is staged and swapped in, how a build is kept off the live tree, how a rebuild is published, what the journal records, how runs keep out of each other's way, and what the ledger records. Each starts from the problem, says how the tool handles it and what was turned down, and ends with its limits, among them the ones the limits page listed. The two figures are SVG files committed beside their .dot sources, with render.sh to redraw them, as in the arch-guard docs.
… files The environment page listed the variables of the command line and of the tests, but not the ones the fault hooks of the tests read, nor what a run takes out of EasyBuild's environment and sets for it. The Org reference gave the staged-rebuild notes as the source of the grace period, and had no exit status 75 and no status for a failed --prepare program. The keys of the ledger configuration, of the build-events files and of the backfill manifest were spread over the how-to guides. Four pages hold them now: the configuration files, with each key and its default; the environment variables, with what EasyBuild gets; the exit statuses of every command, build-events and ledger-maintenance included; and the files of the state root, of an installation, of the build-events outbox and of the package history. The pages that named them in plain text link to them.
The README's usage rebuilt pigz in an EESSI 2026.06 user installation, which no part of the repository runs; it now shows probe, run and status on the example site of the how-to guides, the commands that docs/examples/site.sh runs, and says what rollback does. Its download commands use curl -fsSLO, as the install guide does. The fragment that described the EESSI example goes with it. DESIGN.md said that a run takes one flock, where it takes the state root's lock, a flock and a POSIX record lock; it gave the grace period as the notes' walltime, not measured for "this site"; and among the closed differences it named the Org how-to and reference, the runbook and the README for what the explanation pages and the contributing page now say. The contributing page says outright that the integration test fails without eb or Lmod and skips its bubblewrap case where bwrap does not run. RELEASING.md's local build of a tag passes -Dcommit, which the build of v1.0.0 refuses with: error: invalid option: "commit". TESTMAP.md and the docstring of tests/check_testmap.py cited "Port plan D4", a plan that is not in the repository.
…build
The failure table of the rebuild guide gave as the cause of
eb-staged-rebuild: /opt/site/software/M4/1.4.20 is a missing, not an existing rebuild directory
a dependency that the run would build as well, and the explanation of
the isolation said that the preflight refuses a --robot that would
also build a missing dependency. On the example site, eb --robot
--rebuild installs Example alone: M4/1.4.20 is a build dependency of
GCCcore 15.2.0, which is installed, and EasyBuild's dry run lists it
only because a dry run resolves the dependencies of installed modules
as well.
Both pages say what the dry run lists and that the run refuses a
dependency it marks [ ], and the explanation counts the refusal among
its limits. The guide's advice is to rebuild without --robot once the
dependencies the build needs are installed.
The docstring of docs/examples/eb-standin still called it the stand-in EasyBuild of the documentation walk-through. The walk-through script is gone; the tutorial installs the stand-in as its eb, and runs it with --backend installpath, the one way it answers.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The documentation is a Diátaxis site built from Org pages under
docs/orgmode/, exported and built by Sphinx, with the API generated from the Python package and the Zig build. It has a tutorial, twelve how-to guides, six explanation pages and a reference of the command, configuration, environment, exit status and files.Thirteen code bugs are left as follow-ups:
usernsbackend runs EasyBuild as root, and EasyBuild refuses the build.run --robotis refused for a dependency that EasyBuild would not build.MODULEPATHgets the live module root, not itsalltree.run resumeis the wrong advice.EB_STAGED_REBUILD_FAIL_ATin every install.run --grace-hoursis written to the journal and never used.--softwareand--modulesreach EasyBuild unresolved.EB_STAGED_REBUILD_STATE_ROOTis the default state root ofrunonly.zig/main.zigcan never run.