Repository navigation
docs: Diátaxis site for eb-pipeline - #28
Merged
Merged
Conversation
The documentation was the README, as one page. The stackhands documentation template now builds a site, as it does for eessi-arch-guard: Org pages under docs/orgmode/, exported to reStructuredText by docs/export.el and built by Sphinx with the shibuya theme and the stackhands menu, with warnings and unresolved names as errors. check_site.py checks the internal links and anchors, and that no page names a site; lychee checks the external links. The API is the rustdoc site of the eb_pipeline library, from cargo doc, served beside the pages at api/, with a front page that cargo doc does not write. pixi.lock pins the tools, among them Rust 1.90, the version the msrv job builds with. The docs workflow builds the site for every pull request, checks its external links, also weekly, and publishes it from main to GitHub Pages. The pages so far are the front page, the API page and the contributing page, which says how to record a change, run the tests and build the site.
The fields of the site file structs take the text the schema reference prints for their keys, and the API build now fails on a public item without a doc comment.
docs/examples/tutorial.sh runs the tutorial's commands in order and checks the output the page shows, and a docs job in the ci workflow runs it. The landing page links the tutorial and spells out TOML and CI.
The CLI page gives the usage line, options, variables and exit statuses of every subcommand, with the help text docs/examples/cli.sh prints. The environment page lists the option variables, the EB_PIPELINE__ site file values, the variables a run reads by the names the site file gives, and those it sets for the commands it starts.
What leads to each exit status, the order run picks one in when several apply, and what a run does with the exit status of each tool it starts.
One table per section of the site file, with the type, default and
meaning of every key, and sections on placeholders, {target} in paths,
log rotation and the lines that go through [rebuild]. tests/docs.rs
fails when a table and schema::SECTIONS differ in a key, its order,
its type or its default.
One table per object of each file, the statuses of a line and of a phase, and a real plan and result from docs/examples/files.sh, which the docs job of the CI workflow runs.
The files of a log directory and of the log root, the four locks a run meets, and the order in which the two rotation locks are taken.
Both templates in full, included from templates/, with the names init --list prints and what check and targets say about each; the docs job runs docs/examples/templates.sh.
Each command is in a bash block of its own, followed by what it printed, as the other pages show them.
From the 0.3.0 release, checked against its checksum, or from source with cargo install, with what the release lacks and the failures each way meets. docs/examples/install.sh needs the network, so the docs job does not run it; the contributing page says how to.
From the small-cluster template to a checked site file for one node type, read back with explain and a dry run, with the messages of the common mistakes. docs/examples/site-file.sh runs the committed file, and the docs job runs the script.
The GitHub Actions workflow of examples/ci/, why each target has a run id of its own, when to raise max-parallel and what the concurrency group holds, and a caller script for other CI systems that exits as a run does, which docs/examples/ci.sh runs.
At the end of each run and from cron, under eb-pipeline's rotation lock with flock, with the checks of each and the messages of a run that leaves the rotation to another. docs/examples/rotation.sh holds the locks the way other runs and a rotation do.
The [rebuild] table with a state root per target, what makes a line go through eb-staged-rebuild, the run id each staged line passes, and the messages of a refused or pending line and of the site files that check refuses, from docs/examples/staged.sh.
A [targets.select] table in each target of one build list, what each target plans and runs of it, and where the lines it keeps and leaves out are written, from docs/examples/select.sh.
The title of the guide is the text of every link to it, so the abbreviation reached pages that never spell it out.
What goes wrong when a rotation runs beside a live run or another rotation, how eb-pipeline's lock and eb-keep-logs' lock keep them apart, the order both are taken in, and the limits of flock, with a Graphviz figure of who takes which lock and the script that renders it.
The build directory, the download directory, the state root and the
run id that two targets share when their paths have no {target}, what
each costs a build that runs beside another, what is safe to share, and
what the concurrency group of the example workflow holds.
Why everything about a site lives in one file and the caller stays thin, and how a run goes through its phases and units, what stops it and how one exit status sums up its lines, with a figure of the phases. The tutorial links both pages from what just happened.
Its sections, log rotation included, live in the tutorial, the how-to guides, the explanation and the reference. The tests that read the README read those pages, and the command reference must show --help as the binary prints it. The example workflow names the guide that has the details.
The tutorial starts with cargo --version, which no script ran. The script keeps the toolchain of the user who runs it, as the install script does, so RUSTUP_TOOLCHAIN=1.90.0 prints the version on the page.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The documentation is a Diátaxis site built from Org pages under
docs/orgmode/, exported and built by Sphinx, with the rustdoc API of the library atapi/. It has one tutorial, six how-to guides, four explanation pages and a reference of the command, the site file, the exit codes, and the log tree and its locks. Adocsjob builds the site, andcheck_site.pychecks the internal links.The README is a short introduction with a link to every page. The site link, https://stackhands.github.io/eb-pipeline/, needs GitHub Pages enabled for stackhands/eb-pipeline, with GitHub Actions as the source; until then the Org sources work.
Four possible code bugs are left as follow-ups:
EB_PIPELINE_SITEis the path of the site file on the command line, andScope::envsets it tosite.namefor every command a run starts, so a nestedeb-pipelinecall without--sitereads the site's name as a path.targets.cpu_hardwaresays that every command exports its class asEB_PIPELINE_HW_*; only the source prefetch and build commands do.targets.route.shares_rolists three placeholders; validation and rendering also take{target}and{phase}. The pages list all five.${EB_PIPELINE_MODULES:-}.