A site-neutral EasyBuild pipeline driver. One per-site Tom's Obvious Minimal
Language (TOML) file describes targets, scheduler allocations, paths,
admission, modules, caches, log handling and licences. eb-pipeline turns that
file and a build list into an ordered set of subprocesses, so a continuous
integration (CI) system (Jenkins, GitLab CI, a cron job) becomes a thin caller
that passes a list and a target and reads an exit code and a result file.
It starts eb-stack for stack shard
and stack admit, eb-keep-logs
for failed and rotate, and
eb-staged-rebuild for run,
and never links them in. Nothing about a site lives in this repository: the
templates carry placeholders only, and a test rejects any template value that
looks site-specific.
https://stackhands.github.io/eb-pipeline/, built from the Org pages in
docs/orgmode/:
- Tutorial: Run your first build list.
- How-to guides: install eb-pipeline, write a site file for your cluster, build a list on every target from continuous integration, let users load what a target installs on top of EESSI, rotate the log tree, rebuild installed software through eb-staged-rebuild, build some lines of a list on some targets only.
- Explanation: why one site file drives the build, how a run goes through its phases, why the rotation takes two locks, what the targets of a site file share.
- Reference: the eb-pipeline command, the site file, the plan and result files, the log tree and its locks, the site file templates, environment variables, exit status, and the API of the crate, from rustdoc.
pixi run -e docs docbldThe HTML is written to docs/build/html.
Contributing says how to write a page, run
the tests and record a change.
Rust 1.90 or newer, in a checkout:
cargo install --locked --path . --root ~/.local
eb-pipeline --versionInstall eb-pipeline has the release binaries and what to do when the build fails.
MIT. See LICENSE.
eb-pipeline is one of the stackhands tools for building, installing and maintaining an HPC software stack with EasyBuild on top of EESSI.
| Tool | Job | Docs |
|---|---|---|
| eb-pipeline | Run a site's EasyBuild build lists from one TOML file through eb-stack, eb-keep-logs and eb-staged-rebuild | README |
| eb-stack | Solve, shard and plan an EasyBuild build list; admit builds under a scratch inode budget | docs |
| eb-staged-rebuild | Rebuild an installation in a staged tree and swap it in, so running jobs keep their paths | docs |
| eessi-arch-guard | Reload EESSI in a Slurm task that inherited it for another CPU or GPU | docs |
| eb-local | Install software with EasyBuild on top of EESSI into a directory a user or project owns | docs |
| eb-keep-logs | Keep a failed build's EasyBuild log and command output; rotate a pipeline's log tree | docs |
| rfc9162 | Append-only Merkle logs hashed as RFC 9162, with tiled storage and signed checkpoints | docs |
| zig-common | The shared Zig library under the Zig tools: system calls, checked files, child processes | docs |