Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.d/+hardware-schema-text.bugfix.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
`eb-pipeline schema` says which commands get the hardware class of `[targets.cpu_hardware]`: the source prefetch and build commands of each line, which build for one class of node, as they always have. It said that every command did, and did not say that `[targets.hardware]` is exported only when `cpu_hardware` is set.
1 change: 1 addition & 0 deletions changelog.d/+module-names-as-written.bugfix.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
The module wrapper gives each name of `modules.init` to `module load` as written. It split `EB_PIPELINE_MODULES` with pathname expansion on, so a name with a `*`, `?` or `[` that matched files, in the directory the command started in or under an absolute path, loaded the names of those files instead. `modules.shell` is documented as a POSIX shell, such as `bash` or `sh`, which the wrapper has always needed.
1 change: 1 addition & 0 deletions changelog.d/+shares-placeholders-text.bugfix.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
`eb-pipeline schema` lists all five placeholders that `targets.route.shares_ro` and `shares_rw` take: `{phase_dir}`, `{log_dir}`, `{work_dir}`, `{target}` and `{phase}`. It left out `{target}` and `{phase}`, which `check` accepts and a run replaces in the shares as it does in the route's `args`.
1 change: 1 addition & 0 deletions changelog.d/+site-file-variable.removal.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
The commands of a run get the site file in `EB_PIPELINE_SITE`, as an absolute path, and the site name in the new `EB_PIPELINE_SITE_NAME`; `explain` and the plan show both. `EB_PIPELINE_SITE` held the name, so an `eb-pipeline` that a command started without `--site` tried to read a file of that name and exited 2. A command that reads the site name from `EB_PIPELINE_SITE` must read `EB_PIPELINE_SITE_NAME` instead. In the library, `resolve::explain` takes the site file as its second argument and `Scope` has a `site_file` field.
11 changes: 8 additions & 3 deletions docs/orgmode/howto/write-a-site-file.org
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,8 @@ For a site on top of the European Environment for Scientific Software Installati
EB_PIPELINE_LOG_STAMP = "<log-stamp>"
EB_PIPELINE_MODULES = "EasyBuild"
EB_PIPELINE_RUN_ID = "<run-id>"
EB_PIPELINE_SITE = "example"
EB_PIPELINE_SITE = "$WORK/site.toml"
EB_PIPELINE_SITE_NAME = "example"
EB_PIPELINE_TARGET = "zen4"

[targets.phases.build]
Expand All @@ -99,6 +100,8 @@ For a site on top of the European Environment for Scientific Software Installati
...
#+end_example

=EB_PIPELINE_SITE= is the site file as an absolute path, here with =$WORK= for the directory that holds it, so that a command can run eb-pipeline on the same file.

5. To change a value for one run without editing the file, set it in a variable named =EB_PIPELINE__SECTION__KEY=, here a node-local build directory:

#+begin_src bash
Expand Down Expand Up @@ -129,8 +132,10 @@ eb-pipeline run --site site.toml --list lists/zen4.list --target zen4 --build-ty
set -e
if [ -n "${EB_PIPELINE_UNUSE_PATH:-}" ]; then MODULEPATH=; export MODULEPATH; fi
module purge
for m in ${EB_PIPELINE_MODULES:-}; do module load "$m"; done
exec "$@"' eb-pipeline eb Example-1.0-GCCcore-15.2.0.eb --tmp-logdir=/opt/site/logs/20261117_032712/zen4/easybuild-logs/Example-1.0-GCCcore-15.2.0
set -f
for m in ${EB_PIPELINE_MODULES:-}; do set +f; module load "$m"; done
set +f
exec "$@"' eb-pipeline eb Example-1.0-GCCcore-15.2.0.eb --tmp-logdir=/opt/site/logs/20261117_043420/zen4/easybuild-logs/Example-1.0-GCCcore-15.2.0
...
# [caches] not configured
[logs] TZ=UTC srun --job-name=example-production-logs -p cpu -N 1 --ntasks-per-node=1 -c 1 -t 15:00 eb-keep-logs rotate /opt/site/logs --compress-after 30 --delete-after 365 --lock-timeout 0
Expand Down
10 changes: 8 additions & 2 deletions docs/orgmode/reference/environment.org
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,8 @@ The keys of =[scheduler.env]= come first, and a variable of the table below with
| =EASYBUILD_CUDA_COMPUTE_CAPABILITIES= | the target's =cuda_compute_capabilities= | it is set |
| =EASYBUILD_READ_ONLY_INSTALLDIR= | =filesystems.read_only_installdir_value=, =1= by default | =filesystems.read_only_installdir= is =true= |
| =EASYBUILD_CONFIGS_REPOPATH= | =filesystems.configs_repo= | it is set |
| =EB_PIPELINE_SITE= | =site.name= | always |
| =EB_PIPELINE_SITE= | the site file, as an absolute path | always |
| =EB_PIPELINE_SITE_NAME= | =site.name= | always |
| =EB_PIPELINE_TARGET= | the target's name | always |
| =EB_PIPELINE_BUILD_TYPE= | the build type | always |
| =EB_PIPELINE_RUN_ID= | the run id | always |
Expand All @@ -89,6 +90,9 @@ The keys of =[scheduler.env]= come first, and a variable of the table below with
| the one named by =modules.eessi.accel_override_var=, =EESSI_ACCELERATOR_TARGET_OVERRIDE= by default | the target's =accel_target= | =[modules.eessi]= is on and =accel_target= is set |
| =EESSI_SITE_SOFTWARE_PREFIX= | the target's =site_prefix=, else =filesystems.install_root= | =[modules.eessi]= is on |

=EB_PIPELINE_SITE=, =EB_PIPELINE_TARGET=, =EB_PIPELINE_BUILD_TYPE=, =EB_PIPELINE_RUN_ID= and =EB_PIPELINE_LOG_STAMP= are also the variables of the options above, so a command that runs =eb-pipeline= without those options gets the site file, target, build type, run id and log stamp of the run.
=eb-pipeline= does not read =EB_PIPELINE_SITE_NAME=; =EB_PIPELINE__SITE__NAME=, with two underscores, is the variable that sets =name= in the site file.

=[modules.eessi]= is on when the site file has the table and its =enabled= is not =false=.
=[licences]= is on when its =enabled= and =enforce= are not =false=.
=EESSI= stands for the European Environment for Scientific Software Installations.
Expand All @@ -106,7 +110,7 @@ Some commands get more variables, set over those above.
| =EB_PIPELINE_HW_VENDOR_ID=, =EB_PIPELINE_HW_CPU_FAMILY=, =EB_PIPELINE_HW_MODEL= | the =vendor_id=, =cpu_family= and =model= of the target's =cpu_hardware= for a CPU-role unit, of its =hardware= otherwise | the prefetch and build commands, when the target has a =cpu_hardware= table |
| the keys of =targets.route.env= | their values, with the placeholders of the route replaced | the commands that the route wraps |
| =EB_PIPELINE_ROUTE_RUNTIME_ROOT= | =targets.route.runtime_root= | the same, when it is set |
| =EB_PIPELINE_ROUTE_SHARES_RO=, =EB_PIPELINE_ROUTE_SHARES_RW= | the entries of =targets.route.shares_ro= and =shares_rw=, joined with =:= | the same, when they are not empty |
| =EB_PIPELINE_ROUTE_SHARES_RO=, =EB_PIPELINE_ROUTE_SHARES_RW= | the entries of =targets.route.shares_ro= and =shares_rw=, with their placeholders replaced, joined with =:= | the same, when they are not empty |
| =EB_PIPELINE_ROUTE_RECEIPT_DIGEST= | the value of the variable that =targets.route.receipt_digest_env= names | the same, when it is set |
| =EASYBUILD_OUTPUT_STYLE= | =basic=, whatever =[scheduler.env]= sets, so that =eb --last-log= prints the log path on one line | =eb-keep-logs failed=, after a line fails |
| =TZ= | =UTC=, so that =eb-keep-logs rotate= counts the age of a run from the midnight of its log stamp | =eb-keep-logs rotate= |
Expand All @@ -121,7 +125,9 @@ When =modules.init= is not empty, =modules.unuse_path= is =true= or =modules.she
1. =EB_PIPELINE_SHELL_INIT=: the file it sources first, when set.
2. =EB_PIPELINE_UNUSE_PATH=: when set, it empties =MODULEPATH=.
3. =EB_PIPELINE_MODULES=: it runs =module purge=, then =module load= for each name in it, in order, then the command.
It splits the variable at white space and gives each name to =module load= as written: a name with a pattern in it, such as =GCC/*=, is not matched against file names.

The wrapper is a script for a POSIX shell, such as =bash= or =sh=.
The admit and rotate commands start without it.

** See also
Expand Down
3 changes: 2 additions & 1 deletion docs/orgmode/reference/plan-and-result.org
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,8 @@ cat plan.json
"EB_PIPELINE_LOG_DIR": "$WORK/logs/20261117_090000/cpu",
"EB_PIPELINE_LOG_STAMP": "20261117_090000",
"EB_PIPELINE_RUN_ID": "nightly",
"EB_PIPELINE_SITE": "example",
"EB_PIPELINE_SITE": "$WORK/site.toml",
"EB_PIPELINE_SITE_NAME": "example",
"EB_PIPELINE_TARGET": "cpu"
},
"name": "build",
Expand Down
Loading
Loading