Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,9 +71,9 @@ lint findings remain, and do not comment on the PR.
## Version-one CLI

```text
package inspect | plan | bump
package inspect | plan | bump | latest
recipe easyblock | templates | hierarchy | check | lint | format
stack solve | sbom | order | easystack | sections | from-commit | gap | provided | shadow | batch | shard | footprints | admit | arch | files | generation | cuda-cc | known-failures | oom | bot-classify
stack solve | sbom | order | easystack | sections | from-commit | gap | provided | shadow | batch | shard | capacity | footprints | admit | arch | files | generation | cuda-cc | known-failures | oom | bot-classify
target list | doctor
campaign run | status | finding claim | finding resolve
repro summary
Expand Down
1 change: 1 addition & 0 deletions changelog.d/+docs-drift.doc.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
The docs match the code again: `AGENTS.md` lists `package latest` and `stack capacity`; the `cli.org` MCP list names `eb_stack_footprints` and `eb_stack_admit`; `docs/campaign-boundary.md` cites the current `src/campaign.rs` and `src/foreign.rs` lines and the `run_target` calls; the shard how-to says to plan with EESSI-extend's `EASYBUILD_FILTER_DEPS`, or pass that list as `--filter-deps`; the PyPI and CRAN how-tos say the robot tree is `$EBROOTEASYBUILD/easybuild/easyconfigs` from EESSI's EasyBuild module; the commented pins in the EESSI example policies name the EESSI 2026.06 bundles; and the `provides` and `campaign` API docs have no broken link or stray `///`.
44 changes: 22 additions & 22 deletions docs/campaign-boundary.md
Original file line number Diff line number Diff line change
@@ -1,31 +1,31 @@
# Campaign Execution Boundary Documentation

*Generated from the current repository state (commit `1fd9fb568f60a5531527e82181ee18ce14b36c10`).*
*Line references checked against commit `72ea4e5534c176ba79898f466e7cfbd9e5ff9f19`.*

## 1. Direct Executions performed by `src/campaign.rs`

| Action | Command / Function | Conditions | File:Line Reference |
|--------|-------------------|------------|---------------------|
| Read campaign bundle manifest | `read_json(&manifest_path)` | Always after acquiring lock | `src/campaign.rs:268-269` |
| Extract package name & version from manifest | `manifest.pointer("/package/name")` and `manifest.pointer("/package/version")` | Must exist in manifest | `src/campaign.rs:270-277` |
| Load recipes list | `load_campaign_recipes(&request.bundle)` | Must return at least one recipe | `src/campaign.rs:278-283` |
| Discover Resolvo profile locks | `discover_files(&request.bundle.join("locks"), "json")` | Must find at least one lock file | `src/campaign.rs:284-288` |
| Initialise or load campaign state JSON | `read_json(&request.state_path)` or construct new `CampaignState` | If state file exists, schema must match | `src/campaign.rs:291-319` |
| Record interrupted previous attempt (if any) | `record_interrupted_attempt(&mut state, &request.state_path)` | When previous state was `Running` | `src/campaign.rs:321`, function at `src/campaign.rs:800` |
| Write updated state to disk (atomic) | `write_state(&request.state_path, &state)` | After each state transition | `src/campaign.rs:1377-1390` |
| Pre-flight packaging gate for each recipe | `packaging_gate(&resolved, &[])` | Runs before any build; on error creates a `BuildFinding` with class `Checksum` or `Unknown` | `src/campaign.rs:356` |
| Stage bundle on target | `request.target.stage_bundle(&request.bundle)` | May fail with transport error | `src/campaign.rs:402` |
| Build each recipe on target | `request.target.build_command_with_overlay(...)` then `command.execute()` | Executes EasyBuild command; on failure creates a `BuildFinding` | `src/campaign.rs:532` |
| Verify binaries for each profile | `request.target.verification_command(&program, &args)` then `command.execute()` | Runs verification commands; on failure creates a `BuildFinding` | `src/campaign.rs:629` |
| Finalise campaign state as `Completed` | Set `state.status = CampaignStatus::Completed` and write state | After all builds and verification succeed | `src/campaign.rs:700` |
| Read campaign bundle manifest | `read_json(&manifest_path)` | Always after acquiring lock | `src/campaign.rs:279-280` |
| Extract package name & version from manifest | `manifest.pointer("/package/name")` and `manifest.pointer("/package/version")` | Must exist in manifest | `src/campaign.rs:281-288` |
| Load recipes list | `load_campaign_recipes(&request.bundle)` | Must return at least one recipe | `src/campaign.rs:289-294` |
| Discover Resolvo profile locks | `discover_files(&request.bundle.join("locks"), "json")` | Must find at least one lock file | `src/campaign.rs:295-300` |
| Initialise or load campaign state JSON | `read_json(&request.state_path)` or construct new `CampaignState` | If state file exists, schema must match | `src/campaign.rs:302-332` |
| Record interrupted previous attempt (if any) | `record_interrupted_attempt(&mut state, &request.state_path)` | When previous state was `Running` | `src/campaign.rs:349`, function at `src/campaign.rs:844` |
| Write updated state to disk (atomic) | `write_state(&request.state_path, &state)` | After each state transition | `src/campaign.rs:1515-1529` |
| Pre-flight packaging gate for each recipe | `packaging_gate(&resolved, &[])` | Runs before any build; on error creates a `BuildFinding` with class `Checksum` or `Unknown` | `src/campaign.rs:384` |
| Stage bundle on target | `run_target.stage_bundle(&request.bundle)` | May fail with transport error | `src/campaign.rs:435` |
| Build each recipe on target | `run_target.build_command_with_overlay(...)` then `command.execute()` | Executes EasyBuild command; on failure creates a `BuildFinding` | `src/campaign.rs:571-576` |
| Verify binaries for each profile | `run_target.verification_command(&program, &args)` then `command.execute()` | Runs verification commands; on failure creates a `BuildFinding` | `src/campaign.rs:669-670` |
| Finalise campaign state as `Completed` | Set `state.status = CampaignStatus::Completed` and write state | After all builds and verification succeed | `src/campaign.rs:744` |

## 2. Files Written (but not executed) and Their Consumers

| File | Description | Who/What Consumes It |
|------|-------------|----------------------|
| `state_path` (e.g. `campaign.json`) | Persistent JSON representation of the campaign state, including history, findings, claims | Human operators, other campaign runs, debugging tools |
| `state_path.lock` and `state_path.lock.guard` | Exclusive lock metadata to prevent concurrent campaigns | `CampaignLock::acquire` ensures only one process owns the lock |
| Temporary `*.tmp` files during state writes | Atomic write-then-rename to avoid partial state files | The `write_state` helper (`src/campaign.rs:1377-1390`) |
| Temporary `*.tmp` files during state writes | Atomic write-then-rename to avoid partial state files | The `write_state` helper (`src/campaign.rs:1515-1529`) |
| `campaign.lock` metadata JSON (`*.lock`) | Records host, PID, start ticks for the lock holder | Lock acquisition / release logic (`CampaignLock::acquire` and `Drop`) |
| `campaign.findings` entries inside the state JSON | Records failures, interruptions, and their evidences | Human reviewers, `eb-stack campaign finding` commands |
| `campaign.history` entries inside the state JSON | Chronological log of actions (e.g., "build evaluation on <target>") | Auditing, debugging |
Expand All @@ -36,19 +36,19 @@

| Error Variant | Situation | Where Raised (File:Line) |
|---------------|-----------|--------------------------|
| `CampaignError::InvalidBundle` | Missing required fields in bundle (e.g., no recipes, no locks, missing manifest entries) | `src/campaign.rs:273-288` |
| `CampaignError::UnsupportedSchema` | State file schema version does not match `CAMPAIGN_SCHEMA_VERSION` | `src/campaign.rs:293-294` |
| `CampaignError::StateIdentity` | Existing state file refers to a different package/version than bundle | `src/campaign.rs:296-297` |
| `CampaignError::Busy` | Another process owns the campaign lock (`CampaignLock::acquire`) | `src/campaign.rs:1414-1416` (inside `CampaignLock::acquire`) |
| `CampaignError::Target` (wraps `TargetError`) | A target command that cannot be started is stored as a finding; `run_campaign` returns `Ok` with status `Failed` | Variant at `src/campaign.rs:1528`. Call sites `src/campaign.rs:402`, `444`, `537`, and `630` go through `record_target_command_failure` |
| `CampaignError::Io` / `Json` | File system or JSON (de)serialization failures | `read_json` at `src/campaign.rs:1371-1375` and `write_state` at `src/campaign.rs:1377-1390` |
| `CampaignError::FindingNotFound`, `FindingOwned`, `FindingState` | Operations on campaign findings that are invalid (e.g., claiming an already-owned finding) | `src/campaign.rs:843-876` (`claim_finding`) and `src/campaign.rs:881-910` (`resolve_finding`) |
| `CampaignError::InvalidBundle` | Missing required fields in bundle (e.g., no recipes, no locks, missing manifest entries) | `src/campaign.rs:284-299` |
| `CampaignError::UnsupportedSchema` | State file schema version does not match `CAMPAIGN_SCHEMA_VERSION` | `src/campaign.rs:304-305` |
| `CampaignError::StateIdentity` | Existing state file refers to a different package/version than bundle | `src/campaign.rs:307-308` |
| `CampaignError::Busy` | Another process owns the campaign lock (`CampaignLock::acquire`) | `src/campaign.rs:1553-1555` (inside `CampaignLock::acquire`) |
| `CampaignError::Target` (wraps `TargetError`) | A target command that cannot be started is stored as a finding; `run_campaign` returns `Ok` with status `Failed` | Variant at `src/campaign.rs:1668`. Call sites `src/campaign.rs:485`, `579`, and `673` go through `record_target_command_failure`; a staging failure at `src/campaign.rs:435` is recorded as a `Transport` finding in place |
| `CampaignError::Io` / `Json` | File system or JSON (de)serialization failures | `read_json` at `src/campaign.rs:1509-1513` and `write_state` at `src/campaign.rs:1515-1529` |
| `CampaignError::FindingNotFound`, `FindingOwned`, `FindingState` | Operations on campaign findings that are invalid (e.g., claiming an already-owned finding) | `src/campaign.rs:888-921` (`claim_finding`) and `src/campaign.rs:926-955` (`resolve_finding`) |

### In `src/foreign.rs`

| Error Variant | Situation | Where Raised (File:Line) |
|---------------|-----------|--------------------------|
| `ForeignError::Parse` | Recipe could not be parsed as YAML or Python abstract syntax tree (AST) | `src/foreign.rs:575-598` (conda parsing) and `src/foreign.rs:1418` (Spack parsing) |
| `ForeignError::Parse` | Recipe could not be parsed as YAML or Python abstract syntax tree (AST) | `src/foreign.rs:579-602` (conda parsing) and `src/foreign.rs:1474` (Spack parsing) |
| `ForeignError::Unsupported` | File extension/name not recognised as a supported foreign format | `src/foreign.rs:354` (`parse_foreign_path`; detection is `detect_foreign_format` at `src/foreign.rs:289`) |
| `ForeignError::Io` | Underlying file read error | `src/foreign.rs:351` (`parse_foreign_path`) |

Expand Down
5 changes: 4 additions & 1 deletion docs/orgmode/howto/cran-extend.org
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,10 @@ dependency on the EasyBuild ~R~ module. Packages already listed in
* Inputs

1. a DESCRIPTION file, ~cran.json~, or a one-spec-per-line package list;
2. the European Environment for Scientific Software Installations (EESSI) robot tree;
2. the European Environment for Scientific Software Installations (EESSI) robot tree,
passed as ~--easyconfigs~. It is the easyconfigs directory bundled with
EESSI's EasyBuild: ~$EBROOTEASYBUILD/easybuild/easyconfigs~ after
loading EESSI's ~EasyBuild~ module;
3. a stack policy that locks ~R~ and ~R-bundle-CRAN~ to the versions
in the robot. See ~examples/stacks/eessi-r-extras.toml~.

Expand Down
5 changes: 4 additions & 1 deletion docs/orgmode/howto/pypi-extend.org
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,10 @@ toolchain-built extensions and has no claim on the European Environment for Scie
* Inputs

1. Warehouse-shaped JSON (~info~ + ~urls~) or a ~requirements.txt~;
2. the EESSI robot tree (or a fixture that mirrors its provides);
2. the EESSI robot tree (or a fixture that mirrors its provides), passed
as ~--easyconfigs~. It is the easyconfigs directory bundled with EESSI's
EasyBuild: ~$EBROOTEASYBUILD/easybuild/easyconfigs~ after loading
EESSI's ~EasyBuild~ module;
3. a stack policy that *locks* ~Python~, ~SciPy-bundle~, and
~Python-bundle-PyPI~ to the versions in the robot.
See ~examples/stacks/eessi-python-extras.toml~.
Expand Down
8 changes: 8 additions & 0 deletions docs/orgmode/howto/stack-shard.org
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,14 @@ in that file. ~--weights~ is ~Name/VersionFull~ or the easyconfig
filename, a tab, then seconds. A module the file does not list weighs
~--default-weight~, which is 1. Build the prelude, then the shards.

On an EESSI build host, shard with the dependency filter the build
uses. EESSI-extend sets ~EASYBUILD_FILTER_DEPS~ to the packages EESSI
takes from its compatibility layer, such as binutils, zlib and XZ, and
~stack shard~ reads that variable when ~--filter-deps~ is absent. When
planning without that module loaded, pass the same comma-separated
list as ~--filter-deps~. Otherwise those packages count as missing, and
the ones several shards share land in the prelude.

* Lift CPU-only prerequisites of accelerator roots

EESSI installs accelerator packages under an ~/accel/~ prefix and
Expand Down
2 changes: 1 addition & 1 deletion docs/orgmode/reference/cli.org
Original file line number Diff line number Diff line change
Expand Up @@ -2230,7 +2230,7 @@ plus sandboxed recipe and easyblock tools:
- ~eb_stack_solve~, ~eb_stack_sbom~, ~eb_stack_order~,
~eb_stack_easystack~, ~eb_stack_gap~, ~eb_stack_provided~,
~eb_stack_shadow~, ~eb_stack_batch~, ~eb_stack_shard~, ~eb_stack_capacity~,
~eb_stack_arch~, ~eb_stack_files~,
~eb_stack_footprints~, ~eb_stack_admit~, ~eb_stack_arch~, ~eb_stack_files~,
~eb_stack_generation~, ~eb_stack_cuda_cc~, ~eb_stack_known_failures~,
~eb_stack_oom~, ~eb_stack_bot_classify~, ~eb_stack_sections~,
~eb_stack_from_commit~;
Expand Down
4 changes: 2 additions & 2 deletions examples/stacks/eessi-python-extras.toml
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,10 @@ version = "2026.1"

# [[pins]]
# name = "SciPy-bundle"
# version_requirement = "==2025.06"
# version_requirement = "==2026.05"
# mode = "locked"
#
# [[pins]]
# name = "Python-bundle-PyPI"
# version_requirement = "==2025.04"
# version_requirement = "==2026.04"
# mode = "locked"
4 changes: 2 additions & 2 deletions examples/stacks/eessi-r-extras.toml
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,10 @@ version = "2026.1"

# [[pins]]
# name = "R"
# version_requirement = "==4.4.2"
# version_requirement = "==4.6.1"
# mode = "locked"
#
# [[pins]]
# name = "R-bundle-CRAN"
# version_requirement = "==2025.06"
# version_requirement = "==2026.07"
# mode = "locked"
20 changes: 17 additions & 3 deletions llms-full.txt
Original file line number Diff line number Diff line change
Expand Up @@ -1179,7 +1179,10 @@ toolchain-built extensions and has no claim on the European Environment for Scie
# Inputs

1. Warehouse-shaped JSON (`info` + `urls`) or a `requirements.txt`;
2. the EESSI robot tree (or a fixture that mirrors its provides);
2. the EESSI robot tree (or a fixture that mirrors its provides), passed
as `--easyconfigs`. It is the easyconfigs directory bundled with EESSI's
EasyBuild: `$EBROOTEASYBUILD/easybuild/easyconfigs` after loading
EESSI's `EasyBuild` module;
3. a stack policy that **locks** `Python`, `SciPy-bundle`, and
`Python-bundle-PyPI` to the versions in the robot.
See `examples/stacks/eessi-python-extras.toml`.
Expand Down Expand Up @@ -1339,7 +1342,10 @@ dependency on the EasyBuild `R` module. Packages already listed in
# Inputs

1. a DESCRIPTION file, `cran.json`, or a one-spec-per-line package list;
2. the European Environment for Scientific Software Installations (EESSI) robot tree;
2. the European Environment for Scientific Software Installations (EESSI) robot tree,
passed as `--easyconfigs`. It is the easyconfigs directory bundled with
EESSI's EasyBuild: `$EBROOTEASYBUILD/easybuild/easyconfigs` after
loading EESSI's `EasyBuild` module;
3. a stack policy that locks `R` and `R-bundle-CRAN` to the versions
in the robot. See `examples/stacks/eessi-r-extras.toml`.

Expand Down Expand Up @@ -5834,6 +5840,14 @@ in that file. `--weights` is `Name/VersionFull` or the easyconfig
filename, a tab, then seconds. A module the file does not list weighs
`--default-weight`, which is 1. Build the prelude, then the shards.

On an EESSI build host, shard with the dependency filter the build
uses. EESSI-extend sets `EASYBUILD_FILTER_DEPS` to the packages EESSI
takes from its compatibility layer, such as binutils, zlib and XZ, and
`stack shard` reads that variable when `--filter-deps` is absent. When
planning without that module loaded, pass the same comma-separated
list as `--filter-deps`. Otherwise those packages count as missing, and
the ones several shards share land in the prelude.

# Lift CPU-only prerequisites of accelerator roots

EESSI installs accelerator packages under an `/accel/` prefix and
Expand Down Expand Up @@ -8889,7 +8903,7 @@ plus sandboxed recipe and easyblock tools:
- `eb_stack_solve`, `eb_stack_sbom`, `eb_stack_order`,
`eb_stack_easystack`, `eb_stack_gap`, `eb_stack_provided`,
`eb_stack_shadow`, `eb_stack_batch`, `eb_stack_shard`, `eb_stack_capacity`,
`eb_stack_arch`, `eb_stack_files`,
`eb_stack_footprints`, `eb_stack_admit`, `eb_stack_arch`, `eb_stack_files`,
`eb_stack_generation`, `eb_stack_cuda_cc`, `eb_stack_known_failures`,
`eb_stack_oom`, `eb_stack_bot_classify`, `eb_stack_sections`,
`eb_stack_from_commit`;
Expand Down
4 changes: 2 additions & 2 deletions src/campaign.rs
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ pub enum BuildFindingClass {
/// The job exceeded its time limit.
Timeout,
/// The output did not match any known signature. Left unclassified
/// /// rather than guessed at, so it reaches a human.
/// rather than guessed at, so it reaches a human.
Unknown,
}

Expand Down Expand Up @@ -155,7 +155,7 @@ pub struct BuildFinding {
/// The command that failed, so it can be rerun by hand.
pub command: CommandPlan,
/// Exit status, when the process produced one. Absent when it was
/// /// killed or never started.
/// killed or never started.
pub exit_code: Option<i32>,
/// Attempt number this arose on.
pub attempt: u32,
Expand Down
2 changes: 1 addition & 1 deletion src/provides.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
//! `R-bundle-CRAN`.
//!
//! Expansion is idempotent. Synthetic candidates are marked by
//! [`crate::domain::Candidate::EXT_PROVIDE_MARKER`] in `easyconfig_path` and
//! [`EXT_PROVIDE_MARKER`] in `easyconfig_path` and
//! depend on the parent bundle at its exact version.

use crate::domain::{Candidate, DepReq, ExtEntry};
Expand Down
Loading