Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.d/+component-reach-generation.bugfix.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
`component-reach` resolves the closure over every recipe in the trees, so dependents on `gfbf` and `gompi` toolchains of the generation are in it. On easybuild-easyconfigs develop with the EESSI 2026.06 easystack, curl's closure gains 19 dependents on `gfbf-2026.1` and `gompi-2026.1`, such as R, netCDF, HDF5 and SciPy-bundle, and 12 recipes only they pull in, such as Tk through CMake, so the counts go from `review=106 no-rebuild=16` to `review=133 no-rebuild=20`. Requirements no recipe resolves are listed in `closure_unresolved` instead of being dropped.
10 changes: 9 additions & 1 deletion docs/orgmode/howto/component-reach.org
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ eb-stack component-reach \
The command prints:

#+begin_example
component=curl version=>=8,<9 generation=2026.1 reach=network-facing carriers=23 changed=2 must-rebuild=1 review=106 no-rebuild=16 out=/tmp/component-reach-full
component=curl version=>=8,<9 generation=2026.1 reach=network-facing carriers=23 changed=2 must-rebuild=1 review=133 no-rebuild=20 out=/tmp/component-reach-full
#+end_example

The changed files are ~cURL-8.17.0-GCCcore-15.2.0.eb~ and ~cargo-c-0.10.23-GCCcore-15.2.0.eb~.
Expand Down Expand Up @@ -165,6 +165,14 @@ generation. Classes are the same as ~rebuild-set~: ~must-rebuild~,
~review~, and ~no-rebuild~. This command does not read a prefix or scan
static symbols.

Dependencies resolve over every recipe in the trees, so a recipe on
~gfbf-2026.1~ or ~gompi-2026.1~ reaches the ~GCCcore-15.2.0~ cURL through
the SYSTEM definition of its toolchain. A dependency that no recipe
resolves is listed in ~closure_unresolved~ in the JSON and TOML files, as
the module and the requirement, and the closure has no edge for it.
Without ~--filter-deps~, a stack that takes ~zlib~ or ~binutils~ from its
compatibility layer lists them for every recipe that names them.

* Read the three files

~out/component-reach.md~ is the carrier table and the closure table.
Expand Down
4 changes: 4 additions & 0 deletions docs/orgmode/reference/cli.org
Original file line number Diff line number Diff line change
Expand Up @@ -1891,6 +1891,10 @@ With no ~--easystack~, every easyconfig in the generation is the stack.
With no ~--installed~, every module in the generation is treated as
installed. Pass both to visit only a mounted prefix. The closure classes
are the same as ~rebuild-set~, and they stay inside the generation.
Dependencies resolve over every recipe in the trees, so the SYSTEM
definitions a ~gfbf~ or ~gompi~ hierarchy is read from count. A dependency
that no recipe resolves is listed in ~closure_unresolved~, and the closure
has no edge for it.

The output directory contains ~component-reach.toml~, ~component-reach.json~,
and ~component-reach.md~. The TOML file starts with
Expand Down
1 change: 1 addition & 0 deletions docs/orgmode/reference/formats.org
Original file line number Diff line number Diff line change
Expand Up @@ -715,6 +715,7 @@ One run writes ~component-reach.toml~, ~component-reach.json~, and
| ~closure_counts~ | Row counts, keys ~no-rebuild~, ~review~, ~must-rebuild~ |
| ~closure_rows~ | Rebuild-set rows for ~changed~ |
| ~missing~ | Changed filenames absent from the closure graph |
| ~closure_unresolved~ | Requirements no recipe resolves, so the closure has no edge for them. Omitted when empty |
| ~notes~ | A compat-layer hit, or a note the closure recorded |

~version_reqs~, ~versions~, ~purl~, and ~ecosystem~ are omitted when
Expand Down
15 changes: 14 additions & 1 deletion llms-full.txt
Original file line number Diff line number Diff line change
Expand Up @@ -3623,7 +3623,7 @@ eb-stack component-reach \
The command prints:

```
component=curl version=>=8,<9 generation=2026.1 reach=network-facing carriers=23 changed=2 must-rebuild=1 review=106 no-rebuild=16 out=/tmp/component-reach-full
component=curl version=>=8,<9 generation=2026.1 reach=network-facing carriers=23 changed=2 must-rebuild=1 review=133 no-rebuild=20 out=/tmp/component-reach-full
```

The changed files are `cURL-8.17.0-GCCcore-15.2.0.eb` and `cargo-c-0.10.23-GCCcore-15.2.0.eb`.
Expand Down Expand Up @@ -3738,6 +3738,14 @@ generation. Classes are the same as `rebuild-set`: `must-rebuild`,
`review`, and `no-rebuild`. This command does not read a prefix or scan
static symbols.

Dependencies resolve over every recipe in the trees, so a recipe on
`gfbf-2026.1` or `gompi-2026.1` reaches the `GCCcore-15.2.0` cURL through
the SYSTEM definition of its toolchain. A dependency that no recipe
resolves is listed in `closure_unresolved` in the JSON and TOML files, as
the module and the requirement, and the closure has no edge for it.
Without `--filter-deps`, a stack that takes `zlib` or `binutils` from its
compatibility layer lists them for every recipe that names them.

# Read the three files

`out/component-reach.md` is the carrier table and the closure table.
Expand Down Expand Up @@ -8550,6 +8558,10 @@ With no `--easystack`, every easyconfig in the generation is the stack.
With no `--installed`, every module in the generation is treated as
installed. Pass both to visit only a mounted prefix. The closure classes
are the same as `rebuild-set`, and they stay inside the generation.
Dependencies resolve over every recipe in the trees, so the SYSTEM
definitions a `gfbf` or `gompi` hierarchy is read from count. A dependency
that no recipe resolves is listed in `closure_unresolved`, and the closure
has no edge for it.

The output directory contains `component-reach.toml`, `component-reach.json`,
and `component-reach.md`. The TOML file starts with
Expand Down Expand Up @@ -11047,6 +11059,7 @@ One run writes `component-reach.toml`, `component-reach.json`, and
| `closure_counts` | Row counts, keys `no-rebuild`, `review`, `must-rebuild` |
| `closure_rows` | Rebuild-set rows for `changed` |
| `missing` | Changed filenames absent from the closure graph |
| `closure_unresolved` | Requirements no recipe resolves, so the closure has no edge for them. Omitted when empty |
| `notes` | A compat-layer hit, or a note the closure recorded |

`version_reqs`, `versions`, `purl`, and `ecosystem` are omitted when
Expand Down
40 changes: 32 additions & 8 deletions src/component_reach.rs
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,10 @@ pub struct ReachDocument {
pub closure_rows: Vec<RebuildRow>,
/// Changed filenames missing from the closure graph.
pub missing: Vec<String>,
/// Requirements no recipe in the trees resolves, so the closure has no
/// edge for them.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub closure_unresolved: Vec<String>,
/// Compat-layer hits and closure notes.
pub notes: Vec<String>,
}
Expand Down Expand Up @@ -206,7 +210,8 @@ pub fn component_reach(query: &ReachQuery<'_>) -> ReachDocument {
});
carriers.dedup();
let changed: Vec<String> = changed.into_iter().collect();
let (mut closure_counts, closure_rows, missing, extra) = close(query, &changed);
let (mut closure_counts, closure_rows, missing, closure_unresolved, extra) =
close(query, &changed);
if matches!(model, PurlMatch::Unsupported) {
closure_counts.review = 1;
}
Expand Down Expand Up @@ -238,6 +243,7 @@ pub fn component_reach(query: &ReachQuery<'_>) -> ReachDocument {
closure_counts,
closure_rows,
missing,
closure_unresolved,
notes,
}
}
Expand Down Expand Up @@ -472,17 +478,23 @@ pub fn fetch_easystack_urls(
Ok(out)
}

fn close(
query: &ReachQuery<'_>,
changed: &[String],
) -> (RebuildCounts, Vec<RebuildRow>, Vec<String>, Vec<String>) {
/// Counts, rows, missing filenames, unresolved requirements, and notes.
type Closure = (
RebuildCounts,
Vec<RebuildRow>,
Vec<String>,
Vec<String>,
Vec<String>,
);

fn close(query: &ReachQuery<'_>, changed: &[String]) -> Closure {
let empty = RebuildCounts {
no_rebuild: 0,
review: 0,
must_rebuild: 0,
};
if changed.is_empty() {
return (empty, Vec::new(), Vec::new(), Vec::new());
return (empty, Vec::new(), Vec::new(), Vec::new(), Vec::new());
}
let owned: Vec<Candidate> = query
.candidates
Expand Down Expand Up @@ -518,7 +530,12 @@ fn close(
};
let installed = query.installed.cloned().unwrap_or(modules);
match rebuild_set(&RebuildQuery {
base: owned,
// The whole tree, not only the generation: a gfbf or gompi hierarchy
// is read off its definition recipe, which is a SYSTEM recipe, and
// without it no unpinned dependency of a recipe on that toolchain
// resolves. The roots and the installed set keep the rows in the
// generation.
base: query.candidates.to_vec(),
commits: Default::default(),
easystacks: easystacks.to_vec(),
changed: changed.to_vec(),
Expand All @@ -534,9 +551,16 @@ fn close(
document.counts,
document.rows,
document.missing,
document.unresolved,
document.notes,
),
Err(error) => (empty, Vec::new(), Vec::new(), vec![error.to_string()]),
Err(error) => (
empty,
Vec::new(),
Vec::new(),
Vec::new(),
vec![error.to_string()],
),
}
}

Expand Down
121 changes: 121 additions & 0 deletions tests/component_reach.rs
Original file line number Diff line number Diff line change
Expand Up @@ -254,3 +254,124 @@ fn an_unsupported_purl_type_creates_no_output_directory() {
);
assert!(!out_dir.exists(), "left {}", out_dir.display());
}

/// Run `component-reach` for curl 8 on a tree of `(file, text)` recipes.
fn reach_curl(files: &[(&str, &str)]) -> serde_json::Value {
let dir = tempfile::tempdir().expect("temp");
let tree = dir.path().join("easyconfigs");
std::fs::create_dir_all(&tree).expect("tree");
for (file, text) in files {
std::fs::write(tree.join(file), text).expect("recipe");
}
let out = dir.path().join("out");
let output = Command::new(env!("CARGO_BIN_EXE_eb-stack"))
.env_remove("EASYBUILD_FILTER_DEPS")
.args([
"component-reach",
"--component",
"curl",
"--version",
">=8,<9",
"--generation",
"2026.1",
"--easyconfigs",
])
.arg(&tree)
.args(["--generated-at", "2026-10-22T12:00:00Z", "--out-dir"])
.arg(&out)
.output()
.expect("run");
assert!(
output.status.success(),
"{}",
String::from_utf8_lossy(&output.stderr)
);
serde_json::from_str(&std::fs::read_to_string(out.join("component-reach.json")).unwrap())
.unwrap()
}

const CURL: (&str, &str) = (
"cURL-8.17.0-GCCcore-15.2.0.eb",
"name = 'cURL'\nversion = '8.17.0'\ntoolchain = {'name': 'GCCcore', 'version': '15.2.0'}\n",
);
const R_ON_GFBF: (&str, &str) = (
"R-4.6.1-gfbf-2026.1.eb",
"name = 'R'\nversion = '4.6.1'\ntoolchain = {'name': 'gfbf', 'version': '2026.1'}\ndependencies = [('cURL', '8.17.0')]\n",
);
const NETCDF_ON_GOMPI: (&str, &str) = (
"netCDF-4.10.0-gompi-2026.1.eb",
"name = 'netCDF'\nversion = '4.10.0'\ntoolchain = {'name': 'gompi', 'version': '2026.1'}\ndependencies = [('cURL', '8.17.0')]\n",
);
const GDAL_ON_FOSS: (&str, &str) = (
"GDAL-3.13.0-foss-2026.1.eb",
"name = 'GDAL'\nversion = '3.13.0'\ntoolchain = {'name': 'foss', 'version': '2026.1'}\ndependencies = [('cURL', '8.17.0')]\n",
);

fn closure_dependents(json: &serde_json::Value) -> Vec<String> {
let mut dependents: Vec<String> = json["closure_rows"]
.as_array()
.expect("closure_rows")
.iter()
.map(|row| {
format!(
"{} {}",
row["dependent"].as_str().unwrap(),
row["edge_kind"].as_str().unwrap()
)
})
.collect();
dependents.sort();
dependents
}

/// gfbf and gompi have no built-in hierarchy; it is derived from their
/// definition recipes, which are SYSTEM recipes outside the generation.
#[test]
fn dependents_on_every_toolchain_of_the_generation_are_in_the_closure() {
let json = reach_curl(&[
CURL,
R_ON_GFBF,
NETCDF_ON_GOMPI,
GDAL_ON_FOSS,
(
"gfbf-2026.1.eb",
"name = 'gfbf'\nversion = '2026.1'\ntoolchain = SYSTEM\ndependencies = [('GCC', '15.2.0'), ('FlexiBLAS', '3.5.0', '', ('GCC', '15.2.0'))]\n",
),
(
"gompi-2026.1.eb",
"name = 'gompi'\nversion = '2026.1'\ntoolchain = SYSTEM\ndependencies = [('GCC', '15.2.0'), ('OpenMPI', '5.0.10', '', ('GCC', '15.2.0'))]\n",
),
]);
assert_eq!(
closure_dependents(&json),
[
"GDAL/3.13.0-foss-2026.1 runtime",
"R/4.6.1-gfbf-2026.1 runtime",
"cURL/8.17.0-GCCcore-15.2.0 changed",
"netCDF/4.10.0-gompi-2026.1 runtime",
],
"{json:#}"
);
}

/// Without the gfbf and gompi definitions their recipes cannot resolve cURL.
/// The closure then lacks them, and the report has to say so.
#[test]
fn a_dependency_the_closure_cannot_resolve_is_listed() {
let json = reach_curl(&[CURL, R_ON_GFBF, NETCDF_ON_GOMPI, GDAL_ON_FOSS]);
let unresolved: Vec<&str> = json["closure_unresolved"]
.as_array()
.expect("closure_unresolved")
.iter()
.filter_map(|requirement| requirement.as_str())
.collect();
for requirement in [
"R/4.6.1-gfbf-2026.1 needs cURL ==8.17.0",
"netCDF/4.10.0-gompi-2026.1 needs cURL ==8.17.0",
] {
assert!(
unresolved.contains(&requirement),
"{requirement} not in {unresolved:#?}"
);
}
}
Loading