Repository navigation
docs(shard): use neutral names for the example build lists #840
Workflow file for this run
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| name: Test | |
| on: | |
| push: | |
| branches: [main] | |
| pull_request: | |
| branches: [main] | |
| permissions: | |
| contents: read | |
| concurrency: | |
| group: ${{ github.workflow }}-${{ github.ref }} | |
| cancel-in-progress: true | |
| jobs: | |
| quality: | |
| name: format, clippy, and public metadata | |
| runs-on: ubuntu-latest | |
| steps: | |
| - uses: actions/checkout@v7 | |
| - uses: dtolnay/rust-toolchain@stable | |
| with: | |
| components: rustfmt, clippy | |
| - uses: Swatinem/rust-cache@v2 | |
| - name: Rust formatting | |
| run: cargo fmt --all --check | |
| - name: Clippy | |
| run: cargo clippy --locked --all-targets -- -D warnings | |
| - name: Public metadata contract | |
| run: | | |
| cargo test --locked --test no_helper_script | |
| cargo test --locked --test public_metadata | |
| - name: Cargo source package | |
| run: cargo package --locked | |
| ledger: | |
| name: ledger proofs and writer exclusion | |
| runs-on: ubuntu-latest | |
| steps: | |
| - uses: actions/checkout@v7 | |
| - uses: dtolnay/rust-toolchain@stable | |
| - uses: Swatinem/rust-cache@v2 | |
| - name: Records, proofs, and POSIX writer locks | |
| run: cargo test --locked --test ledger --test ledger_posix_lock --test ledger_lock_aliases --test ledger_retry --test ledger_durability --test ledger_tile_namespace | |
| msrv: | |
| name: Rust 1.90 minimum | |
| runs-on: ubuntu-latest | |
| steps: | |
| - uses: actions/checkout@v7 | |
| - uses: dtolnay/rust-toolchain@master | |
| with: | |
| toolchain: 1.90.0 | |
| - uses: Swatinem/rust-cache@v2 | |
| # `ordered-float` 5.5.1 declares rust-version 1.90. `uuid` 1.29.0 | |
| # declares 1.89.0. Cargo 1.89 rejects the lockfile on ordered-float. | |
| - name: Locked all-targets build | |
| run: cargo build --locked --all-targets | |
| unit: | |
| name: cargo test --lib | |
| runs-on: ubuntu-latest | |
| steps: | |
| - uses: actions/checkout@v7 | |
| - uses: dtolnay/rust-toolchain@stable | |
| - uses: Swatinem/rust-cache@v2 | |
| - name: Library unit tests | |
| run: cargo test --locked --lib | |
| known-bumps: | |
| name: known-bump regression (reproduce_real_prs + bump_emit) | |
| runs-on: ubuntu-latest | |
| steps: | |
| - uses: actions/checkout@v7 | |
| - uses: dtolnay/rust-toolchain@stable | |
| - uses: Swatinem/rust-cache@v2 | |
| - name: Replay frozen maintainer toolchain bumps | |
| run: | | |
| set -euo pipefail | |
| # Library + CLI emit path: foss-2023b -> foss-2024a pairs under | |
| # tests/repro_fixtures/ (GROMACS, ScaFaCoS, MDTraj, Fiona, PuLP, numba) | |
| # plus hierarchy-aware Resolvo cases in bump_emit. | |
| cargo test --locked --test reproduce_real_prs --test bump_emit -- --nocapture | |
| packaging-fixtures: | |
| name: packaging fixtures (eOn + QMCPACK) | |
| runs-on: ubuntu-latest | |
| steps: | |
| - uses: actions/checkout@v7 | |
| - uses: dtolnay/rust-toolchain@stable | |
| - uses: Swatinem/rust-cache@v2 | |
| - name: Parse, resolve, packaging_gate on frozen PR recipe sets | |
| run: | | |
| set -euo pipefail | |
| # Robot-overlay checks skip cleanly when no easyconfigs tree is | |
| # present; resolve + packaging_gate always run against the fixtures. | |
| cargo test --locked \ | |
| --test eon_core_rgpot \ | |
| --test eon_foss_2026_1 \ | |
| --test qmcpack_foss_2026_1 \ | |
| --test eon_packaging \ | |
| --test bundle_component_checksums \ | |
| --test goal_scenario \ | |
| -- --nocapture | |
| solve-reports: | |
| name: solve + build-list + stack-diff | |
| runs-on: ubuntu-latest | |
| steps: | |
| - uses: actions/checkout@v7 | |
| - uses: dtolnay/rust-toolchain@stable | |
| - uses: Swatinem/rust-cache@v2 | |
| - name: Co-select and report fixtures | |
| run: cargo test --locked --test build_list_and_stack_diff -- --nocapture | |
| canonical-workflows: | |
| name: canonical package + campaign workflows | |
| runs-on: ubuntu-latest | |
| steps: | |
| - uses: actions/checkout@v7 | |
| - uses: dtolnay/rust-toolchain@stable | |
| - uses: Swatinem/rust-cache@v2 | |
| - name: Parse, materialize, solve, route, and coordinate campaigns | |
| run: | | |
| cargo test --locked \ | |
| --test container_contract \ | |
| --test foreign_conditions \ | |
| --test foreign_ingest \ | |
| --test foreign_package_plan \ | |
| --test manifest_requires_resolvo \ | |
| --test package_model \ | |
| --test package_config \ | |
| --test profile_emission \ | |
| --test package_workflow \ | |
| --test bump_workflow \ | |
| --test bump_source_patches \ | |
| --test bump_companion \ | |
| --test bump_build_backend \ | |
| --test stack_policy \ | |
| --test canonical_cli \ | |
| --test source_artifact_class \ | |
| --test target_config \ | |
| --test campaign \ | |
| --test ledger \ | |
| --test ledger_tiles \ | |
| --test ledger_tile_namespace \ | |
| --test mcp_v1 | |
| package-closure: | |
| name: package catalog + closure + source discovery | |
| runs-on: ubuntu-latest | |
| steps: | |
| - uses: actions/checkout@v7 | |
| - uses: dtolnay/rust-toolchain@stable | |
| - uses: Swatinem/rust-cache@v2 | |
| - name: Catalog providers, recursive closure, and source-root discovery | |
| run: | | |
| set -euo pipefail | |
| cargo test --locked \ | |
| --test package_catalog \ | |
| --test package_closure \ | |
| --test package_closure_write \ | |
| --test package_sources_discovery \ | |
| -- --nocapture | |
| integration: | |
| name: remaining integration tests | |
| runs-on: ubuntu-latest | |
| steps: | |
| - uses: actions/checkout@v7 | |
| - uses: dtolnay/rust-toolchain@stable | |
| - uses: Swatinem/rust-cache@v2 | |
| - name: Every integration test that is not ignore-only | |
| run: | | |
| cargo test --locked \ | |
| --test arch_known_failures \ | |
| --test arch_unsupported \ | |
| --test backport_list \ | |
| --test baseline_easystack \ | |
| --test batch_versions \ | |
| --test bot_classify \ | |
| --test build_list_and_bare_tuple \ | |
| --test cli_ergonomics \ | |
| --test cli_reference \ | |
| --test cuda_cc \ | |
| --test component_reach \ | |
| --test doc_walkthrough \ | |
| --test eb_robot_paths \ | |
| --test easyblock_layers \ | |
| --test easyblock_owed \ | |
| --test extension_provides \ | |
| --test family_site_layer \ | |
| --test fetch_prs_script \ | |
| --test fix_cost \ | |
| --test from_commit \ | |
| --test generation_lint \ | |
| --test stack_gap \ | |
| --test stack_provided \ | |
| --test stack_shadow \ | |
| --test stack_shard \ | |
| --test inode_footprints_snapshot \ | |
| --test planning_capacity \ | |
| --test planning_storage \ | |
| --test stack_shard_destinations \ | |
| --test stack_admit \ | |
| --test admission_forecast_lifecycle \ | |
| --test stack_shard_cpu_prelude \ | |
| --test stack_shard_sources \ | |
| --test robot_lookup \ | |
| --test robot_lookup_versionprefix_field \ | |
| --test stack_shard_robot_layout \ | |
| --test stack_shard_robot_versionprefix_field \ | |
| --test planner_traversal \ | |
| --test hierarchy_universe \ | |
| --test jenkins_build_list \ | |
| --test lint_fixtures \ | |
| --test llms_txt \ | |
| --test mcp_registry \ | |
| --test mcp_stdio \ | |
| --test module_tree_dump_script \ | |
| --test oom_profile \ | |
| --test optimisation_criteria \ | |
| --test pr_merge_order \ | |
| --test pr_overlap \ | |
| --test pr_overlap_arch \ | |
| --test pr_overlap_easyconfigs \ | |
| --test pr_overlap_easyconfigs_open \ | |
| --test pypi_cran_extend \ | |
| --test rebuild_set \ | |
| --test recipe_commands \ | |
| --test repro_scoring \ | |
| --test ring0_ingest_repro \ | |
| --test sbom_from_lock \ | |
| --test sbom_page_drift \ | |
| --test sbom_schema \ | |
| --test skill_easyblock_fix \ | |
| --test solved_dot \ | |
| --test solver_completeness \ | |
| --test stack_files \ | |
| --test stack_order_filter \ | |
| --test stack_order_version \ | |
| --test source_probe \ | |
| -- --nocapture | |
| cli-smoke: | |
| name: CLI smoke (package bundle + recipe checks) | |
| runs-on: ubuntu-latest | |
| needs: [unit, known-bumps] | |
| steps: | |
| - uses: actions/checkout@v7 | |
| - uses: dtolnay/rust-toolchain@stable | |
| - uses: Swatinem/rust-cache@v2 | |
| - name: Build release binary | |
| run: cargo build --locked --release --bin eb-stack | |
| - name: Tutorial path (GROMACS zero-hand-fed bump) | |
| run: | | |
| set -euo pipefail | |
| BIN=./target/release/eb-stack | |
| OUT=$(mktemp -d) | |
| "$BIN" --version | |
| "$BIN" package bump \ | |
| --source tests/repro_fixtures/gromacs/GROMACS-2024.4-foss-2023b.eb \ | |
| --toolchain foss/2024a \ | |
| --easyconfigs tests/repro_fixtures/universe_foss_2024a \ | |
| --out-dir "$OUT" | |
| RECIPE="$OUT/easyconfigs/g/GROMACS/GROMACS-2024.4-foss-2024a.eb" | |
| test -f "$OUT/package.plan.json" | |
| test -f "$OUT/package.sbom.cdx.json" | |
| test -f "$OUT/locks/default.lock.json" | |
| test -f "$RECIPE" | |
| grep -q "toolchain = {'name': 'foss', 'version': '2024a'}" \ | |
| "$RECIPE" | |
| grep -q "('Python', '3.12.3')" "$RECIPE" | |
| grep -q "('mpi4py', '4.0.1')" "$RECIPE" | |
| # Residual: maintainer-added pybind11 is NOT invented. | |
| ! grep -q pybind11 "$RECIPE" | |
| - name: recipe check on QMCPACK fixture (universe = empty drafts) | |
| run: | | |
| set -euo pipefail | |
| BIN=./target/release/eb-stack | |
| # Fixture alone lacks HDF5/Boost; expect non-zero (loud incompleteness). | |
| if "$BIN" recipe check \ | |
| --recipe fixtures/qmcpack_foss_2026_1/easyconfigs/q/QMCPACK/QMCPACK-4.3.0-foss-2026.1.eb \ | |
| --easyconfigs fixtures/qmcpack_foss_2026_1/easyconfigs; then | |
| echo "expected recipe check to fail without robot deps" >&2 | |
| exit 1 | |
| fi | |
| - name: recipe packaging gate on eOn core fixture | |
| run: | | |
| set -euo pipefail | |
| BIN=./target/release/eb-stack | |
| # Drafts alone leave stack deps missing; exit non-zero is correct. | |
| # Still exercises parse + packaging_gate on the draft-PR snapshot. | |
| if "$BIN" recipe check \ | |
| --recipe fixtures/eon_core_rgpot/easyconfigs/e/eOn/eOn-2.17.2-foss-2026.1.eb \ | |
| --easyconfigs fixtures/eon_core_rgpot/easyconfigs \ | |
| --require-configopt=-Dwith_rgpot=true; then | |
| echo "expected recipe check to fail without robot deps" >&2 | |
| exit 1 | |
| fi | |
| portable-candidate: | |
| name: portable Linux candidate (no publication) | |
| runs-on: ubuntu-22.04 | |
| steps: | |
| - uses: actions/checkout@v7 | |
| with: | |
| persist-credentials: false | |
| - uses: dtolnay/rust-toolchain@stable | |
| with: | |
| targets: x86_64-unknown-linux-musl | |
| - uses: Swatinem/rust-cache@v2 | |
| with: | |
| key: portable-candidate-musl | |
| - name: Install portable C toolchain | |
| run: sudo apt-get update && sudo apt-get install --no-install-recommends -y musl-tools | |
| - name: Build baseline static candidate | |
| shell: bash | |
| run: | | |
| set -euo pipefail | |
| unset CARGO_ENCODED_RUSTFLAGS | |
| export RUSTFLAGS='-C target-cpu=x86-64' | |
| export CC_x86_64_unknown_linux_musl=musl-gcc | |
| cargo build --locked --release --target x86_64-unknown-linux-musl --bin eb-stack | |
| BIN=target/x86_64-unknown-linux-musl/release/eb-stack | |
| if readelf -l "$BIN" | grep -q INTERP; then | |
| echo "portable candidate has a dynamic interpreter" >&2 | |
| exit 1 | |
| fi | |
| "$BIN" --version | |
| "$BIN" stack capacity --snapshot tests/planning_capacity_fixtures/snapshot.json \ | |
| --partition local --target example/host --feature scratch-node \ | |
| --cpus 4 --memory-mib 4096 --exclusive-node true --max-slots-per-node 1 \ | |
| --generated-at 2026-01-01T12:00:10Z --max-age-seconds 60 \ | |
| > target/portable-capacity-check.json | |
| "$BIN" stack shard --help > target/portable-shard-help.txt | |
| grep -q -- --storage-policy target/portable-shard-help.txt | |
| - name: Package source-bound candidate | |
| shell: bash | |
| run: | | |
| set -euo pipefail | |
| python3 - <<'PY' | |
| import hashlib, json, os, pathlib, shutil, subprocess | |
| source = subprocess.check_output(["git", "rev-parse", "HEAD"], text=True).strip() | |
| tree = subprocess.check_output(["git", "rev-parse", "HEAD^{tree}"], text=True).strip() | |
| binary = pathlib.Path("target/x86_64-unknown-linux-musl/release/eb-stack") | |
| check = json.loads(pathlib.Path("target/portable-capacity-check.json").read_text()) | |
| assert check["queue_fitting_nodes"] == 1 and check["free_slots"] == 1 | |
| root = pathlib.Path("target/portable-candidate") | |
| name = "eb-stack-" + source + "-x86_64-unknown-linux-musl" | |
| payload = root / name | |
| payload.mkdir(parents=True, exist_ok=True) | |
| shutil.copy2(binary, payload / "eb-stack") | |
| shutil.copy2("LICENSE", payload / "LICENSE") | |
| receipt = { | |
| "schema_version": 1, "candidate": True, "release": False, | |
| "source": source, "source_tree": tree, | |
| "target": "x86_64-unknown-linux-musl", "target_cpu": "x86-64", | |
| "binary_sha256": hashlib.sha256(binary.read_bytes()).hexdigest(), | |
| "dynamic_interpreter": False, | |
| "startup_checks": ["version", "captured-capacity", "shard-storage-help"], | |
| "full_suite_status": "consult the workflow result", | |
| "workflow_run_id": os.environ["GITHUB_RUN_ID"], | |
| "rustc": subprocess.check_output(["rustc", "--version"], text=True).strip(), | |
| } | |
| encoded = (json.dumps(receipt, indent=2, sort_keys=True) + "\n").encode() | |
| (payload / "receipt.json").write_bytes(encoded) | |
| (root / "receipt.json").write_bytes(encoded) | |
| epoch = subprocess.check_output(["git", "show", "-s", "--format=%ct", "HEAD"], text=True).strip() | |
| archive = root / (name + ".tar.xz") | |
| subprocess.run(["tar", "--sort=name", "--mtime=@" + epoch, "--owner=0", | |
| "--group=0", "--numeric-owner", "-cJf", str(archive), | |
| "-C", str(root), name], check=True) | |
| digest = hashlib.sha256(archive.read_bytes()).hexdigest() | |
| (root / (archive.name + ".sha256")).write_text(digest + " " + archive.name + "\n") | |
| PY | |
| - uses: actions/upload-artifact@v7 | |
| with: | |
| name: portable-candidate-x86_64-unknown-linux-musl | |
| path: | | |
| target/portable-candidate/*.tar.xz | |
| target/portable-candidate/*.sha256 | |
| target/portable-candidate/receipt.json | |
| if-no-files-found: error |