Reduce PR CI latency with affected Rust shards#479
Conversation
Signed-off-by: Jeremi Joslin <[email protected]>
Signed-off-by: Jeremi Joslin <[email protected]>
51bb806 to
5b98e43
Compare
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5b98e4357d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if package is not None: | ||
| seeds.add(package) | ||
| continue |
There was a problem hiding this comment.
For ordinary PRs, this continue assumes Cargo reverse dependencies capture every consumer, but repository tests have cross-package filesystem dependencies that Cargo metadata cannot see. For example, registry-notary/tests/config_schema.rs reads crates/registry-relay/profiles/**, while registry-language-server/src/index.rs reads Registryctl's bundled starter and journey catalog; changing those files now selects only the owning package and its Cargo dependents, so the rust-tests matrix in ci.yml omits the Notary or language-server tests that validate them. Add explicit affected-package edges for these external consumers (and regression tests for the mappings), otherwise incompatible profile/config or authoring-fixture changes can pass PR CI.
AGENTS.md reference: AGENTS.md:L47-L54
Useful? React with 👍 / 👎.
| @@ -0,0 +1,349 @@ | |||
| #!/usr/bin/env python3 | |||
There was a problem hiding this comment.
Commit 11035f42f78f81c336761c29b8c8bba39525f359 has no Signed-off-by trailer, so it violates the repository's per-commit DCO requirement and the DCO gate will reject it. Recreate this commit with an appropriate sign-off before merging.
AGENTS.md reference: AGENTS.md:L68-L72
Useful? React with 👍 / 👎.
| - rust-policy | ||
| - rust-quality | ||
| - rust-tests | ||
| - notary-contracts |
There was a problem hiding this comment.
Removing the inline Notary PostgreSQL loop leaves ordinary PR coverage entirely to notary-postgres-conformance.yml, but that workflow's pull_request.paths omits direct runtime dependencies such as crates/registry-platform-config/** and crates/registry-config-report/** (and several transitive workspace dependencies). A change to registry-platform-config, which is used directly by registry-notary, registry-notary-core, and registry-notary-server, previously ran all three PostgreSQL conformance versions through the root Rust job; now the specialized workflow is not triggered and this replacement aggregate only waits for unit tests and API contracts. Expand the specialized workflow's path filters from the actual Notary dependency closure or add an affected PostgreSQL job to root CI.
Useful? React with 👍 / 👎.
1 participant
What changed
mainpushes, while ordinary PRs run only affected components.ciprofile for source tutorial containers so PR validation does not pay release LTO/codegen costs.Rust workspace,Release tooling,Docs, andRelease source proofcontexts, including safe reporting when path-gated work is skipped, and adds a broader stableCI resultaggregate for a later branch-protection switch.Why
PR #471 took 27m53s wall time. Its serial
Rust workspacejob took 27m37s, including check, clippy, a clean rebuild for tests, Relay all-features tests, API contracts, and unrelated Notary PostgreSQL conformance. The source tutorial job added a separate 14m53s critical path, dominated by release-profile builds.The prior
cargo clean, disabled incremental compilation, and disabled target caching were added after hosted runners exhausted their filesystem. This change keeps those disk safeguards. It gains latency by isolating work on fresh standard runners and selecting affected packages, not by restoring an unbounded workspace target cache.Running the classifier against the actual #471 file list selects only the
relayandregistryctlRust shards. This workflow PR itself intentionally exercises the complete matrix because it changesci.yml.No larger or paid runners are introduced. Every job remains on standard GitHub-hosted public-repository runners.
Measured result
Final rebased run 30145696189 is fully green. Every build/test job completed in less than 10 minutes:
The full workflow-edit validation took 12m10s because it deliberately forced all six Rust shards plus every component gate and temporarily saturated the repository's standard-runner concurrency. Ordinary PRs run the affected subset. The actual #471 classifier result has two Rust shards, so it stays below that saturation point.
Disk telemetry from the final run confirms bounded targets: Relay was the largest Rust shard at 8.1 GB with 79 GB still free; registryctl used 2.8 GB with 84 GB free; the platform shard used 869 MB with 87 GB free. The production-shaped tutorial used 6.4 GB across its Cargo home and build target with 77 GB free.
Validation
actionlint -color .github/workflows/ci.yml .github/workflows/notary-postgres-conformance.yml .github/workflows/relay-postgres-conformance.ymlpython3 .github/scripts/test_ci_changes.py(8 tests)python3 -m py_compile .github/scripts/ci_changes.py .github/scripts/run_cargo_packages.py .github/scripts/test_ci_changes.pypython3 release/scripts/check-gates-inventory.py(87 required gates)python3 -m unittest release/scripts/test_check_gates_inventory.py(42 tests)python3 -m unittest release/scripts/test_registry_release.py(68 tests)cargo fmt --checkshellcheck docs/site/scripts/check-registryctl-tutorials.shnpm --prefix docs/site run test:tutorial:registryctl(9 tests)npm --prefix docs/site run check:tutorial:dry-rungit diff --check5b98e4357d19a6835a0c310ee8bf650b114b8656Rollout note
The live branch protection currently requires
Changed paths,Rust workspace,Release tooling,Docs, andRelease source proof. Those contexts remain available, so this PR is mergeable without changing protection first. After this workflow is onmain,CI resultcan replace the individual RegistryStack CI contexts as the stable required check.