ci: Migrate DRA GCS publishing from Release manager to dra-prep-plugin - #7756
ci: Migrate DRA GCS publishing from Release manager to dra-prep-plugin#7756ninalee12 wants to merge 6 commits into
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
The new artifact staging step can drop nested distribution artifacts (and potentially fail early), which would break DRA uploads.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR migrates the Buildkite DRA publishing flow away from the deprecated Release Manager container and onto the elastic/dra-prep Buildkite plugin, then triggers the centralized unified-release-dra-processing pipeline for downstream processing.
Changes:
- Replaced the Release Manager-based DRA step with a generated sub-pipeline that runs
elastic/dra-prepand triggersunified-release-dra-processing. - Added artifact staging and dependency report generation to feed
dra-prep. - Removed Vault credential setup and Release Manager scripts that are no longer needed.
File summaries
| File | Description |
|---|---|
| .buildkite/scripts/stage-dra-artifacts.sh | New helper to download packaged artifacts, generate dependency CSV, and stage inputs for dra-prep. |
| .buildkite/scripts/release-manager.sh | Removed Release Manager invocation script. |
| .buildkite/scripts/dra.sh | New generator script that uploads a DRA prep + annotate + trigger sub-pipeline per workflow. |
| .buildkite/scripts/dra-annotate.sh | New script to annotate the build with a summary link derived from the DRA manifest. |
| .buildkite/scripts/dra_release.sh | Removed legacy DRA wrapper around Release Manager + dependency report generation. |
| .buildkite/pipeline.package.mbp.yml | Switched DRA steps to call the new generator script and removed docker-login plugin usage. |
| .buildkite/hooks/pre-command | Removed Vault-based DRA credential fetching now that Release Manager is gone. |
Review details
- Files reviewed: 7/7 changed files
- Comments generated: 3
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
|
This pull request does not have a backport label. Could you fix it @ninalee12? 🙏
|
There was a problem hiding this comment.
🔵 Needs a closer look
New DRA scripts have failure paths that can exit with non-actionable errors (missing artifacts/manifest), which will make CI failures harder to diagnose.
Review details
Suppressed comments (2)
.buildkite/scripts/stage-dra-artifacts.sh:34
chmod -R a+r "${BASE_DIR}"/*will cause the script to exit with a generic "cannot access" error when no packages were downloaded (unmatched glob), so the later explicit "no artifacts found" check never runs. This makes the failure mode harder to diagnose.
chmod -R a+r "${BASE_DIR}"/*
.buildkite/scripts/dra-annotate.sh:22
- If the dra-prep step doesn't upload a manifest (or artifact download fails silently),
find ... | head -1can yield an empty path andjqwill fail with a non-obvious error. Add a guard that the manifest exists and thatbuild_id/versionwere extracted successfully, so the step fails with a clear message.
manifest=$(find artifacts/dra/fleet-server -name "manifest-*.json" | head -1)
build_id=$(jq -r '.build_id' "${manifest}")
version=$(jq -r '.version' "${manifest}")
url="https://artifacts-${WORKFLOW}.elastic.co/dra-builds/${BUILDKITE_PIPELINE_SLUG}/${BUILDKITE_BUILD_NUMBER}/${build_id}/summary-${version}.html"
- Files reviewed: 7/7 changed files
- Comments generated: 0 new
- Review effort level: Lite
ninalee12
added
backport-8.x
There was a problem hiding this comment.
🟡 Changes recommended
The snapshot dependency report naming behavior appears to regress from the previous DRA flow and may break downstream consumers expecting the prior -SNAPSHOT suffix.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
- Files reviewed: 7/7 changed files
- Comments generated: 1
- Review effort level: Lite
…lename mage getVersion only appends -SNAPSHOT when the SNAPSHOT env var is set, which happens in the separate package.sh job. Neither dra.sh (stack_version passed to dra-prep plugin) nor stage-dra-artifacts.sh (dependency CSV filename) set it, so snapshot builds used the bare version instead of matching the packaged artifacts.
dra.sh and stage-dra-artifacts.sh both needed to append "-SNAPSHOT" to the mage-derived version for snapshot builds. Move that into a shared resolve_dra_version() helper so the two copies can't drift again.
There was a problem hiding this comment.
🔵 Needs a closer look
The new artifact staging/annotation scripts have confirmed failure modes that can drop nested artifacts and bypass intended error handling under set -euo pipefail.
Review details
Suppressed comments (2)
Previously missed (2) — in code that hasn't changed since the last review.
.buildkite/scripts/stage-dra-artifacts.sh:44
- The staging step only copies top-level files from build/distributions into artifacts/ (via
find ... -maxdepth 1 -type f), but the packaging job uploadsbuild/distributions/**/*(see .buildkite/scripts/package.sh:33), so any nested artifacts/directories will be dropped and the dra-prep plugin may not see the full artifact set. Also, using the${BASE_DIR}/*glob makes the failure mode an unhelpfulchmod: cannot access ...rather than a clear "no artifacts downloaded" error.
Consider failing fast with an explicit check, and copy the entire distributions tree into artifacts/ preserving directory structure.
.buildkite/scripts/dra-annotate.sh:19
- With
set -euo pipefail,manifest=$(find ... | head -1)will exit the script early if theartifacts/dra/fleet-serverdirectory is missing (because the pipeline fails under pipefail), so the subsequent-zcheck and custom error message won't run. Capture at most one match directly fromfindand tolerate a missing directory so the explicit error path is used.
- Files reviewed: 8/8 changed files
- Comments generated: 0 new
- Review effort level: Lite
What is the problem this PR solves?
Release manager will be decommisioned as part of https://github.com/elastic/platform-engineering-productivity/issues/2257
This PR replaces release-manager
How does this PR solve the problem?
Replaces the Release Manager Docker step with elastic/dra-prep-buildkite-plugin +
a trigger to unified-release-dra-processing.
Testing
Run a dry run for the snapshot and staging pipelines
Checklist
./changelog/fragmentsusing the changelog toolRelated issues