Publish the Helm chart to the OCI registry from the publish pipeline - #377
Draft
wallrj wants to merge 5 commits into
Draft
Publish the Helm chart to the OCI registry from the publish pipeline#377wallrj wants to merge 5 commits into
wallrj wants to merge 5 commits into
Conversation
This bug has been latent since commit 7ba3cf2 added the cmctl check. It caused tests to panic Signed-off-by: Ashley Davis <[email protected]>
The aim is for this to replace the need for hack/push_and_sign_chart.sh and to eventually make charts.jetstack.io downstream of the OCI registry. Signed-off-by: Ashley Davis <[email protected]>
The helmchartpr action opened a pull request against jetstack/jetstack-charts with the release's Helm chart. Since the publish GitHub token was rotated to the cert-manager-bot account (cert-manager/cert-manager#9237), that token cannot reach the jetstack org, so the action can no longer run. The charts.jetstack.io repository is now synced from the OCI registry by a workflow in jetstack-charts instead, so nothing needs to open that PR. Delete the action, its GitHub repository manager, the flags and Cloud Build substitutions that only it read, and the Cloud Build job that checked the bot's permissions on jetstack-charts. Co-Authored-By: Claude Fable 5.1 <[email protected]> Signed-off-by: Richard Wall <[email protected]>
The publish job now installs cosign v3. cosign v3 uploads to the transparency log, writes the new bundle format and reads a signing config by default. cosign v1.13.6 did none of those for key-based signing unless COSIGN_EXPERIMENTAL was set. The chart signing code already turned all three off; image signing did not, so with a v3 binary it would have tried to create and verify a tlog entry and failed on our SHA-512 KMS key. Move the chart options into the cosign package as DefaultSignOptions and DefaultVerifyOptions and use them for container images and manifest lists too. Route image signing through the injectable runner so tests can fake it, which answers the Copilot comment on cert-manager#290. cosign verify-blob has the same problem: staging signs metadata.json with --tlog-upload=false, and cosign v2+ fails verification when it cannot find a tlog entry. Pass --insecure-ignore-tlog=true, which is what the chart verification already does. Co-Authored-By: Claude Fable 5.1 <[email protected]> Signed-off-by: Richard Wall <[email protected]>
Install helm the same way as cmrel, cosign and crane, with go install at a pinned tag, instead of piping an unpinned script from the helm main branch into sh. The publish job runs with KMS signing access, so every tool it runs must come from a pinned source. Use the same golang image as the other steps so Renovate keeps them in step. Pin cosign v3.1.3, crane v0.22.1 and helm v4.3.0, the current releases. Co-Authored-By: Claude Fable 5.1 <[email protected]> Signed-off-by: Richard Wall <[email protected]>
Contributor
|
Skipping CI for Draft Pull Request. |
cert-manager-prow
Bot
added
do-not-merge/work-in-progress
Contributor
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
cert-manager-prow
Bot
added
the
size/XXL
This was referenced Sep 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This revives #290 by @SgtCoDFish. The two commits from that PR are rebased onto master unchanged, with three commits on top.
What changes
helmchartocipublish action. It pushes the chart and its.provfile toquay.io/jetstack/charts, signs the tag with cosign, verifies the signature, then copies the chart to the tag without thevprefix and signs that too. This is whathack/push_and_sign_chart.shdoes by hand today. The script stays until this action has run on a real release.helmchartpraction is removed. The publish GitHub token was rotated to thecert-manager-botaccount after Release cert-manager v1.21.2 cert-manager#9237. That token cannot reachjetstack/jetstack-charts, so the action cannot run. A workflow injetstack-chartsnow syncscharts.jetstack.iofrom the OCI registry instead, so we no longer need cmrel to open that PR. The GitHub repository manager, its flags, the Cloud Build substitutions and thegcb/test/helmjob all went with it.cosign signcall now passes--tlog-upload=false --new-bundle-format=false --use-signing-config=false, andcosign verify-blobpasses--insecure-ignore-tlog=true. That keeps the behaviour we have with cosign v1, where key-based signing never touched the transparency log. Without this, the metadata check and image signing would break on the v3 binary, not just chart signing.go installat a pinned tag. Add OCI signing as part of existing publish pipeline #290 piped an unpinned script from the helmmainbranch intosh. The publish job has KMS signing access, so its tools must come from pinned sources like cmrel, cosign and crane already do.What I checked
go test ./...andmake presubmitpass.go installof cosign v3.1.3, crane v0.22.1 and helm v4.3.0 at the pinned tags succeeds.--tlog-upload,--new-bundle-format,--signature-digest-algorithmand--signature, then proceeds as far as calling KMS.DOCKER_CONFIGsecret covershelm pushto quay.io.This has not run against a real release yet. The plan from #290 stands: try it on a
v1.22.0-alpha.0publish before retiring the script.Evidence
COSIGN_EXPERIMENTALis set: https://github.com/sigstore/cosign/blob/v1.13.6/cmd/cosign/cli/sign/sign.go#L69-L73--tlog-upload,--new-bundle-formatand--use-signing-configto true: https://github.com/sigstore/cosign/blob/v3.1.3/cmd/cosign/cli/options/sign.go#L125-L170metadata.jsonwith--tlog-upload=false, so there is no tlog entry forverify-blobto find: https://github.com/cert-manager/cert-manager/blob/650eb5fbf50a7fbcf959f78079964196d0c80a62/make/release.mk#L90charts.jetstack.iosync from OCI lives in jetstack/jetstack-charts#560, on a private repository. It adds cert-manager to the existing hourlyoci-syncjob, and carries the v1.21.2 chart as its first sync.Follow-ups, not in this PR
_COSIGN_REPO_REF,_CRANE_REPO_REFand_HELM_REPO_REFpins. They need a custom manager in cert-manager/renovate-config.gcb/publish/cloudbuild.yaml. Whichever merges last needs a rebase.hack/push_and_sign_chart.sh, the warning that points at it incmd/cmrel/cmd/const.go, and the step in the website release process, once this has published a release.--signing-configfile before cosign removes--tlog-upload.Closes #290.
[Claude Fable 5.1]