Repository navigation
chore(deps): update kubernetes-version - #591
renovate[bot] wants to merge 1 commit into
Conversation
|
Note 7 resources changed across 3 apps Blast radius
konflate 0.6.4 · rendered |
|
Held: this bump cannot pass CI until cert-manager supports Kubernetes 1.37. Triage of the three failing checks at head dbbb59d:
The cert-manager gate is the blocker that cannot be fixed on this branch: the newest cert-manager release (v1.21.2, 2026-09-11) supports k8s 1.33-1.36 and no release supports 1.37 yet. This is the exact premature-bump scenario the gate was built to catch (issue #322: Renovate bumped Kubernetes to 1.37 while cert-manager v1.21.x only supported up to 1.36, silently breaking the daily air-gapped workflow). Recommendation: hold kubernetes-version at 1.36 until cert-manager ships 1.37 support, then let Renovate recreate this group PR. If the weekly recreation is noisy, a Renovate |
a7b20b2 to
72da2e2
Compare
shrinedogg
left a comment
shrinedogg
left a comment
There was a problem hiding this comment.
Batch review (2026-10-12): this bump is internally inconsistent and cannot go green as-is. Not merging.
- airgap/images.txt:34-38 and :58-62 - kube-apiserver/controller-manager/proxy/scheduler are pinned to v1.37.1 while the target Kubernetes version is v1.37.0 (kindest/node, per the local-host topology pins). The v1.37.0 release deploys all four at :v1.37.0. This is the "Air-gap image digest pins" failure.
- airgap/images.txt:33/:57 (coredns v1.14.7) and :37/:61 (etcd 3.7.2-0) are 1.37.1-generation values; v1.37.0 deploys coredns 1.14.6 and etcd 3.7.0-0. Both are tracked components and will fail the same gate after item 1 is fixed.
- mgmt/local-host/clusters/docker/cluster-class.yaml:84,99 and management/cluster-class.yaml:89,104 - customImage carries the new tag kindest/node:v1.37.0 with the STALE v1.36.4 digest sha256:099e0493... Pulling by digest would silently deploy v1.36.4 nodes into clusters declaring v1.37.0, and no gate covers it (the digest test is scoped to airgap/). The v1.37.0 digest is sha256:a1ed56cfb0e7b93589bdf9738cd566405a265939e3620fc4f5de89adff580ae5.
- mgmt/local-talos/clusters/management/cluster.yaml:63 (TalosControlPlane v1.37.1) and mise.toml:5 (kubectl 1.37.1) split the single kubernetes-version group. kindest/node:v1.37.1 does not exist yet (registry 404), so a uniform 1.37.1 is not possible; a uniform bump means both at 1.37.0.
- "Zarf mise asset pin" failure: cert-manager 1.21.2 (bootstrap.toml:38) supports k8s 1.33-1.36 and no supported release covers 1.37. This cannot pass on its own: #458 (drops this gate) must land first, or #591 must rebase onto it; #458's validated pairings already include (1.21.2, 1.37.0, air-gapped nightly 2026-09-21).
- Stale v1.36.4 residue: airgap/images.txt:50, airgap/scripts/stage-and-create-cluster.sh:6, airgap/scripts/build-config-artifact.sh:175, docs/operations.md:697, docs/airgap.md:138-139 and :316.
Fix path: one uniform v1.37.0 across kindest/node + kube components + digests, most reliably via the repo's own python3 airgap/tests/test-airgap-kubeadm-images.py --fix (the documented reconciliation step); fix the four cluster-class.yaml digests; revert the Talos/kubectl split to 1.37.0; land #458 first. Alternatively close and re-bump as a uniform 1.37.1 once kindest publishes it.
shrinedogg
left a comment
shrinedogg
left a comment
There was a problem hiding this comment.
Hold-verification review at head 72da2e2 (unchanged since the review was captured; CI at this head: "Air-gap image digest pins" fail, "Zarf mise asset pin" fail). Fresh evidence collected at this head; the hold remains in force. Not merging.
Verified locally (all commands run at this head, checked out from pull/591):
- python3 airgap/tests/test-airgap-kubeadm-images.py -> exit 1, 12 pin mismatches (identical to the red CI check; per-line list in finding 1).
- python3 tests/test-cert-manager-kubernetes-support.py -> exit 1: "cert-manager 1.21 (bootstrap.toml) supports Kubernetes 1.33 -> 1.36, but the pinned Kubernetes version is v1.37.0"; cross-checked against cert-manager's own support table (raw.githubusercontent.com/cert-manager/website/master content/docs/releases/README.md): currently supported releases 1.20 (k8s 1.32 -> 1.35) and 1.19 (1.31 -> 1.35); 1.21 is listed as upcoming with 1.33 -> 1.36. No release line supports 1.37 today.
- Docker Hub registry API for kindest/node: tag v1.36.4 resolves to sha256:099e049362a1526b2db71494e1947aae99bd16290d7c895f2b7ea312e3cbfaed, tag v1.37.0 resolves to sha256:a1ed56cfb0e7b93589bdf97c8cd566405a265939e3620fc4f5de89adff580ae5.
- Pulled the pinned kindest/node:v1.37.0@a1ed56cf image and inspected /kind/manifests: it bakes kindnetd v20260820-69b56db7, local-path-provisioner v20260820-69b56db7, local-path-helper v20260131-7181c60a. All three are already listed in airgap/images.txt lines 53-56, so the CNI preload coupling documented in docs/airgap.md holds at v1.37.0.
- grep of airgap/zarf.yaml at this head: no image pins for any of the bumped components (only kubectl command invocations), so the package listing is not silently diverging from images.txt.
- mise.toml kubectl 1.37.0 -> 1.37.1: no gate depends on it; CI "Build, lint, and test", kustomize builds, and Renovate digest-pinning checks are green.
Blocking findings:
-
airgap/images.txt control-plane component pins disagree with what kubeadm of the pinned kindest/node v1.37.0 deploys (12 lines). Target version v1.37.0 comes from the topology pins (mgmt/local-host/clusters/docker/cluster.yaml:21, mgmt/local-host/clusters/management/cluster.yaml:24), which the contract test treats as authoritative. Mismatches:
- images.txt:33 and :57 coredns v1.14.7, kubeadm v1.37.0 deploys v1.14.6
- images.txt:34 and :58 kube-apiserver v1.37.1, deploys v1.37.0
- images.txt:35 and :59 kube-controller-manager v1.37.1, deploys v1.37.0
- images.txt:36 and :60 kube-proxy v1.37.1, deploys v1.37.0
- images.txt:37 and :61 etcd 3.7.2-0, deploys 3.7.0-0
- images.txt:38 and :62 kube-scheduler v1.37.1, deploys v1.37.0
This is the group bump's internal incoherence: the kindest/node minor update moves the air-gapped clusters to 1.37.0 while the kubernetes/kubernetes patch update drags the component pins to 1.37.1. The tracked components must match the kubeadm baked in the node image. Fix path: the contract test's --fix rewrites the tracked pins (tags plus registry digests) from kubeadm v1.37.0, or the group must be restructured so component pins follow the node image rather than the standalone kubernetes/kubernetes release.
-
cert-manager 1.21.2 (bootstrap.toml:38) does not support the pinned Kubernetes version. The PR sets 1.37.0 on both local-host clusters (docker/cluster.yaml:21, management/cluster.yaml:24) and pushes the Talos management cluster to v1.37.1 (mgmt/local-talos/clusters/management/cluster.yaml:63), all above cert-manager's stated ceiling of 1.36 (see verified-locally list; same failure as the red CI check). This gate exists because exactly this pairing silently broke the air-gapped workflow before (issue #322). Fix path: land cert-manager support for 1.37 first (bump the bootstrap.toml pin when a release supports it) or hold the Kubernetes minor until then.
-
Warning: four customImage pins pair the new tag with the old digest. mgmt/local-host/clusters/docker/cluster-class.yaml:84 and :99, and mgmt/local-host/clusters/management/cluster-class.yaml:89 and :104, now read kindest/node:v1.37.0@sha256:099e049362..., but 099e0493... is the digest of v1.36.4; v1.37.0 is sha256:a1ed56cf... (registry-verified above; this same PR's images.txt:45-46 carries a1ed56cf for v1.37.0). On main these lines were tag+digest consistent; the PR changed the tag and left the digest, so the Renovate custom manager for these files updates the tag without the digest. A tag+digest pull of a mismatched pair fails, which breaks CAPD node creation on the local-host profile. Fix: extend the custom manager (or autoReplaceStringTemplate) to bump tag and digest together, then regenerate.
-
Warning: docs left behind by an incomplete sweep. docs/airgap.md:138-139 still document archive filenames kindest_node_v1.36.4_mgmt.tar and kindest_node_v1.36.4.tar, but build-package.sh now writes kindest_node_v1.37.0_mgmt.tar and kindest_node_v1.37.0.tar. docs/operations.md:697 still says "The workload uses Kubernetes v1.36.4". docs/airgap.md:316 and the airgap/images.txt:50 section header "(v1.36.4 generation)" describe the previous node generation. Suggestion: fold these into the same human-completes pass as findings 1-3 so the runbook describes the versions that actually ship.
Looks good (checked, no issue): the stage-and-create-cluster.sh KIND_NODE_IMAGE default and the build-config-artifact.sh customImage anchor both carry the correct new tag+digest pair (a1ed56cf); the cluster-class topology version bumps are consistent with the kubeadm target; the Talos kubernetes/kubernetes patch bump is orthogonal to the two gates and not blocked by this review beyond finding 2's support ceiling.
Cross-reference: the earlier batch review (2026-10-12) flagged items 1 and 2 on an older head; findings 3 and 4 are new at this head.
) Remove the CI gate that rejected cert-manager/Kubernetes pairings outside the upstream published window. The gate was originally added to catch chart/image drift (polarsquad#322), but that failure mode is already covered by the surviving tests/test-cert-manager-version-consistency.py. The gate then blocked PR polarsquad#591 (v1.37.0 bump) even though the exact pairing passed the scheduled air-gapped nightly on 2026-09-21 (issue polarsquad#384). Replace with a non-blocking notice: tests/test-cert-manager-kubernetes-support.py prints a NOTICE when the pinned pairing is not in VALIDATED_PAIRINGS and exits 0 in all cases; it only exits 1 on unparseable pins. The dedicated .github/workflows/airgap-image-existence.yml (path-filtered, polarsquad#465) already covers the unconditional digest checks removed here from validate.yml. Seed VALIDATED_PAIRINGS with both the current pin (1.21.2, 1.36.4, within the cert-manager 1.21 published window 1.33-1.36) and the air-gapped nightly result (1.21.2, 1.37.0, run 35554802424, 2026-09-21). Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Remove the CI gate that rejected cert-manager/Kubernetes pairings outside the upstream published window. The gate was originally added to catch chart/image drift (#322), but that failure mode is already covered by the surviving tests/test-cert-manager-version-consistency.py. The gate then blocked PR #591 (v1.37.0 bump) even though the exact pairing passed the scheduled air-gapped nightly on 2026-09-21 (issue #384). Replace with a non-blocking notice: tests/test-cert-manager-kubernetes-support.py prints a NOTICE when the pinned pairing is not in VALIDATED_PAIRINGS and exits 0 in all cases; it only exits 1 on unparseable pins. The dedicated .github/workflows/airgap-image-existence.yml (path-filtered, #465) already covers the unconditional digest checks removed here from validate.yml. Seed VALIDATED_PAIRINGS with both the current pin (1.21.2, 1.36.4, within the cert-manager 1.21 published window 1.33-1.36) and the air-gapped nightly result (1.21.2, 1.37.0, run 35554802424, 2026-09-21). Co-authored-by: Claude Sonnet 4.6 <[email protected]>
72da2e2 to
98aedf0
Compare
- fix kindest/node digest mismatch in docker and management cluster-class.yaml (v1.37.0 tag paired with v1.36.4 sha256:099e049... digest; replace with sha256:a1ed56c...) - fix renovate.json5 "kindest/node image pins" manager to capture and replace @sha256 digest alongside the tag, preventing the same defect on future bumps - add currentDigest assertion for kindest/node in tests/test-renovate-coverage.py Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
edb5b60 to
98aedf0
Compare
98aedf0 to
34aad82
Compare
shrinedogg
left a comment
shrinedogg
left a comment
There was a problem hiding this comment.
REQUEST_CHANGES (hold): re-review at head 34aad82. The rebase onto current main today resolved the cert-manager gate finding via #458, but the digest-pin failure and the stale-digest customImage lines remain. Not merging.
Prior findings, item by item (CHANGES_REQUESTED review of 2026-10-13 at 72da2e2):
- images.txt component pins vs what kubeadm v1.37.0 deploys (12 lines): STILL OPEN. Identical 12 mismatches in the red check at this head (finding 1 below) and in the local run of the same test.
- cert-manager 1.21.2 vs k8s 1.37 support gate: RESOLVED via #458 (merged before this rebase). tests/test-cert-manager-kubernetes-support.py is now notice-only (docstring: "Notice (never fail)... the upstream-window gate that blocked every bootstrap.toml PR was removed in #458") and VALIDATED_PAIRINGS includes (1.21.2, 1.37.0, air-gapped nightly 35554802424). CI "Zarf mise asset pin" is green at this head.
- Stale digest on the four customImage lines: STILL OPEN (finding 2 below).
- Docs incomplete sweep: STILL OPEN in part; line numbers shifted by the rebase (finding 4 below).
- kubernetes-version group split (Talos v1.37.1, kubectl 1.37.1 vs kindest/node v1.37.0): STILL OPEN (finding 3 below).
Remaining blocking findings:
-
Required check "Air-gap image digest pins" is red at head 34aad82 (run 38925234002, job 117054961038; mergeStateStatus UNSTABLE). The 12 mismatches, verbatim from the job log:
- airgap/images.txt:33 and :57 coredns v1.14.7, kubeadm v1.37.0 deploys v1.14.6
- airgap/images.txt:34 and :58 kube-apiserver v1.37.1, deploys v1.37.0
- airgap/images.txt:35 and :59 kube-controller-manager v1.37.1, deploys v1.37.0
- airgap/images.txt:36 and :60 kube-proxy v1.37.1, deploys v1.37.0
- airgap/images.txt:37 and :61 etcd 3.7.2-0, deploys 3.7.0-0
- airgap/images.txt:38 and :62 kube-scheduler v1.37.1, deploys v1.37.0
Root cause is unchanged: the kindest/node minor group moves the node image to v1.37.0 while the kubernetes/kubernetes patch group drags the images.txt component pins to v1.37.1, and the contract test treats the node image (topology pins v1.37.0 in mgmt/local-host/clusters/docker/cluster.yaml:21 and mgmt/local-host/clusters/management/cluster.yaml:24) as authoritative. Fix path: python3 airgap/tests/test-airgap-kubeadm-images.py --fix (reconciles the tracked tags and registry digests from kubeadm v1.37.0), or restructure the group so the component pins follow the node image. A uniform v1.37.1 is not possible yet: kindest/node v1.37.1 is a registry 404 (verified below).
-
Four customImage lines pair the new tag with the old digest: mgmt/local-host/clusters/docker/cluster-class.yaml:84 and :99, mgmt/local-host/clusters/management/cluster-class.yaml:89 and :104 read kindest/node:v1.37.0@sha256:099e049362..., and 099e0493... is the digest of v1.36.4 (registry-verified: v1.36.4 -> sha256:099e0493..., v1.37.0 -> sha256:a1ed56cfb0e7b93589bdf9738cd566405a265939e3620fc4f5de89adff580ae5, the digest images.txt:45-46 and the airgap scripts already carry). This is the tooling gap that #605 fixes (the custom manager's tag-only update). Unblock: land #605, then re-cut this PR so the manager re-captures the digest and the lines become v1.37.0@a1ed56cf...; no gate covers these lines, so finding 1 going green does not imply this is fixed.
-
Warning: the kubernetes-version group is still split. mise.toml:5 kubectl 1.37.1 and mgmt/local-talos/clusters/management/cluster.yaml:63 v1.37.1 versus kindest/node v1.37.0 everywhere else. With kindest/node v1.37.1 unpublished (registry 404), a uniform bump means 1.37.0 across the group; alternatively re-cut the whole PR as a uniform 1.37.1 once kindest publishes it.
-
Warning: stale v1.36.4 residue survives the rebase (line numbers from the current head): docs/operations.md:772 "The workload uses Kubernetes v1.36.4"; docs/airgap.md:138-139 archive filenames kindest_node_v1.36.4_mgmt.tar and kindest_node_v1.36.4.tar (build-package.sh now writes v1.37.0 names); docs/airgap.md:316 "v1.36.4 bakes registry.k8s.io/pause:3.10"; airgap/images.txt:50 section header "(v1.36.4 generation)"; docs/dependencies.md:126 "The repo currently pins Kubernetes v1.36.4"; comment residue in airgap/scripts/stage-and-create-cluster.sh:6 and airgap/scripts/build-config-artifact.sh:175.
Verified (commands + results):
- gh pr view 591 --json headRefOid -> 34aad82; local checkout of pull/591/head matches; diff vs origin/main c48201a is 10 files, 28 additions, 28 deletions.
- gh pr checks 591 -> only "Air-gap image digest pins" fail; the failing run 38925234002 is on head 34aad82 (verified via the actions/runs API head_sha).
- python3 airgap/tests/test-airgap-kubeadm-images.py at this head -> exit 1 with the same 12 mismatches as the job log.
- Docker Hub registry API for kindest/node: v1.36.4 -> sha256:099e049362..., v1.37.0 -> sha256:a1ed56cfb0..., v1.37.1 -> 404.
- git grep v1.36.4 at this head -> the residue list in finding 4.
- git show origin/main:tests/test-cert-manager-kubernetes-support.py -> notice-only gate with (1.21.2, 1.37.0) in VALIDATED_PAIRINGS; CI "Zarf mise asset pin" pass at this head.
Cross-reference: #605 (fix: capture kindest/node digest in Renovate custom manager; fresh review, all checks green) is the unblock for finding 2. Merge it, re-run this PR's bump so the manager re-captures the digest, then re-verify findings 1 and 2 at the new head.
| registry.k8s.io/etcd:3.6.8-0@sha256:397189418d1a00e500c0605ad18d1baf3b541a1004d768448c367e48071622e5 | ||
| registry.k8s.io/kube-scheduler:v1.36.4@sha256:7d84d58f091f16399673c87ccf579de50ac6ede39a2d6b82946c2dd5d7838ffc | ||
| registry.k8s.io/coredns/coredns:v1.14.7@sha256:7efd3c635b03efd68c4e8398fc45f0d993d0e9ab016f72c1cefb0fd6d01aa286 | ||
| registry.k8s.io/kube-apiserver:v1.37.1@sha256:e190f914a6cc21cab9268485d1ae7282fdeab1315ac64b668aa42d732fd1061d |
There was a problem hiding this comment.
Part of the 12 mismatches in the red "Air-gap image digest pins" check (run 38925234002, job 117054961038, same 12 in the local run of airgap/tests/test-airgap-kubeadm-images.py). This line is v1.37.1; kubeadm v1.37.0 (baked in the pinned kindest/node) deploys v1.37.0. The contract test treats the node image as authoritative (topology pins v1.37.0 in the two local-host cluster.yaml files), so the whole 1.37.1 component block (lines 33-38 and 57-62: coredns v1.14.7 vs v1.14.6, kube-apiserver/controller-manager/proxy/scheduler v1.37.1 vs v1.37.0, etcd 3.7.2-0 vs 3.7.0-0) must be reconciled to what kubeadm v1.37.0 deploys, tags and registry digests both. Fix path: python3 airgap/tests/test-airgap-kubeadm-images.py --fix. A uniform v1.37.1 is not possible yet: kindest/node v1.37.1 is a registry 404.
| backend: | ||
| docker: | ||
| customImage: kindest/node:v1.36.4@sha256:099e049362a1526b2db71494e1947aae99bd16290d7c895f2b7ea312e3cbfaed | ||
| customImage: kindest/node:v1.37.0@sha256:099e049362a1526b2db71494e1947aae99bd16290d7c895f2b7ea312e3cbfaed |
There was a problem hiding this comment.
New tag with the old digest: this line and line 99 read kindest/node:v1.37.0@sha256:099e0493..., and 099e0493... is the digest of v1.36.4 (Docker Hub registry-verified: v1.36.4 -> 099e0493..., v1.37.0 -> a1ed56cfb0e7b93589bdf9738cd566405a265939e3620fc4f5de89adff580ae5). This is the tag-only-update gap in the kindest custom manager that PR #605 fixes. Land #605 and re-cut this PR so the manager re-captures the digest (both lines must become v1.37.0@a1ed56cf...); no gate covers these lines, so the digest-pin check going green does not imply this is fixed.
| backend: | ||
| docker: | ||
| customImage: kindest/node:v1.36.4@sha256:099e049362a1526b2db71494e1947aae99bd16290d7c895f2b7ea312e3cbfaed | ||
| customImage: kindest/node:v1.37.0@sha256:099e049362a1526b2db71494e1947aae99bd16290d7c895f2b7ea312e3cbfaed |
There was a problem hiding this comment.
Same as the docker cluster class (inline comment there): lines 89 and 104 pair kindest/node:v1.37.0 with the v1.36.4 digest sha256:099e0493...; the v1.37.0 digest is sha256:a1ed56cfb0e7b93589bdf9738cd566405a265939e3620fc4f5de89adff580ae5. Blocked on #605 (digest capture in the custom manager) plus a re-cut of this PR.
) - extend "kindest/node image pins in management cluster manifests" matchStrings to capture @sha256 digest, so Renovate updates tag and digest atomically - remove autoReplaceStringTemplate so the default replacement applies - add both cluster-class.yaml files to test-renovate-coverage.py EXPECTED with a currentDigest assertion Without this, the Renovate kubernetes-version group PR updates the tag but leaves a stale digest in the management cluster-class files, breaking the air-gap artifact build and the kubeadm image gate. Co-authored-by: Claude Haiku 5.5 <[email protected]>
34aad82 to
9ee7514
Compare
|
Note 7 resources changed across 3 apps Blast radius
konflate 0.6.4 · rendered |
shrinedogg
left a comment
shrinedogg
left a comment
There was a problem hiding this comment.
REQUEST_CHANGES (hold): re-review at head 9ee7514 (unchanged since the wave capture). The digest-pin mismatch and the red required gate persist; the two digest findings from the previous review are resolved. Not merging.
Prior findings, item by item (CHANGES_REQUESTED review of 2026-10-17 at 34aad82):
- images.txt component pins vs what kubeadm v1.37.0 deploys (12 lines): STILL OPEN. Identical 12 mismatches at this head (finding 1 below).
- cert-manager vs k8s support gate: RESOLVED and stays resolved. tests/test-cert-manager-kubernetes-support.py at this head is notice-only ("Notice (never fail)... the upstream-window gate that blocked every bootstrap.toml PR was removed in #458").
- Stale v1.36.4 digest on the four customImage lines: RESOLVED at this head. All four read kindest/node:v1.37.0@sha256:a1ed56cf... (mgmt/local-host/clusters/docker/cluster-class.yaml:84 and :99, mgmt/local-host/clusters/management/cluster-class.yaml:89 and :104), and a1ed56cf... is the registry digest of v1.37.0. #605 (digest capture in the custom manager) merged 2026-10-18T04:40Z and this PR was re-cut after it.
- Docs and comment residue: STILL OPEN (warning 2 below), same set as the previous review, line numbers re-verified at this head.
- kubernetes-version group split (Talos v1.37.1, kubectl 1.37.1 vs kindest/node v1.37.0): STILL OPEN (warning 1 below).
Blocking finding:
- Required check "Air-gap image digest pins" is red at head 9ee7514 (run 39098022373; the run's head_sha verifies equal to 9ee7514). Every other check at this head is green. The 12 mismatches, verbatim from the job log:
- airgap/images.txt:33 and :57 coredns/coredns:v1.14.7, kubeadm v1.37.0 deploys v1.14.6
- airgap/images.txt:34 and :58 kube-apiserver:v1.37.1, deploys v1.37.0
- airgap/images.txt:35 and :59 kube-controller-manager:v1.37.1, deploys v1.37.0
- airgap/images.txt:36 and :60 kube-proxy:v1.37.1, deploys v1.37.0
- airgap/images.txt:37 and :61 etcd:3.7.2-0, deploys 3.7.0-0
- airgap/images.txt:38 and :62 kube-scheduler:v1.37.1, deploys v1.37.0
Root cause unchanged: the kindest/node minor group pins the node image to v1.37.0 (mgmt/local-host/clusters/docker/cluster.yaml:21, mgmt/local-host/clusters/management/cluster.yaml:24, airgap/images.txt:45-46) while the kubernetes/kubernetes patch group drags the images.txt component pins to v1.37.1, and the gate treats the node image's kubeadm as authoritative. A lockstep 1.37.1 set cannot land today: kindest/node has no v1.37.1 tag. Docker Hub tags matching "1.37" are exactly [v1.37.0, v1.37.0-rc.1] (checked 2026-10-20). Unblock condition: kindest/node v1.37.1 published upstream, then a Renovate re-cut or manual re-bundle of the whole kubernetes-version group at 1.37.1; until then the component pins must follow kubeadm v1.37.0 (python3 airgap/tests/test-airgap-kubeadm-images.py --fix reconciles the tracked tags and registry digests from kubeadm v1.37.0).
Warning 1 (non-blocking): the kubernetes-version group remains split at this head: mise.toml:5 kubectl 1.37.1 and mgmt/local-talos/clusters/management/cluster.yaml:63 TalosControlPlane v1.37.1 versus kindest/node v1.37.0 everywhere else. The environments are intentionally independent and the kind side is capped by the missing node image; converge them in the 1.37.1 re-cut.
Warning 2 (non-blocking): stale v1.36.4 residue survives at this head (line numbers from the head): airgap/images.txt:50 section header "(v1.36.4 generation)" above 1.37.1 lines; docs/operations.md:772 "The workload uses Kubernetes v1.36.4"; docs/airgap.md:138-139 archive names kindest_node_v1.36.4_mgmt.tar and kindest_node_v1.36.4.tar (build-package.sh:109-110 now writes v1.37.0 names); docs/airgap.md:316; docs/dependencies.md:126; comment residue airgap/scripts/stage-and-create-cluster.sh:6 and airgap/scripts/build-config-artifact.sh:175. Sweep these in the same re-cut.
Artifact verification (everything else about the bump checks out):
- Every registry.k8s.io pin in this PR exists and its tag resolves to exactly the pinned digest (registry HEAD, checked 2026-10-20): kube-apiserver v1.37.1 -> sha256:e190f914..., kube-controller-manager v1.37.1 -> sha256:d470c1b8..., kube-proxy v1.37.1 -> sha256:b9ae667e..., kube-scheduler v1.37.1 -> sha256:b6e2474a..., coredns/coredns v1.14.7 -> sha256:7efd3c63..., etcd 3.7.2-0 -> sha256:7c6c2398... Both sections carry identical digests.
- kindest/node:v1.37.0 -> sha256:a1ed56cf... (Docker Hub tag API), identical to every node pin in the PR (images.txt:45-46, both cluster-class files, all three scripts).
- kubernetes/kubernetes v1.37.1 exists (git tag d2b770f4...), so the Talos and kubectl pins reference a real release.
- mise.toml parses (tomllib, kubectl = 1.37.1); YAML validity at this head is covered by green CI (Lint YAML, Build kustomize overlays).
- CI "Air-gap image existence" is green at this head: every images.txt digest is pullable, so the partial bump would not break the bundle build, only the version contract.
Release notes: Kubernetes v1.36.4 -> v1.37.0 (CHANGELOG-1.37, checked 2026-10-20): kubeadm v1beta3 config API removed (k8s#136016) with no hand-written kubeadm configs in this repo (grep clean); kubeadm now explicitly sets KubeProxyConfiguration.mode=iptables and kube-proxy warns when unset (k8s#139777, benign for kind); default etcd updated to 3.7.0 (k8s#140333), consistent with the 3.7.x pin trajectory but kubeadm v1.37.0 deploys 3.7.0-0, not the pinned 3.7.2-0; kubeadm passes graduated etcd flags instead of the experimental ones removed in etcd 3.7 (k8s#132838). None of these interact with the repo's pinned topology beyond finding 1.
Verified (commands and results):
- gh pr view 591 --json headRefOid -> 9ee7514 (unchanged since capture; this head was reviewed).
- gh pr checks 591 -> only "Air-gap image digest pins" fail; gh api actions/runs/39098022373 -> head_sha 9ee7514..., conclusion failure.
- gh run view 39098022373 --log-failed -> the 12 verbatim mismatches quoted above, target "kindest/node: v1.37.0".
- airgap/images.txt read at the head: lines 33-38 and 57-62 carry the 1.37.1-generation tags; lines 45-46 carry kindest/node v1.37.0@a1ed56cf.
- Docker Hub API: /repositories/kindest/node/tags?name=1.37 -> [v1.37.0, v1.37.0-rc.1]; /tags/v1.37.0 -> sha256:a1ed56cf...
- registry.k8s.io HEAD per tag: six of six digests match the pins exactly (values above).
- git ls-remote kubernetes/kubernetes v1.37.1 -> d2b770f4c94636a992534a8d3156b6b2d8e82ac3.
- mise exec -- python3 -c tomllib.load(mise.toml) -> OK, kubectl 1.37.1.
- grep at the head for kindest/node:v1.37.0@ -> all 10 pin sites carry a1ed56cf, no old-digest stragglers; grep for kubeadm.k8s.io apiVersion -> no hand-written kubeadm configs.
Cross-reference: #605 (merged) fixed the custom manager's digest capture and resolved prior finding 3 at this head. The remaining unblock is upstream only (kindest/node v1.37.1) plus the re-cut that follows it.
| registry.k8s.io/etcd:3.6.8-0@sha256:397189418d1a00e500c0605ad18d1baf3b541a1004d768448c367e48071622e5 | ||
| registry.k8s.io/kube-scheduler:v1.36.4@sha256:7d84d58f091f16399673c87ccf579de50ac6ede39a2d6b82946c2dd5d7838ffc | ||
| registry.k8s.io/coredns/coredns:v1.14.7@sha256:7efd3c635b03efd68c4e8398fc45f0d993d0e9ab016f72c1cefb0fd6d01aa286 | ||
| registry.k8s.io/kube-apiserver:v1.37.1@sha256:e190f914a6cc21cab9268485d1ae7282fdeab1315ac64b668aa42d732fd1061d |
There was a problem hiding this comment.
Blocking: this pin and its 11 siblings (:33-38 and :57-62) disagree with what kubeadm v1.37.0 deploys (kube-apiserver v1.37.0), and the gate treats the node image's kubeadm as authoritative, so the required check is red at this head. The digest itself is correct (registry HEAD: v1.37.1 -> sha256:e190f914...). Root cause: kindest/node has no v1.37.1 tag ([v1.37.0, v1.37.0-rc.1] only), so the kubernetes/kubernetes patch group ran ahead of the kind group. Unblock: kindest/node v1.37.1 published upstream, then re-cut the lockstep 1.37.1 set; or follow node v1.37.0 via test-airgap-kubeadm-images.py --fix.
| registry.k8s.io/kube-proxy:v1.36.4@sha256:b33fcdd8319290736e566f5fc2225ca07d3c2dfc025fb006342791aa31be4603 | ||
| registry.k8s.io/etcd:3.6.8-0@sha256:397189418d1a00e500c0605ad18d1baf3b541a1004d768448c367e48071622e5 | ||
| registry.k8s.io/kube-scheduler:v1.36.4@sha256:7d84d58f091f16399673c87ccf579de50ac6ede39a2d6b82946c2dd5d7838ffc | ||
| registry.k8s.io/coredns/coredns:v1.14.7@sha256:7efd3c635b03efd68c4e8398fc45f0d993d0e9ab016f72c1cefb0fd6d01aa286 |
There was a problem hiding this comment.
Suggestion (non-blocking): the section header above this block, airgap/images.txt:50, still reads "Workload node containerd store [W] (v1.36.4 generation)" while the block now pins 1.37.1-generation tags. Update the header in the same re-cut (same for docs/operations.md:772, docs/airgap.md:138-139 and :316, docs/dependencies.md:126, and the comments at airgap/scripts/stage-and-create-cluster.sh:6 and airgap/scripts/build-config-artifact.sh:175).
This branch has not been deployed
2 participants
This PR contains the following updates:
v1.36.4→v1.37.01.36.4→1.37.01.37.0→1.37.1v1.14.2→v1.14.73.6.8-0→3.7.2-0v1.36.4→v1.37.1v1.36.4→v1.37.1v1.36.4→v1.37.1v1.36.4→v1.37.1Warning
Some dependencies could not be looked up. Check the Dependency Dashboard for more information.
Release Notes
kubernetes/kubernetes
v1.37.1Compare Source
See kubernetes-announce@. Additional binary downloads are linked in the CHANGELOG.
See the CHANGELOG for more details.
Configuration
📅 Schedule: (in timezone Europe/Helsinki)
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
👻 Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.
This PR was generated by Mend Renovate. View the repository job log.