Repository navigation
fix: honour optional on ConfigMap and Secret volumes - #590
Open
NopeCharon wants to merge 1 commit into
Open
NopeCharon wants to merge 1 commit into
NopeCharon wants to merge 1 commit into
Conversation
A configMap or secret volume marked optional whose object does not exist was treated like a required one: remoteExecutionHandleVolumes kept the pod in Initializing, polled for the object every 2s and, after 5 minutes, failed the pod with "CFGMaps/Secrets not found". A configMap source of a projected volume failed the pod right away, also when it was optional or only one of its listed items was missing. The kubelet mounts an empty volume for a missing optional object, and skips the missing keys of an optional projected source. Do the same: - configMap / secret volumes: on NotFound for an optional object, send the plugin an empty ConfigMap/Secret with that name instead of waiting. Plugins look retrieved objects up by name (the Slurm plugin fails the mount when one is absent), so an empty object is what yields an empty mount there. - projected configMap sources: an optional ConfigMap that is missing projects no keys, and missing keys listed in items are skipped. Other errors, and objects that are not optional, keep the existing behaviour. Fixes interlink-hq#589 Signed-off-by: NopeCharon <[email protected]>
✅ Deploy Preview for interlink-dev canceled.
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
On a virtual node, a pod that mounts a ConfigMap or Secret volume marked
optional: truecan't start when that object doesn't exist. The virtual kubelet treats the volume as required. It keeps the pod inInitializing, polls for the object every 2s, and after 5 minutes fails the pod withCFGMaps/Secrets not found. AconfigMapsource of a projected volume fails the pod right away, also when the source is optional or only one of its listeditemsis missing. On a regular node the kubelet mounts an empty volume and the pod runs, so workloads that rely on optional config (common in Helm charts) break only when they are offloaded.This PR handles
optionalthe way the kubelet does. Only a NotFound error counts, so any other error, and any non-optional object, keeps the current wait-and-retry behaviour:configMap/secretvolumes: when an optional object is missing, the plugin gets an empty ConfigMap/Secret with the same name instead of nothing. Plugins look up retrieved objects by name. For example, the Slurm plugin'sprepareMountserrors out when the object is absent, but it mounts an empty object as an empty volume without error.configMapsources: an optional ConfigMap that is missing adds no keys, and missing keys listed initemsare skipped.Tests:
TestRemoteExecutionHandleVolumesOptionalMissing: missing optional ConfigMap and Secret volumes give empty objects with the right name and namespace, and the pod moves toPending. Without the fix this test sits in the 5-minute wait.TestRemoteExecutionHandleProjectedSourceConfigMap: new cases for a missing optional ConfigMap, a missing key of an optional ConfigMap, and (for contrast) a missing required ConfigMap, which still returns an error.go test -race ./pkg/...andgolangci-lint run(v2.14.0, as in CI) pass.Related issue : #589