Skip to content

[self-managed] Include overridden Pylon image in stack release inventory #2139

Description

@sbaum1994

Describe the bug

The generated self-managed stack release inventory omits the Pylon worker sidecar selected by the stack when the LLM add-on is enabled.

The helm-nvcf-api chart defines worker sidecar images under api.remoteConfig.configData.nvcf.sidecars. Most references use ${nvcf.sidecars.hostname} and ${nvcf.sidecars.repository} placeholders. The chart helper resolves those placeholders and emits release-artifact-*-image annotations that inventory creation detects.

The self-managed stack replaces the API chart's Pylon placeholder reference with the concrete effective reference <sidecar-host>/<sidecar-repository>/pylon:0.18.0. The chart helper only emits an annotation for values that still contain ${nvcf.sidecars.*}, so it skips the concrete Pylon value. The inventory scanner also does not parse the embedded nvcf-api.yaml payload in the remote-config ConfigMap.

As a result, pylon:0.18.0 is used for newly created LLM function versions but is absent from nvcf-self-managed-stack-inventory.json.

Steps or code to reproduce bug

  1. Download the inventory published for self-managed stack 1.0.1:
curl -sSL \
  https://github.com/NVIDIA/nvcf/releases/download/deploy%2Fstacks%2Fself-managed%2Fv1.0.1/nvcf-self-managed-stack-inventory.json |
  jq '.artifacts[] | select(.name == "pylon")'

The query returns no artifact.

  1. Inspect the self-managed stack's Pylon override in global.yaml.gotmpl. With addons.llm.enabled=true, it constructs a concrete pylon:0.18.0 reference.

  2. Inspect the API chart's nvcf-api.sidecarReleaseArtifacts helper. It emits inventory annotations only when a sidecar value contains ${nvcf.sidecars.*}.

The full inventory profile enables addons.llm, so Pylon should be present in the rendered release inventory.

Expected behavior

The self-managed stack inventory includes the exact Pylon image selected by the stack:

nvcr.io/nvidia/nvcf/pylon:0.18.0

It should remain classified as optional because Pylon is used only when the LLM add-on is enabled.

Additional context

Suggested fix:

  • Extend nvcf-api.sidecarReleaseArtifacts to emit annotations for both placeholder-based references and already concrete image references.
  • For placeholder references, retain the current hostname and repository substitution.
  • For concrete references, validate that the value is a resolved image reference and emit it unchanged.
  • Add a chart test covering a concrete llm-router-client-image override.
  • Add or update inventory coverage to assert that the self-managed full profile contains Pylon and classifies it as optional.
  • Once the generated inventory is authoritative for Pylon, remove or simplify the source-only Pylon pin fallback in tools/docs-version-sync/stack_source.go.

Related to #1831.


By submitting this issue, you agree to follow our code of conduct and our contributing guidelines.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs-triageIssue or PR awaiting maintainer triage

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions