Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (8)
Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review. 📝 WalkthroughWalkthroughThe pull request adds SDN17-02 for Kubernetes DRA-based IMEX validation. The new check inspects compute-domain device classes, GPU nodes, NVLink labels, and ResourceSlices. The Network suite, tests, requirement mappings, and test plans now include this validation. ChangesKubernetes DRA IMEX validation
Priority: ⬇️ Low Estimated code review effort: 3 (Moderate) | ~25 minutes Change: Refactor Sequence Diagram(s)sequenceDiagram
participant NetworkSuite
participant ImexComputeDomainCapabilityCheck
participant KubernetesAPI
NetworkSuite->>ImexComputeDomainCapabilityCheck: run SDN17-02 validation
ImexComputeDomainCapabilityCheck->>KubernetesAPI: read cluster capability and resources
KubernetesAPI-->>ImexComputeDomainCapabilityCheck: return device classes, nodes, and ResourceSlices
ImexComputeDomainCapabilityCheck-->>NetworkSuite: return validation result
Suggested reviewers: Merge Risk: ⚪ Minimal · up to No actionable merge-blocking issue remains in the Kubernetes DRA validation or its suite and traceability integration. 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
…17-02) SDN17-02 is the Kubernetes half of SDN17: the same "IMEX without tenant installation" requirement as SDN17-01, read as cluster objects rather than as packages on a host. ImexComputeDomainCapabilityCheck asserts two cluster-side facts - the resource-allocation driver's compute-domain device classes are registered cluster-wide, and every GPU node in the NVLink clique publishes compute-domain resources through the driver's per-node plugin. Publication is asserted from the published resource, not from a running pod: a ResourceSlice for the compute-domain driver exists only once that node's plugin registered with the node agent and published successfully, which a pod in a Running state does not establish. Scope never comes from a node's self-report. The step reports every GPU node the cluster accounts for, and a GPU node arriving without the NVLink clique label fails rather than dropping out of the asserted set - otherwise a node opts itself out of being tested. An empty clique-labelled set fails for the same reason, so the check cannot succeed vacuously on a cluster with no NVLink nodes at all. The asserted-node count is recomputed in the check rather than read from `nodes_validated`, so a run that examines eight nodes and validates none of them fails instead of reporting eight checked and passing. Nothing is asserted about any host IMEX daemon. The driver supports two ownership modes and an active host daemon is a defect under one and a requirement under the other, so `daemon_ownership_mode` is carried as evidence only - both modes pass. - Validation wired into suites/network.yaml (fabric_topology group, next to SDN17-01), gated on kubernetes so a vm/bare_metal run of the suite neither runs it nor provisions a cluster for it. Labels synced into docs/test-plan.yaml and the rendered adoc, with the plan item and traceability matrix entry added - shared/network/imex_compute_domain_test.py: a real provider-neutral reference. Everything is read from the cluster API - nothing shells into a node and nothing invokes IMEX tooling, which would need the IMEX command service enabled and so would fail on a correctly configured cluster. Device classes and the per-node driver are matched by role rather than exact name, since those names move with the driver version while the role does not. A cluster advertising no multi-node NVLink capability (the ComputeDomain CRD is absent) emits a structured skip; that gate is deliberately a different object from the device classes the check asserts on, so the assertion cannot certify itself. An unreadable API is a hard failure rather than that skip, since an unserved API group answers with an empty successful listing - aws and my-isv wire the shared reference as-is, with no arguments: a node list on the command line would hand the choice of what gets tested back to the provider - output_schemas.py: registers the imex_compute_domain step schema - Unit tests covering pass under both ownership modes, unregistered device classes, an unpublished node, absent contract fields not reading as true, zero asserted nodes, an empty node list, a clique-less GPU node staying in scope, the reported validated count not being trusted, malformed input, and the skip; plus script tests for the cluster reads, role matching, GPU-node scoping on both DRA-only and device-plugin clusters, the skip gate, the unreachable-cluster failure, and the bounded kubectl invocation Signed-off-by: Cursor Agent <[email protected]> Co-authored-by: Alexandre Begnoche <[email protected]>
Cleanups from a quality review of the SDN17-02 implementation; no behavior change beyond the step timeout. Extract `_validate_node_reports` for the per-node shape guard that was restated verbatim by three IMEX checks, keeping every error string identical. Drop the `clique_nodes` list carried over from `ImexServicePresenceCheck`, where the filter narrows the asserted set - here it narrows nothing, so `len(clique_nodes)` was always `len(nodes)`. In the probe, remove the argparse machinery: the script takes no arguments by design, and `ProviderArgumentParser` was a third verbatim copy. Replace the `by_name` dedupe in `_scoped_nodes` with a filtered sort, since node names are unique in a listing. Raise the step timeout above the probe's own worst case (4 bounded kubectl calls x 30s), so a degraded cluster yields the script's structured failure JSON instead of being killed mid-run. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Signed-off-by: Alexandre Begnoche <[email protected]>
…ctly SDN17-02 asserts only cluster objects - the driver's compute-domain device classes and the ResourceSlices its per-node plugin publishes. Nothing about it is a provider's to perform or to report, which the step-backed form made plain: the script took no arguments so a provider could not choose what gets tested, and both provider wirings invoked the same shared reference with nothing to fill in. A step exists so a script can do what a validation cannot - cloud SDK calls, SSH, mutation. SDN17-01 and SDN21-01 are script-backed because they SSH into nodes; this check runs four read-only kubectl reads, which is what the live-probe validations in the k8s suite already do. Move the probe into ImexComputeDomainCapabilityCheck, reading through isvtest.core.k8s so the check picks up the KUBECTL override, K8S_PROVIDER microk8s/k3s detection, and in-cluster kubeconfig handling that the copied kubectl layer had already drifted from. The CRD-absence gate now skips directly instead of round-tripping a `skipped` payload through JSON. Removes the shared script, its schema entry, and both provider wirings; every provider with a cluster now gets the check without wiring anything. Assertion semantics are unchanged: scope is still the cluster's own GPU accounting, an unlabelled GPU node still fails rather than leaving the set, zero asserted nodes is still a failure, and the daemon ownership mode is still evidence only. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Signed-off-by: Alexandre Begnoche <[email protected]>
abegnoche
force-pushed
the
sdn17-02-compute-domain-live-probe
branch
from
September 14, 2026 17:32
d16ed88 to
a7a8746
Compare
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
abegnoche
marked this pull request as ready for review
September 14, 2026 19:35
🔐 TruffleHog Secret Scan✅ No secrets or credentials found! Your code has been scanned for 700+ types of secrets and credentials. All clear! 🎉 🕐 Last updated: 2026-09-14 19:36:28 UTC | Commit: a7a8746 |
marranagu
approved these changes
Sep 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements SDN17-02. Closes #598.
What it asserts
On a cluster that offers multi-node NVLink through the NVIDIA DRA driver, IMEX must be available without the tenant installing anything.
ImexComputeDomainCapabilityCheckreads that as cluster objects:compute-domain*.nvidia.comdevice classes are registered cluster-wide, andResourceSlices through the driver's per-node plugin.Scope is the cluster's own GPU accounting, so a GPU node arriving without the clique label fails rather than dropping out of the set, and zero asserted nodes fails rather than passing vacuously. A cluster with the ComputeDomain CRD unregistered is out of scope and skips; an unreadable API group fails instead. Host IMEX daemon state is reported as evidence only, since the driver's two ownership modes disagree on whether one should be running.
Changes
isvtest/src/isvtest/validations/network.py: the check, reading the cluster throughisvtest.core.k8s(four read-onlykubectlcalls, no step and no provider wiring, so every provider with a cluster gets it).isvctl/configs/suites/network.yaml: suite entry,requires: [kubernetes].Testing
20 unit tests against a canned cluster cover skip, failure, and pass paths.
make test,make lint,make demo-test, anduvx pre-commit run -apass.Verified live on a GB200 cluster: skipped before the DRA compute-domain driver was installed, failed naming a clique GPU node that published no ResourceSlice, and passed once every remaining GPU node published (ownership reported as
driver).