Skip to content

feat(dashboard): add Cluster overview with platform telemetry, restricted to root - #255

Closed
wesleip wants to merge 4 commits into
virtfoundry:mainfrom
wesleip:dashboard/cluster-overview
Closed

feat(dashboard): add Cluster overview with platform telemetry, restricted to root#255
wesleip wants to merge 4 commits into
virtfoundry:mainfrom
wesleip:dashboard/cluster-overview

Conversation

@wesleip

@wesleip wesleip commented Oct 9, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Summary

Add a Cluster overview section to the dashboard that surfaces four new platform-wide telemetry fields on GET /api/v1/dashboard/summary:

  • hosts — node count, ready nodes, allocatable CPU and memory (millicores / bytes), distinct kubelet versions, OS images and architectures. Sourced from core/v1/nodes via KubeVirtDriver.ClusterMetrics.
  • hosts.usage — real CPU and memory consumption aggregated from metrics.k8s.io/v1beta1/nodes (Tier 2 — requires metrics-server, documented prerequisite in helm-charts). Served with a 30s in-memory cache, fail-soft on 404/403/5xx/timeout.
  • storage — tenant-scoped PVC allocated capacity (bound_bytes = spec.requests sum of Bound PVCs vs total_bytes reserved) plus the count. Never real guest consumption — the API cannot see filesystem usage. Sourced from the existing PVC list path in KubeVirtDriver.
  • addons — cluster-wide health of critical dependencies, discovered dynamically from CustomResourceDefinitions listed metadata-only (PartialObjectMetadataList Accept header, 4s call timeout — names and labels only, never the OpenAPI schemas). Resolved via the Helm label app.kubernetes.io/name with fallback to part-of / component, plus a small alias table over the API group derived from the CRD object name (<plural>.<group>): kubevirt.io, cdi.kubevirt.io, k8s.cni.cncf.io, cert-manager.io, istio.io, monitoring.coreos.com, tekton.dev, argoproj.io. Any chart that follows the convention shows up automatically — no hard-coded list.

The UI renders a Cluster overview card with four tiles (Nodes, Ready nodes, Allocatable CPU, Allocatable memory), a usage bar on the capacity tiles with a < 60% primary / 60–85% warning / ≥ 85% error colour ramp, a Storage tile in the stats row ("{bound} of {total} allocated"), and a Cluster addons strip with one dot per discovered addon. The dashboard polls every 10s when /ws/events is up and 5s when it is down, so the usage bars and the addons strip stay current without a manual refresh.

Security: the entire section is now gated to platform.RoleRoot at both layers — the service omits the hosts, storage and addons fields from the JSON response for any non-root actor, and the React side hides the card when selectIsRoot is false. Tenant admins and tenant users keep the rest of the dashboard (volumes, VPCs, SGs, networks, recent VMs, health badge) but the platform-wide telemetry is gone — both from the rendered page and from the JSON response.

Four commits on dashboard/cluster-overview:

  1. feat(dashboard): add cluster overview backend telemetry — types, four driver methods, service injection, RBAC contract dynamicClient section, design doc
  2. feat(ui): render cluster overview in the dashboard — UI, i18n, polling
  3. feat(security): restrict the Cluster overview section to root — role check, hidden field, hidden card
  4. fix(dashboard): address cluster overview review feedback — usage fail-soft at the service layer, panic-free quantity parsing, bounded metadata-only addons probe, bound_bytes rename, no mutex copy in WithNamespace

Test plan

  • go test ./... — all packages pass, 0 failures
  • cd ui && npm test (vitest) — 5 files, 27 tests, 0 failures
  • cd ui && npx tsc --noEmit — 0 errors new to this branch (pre-existing isFetching unused warning unrelated)
  • rbac_contract_test passes — the dynamicClient entries (metrics.k8s.io/nodes, apiextensions.k8s.io/customresourcedefinitions) in docs/rbac-contract.yaml are documented for the chart; the AST scan does not see Discovery().RESTClient()
  • Tested on a local kind cluster with metrics-server installed (Tier 1 + Tier 2 verified)
  • Verified as root: hosts, storage, addons populated, all dots green
  • Verified as tenant_admin: section hidden, hosts / storage / addons absent from the JSON response
  • CHANGELOG entry under Unreleased (Added / Changed / Security)
  • Cluster deploy from helm-charts: feat(chart): grant the API read access for the dashboard cluster overview helm-charts#141 — API ClusterRole grants apiextensions.k8s.io/customresourcedefinitions and metrics.k8s.io/nodes get/list (rules first per RBAC ordering). MINOR bump of core + helm-charts together per RELEASES.md stays with the release PR.
  • Wiki updated (homelab ops, metrics-server requirement) — follow-up in helm-charts docs

Conventions

  • Commit messages are in English and follow Conventional Commits (feat(dashboard): …, feat(ui): …, feat(security): …)
  • Branch name is descriptive (dashboard/cluster-overview)
  • No direct commits to main; PR is opened from a feature branch
  • Cross-repo coordination noted — helm-charts change is a follow-up; version bump will land together per RELEASES.md
  • Backward compatible: all new fields are additive (omitempty); older clients keep working
  • Local equivalent of required CI checks: go build ./..., go test ./..., npm run build
  • Homelab gate per AGENTS.md — change was deployed to a local kind cluster and verified

wesleip added 3 commits October 9, 2026 16:26
@wesleip
feat(dashboard): add cluster overview backend telemetry
b740708 
Extend GET /api/v1/dashboard/summary with three new additive fields
that surface platform-wide telemetry for the operator:

  - hosts:    node count, ready nodes, allocatable CPU and memory
              (millicores / bytes), distinct kubelet versions, OS images
              and architectures. Sourced from core/v1/nodes via
              KubeVirtDriver.ClusterMetrics.
  - hosts.usage: real CPU and memory consumption aggregated from
              metrics.k8s.io/v1beta1/nodes (Tier 2 — requires
              metrics-server, documented prerequisite in helm-charts).
              Served with a 30s in-memory cache, and degrades gracefully
              when the API is absent, Forbidden, or transiently
              unavailable.
  - storage:  tenant-scoped PVC capacity and consumption (Bound vs
              reserved) plus the count. Sourced from the existing PVC
              list path in KubeVirtDriver.
  - addons:   cluster-wide health of critical dependencies
              (KubeVirt, CDI, Multus, metrics-server, networking, and
              any chart that follows the Helm label conventions),
              discovered dynamically from CustomResourceDefinitions
              with the 'app.kubernetes.io/name' label and a small alias
              table for the well-known API groups. Each entry reports
              'ok' (Established) or 'degraded'.

All four reads are fail-soft: 404, 403, 401, 503, timeouts, and
connection-refused return (nil, nil) so the endpoint never 500s.
Memory mode and nil client leave the fields empty.

docs/rbac-contract.yaml documents the dynamic-client access to
metrics.k8s.io in a new dynamicClient section — rbac_contract_test
keeps passing because the typed client AST scan does not see
Discovery().RESTClient().

PlatformService gains a kv field so the service can read the
hypervisor. CHANGELOG entry under Unreleased. A new
2026-10-08-dashboard-host-metrics.md captures the design and the
locked decisions for this slice.
@wesleip
feat(ui): render cluster overview in the dashboard
d8ea32b 
Surface the new hosts/storage/addons fields from the previous slice
on the existing Dashboard page:

  - Rename the section title from 'Cluster nodes' to 'Cluster
    overview' to fit the broader scope (nodes + capacity + addons).
  - Four cards inside the section: Nodes, Ready nodes, Allocatable
    CPU, Allocatable memory. The CPU/memory cards now carry a
    usage bar (used / allocatable) and a percent line, with a
    colour ramp on 60% / 85% thresholds. A small 'Todos prontos' /
    'N degraded' / 'None ready' badge sits next to the Ready
    nodes count.
  - The four cards share a common baseline via h-full + flex-1 so
    the percent and the bars line up at the bottom regardless of
    how much content the other cards carry.
  - A new 'Storage' card replaces the redundant 'Networks' tile in
    the stats row, formatted with auto-scaled units (B / KiB / MiB
    / GiB / TiB) and a percent line.
  - A 'Cluster addons' strip below the Kubelet / OS / Arch footer
    renders one dot per critical dependency, green and pulsing
    when Established, an outlined ring when absent, yellow and
    pulsing when unreachable. The list is discovered dynamically
    by the backend — only addons actually present on the cluster
    are shown.
  - The dashboard query polls every 10s when /ws/events is up and
    5s when it is down, so the usage bars and the addons strip
    stay current without a manual refresh. The existing
    realtime-invalidation path still drives VM transitions.

The polling is on a fixed schedule rather than the previous
'health == ok' gate because the underlying data sources
(metrics.k8s.io, the addon probe) have no /ws/events counterpart.

i18n: PT and EN parity for every new key, asserted by i18n.test.ts.
typecheck: tsc --noEmit reports 0 errors new to this commit (the
'isFetching' unused warning is pre-existing).
@wesleip
feat(security): restrict the Cluster overview section to root
63dcc8d 
The dashboard Cluster overview (node counts, capacity, addons) is
platform-wide telemetry, not tenant-scoped. Hide it from any actor
whose role is not 'root'.

Two layers:

  1. service.DashboardSummary now takes a role argument. When the
     role is anything other than platform.RoleRoot, the service
     skips populateHosts, populateStorage and populateAddons; the
     JSON response carries no 'hosts', 'storage' or 'addons' field,
     so nothing leaks through the Network tab or other consumers.
  2. The Dashboard React component gates the entire <SurfaceCard>
     with selectIsRoot, so non-root never even renders the card.

Tenant admins and tenant users still see the rest of the dashboard:
volume / VPC / SG / network counts, recent activity, the VM
running% badge, and the health card. Only the platform-wide
telemetry is gone.

The handler passes actor.Role (empty string when unauthenticated,
which is also not root). Two new tests
(TestDashboardSummaryHidesClusterOverviewForNonRoot and
TestDashboardSummaryShowsClusterOverviewForRoot) lock the
behaviour; the existing hosts/storage/addons tests now pass
platform.RoleRoot explicitly so they keep exercising the populated
path.
Matheus-Thurler
Matheus-Thurler reviewed Oct 11, 2026
View reviewed changes

@Matheus-Thurler Matheus-Thurler left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good structure: root-only gating in both service and UI, 30s cache and fail-soft on metrics. Points to address:

  1. RBAC is missing in the chart. The API ClusterRole in helm-charts/charts/virtfoundry/templates/rbac.yaml has no apiextensions.k8s.io/customresourcedefinitions (get/list) nor metrics.k8s.io/nodes (get/list). Without them the addons strip is always empty and usage is absent, silently, since Forbidden is swallowed. Please add the rules (and docs/rbac-contract.yaml entries) in a helm-charts PR, rules first per our RBAC ordering.
  2. populateHosts can 500 the whole dashboard. If ClusterUsage returns an error that is not caught by isMetricsAbsent, DashboardSummary fails for root. Usage is optional: log and leave Hosts.Usage nil instead of returning the error.
  3. resource.MustParse can panic on a malformed quantity from metrics.k8s.io. Use resource.ParseQuantity and skip the item on error.
  4. probeAddons lists every CRD with full schemas and has no timeout of its own. Please add a call timeout like the usage call, and request metadata only if possible (PartialObjectMetadataList Accept header).
  5. Storage "used" is not consumption. It sums requests of Bound PVCs, which is allocated capacity. Rename it (allocated/bound) or the UI label, so it is not read as real usage.

@wesleip
fix(dashboard): address cluster overview review feedback
79c6ed1 
- populateHosts: cluster usage is optional — an unexpected ClusterUsage
  error is logged and leaves hosts.usage nil instead of failing the
  whole dashboard summary (500) for root.
- fetchClusterUsage: parse node quantities with resource.ParseQuantity
  and skip the item on error; resource.MustParse could panic on a
  malformed metrics.k8s.io payload.
- probeAddons: bound the CRD list with a 4s timeout (same bound as the
  usage call) and request a PartialObjectMetadataList so only names and
  labels cross the wire, not full OpenAPI schemas. The API group is
  derived from the CRD object name (<plural>.<group>); Established is
  not part of ObjectMeta, so a discovered addon is now ok by presence.
  The apiextensions.k8s.io/customresourcedefinitions get/list grant is
  listed under dynamicClient in docs/rbac-contract.yaml.
- StorageSummary: rename used_bytes to bound_bytes and reword the UI
  label to "allocated" — the figures are spec.requests sums, i.e.
  allocated capacity, not real guest consumption.
- WithNamespace: build a fresh driver instead of copying the struct —
  the copy cloned the new cache mutexes (go vet copylocks) while
  sharing the cache entries.

Chart grants land in virtfoundry/helm-charts#141 (rules first).
@wesleip

wesleip commented Oct 11, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks for the review, @Matheus-Thurler — all five points addressed in 79c6ed1, pushed and ready for another look.

  1. RBAC in the chart — opened feat(chart): grant the API read access for the dashboard cluster overview helm-charts#141 (rules first): API ClusterRole now grants apiextensions.k8s.io/customresourcedefinitions get/list and metrics.k8s.io/nodes get/list, both read-only, with README + CHANGELOG entries. On the core side, the CRD grant was also missing from the contract — added under dynamicClient in docs/rbac-contract.yaml (metrics.k8s.io/nodes was already there). helm template + helm lint + security-gates.sh all pass.
  2. populateHosts 500 — fixed: usage is optional. Any ClusterUsage error not absorbed by isMetricsAbsent is now logged ([dashboard] cluster usage unavailable) and leaves hosts.usage nil; the summary no longer fails for root. Covered by TestDashboardSummaryUsageErrorDoesNotFail (undecodable metrics body → 200 with usage nil).
  3. resource.MustParse panic — replaced with resource.ParseQuantity; a node with a malformed quantity is skipped with a log line instead of panicking or failing the call. Covered by TestClusterUsageSkipsMalformedQuantities.
  4. probeAddons payload/timeout — the CRD list now has a 4s call timeout (same bound as the usage call) and requests PartialObjectMetadataList via the Accept header, so only names and labels cross the wire. spec.group is derived from the CRD object name (<plural>.<group>), which made metadata-only possible. One trade-off to flag: status.conditions is not part of ObjectMeta, so Established is no longer observable — a discovered addon is reported ok by presence rather than distinguishing degraded. Given the strip only renders ok dots anyway and the payload saving is significant on operator-heavy clusters, this looked like the right call; happy to revisit if you'd rather keep the Established signal.
  5. Storage "used" — renamed end to end: used_bytes → bound_bytes in the API, and the tile label now reads "allocated" ("{bound} of {total} allocated" / "{bound} de {total} alocados") so it is not read as real consumption. Go doc comments state both figures are spec.requests sums.

Also fixed a latent issue the branch introduced and go vet caught: WithNamespace copied the driver struct by value, cloning the new cache mutexes while sharing the cache entries (copylocks). It now builds a fresh driver — namespace copies never serve cluster telemetry, so they start with empty caches.

Re-ran everything: go build ./..., go vet (clean on the touched packages), go test ./... (all pass, incl. rbac_contract_test), cd ui && npm test (27/27), npx tsc --noEmit (no new errors vs. the branch base), npm run build.

@Matheus-Thurler Matheus-Thurler mentioned this pull request Oct 16, 2026
Merged
3 of 4 tasks
Matheus-Thurler added a commit that referenced this pull request Oct 16, 2026
@Matheus-Thurler @wesleip
feat(dashboard): cluster overview on the redesign (rebase of #255) (#261
788e26c 
)

* feat(dashboard): add cluster overview backend telemetry

Extend GET /api/v1/dashboard/summary with three new additive fields
that surface platform-wide telemetry for the operator:

  - hosts:    node count, ready nodes, allocatable CPU and memory
              (millicores / bytes), distinct kubelet versions, OS images
              and architectures. Sourced from core/v1/nodes via
              KubeVirtDriver.ClusterMetrics.
  - hosts.usage: real CPU and memory consumption aggregated from
              metrics.k8s.io/v1beta1/nodes (Tier 2 — requires
              metrics-server, documented prerequisite in helm-charts).
              Served with a 30s in-memory cache, and degrades gracefully
              when the API is absent, Forbidden, or transiently
              unavailable.
  - storage:  tenant-scoped PVC capacity and consumption (Bound vs
              reserved) plus the count. Sourced from the existing PVC
              list path in KubeVirtDriver.
  - addons:   cluster-wide health of critical dependencies
              (KubeVirt, CDI, Multus, metrics-server, networking, and
              any chart that follows the Helm label conventions),
              discovered dynamically from CustomResourceDefinitions
              with the 'app.kubernetes.io/name' label and a small alias
              table for the well-known API groups. Each entry reports
              'ok' (Established) or 'degraded'.

All four reads are fail-soft: 404, 403, 401, 503, timeouts, and
connection-refused return (nil, nil) so the endpoint never 500s.
Memory mode and nil client leave the fields empty.

docs/rbac-contract.yaml documents the dynamic-client access to
metrics.k8s.io in a new dynamicClient section — rbac_contract_test
keeps passing because the typed client AST scan does not see
Discovery().RESTClient().

PlatformService gains a kv field so the service can read the
hypervisor. CHANGELOG entry under Unreleased. A new
2026-10-08-dashboard-host-metrics.md captures the design and the
locked decisions for this slice.

* feat(ui): render cluster overview in the dashboard

Surface the new hosts/storage/addons fields from the previous slice
on the existing Dashboard page:

  - Rename the section title from 'Cluster nodes' to 'Cluster
    overview' to fit the broader scope (nodes + capacity + addons).
  - Four cards inside the section: Nodes, Ready nodes, Allocatable
    CPU, Allocatable memory. The CPU/memory cards now carry a
    usage bar (used / allocatable) and a percent line, with a
    colour ramp on 60% / 85% thresholds. A small 'Todos prontos' /
    'N degraded' / 'None ready' badge sits next to the Ready
    nodes count.
  - The four cards share a common baseline via h-full + flex-1 so
    the percent and the bars line up at the bottom regardless of
    how much content the other cards carry.
  - A new 'Storage' card replaces the redundant 'Networks' tile in
    the stats row, formatted with auto-scaled units (B / KiB / MiB
    / GiB / TiB) and a percent line.
  - A 'Cluster addons' strip below the Kubelet / OS / Arch footer
    renders one dot per critical dependency, green and pulsing
    when Established, an outlined ring when absent, yellow and
    pulsing when unreachable. The list is discovered dynamically
    by the backend — only addons actually present on the cluster
    are shown.
  - The dashboard query polls every 10s when /ws/events is up and
    5s when it is down, so the usage bars and the addons strip
    stay current without a manual refresh. The existing
    realtime-invalidation path still drives VM transitions.

The polling is on a fixed schedule rather than the previous
'health == ok' gate because the underlying data sources
(metrics.k8s.io, the addon probe) have no /ws/events counterpart.

i18n: PT and EN parity for every new key, asserted by i18n.test.ts.
typecheck: tsc --noEmit reports 0 errors new to this commit (the
'isFetching' unused warning is pre-existing).

* feat(security): restrict the Cluster overview section to root

The dashboard Cluster overview (node counts, capacity, addons) is
platform-wide telemetry, not tenant-scoped. Hide it from any actor
whose role is not 'root'.

Two layers:

  1. service.DashboardSummary now takes a role argument. When the
     role is anything other than platform.RoleRoot, the service
     skips populateHosts, populateStorage and populateAddons; the
     JSON response carries no 'hosts', 'storage' or 'addons' field,
     so nothing leaks through the Network tab or other consumers.
  2. The Dashboard React component gates the entire <SurfaceCard>
     with selectIsRoot, so non-root never even renders the card.

Tenant admins and tenant users still see the rest of the dashboard:
volume / VPC / SG / network counts, recent activity, the VM
running% badge, and the health card. Only the platform-wide
telemetry is gone.

The handler passes actor.Role (empty string when unauthenticated,
which is also not root). Two new tests
(TestDashboardSummaryHidesClusterOverviewForNonRoot and
TestDashboardSummaryShowsClusterOverviewForRoot) lock the
behaviour; the existing hosts/storage/addons tests now pass
platform.RoleRoot explicitly so they keep exercising the populated
path.

* fix(dashboard): address cluster overview review feedback

- populateHosts: cluster usage is optional — an unexpected ClusterUsage
  error is logged and leaves hosts.usage nil instead of failing the
  whole dashboard summary (500) for root.
- fetchClusterUsage: parse node quantities with resource.ParseQuantity
  and skip the item on error; resource.MustParse could panic on a
  malformed metrics.k8s.io payload.
- probeAddons: bound the CRD list with a 4s timeout (same bound as the
  usage call) and request a PartialObjectMetadataList so only names and
  labels cross the wire, not full OpenAPI schemas. The API group is
  derived from the CRD object name (<plural>.<group>); Established is
  not part of ObjectMeta, so a discovered addon is now ok by presence.
  The apiextensions.k8s.io/customresourcedefinitions get/list grant is
  listed under dynamicClient in docs/rbac-contract.yaml.
- StorageSummary: rename used_bytes to bound_bytes and reword the UI
  label to "allocated" — the figures are spec.requests sums, i.e.
  allocated capacity, not real guest consumption.
- WithNamespace: build a fresh driver instead of copying the struct —
  the copy cloned the new cache mutexes (go vet copylocks) while
  sharing the cache entries.

Chart grants land in virtfoundry/helm-charts#141 (rules first).

* fix(ui): adapt cluster overview to redesigned dashboard

Render the overview with StatsCard/StatusBadge/SurfaceCard and add demo fixtures for hosts, storage and addons.

---------

Co-authored-by: Weslei Paulo Pereira <[email protected]>
@Matheus-Thurler

Matheus-Thurler commented Oct 16, 2026

Copy link
Copy Markdown
Contributor

Superseded by #260 (rebased on the new UI).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@RodrigoGoncalves-dev RodrigoGoncalves-dev Awaiting requested review from RodrigoGoncalves-dev RodrigoGoncalves-dev is a code owner
@Matheus-Thurler Matheus-Thurler Awaiting requested review from Matheus-Thurler Matheus-Thurler is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@wesleip @Matheus-Thurler