Skip to content

fix(exporter): omit phpfpm_pod on statically configured targets - #22

Merged
dimoschi merged 2 commits into
masterfrom
fix/omit-empty-pod-label
Sep 13, 2026
Merged

dimoschi merged 2 commits into
masterfrom
fix/omit-empty-pod-label

Conversation

@dimoschi

Copy link
Copy Markdown
Collaborator

Problem

On VM deployments every series carried an empty pod label:

phpfpm_scrape_failures{phpfpm_pod="",pool="",scrape_uri="tcp://127.0.0.1:1/status"} 1
phpfpm_up{phpfpm_pod="",pool="",scrape_uri="tcp://127.0.0.1:1/status"} 0

Only Kubernetes auto-tracking has a pod name to put there. pm.Add(uri, "") on the static path left it empty, but the label was still in the descriptor, so it was emitted regardless.

Prometheus states that "labels with an empty label value are considered equivalent to labels that do not exist", so the label conveyed nothing — it was noise in the exposition output for every non-Kubernetes deployment.

Fix

The label set now follows the discovery mode. NewExporter takes options, and internal/server passes phpfpm.WithPodLabel() only when --k8s.autotracking is set.

$ php-fpm_exporter server --phpfpm.scrape-uri tcp://127.0.0.1:1/status
phpfpm_scrape_failures{pool="",scrape_uri="tcp://127.0.0.1:1/status"} 1
phpfpm_up{pool="",scrape_uri="tcp://127.0.0.1:1/status"} 0

The choice has to be made per exporter, not per pool: client_golang rejects "descriptors reported by collector have inconsistent label names or help strings for the same fully-qualified name" (registry.go:336), so one process cannot mix both label sets. That is fine, because server runs in exactly one discovery mode.

Impact

  • Kubernetes output is unchanged. phpfpm_pod is still emitted and populated.
  • No series churn on VMs. An empty label already equalled an absent one, so removing it does not change series identity. Existing VM series carry on unbroken.
  • The grafana/ dashboard is unaffected. It selects on metric names only and never references phpfpm_pod — I checked all 11 phpfpm_* references in kubernetes-php-fpm.json. AGENTS.md implied the dashboard keys off these labels; that is corrected.

Why not rename it to something environment-agnostic

Considered and rejected. On a VM there is no value for such a label that is not already available: the host is Prometheus's own instance, the pool is pool, and the socket is scrape_uri. A rename would produce a label that is redundant on VMs while remaining Kubernetes-specific in practice, and it would change the identity of every existing Kubernetes series.

Dropping the label entirely was also rejected: with one exporter per namespace fanning out to many pods, Prometheus's scrape target is the exporter, so relabeling cannot recover pod identity. The exporter has to emit it.

Tests

  • descriptors carry the label in Kubernetes mode and not in static mode
  • collected series carry no phpfpm_pod at all for static targets
  • a real prometheus.Registry gathers cleanly in both modes, which is what proves the descriptors and the metric values agree
  • an end-to-end /metrics assertion over HTTP for the static case
  • newExporter maps the discovery mode to the label set in both directions, so auto-tracking silently losing the label is caught too

Verification

$ mise exec -- go test ./... -race
ok  	github.com/hackthebox/php-fpm_exporter/internal/server	3.711s
ok  	github.com/hackthebox/php-fpm_exporter/phpfpm	1.485s
ok  	github.com/hackthebox/php-fpm_exporter/phpfpm/fcgi

golangci-lint (v2.12.2, as CI pins): 5 issues, unchanged from master, none new. Label names became constants because two label-set families would otherwise repeat each string past the goconst threshold.

Also confirmed by running the built binary before and after, which is how the bug was reproduced in the first place.

Only Kubernetes auto-tracking has a pod name to put in phpfpm_pod. Static
--phpfpm.scrape-uri targets published phpfpm_pod="" on every series, which is
noise on VM deployments: Prometheus treats an empty label value as equivalent to
the label being absent, so it conveyed nothing.

The label set now follows the discovery mode. NewExporter takes options and
internal/server passes phpfpm.WithPodLabel() only under --k8s.autotracking.

The choice has to be per exporter rather than per pool, because client_golang
refuses two descriptors that share a metric name but disagree on label names.
That is safe because the server runs in exactly one discovery mode.

Kubernetes output is unchanged. Because an empty label already equalled an absent
one, dropping it does not change series identity for existing VM targets either.

Label names are now constants, since two label-set families would otherwise
repeat each string.

The grafana/ dashboard selects on metric names only and never on phpfpm_pod, so
it is unaffected; AGENTS.md claimed otherwise and is corrected.
The static half was covered, so nothing would have caught auto-tracking silently
losing phpfpm_pod, which is the same wiring bug in reverse and would have hidden
which pod a series came from.
@dimoschi
dimoschi merged commit 87a97c2 into master Sep 13, 2026
7 checks passed
@dimoschi
dimoschi deleted the fix/omit-empty-pod-label branch September 13, 2026 18:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant