Skip to content

fix: honor LISTEN when metrics are pushed - #375

Merged
mayankpande88 merged 2 commits into
mainfrom
fix/listen-in-push-mode
Oct 8, 2026
Merged

mayankpande88 merged 2 commits into
mainfrom
fix/listen-in-push-mode

Conversation

@mayankpande88

Copy link
Copy Markdown
Contributor

Summary

When metrics are pushed (--metrics-endpoint / METRICS_ENDPOINT), the agent forced its HTTP endpoint onto 127.0.0.1:10300 and ignored LISTEN / --listen. If that port was taken, the agent exited and stopped collecting entirely. On a host that pushes, a port conflict therefore cost all data, and two pushing agents couldn't run side by side.

  • LISTEN / --listen is now honoured whenever it is set.
  • Unset, the defaults are unchanged: 0.0.0.0:80, or 127.0.0.1:10300 when pushing.
  • When pushing, a port that can't be bound is logged and the agent keeps running, because nothing needs to scrape that endpoint. In scrape mode a failed bind still stops the agent, as before.
Engineering detail

Change: --listen no longer has a hard default. flags.init fills it in only when it's empty, so an explicit value always wins. The 10300 override came from 1faeef3.

Compatibility: the Kubernetes deployment doesn't set LISTEN, so it keeps 0.0.0.0:80 when scraped and 127.0.0.1:10300 when pushing. install.sh only passes LISTEN through if the user sets it.

CI: gofmt, goimports, vet, golangci-lint, go test (excluding /containers) and the build all pass in a Linux container with Go 1.26.5.

Local e2e: I built agent binaries from this branch and from main and ran them as systemd services on a local Debian 12 VM (kernel 6.1), pushing to a local VictoriaMetrics.

  • Push mode, no LISTEN: both builds listened on 127.0.0.1:10300.
  • Push mode, LISTEN=127.0.0.1:10352: this branch listened on :10352; main still listened on 127.0.0.1:10300.
  • Port 10300 already taken: this branch logged "local metrics endpoint unavailable" and kept sending metrics. Main logged "address already in use" and exited.
  • Scrape mode, no LISTEN: this branch listened on :80.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the ListenAddress flag configuration to conditionally default to 0.0.0.0:80 or 127.0.0.1:10300 depending on whether a metrics endpoint is configured. Additionally, it ensures that a failure to start the local HTTP server does not stop the agent when metrics are being pushed. Feedback suggests using klog.Exitln instead of klog.Errorln when the metrics endpoint is not configured, ensuring the agent exits with a non-zero status so that orchestrators can detect the failure and restart it.

Comment thread main.go
With --metrics-endpoint set, the listen address was forced to
127.0.0.1:10300 even when LISTEN or --listen was given, and a taken port
made the agent exit, so two pushing agents could not share a host and a
port conflict stopped all collection. LISTEN now wins whenever it is
set; unset, the defaults stay as before (0.0.0.0:80, or 127.0.0.1:10300
when pushing). When pushing, a failed bind is logged and the agent keeps
running, since nothing scrapes the endpoint.
In scrape mode a failed bind made main return, so the agent exited 0
and a service manager restarting only on failure would leave it down.
@mayankpande88
mayankpande88 force-pushed the fix/listen-in-push-mode branch from 2649bf3 to bf5fc20 Compare October 8, 2026 09:18
@mayankpande88
mayankpande88 merged commit ebbf6ad into main Oct 8, 2026
7 checks passed
@mayankpande88
mayankpande88 deleted the fix/listen-in-push-mode branch October 8, 2026 09:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants