fix: report no requests figure where the engine exposes none - #247
Open
outofcoffee wants to merge 3 commits into
Open
outofcoffee wants to merge 3 commits into
outofcoffee wants to merge 3 commits into
Conversation
added 3 commits
September 25, 2026 20:31
The parser read the requests figure from request_success_total for every engine family, but that metric is vLLM's: llama.cpp serves no cumulative request counter, so the figure was always zero. The statistics now carry the request count only where the engine family's metrics expose one - a missing figure, not a zero the engine never produced - and the token block draws the line only for a figure the statistics carry.
The parser and the token block are exercised for the absent case at each surface that draws the block - the daemon's metrics reply, the dashboard tile, the serve view and the metrics formats - and for the wire shape that makes an absent figure distinguishable from a genuine zero.
outofcoffee
marked this pull request as ready for review
September 25, 2026 23:03
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The llama.cpp metrics tile's
requestsfigure was always 0, even though theother engine-sourced counters worked. The collector read the figure from
request_success_totalfor every engine family, but that metric is vLLM's:llama.cpp's server exposes no cumulative request counter at all, so the figure
was a zero no engine ever produced.
Summary
metrics expose a cumulative request counter (vLLM's
request_success_total); a family that exposes none (llama.cpp, today)omits the field rather than reporting a zero.
TokenStats.Requestsbecomes optional (*int,omitempty) across thedaemon's reply, the remote relay, and the
metricsJSON output.metricsbar and tableformats, the serve view — draws the
requests:line only for a figure thestatistics carry.
fabricating a
llamacpp:request_success_totalline a real engine wouldnever serve.
field as optional.
Implementation details
The absence is modelled as a pointer, not a zero the renderers hide: a
genuine zero from an engine that exposes the counter (started, nothing served
yet) still serialises, while a family that exposes none omits the field
entirely. That keeps "no figure" and "a zero figure" distinguishable in the
data, the same rule the engine-metrics spec already applies to the last-active
pair and the system stats — and it is what lets every rendering surface drop
the line without special-casing a runner.
Verified against a live llama.cpp server run with
--metrics: the collectoryields the token counters with no requests field, and no surface draws a line
for it.
Fixes #212.