feat: add performance view with time to first token and concurrency

docs/concepts/architecture.md

@@ -129,10 +129,16 @@ under-count real spend.
 ### Streaming is bytes-through, with a bounded usage sniffer
 
 Streamed responses are relayed as raw bytes, never re-framed or re-serialized.
-A line scanner with a bounded buffer parses only lines that contain
-`"usage"`; everything else passes untouched. Billing does not require
-understanding the stream, only spotting the usage object, and re-framing is
-where proxies corrupt SSE.
+A line scanner with a bounded buffer parses lines that contain `"usage"`;
+everything else passes untouched. Billing does not require understanding the
+stream, only spotting the usage object, and re-framing is where proxies corrupt
+SSE.
+
+One exception: until the first event with generated content, the scanner also
+parses each `data:` line to record the time to first token. It keeps only a
+timestamp, never part of the line, and stops checking after the first match.
+The first byte is not used because Anthropic sends `message_start` and `ping`
+before any token.
 
 ### Proxy metadata lives in headers, never in bodies
 

docs/guides/usage.md

@@ -19,8 +19,8 @@ ever persisted**, and there is no flag to turn that on; see
 [../concepts/architecture.md](../concepts/architecture.md#the-schema-is-content-free-by-construction).
 
 Per event: principal, key, provider, model, upstream name, endpoint, client,
-status code, outcome, error kind, cancelled and streamed flags, duration and
-cost. Attached to it, one row per unit: `(unit, quantity, unit_price,
+status code, outcome, error kind, cancelled and streamed flags, duration, time
+to first token (streamed requests only) and cost. Attached to it, one row per unit: `(unit, quantity, unit_price,
 priced)`.
 
 | Field | Values |
@@ -153,6 +153,7 @@ not visibility.
 | `GET /stats/requests` | One page of the request metadata log, newest first, failures included. `limit` (max 500) and `offset` page it; the response carries `{requests, limit, offset, total}` |
 | `GET /stats/requests/facets` | The distinct principals, keys, providers, models, clients and tags in the window, for filter menus. Each list caps at 500 values |
 | `GET /stats/errors` | The errors dashboard in one call: a gap-filled series of counts per outcome, plus a breakdown per (provider, model, endpoint, client, tags, outcome, error_kind, status_code) with request count, average duration, last-seen and time-to-outcome bands |
+| `GET /stats/performance` | The performance view in one call; see [Read performance](#read-performance) |
 
 All of them take the same filters:
 
@@ -199,6 +200,44 @@ Two things to know when reading `/stats/summary`:
 - `/stats/errors` includes cells with outcome `ok`, so error rates per
   dimension have their denominator.
 
+## Read performance
+
+The Performance view under the Usage tab shows how fast requests completed.
+`GET /stats/performance` serves the same data. It takes the filters above and
+`bucket=hour|day|week|month` (default `day`).
+
+```bash
+curl -s "$P/stats/performance?since=2026-07-01&bucket=day" -H "authorization: Bearer $KEY"
+```
+
+The response holds a `summary`, a gap-filled `series` with one entry per
+bucket, and a `models` list with one entry per (provider, model). Each entry
+reports these figures:
+
+| Figure | Definition |
+|---|---|
+| `duration_ms` | Time from the upstream call to the last byte |
+| `ttft_ms` | Time to first token: from the upstream call to the first event with generated content. Streamed requests only |
+| `tokens_per_second` | Output tokens divided by (duration minus time to first token). Streamed requests with both figures only |
+| `concurrency` | Requests in flight at the same time: `average` and `peak`. Summary and series only |
+
+The three latency figures are `{mean, p50, p95}`. They count only requests
+with outcome `ok` that were not cancelled. `requests` counts every request in
+the entry, and `measured` counts the requests that qualified. A figure with no
+qualifying request is `null`, never zero.
+
+Two things to know when reading `/stats/performance`:
+
+- A request has a start (end time minus duration) and an end. Latency figures
+  count it in the bucket where it ended. Concurrency splits it across every
+  bucket it spans.
+- Requests match the window by their end time. A request still running, or
+  one that ended after `until`, is not counted. A request that started before
+  the first bucket adds to concurrency only from that bucket on.
+
+A range with more than 500,000 requests returns 400 `range_too_large`. Narrow
+the range or add a filter.
+
 ## Scrape metrics
 
 `GET /metrics` is unauthenticated and exposes Prometheus text with aggregate

docs/reference/api.md

@@ -92,6 +92,7 @@ together; a pair nothing carries simply matches nothing. `outcome` takes
 | `GET /stats/requests?limit&offset` | One page of the filtered request metadata log (never content), newest first; returns `{requests, limit, offset, total}` |
 | `GET /stats/requests/facets?since&until` | Distinct principals, keys, providers, models, clients and tags in the window, for the explorer's filter options |
 | `GET /stats/errors?bucket&since&until` | The errors dashboard in one call: a gap-filled series of counts per outcome, plus a breakdown per (provider, model, endpoint, client, tags, outcome, error_kind, status_code) with request count, average duration, last-seen and time-to-outcome bands (<1s, 1-5s, 5-15s, 15-30s, 30-60s, 60-120s, >=120s). Rows with outcome `ok` are included so error rates have their denominator. |
+| `GET /stats/performance?bucket&since&until` | The performance view in one call: a summary, a gap-filled series and a per-model breakdown. Each holds `requests`, `measured`, and `duration_ms`, `ttft_ms` and `tokens_per_second` as `{mean, p50, p95}`, null when no request qualifies. The summary and the series also hold `concurrency` as `{average, peak}`. A range with more than 500,000 requests is a 400 `range_too_large`. |
 | `GET /stats/rate-limits` | Current rate-limit snapshot per configured provider: limits, remaining, worst remaining in the current minute, reset times, `observed_at` and a `stale` flag when the reading is older than five minutes. |
 | `GET /stats/rate-limits/series?provider&since&until` | Stored minute-bucket rate-limit samples for one provider. Each row carries the last and worst remaining values for that minute. |
 

docs/reference/openapi.yaml

@@ -2709,6 +2709,13 @@ paths:
                           type: boolean
                         duration_ms:
                           type: integer
+                        ttft_ms:
+                          type: [integer, 'null']
+                          description: >
+                            Time to first token in milliseconds, from the
+                            upstream call to the first event with generated
+                            content. Null on unary requests and on streams
+                            that ended before any content.
                         units:
                           type: object
                           additionalProperties:

internal/server/admin.go

@@ -1166,8 +1166,12 @@ func (s *Server) serveRequestLog(w http.ResponseWriter, r *http.Request) {
 			"cost":        nil,
 			"unpriced":    ev.Unpriced,
 			"duration_ms": ev.DurationMs,
+			"ttft_ms":     nil,
 			"units":       ev.Units,
 		}
+		if ev.TTFTMs.Valid {
+			view["ttft_ms"] = ev.TTFTMs.Int64
+		}
 		if ev.StatusCode.Valid {
 			view["status_code"] = ev.StatusCode.Int64
 		}

internal/server/export_test.go

@@ -0,0 +1,8 @@
+package server
+
+// SetPerformanceEventCap lowers the event cap for a test and returns the restore.
+func SetPerformanceEventCap(n int) func() {
+	old := performanceEventCap
+	performanceEventCap = n
+	return func() { performanceEventCap = old }
+}

internal/server/firstcontent_internal_test.go

@@ -0,0 +1,44 @@
+package server
+
+import "testing"
+
+func TestOpenAIFirstContent(t *testing.T) {
+	cases := map[string]bool{
+		`data: {"choices":[{"index":0,"delta":{"role":"assistant","content":""}}]}`:      false,
+		`data: {"choices":[{"index":0,"delta":{"role":"assistant","content":null}}]}`:    false,
+		`data: {"choices":[{"index":0,"delta":{"content":"hi"}}]}`:                       true,
+		`data: {"choices":[{"index":0,"delta":{"reasoning_content":"hmm"}}]}`:            true,
+		`data: {"choices":[{"index":0,"delta":{"reasoning":"hmm"}}]}`:                    true,
+		`data: {"choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"id":"c1"}]}}]}`: true,
+		`data: {"choices":[{"index":0,"delta":{"tool_calls":[]}}]}`:                      false,
+		`data: {"choices":[{"index":0,"text":"legacy"}]}`:                                true,
+		`data: {"choices":[],"usage":{"prompt_tokens":1,"completion_tokens":1}}`:         false,
+		`data: {"choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}`:              false,
+		`data: [DONE]`:   false,
+		`: keep-alive`:   false,
+		`data: not json`: false,
+		`data:{"choices":[{"index":0,"delta":{"content":"no space after the colon"}}]}`:        true,
+		`data: {"choices":[{"index":0,"delta":{"content":[{"type":"text","text":"parts"}]}}]}`: true,
+	}
+	for line, want := range cases {
+		if got := openAIFirstContent([]byte(line)); got != want {
+			t.Errorf("openAIFirstContent(%s) = %v, want %v", line, got, want)
+		}
+	}
+}
+
+func TestAnthropicFirstContent(t *testing.T) {
+	cases := map[string]bool{
+		`event: content_block_delta`: false,
+		`data: {"type":"message_start","message":{"usage":{"input_tokens":1}}}`: false,
+		`data: {"type":"ping"}`: false,
+		`data: {"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}}`: false,
+		`data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"a"}}`:  true,
+		`data: {"type":"message_delta","usage":{"output_tokens":3}}`:                               false,
+	}
+	for line, want := range cases {
+		if got := anthropicFirstContent([]byte(line)); got != want {
+			t.Errorf("anthropicFirstContent(%s) = %v, want %v", line, got, want)
+		}
+	}
+}

internal/server/openai.go

@@ -3,6 +3,7 @@ package server
 import (
 	"bytes"
 	"context"
+	"database/sql"
 	"encoding/json"
 	"errors"
 	"io"
@@ -272,6 +273,7 @@ func (s *Server) relayStream(w http.ResponseWriter, r *http.Request, auth *Auth,
 	flusher, _ := w.(http.Flusher)
 
 	var usage map[string]any
+	var ttft sql.NullInt64
 	outcome, kind := "ok", ""
 	cancelled := false
 	var pending []byte
@@ -293,6 +295,7 @@ func (s *Server) relayStream(w http.ResponseWriter, r *http.Request, auth *Auth,
 				if idx < 0 {
 					break
 				}
+				ttft = firstContentMs(ttft, pending[:idx], started, openAIFirstContent)
 				if u := parseSSEUsage(pending[:idx]); u != nil {
 					usage = mergeUsage(usage, u)
 				}
@@ -317,12 +320,13 @@ func (s *Server) relayStream(w http.ResponseWriter, r *http.Request, auth *Auth,
 	}
 	// Some upstreams end the stream without a trailing newline; scan the
 	// leftover partial line for a usage chunk too.
+	ttft = firstContentMs(ttft, pending, started, openAIFirstContent)
 	if u := parseSSEUsage(pending); u != nil {
 		usage = mergeUsage(usage, u)
 	}
 	s.recordUsageAsync(auth, route, endpoint, usageOutcome{
 		StatusCode: resp.StatusCode, Outcome: outcome, ErrorKind: kind, Cancelled: cancelled,
-		Streamed: true, Usage: usage, DurationMs: time.Since(started).Milliseconds(),
+		Streamed: true, Usage: usage, DurationMs: time.Since(started).Milliseconds(), TTFTMs: ttft,
 	})
 }