Repository navigation
M16.1 Record and count API requests #64
Copy link
Copy link
Closed
Labels
enhancementNew feature or requestNew feature or request
Description
Part of #63. Plan in docs/architecture.md › API monitoring (M16).
yuva-api
- Contract:
kind(site,api) on analytics properties withapdex_msandmonitor_hour_days;POST /v1/analytics/properties/{id}/requests(1–1,000 records,analytics:write, gzip,202withacceptedanderrors);dataset: apiin the query operation with the M16.1 metrics (requests,requests_per_minute,errors,error_rate,client_errors,client_error_rate,p50,p95,p99,avg_duration) by route, method, status, status class, environment andtime:minute|hour|day, and ranges15m,1h,6h,24h. Lint, regenerate. - Records: method, route or path, status (0 for no response), duration, request and response size, time, optional consumer, API version, environment, service, release,
ip,sample_rate. No body, header, query string or user agent accepted; the address shortened, looked up for the country and dropped in the first function. - Route inference from
path(ids, UUIDs, dates, long hex or base64url, e-mail-shaped segments →{id}), 404/405 without a pattern →(unmatched), route rules, caps with(other). - Duration sketches (base 2^(1/32), sparse varint
bytea) and size sketches (scale 2) with merge, quantile and Apdex in Go, with tests against exact percentiles. - Per-process aggregation with a random writer id, cumulative upserts every 10 s, the last flush 2 minutes after a bucket ends, late records under a fresh writer, drain on shutdown.
- Tables with
workspace_id:monitor_minute(day partitions, 8 days),monitor_hour,monitor_dim_hour(month partitions),monitor_day,monitor_dim_day,monitor_routes,monitor_names,monitor_rollup_hours,monitor_ingest_stats; partitions in the aggregator andanalytics_partitions; themonitor_rollupfold on theanalyticsqueue. -
usage_counters.monitor_requests,YUVA_MONITOR_MONTHLY_REQUESTS,yuva workspace monitor-limit,YUVA_MONITOR_RECORDS_PER_MINUTE,YUVA_MONITOR_BUFFER_KEYS,YUVA_MONITOR_MINUTE_DAYS,YUVA_MONITOR=off. -
sdk/go/analytics:Middlewarefornet/httpthat readsr.Patternafter the mux ran (method and host stripped) or uses aRoutefunction, batching 500 records or 5 seconds through the existing queue and backoff, optionalSampleRatefor successful requests only. - Load test: 300M records a month on one property (500 routes, 5,000 consumers, bursts of 2,000 records a second): ingest p99 under 100 ms, data visible within 15 s, 30-day overview p95 under 1 s.
yuva-panel
- Creating an API property; API overview: requests, error rates and p95 over time, the last 60 minutes by minute; routes table sorted by traffic, errors or latency; a route's page with its history and status codes; caps reached shown. en and tr.
yuva-docs
- "API monitoring" guide with the Go middleware and curl, what is and is not recorded; en, tr, de.
Done when
A Go service with the middleware shows its routes in the panel within a minute, with counts and error rates equal to the requests sent and p50, p95 and p99 within 1.1% of the exact values, and the next day the same numbers from folded days; a raw path with an id never appears as its own route and 404 probes stay in (unmatched); Postgres holds no row per request and no IP address; a key without analytics:write gets 403; the load test meets its targets.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request
Activity
productdevbook commentedon Oct 14, 2026
API part shipped locally (634d9d2, 4fcac89, a2f178e, afc19d6); load test at 2,000 records/s against 30 days of history: ingest p99 31.7 ms, visible after 10.1 s, 30-day overview p95 515 ms.
Still open in this step:
hiddencolumn exists).over_limit).docs/architecture.mdsays.docs: plan API monitoring (M16): request rollups, every API stack, da…
test(api): API monitoring load test at 300M requests a month
docs: API monitoring guide and settings in en, tr and de
feat(web): API properties with an overview, a routes table and a page…
feat(api): hide API routes, tell owners about the request records lim…
docs: API monitoring guide covers hiding routes, the request limit no…
feat(web): API property settings and hiding routes
fix(web): hidden route pages show no figures, latency axes fit, route…
fix(api): API time series fill every bucket once and label minutes an…
fix(web): API charts read the offset in minute and hour labels
productdevbook commentedon Oct 14, 2026
This step's code is no longer on main; it lives in #72.