Repository navigation
feat(metrics): add opt-in native edge request metrics - #30
Conversation
Record authenticated native Worker generations in the D1 usage ledger and expose isolate-local Prometheus counters behind NATIVE_EDGE_METRICS_ENABLED, default off. Adds a static lifecycle hook chain for native dispatch.
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configuration
You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
multillm-proxy | 220fbde | Oct 08 2026, 09:57 PM |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 220fbde9f2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| async pull(controller) { | ||
| try { | ||
| const result = await reader.read(); | ||
| if (result.done) { await finish(); controller.close(); return; } |
There was a problem hiding this comment.
Close the response before awaiting ledger finalization
When D1 is slow or unavailable, the terminal read of every enabled native response waits for recordNativeUsage() before closing the downstream stream, so clients using response.text(), response.json(), or reading through EOF inherit D1 latency and can hang even though the complete upstream body has arrived. Since nativeGenerationFetch() already registers the finalization promise with ctx.waitUntil, start finalization without awaiting it before controller.close() so best-effort telemetry cannot delay served responses.
Useful? React with 👍 / 👎.
| observer.feed(result.value); controller.enqueue(result.value); | ||
| } catch (error) { await finish(options.signal?.aborted ? "canceled" : "transport_error"); controller.error(error); } | ||
| }, | ||
| async cancel(reason) { await finish("canceled"); await reader.cancel(reason); }, |
There was a problem hiding this comment.
Preserve completion when the OpenCode wrapper cancels
With OPENCODE_EDGE_FETCH=true, NATIVE_EDGE_METRICS_ENABLED=true, and the legacy streaming /opencode/chat/completions route, createOpencodeStreamResponse() returns as soon as it sees [DONE] and unconditionally cancels the observed upstream body in its finally block. This cancellation forces reason="canceled" here even after the observer parsed complete usage and [DONE], so successful requests are persisted as status 499 and inflate error/cancellation metrics; distinguish downstream abandonment from cancellation after a recognized terminal event, or avoid wrapping this internal transform as a client cancellation.
Useful? React with 👍 / 👎.
|
|
||
| /** Serialize native telemetry into the existing strict ledger; richer fields need a migration. */ | ||
| export async function recordNativeUsage(env, event) { | ||
| const model = event.model ? `${event.provider}:${event.model}` : null; |
There was a problem hiding this comment.
Avoid prefixing already-qualified model IDs
When a native request supplies an already-qualified model such as openai:gpt-4, priceUsage() deliberately uses that identifier unchanged, but this serialization stores it as linkapi:openai:gpt-4 (or the corresponding edge provider). That fragments usage summaries under a model name different from the configured pricing key and from the Container accounting path, whose _qualified() helper also leaves IDs containing : unchanged; only prepend event.provider when the model is not already qualified.
Useful? React with 👍 / 👎.
| const known = usage.input_tokens !== null && usage.output_tokens !== null; | ||
| const outcome = reason ?? options.outcome ?? (response.status >= 400 || parsed.failed ? "upstream_error" | ||
| : parsed.completed && known && hasProviderUsage ? "success" : "unknown"); |
There was a problem hiding this comment.
Treat embedding output tokens as measured zero
For a native LinkAPI /v1/embeddings response, the standard usage object reports prompt_tokens and total_tokens but has no completion/output-token field. Leaving output_tokens null makes known false, so every otherwise successful embedding response is persisted as status 520 with unknown usage and no configured cost; the existing Container accounting explicitly normalizes embeddings to zero output tokens, and the native finalization path needs the same endpoint-specific normalization.
Useful? React with 👍 / 👎.
| const name = (model.includes(":") ? model : `${provider}:${model}`).toLowerCase(); | ||
| const price = table?.[name] ?? table?.[`${provider}:*`] ?? table?.["*"]; |
There was a problem hiding this comment.
Normalize pricing-table keys before lookup
The requested model is lowercased before lookup, but the keys parsed from MODEL_PRICING_USD_PER_MILLION are left unchanged. An existing valid configuration such as {"LINKAPI:MODEL":{"input":1,"output":2}} is normalized and priced by the Container's CostService, while the native path silently records a null cost for the same model; normalize parsed keys to lowercase, as the shared pricing implementation does, so enabling native dispatch does not change pricing based only on key casing.
Useful? React with 👍 / 👎.
| function priceUsage(env, provider, model, usage, known) { | ||
| if (!known || !model) return null; |
There was a problem hiding this comment.
Apply flat image prices without requiring token usage
For a native /images/generations response using a flat-only pricing entry such as {"linkapi:image-model":{"request":0.04}}, standard image payloads commonly contain no token usage. The early known check therefore returns null before the flatOnly handling can run, and finalization records the successful request as status 520 with no cost; this conflicts with the existing MODEL_PRICING_USD_PER_MILLION contract, where request-only prices explicitly do not require provider token counts.
Useful? React with 👍 / 👎.
d45aba5
into
integrate/gateway-waves-1-2
Summary
Adds native edge request metrics behind
NATIVE_EDGE_METRICS_ENABLED, which defaults to false. While the flag is off, native Worker dispatch, forwarded requests and the Prometheus output are unchanged.ctx.waitUntil.PROMETHEUS_ENABLEDalso on,/v1/metrics/prometheusadds isolate-local native counters with fixed provider and outcome labels, at most 120 series.worker/gateway-lifecycle.mjsadds a staticauthorize/admit/before_dispatch/observe/finalizehook chain for native dispatch. Hooks are code-registered, with no dynamic loading.There are no new D1 tables, routes or browser-readable headers. Details are in
docs/edge-request-metrics.md.Integration
The Wave 3 integration PR wires the other Wave 3 features into the native lifecycle:
Shared registrations
tests/test_edge_request_metrics_worker.mjstotest:worker.worker/gateway-extensions.mjs,worker/gateway-lifecycle.mjsandworker/request-telemetry.mjsto the secret firewall inventory.gateway-extensions.mjsimport rewrite to the shared Worker test loader.NATIVE_EDGE_METRICS_ENABLEDis read only by the Worker, so it is not forwarded to the Container.Tests
tests/test_edge_request_metrics_worker.mjs: new.npm run test:worker: 1,247 passed.cf:migrations:check,test:deploy, free routes, client headers and thewrangler deploy --dry-runbundle all pass.tests/test_frontend_build.mjsfails locally because@tailwindcss/cliis not installed. It fails the same way on main.This is part of Wave 3. It is based on the Wave 2 integration branch (#25), and the Wave 3 integration PR adds the README line.