Skip to content

feat(analytics,proxy): routing headers visualization and routing mode query param - #503

Open
yanaSelin wants to merge 1 commit into
mainfrom
EPMCDME-14083
Open

yanaSelin wants to merge 1 commit into
mainfrom
EPMCDME-14083

Conversation

@yanaSelin

@yanaSelin yanaSelin commented Aug 25, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds Switchyard/LiteLLM routing awareness across the Claude plugin: the statusline shows what a
router alias actually dispatched to, codemie models list distinguishes router aliases from
concrete deployments, and the analytics report prices and visualizes routing/classifier cost per
turn and per session. Also converts the Claude statusline to a bundled TypeScript artifact and
tightens explicit-model-choice handling across the CLI.

Changes

Statusline

  • Convert the Claude statusline from a loose .mjs script to TypeScript with normal project
    imports, then bundle it into a single esbuild artifact; deploy only the bundled artifact
    (drop shim-file deployment, exclude .ts sources from the Claude plugin asset copy).
  • Add a "routed to" widget: when the configured model alias is a Switchyard virtual router or a
    declared LiteLLM auto-router (isRouterCatalogEntry), show the actual backend model the last
    turn dispatched to instead of just the alias name.
  • Surface classifier (routing LLM) cost and tokens, and a routing-tier indicator, sourced from
    the same transcript-based cost accounting the analytics report uses.

Model list & resolution

  • codemie models list: add a TYPE column (router vs. model) and widen the ID column so
    long free-form router alias names no longer truncate.
  • codemie-codex: accept an explicitly requested --model/env model even when it is not listed
    in the live catalog (router aliases resolve per request and are not always published as
    deployments), gated to an explicit choice via a new CODEMIE_MODEL_SOURCE (cli/env/default)
    marker set by AgentCLI; also recognize gpt-smart-router/gpt-fast-router-style aliases as
    Codex-compatible.
  • Fix subagent-default-tier pinning to skip itself when the session model differs from the only
    provisioned tier, so a router alias assigned to CODEMIE_HAIKU_MODEL/CODEMIE_OPUS_MODEL
    no longer gets silently overridden for subagents declaring model: inherit.
  • Fetch the full model descriptor (fetchCodeMieLlmModels) instead of ID-only, so router entries
    carry their is_router/litellm_router.is_router flag through to both the CLI model list and
    the statusline's routing signal.

Analytics (cost & routing report)

  • Report: per-turn model/routing timeline, a "Routing" KPI section (sessions routed, classifier
    cost), and a routing-tier chart on the session page; classifier cost/tokens roll into session
    and summary totals.
  • New bedrock-pricing.mjs: detects Amazon Bedrock's regional/cross-region-inference pricing
    premium from the raw backend model id and applies it to the rate card.
  • Fix model-name normalization to also strip own Bedrock alias shape
    (bedrock/us.anthropic.claude-sonnet-5, no version suffix), which previously slipped through
    unnormalized and showed up as a raw deployment path next to a clean model name in the same
    session's model breakdown.
  • New background-request-normalizer proxy plugin: force a concrete haiku deployment for Claude
    Code's own background requests (e.g. auto-title generation) instead of deferring to the
    upstream classifier, which sometimes misrouted these tiny calls to the expensive tier — one
    observed case billed a title-gen call on par with the session's most expensive real turn.

Other

  • Fix local-dev SSO API base resolution to not require the code-assistant-api prefix.

Testing

  • npm run typecheck
  • npm run lint
  • Manual testing: generated HTML analytics report against real routed/unrouted sessions;
    verified statusline "routed to" widget and codemie models list TYPE column locally.

Checklist

  • Code follows project standards
  • CI is green (npm run ci)
  • No merge conflicts with main

EPMCDME-14886

EPMCDME-14887

@codemie-ai

Copy link
Copy Markdown
Owner

[AUTO_CLOSE_WARNING] ⏰ This pull request has had no activity for 7 days (last activity: 2026-09-11).

It will be automatically closed no earlier than 2026-09-25 (UTC) (~6 more days).

Any activity from anyone — a comment, a push, a review, reopening — restarts the countdown from that moment. This warning will be deleted automatically once that happens.

@yanaSelin
yanaSelin force-pushed the EPMCDME-14083 branch 2 times, most recently from 8ebe043 to 5a626d5 Compare September 22, 2026 15:14
… bundling (EPMCDME-14083)

Squashed history of this branch (21 commits) into one for a clean rebase onto main.

- Extract Switchyard/LiteLLM routing headers into a shared domain layer (routing-headers.mjs),
  injected onto response bodies by a new proxy plugin and read back by the analytics cost
  engine and the Claude statusline.
- Add classifier (routing LLM) cost reporting and a Routing KPI section in the analytics report.
- Add Bedrock regional-endpoint pricing detection/premium (bedrock-pricing.mjs).
- Force a concrete haiku deployment for Claude Code's own background requests (title-gen, etc.)
  instead of deferring to the upstream classifier, which sometimes misrouted them to a pricier tier.
- Convert the Claude statusline to TypeScript and bundle it with esbuild instead of deploying
  loose .mjs sources; deploy the bundled artifact only.
- Preserve explicit --model/env model choices during subagent-default-tier resolution; fix the
  stale AgentCLI test this depends on to expect the intended 3-way CODEMIE_MODEL_SOURCE value.
- Resolve local-dev SSO API base without the code-assistant-api prefix.

Refs: EPMCDME-14083

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants