Skip to content

feat: move admin analytics reads to daily facts - #641

Draft
FlyM1ss wants to merge 29 commits into
mainfrom
feat/admin-layer-prB
Draft

FlyM1ss wants to merge 29 commits into
mainfrom
feat/admin-layer-prB

Conversation

@FlyM1ss

@FlyM1ss FlyM1ss commented Oct 15, 2026 •

Copy link
Copy Markdown
Collaborator

DO NOT MERGE until 2026-10-23 at the earliest (30 complete fact days) AND the prod read-only gate below has been re-run and passes. Merging deploys prod and drops two tables irreversibly.

Admin layer redesign PR B (design docs/superpowers/specs/2026-09-15-admin-layer-redesign-design.md, plan docs/superpowers/plans/2026-09-15-admin-layer-redesign-prB-read-paths-delete.md). It moves every /api/admin/analytics/* read path off the legacy snapshot tables and onto user_daily_facts / user_activity / rollups. It deletes states.py, lifecycle_backfill.py, maintenance.py and the reaper sweep. It drops user_analytics_snapshots and user_lifecycle_daily_snapshots at boot.

Gate (run read-only on prod before lifting draft)

  • At least 30 complete fact days exist. The first complete day is 2026-09-23.
  • Legacy lifecycle days with no fact row: 18 on 2026-10-15 (07-11..07-28, 283 rows). These get rolled into anonymous lifecycle_segment_count/lifecycle_transition rollups before the drop. Re-count them, and confirm that every legacy segment is one of the six lifecycle segments.
  • user_analytics_snapshots.activated_at exists, so the seed can read it.
  • user_activity sanity: the users without a row are never-active users (123/271 on 2026-10-15).
  • Fact days with no lifecycle_transitions row (the PR A history copy, 07-29..09-22, plus no-movement days). The first boot derives their log from consecutive fact days before the drop; the count is informational.

Deploy

  • Roll forward only. A rollback to main recreates empty legacy tables.
  • The migration runs on the daily-facts worker thread, not the startup hook. Order: seed user_activity, legacy lifecycle rollup, history transition backfill, drop. Any of those failing skips the drop. The activation_users rollup backfill runs regardless of that chain. A failure (e.g. the drop's 5 s lock_timeout) is retried before each worker tick until it succeeds; the boot log prints the report.
  • The activation_users backfill fills every rolled-up day inside the 180-day raw-event retention from one grouped query. No operator step is needed: backfill_analytics would not do it (it rebuilds rollups only for days where it inserts new events).
  • The outgoing instance logs snapshot_maintenance_failed every 60 s until it exits. This is harmless.

Value changes

  • Retention, /groups and /lifecycle read the PR A history copy, which holds zero activity before about 2026-09-23. Retention marks those cells incomplete, and /groups now reports partial. Recomputing those days from raw events is a follow-up (admin: recompute PR A's copied fact days from raw events before they expire #645).
  • The "ATL platform model cost" figure comes from agent_runs.est_cost_usd and now excludes BYOK runs. Facts written since 2026-09-23 still carry BYOK estimates. Runs with no owner_user_id are excluded. "ATL Credits debited" now sums ledger consumption.
  • include_internal=true no longer widens retention, /groups runs/cost, lifecycle segments, state counts, attention, the overview's funnel/failures/tokens/active/conversion/repeat, or /commercial platform cost (today's raw slice is internal-free like the rollups).
  • Excluding an account or promoting it to admin removes its existing fact rows from every fact read (state counts, attention, history, transitions, /groups, expiry rollups) on the next request.
  • Before tonight's facts land (or while the job fails), /lifecycle current counts and /operational state counts read the latest landed day (stale=true, status partial, coverage = that day) instead of zeros. Overview attention reads the day before yesterday at most, else it is unavailable. /groups is partial when any past day in the window has no fact rows.
  • The activation funnel is unavailable while any rolled-up day in the window lacks activation_users rows.
  • Retention counts a ledger movement (purchase, grant, debit) as activity, as on main.
  • /users:
    • The filter and the priority placement use yesterday's segment, while each row shows the live segment.
    • Before tonight's facts land, the page falls back to the latest landed day, and as_of says which day that is.
    • A within-tier tie is no longer broken by lifetime net purchased.
  • /overview drops user_state_counts, and /users?status= returns 422. No frontend reads either.
  • The activation funnel and active/conversion/repeat ignore the billing/provider/model filters. The rollups have no such dimensions. The funnel sums per-day distinct users, so a user who repeats a step on two days counts twice. That is kept because an exact count needs a raw read on every load.
  • top_failure_categories[].affected_users is renamed event_count and counts failure events, filtered or not. On main it counted distinct users from a raw read. The page column now reads "Events". rollup_day stops writing affected_users rows. Closes admin: decide what the overview's failure count means #647.
  • Attention recent_runs = the sum of runs_requested over the 30 days through yesterday.
  • Profile:
    • The state follows the live axes.
    • Run/cost/spend figures are as of yesterday.
    • Tokens, provider mix and lane mix come from the newest 100 events. Milestones are first occurrences in the selected range, from one grouped MIN() (≤ 2 rows): a second raw-event read beside the timeline page, recorded as an amendment to design §6.12.
    • Sessions: a session crossing the 30-day cutoff is read whole.
    • evidence_event_ids is always [].
    • Transitions: one row per day.

Follow-ups

🤖 Generated with Claude Code

FlyM1ss and others added 26 commits October 15, 2026 13:00
Adds resolve_group_badge (admin > user_group > tier precedence),
operational_reason, and operational_result_from_code, which rebuilds a
display-safe result from a stored reason code. None maps to the healthy
no_supported_issue result, since history-copy rows carry a NULL code.

Co-Authored-By: Claude Haiku 5.5 <[email protected]>
run_startup_migrations now seeds user_activity from the legacy row one last
time, then calls the new drop_legacy_snapshot_tables() on the analytics base
store (both twins, DROP TABLE IF EXISTS). The seed returns 0 once the table is
gone, so later boots are no-ops. The snapshot models, the store methods, the
history copy and the legacy column migrations are deleted. app.py runs the
migration in its own try, so a failure there still starts the worker.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Removes user_state_counts from GET /overview and the status filter from
GET /users (now 422), end to end: model, filters, SQL predicates, router,
fixtures and tests. _legacy_state stays for the profile's state summary.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…s and allowlist reasons

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…shot tables

run_startup_migrations is now seed -> legacy rollup -> drop. Every
user_lifecycle_daily_snapshots day with no fact row and no lifecycle-owned
rollup is written as anonymous lifecycle_segment_count / lifecycle_transition
rows in main's rollup_lifecycle_day shape, so the drop no longer destroys
movement history older than the PR A 56-day copy. Either step raising skips
the drop. Missing legacy tables are detected from the catalogue (sqlite_master,
to_regclass) instead of English error text. rollup_lifecycle_day warns when a
day has transitions but no facts; runs_by_* semantics documented.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
_GROUP_TOTALS_SQL counts partial rows per group; get_groups downgrades to
partial when the (selected) window holds any, so the PR A history copy's
zero-filled days are no longer labelled ready.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
aggregate_operator_cost_for_day (both twins) skips runs whose
metadata.llm_execution.billing_mode is byok, so the profile's ATL platform
model cost, /groups atl_cost_micro_usd and the permanent operator_cost_by_*
rollups no longer include the user's own-key spend. Runs with no lane or
malformed metadata keep counting.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
…ifecycle history

Retention cohorts always come from the facts-eligible population (facts hold
no internal users, so admitting them counted eligible-never-retained), and
/lifecycle history always falls back to the anonymous rollups. The lifecycle
raw-read test now asserts every section is available, so a swallowed read
fails it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
When yesterday has no fact rows, the users page joins the latest landed
snapshot_date (one MAX() read, both twins via a shared template), stamps the
operational as_of from it, and prints
WARNING: analytics.read.users_facts_stale, instead of silently reading every
row as healthy/unpaid with an empty priority view.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
… failures

Friction availability now equals funnel availability (top_failure_categories
is computed from the funnel's reads); the count_states_for_date probe whose
result was discarded is gone. Every silent except branch on the overview,
lifecycle and groups read paths prints
WARNING: analytics.read.<section>_failed category=<class>, never the message.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
backfill_analytics docstring and the script's --help now say inserted events
update only the raw log and day rollups. CLAUDE.md marks the two snapshot
recompute functions it names as deleted.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@vercel

vercel Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
agentic-trading-lab Ready Ready Preview Oct 17, 2026 12:24am UTC

FlyM1ss and others added 2 commits October 16, 2026 19:32
…tion

Review findings on #641 (store, job and migration side):
- Derive lifecycle_transitions for the PR A history-copy days from
  consecutive fact days (first copied day from the legacy table) before
  the drop.
- The daily job replaces a day's transition log instead of upserting
  movers only, re-derives the next day's log after a late recompute, and
  re-queues a recompute that fails part-way.
- Fact reads apply admin/excluded filtering at read time; the replace
  spares rows the filter only hides.
- rollup_day writes activation_users zero or not; the startup migration
  fills rolled-up days inside raw retention, independent of the drop.
- The migration runs once, on the worker, retried before each tick;
  app.py no longer runs it on the event loop.
- Delete the dead rollup_current_day.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Review findings on #641 (read side):
- Overview attention, /lifecycle current and /operational state counts
  fall back to the latest landed fact day (labelled stale, or refused when
  the panel cannot say so) instead of empty counts; /groups reads coverage
  instead of assuming start < today landed.
- The funnel is unavailable while a rolled-up day lacks activation_users.
- Today's overview slice and the platform-cost read are internal-free,
  like the rollups they add to.
- Retention counts ledger activity again; profile milestones are first
  occurrences (design SS6.12 amended); sessions crossing the 30-day cutoff
  are read whole.
- The stale-facts WARNING logs once per stale day, not per request.
- Drop the dead use_anonymous_rollups parameter and double chunking.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
top_failure_categories summed per-day distinct users unfiltered (user-days)
and events under a billing/provider/model filter, under a field named
affected_users. The anonymous rollups can give an exact event count and
nothing else, so both paths now sum event_count rows plus today's events.
The field is renamed event_count, the page column reads "Events", and
rollup_day stops writing the affected_users family nothing reads. Design
D13 amended.

Closes #647

Co-Authored-By: Claude Opus 5.5 <[email protected]>

This branch was successfully deployed

1 active deployment
Preview — b32c843e Deployed Oct 17, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

admin: decide what the overview's failure count means

1 participant