diff --git a/CLAUDE.md b/CLAUDE.md index acb4fe7c2..514eed3ed 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -85,7 +85,7 @@ python dashboard/scripts/backtest_hourly_agent.py # main hourly agent backte - `ATL_BACKTEST_WORKER` (**never set by an operator**): `run_backtest_background` sets it to `1` in a dashboard backtest child's environment and nowhere else. Every Postgres store twin that owns a schema — **eleven of the twelve pairs in `tests/test_store_twin_parity.py::_TWINS`**; the twelfth, `PostgresValueAnalyticsStore`, composes the other stores and has no `_init_schema`, which that file's `_NO_OWN_DDL_TWINS` already records — routes its constructor's DDL through `db_url.init_schema_unless_worker`, which skips it on the literal `1` and prints ` backend: schema init skipped (backtest worker)`, and otherwise runs it, times it into `db_url.schema_init_seconds()` and prints ` backend: schema init s`. The parent ran that DDL at import, and the child was paying a pooled Neon checkout plus a batch of round trips per store before reading a bar: on today's import graph six twins construct in a child — `PostgresBacktestDatabase` (55 statements), `PostgresModelProviderStore` (18), `PostgresAnalyticsStore` (16), `PostgresAgentStore` (14), `PostgresCreditsStore` (3), `BrokerConnectionStorePostgres` (1), 107 statements over six checkouts. A warm schema does not shrink that: 72 of the 107 are `ADD COLUMN IF NOT EXISTS` and the other five `ALTER`s are constraint drops and re-adds, so they round-trip whether or not the column is there. ⚠ **`_init_schema` is not purely DDL, so the guard skips more than its name says.** Three of the eleven twins run data statements inside it — `PostgresCreditsStore` seeds the `default` grant pool and backfills settled reservation amounts plus ledger bucket/operation columns, `PostgresModelProviderStore` scrubs `api_key_enc` on revoked credentials, seeds `SEEDED_PROVIDERS` and backfills the CommonStack model allowlist into an already-seeded row, `PostgresUserStore` repairs `users.user_group` values outside the allowed set. A child skips those too, and they are safe to skip **only because the parent applies them at boot before it spawns anything** — a property of how dashboard backtests are launched rather than of this code, so a worker that ever ran without a parent ahead of it (a bare CLI run with the flag exported, a future worker-only service) would silently not have them. `test_backtest_worker_schema_skip.py` holds that set as a declared registry and fails in both directions, so a fourth cannot arrive unnoticed and a stale entry cannot linger; read it before adding a migration to any guarded `_init_schema`. **Both log directions are deliberate:** the timed line is the parent's own boot cost, which is the only pre-change baseline the deploy that removes the child's copy can still produce, and a silent skip would look exactly like a build with no guard. ⚠ **All eleven are guarded, not just those six, on purpose.** Which stores a child imports is an accident of the import graph, and the accident arrives by **two independent routes**. The script imports `db` deliberately (`backtest_hourly_agent.py:51`) and then reaches `credits_store` and `model_provider_store` through the two service imports at `:188-189` (`credits/service.py:27`, `model_providers/service.py:38`), with `model_providers/repository.py:13-14` pulling in `agents/repository.py` and `brokers/repository.py` — five of the six twins, with `domain/analytics/service.py` never imported. Only the sixth, `analytics_store`, needs the analytics chain those same service modules also pull in (`domain/analytics/instrumentation.py:15` → `analytics/service.py:17`, whose module-level `analytics_service = _build_analytics_service()` at `:253` then constructs a value store that resolves `credits_store`, `model_provider_store`, `agent_store` and `run_store` as constructor-time defaults — the first three already built by route one, `run_store` (like `analytics_store`) built here for the first time). `run_store` does not move the count: it is a seventh module-level singleton (`domain/runs/repository.py:469`), not a seventh twin — `domain/runs/` has no `repository_postgres.py`, so protocol runs are SQLite either way and nothing here guards them. Neither route is legible from the child's own import block, which names `db` and two service modules and nothing else, so one new module-level import in the engine's reach would otherwise re-add an unguarded payer with nothing going red. `test_backtest_worker_schema_skip.py` pins the invariant that survives that, in the AST rather than as a substring: no non-test `*_postgres.py` calls **any** `self._…schema…()` method straight from `__init__`, any twin defining a schema method names `init_schema_unless_worker`, and guarded ∪ exempt equals `_TWINS`. It matched the literal `"self._init_schema()"` until 2026-09-21, which is a check on one spelling — a twin calling `self._ensure_schema()` passed it while doing the forbidden thing. One shared helper rather than eleven copies, for the reason `db_url.py`'s header gives about its scrubber — an inverted copy fails in the *parent*, as an `UndefinedTable` on prod from a store nobody edited, and a dropped log line makes "skipped" and "never constructed" the same empty stdout. Named for the process role, not the DDL, so nobody exports it globally and later child-only behaviour has a home. The child also publishes pre-loop **phases** to its progress file (`engine.py:PROGRESS_PHASES`: `starting`, `loading_bars`, `indicators`, `first_decision`, `running`, `saving`), which `/backtest/status` turns into the card's sentence and which double as the start-up measurement — read `phases[]` off a run's progress file for the per-phase timings, or the child's `⏱ phase …` stdout lines out of the service log, which is the copy that survives (the parent unlinks the progress file when the run ends). Every finished entry carries `duration_seconds`, measured on `time.monotonic()` — do **not** re-derive a duration by differencing `started_at`/`ended_at`, which are deliberately wall-clock stamps kept for correlating against a log line: `time.time()` is not monotonic, so an NTP step between two reads distorts the interval and a backward step yields a negative one. ⚠ `starting` is the one phase whose duration *is* that wall-clock difference, and it cannot be otherwise — it spans the parent→child boundary (it opens at the parent's `--launched-at` and closes in the child) and `monotonic()` has a **per-process epoch**, so the parent's reading and the child's are not comparable at all. `_init_progress_phases` leaves the steady mark `None` to say exactly that, and `_set_progress_phase` reads `None` as "fall back to the wall clock"; every later phase is entirely in-child and uses the steady clock. The `starting` entry additionally carries `child_entered_at`, `imports_done_at` and `schema_init_seconds`, splitting that one interval into spawn, imports-including-stores, of-which-DDL and preflight: one phase name, four numbers, because a single figure covering all four cannot say which of them an optimisation moved. ⚠ **That exemption covers the phase's total and exactly one of those four splits — do not read it as covering the record.** Only `spawn+interpreter` (`child_entered_at - started_at`) reaches back into the parent; `imports+stores` and `preflight` both begin and end inside the child, so both are measured on the steady clock and published as `imports_seconds` and `preflight_seconds` beside the stamps. The launch script takes a second, steady pair for this (`CHILD_ENTERED_STEADY`/`IMPORTS_DONE_STEADY`, handed over in `startup_clock`) and the engine closes `preflight` against its own `steady_clock()` read, which is the same clock in the same process. Those two marks are deliberately **not** published — a raw `monotonic()` reading has a per-process epoch and is meaningless in a file — so what `phases[]` gains is the two durations, not the marks. The wall-clock fallback for a caller that hands over only the three stamps is kept and is *silent*, so it is pinned in the source rather than at runtime: `test_backtest_launch_phases.py` AST-checks that the script still hands over the steady pair, that both are `time.monotonic()` reads, and that they bracket the imports. Shipping the #509 fix with these two differences still in the breakdown line is the mistake that guard exists to prevent repeating. `loading_bars` carries `fetch_seconds` for the same reason and prints `fetch s | aggregate+verify s` on the transition out: the phase is one name, but with the bar cache warm the residual **is** the aggregation cost, measured rather than inferred — which is what decides whether caching the aggregated output is worth a second change. `record_phase_metric` attaches a number to whichever phase is open — and drops it when none is — and extras are cleared on every transition, so a figure can never be reported against the wrong phase. The startup clock is seeded only when `--launched-at` actually opens `starting`; a bare CLI run passes the clock without a launch time, and those keys must not surface on `loading_bars`. `tests/conftest.py` strips the flag. - `MAX_AI_HEDGE_FUND_TRADING_DAYS` (optional, default **10**, range 0–60, **0 disables the hosted runtime**): the pre-flight bound `POST /backtest/run` applies to an AI Hedge Fund window (`_enforce_ai_hedge_fund_window` in `api/routers/backtests.py`). The hosted runtime spends one upstream **subprocess per trading day**, each loading its own lookback window beside a parent already holding uvicorn, FastAPI and the Postgres pools, and the kernel's victim is the whole web process — one backtest denies service to everyone (issue #308). An over-long window answers **422** naming the bound and the requested size (the caller can shorten it); `0` answers **503** (the deployment's own configuration, nothing the caller can change). Measured in trading days via `_estimated_decision_days`, the same upper bound `_backtest_subprocess_timeout` sizes the parent budget from, so the two cannot disagree about how big a window is. Junk/negative/over-ceiling values log and fall back rather than raising at import — a bare `int()` at module scope in this very module once killed app boot. **The default stays 10 even though prod is now 2GB** (2026-09-11): nothing has ever measured one child's resident set, so 10 was a guess against the old ceiling and a larger guess against a larger ceiling is the same mistake with more RAM behind it. Profile a run, then raise it from the Render dashboard — no deploy needed. - `MAX_LEGACY_ACTIVE_PER_SESSION` / `MAX_LEGACY_ACTIVE_GLOBAL` (optional, defaults **5** / **50**, 0 disables): concurrency budgets for the legacy `/api/v1/backtest/*` surface. That surface authenticates nothing — `_require_session` accepts any non-empty `X-Session-Id` — and writes **no `protocol_runs` row**, so the per-agent, per-account and global protocol caps are all blind to its runs; before these it was the one unbounded path into the same engine. The per-session budget bounds a looping client (its key is caller-chosen, so it is not a bound against someone who rotates it); the global one is the memory bound that holds regardless, since every live session pins a loaded bar window. Enforced via `start_backtest(enforce_session_cap=True)`, which **only** the legacy route passes — the protocol surfaces reach the same function through `run_service.create_run`, which has already applied its three caps. -- `ANALYTICS_DAILY_JOB_INTERVAL_SECONDS` (optional, default **300**, range 5–3600): how often the analytics daily-facts worker (`domain/analytics/daily_job.py`, started from `app.py` beside the run reaper) wakes to ask whether yesterday's `user_daily_facts` row set is due. It runs on **its own thread**, not the 60-second reaper tick — the reaper exists to keep run heartbeats fresh against `RUN_HEARTBEAT_STALE_SECONDS`, and a whole-population batch across three databases on that thread would let a slow analytics night mark live runs as orphaned (design D23). An idle wake costs exactly one store call (the refused day claim on `analytics_projection_jobs`), so the interval buys freshness after midnight, not cost. Junk or out-of-range values log and fall back to 300 rather than raising at import — a bare `int()` at module scope has killed app boot in this repo before. `tests/conftest.py` strips it. The worker also runs the idempotent `user_activity` seed and the eight-week history copy (`domain/analytics/facts_migration.py`) once before its first tick, and owns `rollup_day` and the analytics retention coordinator; the reaper keeps only the two throttled snapshot repairs in `maintenance.py` until PR B deletes them. Design: `docs/superpowers/specs/2026-09-15-admin-layer-redesign-design.md` §6.9, §6.12. +- `ANALYTICS_DAILY_JOB_INTERVAL_SECONDS` (optional, default **300**, range 5–3600): how often the analytics daily-facts worker (`domain/analytics/daily_job.py`, started from `app.py` beside the run reaper) wakes to ask whether yesterday's `user_daily_facts` row set is due. It runs on **its own thread**, not the 60-second reaper tick — the reaper exists to keep run heartbeats fresh against `RUN_HEARTBEAT_STALE_SECONDS`, and a whole-population batch across three databases on that thread would let a slow analytics night mark live runs as orphaned (design D23). An idle wake costs exactly one store call (the refused day claim on `analytics_projection_jobs`), so the interval buys freshness after midnight, not cost. Junk or out-of-range values log and fall back to 300 rather than raising at import — a bare `int()` at module scope has killed app boot in this repo before. `tests/conftest.py` strips it. The worker is also the only caller of the startup migration (`domain/analytics/facts_migration.py`: the idempotent `user_activity` re-seed, the legacy lifecycle rollup and history transition backfill, the drop of the two five-state snapshot tables, then the `activation_users` rollup backfill), run before its first tick and retried before each later one until it succeeds — never synchronously in `app.py`'s startup hook, where it blocked the event loop — and owns `rollup_day` and the analytics retention coordinator; PR B deleted `maintenance.py`, `states.py` and `lifecycle_backfill.py`, so the reaper carries no analytics sweep. Design: `docs/superpowers/specs/2026-09-15-admin-layer-redesign-design.md` §6.9, §6.12. - `LEADERBOARD_DAILY_AUTO_DEPLOY` (optional, **strict opt-in — this one spends money**): when truthy, serving the *public, unauthenticated* `GET /api/v1/leaderboard?period=daily` may start a background thread that runs `deploy_model_run` for every competition LLM entry — real billable API calls initiated by an anonymous request. It is therefore off unless explicitly set (`1`/`true`/`yes`/`on`); do **not** restore the old "on whenever `RENDER` is unset" default, which armed it on the Docker image, every self-host and fork, and the test suite (`tests/conftest.py` deliberately strips `RENDER`, and strips this var too). Prod previously relied on the nightly refresh job instead — **but that schedule is paused as of PR #352** (see `LEADERBOARD_DAILY_REFRESH_SECRET` above), so prod currently runs *neither* this flag nor the cron. Nothing refreshes the daily board automatically today, by design: the Live Trading Leaderboard that replaced its tab is a Season 0 preview with no advance engine yet. Do not "restore" automation here to close the gap — arming this flag is the anonymous-billable-request path the strict opt-in exists to prevent. Note the in-progress guard and the per-window state file (`storage/data/leaderboard_daily_refresh.json`) are **per-process**: fine for the current single-instance Render deploy, but a multi-replica deploy would duplicate model deploys. - `NEWS_FEED_FROZEN` (optional, **unset = frozen** while FinSearch's feed timers are paused, since 2026-10-11): read per request by `api/routers/news.py::news_feed_frozen()`. Frozen, `GET /api/news/signals` never calls FinSearch and serves `dashboard/config/news_demo_snapshot.json` (the last live payload); the loader adds `demo: true` and `staleness_hours: null`, so both Home news consumers (`home-page.js`, `home-news-signals.js`) show "Demo news" and fixed dates instead of a frozen "Updated 0.2h ago" and ever-growing "3w ago". To resume: re-enable the FinSearch timers on the droplet first, then set `NEWS_FEED_FROZEN=0` in the Render dashboard (no deploy). Junk reads as frozen and warns once. A snapshot that fails to load fails closed to `unavailable` and is retried on the next request. The adapter (see *Fail-closed is not fail-visible*) still backs `/api/v2` backtest steps (`?as_of=`) either way. `tests/conftest.py` strips it. - Alpaca paper-trading credentials also live in `credentials/alpaca.json` (gitignored; see `credentials/alpaca.json.example`). @@ -99,7 +99,7 @@ Pipeline is **backtest → SQLite → API → dashboard**. The backend is layere - **`execution/`** — v2 execution backends binding domain engines to the `/api/v2` contract: `base.py` (interface), `backtest_backend.py` (implemented), `paper_backend.py` (**stub** — raises `NotImplementedError`; Phase B not built). Deliberately at the backend root (not `domain/`) so it can bridge domain→API without tripping the `domain/`→`api/` import ban. - **`infrastructure/`** — `llm/` (the `validator` security boundary, `token_cost`, `backtest_harness`/gateway client), `market_data/` (Alpaca bars), and `brokers/` (`alpaca_paper.py`, the isolated Alpaca paper-trading HTTP adapter). - **Backend-root modules** — `middleware.py` (session enforcement + CSP), `users.py` (auth/bcrypt/session store), `cache.py` (TTL cache for paper-trading responses), `baseline_generator.py`/`baseline_resolver.py`/`baselines_endpoint.py` (shared baseline equity-curve generation + DJIA/buy-hold baselines for backtests and paper trading), `llm_integration_example.py` (reference safe-LLM pattern). (`engines/` and `services/` are **not** packages — they were pre-refactor compatibility shims, deleted once their code moved under `domain/`; `test_architecture_boundaries.py`'s `_DELETED_SHIMS` list asserts they stay non-importable.) -- **Persistence** (`database.py` + per-store repositories like `domain/runs/repository.py`, `domain/strategies/repository.py`): thin SQLite wrappers over `DATABASE_PATH` in **WAL journal mode** (readers aren't blocked by finalize's heavy writes); schema is created lazily and self-migrates. `agent_runs` carries a JSON `metadata` column recording the effective `LLM_MAX_OUTPUT_TOKENS` per run. `domain/analytics/value_repository.py`'s `ValueAnalyticsStore` had 21 `is_postgres` branches and no Postgres twin until the 2026-09-15 admin layer redesign's PR T split it into `value_repository_postgres.py`'s `PostgresValueAnalyticsStore`, registered in `tests/test_store_twin_parity.py::_TWINS` like every other store pair. **Construct that pair only through `build_value_analytics_store()`**, which dispatches on the resolved `analytics_base`'s `database_url`: naming either class directly pairs a dialect with the wrong placeholder style, and because CI and local runs are both SQLite, the mismatch raises on the Postgres deployment alone. Both constructors now refuse a base of the wrong dialect rather than failing at the first query. That same PR added a **fourth parity axis** covering all 12 pairs: `_DUPLICATED_BODIES` in `test_store_twin_parity.py` declares, per pair, which method bodies are byte-identical across the twins, and `test_duplicated_bodies_match_their_declaration` fails both ways — on a declared body that stops matching (a fix applied to one copy only, which every other axis passes) and on a body that becomes identical without being declared. Adding a method to both halves of any twin pair can therefore turn the build red on its own; the fix is to share the implementation or name it with a reason, never to delete the name. +- **Persistence** (`database.py` + per-store repositories like `domain/runs/repository.py`, `domain/strategies/repository.py`): thin SQLite wrappers over `DATABASE_PATH` in **WAL journal mode** (readers aren't blocked by finalize's heavy writes); schema is created lazily and self-migrates. `agent_runs` carries a JSON `metadata` column recording the effective `LLM_MAX_OUTPUT_TOKENS` per run. `domain/analytics/value_repository.py`'s `ValueAnalyticsStore` had 21 `is_postgres` branches and no Postgres twin until the 2026-09-15 admin layer redesign's PR T split it into `value_repository_postgres.py`'s `PostgresValueAnalyticsStore`, registered in `tests/test_store_twin_parity.py::_TWINS` like every other store pair. **Construct that pair only through `build_value_analytics_store()`**, which dispatches on the resolved `analytics_base`'s `database_url`: naming either class directly pairs a dialect with the wrong placeholder style, and because CI and local runs are both SQLite, the mismatch raises on the Postgres deployment alone. Both constructors now refuse a base of the wrong dialect rather than failing at the first query. That same PR added a **fourth parity axis** covering all 12 pairs: `_DUPLICATED_BODIES` in `test_store_twin_parity.py` declares, per pair, which method bodies are byte-identical across the twins, and `test_duplicated_bodies_match_their_declaration` fails both ways — on a declared body that stops matching (a fix applied to one copy only, which every other axis passes) and on a body that becomes identical without being declared. Adding a method to both halves of any twin pair can therefore turn the build red on its own; the fix is to share the implementation or name it with a reason, never to delete the name. Analytics persistence after PR B: `user_activity` (current state, never swept), `user_daily_facts` and `lifecycle_transitions` (180 days, rolled up into `analytics_daily_rollups` before they expire) and the anonymous `analytics_daily_rollups` themselves (indefinite). The five-state snapshot tables are gone — one daily fact row per user replaced a per-event recompute — so a read path that wants "state" reads yesterday's fact or computes it live, never a snapshot. - **Frontend** — `dashboard/frontend/` is the served static root and holds **both** UIs: the **landing page** (`index.html` + `assets/`) served at **`/`**, and the vanilla-JS + Chart.js **dashboard** (`app.html`, `app.js`, `styles.css`, no build step) served at **`/app`**. The landing page is a Vite/React marketing site whose **source** lives in `dashboard/landing/` (Replit-exported, de-monorepo'd; `npm run build`); its build output ships as `frontend/index.html` + `frontend/assets/`. `app.py` adds a `/app/`→`/app` 308 redirect so the dashboard's relative asset paths resolve. Vercel deploys the static `dashboard/frontend`. - **Paths** (`dashboard/backend/paths.py`): single source of truth for on-disk locations. @@ -298,7 +298,7 @@ test passing period `"live"` or season `3` passes anyway). One design governs the administrator layer: `docs/superpowers/specs/2026-09-15-admin-layer-redesign-design.md`, with per-PR plans at `docs/superpowers/plans/2026-09-15-admin-layer-redesign-pr*.md`. It supersedes the 2026-08-26 → 2026-09-13 admin analytics specs (the two it absorbed are deleted; the two it kept are narrowed in place). Read it before touching `domain/analytics/`, `api/routers/admin_analytics.py`, `admin-tabs.js` or the admin page. Three facts it establishes that are easy to get wrong from the code alone: `/admin-analytics` is a static mock whose one redirect in `admin-tabs.js` made the wired in-app dashboard (`js/admin-analytics-value.js`, nine live `/api/admin/analytics/*` endpoints) unreachable; both 2026-09-11 outage burners (`states.py` 15-minute stale window on the 60-second reaper, `service.py` `project_snapshots=True`) were live until **PR 0 (#485, merged 2026-09-17)** killed them; and `ValueAnalyticsStore` carried 21 inline `is_postgres` branches with no Postgres twin — invisible to the parity guard, which starts from `*_postgres.py` files on disk — until **PR T (#498)** split it. The last two are now history: the design doc is written against the pre-PR-0 state, so read those two claims as describing the state it was written for, not the tree. See the Persistence bullet above for the rule that replaced the third. -PR 0 killed the synchronous recompute in the web process only. `instrumentation._recalculate_snapshot` fell back to `states.recalculate_user_snapshots` whenever no recalculator was registered, and only `app.py`'s startup registered one. So every dashboard backtest child kept rebuilding snapshots per event, at ~5 s each and ~31 s of every model call, until #522 made the unregistered default inert. **Do not restore that fallback.** A default that is expensive until something disables it is inherited by every entry point that skips `app.py`'s startup. `disable_synchronous_projection()`, the boot-time no-op PR 0 added, is deleted: nothing in production registers a recalculator. Design: `docs/superpowers/specs/2026-09-23-llm-backtest-step-latency-design.md` §4. +PR 0 killed the synchronous recompute in the web process only. `instrumentation._recalculate_snapshot` fell back to `states.recalculate_user_snapshots` (both since deleted) whenever no recalculator was registered, and only `app.py`'s startup registered one. So every dashboard backtest child kept rebuilding snapshots per event, at ~5 s each and ~31 s of every model call, until #522 made the unregistered default inert. **Do not restore that fallback.** A default that is expensive until something disables it is inherited by every entry point that skips `app.py`'s startup. `disable_synchronous_projection()`, the boot-time no-op PR 0 added, is deleted: nothing in production registers a recalculator. Design: `docs/superpowers/specs/2026-09-23-llm-backtest-step-latency-design.md` §4. ### External agents & Agent-Environment Protocol (`/api/v1`) @@ -355,4 +355,4 @@ The FinSearch news adapter (`dashboard/backend/integrations/news_sentiment.py`) **Merging to Open-Finance-Lab `main` auto-deploys prod**: a CI job hits the Render Deploy Hook once backend tests pass on `main` (PR #95, live since 2026-07-11) — no manual trigger or fork-sync needed. (Render's own `autoDeploy` is `"no"`; the CI hook drives every deploy.) - **Phantom `test_deleted_shim_is_not_importable` failures = stale bytecode, not a regression.** If those cases fail locally with `DID NOT RAISE ModuleNotFoundError`, it's leftover `dashboard/backend/{engines,services}/__pycache__/*.pyc` from the pre-refactor layout, which Python resolves as a PEP-420 namespace package. The dirs are untracked so CI is green; `rm -rf dashboard/backend/engines dashboard/backend/services` clears it. - **User accounts were silently lost on every prod redeploy until 2026-07 (see `docs/superpowers/plans/2026-07-08-user-account-persistence-fix.md`).** `users.py` originally shared `DB_PATH` with backtest data; on the live Render service (then free tier, `disk: null` — still true today, `DATABASE_PATH` unset) that file resets to the git-committed seed DB on every deploy, deleting the `users`/`auth_sessions` tables with no error surfaced anywhere. The fix is an optional Postgres backend selected via `USERS_DATABASE_URL` — set it in prod (see the **Prod deploy reality** bullet above); leave it unset for local dev/tests, which keep using SQLite exactly as before. The same fix was extended to agents, agent versions, and strategies in 2026-07 via `CONTENT_DATABASE_URL` (see `docs/superpowers/specs/2026-07-15-agent-strategy-persistence-design.md`) — before that, every registered agent and issued API key died on each deploy, breaking all SDK/Discord integrations, with `resolve_api_key()` as the sole auth path for `/api/v1` and `/api/v2`. -- **Anything registered with `register_reaper_sweep` runs on the same 60-second tick** (`RUN_REAPER_INTERVAL_SECONDS`, `domain/runs/service.py`) — there is no per-sweep interval, so a sweep that does per-user work multiplies its cost by every tick, not by however slow that work actually is. The analytics snapshot-repair sweep registered there (`domain/analytics/maintenance.py::run_analytics_maintenance` → `states.py::repair_stale_snapshots`) was one of two September 2026 outage burners for exactly this reason; it is throttled to a 24-hour staleness window by PR 0, left on the reaper by PR A (which gives the daily-facts job its own worker thread, `domain/analytics/daily_job.py`, precisely so that job is *not* another reaper sweep, and moves `rollup_day` and the retention coordinator onto it), and deleted outright by PR B once every read path has moved to `user_daily_facts`. The admin layer's full architecture — the analytics data model, the daily job, the nine `/api/admin/analytics` endpoints, and the `/admin` page — is designed at `docs/superpowers/specs/2026-09-15-admin-layer-redesign-design.md`; read it before touching anything under `domain/analytics/` or the admin frontend. +- **Anything registered with `register_reaper_sweep` runs on the same 60-second tick** (`RUN_REAPER_INTERVAL_SECONDS`, `domain/runs/service.py`) — there is no per-sweep interval, so a sweep that does per-user work multiplies its cost by every tick, not by however slow that work actually is. The analytics snapshot-repair sweep registered there (`domain/analytics/maintenance.py::run_analytics_maintenance` → `states.py::repair_stale_snapshots`) was one of two September 2026 outage burners for exactly this reason; it is throttled to a 24-hour staleness window by PR 0, left on the reaper by PR A (which gives the daily-facts job its own worker thread, `domain/analytics/daily_job.py`, precisely so that job is *not* another reaper sweep, and moves `rollup_day` and the retention coordinator onto it), and deleted outright by PR B after every read path moved to `user_daily_facts` (the tables it fed are dropped at boot by `facts_migration.py`). The admin layer's full architecture — the analytics data model, the daily job, the nine `/api/admin/analytics` endpoints, and the `/admin` page — is designed at `docs/superpowers/specs/2026-09-15-admin-layer-redesign-design.md`; read it before touching anything under `domain/analytics/` or the admin frontend. diff --git a/dashboard/backend/api/routers/admin_analytics.py b/dashboard/backend/api/routers/admin_analytics.py index 50403683c..609faf9d8 100644 --- a/dashboard/backend/api/routers/admin_analytics.py +++ b/dashboard/backend/api/routers/admin_analytics.py @@ -49,7 +49,6 @@ _PROVIDER_ID_PATTERN = re.compile(r"^[a-z0-9_]{2,64}$") _MODEL_ID_PATTERN = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._/\-:]{0,255}$") _POSITIVE_INTEGER_PATTERN = re.compile(r"^[0-9]+$") -_USER_STATES = {"blocked", "needs_attention", "dormant", "onboarding", "active"} _ACTIVITY_SECTIONS = {"timeline", "runs", "usage", "sessions"} _LIFECYCLE_SEGMENTS = {"new", "onboarding", "growing", "core", "at_risk", "dormant"} _OPERATIONAL_STATES = {"blocked", "needs_attention", "healthy"} @@ -209,7 +208,6 @@ def _value_user_filters(request: Request) -> tuple[UserValueFilters, int, int]: request, { "q", - "status", "lifecycle_segment", "operational_state", "commercial_tier", @@ -241,9 +239,6 @@ def _value_user_filters(request: Request) -> tuple[UserValueFilters, int, int]: user_group = parse_user_group(user_group) except ValueError: _invalid_query() - legacy_status = values.get("status") - if legacy_status is not None and legacy_status not in _USER_STATES: - _invalid_query() from_date = ( _parse_date(values["last_meaningful_activity_from"]) @@ -278,7 +273,6 @@ def _value_user_filters(request: Request) -> tuple[UserValueFilters, int, int]: priority=( _parse_bool(values["priority"]) if "priority" in values else False ), - legacy_status=legacy_status, include_internal=( _parse_bool(values["include_internal"]) if "include_internal" in values diff --git a/dashboard/backend/api/routers/backtests.py b/dashboard/backend/api/routers/backtests.py index 8ffef19fd..7b784b897 100644 --- a/dashboard/backend/api/routers/backtests.py +++ b/dashboard/backend/api/routers/backtests.py @@ -1194,7 +1194,7 @@ def _finalize_slot_locked( # Still `backtest_failed`: the user asked for a backtest, got nothing, # and was billed. A separate event name would need nine registrations # and would drop timeouts out of `metrics.py`'s `_TERMINAL_FAILURE`, - # `states.py`'s consecutive-failure alert and `backfill.py`'s terminal + # `lifecycle.py`'s consecutive-failure alert and `backfill.py`'s terminal # map -- the three things that exist to notice runs failing. What was # missing is the reason, not the outcome. return { diff --git a/dashboard/backend/app.py b/dashboard/backend/app.py index 4b734be17..658a7d0b9 100644 --- a/dashboard/backend/app.py +++ b/dashboard/backend/app.py @@ -355,34 +355,18 @@ def bar_cache_background(): except Exception as e: print(f"⚠️ Research sweeper start error: {e}") - try: - from dashboard.backend.domain.analytics.maintenance import ( - run_analytics_maintenance, - ) - from dashboard.backend.domain.runs.service import register_reaper_sweep - - register_reaper_sweep(run_analytics_maintenance) - print("🧹 Analytics maintenance sweep registered with the reaper") - except Exception as e: - print( - "WARNING: analytics.maintenance_registration_failed " - f"category={type(e).__name__}" - ) - try: # Admin layer redesign PR A (design D23, SS6.9): the daily-facts job # runs on its own thread, not as a reaper sweep -- a whole-population # batch across three databases on the heartbeat thread would let a # slow analytics night mark live runs as orphaned. The worker also - # owns rollup_day and the retention coordinator now, and runs the - # idempotent user_activity seed + history copy once before ticking. + # owns rollup_day and the retention coordinator now, and PR B's + # startup migration (re-seed user_activity, roll up and derive the + # legacy history, drop the snapshot tables) runs on it before its + # first tick -- never here, where it would block the event loop. from dashboard.backend.domain.analytics.daily_job import ( start_daily_facts_worker, ) - from dashboard.backend.domain.analytics.facts_migration import ( - run_startup_migrations, - ) - run_startup_migrations() start_daily_facts_worker() print("📊 Analytics daily-facts worker started") except Exception as e: diff --git a/dashboard/backend/database.py b/dashboard/backend/database.py index 5b19da4bd..3b1b314f3 100644 --- a/dashboard/backend/database.py +++ b/dashboard/backend/database.py @@ -1018,6 +1018,12 @@ def aggregate_operator_cost_for_day(self, day: date) -> Dict[int, int]: ``est_cost_usd`` is a float; converted with ``round(value * 1_000_000)`` and clamped at zero so a negative stored value cannot violate ``user_daily_facts.operator_cost_micro``'s CHECK. + + BYOK runs are excluded: ``engine.py`` stamps ``est_cost_usd`` on them + too, from the user's own provider bill, which is not operator spend + (design SS6.7 "for operator-funded runs"). The lane is read from + ``metadata.llm_execution.billing_mode``; a run that records none (rule + based, pre-handoff) or whose metadata is not valid JSON keeps counting. """ start = f"{day.isoformat()} 00:00:00" end = f"{(day + timedelta(days=1)).isoformat()} 00:00:00" @@ -1029,6 +1035,12 @@ def aggregate_operator_cost_for_day(self, day: date) -> Dict[int, int]: FROM agent_runs WHERE owner_user_id IS NOT NULL AND updated_at >= ? AND updated_at < ? + AND COALESCE( + CASE WHEN json_valid(metadata) + THEN json_extract(metadata, '$.llm_execution.billing_mode') + END, + '' + ) <> 'byok' GROUP BY owner_user_id """, (start, end), diff --git a/dashboard/backend/database_postgres.py b/dashboard/backend/database_postgres.py index dc5282418..800e44ea7 100644 --- a/dashboard/backend/database_postgres.py +++ b/dashboard/backend/database_postgres.py @@ -1105,7 +1105,9 @@ def get_runs_by_mode(self, mode: str) -> List[Dict]: return [BacktestDatabase._parse_run_row(row) for row in rows] def aggregate_operator_cost_for_day(self, day: date) -> Dict[int, int]: - """See the SQLite twin.""" + """See the SQLite twin. ``metadata`` is TEXT here as well; ``IS JSON + OBJECT`` (PostgreSQL 16+, prod and CI run 18) guards the cast so one + malformed row cannot fail the whole day's statement.""" start = f"{day.isoformat()} 00:00:00" end = f"{(day + timedelta(days=1)).isoformat()} 00:00:00" with self._get_connection() as conn: @@ -1116,6 +1118,12 @@ def aggregate_operator_cost_for_day(self, day: date) -> Dict[int, int]: FROM agent_runs WHERE owner_user_id IS NOT NULL AND updated_at >= %s AND updated_at < %s + AND COALESCE( + CASE WHEN metadata IS JSON OBJECT + THEN metadata::jsonb -> 'llm_execution' ->> 'billing_mode' + END, + '' + ) <> 'byok' GROUP BY owner_user_id """, (start, end), diff --git a/dashboard/backend/domain/analytics/backfill.py b/dashboard/backend/domain/analytics/backfill.py index 472fbf94c..e6d4b45f4 100644 --- a/dashboard/backend/domain/analytics/backfill.py +++ b/dashboard/backend/domain/analytics/backfill.py @@ -108,7 +108,6 @@ class BackfillReport(BaseModel): write_failures: int = Field(default=0, ge=0) repair_failures: int = Field(default=0, ge=0) affected_users: int = Field(default=0, ge=0) - repaired_snapshots: int = Field(default=0, ge=0) rebuilt_rollup_days: int = Field(default=0, ge=0) source_event_ids: list[str] = Field(default_factory=list) @@ -509,13 +508,20 @@ def backfill_analytics( dry_run: bool = False, source: BackfillSource | None = None, store: Any = analytics_store, - recalculate_snapshot: Any | None = None, rebuild_rollup: Any | None = None, ) -> BackfillReport: """Reconstruct safe events inside ``[now - days, before]``. - Bulk repairs run once per affected subject/day after all inserts complete. + Rollup repairs run once per affected day after all inserts complete. Dry runs enumerate and validate sources but perform no writes at all. + + Inserted events reach the raw log and the day rollups only. They do not + update ``user_activity`` (activation date, last meaningful activity) or + ``user_daily_facts``: events go in through ``store.append_event``, not the + ingestion path that maintains ``user_activity``, and the daily job's + late-arrival sweep recomputes facts only for the last + ``RECOMPUTE_LOOKBACK_DAYS`` days. A historical backfill therefore moves no + one's activation date or lifecycle segment. """ if isinstance(days, bool) or not isinstance(days, int) or not 1 <= days <= 180: @@ -593,14 +599,6 @@ def backfill_analytics( if candidate.occurred_at.date() < current.date(): affected_days.add(candidate.occurred_at.date()) - if recalculate_snapshot is None: - from .states import AnalyticsStateStore, recalculate_user_snapshot - - state_store = AnalyticsStateStore(store) - - def recalculate_snapshot(user_id: int, **kwargs: Any) -> Any: - return recalculate_user_snapshot(user_id, store=state_store, **kwargs) - if rebuild_rollup is None: from .rollups import AnalyticsRollupStore, rollup_day @@ -610,14 +608,7 @@ def rebuild_rollup(day: date, **kwargs: Any) -> Any: return rollup_day(day, store=rollup_store, **kwargs) repair_failures = 0 - repaired_snapshots = 0 rebuilt_rollup_days = 0 - for user_id in sorted(affected_users): - try: - recalculate_snapshot(user_id, now=current) - repaired_snapshots += 1 - except Exception: - repair_failures += 1 for day in sorted(affected_days): try: rebuild_rollup(day, now=datetime.combine( @@ -640,7 +631,6 @@ def rebuild_rollup(day: date, **kwargs: Any) -> Any: write_failures=write_failures, repair_failures=repair_failures, affected_users=len(affected_users), - repaired_snapshots=repaired_snapshots, rebuilt_rollup_days=rebuilt_rollup_days, source_event_ids=inserted_ids, ) diff --git a/dashboard/backend/domain/analytics/daily_facts.py b/dashboard/backend/domain/analytics/daily_facts.py index 9586736fa..f3b0da863 100644 --- a/dashboard/backend/domain/analytics/daily_facts.py +++ b/dashboard/backend/domain/analytics/daily_facts.py @@ -2,7 +2,7 @@ Every query here is parameterised by a date, never by a user id. That is the property the read-budget test enforces, and it is the difference between this -module and the snapshot sweep it replaces: cost grows with days, not with +module and the per-user snapshot sweep it replaced: cost grows with days, not with users times minutes (design SS6.9, SS6.12). Scheduling lives in ``daily_job.py`` (its own worker thread, D23). This module @@ -110,6 +110,44 @@ def _count(totals: Any, field: str) -> int: return int(getattr(totals, field)) if totals is not None else 0 +def _rederive_next_day_transitions( + day: date, facts: list[UserDailyFact], *, store: Any, now: datetime +) -> int: + """Rebuild ``day + 1``'s transition log against ``day``'s rewritten facts. + + A transition's ``from_segment`` is the previous day's segment, so + recomputing a past day can invalidate the next day's log without + touching its facts. Only the late-arrival sweep calls this (the target + day's successor is today, which has no facts). ``inactive_days`` belongs + to the later day's own evaluation, so an existing row keeps its value; a + move that only appears now has none to keep and takes the column's 0. + """ + next_day = day + timedelta(days=1) + later = store.list_facts_for_date(next_day) + if not later: + return 0 + existing = {row.user_id: row for row in store.list_transitions_for_date(next_day)} + segment_on_day = {row.user_id: row.lifecycle_segment for row in facts} + rows: list[LifecycleTransitionRow] = [] + for fact in later: + before = segment_on_day.get(fact.user_id) + if before is None or before == fact.lifecycle_segment: + continue + kept = existing.get(fact.user_id) + rows.append( + LifecycleTransitionRow( + user_id=fact.user_id, + snapshot_date=next_day, + from_segment=before, + to_segment=fact.lifecycle_segment, + inactive_days=kept.inactive_days if kept is not None else 0, + data_quality=fact.data_quality, + created_at=now, + ) + ) + return store.replace_lifecycle_transitions(next_day, rows) + + def _compute_day( day: date, *, @@ -117,12 +155,17 @@ def _compute_day( store: Any, run_history_store: Any, rollup: Callable[..., Any], + rederive_next: bool = False, ) -> _DayOutcome: """Steps 1-6 of design SS6.9 for one day, each wrapped individually. Step order differs from the design's numbering in one place: the eligible population is read first, because the operational step (5) answers for exactly those ids. Query count is unchanged. + + ``rederive_next`` is set for a late-arrival recompute, whose successor + day already has facts and therefore a transition log keyed to this + day's segments. """ from .rollups import AnalyticsRollupStore @@ -158,7 +201,7 @@ def subjects_step() -> list[dict[str, Any]]: return _DayOutcome(failed_steps=tuple(failed)) eligible_ids = [int(subject["id"]) for subject in subjects] - # 1. Anonymous rollups for D (moved here from run_analytics_maintenance). + # 1. Anonymous rollups for D. step( "rollup", lambda: rollup(day, store=AnalyticsRollupStore(analytics), now=_midnight(now.date())), @@ -320,7 +363,14 @@ def transitions_step() -> int: created_at=now, ) ) - return store.append_lifecycle_transitions(changed) + # Whole-day replace: a recompute that puts a user back on yesterday's + # segment must also remove the move the first run recorded. + written = store.replace_lifecycle_transitions(day, changed) + if rederive_next: + written += _rederive_next_day_transitions( + day, [row for row, _days in rows_with_days], store=store, now=now + ) + return written transitions = step("transitions", transitions_step, 0) return _DayOutcome( @@ -385,14 +435,22 @@ def run_daily_facts( "WARNING: analytics.daily_facts.recompute_scan_failed " f"category={type(exc).__name__[:80]}" ) - # One store call after the loop, never inside it: the lease belongs to + # Store calls after the loop, never inside it: the lease belongs to # the job row, not the day, and the discipline guard forbids store - # calls inside loops (design SS6.11 rule 7). A failed stale day stays - # in list_days_needing_recompute, so the next tick retries it. + # calls inside loops (design SS6.11 rule 7). A failed stale day is + # re-queued explicitly: a step that failed after the facts upsert had + # already refreshed calculated_at, which is what takes a day off + # list_days_needing_recompute -- so without it, the day's partial rows + # (or a stale next-day transition log) would stand for good. failed_recomputes: list[date] = [] for stale_day in stale_days[:MAX_RECOMPUTES_PER_TICK]: stale_outcome = _compute_day( - stale_day, now=current, store=store, run_history_store=runs, rollup=rollup_fn + stale_day, + now=current, + store=store, + run_history_store=runs, + rollup=rollup_fn, + rederive_next=True, ) if stale_outcome.failed_steps: failed_recomputes.append(stale_day) @@ -403,6 +461,7 @@ def run_daily_facts( continue recomputed.append(stale_day) if failed_recomputes: + store.requeue_days_for_recompute(failed_recomputes) store.release_projection_day(DAILY_FACTS_JOB, now=current) report = DailyFactsReport( diff --git a/dashboard/backend/domain/analytics/daily_job.py b/dashboard/backend/domain/analytics/daily_job.py index 0007d5dbd..86e275cbe 100644 --- a/dashboard/backend/domain/analytics/daily_job.py +++ b/dashboard/backend/domain/analytics/daily_job.py @@ -7,9 +7,11 @@ inside the single web process (design SS2.3): no external scheduler, no second service. -Every tick is ``daily_facts.run_daily_facts``; the first thing the thread does, -once, is ``facts_migration.run_startup_migrations`` (seed ``user_activity``, -copy the legacy history). Both are idempotent, so a restart costs nothing. +Every tick is ``daily_facts.run_daily_facts``; the first thing the thread does +is ``facts_migration.run_startup_migrations`` (re-seed ``user_activity``, roll +up and derive the legacy history, then drop the legacy snapshot tables), and +until that succeeds it is retried before each tick. Both are idempotent, so a +restart costs nothing. """ from __future__ import annotations @@ -98,15 +100,29 @@ def start_daily_facts_worker( tick_fn = tick if tick is not None else _default_tick prepare_fn = prepare if prepare is not None else _default_prepare - def _loop() -> None: + def _prepare() -> bool: try: - prepare_fn() + report = prepare_fn() except Exception as exc: print( "WARNING: analytics.facts_migration_failed " f"category={type(exc).__name__[:80]}" ) + return False + if report is not None: + print(f"🧹 Analytics startup migration: {report!r}", flush=True) + return True + + def _loop() -> None: + # The only caller of the startup migration (app.py used to run it + # synchronously as well, on the event loop). A failure is retried + # before each tick rather than only at the next boot: the drop it + # guards is the step most likely to lose a lock to the outgoing + # instance on a deploy. + prepared = _prepare() while not stop.wait(interval): + if not prepared: + prepared = _prepare() try: tick_fn() except Exception as exc: diff --git a/dashboard/backend/domain/analytics/facts_migration.py b/dashboard/backend/domain/analytics/facts_migration.py index 98d8fa95a..cd4dc6160 100644 --- a/dashboard/backend/domain/analytics/facts_migration.py +++ b/dashboard/backend/domain/analytics/facts_migration.py @@ -1,10 +1,20 @@ -"""Startup migrations for the fact tables PR A creates. - -Both are idempotent and both run again on every boot; the worker thread in -``daily_job.py`` calls ``run_startup_migrations`` once before its first tick. -PR B re-runs the same seed immediately before dropping the legacy tables, -because the legacy snapshot row keeps being maintained between the two PRs -(design SS6.9 "Migration of existing history", SS11). +"""Startup migration from the five-state snapshot tables to the fact tables. + +Runs once per boot, as the daily-facts worker's ``prepare`` step +(``daily_job.py``), off the web process's event loop. A failed run is retried +before each later tick until one succeeds. Every step is idempotent, so a +retry, or the next boot, costs only the checks that find nothing to do. + +``user_activity`` was first seeded from ``user_analytics_snapshots`` when it +was created; the legacy row kept being maintained until the snapshot sweep was +deleted, so the same seed runs again immediately before both legacy tables are +dropped (design SS6.9 "Migration of existing history", SS11). Between the two, +every ``user_lifecycle_daily_snapshots`` day that no fact row covers is rolled +up into the anonymous lifecycle rollups, the only copy of it the drop leaves, +and the fact days the PR A history copy wrote without a transition log get one. +After the first boot that drops them, those reads find nothing and the drop is +``IF EXISTS``. Last, rolled-up days that predate the ``activation_users`` +rollup metric are filled from the raw events still inside retention. """ from __future__ import annotations @@ -15,14 +25,13 @@ from pydantic import BaseModel, ConfigDict, Field -HISTORY_COPY_DAYS = 56 - - class StartupMigrationReport(BaseModel): model_config = ConfigDict(extra="forbid", frozen=True) activity_seeded: int = Field(default=0, ge=0) - history_copied: int = Field(default=0, ge=0) + legacy_days_rolled_up: int = Field(default=0, ge=0) + transitions_backfilled: int = Field(default=0, ge=0) + activation_days_backfilled: int = Field(default=0, ge=0) def _utc_now(value: datetime | None) -> datetime: @@ -32,34 +41,67 @@ def _utc_now(value: datetime | None) -> datetime: return current.astimezone(timezone.utc) -def migrate_lifecycle_history(*, value_store: Any, now: datetime) -> int: - """Copy the last eight weeks of legacy daily snapshots; returns rows inserted.""" - current = _utc_now(now) - return value_store.copy_daily_snapshot_history( - since=current.date() - timedelta(days=HISTORY_COPY_DAYS), now=current - ) - - def run_startup_migrations( *, value_store: Any = None, now: datetime | None = None, ) -> StartupMigrationReport: - """Seed ``user_activity`` from the legacy row, then copy the history.""" + """Re-seed ``user_activity``, roll up and derive legacy history, drop, fill. + + The order is the point: the drop must never run before the seed has read + the final legacy row, or every pre-existing activation date is lost, nor + before the legacy lifecycle days that no fact row covers are rolled up, + or the movement history they hold is lost, nor before the transition + backfill has read the legacy day preceding the first copied fact day. + Any of those raising skips the drop; the next attempt tries again. + + The activation backfill shares none of that chain, so it runs whether + the chain succeeded or not -- a drop that keeps losing its lock must not + also keep the funnel unavailable. The first error (chain, then backfill) + is raised once both have had their turn, so the worker retries. + """ current = _utc_now(now) if value_store is None: from .repository import analytics_store from .value_repository import build_value_analytics_store value_store = build_value_analytics_store(analytics_store) - seeded = value_store.seed_activity_from_snapshots(now=current) - copied = migrate_lifecycle_history(value_store=value_store, now=current) - return StartupMigrationReport(activity_seeded=seeded, history_copied=copied) + from .retention import RAW_EVENT_RETENTION_DAYS + from .rollups import AnalyticsRollupStore, rollup_legacy_lifecycle_days + + errors: list[Exception] = [] + seeded, rolled_up, transitions = 0, (), 0 + try: + seeded = value_store.seed_activity_from_snapshots(now=current) + rolled_up = rollup_legacy_lifecycle_days(store=value_store) + transitions = value_store.backfill_transitions_from_history(now=current) + value_store.analytics_base.drop_legacy_snapshot_tables() + except Exception as exc: + errors.append(exc) + activation_days: tuple[Any, ...] = () + try: + # The oldest day whose raw events are still whole: retention deletes + # events older than RAW_EVENT_RETENTION_DAYS, so the boundary day may + # be partly gone and is left alone. + activation_days = AnalyticsRollupStore( + value_store.analytics_base + ).backfill_activation_users( + since=current.date() - timedelta(days=RAW_EVENT_RETENTION_DAYS - 1), + until=current.date(), + ) + except Exception as exc: + errors.append(exc) + if errors: + raise errors[0] + return StartupMigrationReport( + activity_seeded=seeded, + legacy_days_rolled_up=len(rolled_up), + transitions_backfilled=transitions, + activation_days_backfilled=len(activation_days), + ) __all__ = [ - "HISTORY_COPY_DAYS", "StartupMigrationReport", - "migrate_lifecycle_history", "run_startup_migrations", ] diff --git a/dashboard/backend/domain/analytics/instrumentation.py b/dashboard/backend/domain/analytics/instrumentation.py index b17bb5201..f31d47b46 100644 --- a/dashboard/backend/domain/analytics/instrumentation.py +++ b/dashboard/backend/domain/analytics/instrumentation.py @@ -45,42 +45,6 @@ "credits_settled", "credits_refunded", } -SNAPSHOT_RELEVANT_EVENTS = ( - _ACCOUNT_EVENTS - | _CREDENTIAL_EVENTS - | _AGENT_EVENTS - | _RUN_EVENTS - | _RESOURCE_EVENTS - | {"safe_error_recorded"} -) -# None means "no synchronous projection". That is the default in every process -# and nothing in production registers a callback, so no process rebuilds -# snapshots on the event path. Before #522, None fell back to -# states.recalculate_user_snapshots: ~5s per event in the child, ~31s of every -# model call. Snapshots now catch up only through the reaper's repair sweep, -# throttled to a 24-hour staleness window, for worker-emitted events exactly -# as for web-emitted ones. The daily job writes user_daily_facts, not snapshots. -_snapshot_recalculator: Any = None - - -def register_snapshot_recalculator(callback: Any) -> None: - global _snapshot_recalculator - if callback is not None and not callable(callback): - raise TypeError("snapshot recalculator must be callable") - _snapshot_recalculator = callback - - -def _recalculate_snapshot(user_id: int, event_name: str) -> None: - if event_name not in SNAPSHOT_RELEVANT_EVENTS: - return - callback = _snapshot_recalculator - if callback is None: - # Inert by default (#522). A default that is expensive until something - # disables it is inherited by every entry point that skips app.py's - # startup: the backtest child, CLI scripts, a future worker service. - # Do not add a fallback here. - return - callback(user_id) def _safe_warning(exc: BaseException) -> None: @@ -135,10 +99,7 @@ def _emit(*, allowed_names: set[str], group: str, **kwargs: Any) -> None: source_record_id=source_record_id, version=version, ) - service = get_analytics_service() - result = service.try_record_server_event(**kwargs) - if result is not None and not getattr(service, "project_snapshots", False): - _recalculate_snapshot(int(kwargs["user_id"]), str(event_name)) + get_analytics_service().try_record_server_event(**kwargs) except Exception as exc: _safe_warning(exc) diff --git a/dashboard/backend/domain/analytics/lifecycle.py b/dashboard/backend/domain/analytics/lifecycle.py index 59716ac62..01b69b753 100644 --- a/dashboard/backend/domain/analytics/lifecycle.py +++ b/dashboard/backend/domain/analytics/lifecycle.py @@ -397,6 +397,75 @@ def activation_cohort_week(activated_at: datetime) -> date: return activated_day - timedelta(days=activated_day.weekday()) +_OPERATIONAL_STATE_BY_REASON: dict[str, OperationalState] = { + "account_restricted": "blocked", + "billing_lane_unavailable": "blocked", + "provider_disabled": "blocked", + "invalid_default_credential": "needs_attention", + "three_consecutive_failed_runs": "needs_attention", + "run_deadline_exceeded": "needs_attention", + "no_supported_issue": "healthy", +} +_UNKNOWN_OPERATIONAL_REASON = "An operational issue was recorded." + + +def resolve_group_badge( + *, + role: str, + user_group: str, + tier: CommercialTier, +) -> str: + """The single group label admin views show for one user. + + Three orthogonal axes, one badge, fixed precedence (design §6.2, D11): an + admin reads as an admin, because that is the fact that changes what they + can do; a named ``user_group`` beats tier because a human assigned it; + ``unknown`` is the column's default, not a label, so it falls through to + the tier. Do not recompute the precedence in JavaScript: the server owns + that rule. + """ + + if role == "admin": + return "admin" + if user_group and user_group != "unknown": + return user_group + return "free" if tier == "unpaid" else "paid" + + +def operational_reason(reason_code: str) -> str: + """Display text for a stored operational reason code, never the code itself.""" + + return _OPERATIONAL_REASONS.get(reason_code, _UNKNOWN_OPERATIONAL_REASON) + + +def operational_result_from_code( + reason_code: str | None, + *, + as_of: datetime, +) -> OperationalResult: + """Rebuild a display-safe result from yesterday's stored reason code. + + ``user_daily_facts`` stores the code, not the evidence: the one dynamic + evidence line (the consecutive-failure count) is not recoverable from a + code, so it falls back to the reason text rather than inventing a number. + History-copy rows carry state ``healthy`` with a NULL code, so ``None`` + rebuilds as the healthy result rather than as an unknown issue. + """ + + calculation_time = _require_utc(as_of, "as_of") + code = "no_supported_issue" if reason_code is None else reason_code + state = _OPERATIONAL_STATE_BY_REASON.get(code, "needs_attention") + reason = operational_reason(code) + evidence = _OPERATIONAL_EVIDENCE.get(code, reason) + return OperationalResult( + state=state, + reason_code=code, + reason=reason, + evidence=(evidence,), + calculated_at=calculation_time, + ) + + __all__ = [ "CommercialTier", "LifecycleInputs", @@ -411,4 +480,7 @@ def activation_cohort_week(activated_at: datetime) -> date: "commercial_tier", "consecutive_failed_terminal_runs", "is_lifecycle_activity", + "operational_reason", + "operational_result_from_code", + "resolve_group_badge", ] diff --git a/dashboard/backend/domain/analytics/lifecycle_backfill.py b/dashboard/backend/domain/analytics/lifecycle_backfill.py deleted file mode 100644 index 9164bec17..000000000 --- a/dashboard/backend/domain/analytics/lifecycle_backfill.py +++ /dev/null @@ -1,460 +0,0 @@ -"""Bounded, resumable lifecycle history reconstruction.""" - -from __future__ import annotations - -import base64 -import json -from datetime import date, datetime, time, timedelta, timezone -from typing import Literal, Protocol, Sequence - -from pydantic import BaseModel, ConfigDict, Field, field_validator - -from .lifecycle import LifecycleInputs, calculate_lifecycle, is_lifecycle_activity -from .repository import analytics_store -from .repository_common import positive_limit, positive_user_id -from .rollups import AnalyticsRollupStore -from .value_repository import ( - ProjectionJob, - UserLifecycleDailySnapshot, - build_value_analytics_store, -) - - -UTC = timezone.utc -BACKFILL_JOB_NAME = "lifecycle_previous_8_weeks" -MAX_BACKFILL_DAYS = 56 - - -def _utc(value: datetime, field_name: str) -> datetime: - if value.tzinfo is None or value.utcoffset() is None: - raise ValueError(f"{field_name} must include a timezone") - return value.astimezone(UTC) - - -def _timestamp(value: object) -> datetime: - parsed = datetime.fromisoformat(str(value)) - if parsed.tzinfo is None or parsed.utcoffset() is None: - parsed = parsed.replace(tzinfo=UTC) - return parsed.astimezone(UTC) - - -class LifecycleBackfillUser(BaseModel): - model_config = ConfigDict(extra="forbid", frozen=True) - - user_id: int = Field(gt=0) - created_at: datetime - - @field_validator("created_at") - @classmethod - def require_created_timezone(cls, value: datetime) -> datetime: - return _utc(value, "created_at") - - -class LifecycleBackfillEvidence(BaseModel): - model_config = ConfigDict(extra="forbid", frozen=True) - - event_name: str = Field(min_length=1, max_length=64) - occurred_at: datetime - - @field_validator("occurred_at") - @classmethod - def require_occurred_timezone(cls, value: datetime) -> datetime: - return _utc(value, "occurred_at") - - -class LifecycleBackfillCursor(BaseModel): - model_config = ConfigDict(extra="forbid", frozen=True) - - last_user_id: int = Field(gt=0) - window_start: date - window_end: date - - def encode(self) -> str: - payload = json.dumps( - { - "last_user_id": self.last_user_id, - "window_start": self.window_start.isoformat(), - "window_end": self.window_end.isoformat(), - }, - sort_keys=True, - separators=(",", ":"), - ).encode("ascii") - return base64.urlsafe_b64encode(payload).decode("ascii").rstrip("=") - - @classmethod - def decode(cls, value: str) -> "LifecycleBackfillCursor": - if not isinstance(value, str) or not value or value != value.strip(): - raise ValueError("cursor must be a non-empty encoded string") - try: - padded = value + "=" * (-len(value) % 4) - payload = base64.b64decode( - padded.encode("ascii"), - altchars=b"-_", - validate=True, - ) - data = json.loads(payload.decode("ascii")) - return cls.model_validate(data) - except Exception as exc: - raise ValueError("cursor is invalid") from exc - - -class LifecycleBackfillReport(BaseModel): - model_config = ConfigDict(extra="forbid", frozen=True) - - processed_users: int = Field(ge=0) - written_rows: int = Field(ge=0) - partial_days: int = Field(ge=0) - next_cursor: str | None = None - complete: bool - - -class LifecycleBackfillSourceContract(Protocol): - def list_users( - self, - *, - after_user_id: int, - limit: int, - ) -> Sequence[LifecycleBackfillUser]: ... - - def list_evidence( - self, - user_id: int, - *, - start: datetime, - end: datetime, - ) -> Sequence[LifecycleBackfillEvidence]: ... - - def quality_for( - self, - snapshot_date: date, - *, - required_from: date, - ) -> Literal["complete", "partial"]: ... - - -class LifecycleBackfillSource: - """Read display-safe lifecycle evidence from authoritative local stores.""" - - def __init__( - self, - *, - analytics_base=None, - value_store=None, - complete_from: date | None = None, - ) -> None: - self.analytics_base = analytics_base or analytics_store - self.value_store = value_store or build_value_analytics_store(self.analytics_base) - self.rollups = AnalyticsRollupStore(self.analytics_base) - self.complete_from = complete_from - self.is_postgres = hasattr(self.analytics_base, "database_url") - - def list_users( - self, - *, - after_user_id: int, - limit: int, - ) -> tuple[LifecycleBackfillUser, ...]: - cursor = max(0, int(after_user_id)) - page_size = positive_limit(limit, maximum=501) - placeholder = "%s" if self.is_postgres else "?" - sql = f""" - SELECT users.id, users.created_at - FROM users - LEFT JOIN analytics_subject_settings AS settings - ON settings.user_id = users.id - WHERE users.id > {placeholder} - AND users.role <> 'admin' - AND COALESCE(settings.excluded, {"FALSE" if self.is_postgres else "0"}) = {"FALSE" if self.is_postgres else "0"} - ORDER BY users.id - LIMIT {placeholder} - """ - with self.analytics_base._get_connection() as conn: - if self.is_postgres: - with conn.cursor() as db_cursor: - db_cursor.execute(sql, (cursor, page_size)) - rows = db_cursor.fetchall() - else: - rows = conn.execute(sql, (cursor, page_size)).fetchall() - return tuple( - LifecycleBackfillUser( - user_id=int(row["id"]), - created_at=_timestamp(row["created_at"]), - ) - for row in rows - ) - - def list_evidence( - self, - user_id: int, - *, - start: datetime, - end: datetime, - ) -> tuple[LifecycleBackfillEvidence, ...]: - subject_id = positive_user_id(user_id) - window_start = _utc(start, "start") - window_end = _utc(end, "end") - if window_end <= window_start: - raise ValueError("end must be later than start") - events = self.rollups.list_events( - start=window_start, - end=window_end, - include_internal=True, - user_id=subject_id, - ) - evidence = [ - LifecycleBackfillEvidence( - event_name=event.event_name, - occurred_at=event.occurred_at, - ) - for event in events - if event.event_name == "backtest_completed" or is_lifecycle_activity(event) - ] - credit_activity = self.value_store.list_credit_activity( - [subject_id], - start=window_start, - end=window_end, - ).get(subject_id, ()) - evidence.extend( - LifecycleBackfillEvidence( - event_name="credits_settled", - occurred_at=timestamp, - ) - for timestamp in credit_activity - ) - return tuple(sorted(evidence, key=lambda item: item.occurred_at)) - - def quality_for( - self, - snapshot_date: date, - *, - required_from: date, - ) -> Literal["complete", "partial"]: - del required_from - if self.complete_from is None or snapshot_date < self.complete_from: - return "partial" - return "complete" - - -def _window_days(start: date, end: date) -> tuple[date, ...]: - if not isinstance(start, date) or isinstance(start, datetime): - raise ValueError("start must be a date") - if not isinstance(end, date) or isinstance(end, datetime): - raise ValueError("end must be a date") - if end <= start: - raise ValueError("end must be later than start") - length = (end - start).days - if length > MAX_BACKFILL_DAYS: - raise ValueError("backfill window must contain at most 56 days") - return tuple(start + timedelta(days=offset) for offset in range(length)) - - -def _inputs_at( - user: LifecycleBackfillUser, - evidence: Sequence[LifecycleBackfillEvidence], - *, - cutoff: datetime, -) -> LifecycleInputs: - available = [ - item for item in evidence if user.created_at <= item.occurred_at < cutoff - ] - successes = [ - item.occurred_at - for item in available - if item.event_name == "backtest_completed" - ] - meaningful = [item.occurred_at for item in available if is_lifecycle_activity(item)] - window_start = cutoff - timedelta(days=30) - return LifecycleInputs( - user_id=user.user_id, - created_at=user.created_at, - first_successful_backtest_at=min(successes) if successes else None, - last_meaningful_activity_at=max(meaningful) if meaningful else None, - active_days_30d=len( - { - timestamp.date() - for timestamp in meaningful - if window_start <= timestamp < cutoff - } - ), - successful_backtests_30d=sum( - 1 for timestamp in successes if window_start <= timestamp < cutoff - ), - ) - - -def backfill_lifecycle_history( - *, - start: date, - end: date, - batch_size: int = 100, - cursor: str | None = None, - now: datetime | None = None, - source: LifecycleBackfillSourceContract, - store, -) -> LifecycleBackfillReport: - days = _window_days(start, end) - current = _utc(now or datetime.now(UTC), "now") - if end > current.date() + timedelta(days=1): - raise ValueError("backfill window cannot extend beyond the current UTC day") - page_size = positive_limit(batch_size, maximum=500) - after_user_id = 0 - if cursor is not None: - decoded = LifecycleBackfillCursor.decode(cursor) - if decoded.window_start != start or decoded.window_end != end: - raise ValueError("cursor window does not match the requested window") - after_user_id = decoded.last_user_id - - page = tuple(source.list_users(after_user_id=after_user_id, limit=page_size + 1)) - page_ids = [user.user_id for user in page] - if ( - len(page) > page_size + 1 - or page_ids != sorted(set(page_ids)) - or any(user_id <= after_user_id for user_id in page_ids) - ): - raise ValueError("backfill source returned an invalid user page") - users = page[:page_size] - partial_dates: set[date] = set() - written_rows = 0 - evidence_start = datetime.combine( - start - timedelta(days=30), - time.min, - tzinfo=UTC, - ) - evidence_end = min( - datetime.combine(end, time.min, tzinfo=UTC), - current + timedelta(microseconds=1), - ) - existing = { - (snapshot.snapshot_date, snapshot.user_id): snapshot - for snapshot in store.list_daily_snapshots( - start=start, - end=end, - user_ids=[user.user_id for user in users], - ) - } - for user in users: - evidence = source.list_evidence( - user.user_id, - start=evidence_start, - end=evidence_end, - ) - for snapshot_date in days: - cutoff = min( - datetime.combine( - snapshot_date + timedelta(days=1), - time.min, - tzinfo=UTC, - ), - current, - ) - if user.created_at >= cutoff: - continue - result = calculate_lifecycle( - _inputs_at(user, evidence, cutoff=cutoff), - cutoff, - ) - quality = source.quality_for( - snapshot_date, - required_from=snapshot_date - timedelta(days=29), - ) - prior = existing.get((snapshot_date, user.user_id)) - if ( - prior is not None - and prior.data_quality == "complete" - and quality == "partial" - ): - continue - if quality == "partial": - partial_dates.add(snapshot_date) - store.upsert_daily_snapshot( - UserLifecycleDailySnapshot( - snapshot_date=snapshot_date, - user_id=user.user_id, - lifecycle_segment=result.segment, - lifecycle_reason_code=result.reason_code, - data_quality=quality, - calculated_at=current, - ) - ) - written_rows += 1 - - complete = len(page) <= page_size - next_cursor = None - if not complete and users: - next_cursor = LifecycleBackfillCursor( - last_user_id=users[-1].user_id, - window_start=start, - window_end=end, - ).encode() - return LifecycleBackfillReport( - processed_users=len(users), - written_rows=written_rows, - partial_days=len(partial_dates), - next_cursor=next_cursor, - complete=complete, - ) - - -def run_lifecycle_backfill_batch( - *, - now: datetime | None = None, - batch_size: int = 100, - source: LifecycleBackfillSourceContract | None = None, - store=None, -) -> LifecycleBackfillReport: - current = _utc(now or datetime.now(UTC), "now") - values = store or build_value_analytics_store() - history_source = source or LifecycleBackfillSource( - analytics_base=values.analytics_base, - value_store=values, - ) - job = values.get_projection_job(BACKFILL_JOB_NAME) - if job is None: - end = current.date() + timedelta(days=1) - job = ProjectionJob( - job_name=BACKFILL_JOB_NAME, - window_start=end - timedelta(days=MAX_BACKFILL_DAYS), - window_end=end, - status="pending", - updated_at=current, - ) - if job.status == "complete": - return LifecycleBackfillReport( - processed_users=0, - written_rows=0, - partial_days=0, - complete=True, - ) - report = backfill_lifecycle_history( - start=job.window_start, - end=job.window_end, - batch_size=batch_size, - cursor=job.cursor, - now=current, - source=history_source, - store=values, - ) - values.save_projection_job( - ProjectionJob( - job_name=job.job_name, - window_start=job.window_start, - window_end=job.window_end, - cursor=report.next_cursor, - status="complete" if report.complete else "running", - updated_at=current, - ) - ) - return report - - -__all__ = [ - "BACKFILL_JOB_NAME", - "LifecycleBackfillCursor", - "LifecycleBackfillEvidence", - "LifecycleBackfillReport", - "LifecycleBackfillSource", - "LifecycleBackfillSourceContract", - "LifecycleBackfillUser", - "backfill_lifecycle_history", - "run_lifecycle_backfill_batch", -] diff --git a/dashboard/backend/domain/analytics/lifecycle_reads.py b/dashboard/backend/domain/analytics/lifecycle_reads.py index 74fd7d452..2d9bd1572 100644 --- a/dashboard/backend/domain/analytics/lifecycle_reads.py +++ b/dashboard/backend/domain/analytics/lifecycle_reads.py @@ -5,8 +5,11 @@ ``user_activity`` and ``user_daily_facts`` rather than a 180-day scan of ``analytics_events`` per user (design SS6.6). -Nothing in ``api/`` calls this module yet. PR B wires it into the one-user -profile route; the daily job (``daily_facts.py``) is its only caller in PR A. +Two kinds of caller: the daily job (``daily_facts.py``), which passes the end +of the day it is computing as ``as_of``, and the live read paths +(``query_service.py`` for one user's axes, ``value_queries.py`` for the profile +and users list), which pass ``now``. The clamp below is what lets one function +serve both. """ from __future__ import annotations diff --git a/dashboard/backend/domain/analytics/maintenance.py b/dashboard/backend/domain/analytics/maintenance.py deleted file mode 100644 index ce877f678..000000000 --- a/dashboard/backend/domain/analytics/maintenance.py +++ /dev/null @@ -1,100 +0,0 @@ -"""Throttled repair of the legacy Analytics snapshot rows. - -Admin layer redesign PR A moved the day rollup and the lifecycle history backfill off -this reaper tick: the daily-facts job (``daily_facts.py``, on its own worker -thread, ``daily_job.py``) owns the day rollup now, so the two can never write -the same rows (design SS6.9 step 1, D23), and the eight-week history copy in -``facts_migration.py`` replaces the history copy. What remains is the two -24-hour-throttled snapshot repairs PR 0 left, which keep -``user_analytics_snapshots`` maintained for the read paths that still use it. -PR B moves those read paths and deletes this module. -""" - -from __future__ import annotations - -from datetime import datetime, timezone -from typing import Any - -from pydantic import BaseModel, ConfigDict, Field - - -class AnalyticsMaintenanceReport(BaseModel): - model_config = ConfigDict(extra="forbid", frozen=True) - - repaired_snapshots: int = Field(default=0, ge=0) - repaired_value_snapshots: int = Field(default=0, ge=0) - failures: int = Field(default=0, ge=0) - - -def _current_utc(value: datetime | None) -> datetime: - current = value or datetime.now(timezone.utc) - if current.tzinfo is None or current.utcoffset() is None: - raise ValueError("now must include a timezone") - return current.astimezone(timezone.utc) - - -def run_analytics_maintenance( - *, - now: datetime | None = None, - snapshot_limit: int = 100, - repair_snapshots: Any | None = None, - repair_value_snapshots: Any | None = None, -) -> AnalyticsMaintenanceReport: - """Repair bounded batches of stale dual-axis and legacy snapshots.""" - - current = _current_utc(now) - if isinstance(snapshot_limit, bool) or not isinstance(snapshot_limit, int): - raise ValueError("snapshot_limit must be an integer") - page_size = max(1, min(snapshot_limit, 100)) - failures = 0 - - if repair_snapshots is None: - from .states import repair_stale_snapshots - - repair_snapshots = repair_stale_snapshots - if repair_value_snapshots is None: - from .states import repair_stale_value_snapshots - - repair_value_snapshots = repair_stale_value_snapshots - - repaired_values = 0 - try: - repaired_values = int( - repair_value_snapshots( - now=current, - limit=page_size, - ) - ) - except Exception as exc: - failures += 1 - print( - "WARNING: analytics.value_snapshot_maintenance_failed " - f"category={type(exc).__name__[:80]}" - ) - - repaired = 0 - try: - repaired = int( - repair_snapshots( - now=current, - limit=page_size, - ) - ) - except Exception as exc: - failures += 1 - print( - "WARNING: analytics.snapshot_maintenance_failed " - f"category={type(exc).__name__[:80]}" - ) - - return AnalyticsMaintenanceReport( - repaired_snapshots=max(0, repaired), - repaired_value_snapshots=max(0, repaired_values), - failures=failures, - ) - - -__all__ = [ - "AnalyticsMaintenanceReport", - "run_analytics_maintenance", -] diff --git a/dashboard/backend/domain/analytics/metrics.py b/dashboard/backend/domain/analytics/metrics.py index 7ad4e39d6..971c2c58e 100644 --- a/dashboard/backend/domain/analytics/metrics.py +++ b/dashboard/backend/domain/analytics/metrics.py @@ -15,6 +15,15 @@ _MEANINGFUL_EXPERIENCE_EVENTS = {"page_viewed"} _TERMINAL_SUCCESS = "backtest_completed" _TERMINAL_FAILURE = "backtest_failed" +# The overview's activation funnel, in funnel order. Shared by the rollup that +# writes one ``activation_users`` row per name per day and the query service +# that reads them back. +ACTIVATION_EVENTS: tuple[str, ...] = ( + "account_signed_up", + "credential_verified", + "agent_created", + "backtest_completed", +) def _utc(value: datetime, field_name: str) -> datetime: @@ -255,6 +264,7 @@ def calculate_overview_metrics( __all__ = [ + "ACTIVATION_EVENTS", "AnalyticsMetricFilters", "AnalyticsOverviewMetrics", "backtest_success_rate", diff --git a/dashboard/backend/domain/analytics/query_service.py b/dashboard/backend/domain/analytics/query_service.py index aa414da1d..3d8e05c2f 100644 --- a/dashboard/backend/domain/analytics/query_service.py +++ b/dashboard/backend/domain/analytics/query_service.py @@ -5,14 +5,25 @@ import json from collections import Counter, defaultdict from dataclasses import dataclass -from datetime import datetime, timedelta, timezone -from typing import Any, Literal +from datetime import date, datetime, timedelta, timezone +from typing import Any, Literal, Sequence from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator +from .lifecycle import ( + CommercialTier, + LifecycleResult, + OperationalResult, + OperationalSignals, + calculate_lifecycle, + calculate_operational_state, + operational_result_from_code, + resolve_group_badge, +) +from .lifecycle_reads import build_lifecycle_inputs from .metrics import ( + ACTIVATION_EVENTS, AnalyticsMetricFilters, - calculate_overview_metrics, is_meaningful_event, ) from .models import AnalyticsEventRecord @@ -27,14 +38,9 @@ ) from .rollups import AnalyticsRollupStore, DailyRollup from .usage_cost import atl_cost_micro, byok_estimate_micro, byok_rollup_lane -from .states import ( - AnalyticsStateStore, - UserAnalyticsSnapshot, - calculate_user_state, -) +from .value_repository import build_value_analytics_store ActivitySection = Literal["timeline", "runs", "usage", "sessions"] -_USER_STATES = {"blocked", "needs_attention", "dormant", "onboarding", "active"} _ATTENTION_STATES = {"blocked", "needs_attention"} # The two BillingLaneDay call-count fields; also models.ALLOWED_BILLING_MODES. _BILLING_LANES = ("platform_credits", "byok") @@ -46,12 +52,7 @@ "byok_estimated_micro", "byok_unpriced", ) -_ACTIVATION_EVENTS = ( - "account_signed_up", - "credential_verified", - "agent_created", - "backtest_completed", -) +_ACTIVATION_EVENTS = ACTIVATION_EVENTS _USAGE_EVENTS = { "model_usage_recorded", "credits_reserved", @@ -103,7 +104,7 @@ class FailureCategoryCount(BaseModel): model_config = ConfigDict(extra="forbid", frozen=True) error_category: str - affected_users: int = Field(ge=0) + event_count: int = Field(ge=0) class AnalyticsStateSummary(BaseModel): @@ -249,7 +250,6 @@ class AnalyticsOverview(BaseModel): daily_completed_runs: dict[str, int] billing_lane_mix: list[BillingLaneDay] activation_funnel: dict[str, int] - user_state_counts: dict[str, int] top_failure_categories: list[FailureCategoryCount] users_needing_attention: list[AnalyticsUserListItem] last_updated: datetime @@ -257,15 +257,50 @@ class AnalyticsOverview(BaseModel): availability: dict[str, PanelAvailability] -def _all_users(user_store: Any) -> list[dict[str, Any]]: - users: list[dict[str, Any]] = [] - offset = 0 - while True: - page = user_store.list_users_admin(limit=500, offset=offset) - users.extend(page) - if len(page) < 500: - return users - offset += len(page) +class RunOutcomeCounts(BaseModel): + model_config = ConfigDict(extra="forbid", frozen=True) + + requested: int = Field(default=0, ge=0) + completed: int = Field(default=0, ge=0) + failed: int = Field(default=0, ge=0) + cancelled: int = Field(default=0, ge=0) + + +class UserMetricsShared(BaseModel): + """One user's numbers as the user may see them (design §6.13). + + ``get_user_metrics`` builds this for ``audience="user"`` and + ``UserMetricsAdmin`` for ``audience="admin"``. Two models rather than one + with nullable admin fields: the admin-only fields are *absent* from the + user projection, not nulled -- a null still tells the subject the field + exists -- and absence must hold on every serialisation path, including + ``model_dump_json`` and FastAPI's, which a ``model_dump`` override would + not reach. A future ``GET /api/me/usage`` is this call with + ``audience="user"``. + """ + + model_config = ConfigDict(extra="forbid", frozen=True) + + user_id: int = Field(gt=0) + facts_as_of: date + runs_7d: RunOutcomeCounts + runs_30d: RunOutcomeCounts + own_spend_micro: int = Field(default=0, ge=0) + operator_cost_micro: int = Field(default=0, ge=0) + tier: CommercialTier = "unpaid" + credit_balance_micro: int = Field(default=0, ge=0) + data_quality: Literal["complete", "partial"] = "complete" + + +class UserMetricsAdmin(UserMetricsShared): + """The admin projection: the shared numbers plus the admin-only axes.""" + + active_days_30d: int = Field(ge=0, le=30) + lifecycle: LifecycleResult + operational: OperationalResult + user_group: str + group_badge: str + access_log: Sequence[str] class AnalyticsQueryStore: @@ -274,35 +309,8 @@ class AnalyticsQueryStore: def __init__(self, base_store: Any = analytics_store): self.base_store = base_store self.rollups = AnalyticsRollupStore(base_store) - self.states = AnalyticsStateStore(base_store) self.is_postgres = hasattr(base_store, "database_url") - def list_snapshots(self) -> dict[int, UserAnalyticsSnapshot]: - if self.is_postgres: - with self.base_store._get_connection() as conn: - with conn.cursor() as cur: - cur.execute("SELECT * FROM user_analytics_snapshots ORDER BY user_id") - rows = cur.fetchall() - else: - with self.base_store._get_connection() as conn: - rows = conn.execute( - "SELECT * FROM user_analytics_snapshots ORDER BY user_id" - ).fetchall() - result = {} - for row in rows: - snapshot = UserAnalyticsSnapshot( - user_id=int(row["user_id"]), - status=str(row["status"]), - reason_code=str(row["reason_code"]), - human_readable_reason=str(row["human_readable_reason"]), - evidence_event_ids=list( - json.loads(str(row["evidence_event_ids_json"])) - ), - calculated_at=_parse_timestamp(row["calculated_at"]), - ) - result[snapshot.user_id] = snapshot - return result - def list_metric_events( self, *, @@ -399,61 +407,6 @@ def list_metric_events( ) return events - def summarize_users( - self, - *, - user_ids: set[int], - start: datetime, - end: datetime, - ) -> dict[int, dict[str, Any]]: - """Return bounded 30-day summaries without hydrating event models.""" - - if not user_ids: - return {} - result: dict[int, dict[str, Any]] = {} - ordered_ids = sorted(positive_user_id(user_id) for user_id in user_ids) - for offset in range(0, len(ordered_ids), 500): - chunk = ordered_ids[offset : offset + 500] - placeholder = "%s" if self.is_postgres else "?" - id_placeholders = ", ".join(placeholder for _ in chunk) - params: list[Any] = [utc_iso(start), utc_iso(end), *chunk] - sql = f""" - SELECT user_id, - MAX(CASE - WHEN event_group <> 'experience' - OR event_name = 'page_viewed' - THEN occurred_at - END) AS last_meaningful_activity, - SUM(CASE WHEN event_group = 'run' THEN 1 ELSE 0 END) - AS recent_runs, - SUM(CASE WHEN event_name = 'backtest_failed' THEN 1 ELSE 0 END) - AS recent_failures - FROM analytics_events - WHERE occurred_at >= {placeholder} - AND occurred_at < {placeholder} - AND user_id IN ({id_placeholders}) - GROUP BY user_id - """ - if self.is_postgres: - with self.base_store._get_connection() as conn: - with conn.cursor() as cur: - cur.execute(sql, params) - rows = cur.fetchall() - else: - with self.base_store._get_connection() as conn: - rows = conn.execute(sql, params).fetchall() - for row in rows: - result[int(row["user_id"])] = { - "last_meaningful_activity": ( - _parse_timestamp(row["last_meaningful_activity"]) - if row["last_meaningful_activity"] is not None - else None - ), - "recent_runs": int(row["recent_runs"] or 0), - "recent_failures": int(row["recent_failures"] or 0), - } - return result - def list_activity_rows( self, *, @@ -516,39 +469,95 @@ def list_activity_rows( (int(row["sequence"]), _row_to_event(row)) for row in page_rows ], next_cursor + def first_event_times( + self, + *, + user_id: int, + event_names: Sequence[str], + start: datetime, + end: datetime, + ) -> dict[str, datetime]: + """Each named event's first ``occurred_at`` for one user in ``[start, end)``. + + One grouped, indexed read returning at most ``len(event_names)`` + rows: the profile's activation milestones are first occurrences, + which the newest-100 timeline page cannot answer for a user whose + first verification is older than their last hundred events. + """ + subject_id = positive_user_id(user_id) + names = sorted(set(event_names)) + if not names: + return {} + placeholder = "%s" if self.is_postgres else "?" + sql = f""" + SELECT event_name, MIN(occurred_at) AS first_at + FROM analytics_events + WHERE user_id = {placeholder} + AND event_name IN ({", ".join(placeholder for _ in names)}) + AND occurred_at >= {placeholder} AND occurred_at < {placeholder} + GROUP BY event_name + """ + params = [ + subject_id, + *names, + utc_iso(_utc(start, "start")), + utc_iso(_utc(end, "end")), + ] + if self.is_postgres: + with self.base_store._get_connection() as conn: + with conn.cursor() as cur: + cur.execute(sql, params) + rows = cur.fetchall() + else: + with self.base_store._get_connection() as conn: + rows = conn.execute(sql, params).fetchall() + return { + str(row["event_name"]): _parse_timestamp(row["first_at"]) + for row in rows + if row["first_at"] is not None + } + def list_session_rows( self, *, user_id: int, limit: int, cursor: str | None, + start: datetime, ) -> tuple[list[dict[str, Any]], str | None]: + """Sessions with any experience event at or after ``start``, whole. + + The window selects *sessions*, not events: a session that began + before ``start`` and continued after it is read with all of its + events, so its count and visible time are not cut at the boundary. + Sessions wholly before ``start`` are out of scope (design: the + activity route's 30-day sessions window). + """ subject_id = positive_user_id(user_id) page_size = positive_limit(limit) + window_start = utc_iso(_utc(start, "start")) + placeholder = "%s" if self.is_postgres else "?" + sql = f""" + SELECT * FROM analytics_events + WHERE user_id = {placeholder} + AND event_group = 'experience' + AND session_id IN ( + SELECT session_id FROM analytics_events + WHERE user_id = {placeholder} AND occurred_at >= {placeholder} + AND event_group = 'experience' + AND session_id IS NOT NULL + ) + ORDER BY occurred_at DESC, sequence DESC + """ + params = (subject_id, subject_id, window_start) if self.is_postgres: with self.base_store._get_connection() as conn: with conn.cursor() as cur: - cur.execute( - """ - SELECT * FROM analytics_events - WHERE user_id = %s AND event_group = 'experience' - AND session_id IS NOT NULL - ORDER BY occurred_at DESC, sequence DESC - """, - (subject_id,), - ) + cur.execute(sql, params) rows = cur.fetchall() else: with self.base_store._get_connection() as conn: - rows = conn.execute( - """ - SELECT * FROM analytics_events - WHERE user_id = ? AND event_group = 'experience' - AND session_id IS NOT NULL - ORDER BY occurred_at DESC, sequence DESC - """, - (subject_id,), - ).fetchall() + rows = conn.execute(sql, params).fetchall() grouped: dict[str, dict[str, Any]] = {} for row in rows: event = _row_to_event(row) @@ -596,18 +605,91 @@ def list_session_rows( return page_rows, next_cursor -def _snapshot_summary(snapshot: UserAnalyticsSnapshot) -> AnalyticsStateSummary: - return AnalyticsStateSummary( - status=snapshot.status, - reason_code=snapshot.reason_code, - human_readable_reason=snapshot.human_readable_reason, - evidence_event_ids=snapshot.evidence_event_ids, - calculated_at=snapshot.calculated_at, +def _availability(available: bool = True) -> PanelAvailability: + return PanelAvailability(available=available) + + +def _warn_read_failed(section: str, exc: BaseException) -> None: + """One log line for a read the page degrades to unavailable/partial. + + The page already says "incomplete"; this is what lets an operator tell a + Postgres-only SQL bug from an outage in the Render log. Class name only: + an exception message can carry row data. + """ + print( + f"WARNING: analytics.read.{section}_failed " + f"category={type(exc).__name__[:80]}", + flush=True, ) -def _availability(available: bool = True) -> PanelAvailability: - return PanelAvailability(available=available) +def _legacy_state(operational_state: str, lifecycle_segment: str) -> str: + """Map the live operational state and lifecycle segment onto the legacy + five-state label (blocked, needs_attention, dormant, onboarding, active) + that the user profile's ``state`` summary still reports. Operational + problems win; otherwise the lifecycle segment decides.""" + + if operational_state in {"blocked", "needs_attention"}: + return operational_state + if lifecycle_segment == "dormant": + return "dormant" + if lifecycle_segment in {"new", "onboarding"}: + return "onboarding" + return "active" + + +def _is_total_row(row: DailyRollup) -> bool: + """A rollup's undimensioned day total, not one of its per-dimension rows.""" + + return not ( + row.event_name + or row.billing_mode + or row.provider_id + or row.model_id + or row.outcome + or row.error_category + or row.user_state + ) + + +def _latest_rollup_value(rows: Sequence[DailyRollup], metric_name: str) -> int | None: + """The metric's total on the latest rolled-up day: "as of yesterday".""" + + latest: DailyRollup | None = None + for row in rows: + if row.metric_name == metric_name and _is_total_row(row): + if latest is None or row.rollup_date > latest.rollup_date: + latest = row + return None if latest is None else latest.value_count + + +def _sum_rollup(rows: Sequence[DailyRollup], metric_name: str) -> int: + return sum( + row.value_count + for row in rows + if row.metric_name == metric_name and _is_total_row(row) + ) + + +def _rollup_token_total( + rows: Sequence[DailyRollup], + metric_name: str, + filters: AnalyticsMetricFilters, + *, + dimensional: bool, +) -> int: + """A token metric over completed days: the day totals, or -- when a + billing/provider/model filter is set -- the per-usage-dimension rows.""" + + if not dimensional: + return _sum_rollup(rows, metric_name) + return sum( + row.value_count + for row in rows + if row.metric_name == metric_name + and (row.billing_mode or row.provider_id or row.model_id) + and _matches_rollup_dimensions(row, filters) + ) def _matches_rollup_dimensions( @@ -667,12 +749,55 @@ def _platform_cost_micro( ) +def _run_outcome_counts(totals: Any) -> RunOutcomeCounts: + if totals is None: + return RunOutcomeCounts() + return RunOutcomeCounts( + requested=totals.runs_requested, + completed=totals.runs_completed, + failed=totals.runs_failed, + cancelled=totals.runs_cancelled, + ) + + +def landed_fact_day( + value_store: Any, + *, + yesterday: date, + warned: set[tuple[date, date | None]], +) -> date | None: + """The fact day a read "as of yesterday" should use, or None if none landed. + + Facts land once a night. Until they do -- or all day, when the job's + facts step fails -- yesterday has no rows, and a read of it returns empty + counts indistinguishable from "nobody is blocked". Every such read + joins the newest landed day instead and labels it (``stale`` plus the + coverage dates; ``/users`` stamps ``as_of``), or, where its response has + no field to label it with, refuses a day older than the normal gap. + + The WARNING is printed once per (expected, latest) pair for the calling + service, not once per request: a failed job would otherwise write it on + every page load, filter change and pagination click all day. + """ + latest = value_store.latest_fact_date(on_or_before=yesterday) + if latest != yesterday and (yesterday, latest) not in warned: + warned.add((yesterday, latest)) + print( + "WARNING: analytics.read.facts_stale " + f"latest={latest.isoformat() if latest else 'none'} " + f"expected={yesterday.isoformat()}", + flush=True, + ) + return latest + + class AnalyticsQueryService: def __init__( self, *, store: Any = analytics_store, user_store: Any | None = None, + value_store: Any | None = None, ): if user_store is None: from dashboard.backend.users import user_store as default_user_store @@ -681,6 +806,25 @@ def __init__( self.store = store self.user_store = user_store self.query_store = AnalyticsQueryStore(store) + self._value_store = value_store + self._stale_fact_warnings: set[tuple[date, date | None]] = set() + + @property + def value_store(self) -> Any: + """The fact-table store, built on first use. + + Not in ``__init__``: the module-level ``analytics_query_service`` is + constructed at import, and building the default twin resolves the + credits / provider / agent / run singletons -- on the web boot path + and in every backtest child that imports this module. + """ + if self._value_store is None: + self._value_store = build_value_analytics_store(analytics_base=self.store) + return self._value_store + + @value_store.setter + def value_store(self, value: Any) -> None: + self._value_store = value def get_overview( self, @@ -713,58 +857,135 @@ def get_overview( lane_days: dict[str, dict[str, int]] = {} funnel: dict[str, int] = {} failures: list[FailureCategoryCount] = [] - raw_events: list[_MetricEvent] = [] - # An empty window has nothing to read, which is not a failed read. - raw_events_read = effective_end <= filters.start - - if effective_end > filters.start: + dimensional = any( + value is not None + for value in ( + filters.billing_mode, + filters.provider_id, + filters.model_id, + ) + ) + # The one raw-event read: the current UTC day, only the part of it the + # window covers (design §6.15 item 2). Completed days come from the + # rollups, which already hold the trailing context the old + # window-wide scan existed to supply. + # + # Always without internal accounts: ``rollup_day`` writes the + # completed days internal-free, so admitting them into today's slice + # under ``include_internal`` summed two populations into one figure + # that matched neither setting. The flag is accepted and echoed, and + # widens nothing here (as on /retention, /groups and /lifecycle). + today_start = datetime.combine( + current.date(), datetime.min.time(), tzinfo=timezone.utc + ) + scan_start = max(filters.start, today_start) + today_events: list[_MetricEvent] = [] + # An empty slice has nothing to read, which is not a failed read. + raw_events_read = effective_end <= scan_start + if not raw_events_read: try: - raw_events = self.query_store.list_metric_events( - start=filters.start, + today_events = self.query_store.list_metric_events( + start=scan_start, end=effective_end, - include_internal=filters.include_internal, + include_internal=False, ) raw_events_read = True - raw_metrics = calculate_overview_metrics( - raw_events, - start=filters.start, - end=effective_end, - billing_mode=filters.billing_mode, - provider_id=filters.provider_id, - model_id=filters.model_id, + except Exception as exc: + _warn_read_failed("overview_events", exc) + today_events = [] + current_events = [ + event for event in today_events if _event_matches_filters(event, filters) + ] + rollups: list[DailyRollup] = [] + rollups_read = False + try: + historical_end = min(effective_end.date(), current.date()) + rollups = self.query_store.rollups.list_rollups( + start=filters.start.date(), + end=historical_end, + ) + rollups_read = True + except Exception as exc: + _warn_read_failed("overview_rollups", exc) + rollups = [] + + # Trailing context, as of the latest completed day in the window. The + # rollups hold these as day totals only, so a billing/provider/model + # filter does not narrow them. + if rollups_read: + active_users = _latest_rollup_value(rollups, "rolling_active_users_7d") + mature = _sum_rollup(rollups, "mature_signup_cohort_users") + converted = _sum_rollup(rollups, "first_success_within_7d_users") + conversion = None if mature == 0 else converted / mature + first_success = _latest_rollup_value(rollups, "users_with_first_success") + repeat = _latest_rollup_value(rollups, "users_with_repeat_success_24h_30d") + if first_success: + repeat_rate = (repeat or 0) / first_success + else: + availability["snapshot"] = _availability(False) + + if rollups_read and raw_events_read: + # Completed days sum their distinct users per day, so a user who + # repeats a step on two days counts twice (the daily tier's trade, + # design §6.3). activation_users rows carry no billing/provider/ + # model, so today's half is unfiltered too. + funnel = { + event_name: sum( + row.value_count + for row in rollups + if row.metric_name == "activation_users" + and row.event_name == event_name ) - active_users = raw_metrics.active_users_7d - conversion = raw_metrics.first_success_conversion - repeat_rate = raw_metrics.repeat_run_rate - funnel = { - event_name: len( - { - event.user_id - for event in raw_events - if event.event_name == event_name - and _event_matches_filters(event, filters) - } - ) - for event_name in _ACTIVATION_EVENTS - } - failure_users: dict[str, set[int]] = defaultdict(set) - for event in raw_events: - if event.error_category and _event_matches_filters(event, filters): - failure_users[event.error_category].add(event.user_id) - failures = [ - FailureCategoryCount( - error_category=category, - affected_users=len(users), - ) - for category, users in sorted( - failure_users.items(), - key=lambda item: (-len(item[1]), item[0]), - )[:10] - ] - except Exception: - availability["snapshot"] = _availability(False) + + len( + { + event.user_id + for event in today_events + if event.event_name == event_name + } + ) + for event_name in _ACTIVATION_EVENTS + } + # Design D13: events per category, filtered or not. A distinct-user + # count over the range would need a raw read or a rollup keyed by + # user, and summing per-day distinct users counts user-days, so + # the one figure the rollups give exactly is the event count + # (#647). + failure_counts: dict[str, int] = defaultdict(int) + for row in rollups: + if ( + row.error_category + and row.metric_name == "event_count" + and _matches_rollup_dimensions(row, filters) + ): + failure_counts[row.error_category] += row.value_count + for event in current_events: + if event.error_category: + failure_counts[event.error_category] += 1 + failures = [ + FailureCategoryCount(error_category=category, event_count=count) + for category, count in sorted( + failure_counts.items(), + key=lambda item: (-item[1], item[0]), + )[:10] + ] + # A completed day rolled up before activation_users existed has + # every other rollup row and none of these, and summed as zero. + # rollup_day now writes them zero or not and the startup + # migration fills the old days inside raw retention; a day still + # without them makes the funnel an undercount, which it must not + # publish as a figure. + rolled_up_days = { + row.rollup_date for row in rollups if row.metric_name == "daily_active_users" + } + activation_days = { + row.rollup_date for row in rollups if row.metric_name == "activation_users" + } + if not rolled_up_days <= activation_days: + funnel = {} availability["funnel"] = _availability(False) - availability["friction"] = _availability(False) + else: + availability["funnel"] = _availability(False) + availability["friction"] = _availability(False) try: if not raw_events_read: @@ -773,19 +994,8 @@ def get_overview( # publish a rollup-only figure as the whole range -- the quiet # answer a failed read must never be mistaken for. raise AnalyticsStoreError("raw analytics events are unavailable") - historical_end = min(effective_end.date(), current.date()) - rollups = self.query_store.rollups.list_rollups( - start=filters.start.date(), - end=historical_end, - ) - dimensional = any( - value is not None - for value in ( - filters.billing_mode, - filters.provider_id, - filters.model_id, - ) - ) + if not rollups_read: + raise AnalyticsStoreError("analytics rollups are unavailable") if dimensional: historical_completed = sum( row.value_count @@ -818,17 +1028,6 @@ def get_overview( and not row.provider_id and not row.model_id ) - today_start = datetime.combine( - current.date(), datetime.min.time(), tzinfo=timezone.utc - ) - current_events = [ - event - for event in raw_events - if max(filters.start, today_start) - <= event.occurred_at - < effective_end - and _event_matches_filters(event, filters) - ] current_completed = sum( event.event_name == "backtest_completed" for event in current_events ) @@ -843,17 +1042,16 @@ def get_overview( platform_cost = ( _platform_cost_micro(rollups, current_events, filters) / 1_000_000 ) - input_tokens = sum( - int(event.properties.get("input_tokens", 0)) - for event in raw_events - if event.event_name == "model_usage_recorded" - and _event_matches_filters(event, filters) - ) - output_tokens = sum( - int(event.properties.get("output_tokens", 0)) - for event in raw_events - if event.event_name == "model_usage_recorded" - and _event_matches_filters(event, filters) + input_tokens, output_tokens = ( + _rollup_token_total( + rollups, metric_name, filters, dimensional=dimensional + ) + + sum( + int(event.properties.get(metric_name, 0)) + for event in current_events + if event.event_name == "model_usage_recorded" + ) + for metric_name in ("input_tokens", "output_tokens") ) for row in rollups: key = row.rollup_date.isoformat() @@ -954,80 +1152,69 @@ def get_overview( lane_days = { day: lane for day, lane in lanes.items() if any(lane.values()) } - except Exception: + except Exception as exc: + _warn_read_failed("overview_growth", exc) availability["growth"] = _availability(False) - state_counts: dict[str, int] = {} + # The friction panel (top_failure_categories) is computed from the + # funnel's reads, so it is unavailable exactly when they failed (set + # above) -- not when only the funnel's activation rows are missing. + + # Who needs attention, as of the latest landed fact day (design D20): + # yesterday once the night's job lands, the day before until then. + # The daily job writes facts for non-internal accounts only, so + # include_internal does not widen it. + yesterday = current.date() - timedelta(days=1) attention: list[AnalyticsUserListItem] = [] - snapshots: dict[int, UserAnalyticsSnapshot] = {} - excluded: set[int] = set() - snapshots_available = False try: - snapshots = self.query_store.list_snapshots() - excluded = ( - set() - if filters.include_internal - else self.store.list_excluded_user_ids(include_admin_accounts=True) + facts_day = landed_fact_day( + self.value_store, + yesterday=yesterday, + warned=self._stale_fact_warnings, ) - state_counts = dict( - Counter( - snapshot.status - for user_id, snapshot in snapshots.items() - if user_id not in excluded - ) + if facts_day is None or facts_day < yesterday - timedelta(days=1): + # No fact day has landed, or only one older than the normal + # "tonight's job has not run yet" gap. An empty list would read + # as "nobody needs attention", and this panel has no as-of + # field to label an old list with, so it says unavailable. + raise AnalyticsStoreError("no recent analytics fact day has landed") + candidates = self.value_store.list_attention_candidates( + day=facts_day, + window_start=facts_day - timedelta(days=29), + limit=10, ) - snapshots_available = True - except Exception: - availability["friction"] = _availability(False) - try: - if not snapshots_available: - raise RuntimeError("Analytics snapshots are unavailable") - attention_ids = { - user_id - for user_id, snapshot in snapshots.items() - if user_id not in excluded - and snapshot.status in _ATTENTION_STATES - } - summaries = self.query_store.summarize_users( - user_ids=attention_ids, - start=current - timedelta(days=30), - end=current + timedelta(microseconds=1), + activity = ( + self.value_store.list_activity( + [candidate.user_id for candidate in candidates] + ) + if candidates + else {} ) - identities = { - int(user["id"]): user - for user in _all_users(self.user_store) - if int(user["id"]) in attention_ids - } - candidates = [] - for user_id in attention_ids: - user = identities.get(user_id) - if user is None: - continue - snapshot = snapshots[user_id] - summary = summaries.get(user_id, {}) - candidates.append( + for candidate in candidates: + reason = operational_result_from_code( + candidate.operational_reason_code, as_of=current + ) + clock = activity.get(candidate.user_id) + attention.append( AnalyticsUserListItem( - user_id=user_id, - display_name=str(user.get("display_name") or ""), - email=str(user.get("email") or ""), - joined_at=_parse_timestamp(user["created_at"]), - status=snapshot.status, - reason_code=snapshot.reason_code, - human_readable_reason=snapshot.human_readable_reason, - last_meaningful_activity=summary.get( - "last_meaningful_activity" + user_id=candidate.user_id, + display_name=candidate.display_name, + email=candidate.email, + joined_at=candidate.created_at, + status=candidate.operational_state, + reason_code=reason.reason_code, + human_readable_reason=reason.reason, + last_meaningful_activity=( + clock.last_meaningful_activity_at if clock else None ), - recent_runs=int(summary.get("recent_runs", 0)), - recent_failures=int(summary.get("recent_failures", 0)), - profile_path=f"/admin/analytics/users/{user_id}", + recent_runs=candidate.recent_runs, + recent_failures=candidate.recent_failures, + profile_path=f"/admin/analytics/users/{candidate.user_id}", ) ) - candidates.sort( - key=lambda item: (item.recent_failures, item.user_id), - reverse=True, - ) - attention = candidates[:10] - except Exception: + except Exception as exc: + _warn_read_failed("overview_attention", exc) + attention = [] availability["attention"] = _availability(False) return AnalyticsOverview( @@ -1054,7 +1241,6 @@ def get_overview( for day, counts in sorted(lane_days.items()) ], activation_funnel=funnel, - user_state_counts=state_counts, top_failure_categories=failures, users_needing_attention=attention, last_updated=current, @@ -1093,10 +1279,12 @@ def get_platform_model_cost_micro( current_events = ( [ event + # Internal-free, like the rollups it is added to -- the + # overview's own figure reads the same way (see there). for event in self.query_store.list_metric_events( start=current_start, end=effective_end, - include_internal=filters.include_internal, + include_internal=False, ) if _event_matches_filters(event, filters) ] @@ -1111,6 +1299,41 @@ def get_platform_model_cost_micro( ) return None + def get_user_axes( + self, user_id: int, *, now: datetime + ) -> tuple[LifecycleResult, OperationalResult]: + """Live segment and live operational state for exactly one user. + + The segment's inputs are ``user_activity`` plus the trailing 30 fact + days through yesterday, evaluated ``as_of=now``; today's runs reach + the segment tomorrow (the accepted one-day lag, design §6.3). + """ + subject_id = positive_user_id(user_id) + current = _utc(now, "now") + user = self.user_store.get_user_admin(subject_id) + if user is None: + raise LookupError("Analytics user was not found") + yesterday = current.date() - timedelta(days=1) + activity = self.value_store.get_activity(subject_id) + totals = self.value_store.sum_recent_facts( + [subject_id], start=yesterday - timedelta(days=29), end=yesterday + ).get(subject_id) + lifecycle = calculate_lifecycle( + build_lifecycle_inputs( + subject_id, + created_at=_parse_timestamp(user["created_at"]), + activity=activity, + totals=totals, + as_of=current, + ), + current, + ) + facts = self.value_store.get_operational_facts(subject_id, now=current) + operational = calculate_operational_state( + OperationalSignals(**facts.model_dump()), current + ) + return lifecycle, operational + def get_user_profile( self, *, @@ -1118,7 +1341,18 @@ def get_user_profile( now: datetime | None = None, start: datetime | None = None, end: datetime | None = None, + axes: tuple[LifecycleResult, OperationalResult] | None = None, ) -> AnalyticsUserProfile: + """One user's profile: live axes, fact-window sums, one timeline page. + + The run, cost and spend figures sum ``user_daily_facts`` over the + selected window clipped to yesterday ("as of yesterday", design + §6.3). The raw-event scan is the first timeline page (design §6.12), + and every field computed from it -- lane mix, provider, client, top + page, tokens -- is bounded by that page, not by the window. The two + event milestones are first occurrences, so they come from one grouped + MIN() read over the selected range instead. + """ subject_id = positive_user_id(user_id) current = _utc(now or datetime.now(timezone.utc), "now") range_start = ( @@ -1132,33 +1366,42 @@ def get_user_profile( user = self.user_store.get_user_admin(subject_id) if user is None: raise LookupError("Analytics user was not found") - snapshot = self.query_store.states.get_snapshot(subject_id) - if snapshot is None: - snapshot = calculate_user_state( - subject_id, - now=current, - store=self.query_store.states, - ) - events = self.query_store.rollups.list_events( - start=range_start, - end=range_end, - include_internal=True, - user_id=subject_id, + lifecycle, operational = axes or self.get_user_axes(subject_id, now=current) + yesterday = current.date() - timedelta(days=1) + window_start = range_start.date() + window_end = min((range_end - timedelta(microseconds=1)).date(), yesterday) + totals = ( + self.value_store.sum_recent_facts( + [subject_id], start=window_start, end=window_end + ).get(subject_id) + if window_end >= window_start + else None ) - ordered = sorted(events, key=lambda event: event.occurred_at, reverse=True) - meaningful = [event for event in ordered if is_meaningful_event(event)] - milestones: dict[str, datetime] = {} - for event_name in _ACTIVATION_EVENTS: - matches = [ - event.occurred_at for event in events if event.event_name == event_name - ] - if matches: - milestones[event_name] = min(matches) - lane_counts = Counter( - event.billing_mode - for event in events - if event.event_name == "model_usage_recorded" and event.billing_mode + activity = self.value_store.get_activity(subject_id) + + # The profile's one raw-event read: the first timeline page (design §6.12). + rows, _next_cursor = self.query_store.list_activity_rows( + user_id=subject_id, section="timeline", limit=100, cursor=None ) + events = [event for _sequence, event in rows] # newest first + meaningful = [event for event in events if is_meaningful_event(event)] + joined_at = _parse_timestamp(user["created_at"]) + milestones: dict[str, datetime] = {"account_signed_up": joined_at} + if activity is not None and activity.activated_at is not None: + milestones["backtest_completed"] = activity.activated_at + # First occurrences in the selected range, as main computed them -- + # never min() over the timeline page, which holds only the newest + # 100 events and so loses (or misdates) an older first verification. + milestones.update( + self.query_store.first_event_times( + user_id=subject_id, + event_names=("credential_verified", "agent_created"), + start=range_start, + end=range_end, + ) + ) + usage = [event for event in events if event.event_name == "model_usage_recorded"] + lane_counts = Counter(event.billing_mode for event in usage if event.billing_mode) page_counts = Counter( event.page_view for event in events @@ -1167,7 +1410,7 @@ def get_user_profile( provider_event = next( ( event - for event in ordered + for event in events if event.provider_id and event.event_name in {"credential_defaulted", "credential_verified", "model_usage_recorded"} @@ -1177,70 +1420,143 @@ def get_user_profile( client_event = next( ( event - for event in ordered + for event in events if event.country_code or event.device_category or event.browser_family ), None, ) - recent = [ - AnalyticsFootprintItem( - event_id=event.event_id, - event_name=event.event_name, - occurred_at=event.occurred_at, - page_view=event.page_view, - provider_id=event.provider_id, - model_id=event.model_id, - billing_mode=event.billing_mode, - outcome=event.outcome, - error_category=event.error_category, - ) - for event in meaningful[:20] - ] + state_status = _legacy_state(operational.state, lifecycle.segment) + state_reason = ( + operational if state_status in _ATTENTION_STATES else lifecycle + ) return AnalyticsUserProfile( user_id=subject_id, display_name=str(user.get("display_name") or ""), email=str(user.get("email") or ""), - joined_at=_parse_timestamp(user["created_at"]), - last_meaningful_activity=(meaningful[0].occurred_at if meaningful else None), - state=_snapshot_summary(snapshot), + joined_at=joined_at, + last_meaningful_activity=( + activity.last_meaningful_activity_at if activity is not None else None + ), + state=AnalyticsStateSummary( + status=state_status, + reason_code=state_reason.reason_code, + human_readable_reason=state_reason.reason, + # The five-state evidence list had one source, + # calculate_user_state, and the profile no longer calls it. + evidence_event_ids=[], + calculated_at=current, + ), primary_billing_lane=(lane_counts.most_common(1)[0][0] if lane_counts else None), default_provider=(provider_event.provider_id if provider_event else None), country_code=(client_event.country_code if client_event else None), device_category=(client_event.device_category if client_event else None), browser_family=(client_event.browser_family if client_event else None), activation_milestones=milestones, - recent_footprint=recent, + recent_footprint=[ + AnalyticsFootprintItem( + event_id=event.event_id, + event_name=event.event_name, + occurred_at=event.occurred_at, + page_view=event.page_view, + provider_id=event.provider_id, + model_id=event.model_id, + billing_mode=event.billing_mode, + outcome=event.outcome, + error_category=event.error_category, + ) + for event in meaningful[:20] + ], run_summary={ - "requested": sum(event.event_name == "backtest_requested" for event in events), - "completed": sum(event.event_name == "backtest_completed" for event in events), - "failed": sum(event.event_name == "backtest_failed" for event in events), - "cancelled": sum(event.event_name == "backtest_cancelled" for event in events), + "requested": totals.runs_requested if totals else 0, + "completed": totals.runs_completed if totals else 0, + "failed": totals.runs_failed if totals else 0, + "cancelled": totals.runs_cancelled if totals else 0, }, billing_lane_mix=dict(lane_counts), input_tokens=sum( - int(event.properties.get("input_tokens", 0)) - for event in events - if event.event_name == "model_usage_recorded" + int(event.properties.get("input_tokens", 0)) for event in usage ), output_tokens=sum( - int(event.properties.get("output_tokens", 0)) - for event in events - if event.event_name == "model_usage_recorded" + int(event.properties.get("output_tokens", 0)) for event in usage ), - platform_model_cost_usd=sum( - atl_cost_micro(event.billing_mode, event.properties) - for event in events - if event.event_name == "model_usage_recorded" - ) + platform_model_cost_usd=(totals.operator_cost_micro if totals else 0) / 1_000_000, - credits_debited_micro=sum( - int(event.properties.get("amount_micro", 0)) - for event in events - if event.event_name == "credits_settled" - ), + credits_debited_micro=totals.own_spend_micro if totals else 0, top_product_page=(page_counts.most_common(1)[0][0] if page_counts else None), ) + def get_user_metrics( + self, + user_id: int, + *, + audience: Literal["admin", "user"], + now: datetime | None = None, + ) -> UserMetricsShared: + """Build one user's metric struct for one audience (design §6.13). + + ``audience="user"`` returns ``UserMetricsShared`` and never reads the + admin-only sources (the axes, the access log); ``audience="admin"`` + returns ``UserMetricsAdmin``. The shared numbers come from the same + reads either way, so the two projections cannot disagree on them. + Nothing calls this yet and it is on no response model. + """ + if audience not in {"admin", "user"}: + raise ValueError("audience must be admin or user") + subject_id = positive_user_id(user_id) + current = _utc(now or datetime.now(timezone.utc), "now") + user = self.user_store.get_user_admin(subject_id) + if user is None: + raise LookupError("Analytics user was not found") + yesterday = current.date() - timedelta(days=1) + week = self.value_store.sum_recent_facts( + [subject_id], start=yesterday - timedelta(days=6), end=yesterday + ).get(subject_id) + month = self.value_store.sum_recent_facts( + [subject_id], start=yesterday - timedelta(days=29), end=yesterday + ).get(subject_id) + yesterday_start = datetime.combine(yesterday, datetime.min.time(), tzinfo=timezone.utc) + commercial = self.value_store.list_commercial_values( + [subject_id], + start=yesterday_start, + end=yesterday_start + timedelta(days=1), + ).get(subject_id) + tier: CommercialTier = ( + commercial.commercial_tier if commercial is not None else "unpaid" + ) + shared = dict( + user_id=subject_id, + facts_as_of=yesterday, + runs_7d=_run_outcome_counts(week), + runs_30d=_run_outcome_counts(month), + own_spend_micro=month.own_spend_micro if month else 0, + operator_cost_micro=month.operator_cost_micro if month else 0, + tier=tier, + credit_balance_micro=( + commercial.total_available_micro if commercial is not None else 0 + ), + data_quality=( + "complete" if month is not None and month.days_present >= 30 else "partial" + ), + ) + if audience == "user": + return UserMetricsShared(**shared) + lifecycle, operational = self.get_user_axes(subject_id, now=current) + user_group = str(user.get("user_group") or "unknown") + return UserMetricsAdmin( + **shared, + active_days_30d=min(30, month.active_days) if month else 0, + lifecycle=lifecycle, + operational=operational, + user_group=user_group, + group_badge=resolve_group_badge( + role=str(user.get("role") or "user"), user_group=user_group, tier=tier + ), + access_log=[ + str(row["section"]) + for row in self.store.list_admin_access(subject_id, limit=10) + ], + ) + def get_user_activity( self, *, @@ -1248,15 +1564,18 @@ def get_user_activity( section: ActivitySection, limit: int, cursor: str | None, + now: datetime | None = None, ) -> AnalyticsActivityPage: if section not in {"timeline", "runs", "usage", "sessions"}: raise ValueError("section must be a supported Analytics activity section") page_size = positive_limit(limit) if section == "sessions": + current = _utc(now or datetime.now(timezone.utc), "now") rows, next_cursor = self.query_store.list_session_rows( user_id=user_id, limit=page_size, cursor=cursor, + start=current - timedelta(days=30), ) return AnalyticsActivityPage( items=[ @@ -1376,6 +1695,9 @@ def get_value_analytics_query_service() -> ValueAnalyticsQueryService: "AnalyticsUserProfile", "FailureCategoryCount", "PanelAvailability", + "RunOutcomeCounts", + "UserMetricsAdmin", + "UserMetricsShared", "get_analytics_query_service", "get_value_analytics_query_service", ] diff --git a/dashboard/backend/domain/analytics/repository.py b/dashboard/backend/domain/analytics/repository.py index 2aaf6b212..f1efa1722 100644 --- a/dashboard/backend/domain/analytics/repository.py +++ b/dashboard/backend/domain/analytics/repository.py @@ -123,48 +123,6 @@ ) ); -CREATE TABLE IF NOT EXISTS user_analytics_snapshots ( - user_id INTEGER PRIMARY KEY, - status TEXT NOT NULL CHECK ( - status IN ('blocked', 'needs_attention', 'dormant', 'onboarding', 'active') - ), - reason_code TEXT NOT NULL CHECK (length(reason_code) BETWEEN 1 AND 100), - human_readable_reason TEXT NOT NULL - CHECK (length(human_readable_reason) BETWEEN 1 AND 500), - evidence_event_ids_json TEXT NOT NULL DEFAULT '[]' - CHECK (length(evidence_event_ids_json) <= 4096), - calculated_at TEXT NOT NULL, - lifecycle_segment TEXT, - lifecycle_reason_code TEXT, - lifecycle_reason TEXT, - lifecycle_evidence_json TEXT NOT NULL DEFAULT '[]', - operational_state TEXT, - operational_reason_code TEXT, - operational_reason TEXT, - operational_evidence_json TEXT NOT NULL DEFAULT '[]', - activated_at TEXT, - last_meaningful_activity_at TEXT, - inactive_days INTEGER NOT NULL DEFAULT 0, - active_days_30d INTEGER NOT NULL DEFAULT 0, - successful_backtests_30d INTEGER NOT NULL DEFAULT 0, - FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE -); - -CREATE TABLE IF NOT EXISTS user_lifecycle_daily_snapshots ( - snapshot_date TEXT NOT NULL CHECK (length(snapshot_date) = 10), - user_id INTEGER NOT NULL, - lifecycle_segment TEXT NOT NULL, - lifecycle_reason_code TEXT NOT NULL, - data_quality TEXT NOT NULL CHECK (data_quality IN ('complete', 'partial')), - calculated_at TEXT NOT NULL, - PRIMARY KEY (snapshot_date, user_id), - FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE -); -CREATE INDEX IF NOT EXISTS idx_lifecycle_daily_segment - ON user_lifecycle_daily_snapshots(snapshot_date, lifecycle_segment); -CREATE INDEX IF NOT EXISTS idx_lifecycle_daily_user - ON user_lifecycle_daily_snapshots(user_id, snapshot_date DESC); - CREATE TABLE IF NOT EXISTS analytics_projection_jobs ( job_name TEXT PRIMARY KEY, window_start TEXT NOT NULL, @@ -266,6 +224,11 @@ ON admin_analytics_access_log(admin_user_id, accessed_at DESC, sequence DESC); """ +# The five-state snapshot tables PR B retires. Neither DDL declares them any +# more; ``drop_legacy_snapshot_tables`` removes them from a database that +# still has them. Child first, though neither references the other. +LEGACY_SNAPSHOT_TABLES = ("user_lifecycle_daily_snapshots", "user_analytics_snapshots") + _EVENT_COLUMNS = ( "event_id", @@ -343,38 +306,19 @@ def _get_connection(self) -> Iterator[sqlite3.Connection]: def _init_schema(self) -> None: with self._get_connection() as conn: conn.executescript(ANALYTICS_SQLITE_DDL) - self._migrate_value_columns(conn) self._migrate_error_category_constraint(conn) - @staticmethod - def _migrate_value_columns(conn: sqlite3.Connection) -> None: - """Add value-analytics columns to installations created by PR 1.""" - columns = { - str(row[1]) - for row in conn.execute( - "PRAGMA table_info(user_analytics_snapshots)" - ).fetchall() - } - additions = { - "lifecycle_segment": "TEXT", - "lifecycle_reason_code": "TEXT", - "lifecycle_reason": "TEXT", - "lifecycle_evidence_json": "TEXT NOT NULL DEFAULT '[]'", - "operational_state": "TEXT", - "operational_reason_code": "TEXT", - "operational_reason": "TEXT", - "operational_evidence_json": "TEXT NOT NULL DEFAULT '[]'", - "activated_at": "TEXT", - "last_meaningful_activity_at": "TEXT", - "inactive_days": "INTEGER NOT NULL DEFAULT 0", - "active_days_30d": "INTEGER NOT NULL DEFAULT 0", - "successful_backtests_30d": "INTEGER NOT NULL DEFAULT 0", - } - for name, definition in additions.items(): - if name not in columns: - conn.execute( - f"ALTER TABLE user_analytics_snapshots ADD COLUMN {name} {definition}" - ) + def drop_legacy_snapshot_tables(self) -> None: + """Drop the five-state snapshot tables (design SS11, SS13 row B). + + ``IF EXISTS`` on both, so the second boot after the deploy that drops + them, and a fresh install whose DDL never created them, are no-ops. + ``facts_migration.run_startup_migrations`` calls it only after + ``seed_activity_from_snapshots`` has read the final legacy row. + """ + with self._get_connection() as conn: + for table in LEGACY_SNAPSHOT_TABLES: + conn.execute(f"DROP TABLE IF EXISTS {table}") @staticmethod def _migrate_error_category_constraint(conn: sqlite3.Connection) -> None: diff --git a/dashboard/backend/domain/analytics/repository_postgres.py b/dashboard/backend/domain/analytics/repository_postgres.py index 5a1749636..99904207f 100644 --- a/dashboard/backend/domain/analytics/repository_postgres.py +++ b/dashboard/backend/domain/analytics/repository_postgres.py @@ -10,7 +10,12 @@ from dashboard.backend.db_url import init_schema_unless_worker, require_postgres_url from .models import AnalyticsEventRecord, AppendEventResult, RetentionResult -from .repository import _EVENT_COLUMNS, _event_values, _row_to_event +from .repository import ( + LEGACY_SNAPSHOT_TABLES, + _EVENT_COLUMNS, + _event_values, + _row_to_event, +) from .repository_common import ( AnalyticsIdempotencyConflictError, AnalyticsStoreError, @@ -113,46 +118,6 @@ ) ); -CREATE TABLE IF NOT EXISTS user_analytics_snapshots ( - user_id INTEGER PRIMARY KEY REFERENCES users(id) ON DELETE CASCADE, - status TEXT NOT NULL CHECK ( - status IN ('blocked', 'needs_attention', 'dormant', 'onboarding', 'active') - ), - reason_code TEXT NOT NULL CHECK (length(reason_code) BETWEEN 1 AND 100), - human_readable_reason TEXT NOT NULL - CHECK (length(human_readable_reason) BETWEEN 1 AND 500), - evidence_event_ids_json TEXT NOT NULL DEFAULT '[]' - CHECK (length(evidence_event_ids_json) <= 4096), - calculated_at TEXT NOT NULL, - lifecycle_segment TEXT, - lifecycle_reason_code TEXT, - lifecycle_reason TEXT, - lifecycle_evidence_json TEXT NOT NULL DEFAULT '[]', - operational_state TEXT, - operational_reason_code TEXT, - operational_reason TEXT, - operational_evidence_json TEXT NOT NULL DEFAULT '[]', - activated_at TEXT, - last_meaningful_activity_at TEXT, - inactive_days INTEGER NOT NULL DEFAULT 0, - active_days_30d INTEGER NOT NULL DEFAULT 0, - successful_backtests_30d INTEGER NOT NULL DEFAULT 0 -); - -CREATE TABLE IF NOT EXISTS user_lifecycle_daily_snapshots ( - snapshot_date TEXT NOT NULL CHECK (length(snapshot_date) = 10), - user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE, - lifecycle_segment TEXT NOT NULL, - lifecycle_reason_code TEXT NOT NULL, - data_quality TEXT NOT NULL CHECK (data_quality IN ('complete', 'partial')), - calculated_at TEXT NOT NULL, - PRIMARY KEY (snapshot_date, user_id) -); -CREATE INDEX IF NOT EXISTS idx_lifecycle_daily_segment - ON user_lifecycle_daily_snapshots(snapshot_date, lifecycle_segment); -CREATE INDEX IF NOT EXISTS idx_lifecycle_daily_user - ON user_lifecycle_daily_snapshots(user_id, snapshot_date DESC); - CREATE TABLE IF NOT EXISTS analytics_projection_jobs ( job_name TEXT PRIMARY KEY, window_start TEXT NOT NULL, @@ -268,25 +233,6 @@ def _init_schema(self) -> None: "ALTER TABLE analytics_events " "DROP CONSTRAINT IF EXISTS analytics_events_error_category_check" ) - for column, definition in { - "lifecycle_segment": "TEXT", - "lifecycle_reason_code": "TEXT", - "lifecycle_reason": "TEXT", - "lifecycle_evidence_json": "TEXT NOT NULL DEFAULT '[]'", - "operational_state": "TEXT", - "operational_reason_code": "TEXT", - "operational_reason": "TEXT", - "operational_evidence_json": "TEXT NOT NULL DEFAULT '[]'", - "activated_at": "TEXT", - "last_meaningful_activity_at": "TEXT", - "inactive_days": "INTEGER NOT NULL DEFAULT 0", - "active_days_30d": "INTEGER NOT NULL DEFAULT 0", - "successful_backtests_30d": "INTEGER NOT NULL DEFAULT 0", - }.items(): - cur.execute( - "ALTER TABLE user_analytics_snapshots " - f"ADD COLUMN IF NOT EXISTS {column} {definition}" - ) cur.execute( """ ALTER TABLE analytics_events @@ -304,6 +250,15 @@ def _init_schema(self) -> None: """ ) + def drop_legacy_snapshot_tables(self) -> None: + """See the SQLite twin. Both DROPs commit in one transaction.""" + with self._get_connection() as conn: + with conn.cursor() as cur: + # Old instance holds a lock? Fail loudly, retry next boot; never stall startup. + cur.execute("SET LOCAL lock_timeout = '5s'") + for table in LEGACY_SNAPSHOT_TABLES: + cur.execute(f"DROP TABLE IF EXISTS {table}") + @staticmethod def _select_event(cur, field: str, value: str | None): if value is None: diff --git a/dashboard/backend/domain/analytics/retention.py b/dashboard/backend/domain/analytics/retention.py index 6cea6011a..c7eda8719 100644 --- a/dashboard/backend/domain/analytics/retention.py +++ b/dashboard/backend/domain/analytics/retention.py @@ -80,33 +80,19 @@ def run_once(self, now: datetime | None = None) -> RetentionResult: if self.value_store is not None: lifecycle_before = current.date() - timedelta(days=RAW_EVENT_RETENTION_DAYS) - expiring_days = self.value_store.list_expiring_daily_dates( + expiring_days = self.value_store.list_expiring_fact_dates( before=lifecycle_before, limit=min(self.batch_size, self.max_batches), ) - # Aggregate every selected day before deleting any of them so - # adjacent-day transitions remain available throughout the batch. + # Aggregate every selected day before deleting any of it (design + # SS11: the anonymous rollups are what outlives the 180-day + # horizon). Facts and the transition log for one day expire + # together, in one transaction. for day in expiring_days: rollup_lifecycle_day(day, store=self.value_store) - expiring_set = set(expiring_days) - for boundary_day in sorted( - { - day + timedelta(days=1) - for day in expiring_days - if day + timedelta(days=1) not in expiring_set - } - ): - boundary_rows = self.value_store.list_daily_snapshots( - start=boundary_day, - end=boundary_day + timedelta(days=1), - ) - if boundary_rows: - rollup_lifecycle_day(boundary_day, store=self.value_store) for day in expiring_days: - lifecycle_deleted += self.value_store.delete_daily_snapshots_for_date( - day - ) - has_more_lifecycle = self.value_store.has_daily_before(lifecycle_before) + lifecycle_deleted += self.value_store.expire_facts_for_date(day) + has_more_lifecycle = self.value_store.has_facts_before(lifecycle_before) return RetentionResult( raw_events_deleted=raw_deleted, diff --git a/dashboard/backend/domain/analytics/rollups.py b/dashboard/backend/domain/analytics/rollups.py index b5e35636b..43916968c 100644 --- a/dashboard/backend/domain/analytics/rollups.py +++ b/dashboard/backend/domain/analytics/rollups.py @@ -8,19 +8,16 @@ from collections import Counter, defaultdict from datetime import date, datetime, time, timedelta, timezone -from typing import Any, Iterable, Mapping, Sequence +from typing import Any, Iterable, Sequence from pydantic import BaseModel, ConfigDict, Field -from .metrics import calculate_overview_metrics +from .metrics import ACTIVATION_EVENTS, calculate_overview_metrics from .models import AnalyticsEventRecord from .repository import _row_to_event, analytics_store from .repository_common import utc_iso from .usage_cost import atl_cost_micro, byok_estimate_micro -from .value_repository import ( - LIFECYCLE_ROLLUP_METRICS, - UserLifecycleDailySnapshot, -) +from .value_repository import LIFECYCLE_ROLLUP_METRICS class DailyRollup(BaseModel): @@ -95,49 +92,20 @@ def list_events( return [event for event in events if event.user_id not in excluded] def replace_day(self, day: date, rows: Iterable[DailyRollup]) -> None: - values = list(rows) - columns = ( - "rollup_date", - "metric_name", - "event_name", - "billing_mode", - "provider_id", - "model_id", - "outcome", - "error_category", - "user_state", - "value_count", - "value_sum_micro", - "updated_at", - ) - payloads = [ - ( - row.rollup_date.isoformat(), - row.metric_name, - row.event_name, - row.billing_mode, - row.provider_id, - row.model_id, - row.outcome, - row.error_category, - row.user_state, - row.value_count, - row.value_sum_micro, - utc_iso(row.updated_at), - ) - for row in values - ] + columns = _ROLLUP_TABLE_COLUMNS + payloads = [_rollup_payload(row) for row in rows] + preserved = sorted(LIFECYCLE_ROLLUP_METRICS) if self.is_postgres: placeholders = ", ".join("%s" for _ in columns) with self.base_store._get_connection() as conn: with conn.cursor() as cur: cur.execute( - """ + f""" DELETE FROM analytics_daily_rollups WHERE rollup_date = %s - AND metric_name NOT IN (%s, %s) + AND metric_name NOT IN ({", ".join("%s" for _ in preserved)}) """, - (day.isoformat(), *sorted(LIFECYCLE_ROLLUP_METRICS)), + (day.isoformat(), *preserved), ) if payloads: cur.executemany( @@ -149,12 +117,12 @@ def replace_day(self, day: date, rows: Iterable[DailyRollup]) -> None: placeholders = ", ".join("?" for _ in columns) with self.base_store._get_connection() as conn: conn.execute( - """ + f""" DELETE FROM analytics_daily_rollups WHERE rollup_date = ? - AND metric_name NOT IN (?, ?) + AND metric_name NOT IN ({", ".join("?" for _ in preserved)}) """, - (day.isoformat(), *sorted(LIFECYCLE_ROLLUP_METRICS)), + (day.isoformat(), *preserved), ) if payloads: conn.executemany( @@ -163,6 +131,106 @@ def replace_day(self, day: date, rows: Iterable[DailyRollup]) -> None: payloads, ) + def backfill_activation_users(self, *, since: date, until: date) -> tuple[date, ...]: + """Write ``activation_users`` for rolled-up days that predate the metric. + + ``rollup_day`` gained the metric in PR B, so every day rolled up + before it holds none, and the overview's funnel summed those days as + zero. One grouped read over the raw events of the missing days -- + distinct users per step per day, counted in SQL -- fills them, where + a ``rollup_day`` rebuild per day would re-read 31 days of events for + each one. Only days in ``[since, until)`` that were rolled up (they + hold ``daily_active_users``) are written, so a day whose raw events + have expired is never filled with zeros; admin and excluded accounts + are left out, as ``list_events`` leaves them out. A filled day no + longer matches, so a re-run writes nothing. Returns the days filled. + """ + p = "%s" if self.is_postgres else "?" + missing_sql = f""" + SELECT DISTINCT r.rollup_date FROM analytics_daily_rollups AS r + WHERE r.metric_name = 'daily_active_users' + AND r.rollup_date >= {p} AND r.rollup_date < {p} + AND NOT EXISTS ( + SELECT 1 FROM analytics_daily_rollups AS a + WHERE a.rollup_date = r.rollup_date + AND a.metric_name = 'activation_users' + ) + ORDER BY r.rollup_date + """ + events = sorted(ACTIVATION_EVENTS) + counts_sql = f""" + SELECT substr(occurred_at, 1, 10) AS day, event_name, + COUNT(DISTINCT user_id) AS users + FROM analytics_events + WHERE occurred_at >= {p} AND occurred_at < {p} + AND event_name IN ({", ".join(p for _ in events)}) + AND user_id NOT IN (SELECT id FROM users WHERE role = 'admin') + AND user_id NOT IN ( + SELECT user_id FROM analytics_subject_settings WHERE excluded + ) + GROUP BY substr(occurred_at, 1, 10), event_name + """ + insert_sql = ( + f"INSERT INTO analytics_daily_rollups ({', '.join(_ROLLUP_TABLE_COLUMNS)}) " + f"VALUES ({', '.join(p for _ in _ROLLUP_TABLE_COLUMNS)})" + ) + + def fill(run: Any) -> tuple[date, ...]: + days = tuple( + date.fromisoformat(str(row["rollup_date"])) + for row in run(missing_sql, (since.isoformat(), until.isoformat())) + ) + if not days: + return () + counts = { + (str(row["day"]), str(row["event_name"])): int(row["users"]) + for row in run( + counts_sql, + ( + utc_iso(_day_start(days[0])), + utc_iso(_day_start(days[-1] + timedelta(days=1))), + *events, + ), + ) + } + payloads = [ + _rollup_payload( + _row( + day, + "activation_users", + count=counts.get((day.isoformat(), event_name), 0), + event_name=event_name, + updated_at=_day_start(day + timedelta(days=1)), + ) + ) + for day in days + for event_name in ACTIVATION_EVENTS + ] + run(insert_sql, payloads, many=True) + return days + + if self.is_postgres: + with self.base_store._get_connection() as conn: + with conn.cursor() as cur: + + def run_postgres(sql: str, params: Any, many: bool = False) -> Any: + if many: + cur.executemany(sql, params) + return [] + cur.execute(sql, params) + return cur.fetchall() + + return fill(run_postgres) + with self.base_store._get_connection() as conn: + + def run_sqlite(sql: str, params: Any, many: bool = False) -> Any: + if many: + conn.executemany(sql, params) + return [] + return conn.execute(sql, params).fetchall() + + return fill(run_sqlite) + def list_rollups(self, *, start: date, end: date) -> list[DailyRollup]: params = (start.isoformat(), end.isoformat()) if self.is_postgres: @@ -210,6 +278,39 @@ def list_rollups(self, *, start: date, end: date) -> list[DailyRollup]: ] +_ROLLUP_TABLE_COLUMNS = ( + "rollup_date", + "metric_name", + "event_name", + "billing_mode", + "provider_id", + "model_id", + "outcome", + "error_category", + "user_state", + "value_count", + "value_sum_micro", + "updated_at", +) + + +def _rollup_payload(row: DailyRollup) -> tuple[Any, ...]: + return ( + row.rollup_date.isoformat(), + row.metric_name, + row.event_name, + row.billing_mode, + row.provider_id, + row.model_id, + row.outcome, + row.error_category, + row.user_state, + row.value_count, + row.value_sum_micro, + utc_iso(row.updated_at), + ) + + def _row( day: date, metric_name: str, @@ -233,7 +334,6 @@ def rollup_day( day: date, *, store: AnalyticsRollupStore | None = None, - state_counts: Mapping[str, int] | None = None, include_internal: bool = False, now: datetime | None = None, ) -> list[DailyRollup]: @@ -311,7 +411,6 @@ def rollup_day( ), _row(day, "input_tokens", count=overview.input_tokens, updated_at=current), _row(day, "output_tokens", count=overview.output_tokens, updated_at=current), - _row(day, "affected_users", count=overview.affected_users, updated_at=current), ] event_dimensions = Counter( @@ -443,31 +542,25 @@ def rollup_day( ), ] ) - failure_users: dict[str, set[int]] = defaultdict(set) + # Distinct users per activation step per day: the overview's funnel sums + # these across completed days instead of rescanning the window. Written + # zero or not, like byok_unpriced_calls: presence is how the funnel tells + # a day with no activations from a day rolled up before this metric + # existed, which it must not read as zero. + activation_users: dict[str, set[int]] = defaultdict(set) for event in day_events: - if event.error_category: - failure_users[event.error_category].add(event.user_id) - for category, users in sorted(failure_users.items()): - rows.append( - _row( - day, - "affected_users", - count=len(users), - error_category=category, - updated_at=current, - ) - ) - for status, count in sorted((state_counts or {}).items()): + if event.event_name in ACTIVATION_EVENTS: + activation_users[event.event_name].add(event.user_id) + for event_name in ACTIVATION_EVENTS: rows.append( _row( day, - "user_state_count", - count=int(count), - user_state=status, + "activation_users", + count=len(activation_users[event_name]), + event_name=event_name, updated_at=current, ) ) - rows.sort( key=lambda row: ( row.metric_name, @@ -484,27 +577,8 @@ def rollup_day( return rows -def rollup_current_day( - *, - store: AnalyticsRollupStore | None = None, - now: datetime | None = None, - include_internal: bool = False, -): - current = now or datetime.now(timezone.utc) - start = _day_start(current.astimezone(timezone.utc).date()) - aggregate_store = store or AnalyticsRollupStore() - events = aggregate_store.list_events( - start=start - timedelta(days=30), - end=current, - include_internal=include_internal, - ) - return calculate_overview_metrics(events, start=start, end=current) - - -def _snapshot_quality(rows: Sequence[UserLifecycleDailySnapshot]) -> str: - return ( - "partial" if any(row.data_quality == "partial" for row in rows) else "complete" - ) +def _quality(rows: Sequence[Any]) -> str: + return "partial" if any(row.data_quality == "partial" for row in rows) else "complete" def rollup_lifecycle_day( @@ -512,74 +586,195 @@ def rollup_lifecycle_day( *, store, ) -> tuple[DailyRollup, ...]: - """Persist anonymous lifecycle counts and changed-segment transitions.""" + """Persist anonymous lifecycle counts, transitions and the long-term + tier / user_group families for one day (design SS6.7, SS11). + + Reads the day's fact rows and transition log; writes nothing keyed by a + user. The retention sweep calls it for every expiring day before that + day's user-keyed rows are deleted. ``runs_by_tier`` / + ``runs_by_user_group`` count ``runs_completed``, not the + ``runs_requested`` that ``/groups`` reports as ``total_runs``. A day with + no facts writes nothing: + its facts may already have expired, and replacing its lifecycle rollups + with an empty set would erase the history this function exists to keep. + """ + + facts = store.list_facts_for_date(day) + transitions = store.list_transitions_for_date(day) + if not facts: + if transitions: + # Unreachable while facts and transitions expire together; if it + # ever happens the day's movement is lost, so say so. + print( + "WARNING: analytics.rollup.lifecycle_day_without_facts " + f"day={day.isoformat()} transitions={len(transitions)}", + flush=True, + ) + return () + updated_at = _day_start(day + timedelta(days=1)) + rows: list[DailyRollup] = [] - current = store.list_daily_snapshots( - start=day, - end=day + timedelta(days=1), - ) - previous = store.list_daily_snapshots( - start=day - timedelta(days=1), - end=day, - ) + by_segment: dict[str, list[Any]] = defaultdict(list) + for fact in facts: + by_segment[fact.lifecycle_segment].append(fact) + for segment, group in sorted(by_segment.items()): + rows.append( + _row( + day, + "lifecycle_segment_count", + count=len(group), + user_state=segment, + outcome=_quality(group), + updated_at=updated_at, + ) + ) + + by_transition: dict[tuple[str, str], list[Any]] = defaultdict(list) + for transition in transitions: + by_transition[(transition.from_segment, transition.to_segment)].append( + transition + ) + for (from_segment, to_segment), group in sorted(by_transition.items()): + rows.append( + _row( + day, + "lifecycle_transition", + count=len(group), + event_name=from_segment, + user_state=to_segment, + outcome=_quality(group), + updated_at=updated_at, + ) + ) + + # ``runs_by_*`` count runs_completed -- finished runs -- unlike /groups' + # ``total_runs``, which counts runs_requested. These rows are permanent + # once the day's facts expire, so the semantics are fixed here. + for dimension in ("tier", "user_group"): + by_value: dict[str, list[Any]] = defaultdict(list) + for fact in facts: + by_value[getattr(fact, dimension)].append(fact) + for value, group in sorted(by_value.items()): + quality = _quality(group) + rows.append( + _row( + day, + f"runs_by_{dimension}", + count=sum(fact.runs_completed for fact in group), + user_state=value, + outcome=quality, + updated_at=updated_at, + ) + ) + rows.append( + _row( + day, + f"operator_cost_by_{dimension}", + count=len(group), + sum_micro=sum(fact.operator_cost_micro for fact in group), + user_state=value, + outcome=quality, + updated_at=updated_at, + ) + ) + rows.append( + _row( + day, + f"own_spend_by_{dimension}", + count=len(group), + sum_micro=sum(fact.own_spend_micro for fact in group), + user_state=value, + outcome=quality, + updated_at=updated_at, + ) + ) + + rows.sort(key=lambda row: (row.metric_name, row.event_name, row.user_state)) + # Transitions come from their own table for ``day``, so there is no + # "previous day missing" case left to protect against. + store.replace_lifecycle_rollups(day, rows, replace_transitions=True) + return tuple(rows) + + +def _legacy_lifecycle_rows( + day: date, current: Sequence[Any], previous: Sequence[Any] +) -> list[DailyRollup]: + """Main's ``rollup_lifecycle_day`` over legacy snapshot rows, unchanged in shape: + a count per segment, and a transition per (previous, current) segment pair + for users present on both days whose segment moved.""" updated_at = _day_start(day + timedelta(days=1)) rows: list[DailyRollup] = [] - by_segment: dict[str, list[UserLifecycleDailySnapshot]] = defaultdict(list) + by_segment: dict[str, list[Any]] = defaultdict(list) for snapshot in current: by_segment[snapshot.lifecycle_segment].append(snapshot) - for segment, snapshots in sorted(by_segment.items()): + for segment, group in sorted(by_segment.items()): rows.append( _row( day, "lifecycle_segment_count", - count=len(snapshots), + count=len(group), user_state=segment, - outcome=_snapshot_quality(snapshots), + outcome=_quality(group), updated_at=updated_at, ) ) - previous_by_user = {snapshot.user_id: snapshot for snapshot in previous} - transition_snapshots: dict[tuple[str, str], list[UserLifecycleDailySnapshot]] = ( - defaultdict(list) - ) - transition_quality: dict[tuple[str, str], list[UserLifecycleDailySnapshot]] = ( - defaultdict(list) - ) + moved: dict[tuple[str, str], list[Any]] = defaultdict(list) + moved_quality: dict[tuple[str, str], list[Any]] = defaultdict(list) for snapshot in current: prior = previous_by_user.get(snapshot.user_id) if prior is None or prior.lifecycle_segment == snapshot.lifecycle_segment: continue key = (prior.lifecycle_segment, snapshot.lifecycle_segment) - transition_snapshots[key].append(snapshot) - transition_quality[key].extend((prior, snapshot)) - for (from_segment, to_segment), snapshots in sorted(transition_snapshots.items()): + moved[key].append(snapshot) + moved_quality[key].extend((prior, snapshot)) + for (from_segment, to_segment), group in sorted(moved.items()): rows.append( _row( day, "lifecycle_transition", - count=len(snapshots), + count=len(group), event_name=from_segment, user_state=to_segment, - outcome=_snapshot_quality( - transition_quality[(from_segment, to_segment)] - ), + outcome=_quality(moved_quality[(from_segment, to_segment)]), updated_at=updated_at, ) ) rows.sort(key=lambda row: (row.metric_name, row.event_name, row.user_state)) - store.replace_lifecycle_rollups( - day, - rows, - replace_transitions=bool(previous), - ) - return tuple(rows) + return rows + + +def rollup_legacy_lifecycle_days(*, store) -> tuple[date, ...]: + """Roll the legacy days no fact row covers into anonymous lifecycle rollups. + + The startup migration's step between the ``user_activity`` seed and the + drop of ``user_lifecycle_daily_snapshots`` (design SS6.9, SS11): the PR A + history copy reached back only 56 days, so older legacy days live in that + table alone, and ``_history`` already falls back to these two families for + any day without facts. Days that already hold a lifecycle-owned rollup are + excluded by the store's read, so nothing is overwritten and a re-run (a + boot whose drop failed) writes nothing new. Transitions are written only + when the previous legacy day exists, exactly as main's writer did. + Returns the days rolled up. + """ + history = store.list_legacy_lifecycle_history() + by_day: dict[date, list[Any]] = defaultdict(list) + for snapshot in history.snapshots: + by_day[snapshot.snapshot_date].append(snapshot) + for day in history.days: + previous = by_day.get(day - timedelta(days=1), []) + store.replace_lifecycle_rollups( + day, + _legacy_lifecycle_rows(day, by_day.get(day, []), previous), + replace_transitions=bool(previous), + ) + return history.days __all__ = [ "AnalyticsRollupStore", "DailyRollup", - "rollup_current_day", "rollup_day", + "rollup_legacy_lifecycle_days", "rollup_lifecycle_day", ] diff --git a/dashboard/backend/domain/analytics/service.py b/dashboard/backend/domain/analytics/service.py index 5994b3d47..09ce53345 100644 --- a/dashboard/backend/domain/analytics/service.py +++ b/dashboard/backend/domain/analytics/service.py @@ -39,49 +39,17 @@ def __init__( self, store, *, - state_store=None, value_store=None, - project_snapshots: bool = False, maintain_activity: bool = False, ): - if not isinstance(project_snapshots, bool): - raise TypeError("project_snapshots must be a boolean") if not isinstance(maintain_activity, bool): raise TypeError("maintain_activity must be a boolean") - if project_snapshots and (state_store is None or value_store is None): - raise ValueError("snapshot stores are required when projection is enabled") if maintain_activity and value_store is None: raise ValueError("a value store is required to maintain user activity") self.store = store - self.state_store = state_store self.value_store = value_store - self.project_snapshots = project_snapshots self.maintain_activity = maintain_activity - def _try_recalculate_snapshots( - self, - *, - user_id: int, - event_name: str, - now: datetime, - ) -> None: - if not self.project_snapshots: - return - try: - from .states import recalculate_user_snapshots - - recalculate_user_snapshots( - user_id, - now=now, - state_store=self.state_store, - value_store=self.value_store, - ) - except Exception as exc: - print( - "WARNING: analytics.value_projection_failed " - f"event={event_name} category={type(exc).__name__[:80]}" - ) - def _try_record_activity( self, *, @@ -204,19 +172,8 @@ def record_server_event( result = self.store.append_event(event) from .lifecycle import is_lifecycle_activity - lifecycle_activity = is_lifecycle_activity(event) - if result.created and lifecycle_activity: + if result.created and is_lifecycle_activity(event): self._try_record_activity(user_id=subject_id, event=event, now=received) - projection_relevant = lifecycle_activity or event_name in { - "account_signed_up", - "safe_error_recorded", - } - if result.created and projection_relevant: - self._try_recalculate_snapshots( - user_id=subject_id, - event_name=event_name, - now=received, - ) return result def try_record_server_event(self, **kwargs) -> AppendEventResult | None: @@ -271,22 +228,13 @@ def record_admin_profile_access( def _build_analytics_service() -> AnalyticsService: - from .states import AnalyticsStateStore from .value_repository import build_value_analytics_store - state_store = AnalyticsStateStore(analytics_store) return AnalyticsService( store=analytics_store, - state_store=state_store, value_store=build_value_analytics_store(analytics_store), - # PR 0 (burner kill): the live singleton no longer recomputes a - # snapshot synchronously inside record_server_event. Task 3 of the - # same plan disarms instrumentation.py's own fallback recalculator, - # which otherwise steps in for exactly this state (see that task -- - # flipping this flag alone is not sufficient). - project_snapshots=False, - # PR A replaces the per-event snapshot recompute with the activity - # upsert. PR B deletes the projection flag entirely. + # The activity upsert is the only per-event projection (design SS6.5); + # PR B deleted the per-event snapshot recompute it replaced. maintain_activity=True, ) diff --git a/dashboard/backend/domain/analytics/states.py b/dashboard/backend/domain/analytics/states.py deleted file mode 100644 index 200d42789..000000000 --- a/dashboard/backend/domain/analytics/states.py +++ /dev/null @@ -1,804 +0,0 @@ -"""Legacy and dual-axis Analytics snapshots with bounded repair logic.""" - -from __future__ import annotations - -import json -from datetime import datetime, time, timedelta, timezone -from typing import Any - -from pydantic import BaseModel, ConfigDict, Field, field_validator - -from .lifecycle import ( - LifecycleInputs, - OperationalSignals, - calculate_lifecycle, - calculate_operational_state, - is_lifecycle_activity, -) -from .metrics import is_meaningful_event -from .models import AnalyticsEventRecord -from .repository import analytics_store -from .repository_common import positive_limit, positive_user_id, utc_iso -from .rollups import AnalyticsRollupStore -from .value_repository import ( - UserLifecycleDailySnapshot, - UserValueSnapshot, - build_value_analytics_store, -) - - -_REASONS = { - "billing_lane_unavailable": "The latest attempted run has no usable billing lane.", - "provider_disabled": "The selected model provider is currently unavailable.", - "account_restricted": "The account is currently restricted from model spending.", - "invalid_default_credential": "The default model credential is invalid.", - "three_consecutive_failed_runs": "The three newest terminal runs failed within 24 hours.", - "run_deadline_exceeded": "A run remains non-terminal beyond its safe deadline.", - "no_meaningful_activity_30d": "No meaningful product activity was observed for 30 days.", - "no_successful_run": "The user has not completed a successful backtest yet.", - "recent_successful_run_and_activity": "A successful run and recent meaningful activity are present.", - "successful_run_missing": "A successful run is expected but cannot be found.", -} - - -class UserAnalyticsSnapshot(BaseModel): - model_config = ConfigDict(extra="forbid", frozen=True) - - user_id: int = Field(gt=0) - status: str - reason_code: str = Field(min_length=1, max_length=100) - human_readable_reason: str = Field(min_length=1, max_length=500) - evidence_event_ids: list[str] = Field(default_factory=list, max_length=5) - calculated_at: datetime - - @field_validator("status") - @classmethod - def validate_status(cls, value: str) -> str: - allowed = {"blocked", "needs_attention", "dormant", "onboarding", "active"} - if value not in allowed: - raise ValueError("unsupported Analytics user state") - return value - - @field_validator("calculated_at") - @classmethod - def require_timezone(cls, value: datetime) -> datetime: - if value.tzinfo is None or value.utcoffset() is None: - raise ValueError("calculated_at must include a timezone") - return value.astimezone(timezone.utc) - - -def _parse_timestamp(value: object) -> datetime: - parsed = datetime.fromisoformat(str(value)) - if parsed.tzinfo is None or parsed.utcoffset() is None: - parsed = parsed.replace(tzinfo=timezone.utc) - return parsed.astimezone(timezone.utc) - - -class AnalyticsStateStore: - """Snapshot companion operations over PR 1 tables.""" - - def __init__(self, base_store=analytics_store): - self.base_store = base_store - self.rollups = AnalyticsRollupStore(base_store) - self.is_postgres = hasattr(base_store, "database_url") - - def get_user(self, user_id: int) -> dict[str, Any] | None: - subject_id = positive_user_id(user_id) - if self.is_postgres: - with self.base_store._get_connection() as conn: - with conn.cursor() as cur: - cur.execute("SELECT * FROM users WHERE id = %s", (subject_id,)) - row = cur.fetchone() - else: - with self.base_store._get_connection() as conn: - row = conn.execute( - "SELECT * FROM users WHERE id = ?", (subject_id,) - ).fetchone() - return dict(row) if row is not None else None - - def list_user_events( - self, - user_id: int, - *, - now: datetime, - days: int = 180, - ) -> list[AnalyticsEventRecord]: - return self.rollups.list_events( - start=now - timedelta(days=days), - end=now + timedelta(microseconds=1), - include_internal=True, - user_id=positive_user_id(user_id), - ) - - def upsert_snapshot( - self, - snapshot: UserAnalyticsSnapshot, - ) -> UserAnalyticsSnapshot: - payload = ( - snapshot.user_id, - snapshot.status, - snapshot.reason_code, - snapshot.human_readable_reason, - json.dumps(snapshot.evidence_event_ids, separators=(",", ":")), - utc_iso(snapshot.calculated_at), - ) - if self.is_postgres: - with self.base_store._get_connection() as conn: - with conn.cursor() as cur: - cur.execute( - """ - INSERT INTO user_analytics_snapshots ( - user_id, status, reason_code, human_readable_reason, - evidence_event_ids_json, calculated_at - ) VALUES (%s, %s, %s, %s, %s, %s) - ON CONFLICT(user_id) DO UPDATE SET - status = EXCLUDED.status, - reason_code = EXCLUDED.reason_code, - human_readable_reason = EXCLUDED.human_readable_reason, - evidence_event_ids_json = EXCLUDED.evidence_event_ids_json, - calculated_at = EXCLUDED.calculated_at - """, - payload, - ) - else: - with self.base_store._get_connection() as conn: - conn.execute( - """ - INSERT INTO user_analytics_snapshots ( - user_id, status, reason_code, human_readable_reason, - evidence_event_ids_json, calculated_at - ) VALUES (?, ?, ?, ?, ?, ?) - ON CONFLICT(user_id) DO UPDATE SET - status = excluded.status, - reason_code = excluded.reason_code, - human_readable_reason = excluded.human_readable_reason, - evidence_event_ids_json = excluded.evidence_event_ids_json, - calculated_at = excluded.calculated_at - """, - payload, - ) - return snapshot - - def upsert_combined_snapshot( - self, - legacy: UserAnalyticsSnapshot, - value: UserValueSnapshot, - ) -> tuple[UserAnalyticsSnapshot, UserValueSnapshot]: - """Persist compatibility and dual-axis fields in one database statement.""" - - if legacy.user_id != value.user_id: - raise ValueError("snapshot user IDs must match") - if legacy.calculated_at != value.calculated_at: - raise ValueError("snapshot calculation times must match") - - def evidence(items: Any) -> str: - return json.dumps(list(items), separators=(",", ":"), ensure_ascii=True) - - payload = ( - legacy.user_id, - legacy.status, - legacy.reason_code, - legacy.human_readable_reason, - evidence(legacy.evidence_event_ids), - value.lifecycle_segment, - value.lifecycle_reason_code, - value.lifecycle_reason, - evidence(value.lifecycle_evidence), - value.operational_state, - value.operational_reason_code, - value.operational_reason, - evidence(value.operational_evidence), - utc_iso(value.activated_at) if value.activated_at else None, - ( - utc_iso(value.last_meaningful_activity_at) - if value.last_meaningful_activity_at - else None - ), - value.inactive_days, - value.active_days_30d, - value.successful_backtests_30d, - utc_iso(value.calculated_at), - ) - columns = """ - user_id, status, reason_code, human_readable_reason, - evidence_event_ids_json, lifecycle_segment, lifecycle_reason_code, - lifecycle_reason, lifecycle_evidence_json, operational_state, - operational_reason_code, operational_reason, - operational_evidence_json, activated_at, - last_meaningful_activity_at, inactive_days, active_days_30d, - successful_backtests_30d, calculated_at - """ - updates = """ - status=excluded.status, - reason_code=excluded.reason_code, - human_readable_reason=excluded.human_readable_reason, - evidence_event_ids_json=excluded.evidence_event_ids_json, - lifecycle_segment=excluded.lifecycle_segment, - lifecycle_reason_code=excluded.lifecycle_reason_code, - lifecycle_reason=excluded.lifecycle_reason, - lifecycle_evidence_json=excluded.lifecycle_evidence_json, - operational_state=excluded.operational_state, - operational_reason_code=excluded.operational_reason_code, - operational_reason=excluded.operational_reason, - operational_evidence_json=excluded.operational_evidence_json, - activated_at=excluded.activated_at, - last_meaningful_activity_at=excluded.last_meaningful_activity_at, - inactive_days=excluded.inactive_days, - active_days_30d=excluded.active_days_30d, - successful_backtests_30d=excluded.successful_backtests_30d, - calculated_at=excluded.calculated_at - """ - placeholders = ", ".join(["%s" if self.is_postgres else "?"] * len(payload)) - sql = f""" - INSERT INTO user_analytics_snapshots ({columns}) - VALUES ({placeholders}) - ON CONFLICT(user_id) DO UPDATE SET {updates} - """ - with self.base_store._get_connection() as conn: - if self.is_postgres: - with conn.cursor() as cur: - cur.execute(sql, payload) - else: - conn.execute(sql, payload) - return legacy, value - - def get_snapshot(self, user_id: int) -> UserAnalyticsSnapshot | None: - subject_id = positive_user_id(user_id) - if self.is_postgres: - with self.base_store._get_connection() as conn: - with conn.cursor() as cur: - cur.execute( - "SELECT * FROM user_analytics_snapshots WHERE user_id = %s", - (subject_id,), - ) - row = cur.fetchone() - else: - with self.base_store._get_connection() as conn: - row = conn.execute( - "SELECT * FROM user_analytics_snapshots WHERE user_id = ?", - (subject_id,), - ).fetchone() - if row is None: - return None - return UserAnalyticsSnapshot( - user_id=int(row["user_id"]), - status=str(row["status"]), - reason_code=str(row["reason_code"]), - human_readable_reason=str(row["human_readable_reason"]), - evidence_event_ids=list(json.loads(str(row["evidence_event_ids_json"]))), - calculated_at=_parse_timestamp(row["calculated_at"]), - ) - - def list_stale_user_ids( - self, - *, - before: datetime | None = None, - now: datetime | None = None, - limit: int, - include_time_transitions: bool = False, - ) -> list[int]: - page_size = positive_limit(limit) - if before is None and now is None: - raise ValueError("before or now is required") - current = _require_utc(now, "now") if now is not None else None - stale_before = before or (current - timedelta(minutes=15)) - cutoff = utc_iso(_require_utc(stale_before, "before")) - transition_sql = "" - transition_params: list[str] = [] - if include_time_transitions: - if current is None: - raise ValueError("now is required for time transitions") - day_start = datetime.combine(current.date(), time.min, tzinfo=timezone.utc) - placeholder = "%s" if self.is_postgres else "?" - transition_sql = f""" - OR snapshots.lifecycle_segment IS NULL - OR snapshots.calculated_at < {placeholder} - OR NOT EXISTS ( - SELECT 1 - FROM user_lifecycle_daily_snapshots AS daily - WHERE daily.user_id = users.id - AND daily.snapshot_date = {placeholder} - ) - """ - transition_params = [utc_iso(day_start), current.date().isoformat()] - if self.is_postgres: - with self.base_store._get_connection() as conn: - with conn.cursor() as cur: - cur.execute( - f""" - SELECT users.id - FROM users - LEFT JOIN user_analytics_snapshots AS snapshots - ON snapshots.user_id = users.id - LEFT JOIN analytics_subject_settings AS settings - ON settings.user_id = users.id - WHERE users.role <> 'admin' - AND COALESCE(settings.excluded, FALSE) = FALSE - AND ( - snapshots.user_id IS NULL - OR snapshots.calculated_at < %s - {transition_sql} - ) - ORDER BY users.id - LIMIT %s - """, - (cutoff, *transition_params, page_size), - ) - rows = cur.fetchall() - else: - with self.base_store._get_connection() as conn: - rows = conn.execute( - f""" - SELECT users.id - FROM users - LEFT JOIN user_analytics_snapshots AS snapshots - ON snapshots.user_id = users.id - LEFT JOIN analytics_subject_settings AS settings - ON settings.user_id = users.id - WHERE users.role <> 'admin' - AND COALESCE(settings.excluded, 0) = 0 - AND ( - snapshots.user_id IS NULL - OR snapshots.calculated_at < ? - {transition_sql} - ) - ORDER BY users.id - LIMIT ? - """, - (cutoff, *transition_params, page_size), - ).fetchall() - return [int(row["id"]) for row in rows] - - -def _ordered_evidence(events: list[AnalyticsEventRecord]) -> list[str]: - by_id = sorted(events, key=lambda event: event.event_id) - ordered = sorted( - by_id, - key=lambda event: event.occurred_at.astimezone(timezone.utc), - reverse=True, - ) - return [event.event_id for event in ordered[:5]] - - -def _snapshot( - *, - user_id: int, - status: str, - reason_code: str, - evidence: list[AnalyticsEventRecord], - now: datetime, -) -> UserAnalyticsSnapshot: - return UserAnalyticsSnapshot( - user_id=user_id, - status=status, - reason_code=reason_code, - human_readable_reason=_REASONS[reason_code], - evidence_event_ids=_ordered_evidence(evidence), - calculated_at=now, - ) - - -def calculate_user_state( - user_id: int, - *, - now: datetime | None = None, - store: AnalyticsStateStore | None = None, -) -> UserAnalyticsSnapshot: - subject_id = positive_user_id(user_id) - current = now or datetime.now(timezone.utc) - if current.tzinfo is None or current.utcoffset() is None: - raise ValueError("now must include a timezone") - current = current.astimezone(timezone.utc) - state_store = store or AnalyticsStateStore() - user = state_store.get_user(subject_id) - if user is None: - raise LookupError("Analytics user was not found") - events = state_store.list_user_events(subject_id, now=current) - events_desc = sorted(events, key=lambda event: event.occurred_at, reverse=True) - attempted_core = any( - event.event_group == "run" - or ( - event.event_name == "safe_error_recorded" - and event.source_record_type == "run" - ) - for event in events - ) - - blocking_categories = { - "credential_missing", - "provider_unavailable", - "credits_unavailable", - # Split out of credits_unavailable; an empty balance still blocks the - # platform lane, so it keeps the operational state it always had. - "credits_insufficient", - # A restricted account (llm_overage, refund review) can run nothing on - # the platform lane -- strictly worse than an empty balance, so it - # must not read healthier than one. - "account_restricted", - "model_not_allowed", - } - resolvers = { - "credential_verified", - "credential_reverified", - "credential_defaulted", - "credits_settled", - "backtest_completed", - } - latest_resolver = next( - (event for event in events_desc if event.event_name in resolvers), - None, - ) - blocker = next( - ( - event - for event in events_desc - if event.event_name == "safe_error_recorded" - and event.error_category in blocking_categories - ), - None, - ) - if ( - attempted_core - and blocker is not None - and ( - latest_resolver is None or blocker.occurred_at > latest_resolver.occurred_at - ) - ): - reason = ( - "provider_disabled" - if blocker.error_category in {"provider_unavailable", "model_not_allowed"} - else "billing_lane_unavailable" - ) - return _snapshot( - user_id=subject_id, - status="blocked", - reason_code=reason, - evidence=[blocker], - now=current, - ) - - invalid_credential = next( - ( - event - for event in events_desc - if event.event_name == "safe_error_recorded" - and event.error_category == "credential_invalid" - ), - None, - ) - credential_resolver = next( - ( - event - for event in events_desc - if event.event_name - in {"credential_verified", "credential_reverified", "credential_defaulted"} - ), - None, - ) - if invalid_credential is not None and ( - credential_resolver is None - or invalid_credential.occurred_at > credential_resolver.occurred_at - ): - return _snapshot( - user_id=subject_id, - status="needs_attention", - reason_code="invalid_default_credential", - evidence=[invalid_credential], - now=current, - ) - - terminal_24h = [ - event - for event in events_desc - if event.event_name - in {"backtest_completed", "backtest_failed", "backtest_cancelled"} - and current - timedelta(hours=24) <= event.occurred_at <= current - ] - newest_three = terminal_24h[:3] - if len(newest_three) == 3 and all( - event.event_name == "backtest_failed" for event in newest_three - ): - return _snapshot( - user_id=subject_id, - status="needs_attention", - reason_code="three_consecutive_failed_runs", - evidence=newest_three, - now=current, - ) - - meaningful = [event for event in events_desc if is_meaningful_event(event)] - last_meaningful = meaningful[0] if meaningful else None - created_at = _parse_timestamp(user["created_at"]) - if ( - last_meaningful is not None - and last_meaningful.occurred_at < current - timedelta(days=30) - ) or (last_meaningful is None and created_at < current - timedelta(days=30)): - return _snapshot( - user_id=subject_id, - status="dormant", - reason_code="no_meaningful_activity_30d", - evidence=([last_meaningful] if last_meaningful else []), - now=current, - ) - - successes = [ - event for event in events_desc if event.event_name == "backtest_completed" - ] - if not successes: - signup = next( - (event for event in events_desc if event.event_name == "account_signed_up"), - None, - ) - return _snapshot( - user_id=subject_id, - status="onboarding", - reason_code="no_successful_run", - evidence=([signup] if signup else []), - now=current, - ) - - return _snapshot( - user_id=subject_id, - status="active", - reason_code="recent_successful_run_and_activity", - evidence=[successes[0]] + ([last_meaningful] if last_meaningful else []), - now=current, - ) - - -def _require_utc(value: datetime, field_name: str) -> datetime: - if value.tzinfo is None or value.utcoffset() is None: - raise ValueError(f"{field_name} must include a timezone") - return value.astimezone(timezone.utc) - - -def _calculate_user_value_snapshot( - user_id: int, - *, - now: datetime, - state_store: AnalyticsStateStore, - value_store, -) -> UserValueSnapshot: - subject_id = positive_user_id(user_id) - user = state_store.get_user(subject_id) - if user is None: - raise LookupError("Analytics user was not found") - - events = [ - event - for event in state_store.list_user_events(subject_id, now=now) - if event.occurred_at.astimezone(timezone.utc) <= now - ] - activity_start = now - timedelta(days=180) - credit_activity = tuple( - timestamp.astimezone(timezone.utc) - for timestamp in value_store.list_credit_activity( - [subject_id], - start=activity_start, - end=now + timedelta(microseconds=1), - ).get(subject_id, ()) - if timestamp.astimezone(timezone.utc) <= now - ) - previous = value_store.get_current_snapshot(subject_id) - - successes = [ - event.occurred_at.astimezone(timezone.utc) - for event in events - if event.event_name == "backtest_completed" - ] - activation_candidates = list(successes) - if ( - previous is not None - and previous.activated_at is not None - and previous.activated_at <= now - ): - activation_candidates.append(previous.activated_at) - activated_at = min(activation_candidates) if activation_candidates else None - - meaningful_activity = [ - event.occurred_at.astimezone(timezone.utc) - for event in events - if is_lifecycle_activity(event) - ] - meaningful_activity.extend(credit_activity) - if ( - previous is not None - and previous.last_meaningful_activity_at is not None - and previous.last_meaningful_activity_at <= now - ): - meaningful_activity.append(previous.last_meaningful_activity_at) - last_meaningful = max(meaningful_activity) if meaningful_activity else None - - window_start = datetime.combine( - now.date() - timedelta(days=29), - time.min, - tzinfo=timezone.utc, - ) - active_days = { - timestamp.date() - for timestamp in meaningful_activity - if window_start <= timestamp <= now - } - successful_backtests = sum( - 1 for timestamp in successes if window_start <= timestamp <= now - ) - lifecycle = calculate_lifecycle( - LifecycleInputs( - user_id=subject_id, - created_at=_parse_timestamp(user["created_at"]), - first_successful_backtest_at=activated_at, - last_meaningful_activity_at=last_meaningful, - active_days_30d=len(active_days), - successful_backtests_30d=successful_backtests, - ), - now, - ) - facts = value_store.get_operational_facts(subject_id, now=now) - operational = calculate_operational_state( - OperationalSignals( - user_id=subject_id, - account_restricted=facts.account_restricted, - usable_billing_lane=facts.usable_billing_lane, - selected_provider_enabled=facts.selected_provider_enabled, - default_credential_status=facts.default_credential_status, - failed_terminal_runs_24h=facts.failed_terminal_runs_24h, - run_beyond_safe_deadline=facts.run_beyond_safe_deadline, - ), - now, - ) - return UserValueSnapshot( - user_id=subject_id, - lifecycle_segment=lifecycle.segment, - lifecycle_reason_code=lifecycle.reason_code, - lifecycle_reason=lifecycle.reason, - lifecycle_evidence=lifecycle.evidence, - operational_state=operational.state, - operational_reason_code=operational.reason_code, - operational_reason=operational.reason, - operational_evidence=operational.evidence, - activated_at=lifecycle.activated_at, - last_meaningful_activity_at=lifecycle.last_meaningful_activity_at, - inactive_days=lifecycle.inactive_days, - active_days_30d=lifecycle.active_days_30d, - successful_backtests_30d=lifecycle.successful_backtests_30d, - calculated_at=now, - ) - - -def calculate_user_value_snapshot( - user_id: int, - *, - now: datetime | None = None, - state_store: AnalyticsStateStore | None = None, - value_store=None, -) -> UserValueSnapshot: - current = _require_utc(now or datetime.now(timezone.utc), "now") - states = state_store or AnalyticsStateStore() - values = value_store or build_value_analytics_store(states.base_store) - snapshot = _calculate_user_value_snapshot( - user_id, - now=current, - state_store=states, - value_store=values, - ) - return values.upsert_current_snapshot(snapshot) - - -def recalculate_user_snapshots( - user_id: int, - *, - now: datetime | None = None, - state_store: AnalyticsStateStore | None = None, - value_store=None, -) -> tuple[UserAnalyticsSnapshot, UserValueSnapshot]: - """Recalculate legacy and dual-axis projections from the same evidence.""" - - current = _require_utc(now or datetime.now(timezone.utc), "now") - states = state_store or AnalyticsStateStore() - values = value_store or build_value_analytics_store(states.base_store) - legacy = calculate_user_state(user_id, now=current, store=states) - value = _calculate_user_value_snapshot( - user_id, - now=current, - state_store=states, - value_store=values, - ) - if getattr(values, "analytics_base", None) is states.base_store: - return states.upsert_combined_snapshot(legacy, value) - states.upsert_snapshot(legacy) - values.upsert_current_snapshot(value) - return legacy, value - - -def recalculate_user_snapshot( - user_id: int, - *, - now: datetime | None = None, - store: AnalyticsStateStore | None = None, -) -> UserAnalyticsSnapshot: - state_store = store or AnalyticsStateStore() - snapshot = calculate_user_state(user_id, now=now, store=state_store) - return state_store.upsert_snapshot(snapshot) - - -def repair_stale_snapshots( - *, - now: datetime | None = None, - limit: int = 100, - stale_after: timedelta = timedelta(hours=24), - store: AnalyticsStateStore | None = None, -) -> int: - current = now or datetime.now(timezone.utc) - if current.tzinfo is None or current.utcoffset() is None: - raise ValueError("now must include a timezone") - state_store = store or AnalyticsStateStore() - user_ids = state_store.list_stale_user_ids( - before=current - stale_after, - limit=limit, - ) - repaired = 0 - for user_id in user_ids: - try: - recalculate_user_snapshot(user_id, now=current, store=state_store) - repaired += 1 - except Exception as exc: - print( - "WARNING: analytics.snapshot_repair_failed " - f"category={type(exc).__name__[:80]}" - ) - return repaired - - -def repair_stale_value_snapshots( - *, - now: datetime | None = None, - limit: int = 100, - state_store: AnalyticsStateStore | None = None, - value_store=None, -) -> int: - current = _require_utc(now or datetime.now(timezone.utc), "now") - states = state_store or AnalyticsStateStore() - values = value_store or build_value_analytics_store(states.base_store) - user_ids = states.list_stale_user_ids( - now=current, - limit=limit, - include_time_transitions=True, - ) - repaired = 0 - for user_id in user_ids: - try: - _legacy, snapshot = recalculate_user_snapshots( - user_id, - now=current, - state_store=states, - value_store=values, - ) - values.upsert_daily_snapshot( - UserLifecycleDailySnapshot( - snapshot_date=current.date(), - user_id=user_id, - lifecycle_segment=snapshot.lifecycle_segment, - lifecycle_reason_code=snapshot.lifecycle_reason_code, - data_quality="complete", - calculated_at=current, - ) - ) - repaired += 1 - except Exception as exc: - print( - "WARNING: analytics.value_snapshot_repair_failed " - f"category={type(exc).__name__[:80]}" - ) - return repaired - - -__all__ = [ - "AnalyticsStateStore", - "UserAnalyticsSnapshot", - "calculate_user_value_snapshot", - "calculate_user_state", - "recalculate_user_snapshot", - "recalculate_user_snapshots", - "repair_stale_snapshots", - "repair_stale_value_snapshots", -] diff --git a/dashboard/backend/domain/analytics/value_queries.py b/dashboard/backend/domain/analytics/value_queries.py index ba4ba0288..91bee5fba 100644 --- a/dashboard/backend/domain/analytics/value_queries.py +++ b/dashboard/backend/domain/analytics/value_queries.py @@ -14,22 +14,29 @@ LifecycleSegment, OperationalResult, OperationalState, - is_lifecycle_activity, + activation_cohort_week, + calculate_lifecycle, + operational_reason, + operational_result_from_code, ) +from .lifecycle_reads import build_lifecycle_inputs from .metrics import AnalyticsMetricFilters from .query_service import ( AnalyticsQueryService, AnalyticsQueryStore, AnalyticsUserProfile, FailureCategoryCount, + _warn_read_failed, + landed_fact_day, ) from .repository import analytics_store from .repository_common import positive_limit, positive_user_id -from .usage_cost import atl_cost_micro from .value_repository import ( + MAX_USER_BATCH, CommercialValueFact, - UserLifecycleDailySnapshot, - UserValueSnapshot, + GroupFactTotals, + UserActivity, + UserPageQuery, build_value_analytics_store, ) from dashboard.backend.domain.user_groups import ( @@ -61,7 +68,6 @@ "invested", "high_value", ) -_TIER_RANK = {tier: rank for rank, tier in enumerate(_COMMERCIAL_TIERS)} _MOVEMENT_WINDOWS: dict[str, tuple[int, Literal["day", "week", "month"]]] = { "5d": (5, "day"), "1w": (7, "day"), @@ -79,23 +85,14 @@ _MAX_HISTORY_SCAN_DAYS = max( MAX_VALUE_RANGE_DAYS, *(days for days, _granularity in _MOVEMENT_WINDOWS.values()) ) -"""Widest span of per-user daily rows one lifecycle request may scan. +"""Widest span of daily fact and rollup rows one lifecycle request may read. `MAX_VALUE_RANGE_DAYS` bounds the *filter* range, but the movement chart reads a window of its own, so the scan is the wider of the two -- 365 days today, double the filter cap. Derived rather than written down so that adding a longer -movement range cannot silently widen every user's scan. +movement range cannot silently widen the scan. """ -_PRIORITY_RANK = { - "blocked": 0, - "needs_attention": 1, - "healthy_at_risk": 2, - "healthy_onboarding": 3, - "none": 4, -} - - def _utc(value: datetime, field_name: str) -> datetime: if value.tzinfo is None or value.utcoffset() is None: raise ValueError(f"{field_name} must include a timezone") @@ -118,13 +115,6 @@ def _period_start(value: date, granularity: Literal["day", "week", "month"]) -> return value.replace(day=1) -def _parse_timestamp(value: object) -> datetime: - parsed = datetime.fromisoformat(str(value)) - if parsed.tzinfo is None or parsed.utcoffset() is None: - parsed = parsed.replace(tzinfo=UTC) - return parsed.astimezone(UTC) - - def _validate_dates(start: date, end: date) -> tuple[date, date]: if not isinstance(start, date) or isinstance(start, datetime): raise ValueError("start must be a date") @@ -297,6 +287,20 @@ class OperationalAnalyticsResponse(BaseModel): availability: SectionAvailability +class OperationalReasonCount(BaseModel): + """One blocking or attention reason and how many users carried it. + + Service-level only in PR B (design §9: PR D puts `top_operational_reasons` + on OperationalAnalyticsResponse in the contract re-cut). + """ + + model_config = ConfigDict(extra="forbid", frozen=True) + + reason_code: str + reason: str + users: int = Field(ge=0) + + class UserGroupSummary(BaseModel): """Display-safe value metrics for one canonical account-source group.""" @@ -335,7 +339,6 @@ class UserValueFilters(BaseModel): last_meaningful_activity_from: datetime | None = None last_meaningful_activity_to: datetime | None = None priority: bool = False - legacy_status: str | None = None include_internal: bool = False @field_validator("q") @@ -362,15 +365,6 @@ def validate_activity_range(self) -> "UserValueFilters": and self.last_meaningful_activity_to < self.last_meaningful_activity_from ): raise ValueError("last meaningful activity range is reversed") - legacy = self.legacy_status - if legacy is not None and legacy not in { - "blocked", - "needs_attention", - "dormant", - "onboarding", - "active", - }: - raise ValueError("legacy_status is unsupported") return self @@ -416,39 +410,18 @@ class ValueUserProfile(AnalyticsUserProfile): recent_lifecycle_transitions: Sequence[LifecycleTransition] -def _lifecycle(snapshot: UserValueSnapshot) -> LifecycleResult: - return LifecycleResult( - segment=snapshot.lifecycle_segment, - reason_code=snapshot.lifecycle_reason_code, - reason=snapshot.lifecycle_reason, - evidence=tuple(snapshot.lifecycle_evidence), - activated_at=snapshot.activated_at, - last_meaningful_activity_at=snapshot.last_meaningful_activity_at, - inactive_days=snapshot.inactive_days, - active_days_30d=snapshot.active_days_30d, - successful_backtests_30d=snapshot.successful_backtests_30d, - calculated_at=snapshot.calculated_at, - ) - - -def _operational(snapshot: UserValueSnapshot) -> OperationalResult: - return OperationalResult( - state=snapshot.operational_state, - reason_code=snapshot.operational_reason_code, - reason=snapshot.operational_reason, - evidence=tuple(snapshot.operational_evidence), - calculated_at=snapshot.calculated_at, - ) - - -def _priority_group(snapshot: UserValueSnapshot) -> PriorityGroup: - if snapshot.operational_state == "blocked": +def _priority_group( + operational_state: str, lifecycle_segment: str | None +) -> PriorityGroup: + """Yesterday's fact values -> group; the same rule as ``_users_page_sql``'s + priority predicate and sort, so a row agrees with where SQL placed it.""" + if operational_state == "blocked": return "blocked" - if snapshot.operational_state == "needs_attention": + if operational_state == "needs_attention": return "needs_attention" - if snapshot.lifecycle_segment == "at_risk": + if lifecycle_segment == "at_risk": return "healthy_at_risk" - if snapshot.lifecycle_segment == "onboarding": + if lifecycle_segment == "onboarding": return "healthy_onboarding" return "none" @@ -464,33 +437,30 @@ def _all_users(user_store: Any) -> list[dict[str, Any]]: offset += len(page) -def _event_utc(value: object) -> datetime: - """Normalize an event timestamp without trusting a naive stored value.""" - - if isinstance(value, datetime): - if value.tzinfo is None or value.utcoffset() is None: - return value.replace(tzinfo=UTC) - return value.astimezone(UTC) - return _parse_timestamp(value) - - -def _safe_cost_micro_usd(event: object) -> int: - """Read only the bounded platform-cost field used by the summary. - - Server analytics events are validated on write, but this endpoint also - reads legacy rows. A malformed or negative value must not make an admin - summary fail or turn into a negative cost. - """ - - if getattr(event, "event_name", None) != "model_usage_recorded": - return 0 - properties = getattr(event, "properties", {}) - if not isinstance(properties, dict): - return 0 - value = properties.get("cost_micro_usd") - if isinstance(value, bool) or not isinstance(value, int) or value < 0: - return 0 - return atl_cost_micro(getattr(event, "billing_mode", None), properties) +def _transition_models( + counts: Counter[tuple[str, str]], + partial: set[tuple[str, str]], + *, + start: date, + end: date, +) -> list[LifecycleTransition]: + """Window-level transition totals, largest first, stamped with the window.""" + return [ + LifecycleTransition( + from_segment=from_segment, + to_segment=to_segment, + users=count, + period_start=start, + period_end=end, + data_quality=( + "partial" if (from_segment, to_segment) in partial else "complete" + ), + ) + for (from_segment, to_segment), count in sorted( + counts.items(), + key=lambda item: (-item[1], item[0]), + ) + ] class ValueAnalyticsQueryService: @@ -511,9 +481,18 @@ def __init__( self.user_store = user_store self.value_store = value_store or build_value_analytics_store(store) self.query_store = query_store or AnalyticsQueryStore(store) + # A defaulted legacy service shares this service's value store, so the + # profile's axes and fact sums read the stores this service was given. self.legacy_service = legacy_service or AnalyticsQueryService( store=store, user_store=user_store, + value_store=self.value_store, + ) + self._stale_fact_warnings: set[tuple[date, date | None]] = set() + + def _landed_fact_day(self, yesterday: date) -> date | None: + return landed_fact_day( + self.value_store, yesterday=yesterday, warned=self._stale_fact_warnings ) def _eligible_users(self, *, include_internal: bool) -> list[dict[str, Any]]: @@ -527,18 +506,6 @@ def _eligible_users(self, *, include_internal: bool) -> list[dict[str, Any]]: def _ids(users: Sequence[dict[str, Any]]) -> list[int]: return [positive_user_id(user["id"]) for user in users] - def _current( - self, - users: Sequence[dict[str, Any]], - ) -> dict[int, UserValueSnapshot]: - result: dict[int, UserValueSnapshot] = {} - ids = self._ids(users) - for offset in range(0, len(ids), 500): - result.update( - self.value_store.list_current_snapshots(ids[offset : offset + 500]) - ) - return result - def _commercial( self, users: Sequence[dict[str, Any]], @@ -595,31 +562,12 @@ def _ledger_days( ], ) - def _daily( - self, - user_ids: Sequence[int], - *, - start: date, - end: date, - ) -> list[UserLifecycleDailySnapshot]: - rows: list[UserLifecycleDailySnapshot] = [] - for offset in range(0, len(user_ids), 500): - rows.extend( - self.value_store.list_daily_snapshots( - start=start, - end=end, - user_ids=user_ids[offset : offset + 500], - ) - ) - return rows - def _history( self, - user_ids: Sequence[int], *, start: date, end: date, - use_anonymous_rollups: bool, + yesterday: date, movement_start: date | None = None, movement_range: str = "5d", ) -> tuple[ @@ -628,6 +576,15 @@ def _history( list[LifecycleTransition], SectionAvailability, ]: + """Population history from ``user_daily_facts`` and ``lifecycle_transitions``. + + Both are grouped in SQL, so the read does not grow with the user table. + Days the facts table no longer holds fall back to the anonymous + lifecycle rollups -- always: facts hold no internal accounts, exactly + like the rollups, so the internal toggle has nothing to keep out of + the fallback. Facts exist only through ``yesterday``, so coverage is + judged against that day rather than against ``end``. + """ if movement_range not in _MOVEMENT_WINDOWS: raise ValueError("unsupported lifecycle movement range") window_days, granularity = _MOVEMENT_WINDOWS[movement_range] @@ -638,19 +595,20 @@ def _history( min(start, selected_movement_start), end - timedelta(days=_MAX_HISTORY_SCAN_DAYS), ) - rows = self._daily( - user_ids, - start=history_start - timedelta(days=1), - end=end, - ) - by_date: dict[date, list[UserLifecycleDailySnapshot]] = defaultdict(list) - for row in rows: - by_date[row.snapshot_date].append(row) - rollups = ( - self.query_store.rollups.list_rollups(start=history_start, end=end) - if use_anonymous_rollups - else [] - ) + daily_counts: dict[date, dict[LifecycleSegment, int]] = {} + daily_quality: dict[date, str] = {} + for row in self.value_store.count_segments_by_date(start=history_start, end=end): + counts = daily_counts.setdefault( + row.snapshot_date, {segment: 0 for segment in _LIFECYCLE_SEGMENTS} + ) + counts[row.lifecycle_segment] = row.users + if row.partial_rows: + daily_quality[row.snapshot_date] = "partial" + else: + daily_quality.setdefault(row.snapshot_date, "complete") + direct_dates = set(daily_counts) + + rollups = self.query_store.rollups.list_rollups(start=history_start, end=end) rollup_counts: dict[date, dict[LifecycleSegment, int]] = defaultdict(dict) rollup_quality: dict[date, str] = {} for row in rollups: @@ -661,20 +619,6 @@ def _history( rollup_quality[row.rollup_date] = "partial" else: rollup_quality.setdefault(row.rollup_date, "complete") - - daily_counts: dict[date, dict[LifecycleSegment, int]] = {} - daily_quality: dict[date, str] = {} - direct_dates = {day for day in by_date if history_start <= day < end} - for day in direct_dates: - counts = Counter(row.lifecycle_segment for row in by_date[day]) - daily_counts[day] = { - segment: int(counts.get(segment, 0)) for segment in _LIFECYCLE_SEGMENTS - } - daily_quality[day] = ( - "partial" - if any(row.data_quality == "partial" for row in by_date[day]) - else "complete" - ) for day, counts in rollup_counts.items(): if day not in direct_dates: daily_counts[day] = { @@ -686,9 +630,8 @@ def _history( weekly: list[WeeklyLifecycleCount] = [] by_week: dict[date, list[date]] = defaultdict(list) for day in daily_counts: - if not start <= day < end: - continue - by_week[_week_start(day)].append(day) + if start <= day < end: + by_week[_week_start(day)].append(day) for week, dates in sorted(by_week.items()): latest = max(dates) weekly.append( @@ -716,18 +659,10 @@ def _history( transition_counts: Counter[tuple[str, str]] = Counter() transition_partial: set[tuple[str, str]] = set() - by_user_date = {(row.user_id, row.snapshot_date): row for row in rows} - for row in rows: - if not start <= row.snapshot_date < end: - continue - previous = by_user_date.get( - (row.user_id, row.snapshot_date - timedelta(days=1)) - ) - if previous is None or previous.lifecycle_segment == row.lifecycle_segment: - continue - key = (previous.lifecycle_segment, row.lifecycle_segment) - transition_counts[key] += 1 - if previous.data_quality == "partial" or row.data_quality == "partial": + for row in self.value_store.count_transitions(start=start, end=end): + key = (row.from_segment, row.to_segment) + transition_counts[key] += row.users + if row.partial_rows: transition_partial.add(key) for row in rollups: if ( @@ -740,24 +675,9 @@ def _history( transition_counts[key] += row.value_count if row.outcome == "partial": transition_partial.add(key) - transitions = [ - LifecycleTransition( - from_segment=from_segment, - to_segment=to_segment, - users=count, - period_start=start, - period_end=end, - data_quality=( - "partial" - if (from_segment, to_segment) in transition_partial - else "complete" - ), - ) - for (from_segment, to_segment), count in sorted( - transition_counts.items(), - key=lambda item: (-item[1], item[0]), - ) - ] + transitions = _transition_models( + transition_counts, transition_partial, start=start, end=end + ) coverage = sorted(day for day in daily_counts if start <= day < end) if not coverage: availability = SectionAvailability( @@ -768,7 +688,7 @@ def _history( partial = ( any(daily_quality[day] == "partial" for day in coverage) or coverage[0] > start - or coverage[-1] < end - timedelta(days=1) + or coverage[-1] < min(end - timedelta(days=1), yesterday) ) availability = SectionAvailability( available=True, @@ -787,25 +707,56 @@ def get_lifecycle( movement_range: str = "5d", now: datetime | None = None, ) -> LifecycleAnalyticsResponse: + """Segment counts are as of the latest landed fact day (design §6.3, §6.10). + + That is yesterday once the night's job lands; until then, or while it + fails, the day before, labelled ``stale`` and ``partial`` rather than + read as an empty population. Fact rows hold no internal or excluded + accounts, so ``include_internal`` widens only the activation and paid + headlines, which read ``user_activity`` and the ledger. History always + falls back to the anonymous rollups for days without facts. + """ start, end = _validate_dates(start, end) if movement_range not in _MOVEMENT_WINDOWS: raise ValueError("unsupported lifecycle movement range") window_days, _granularity = _MOVEMENT_WINDOWS[movement_range] movement_start = end - timedelta(days=window_days) current_time = _utc(now or datetime.now(UTC), "now") - users = self._eligible_users(include_internal=include_internal) - current = self._current(users) - counts = Counter(snapshot.lifecycle_segment for snapshot in current.values()) + yesterday = current_time.date() - timedelta(days=1) + facts_day = self._landed_fact_day(yesterday) + cells = self.value_store.count_states_for_date(facts_day) if facts_day else [] + stale = bool(cells) and facts_day != yesterday segment_counts = { - segment: int(counts.get(segment, 0)) for segment in _LIFECYCLE_SEGMENTS + segment: sum( + cell.users for cell in cells if cell.lifecycle_segment == segment + ) + for segment in _LIFECYCLE_SEGMENTS } availability: dict[str, SectionAvailability] = { "current": SectionAvailability( - available=bool(current) or not users, - status="ready" if len(current) == len(users) else "building", + available=True, + stale=stale, + status=( + "building" if not cells else "partial" if stale else "ready" + ), + coverage_start=facts_day if cells else None, + coverage_end=facts_day if cells else None, ) } try: + activated_users = self.value_store.count_activated_users( + include_internal=include_internal + ) + except Exception as exc: + _warn_read_failed("lifecycle_activation", exc) + activated_users = 0 + availability["current"] = SectionAvailability( + available=bool(cells), + stale=stale, + status="partial", + ) + try: + users = self._eligible_users(include_internal=include_internal) commercial = self._commercial(users, start=start, end=end) paid = sum( fact.lifetime_net_purchased_micro > 0 for fact in commercial.values() @@ -814,7 +765,8 @@ def get_lifecycle( available=True, status="ready", ) - except Exception: + except Exception as exc: + _warn_read_failed("lifecycle_commercial", exc) paid = 0 availability["commercial"] = SectionAvailability( available=False, @@ -822,14 +774,14 @@ def get_lifecycle( ) try: weekly, movement, transitions, history_availability = self._history( - self._ids(users), start=start, end=end, - use_anonymous_rollups=not include_internal, + yesterday=yesterday, movement_start=movement_start, movement_range=movement_range, ) - except Exception: + except Exception as exc: + _warn_read_failed("lifecycle_history", exc) weekly, movement, transitions = [], [], [] history_availability = SectionAvailability( available=False, @@ -839,9 +791,7 @@ def get_lifecycle( return LifecycleAnalyticsResponse( as_of=current_time, headline=LifecycleHeadline( - activated_users=sum( - snapshot.activated_at is not None for snapshot in current.values() - ), + activated_users=activated_users, core_users=segment_counts["core"], at_risk_users=segment_counts["at_risk"], paid_users=paid, @@ -855,27 +805,6 @@ def get_lifecycle( availability=availability, ) - def _retention_quality( - self, - rows: Sequence[UserLifecycleDailySnapshot], - *, - user_ids: Sequence[int], - start: date, - end: date, - ) -> Literal["complete", "partial"]: - selected = [row for row in rows if start <= row.snapshot_date < end] - expected = { - (user_id, start + timedelta(days=offset)) - for user_id in user_ids - for offset in range((end - start).days) - } - observed = {(row.user_id, row.snapshot_date) for row in selected} - if expected - observed or any( - row.data_quality == "partial" for row in selected - ): - return "partial" - return "complete" - @staticmethod def _summary_cell( week: Literal[1, 2, 4], @@ -907,52 +836,69 @@ def get_retention( ) -> RetentionAnalyticsResponse: start, end = _validate_dates(start, end) current_time = _utc(now or datetime.now(UTC), "now") - users = self._eligible_users(include_internal=include_internal) - current = self._current(users) cohort_start = _week_start(start) + # Every Monday below ``end`` is a cohort week, so the last one is the + # Monday of the window's final day; its members activated before the + # next Monday. + # Cohorts are always the facts-eligible population: user_daily_facts + # holds no internal or excluded account, so admitting one under + # ``include_internal`` would count it eligible and never retained, + # deflating every rate it touched. The flag is accepted and ignored. + activations = self.value_store.list_activations( + start=_day_start(cohort_start), + end=_day_start(_week_start(end - timedelta(days=1)) + timedelta(days=7)), + include_internal=False, + ) cohort_users: dict[date, list[int]] = defaultdict(list) - for user_id, snapshot in current.items(): - if snapshot.activated_at is None: - continue - week = _week_start(snapshot.activated_at.date()) + for user_id, activated_at in activations.items(): + week = activation_cohort_week(activated_at) if cohort_start <= week < end: cohort_users[week].append(user_id) ids = sorted( - {user_id for values in cohort_users.values() for user_id in values} + {user_id for members in cohort_users.values() for user_id in members} ) - activity_by_user: dict[int, list[datetime]] = defaultdict(list) - if ids: - activity_end = min(_day_start(end + timedelta(days=35)), current_time) - events = self.query_store.rollups.list_events( - start=_day_start(cohort_start), - end=activity_end + timedelta(microseconds=1), - include_internal=True, - ) - id_set = set(ids) - for event in events: - if event.user_id in id_set and is_lifecycle_activity(event): - activity_by_user[event.user_id].append(event.occurred_at) - for offset in range(0, len(ids), 500): - chunk = ids[offset : offset + 500] - credit_activity = self.value_store.list_credit_activity( - chunk, - start=_day_start(cohort_start), - end=activity_end + timedelta(microseconds=1), - ) - for user_id, timestamps in credit_activity.items(): - activity_by_user[user_id].extend(timestamps) - daily_end = min( + grid_end = min( end + timedelta(days=35), current_time.date() + timedelta(days=1) ) - daily = ( - self._daily( - ids, - start=cohort_start, - end=daily_end, - ) - if ids - else [] - ) + active_dates: dict[int, set[date]] = defaultdict(set) + day_quality: dict[date, str] = {} + if ids: + # list_active_dates chunks by MAX_USER_BATCH itself. + for user_id, days in self.value_store.list_active_dates( + ids, start=cohort_start, end=grid_end + ).items(): + active_dates[user_id].update(days) + # A ledger movement (a purchase, a grant, a debit) retains a user + # as it did on main, which unioned list_credit_activity into the + # event activity. The fact ``active`` flag is set from events + # alone, so without this a cohort member whose only week-2 + # activity was a Credits purchase read as churned. + # list_credit_activity takes one batch at most. + for offset in range(0, len(ids), MAX_USER_BATCH): + for user_id, stamps in self.value_store.list_credit_activity( + ids[offset : offset + MAX_USER_BATCH], + start=_day_start(cohort_start), + end=_day_start(grid_end), + ).items(): + active_dates[user_id].update(stamp.date() for stamp in stamps) + # A day is trustworthy only when it has fact rows and none is partial. + for row in self.value_store.count_segments_by_date( + start=cohort_start, end=grid_end + ): + if row.partial_rows: + day_quality[row.snapshot_date] = "partial" + else: + day_quality.setdefault(row.snapshot_date, "complete") + + def week_quality( + target_start: date, target_end: date + ) -> Literal["complete", "partial"]: + day = target_start + while day < target_end: + if day_quality.get(day) != "complete": + return "partial" + day += timedelta(days=1) + return "complete" cohorts: list[RetentionCohort] = [] all_cells: dict[int, list[RetentionCell]] = {1: [], 2: [], 4: []} @@ -962,12 +908,6 @@ def get_retention( target_start = cohort_week + timedelta(days=week * 7) target_end = target_start + timedelta(days=7) mature = current_time >= _day_start(target_end) - quality = self._retention_quality( - daily, - user_ids=members, - start=target_start, - end=target_end, - ) retained = None eligible = None rate = None @@ -975,10 +915,8 @@ def get_retention( eligible = len(members) retained = sum( any( - _day_start(target_start) - <= timestamp - < _day_start(target_end) - for timestamp in activity_by_user.get(user_id, ()) + target_start <= day < target_end + for day in active_dates.get(user_id, ()) ) for user_id in members ) @@ -989,7 +927,7 @@ def get_retention( retained_users=retained, eligible_users=eligible, rate=rate, - data_quality=quality, + data_quality=week_quality(target_start, target_end), ) cells[week] = cell all_cells[week].append(cell) @@ -1015,12 +953,8 @@ def get_retention( summary_week_2=self._summary_cell(2, all_cells[2]), summary_week_4=self._summary_cell(4, all_cells[4]), availability=SectionAvailability( - available=bool(current) or not users, - status=( - "partial" - if partial - else "ready" if len(current) == len(users) else "building" - ), + available=True, + status="partial" if partial else "ready", coverage_start=cohort_start if cohorts else None, coverage_end=end - timedelta(days=1) if cohorts else None, ), @@ -1115,9 +1049,18 @@ def get_operational( ) -> OperationalAnalyticsResponse: start, end = _validate_dates(start, end) current_time = _utc(now or datetime.now(UTC), "now") - users = self._eligible_users(include_internal=include_internal) - current = self._current(users) - counts = Counter(snapshot.operational_state for snapshot in current.values()) + yesterday = current_time.date() - timedelta(days=1) + # Fact rows hold no internal or excluded accounts, so ``include_internal`` + # widens only the overview figures below. State counts are as of the + # latest landed fact day -- the same day /users falls back to, so the + # two pages agree while the night's job has not landed. + facts_day = self._landed_fact_day(yesterday) + cells = self.value_store.count_states_for_date(facts_day) if facts_day else [] + stale = bool(cells) and facts_day != yesterday + counts = { + state: sum(cell.users for cell in cells if cell.operational_state == state) + for state in _OPERATIONAL_STATES + } overview = self.legacy_service.get_overview( filters=AnalyticsMetricFilters( start=_day_start(start), @@ -1131,19 +1074,17 @@ def get_operational( ) growth_available = overview.availability["growth"].available friction_available = overview.availability["friction"].available - current_complete = len(current) == len(users) - available = bool(current) or not users or growth_available or friction_available - if not current_complete: + available = bool(cells) or growth_available or friction_available + if not cells: + # No fact day has landed: empty counts are not "none blocked". status = "building" - elif growth_available and friction_available: + elif growth_available and friction_available and not stale: status = "ready" else: status = "partial" if available else "unavailable" return OperationalAnalyticsResponse( as_of=current_time, - operational_state_counts={ - state: int(counts.get(state, 0)) for state in _OPERATIONAL_STATES - }, + operational_state_counts=counts, # Unknown whenever the growth read failed, even part-way: the # overview can compute a figure and then fail the block. backtest_success_rate=( @@ -1161,10 +1102,41 @@ def get_operational( top_failure_categories=overview.top_failure_categories, availability=SectionAvailability( available=available, + stale=stale, status=status, + # The day the state counts describe, so a stale fallback says + # how stale it is. + coverage_start=facts_day if cells else None, + coverage_end=facts_day if cells else None, ), ) + def top_operational_reasons( + self, *, day: date, limit: int = 10 + ) -> list[OperationalReasonCount]: + """Why users were blocked or needed attention on ``day`` (design §9). + + One grouped read over the fact rows; the reason text is the rule + table's copy, never the stored code alone. + """ + page_size = positive_limit(limit) + totals: Counter[str] = Counter() + for cell in self.value_store.count_states_for_date(day): + # A NULL code is not a supported reason (it reads as + # no_supported_issue); the row still counts under its state. + if ( + cell.operational_state in {"blocked", "needs_attention"} + and cell.operational_reason_code is not None + ): + totals[cell.operational_reason_code] += cell.users + ranked = sorted(totals.items(), key=lambda item: (-item[1], item[0])) + return [ + OperationalReasonCount( + reason_code=code, reason=operational_reason(code), users=users + ) + for code, users in ranked[:page_size] + ] + def get_groups( self, start: date, @@ -1175,10 +1147,12 @@ def get_groups( ) -> GroupAnalyticsResponse: """Build the fixed-order source summary for one UTC date window. - The account row is the source of truth for group membership. Event and - Credits reads are deliberately batched and independently guarded so a - missing projection leaves six honest rows with partial availability, - rather than dropping the whole summary. + The account row is the source of truth for group membership. Run counts + and operator cost come from one grouped read of ``user_daily_facts``, + which carries the group each day was written under. That read and the + Credits read are independently guarded so a missing projection leaves + six honest rows with partial availability, rather than dropping the + whole summary. """ start, end = _validate_dates(start, end) @@ -1193,7 +1167,8 @@ def get_groups( users_available = True try: all_users = self._eligible_users(include_internal=include_internal) - except Exception: + except Exception as exc: + _warn_read_failed("groups_users", exc) all_users = [] users_available = False @@ -1220,60 +1195,34 @@ def get_groups( users_by_group: Counter[UserGroup] = Counter( group_by_user[user_id] for user_id in user_ids ) - requested_by_group: Counter[UserGroup] = Counter() - successful_users_by_group: dict[UserGroup, set[int]] = defaultdict(set) - completion_days_by_group: dict[ - UserGroup, dict[int, set[date]] - ] = defaultdict(lambda: defaultdict(set)) - cost_by_group: Counter[UserGroup] = Counter() - events_available = True - events: Sequence[Any] = () - period_start = _day_start(start) - period_end = _day_start(end) - try: - list_metric_events = getattr(self.query_store, "list_metric_events", None) - if callable(list_metric_events): - events = list_metric_events( - start=period_start, - end=period_end, - include_internal=include_internal, - ) - else: - # Small synthetic stores used by older callers expose only - # the rollup reader. It is still display-safe; account - # eligibility above remains the authoritative filter. - events = self.query_store.rollups.list_events( - start=period_start, - end=period_end, - include_internal=True, - ) - except Exception: - events_available = False - - for event in events: + # Facts exist only through yesterday, and a past day has them only + # once the night's job lands. Coverage is read, not assumed from the + # calendar: a window holding a day the job has not landed (or failed + # to write) would otherwise read that day as zero runs, "ready". + facts_end = min(end, current_time.date()) + expected_days = { + start + timedelta(days=offset) for offset in range((facts_end - start).days) + } + landed_days: set[date] = set() + facts_available = True + totals: dict[str, GroupFactTotals] = {} + if expected_days: try: - event_user_id = positive_user_id(getattr(event, "user_id")) - if event_user_id not in user_ids: - continue - occurred_at = _event_utc(getattr(event, "occurred_at")) - if not (period_start <= occurred_at < period_end): - continue - group = group_by_user[event_user_id] - event_name = getattr(event, "event_name", None) - except (AttributeError, TypeError, ValueError, KeyError): - # Ignore malformed legacy event rows while preserving a valid - # summary for the rest of the period. - events_available = False - continue - if event_name == "backtest_requested": - requested_by_group[group] += 1 - elif event_name == "backtest_completed": - successful_users_by_group[group].add(event_user_id) - completion_days_by_group[group][event_user_id].add( - occurred_at.date() - ) - elif event_name == "model_usage_recorded": - cost_by_group[group] += _safe_cost_micro_usd(event) + totals = self.value_store.sum_facts_by_user_group(start=start, end=end) + landed_days = { + row.snapshot_date + for row in self.value_store.count_segments_by_date( + start=start, end=facts_end + ) + } + except Exception as exc: + _warn_read_failed("groups_facts", exc) + facts_available = False + landed = bool(landed_days) + if selected_group is not None: + totals = { + group: value for group, value in totals.items() if group == selected_group + } commercial_available = True commercial: dict[int, CommercialValueFact] = {} @@ -1281,7 +1230,8 @@ def get_groups( commercial = self._commercial(user_rows, start=start, end=end) if user_ids and not user_ids.issubset(commercial): commercial_available = False - except Exception: + except Exception as exc: + _warn_read_failed("groups_commercial", exc) commercial_available = False paid_by_group: Counter[UserGroup] = Counter() @@ -1293,20 +1243,16 @@ def get_groups( rows: list[UserGroupSummary] = [] for group in USER_GROUPS: - successful_users = successful_users_by_group[group] - repeat_users = sum( - len(days) >= 2 - for days in completion_days_by_group[group].values() - ) + group_totals = totals.get(group, GroupFactTotals()) rows.append( UserGroupSummary( group=group, label=USER_GROUP_LABELS[group], users=int(users_by_group[group]), - successful_run_users=len(successful_users), - repeat_users=repeat_users, - total_runs=int(requested_by_group[group]), - atl_cost_micro_usd=int(cost_by_group[group]), + successful_run_users=group_totals.successful_run_users, + repeat_users=group_totals.repeat_users, + total_runs=group_totals.total_runs, + atl_cost_micro_usd=group_totals.operator_cost_micro, paid_users=int(paid_by_group[group]), ) ) @@ -1314,12 +1260,22 @@ def get_groups( if not users_available: availability = SectionAvailability(available=False, status="unavailable") else: - complete = events_available and commercial_available + # A partial fact row in the window (a failed source step, or the + # PR A history copy's zero-filled days) makes its zeros unknowns, + # and so does a past day with no fact rows at all. + facts_partial = any(value.partial_rows for value in totals.values()) + complete = ( + landed + and landed_days == expected_days + and facts_available + and commercial_available + and not facts_partial + ) availability = SectionAvailability( available=True, status="ready" if complete else "partial", - coverage_start=start, - coverage_end=end - timedelta(days=1), + coverage_start=min(landed_days) if landed else None, + coverage_end=max(landed_days) if landed else None, ) return GroupAnalyticsResponse( as_of=current_time, @@ -1342,88 +1298,92 @@ def list_users( if isinstance(offset, bool) or not isinstance(offset, int) or offset < 0: raise ValueError("offset must be a non-negative integer") current_time = _utc(now or datetime.now(UTC), "now") - users = self._eligible_users(include_internal=filters.include_internal) - current = self._current(users) - commercial = self._commercial( - users, - start=current_time.date() - timedelta(days=30), - end=current_time.date() + timedelta(days=1), + yesterday = current_time.date() - timedelta(days=1) + query = UserPageQuery( + q=filters.q, + lifecycle_segment=filters.lifecycle_segment, + operational_state=filters.operational_state, + commercial_tier=filters.commercial_tier, + user_group=filters.user_group, + activated=filters.activated, + last_meaningful_activity_from=filters.last_meaningful_activity_from, + last_meaningful_activity_to=filters.last_meaningful_activity_to, + priority=filters.priority, + include_internal=filters.include_internal, ) - legacy = self.query_store.list_snapshots() if filters.legacy_status else {} - selected: list[ValueUserListItem] = [] - needle = filters.q.casefold() if filters.q else None - for user in users: - user_id = int(user["id"]) - snapshot = current.get(user_id) - fact = commercial.get(user_id) - if snapshot is None or fact is None: - continue - user_group = coerce_user_group(user.get("user_group")) - if filters.user_group is not None and user_group != filters.user_group: - continue - group = _priority_group(snapshot) - if filters.priority and group == "none": - continue - if filters.lifecycle_segment not in {None, snapshot.lifecycle_segment}: - continue - if filters.operational_state not in {None, snapshot.operational_state}: - continue - if filters.commercial_tier not in {None, fact.commercial_tier}: - continue - if filters.activated is not None: - if (snapshot.activated_at is not None) != filters.activated: - continue - if filters.last_meaningful_activity_from is not None and ( - snapshot.last_meaningful_activity_at is None - or snapshot.last_meaningful_activity_at - < filters.last_meaningful_activity_from - ): - continue - if filters.last_meaningful_activity_to is not None and ( - snapshot.last_meaningful_activity_at is None - or snapshot.last_meaningful_activity_at - > filters.last_meaningful_activity_to - ): - continue - if filters.legacy_status is not None and ( - user_id not in legacy or legacy[user_id].status != filters.legacy_status - ): - continue - if needle is not None and needle not in ( - f"{user.get('display_name', '')} {user.get('email', '')}".casefold() - ): - continue - selected.append( - ValueUserListItem( - user_id=user_id, - display_name=str(user.get("display_name") or ""), - email=str(user.get("email") or ""), - joined_at=_parse_timestamp(user["created_at"]), - lifecycle=_lifecycle(snapshot), - operational=_operational(snapshot), - commercial_tier=fact.commercial_tier, - lifetime_net_purchased_micro=fact.lifetime_net_purchased_micro, - priority_group=group, - profile_path=f"/admin/analytics/users/{user_id}", + # The page joins yesterday's facts. Before the night's job lands, or + # while it is failing, that join is empty and every row would read + # healthy / unpaid with no signal: join the latest landed day instead, + # stamp the operational as_of from it, and say so in the log (once + # per stale day, not once per page load). + facts_day = self._landed_fact_day(yesterday) or yesterday + rows, total = self.value_store.list_users_page( + query=query, day=facts_day, limit=page_size, offset=offset + ) + ids = [row.user_id for row in rows] + totals = ( + self.value_store.sum_recent_facts( + ids, start=yesterday - timedelta(days=29), end=yesterday + ) + if ids + else {} + ) + commercial = ( + self.value_store.list_commercial_values( + ids, + start=_day_start(yesterday), + end=_day_start(yesterday + timedelta(days=1)), + ) + if ids + else {} + ) + as_of = _day_start(facts_day + timedelta(days=1)) + items: list[ValueUserListItem] = [] + for row in rows: + activity = ( + UserActivity( + user_id=row.user_id, + activated_at=row.activated_at, + last_meaningful_activity_at=row.last_meaningful_activity_at, + updated_at=current_time, ) + if row.activated_at is not None + or row.last_meaningful_activity_at is not None + else None + ) + lifecycle = calculate_lifecycle( + build_lifecycle_inputs( + row.user_id, + created_at=row.created_at, + activity=activity, + totals=totals.get(row.user_id), + as_of=current_time, + ), + current_time, ) - if filters.priority: - selected.sort( - key=lambda item: ( - _PRIORITY_RANK[item.priority_group], - -_TIER_RANK[item.commercial_tier], - -item.lifetime_net_purchased_micro, - -item.lifecycle.inactive_days, - item.user_id, + fact = commercial.get(row.user_id) + items.append( + ValueUserListItem( + user_id=row.user_id, + display_name=row.display_name, + email=row.email, + joined_at=row.created_at, + lifecycle=lifecycle, + operational=operational_result_from_code( + row.operational_reason_code, as_of=as_of + ), + commercial_tier=row.tier, + lifetime_net_purchased_micro=( + fact.lifetime_net_purchased_micro if fact is not None else 0 + ), + priority_group=_priority_group( + row.operational_state, row.lifecycle_segment + ), + profile_path=f"/admin/analytics/users/{row.user_id}", ) ) - else: - selected.sort(key=lambda item: item.user_id) return PaginatedValueUsers( - items=selected[offset : offset + page_size], - total=len(selected), - limit=page_size, - offset=offset, + items=items, total=total, limit=page_size, offset=offset ) def get_user_profile( @@ -1437,30 +1397,39 @@ def get_user_profile( start, end = _validate_dates(start, end) subject_id = positive_user_id(user_id) current_time = _utc(now or datetime.now(UTC), "now") - snapshot = self.value_store.get_current_snapshot(subject_id) - if snapshot is None: - raise LookupError("Analytics value snapshot was not found") - commercial = self.value_store.list_commercial_values( - [subject_id], - start=_day_start(start), - end=_day_start(end), - )[subject_id] + axes = self.legacy_service.get_user_axes(subject_id, now=current_time) legacy = self.legacy_service.get_user_profile( user_id=subject_id, now=current_time, start=_day_start(start), end=_day_start(end), + axes=axes, ) - _weekly, _movement, transitions, _availability = self._history( + commercial = self.value_store.list_commercial_values( [subject_id], - start=start, - end=end, - use_anonymous_rollups=False, - ) + start=_day_start(start), + end=_day_start(end), + )[subject_id] + # This user's own moves from the transition log, each stamped with its + # own day. Never the anonymous rollups: they are population-wide. + transitions = [ + LifecycleTransition( + from_segment=row.from_segment, + to_segment=row.to_segment, + users=1, + period_start=row.snapshot_date, + period_end=row.snapshot_date + timedelta(days=1), + data_quality=row.data_quality, + ) + for row in self.value_store.list_user_transitions( + subject_id, start=start, end=end + ) + ] + lifecycle, operational = axes return ValueUserProfile( **legacy.model_dump(), - lifecycle=_lifecycle(snapshot), - operational=_operational(snapshot), + lifecycle=lifecycle, + operational=operational, commercial=commercial, selected_period_start=start, selected_period_end=end, @@ -1478,6 +1447,7 @@ def get_user_profile( "LifecycleMovementPoint", "LifecycleTransition", "OperationalAnalyticsResponse", + "OperationalReasonCount", "PaginatedValueUsers", "RetentionAnalyticsResponse", "RetentionCell", diff --git a/dashboard/backend/domain/analytics/value_repository.py b/dashboard/backend/domain/analytics/value_repository.py index adfdda795..f8fda9d71 100644 --- a/dashboard/backend/domain/analytics/value_repository.py +++ b/dashboard/backend/domain/analytics/value_repository.py @@ -7,18 +7,20 @@ (``PostgresValueAnalyticsStore``); ``build_value_analytics_store()`` below selects between them the same way ``repository.py``'s ``_build_analytics_store()`` selects between ``AnalyticsStore`` and ``PostgresAnalyticsStore``. Every method -that reads or writes this store's own tables (``user_analytics_snapshots``, -``user_lifecycle_daily_snapshots``, ``analytics_projection_jobs``) has exactly -one code path here; the two methods that branch on a *different* store's -dialect (``list_commercial_values``, ``list_credit_activity``, both reading -``self.credits_base``) are unchanged and identical on both twins, because that -branch was never about which twin this class is -- a caller can pair either -``analytics_base`` with either ``credits_base``, and both twins must handle it. +that reads or writes this store's own tables (``user_daily_facts``, +``user_activity``, ``lifecycle_transitions``, ``analytics_projection_jobs``) +has exactly one code path here; the two methods that branch on a *different* +store's dialect (``list_commercial_values``, ``list_credit_activity``, both +reading ``self.credits_base``) are unchanged and identical on both twins, +because that branch was never about which twin this class is -- a caller can +pair either ``analytics_base`` with either ``credits_base``, and both twins +must handle it. """ from __future__ import annotations -import json +import sqlite3 +from dataclasses import dataclass from datetime import date, datetime, timedelta, timezone from typing import Any, Literal, Mapping, Sequence @@ -42,9 +44,111 @@ LIFECYCLE_SEGMENTS = frozenset( {"new", "onboarding", "growing", "core", "at_risk", "dormant"} ) -LIFECYCLE_ROLLUP_METRICS = frozenset( - {"lifecycle_segment_count", "lifecycle_transition"} +# Long-term families written before user-keyed rows expire (design SS6.7): +# metric_name names the dimension, user_state carries its value. +_DIMENSION_FAMILIES = frozenset( + { + "runs_by_tier", + "runs_by_user_group", + "operator_cost_by_tier", + "operator_cost_by_user_group", + "own_spend_by_tier", + "own_spend_by_user_group", + } +) +LIFECYCLE_ROLLUP_METRICS = ( + frozenset({"lifecycle_segment_count", "lifecycle_transition"}) + | _DIMENSION_FAMILIES ) +_ROLLUP_COLUMNS = ( + "rollup_date", + "metric_name", + "event_name", + "billing_mode", + "provider_id", + "model_id", + "outcome", + "error_category", + "user_state", + "value_count", + "value_sum_micro", + "updated_at", +) + + +def _lifecycle_rollup_payloads( + day: date, rows: Sequence[Any], *, replace_transitions: bool +) -> list[tuple]: + """Validate lifecycle-owned rollup rows and flatten them for INSERT. + + Shared by both twins: the rules are dialect-free, and a rule fixed on one + copy only would be a Postgres-only behaviour divergence. + """ + + payloads = [] + for row in rows: + if row.rollup_date != day or row.metric_name not in LIFECYCLE_ROLLUP_METRICS: + raise ValueError("invalid lifecycle rollup row") + if row.metric_name == "lifecycle_segment_count": + valid_dimensions = ( + not row.event_name and row.user_state in LIFECYCLE_SEGMENTS + ) + valid_sum = row.value_sum_micro == 0 + elif row.metric_name == "lifecycle_transition": + valid_dimensions = ( + replace_transitions + and row.event_name in LIFECYCLE_SEGMENTS + and row.user_state in LIFECYCLE_SEGMENTS + and row.event_name != row.user_state + ) + valid_sum = row.value_sum_micro == 0 + else: + # A long-term dimension family: user_state is the tier or group + # value. Only the cost families carry a sum; the runs families + # carry a count alone. + valid_dimensions = not row.event_name and bool(row.user_state) + valid_sum = ( + row.value_sum_micro == 0 + if row.metric_name.startswith("runs_by_") + else row.value_sum_micro >= 0 + ) + unused_dimensions = ( + row.billing_mode, + row.provider_id, + row.model_id, + row.error_category, + ) + if ( + not valid_dimensions + or any(unused_dimensions) + or row.outcome not in {"complete", "partial"} + or not valid_sum + ): + raise ValueError("invalid lifecycle rollup dimensions") + payloads.append( + ( + row.rollup_date.isoformat(), + row.metric_name, + row.event_name, + row.billing_mode, + row.provider_id, + row.model_id, + row.outcome, + row.error_category, + row.user_state, + row.value_count, + row.value_sum_micro, + utc_iso(row.updated_at), + ) + ) + return payloads + + +def _replaced_lifecycle_metrics(*, replace_transitions: bool) -> list[str]: + return sorted( + LIFECYCLE_ROLLUP_METRICS + - (set() if replace_transitions else {"lifecycle_transition"}) + ) _ACTIVE_RUN_STATUSES = frozenset({"created", "loading", "running"}) _TERMINAL_RUN_STATUSES = frozenset( {"completed", "failed", "cancelled", "closed", "timed_out"} @@ -88,47 +192,6 @@ def _optional_timestamp(value: object) -> datetime | None: return None -class UserValueSnapshot(BaseModel): - model_config = ConfigDict(extra="forbid", frozen=True) - - user_id: int = Field(gt=0) - lifecycle_segment: LifecycleSegment - lifecycle_reason_code: str = Field(min_length=1, max_length=100) - lifecycle_reason: str = Field(min_length=1, max_length=500) - lifecycle_evidence: Sequence[str] = Field(default_factory=tuple, max_length=10) - operational_state: OperationalState - operational_reason_code: str = Field(min_length=1, max_length=100) - operational_reason: str = Field(min_length=1, max_length=500) - operational_evidence: Sequence[str] = Field(default_factory=tuple, max_length=10) - activated_at: datetime | None = None - last_meaningful_activity_at: datetime | None = None - inactive_days: int = Field(ge=0) - active_days_30d: int = Field(ge=0, le=30) - successful_backtests_30d: int = Field(ge=0) - calculated_at: datetime - - @field_validator("activated_at", "last_meaningful_activity_at", "calculated_at") - @classmethod - def require_timezone(cls, value: datetime | None) -> datetime | None: - return _utc(value) if value is not None else None - - -class UserLifecycleDailySnapshot(BaseModel): - model_config = ConfigDict(extra="forbid", frozen=True) - - snapshot_date: date - user_id: int = Field(gt=0) - lifecycle_segment: LifecycleSegment - lifecycle_reason_code: str = Field(min_length=1, max_length=100) - data_quality: Literal["complete", "partial"] - calculated_at: datetime - - @field_validator("calculated_at") - @classmethod - def require_timezone(cls, value: datetime) -> datetime: - return _utc(value) - - class UserActivity(BaseModel): """One user's current activity clock. One row, overwritten in place. @@ -294,6 +357,30 @@ def require_timezone(cls, value: datetime) -> datetime: return _utc(value) +class LegacyLifecycleSnapshot(BaseModel): + """One row of the retired ``user_lifecycle_daily_snapshots`` table. + + Read once, by the startup migration, to roll up the legacy days no fact + row covers before the table is dropped (``rollups.rollup_legacy_lifecycle_days``). + """ + + model_config = ConfigDict(extra="forbid", frozen=True) + + snapshot_date: date + user_id: int = Field(gt=0) + lifecycle_segment: LifecycleSegment + data_quality: Literal["complete", "partial"] + + +class LegacyLifecycleHistory(BaseModel): + """The legacy days still to roll up, plus their rows and the previous day's.""" + + model_config = ConfigDict(extra="forbid", frozen=True) + + days: tuple[date, ...] = () + snapshots: tuple[LegacyLifecycleSnapshot, ...] = () + + _DAILY_FACT_COLUMNS = ( "snapshot_date", "user_id", "lifecycle_segment", "lifecycle_reason_code", "operational_state", "operational_reason_code", "tier", "user_group", "active", @@ -341,6 +428,12 @@ def require_timezone(cls, value: datetime) -> datetime: WHERE snapshot_date >= {p} AND snapshot_date <= {p} GROUP BY snapshot_date """ +# Older than every received_at, so a requeued day compares as stale. +_RECOMPUTE_EPOCH = "1970-01-01T00:00:00+00:00" +_REQUEUE_RECOMPUTE_SQL = """ + UPDATE user_daily_facts SET calculated_at = {p} + WHERE snapshot_date IN ({days}) +""" def _day_bounds_iso(day: date) -> tuple[str, str]: @@ -605,6 +698,99 @@ class CurrentOperationalFacts(BaseModel): run_beyond_safe_deadline: bool = False +class FactStateCount(BaseModel): + """One (operational state, reason, segment) cell of one day's population.""" + + model_config = ConfigDict(extra="forbid", frozen=True) + + operational_state: OperationalState + # Nullable like the column: history-copy rows carry 'healthy' with no code. + operational_reason_code: str | None = None + lifecycle_segment: LifecycleSegment + users: int = Field(ge=0) + partial_rows: int = Field(default=0, ge=0) + + +class SegmentDayCount(BaseModel): + model_config = ConfigDict(extra="forbid", frozen=True) + + snapshot_date: date + lifecycle_segment: LifecycleSegment + users: int = Field(ge=0) + partial_rows: int = Field(default=0, ge=0) + + +class TransitionCount(BaseModel): + model_config = ConfigDict(extra="forbid", frozen=True) + + from_segment: LifecycleSegment + to_segment: LifecycleSegment + users: int = Field(ge=0) + partial_rows: int = Field(default=0, ge=0) + + +class AttentionCandidate(BaseModel): + """A blocked or needs-attention user with their trailing-window run counts.""" + + model_config = ConfigDict(extra="forbid", frozen=True) + + user_id: int = Field(gt=0) + display_name: str + email: str + created_at: datetime + operational_state: OperationalState + operational_reason_code: str | None = None + recent_runs: int = Field(default=0, ge=0) + recent_failures: int = Field(default=0, ge=0) + + +class GroupFactTotals(BaseModel): + model_config = ConfigDict(extra="forbid", frozen=True) + + total_runs: int = Field(default=0, ge=0) + operator_cost_micro: int = Field(default=0, ge=0) + successful_run_users: int = Field(default=0, ge=0) + repeat_users: int = Field(default=0, ge=0) + # Rows the daily job (or the PR A history copy) marked partial: their + # zeros are not known zeros, so /groups must not call the window ready. + partial_rows: int = Field(default=0, ge=0) + + +@dataclass(frozen=True) +class UserPageQuery: + """The users-list filters, dialect-free, so the SQL builder can live here + without importing value_queries (which imports this module).""" + + q: str | None = None + lifecycle_segment: str | None = None + operational_state: str | None = None + commercial_tier: str | None = None + user_group: str | None = None + activated: bool | None = None + last_meaningful_activity_from: datetime | None = None + last_meaningful_activity_to: datetime | None = None + priority: bool = False + include_internal: bool = False + + +class UserPageRow(BaseModel): + model_config = ConfigDict(extra="forbid", frozen=True) + + user_id: int = Field(gt=0) + email: str + display_name: str + role: str + user_group: str + created_at: datetime + activated_at: datetime | None = None + last_meaningful_activity_at: datetime | None = None + lifecycle_segment: LifecycleSegment | None = None + operational_state: OperationalState = "healthy" + operational_reason_code: str = "no_supported_issue" + tier: CommercialTier = "unpaid" + data_quality: Literal["complete", "partial"] | None = None + + class ProjectionJob(BaseModel): model_config = ConfigDict(extra="forbid", frozen=True) @@ -644,93 +830,603 @@ def _validate_window(start: datetime, end: datetime) -> tuple[datetime, datetime return window_start, window_end -def _legacy_seed(snapshot: UserValueSnapshot) -> tuple[str, str, str]: - """Seed compatibility fields only when no legacy projection exists yet.""" - - if snapshot.operational_state == "blocked": - status = "blocked" - reason_code = snapshot.operational_reason_code - reason = snapshot.operational_reason - elif snapshot.operational_state == "needs_attention": - status = "needs_attention" - reason_code = snapshot.operational_reason_code - reason = snapshot.operational_reason - elif snapshot.lifecycle_segment == "dormant": - status = "dormant" - reason_code = snapshot.lifecycle_reason_code - reason = snapshot.lifecycle_reason - elif snapshot.lifecycle_segment in {"new", "onboarding"}: - status = "onboarding" - reason_code = snapshot.lifecycle_reason_code - reason = snapshot.lifecycle_reason - else: - status = "active" - reason_code = snapshot.lifecycle_reason_code - reason = snapshot.lifecycle_reason - return status, reason_code, reason +def _projection_job_name(value: object) -> str: + """Shared by both twins: ``value_repository_postgres.py`` imports it.""" + if ( + not isinstance(value, str) + or not value + or value != value.strip() + or len(value) > 100 + ): + raise ValueError("job_name must be a trimmed non-empty string") + return value -def _current_snapshot_from_row(row: Any) -> UserValueSnapshot | None: - """Shared by both twins. +# -- Set-based fact-table reads (design §6.10). ``{p}`` is the placeholder, +# ``{internal}`` the optional internal-account exclusion, ``{user_clause}`` the +# per-twin id clause. Both twins render these; only the placeholder and the id +# clause differ. ``active`` and ``excluded`` are INTEGER on SQLite and BOOLEAN on +# Postgres, so the SQL says ``WHERE active`` / ``WHERE excluded`` and never +# ``= 1``. - Moved out of the class (was ``ValueAnalyticsStore._current_snapshot_from_row``, - a ``@staticmethod``) so ``value_repository_postgres.py`` can import it - directly, mirroring how ``repository_postgres.py`` imports bare functions - (``_row_to_event``) from ``repository.py``. No caller outside this module - referenced the staticmethod, so this is not a behaviour change. +_INTERNAL_EXCLUSION_SQL = """ + AND u.role <> 'admin' + AND u.id NOT IN ( + SELECT user_id FROM analytics_subject_settings WHERE excluded + ) +""" + + +def _subject_filter(column: str) -> str: + """Read-time exclusion over user-keyed fact rows. Shared by both twins. + + The daily job writes no row for an admin or an excluded account (design + SS6), but a row written before the account was excluded or promoted stays + in the table for up to 180 days. ``main`` filtered every read through + ``_eligible_users``, so an exclusion took effect on the next request; + this fragment keeps that, instead of letting it wait for tomorrow's facts + (and, for history, for the rows to expire). """ - if row is None or _row_value(row, "lifecycle_segment") is None: - return None + return f""" + AND {column} NOT IN (SELECT id FROM users WHERE role = 'admin') + AND {column} NOT IN ( + SELECT user_id FROM analytics_subject_settings WHERE excluded + ) +""" - def seq(name: str) -> tuple[str, ...]: - try: - value = json.loads(_row_value(row, name, "[]")) - if not isinstance(value, list) or not all( - isinstance(item, str) for item in value - ): - return () - return tuple(value) - except (TypeError, ValueError, json.JSONDecodeError): - return () - return UserValueSnapshot( +_COUNT_STATES_SQL = ( + """ + SELECT operational_state, operational_reason_code, lifecycle_segment, + COUNT(*) AS users, + SUM(CASE WHEN data_quality = 'partial' THEN 1 ELSE 0 END) AS partial_rows + FROM user_daily_facts + WHERE snapshot_date = {p} + """ + + _subject_filter("user_id") + + """ + GROUP BY operational_state, operational_reason_code, lifecycle_segment + ORDER BY operational_state, operational_reason_code, lifecycle_segment +""" +) + +_COUNT_SEGMENTS_SQL = ( + """ + SELECT snapshot_date, lifecycle_segment, + COUNT(*) AS users, + SUM(CASE WHEN data_quality = 'partial' THEN 1 ELSE 0 END) AS partial_rows + FROM user_daily_facts + WHERE snapshot_date >= {p} AND snapshot_date < {p} + """ + + _subject_filter("user_id") + + """ + GROUP BY snapshot_date, lifecycle_segment + ORDER BY snapshot_date, lifecycle_segment +""" +) + +_COUNT_TRANSITIONS_SQL = ( + """ + SELECT from_segment, to_segment, + COUNT(*) AS users, + SUM(CASE WHEN data_quality = 'partial' THEN 1 ELSE 0 END) AS partial_rows + FROM lifecycle_transitions + WHERE snapshot_date >= {p} AND snapshot_date < {p} + """ + + _subject_filter("user_id") + + """ + GROUP BY from_segment, to_segment + ORDER BY from_segment, to_segment +""" +) + +_ATTENTION_SQL = ( + """ + SELECT f.user_id, f.operational_state, f.operational_reason_code, + u.display_name, u.email, u.created_at, + COALESCE(SUM(w.runs_requested), 0) AS recent_runs, + COALESCE(SUM(w.runs_failed), 0) AS recent_failures + FROM user_daily_facts AS f + JOIN users AS u ON u.id = f.user_id + LEFT JOIN user_daily_facts AS w + ON w.user_id = f.user_id + AND w.snapshot_date >= {p} AND w.snapshot_date <= {p} + WHERE f.snapshot_date = {p} + AND f.operational_state IN ('blocked', 'needs_attention') + """ + + _subject_filter("f.user_id") + + """ + GROUP BY f.user_id, f.operational_state, f.operational_reason_code, + u.display_name, u.email, u.created_at + ORDER BY recent_failures DESC, f.user_id DESC + LIMIT {p} +""" +) + +_REPLACE_TRANSITIONS_DELETE_SQL = ( + """ + DELETE FROM lifecycle_transitions + WHERE snapshot_date = {p} + """ + + _subject_filter("user_id") +) + +_UPSERT_TRANSITIONS_SQL = f""" + INSERT INTO lifecycle_transitions ({", ".join(_TRANSITION_COLUMNS)}) + VALUES ({", ".join("{p}" for _ in _TRANSITION_COLUMNS)}) + ON CONFLICT(user_id, snapshot_date) DO UPDATE SET {_TRANSITION_UPDATES} +""" + +_FACTS_FOR_DATE_SQL = ( + """ + SELECT * FROM user_daily_facts + WHERE snapshot_date = {p} + """ + + _subject_filter("user_id") + + """ + ORDER BY user_id +""" +) + +# The PR A history copy (``copy_daily_snapshot_history``) filled +# ``user_daily_facts`` from the legacy table and wrote no transition log, and +# the legacy rollup skips any day a fact row covers, so 07-29..09-22 had no +# movement source once ``_history`` stopped deriving it from the legacy rows. +# The startup migration derives it from consecutive fact days, and for a first +# fact day whose predecessor exists only in the legacy table, from that table +# while it still exists. ``{prev}`` is the dialect's "the day before +# cur.snapshot_date" as YYYY-MM-DD text; ``{days}`` / ``{segments}`` are +# placeholder lists. ``inactive_days`` is not on a fact row; 0 is the column +# default and nothing reads it off a transition. +_DAYS_WITHOUT_TRANSITIONS_SQL = """ + SELECT DISTINCT f.snapshot_date + FROM user_daily_facts AS f + WHERE NOT EXISTS ( + SELECT 1 FROM lifecycle_transitions AS t + WHERE t.snapshot_date = f.snapshot_date + ) + ORDER BY f.snapshot_date +""" + +_BACKFILL_TRANSITIONS_SQL = """ + INSERT INTO lifecycle_transitions ( + user_id, snapshot_date, from_segment, to_segment, inactive_days, + data_quality, created_at + ) + SELECT cur.user_id, cur.snapshot_date, prev.lifecycle_segment, + cur.lifecycle_segment, 0, + CASE WHEN cur.data_quality = 'partial' OR prev.data_quality = 'partial' + THEN 'partial' ELSE 'complete' END, + {p} + FROM user_daily_facts AS cur + JOIN {source} AS prev + ON prev.user_id = cur.user_id AND prev.snapshot_date = {prev} + WHERE cur.snapshot_date IN ({days}) + AND prev.lifecycle_segment <> cur.lifecycle_segment + AND prev.lifecycle_segment IN ({segments}) + {guard} + ON CONFLICT (user_id, snapshot_date) DO NOTHING +""" + +# Only for the legacy source: a day whose predecessor has facts takes its +# previous segment from them, never from the table being retired. +_NO_PRIOR_FACT_DAY_SQL = """ + AND NOT EXISTS ( + SELECT 1 FROM user_daily_facts AS pf WHERE pf.snapshot_date = {prev} + ) +""" + + +def _backfill_transition_statements( + *, p: str, prev: str, days: Sequence[str] +) -> list[tuple[str, list[str]]]: + """(sql, params) per source, facts first. Shared by both twins.""" + segments = sorted(LIFECYCLE_SEGMENTS) + day_marks = ", ".join(p for _ in days) + segment_marks = ", ".join(p for _ in segments) + statements = [] + for source, guard in ( + ("user_daily_facts", ""), + ("user_lifecycle_daily_snapshots", _NO_PRIOR_FACT_DAY_SQL.format(prev=prev)), + ): + statements.append( + ( + _BACKFILL_TRANSITIONS_SQL.format( + p=p, + source=source, + prev=prev, + days=day_marks, + segments=segment_marks, + guard=guard, + ), + [*days, *segments], + ) + ) + return statements + +# The legacy days the drop would otherwise destroy: no fact row covers them +# and no lifecycle rollup was ever written for them (design SS6.9 "Movement +# charts keep their history"). {metrics} is the lifecycle-owned families, so a +# day already rolled up -- by an earlier boot whose drop failed, or by the +# retention sweep -- is never overwritten. +_LEGACY_UNROLLED_DATES_SQL = """ + SELECT DISTINCT s.snapshot_date + FROM user_lifecycle_daily_snapshots AS s + WHERE NOT EXISTS ( + SELECT 1 FROM user_daily_facts AS f + WHERE f.snapshot_date = s.snapshot_date + ) + AND NOT EXISTS ( + SELECT 1 FROM analytics_daily_rollups AS r + WHERE r.rollup_date = s.snapshot_date + AND r.metric_name IN ({metrics}) + ) + ORDER BY s.snapshot_date +""" + +_LEGACY_SNAPSHOTS_SQL = """ + SELECT snapshot_date, user_id, lifecycle_segment, data_quality + FROM user_lifecycle_daily_snapshots + WHERE {date_clause} + ORDER BY snapshot_date, user_id +""" + + +def _legacy_snapshot(row: Any) -> LegacyLifecycleSnapshot: + """Shared by both twins.""" + return LegacyLifecycleSnapshot( + snapshot_date=date.fromisoformat(str(_row_value(row, "snapshot_date"))), user_id=int(_row_value(row, "user_id")), lifecycle_segment=_row_value(row, "lifecycle_segment"), - lifecycle_reason_code=_row_value(row, "lifecycle_reason_code"), - lifecycle_reason=_row_value(row, "lifecycle_reason"), - lifecycle_evidence=seq("lifecycle_evidence_json"), - operational_state=_row_value(row, "operational_state") or "healthy", - operational_reason_code=( - _row_value(row, "operational_reason_code") or "no_supported_issue" - ), - operational_reason=( - _row_value(row, "operational_reason") - or "No supported current operational issue was detected." - ), - operational_evidence=seq("operational_evidence_json"), + data_quality=_row_value(row, "data_quality"), + ) + + +def _legacy_read_dates(days: Sequence[date]) -> list[str]: + """Each unrolled day plus the day before it, for its transitions. Shared.""" + wanted = set(days) | {day - timedelta(days=1) for day in days} + return sorted(day.isoformat() for day in wanted) + + +def _sqlite_table_exists(conn: sqlite3.Connection, table: str) -> bool: + """Locale-proof: asks the catalogue rather than parsing an error message.""" + return ( + conn.execute( + "SELECT 1 FROM sqlite_master WHERE type = 'table' AND name = ?", + (table,), + ).fetchone() + is not None + ) + + +_GROUP_TOTALS_SQL = ( + """ + SELECT user_group, + COALESCE(SUM(runs_requested), 0) AS total_runs, + COALESCE(SUM(operator_cost_micro), 0) AS operator_cost_micro, + COUNT(DISTINCT CASE WHEN runs_completed > 0 THEN user_id END) + AS successful_run_users, + COALESCE(SUM(CASE WHEN data_quality = 'partial' THEN 1 ELSE 0 END), 0) + AS partial_rows + FROM user_daily_facts + WHERE snapshot_date >= {p} AND snapshot_date < {p} + """ + + _subject_filter("user_id") + + """ + GROUP BY user_group +""" +) + +_LATEST_FACT_DATE_SQL = """ + SELECT MAX(snapshot_date) AS latest + FROM user_daily_facts + WHERE snapshot_date <= {p} +""" + + +def _latest_fact_date(row: Any) -> date | None: + """Shared by both twins.""" + latest = _row_value(row, "latest") if row is not None else None + return date.fromisoformat(str(latest)) if latest else None + + +_GROUP_REPEAT_SQL = ( + """ + SELECT user_group, COUNT(*) AS repeat_users + FROM ( + SELECT user_group, user_id + FROM user_daily_facts + WHERE snapshot_date >= {p} AND snapshot_date < {p} + AND runs_completed > 0 + """ + + _subject_filter("user_id") + + """ + GROUP BY user_group, user_id + HAVING COUNT(*) >= 2 + ) AS repeaters + GROUP BY user_group +""" +) + +_ACTIVE_DATES_SQL = """ + SELECT user_id, snapshot_date + FROM user_daily_facts + WHERE snapshot_date >= {p} AND snapshot_date < {p} + AND active{user_clause} + ORDER BY user_id, snapshot_date +""" + +_ACTIVATIONS_SQL = """ + SELECT a.user_id, a.activated_at + FROM user_activity AS a + JOIN users AS u ON u.id = a.user_id + WHERE a.activated_at IS NOT NULL + AND a.activated_at >= {p} AND a.activated_at < {p} + {internal} + ORDER BY a.user_id +""" + +_COUNT_ACTIVATED_SQL = """ + SELECT COUNT(*) AS activated + FROM user_activity AS a + JOIN users AS u ON u.id = a.user_id + WHERE a.activated_at IS NOT NULL + {internal} +""" + +_USER_TRANSITIONS_SQL = """ + SELECT * FROM lifecycle_transitions + WHERE user_id = {p} + AND snapshot_date >= {p} AND snapshot_date < {p} + ORDER BY snapshot_date, transition_id +""" + +_TRANSITIONS_FOR_DATE_SQL = ( + """ + SELECT * FROM lifecycle_transitions + WHERE snapshot_date = {p} + """ + + _subject_filter("user_id") + + """ + ORDER BY user_id +""" +) + + +_USERS_PAGE_SELECT = """ + SELECT u.id AS user_id, u.email, u.display_name, u.role, u.user_group, + u.created_at, a.activated_at, a.last_meaningful_activity_at, + f.lifecycle_segment, f.operational_state, f.operational_reason_code, + f.tier, f.data_quality +""" +_USERS_PAGE_FROM = """ + FROM users AS u + LEFT JOIN user_activity AS a ON a.user_id = u.id + LEFT JOIN user_daily_facts AS f + ON f.user_id = u.id AND f.snapshot_date = {p} +""" +_USERS_PAGE_PRIORITY_ORDER = """ + ORDER BY CASE COALESCE(f.operational_state, 'healthy') + WHEN 'blocked' THEN 0 + WHEN 'needs_attention' THEN 1 + ELSE CASE f.lifecycle_segment + WHEN 'at_risk' THEN 2 + WHEN 'onboarding' THEN 3 + ELSE 4 + END + END ASC, + CASE COALESCE(f.tier, 'unpaid') + WHEN 'high_value' THEN 0 + WHEN 'invested' THEN 1 + WHEN 'starter' THEN 2 + ELSE 3 + END ASC, + CASE WHEN a.last_meaningful_activity_at IS NULL THEN 0 ELSE 1 END ASC, + a.last_meaningful_activity_at ASC, + u.id ASC +""" + + +def _users_page_sql( + query: UserPageQuery, *, day: date, limit: int, offset: int, p: str +) -> tuple[str, str, list[Any], list[Any]]: + """Render (page_sql, count_sql, page_params, count_params) for one dialect. + + Every predicate is parameterised; the only interpolated text is the fixed + every filter is a parameter, never interpolated text. The LIKE pattern + comes from users.admin_user_search_pattern so '%' and '_' in a search are + literal, exactly as the account console's own search treats them. + """ + from dashboard.backend.users import admin_user_search_pattern + + where: list[str] = [] + params: list[Any] = [_validate_day(day)] + if not query.include_internal: + where.append( + "u.role <> 'admin' AND u.id NOT IN " + "(SELECT user_id FROM analytics_subject_settings WHERE excluded)" + ) + pattern = admin_user_search_pattern(query.q) + if pattern is not None: + where.append( + f"(lower(u.email) LIKE lower({p}) ESCAPE '\\'" + f" OR lower(u.display_name) LIKE lower({p}) ESCAPE '\\')" + ) + params.extend([pattern, pattern]) + if query.user_group is not None: + where.append(f"u.user_group = {p}") + params.append(query.user_group) + if query.commercial_tier is not None: + where.append(f"COALESCE(f.tier, 'unpaid') = {p}") + params.append(query.commercial_tier) + if query.operational_state is not None: + where.append(f"COALESCE(f.operational_state, 'healthy') = {p}") + params.append(query.operational_state) + if query.lifecycle_segment is not None: + where.append(f"f.lifecycle_segment = {p}") + params.append(query.lifecycle_segment) + if query.activated is True: + where.append("a.activated_at IS NOT NULL") + elif query.activated is False: + where.append("a.activated_at IS NULL") + if query.last_meaningful_activity_from is not None: + where.append(f"a.last_meaningful_activity_at >= {p}") + params.append(utc_iso(_utc(query.last_meaningful_activity_from))) + if query.last_meaningful_activity_to is not None: + where.append(f"a.last_meaningful_activity_at <= {p}") + params.append(utc_iso(_utc(query.last_meaningful_activity_to))) + if query.priority: + where.append( + "(COALESCE(f.operational_state, 'healthy') IN ('blocked', 'needs_attention')" + " OR f.lifecycle_segment IN ('at_risk', 'onboarding'))" + ) + where_sql = (" WHERE " + " AND ".join(where)) if where else "" + from_sql = _USERS_PAGE_FROM.format(p=p) + order_sql = _USERS_PAGE_PRIORITY_ORDER if query.priority else " ORDER BY u.id ASC" + page_sql = ( + f"{_USERS_PAGE_SELECT}{from_sql}{where_sql}{order_sql} LIMIT {p} OFFSET {p}" + ) + count_sql = f"SELECT COUNT(*) AS total{from_sql}{where_sql}" + return page_sql, count_sql, [*params, limit, offset], list(params) + + +def _validate_page_window(limit: int, offset: int) -> tuple[int, int]: + if isinstance(offset, bool) or not isinstance(offset, int) or offset < 0: + raise ValueError("offset must be a non-negative integer") + return positive_limit(limit), offset + + +def _user_page_row(row: Any) -> UserPageRow: + """Shared by both twins.""" + return UserPageRow( + user_id=int(_row_value(row, "user_id")), + email=str(_row_value(row, "email") or ""), + display_name=str(_row_value(row, "display_name") or ""), + role=str(_row_value(row, "role") or "user"), + user_group=str(_row_value(row, "user_group") or "unknown"), + created_at=_timestamp(_row_value(row, "created_at")), activated_at=_optional_timestamp(_row_value(row, "activated_at")), last_meaningful_activity_at=_optional_timestamp( _row_value(row, "last_meaningful_activity_at") ), - inactive_days=int(_row_value(row, "inactive_days", 0)), - active_days_30d=int(_row_value(row, "active_days_30d", 0)), - successful_backtests_30d=int( - _row_value(row, "successful_backtests_30d", 0) + lifecycle_segment=_row_value(row, "lifecycle_segment"), + operational_state=_row_value(row, "operational_state") or "healthy", + operational_reason_code=( + _row_value(row, "operational_reason_code") or "no_supported_issue" ), - calculated_at=_timestamp(_row_value(row, "calculated_at")), + tier=_row_value(row, "tier") or "unpaid", + data_quality=_row_value(row, "data_quality"), ) -def _projection_job_name(value: object) -> str: - """Shared by both twins; see ``_current_snapshot_from_row`` above.""" - if ( - not isinstance(value, str) - or not value - or value != value.strip() - or len(value) > 100 - ): - raise ValueError("job_name must be a trimmed non-empty string") - return value +def _transition_row(row: Any) -> LifecycleTransitionRow: + """Shared by both twins.""" + return LifecycleTransitionRow( + user_id=int(_row_value(row, "user_id")), + snapshot_date=date.fromisoformat(str(_row_value(row, "snapshot_date"))), + from_segment=_row_value(row, "from_segment"), + to_segment=_row_value(row, "to_segment"), + inactive_days=int(_row_value(row, "inactive_days", 0)), + data_quality=_row_value(row, "data_quality"), + created_at=_timestamp(_row_value(row, "created_at")), + ) + + +def _attention_candidate(row: Any) -> AttentionCandidate: + """Shared by both twins.""" + return AttentionCandidate( + user_id=int(_row_value(row, "user_id")), + display_name=str(_row_value(row, "display_name") or ""), + email=str(_row_value(row, "email") or ""), + created_at=_timestamp(_row_value(row, "created_at")), + operational_state=_row_value(row, "operational_state"), + operational_reason_code=_row_value(row, "operational_reason_code"), + recent_runs=int(_row_value(row, "recent_runs", 0) or 0), + recent_failures=int(_row_value(row, "recent_failures", 0) or 0), + ) + + +def _group_totals( + totals: Sequence[Any], repeats: Sequence[Any] +) -> dict[str, GroupFactTotals]: + """Shared by both twins.""" + repeat_by_group = { + str(_row_value(row, "user_group")): int(_row_value(row, "repeat_users", 0) or 0) + for row in repeats + } + return { + str(_row_value(row, "user_group")): GroupFactTotals( + total_runs=int(_row_value(row, "total_runs", 0) or 0), + operator_cost_micro=max(int(_row_value(row, "operator_cost_micro", 0) or 0), 0), + successful_run_users=int(_row_value(row, "successful_run_users", 0) or 0), + repeat_users=repeat_by_group.get(str(_row_value(row, "user_group")), 0), + partial_rows=int(_row_value(row, "partial_rows", 0) or 0), + ) + for row in totals + } + + +def _fact_state_count(row: Any) -> FactStateCount: + """Shared by both twins.""" + return FactStateCount( + operational_state=_row_value(row, "operational_state"), + operational_reason_code=_row_value(row, "operational_reason_code"), + lifecycle_segment=_row_value(row, "lifecycle_segment"), + users=int(_row_value(row, "users", 0)), + partial_rows=int(_row_value(row, "partial_rows", 0) or 0), + ) + + +def _segment_day_count(row: Any) -> SegmentDayCount: + """Shared by both twins.""" + return SegmentDayCount( + snapshot_date=date.fromisoformat(str(_row_value(row, "snapshot_date"))), + lifecycle_segment=_row_value(row, "lifecycle_segment"), + users=int(_row_value(row, "users", 0)), + partial_rows=int(_row_value(row, "partial_rows", 0) or 0), + ) + + +def _transition_count(row: Any) -> TransitionCount: + """Shared by both twins.""" + return TransitionCount( + from_segment=_row_value(row, "from_segment"), + to_segment=_row_value(row, "to_segment"), + users=int(_row_value(row, "users", 0)), + partial_rows=int(_row_value(row, "partial_rows", 0) or 0), + ) + + +def _validate_day(day: date) -> str: + if not isinstance(day, date) or isinstance(day, datetime): + raise ValueError("day must be a date") + return day.isoformat() + + +def _validate_date_window(start: date, end: date) -> tuple[str, str]: + """Half-open ``[start, end)`` over dates, as ISO strings.""" + if not isinstance(start, date) or isinstance(start, datetime): + raise ValueError("start must be a date") + if not isinstance(end, date) or isinstance(end, datetime): + raise ValueError("end must be a date") + if end <= start: + raise ValueError("end must be later than start") + return start.isoformat(), end.isoformat() + + +def _id_chunks(user_ids: Sequence[int]) -> list[list[int]]: + """Dedupe ``user_ids`` and split into ``MAX_USER_BATCH``-sized chunks. + + ``_ids`` refuses more than one batch; a whole-population caller (the + retention cohorts) needs a read that spans several. + """ + if not isinstance(user_ids, (list, tuple)): + raise ValueError("user_ids must be a list or tuple") + values = list(dict.fromkeys(positive_user_id(item) for item in user_ids)) + return [ + values[index : index + MAX_USER_BATCH] + for index in range(0, len(values), MAX_USER_BATCH) + ] class ValueAnalyticsStore: @@ -793,178 +1489,6 @@ def __init__( def _analytics_connection(self): return self.analytics_base._get_connection() - def upsert_current_snapshot(self, snapshot: UserValueSnapshot) -> UserValueSnapshot: - def evidence(value: Sequence[str]) -> str: - return json.dumps( - list(value), - separators=(",", ":"), - ensure_ascii=True, - ) - - legacy_status, legacy_reason_code, legacy_reason = _legacy_seed(snapshot) - values = ( - snapshot.user_id, - legacy_status, - legacy_reason_code, - legacy_reason, - "[]", - snapshot.lifecycle_segment, - snapshot.lifecycle_reason_code, - snapshot.lifecycle_reason, - evidence(snapshot.lifecycle_evidence), - snapshot.operational_state, - snapshot.operational_reason_code, - snapshot.operational_reason, - evidence(snapshot.operational_evidence), - utc_iso(snapshot.activated_at) if snapshot.activated_at else None, - ( - utc_iso(snapshot.last_meaningful_activity_at) - if snapshot.last_meaningful_activity_at - else None - ), - snapshot.inactive_days, - snapshot.active_days_30d, - snapshot.successful_backtests_30d, - utc_iso(snapshot.calculated_at), - ) - columns = """ - user_id, status, reason_code, human_readable_reason, - evidence_event_ids_json, lifecycle_segment, lifecycle_reason_code, - lifecycle_reason, lifecycle_evidence_json, operational_state, - operational_reason_code, operational_reason, - operational_evidence_json, activated_at, - last_meaningful_activity_at, inactive_days, active_days_30d, - successful_backtests_30d, calculated_at - """ - updates = """ - lifecycle_segment=excluded.lifecycle_segment, - lifecycle_reason_code=excluded.lifecycle_reason_code, - lifecycle_reason=excluded.lifecycle_reason, - lifecycle_evidence_json=excluded.lifecycle_evidence_json, - operational_state=excluded.operational_state, - operational_reason_code=excluded.operational_reason_code, - operational_reason=excluded.operational_reason, - operational_evidence_json=excluded.operational_evidence_json, - activated_at=excluded.activated_at, - last_meaningful_activity_at=excluded.last_meaningful_activity_at, - inactive_days=excluded.inactive_days, - active_days_30d=excluded.active_days_30d, - successful_backtests_30d=excluded.successful_backtests_30d, - calculated_at=excluded.calculated_at - """ - with self._analytics_connection() as conn: - conn.execute( - f""" - INSERT INTO user_analytics_snapshots ({columns}) - VALUES ({", ".join(["?"] * len(values))}) - ON CONFLICT(user_id) DO UPDATE SET {updates} - """, - values, - ) - return snapshot - - def get_current_snapshot(self, user_id: int) -> UserValueSnapshot | None: - subject = positive_user_id(user_id) - with self._analytics_connection() as conn: - row = conn.execute( - "SELECT * FROM user_analytics_snapshots WHERE user_id=?", (subject,) - ).fetchone() - return _current_snapshot_from_row(row) - - def list_current_snapshots( - self, - user_ids: Sequence[int], - ) -> dict[int, UserValueSnapshot]: - ids = _ids(user_ids) - if not ids: - return {} - result: dict[int, UserValueSnapshot] = {} - for offset in range(0, len(ids), MAX_USER_BATCH): - chunk = ids[offset : offset + MAX_USER_BATCH] - clause = f"user_id IN ({', '.join('?' for _ in chunk)})" - with self._analytics_connection() as conn: - rows = conn.execute( - f""" - SELECT * - FROM user_analytics_snapshots - WHERE {clause} AND lifecycle_segment IS NOT NULL - ORDER BY user_id - """, - chunk, - ).fetchall() - for row in rows: - user_id = int(_row_value(row, "user_id")) - snapshot = _current_snapshot_from_row(row) - if snapshot is not None: - result[user_id] = snapshot - return result - - def upsert_daily_snapshot( - self, - snapshot: UserLifecycleDailySnapshot, - ) -> UserLifecycleDailySnapshot: - values = ( - snapshot.snapshot_date.isoformat(), - snapshot.user_id, - snapshot.lifecycle_segment, - snapshot.lifecycle_reason_code, - snapshot.data_quality, - utc_iso(snapshot.calculated_at), - ) - sql = """ - INSERT INTO user_lifecycle_daily_snapshots ( - snapshot_date, user_id, lifecycle_segment, - lifecycle_reason_code, data_quality, calculated_at - ) VALUES (?, ?, ?, ?, ?, ?) - ON CONFLICT(snapshot_date, user_id) DO UPDATE SET - lifecycle_segment=excluded.lifecycle_segment, - lifecycle_reason_code=excluded.lifecycle_reason_code, - data_quality=excluded.data_quality, - calculated_at=excluded.calculated_at - """ - with self._analytics_connection() as conn: - conn.execute(sql, values) - return snapshot - - def list_daily_snapshots( - self, - *, - start: date, - end: date, - user_ids: Sequence[int] | None = None, - ) -> list[UserLifecycleDailySnapshot]: - if end <= start: - raise ValueError("end must be later than start") - ids = _ids(user_ids) if user_ids is not None else None - if ids == []: - return [] - params: list[Any] = [start.isoformat(), end.isoformat()] - clause = "" - if ids: - clause = f" AND user_id IN ({','.join('?' for _ in ids)})" - params.extend(ids) - sql = f""" - SELECT * - FROM user_lifecycle_daily_snapshots - WHERE snapshot_date >= ? - AND snapshot_date < ? - {clause} - ORDER BY snapshot_date, user_id - """ - with self._analytics_connection() as conn: - rows = conn.execute(sql, params).fetchall() - return [ - UserLifecycleDailySnapshot( - snapshot_date=date.fromisoformat(str(_row_value(row, "snapshot_date"))), - user_id=int(_row_value(row, "user_id")), - lifecycle_segment=_row_value(row, "lifecycle_segment"), - lifecycle_reason_code=_row_value(row, "lifecycle_reason_code"), - data_quality=_row_value(row, "data_quality"), - calculated_at=_timestamp(_row_value(row, "calculated_at")), - ) - for row in rows - ] - def replace_lifecycle_rollups( self, day: date, @@ -974,72 +1498,11 @@ def replace_lifecycle_rollups( ) -> None: """Replace only lifecycle aggregates, preserving other daily metrics.""" - values = list(rows) - columns = ( - "rollup_date", - "metric_name", - "event_name", - "billing_mode", - "provider_id", - "model_id", - "outcome", - "error_category", - "user_state", - "value_count", - "value_sum_micro", - "updated_at", + payloads = _lifecycle_rollup_payloads( + day, list(rows), replace_transitions=replace_transitions ) - payloads = [] - for row in values: - if ( - row.rollup_date != day - or row.metric_name not in LIFECYCLE_ROLLUP_METRICS - ): - raise ValueError("invalid lifecycle rollup row") - if row.metric_name == "lifecycle_segment_count": - valid_dimensions = ( - not row.event_name and row.user_state in LIFECYCLE_SEGMENTS - ) - else: - valid_dimensions = ( - replace_transitions - and row.event_name in LIFECYCLE_SEGMENTS - and row.user_state in LIFECYCLE_SEGMENTS - and row.event_name != row.user_state - ) - unused_dimensions = ( - row.billing_mode, - row.provider_id, - row.model_id, - row.error_category, - ) - if ( - not valid_dimensions - or any(unused_dimensions) - or row.outcome not in {"complete", "partial"} - or row.value_sum_micro != 0 - ): - raise ValueError("invalid lifecycle rollup dimensions") - payloads.append( - ( - row.rollup_date.isoformat(), - row.metric_name, - row.event_name, - row.billing_mode, - row.provider_id, - row.model_id, - row.outcome, - row.error_category, - row.user_state, - row.value_count, - row.value_sum_micro, - utc_iso(row.updated_at), - ) - ) - placeholders = ", ".join(["?"] * len(columns)) - metrics = ["lifecycle_segment_count"] - if replace_transitions: - metrics.append("lifecycle_transition") + placeholders = ", ".join(["?"] * len(_ROLLUP_COLUMNS)) + metrics = _replaced_lifecycle_metrics(replace_transitions=replace_transitions) metric_placeholders = ", ".join(["?"] * len(metrics)) with self._analytics_connection() as conn: conn.execute( @@ -1053,58 +1516,67 @@ def replace_lifecycle_rollups( if payloads: conn.executemany( f""" - INSERT INTO analytics_daily_rollups ({', '.join(columns)}) + INSERT INTO analytics_daily_rollups ({', '.join(_ROLLUP_COLUMNS)}) VALUES ({placeholders}) """, payloads, ) - def list_expiring_daily_dates( - self, - *, - before: date, - limit: int, - ) -> list[date]: + def list_expiring_fact_dates(self, *, before: date, limit: int) -> list[date]: + """Distinct dates older than ``before`` in either user-keyed table.""" if not isinstance(before, date) or isinstance(before, datetime): raise ValueError("before must be a date") page_size = positive_limit(limit, maximum=1000) sql = """ - SELECT DISTINCT snapshot_date - FROM user_lifecycle_daily_snapshots - WHERE snapshot_date < ? + SELECT snapshot_date FROM ( + SELECT DISTINCT snapshot_date FROM user_daily_facts + WHERE snapshot_date < ? + UNION + SELECT DISTINCT snapshot_date FROM lifecycle_transitions + WHERE snapshot_date < ? + ) AS expiring ORDER BY snapshot_date LIMIT ? """ with self._analytics_connection() as conn: - rows = conn.execute(sql, (before.isoformat(), page_size)).fetchall() + rows = conn.execute( + sql, (before.isoformat(), before.isoformat(), page_size) + ).fetchall() return [ date.fromisoformat(str(_row_value(row, "snapshot_date"))) for row in rows ] - def delete_daily_snapshots_for_date(self, day: date) -> int: + def expire_facts_for_date(self, day: date) -> int: + """Delete one day's facts and transitions in one transaction. + + Both tables or neither: a transition log that outlived its day's + facts would be counted again by any later rollup of that day. + """ if not isinstance(day, date) or isinstance(day, datetime): raise ValueError("day must be a date") with self._analytics_connection() as conn: - cursor = conn.execute( - """ - DELETE FROM user_lifecycle_daily_snapshots - WHERE snapshot_date = ? - """, - (day.isoformat(),), - ) - return max(0, int(cursor.rowcount)) + deleted = 0 + for table in ("user_daily_facts", "lifecycle_transitions"): + cursor = conn.execute( + f"DELETE FROM {table} WHERE snapshot_date = ?", + (day.isoformat(),), + ) + deleted += max(0, int(cursor.rowcount)) + return deleted - def has_daily_before(self, before: date) -> bool: + def has_facts_before(self, before: date) -> bool: if not isinstance(before, date) or isinstance(before, datetime): raise ValueError("before must be a date") sql = """ - SELECT 1 - FROM user_lifecycle_daily_snapshots - WHERE snapshot_date < ? + SELECT 1 FROM user_daily_facts WHERE snapshot_date < ? + UNION ALL + SELECT 1 FROM lifecycle_transitions WHERE snapshot_date < ? LIMIT 1 """ with self._analytics_connection() as conn: - row = conn.execute(sql, (before.isoformat(),)).fetchone() + row = conn.execute( + sql, (before.isoformat(), before.isoformat()) + ).fetchone() return row is not None def record_activity( @@ -1189,7 +1661,7 @@ def list_activity( ``None`` is the daily job's shape: one statement over the whole population, parameterised by nothing. A sequence is batched by - ``MAX_USER_BATCH`` the way ``list_current_snapshots`` already is. + ``MAX_USER_BATCH``. """ result: dict[int, UserActivity] = {} if user_ids is None: @@ -1220,16 +1692,23 @@ def list_activity( def seed_activity_from_snapshots(self, *, now: datetime) -> int: """Copy ``activated_at`` / ``last_meaningful_activity_at`` from the legacy row. - Design SS11: ``user_analytics_snapshots`` is the only table that knows - when an existing user first activated, and it is dropped in PR B. - Ingestion only maintains ``user_activity`` for events arriving after - this deploy, so without this copy every pre-existing activation date - would vanish on the night of the drop. + Design SS11: ``user_analytics_snapshots`` was the only table that + knew when an existing user first activated, and + ``run_startup_migrations`` drops it. Ingestion only maintains + ``user_activity`` for events arriving after the deploy that created + it, so without this copy every pre-existing activation date would + vanish with the drop. Idempotent by construction -- the same MIN/MAX upsert - ``record_activity`` uses -- so PR B can re-run it immediately before - the drop (the legacy row keeps being maintained by the throttled - repair between the two PRs). Returns the number of rows touched. + ``record_activity`` uses -- so it runs immediately before the drop on + every boot, including one where the legacy row moved since the last. + Returns the number of rows touched. + + Returns 0 once the table is gone: ``run_startup_migrations`` drops it + right after this read and runs on every boot, and a fresh install's + DDL never creates it. Absence is read from the catalogue, never from + an error message; any failure (a legacy row missing the columns read + here) still raises. """ stamp = utc_iso(_utc(now, "now")) sql = """ @@ -1258,9 +1737,48 @@ def seed_activity_from_snapshots(self, *, now: datetime) -> int: updated_at = excluded.updated_at """ with self._analytics_connection() as conn: + if not _sqlite_table_exists(conn, "user_analytics_snapshots"): + return 0 cursor = conn.execute(sql, (stamp,)) return max(0, int(cursor.rowcount)) + def list_legacy_lifecycle_history(self) -> LegacyLifecycleHistory: + """The legacy lifecycle days no fact row and no lifecycle rollup covers. + + Read by the startup migration immediately before the drop, so those + days survive it as anonymous rollups. Empty once the table is gone + (every boot after the drop, and a fresh install); any other failure + raises, which skips the drop. + """ + metrics = sorted(LIFECYCLE_ROLLUP_METRICS) + with self._analytics_connection() as conn: + if not _sqlite_table_exists(conn, "user_lifecycle_daily_snapshots"): + return LegacyLifecycleHistory() + day_rows = conn.execute( + _LEGACY_UNROLLED_DATES_SQL.format( + metrics=", ".join("?" for _ in metrics) + ), + metrics, + ).fetchall() + days = tuple( + date.fromisoformat(str(_row_value(row, "snapshot_date"))) + for row in day_rows + ) + if not days: + return LegacyLifecycleHistory() + read_dates = _legacy_read_dates(days) + rows = conn.execute( + _LEGACY_SNAPSHOTS_SQL.format( + date_clause=( + f"snapshot_date IN ({', '.join('?' for _ in read_dates)})" + ) + ), + read_dates, + ).fetchall() + return LegacyLifecycleHistory( + days=days, snapshots=tuple(_legacy_snapshot(row) for row in rows) + ) + def sum_recent_facts( self, user_ids: Sequence[int] | None, @@ -1294,6 +1812,153 @@ def sum_recent_facts( for row in rows } + # -- Fact-table reads (design §6.10). One statement each, never per user. -- + + def count_states_for_date(self, day: date) -> list[FactStateCount]: + params = (_validate_day(day),) + with self._analytics_connection() as conn: + rows = conn.execute(_COUNT_STATES_SQL.format(p="?"), params).fetchall() + return [_fact_state_count(row) for row in rows] + + def count_segments_by_date( + self, *, start: date, end: date + ) -> list[SegmentDayCount]: + params = _validate_date_window(start, end) + with self._analytics_connection() as conn: + rows = conn.execute(_COUNT_SEGMENTS_SQL.format(p="?"), params).fetchall() + return [_segment_day_count(row) for row in rows] + + def count_transitions(self, *, start: date, end: date) -> list[TransitionCount]: + params = _validate_date_window(start, end) + with self._analytics_connection() as conn: + rows = conn.execute( + _COUNT_TRANSITIONS_SQL.format(p="?"), params + ).fetchall() + return [_transition_count(row) for row in rows] + + def list_attention_candidates( + self, *, day: date, window_start: date, limit: int + ) -> list[AttentionCandidate]: + day_iso = _validate_day(day) + if not isinstance(window_start, date) or isinstance(window_start, datetime): + raise ValueError("window_start must be a date") + if window_start > day: + raise ValueError("window_start must not be after day") + params = (window_start.isoformat(), day_iso, day_iso, positive_limit(limit)) + with self._analytics_connection() as conn: + rows = conn.execute(_ATTENTION_SQL.format(p="?"), params).fetchall() + return [_attention_candidate(row) for row in rows] + + def list_users_page( + self, + *, + query: UserPageQuery, + day: date, + limit: int, + offset: int, + ) -> tuple[list[UserPageRow], int]: + """One filtered, sorted, paginated page of users plus the total. + + Two statements, both in SQL, neither parameterised by a user id: the + Python loop this replaces loaded every eligible user and every + snapshot on every page load. + """ + page_size, start = _validate_page_window(limit, offset) + page_sql, count_sql, page_params, count_params = _users_page_sql( + query, day=day, limit=page_size, offset=start, p="?" + ) + with self._analytics_connection() as conn: + rows = conn.execute(page_sql, page_params).fetchall() + total_row = conn.execute(count_sql, count_params).fetchone() + return [_user_page_row(row) for row in rows], int( + _row_value(total_row, "total", 0) or 0 + ) + + def latest_fact_date(self, *, on_or_before: date) -> date | None: + """The newest ``snapshot_date`` at or before the bound; one MAX() read. + + ``/users`` joins yesterday's facts; before the night's job lands (or + while it fails) this names the day it can join instead. + """ + with self._analytics_connection() as conn: + row = conn.execute( + _LATEST_FACT_DATE_SQL.format(p="?"), (_validate_day(on_or_before),) + ).fetchone() + return _latest_fact_date(row) + + def sum_facts_by_user_group( + self, *, start: date, end: date + ) -> dict[str, GroupFactTotals]: + params = _validate_date_window(start, end) + with self._analytics_connection() as conn: + totals = conn.execute(_GROUP_TOTALS_SQL.format(p="?"), params).fetchall() + repeats = conn.execute(_GROUP_REPEAT_SQL.format(p="?"), params).fetchall() + return _group_totals(totals, repeats) + + def list_active_dates( + self, user_ids: Sequence[int], *, start: date, end: date + ) -> dict[int, list[date]]: + chunks = _id_chunks(user_ids) + window = _validate_date_window(start, end) + result: dict[int, list[date]] = { + user_id: [] for chunk in chunks for user_id in chunk + } + with self._analytics_connection() as conn: + for chunk in chunks: + user_clause = f" AND user_id IN ({', '.join('?' for _ in chunk)})" + rows = conn.execute( + _ACTIVE_DATES_SQL.format(p="?", user_clause=user_clause), + [*window, *chunk], + ).fetchall() + for row in rows: + result[int(_row_value(row, "user_id"))].append( + date.fromisoformat(str(_row_value(row, "snapshot_date"))) + ) + return result + + def list_activations( + self, *, start: datetime, end: datetime, include_internal: bool + ) -> dict[int, datetime]: + window_start, window_end = _validate_window(start, end) + sql = _ACTIVATIONS_SQL.format( + p="?", internal="" if include_internal else _INTERNAL_EXCLUSION_SQL + ) + with self._analytics_connection() as conn: + rows = conn.execute( + sql, (utc_iso(window_start), utc_iso(window_end)) + ).fetchall() + return { + int(_row_value(row, "user_id")): _timestamp(_row_value(row, "activated_at")) + for row in rows + } + + def count_activated_users(self, *, include_internal: bool) -> int: + sql = _COUNT_ACTIVATED_SQL.format( + internal="" if include_internal else _INTERNAL_EXCLUSION_SQL + ) + with self._analytics_connection() as conn: + row = conn.execute(sql).fetchone() + return int(_row_value(row, "activated", 0) or 0) + + def list_user_transitions( + self, user_id: int, *, start: date, end: date + ) -> list[LifecycleTransitionRow]: + subject = positive_user_id(user_id) + window = _validate_date_window(start, end) + with self._analytics_connection() as conn: + rows = conn.execute( + _USER_TRANSITIONS_SQL.format(p="?"), (subject, *window) + ).fetchall() + return [_transition_row(row) for row in rows] + + def list_transitions_for_date(self, day: date) -> list[LifecycleTransitionRow]: + params = (_validate_day(day),) + with self._analytics_connection() as conn: + rows = conn.execute( + _TRANSITIONS_FOR_DATE_SQL.format(p="?"), params + ).fetchall() + return [_transition_row(row) for row in rows] + def list_commercial_values( self, user_ids: Sequence[int], @@ -1786,12 +2451,83 @@ def append_lifecycle_transitions( conn.executemany(sql, [_transition_values(row) for row in rows]) return len(rows) + def replace_lifecycle_transitions( + self, day: date, rows: Sequence[LifecycleTransitionRow] + ) -> int: + """Make ``day``'s transition log exactly ``rows``, in one transaction. + + The daily job's writer. ``append_lifecycle_transitions`` upserts the + users whose segment moved and touches nobody else, so a recompute + that finds a user back on the previous day's segment kept the first + run's row forever -- and ``rollup_lifecycle_day`` bakes that row into + the permanent rollups at expiry. The day is recomputed whole, so its + log is replaced whole -- for the accounts the job evaluates. Rows of + an admin or excluded account are left in place: the read filter only + hides them, so an exclusion stays reversible. + """ + day_iso = _validate_day(day) + if any(row.snapshot_date != day for row in rows): + raise ValueError("every transition must belong to day") + with self._analytics_connection() as conn: + conn.execute(_REPLACE_TRANSITIONS_DELETE_SQL.format(p="?"), (day_iso,)) + if rows: + conn.executemany( + _UPSERT_TRANSITIONS_SQL.format(p="?"), + [_transition_values(row) for row in rows], + ) + return len(rows) + + def requeue_days_for_recompute(self, days: Sequence[date]) -> int: + """Put ``days`` back on ``list_days_needing_recompute``. One statement. + + A day leaves that list once its facts' ``calculated_at`` is newer than + its newest event's ``received_at``, and the facts upsert sets it + before the transitions step runs. A late-arrival recompute that then + fails would never be retried. Resetting the stamp to the epoch (it is + read by nothing but the recompute scan) makes the day due again. + """ + values = sorted({_validate_day(day) for day in days}) + if not values: + return 0 + sql = _REQUEUE_RECOMPUTE_SQL.format(p="?", days=", ".join("?" for _ in values)) + with self._analytics_connection() as conn: + cursor = conn.execute(sql, [_RECOMPUTE_EPOCH, *values]) + return max(0, int(cursor.rowcount)) + + def backfill_transitions_from_history(self, *, now: datetime) -> int: + """Derive the transition log for every fact day that has none. + + A startup-migration step, run between the legacy rollup and the drop + (``facts_migration.run_startup_migrations``), and only while + ``user_lifecycle_daily_snapshots`` still exists: it is a one-off for + the PR A history copy's days, not a second writer beside the job. + A day the job already wrote a log for is left alone; a day it wrote + none for (no one moved) derives none again, from the same facts. + Returns the rows inserted. + """ + stamp = utc_iso(_utc(now, "now")) + with self._analytics_connection() as conn: + if not _sqlite_table_exists(conn, "user_lifecycle_daily_snapshots"): + return 0 + days = [ + str(_row_value(row, "snapshot_date")) + for row in conn.execute(_DAYS_WITHOUT_TRANSITIONS_SQL).fetchall() + ] + if not days: + return 0 + inserted = 0 + for sql, params in _backfill_transition_statements( + p="?", prev="date(cur.snapshot_date, '-1 day')", days=days + ): + cursor = conn.execute(sql, [stamp, *params]) + inserted += max(0, int(cursor.rowcount)) + return inserted + def list_facts_for_date(self, day: date) -> list[UserDailyFact]: """Every fact row for one date. Used for the previous day's segments.""" with self._analytics_connection() as conn: rows = conn.execute( - "SELECT * FROM user_daily_facts WHERE snapshot_date = ? ORDER BY user_id", - (day.isoformat(),), + _FACTS_FOR_DATE_SQL.format(p="?"), (_validate_day(day),) ).fetchall() return [_fact_from_row(row) for row in rows] @@ -1823,38 +2559,6 @@ def list_days_needing_recompute(self, *, since: date, until: date) -> list[date] ).fetchall() return _recompute_days(event_rows, fact_rows) - def copy_daily_snapshot_history(self, *, since: date, now: datetime) -> int: - """Copy legacy ``user_lifecycle_daily_snapshots`` rows into the fact table. - - Run, cost and tier columns are filled with their neutral values and - every copied row is ``partial``, so the UI labels the period - "Incomplete data" instead of charting zeros as fact (design SS6.7). - ``tier='unpaid'`` is a placeholder, not a claim, for the same reason. - ``user_group`` is read from ``users`` at the copy (D9). ``ON CONFLICT - DO NOTHING`` makes it both idempotent and unable to clobber a row the - daily job already computed. Returns the number of rows inserted. - """ - stamp = utc_iso(_utc(now, "now")) - sql = """ - INSERT INTO user_daily_facts ( - snapshot_date, user_id, lifecycle_segment, lifecycle_reason_code, - operational_state, operational_reason_code, tier, user_group, active, - runs_requested, runs_completed, runs_failed, runs_cancelled, - operator_cost_micro, own_spend_micro, data_quality, calculated_at - ) - SELECT s.snapshot_date, s.user_id, s.lifecycle_segment, - s.lifecycle_reason_code, 'healthy', NULL, 'unpaid', - users.user_group, 0, 0, 0, 0, 0, 0, 0, 'partial', ? - FROM user_lifecycle_daily_snapshots AS s - JOIN users ON users.id = s.user_id - WHERE s.snapshot_date >= ? - ON CONFLICT(snapshot_date, user_id) DO NOTHING - """ - with self._analytics_connection() as conn: - cursor = conn.execute(sql, (stamp, since.isoformat())) - return max(0, int(cursor.rowcount)) - - def build_value_analytics_store( analytics_base: Any | None = None, credits_base: Any | None = None, @@ -1893,17 +2597,22 @@ def build_value_analytics_store( __all__ = [ + "AttentionCandidate", "CommercialValueFact", "CurrentOperationalFacts", "ProjectionJob", "ActivityUpdate", "DayEventTotals", + "FactStateCount", + "GroupFactTotals", "RecentFactTotals", + "SegmentDayCount", + "TransitionCount", + "UserPageQuery", + "UserPageRow", "LifecycleTransitionRow", "UserActivity", "UserDailyFact", - "UserLifecycleDailySnapshot", - "UserValueSnapshot", "ValueAnalyticsStore", "build_value_analytics_store", ] diff --git a/dashboard/backend/domain/analytics/value_repository_postgres.py b/dashboard/backend/domain/analytics/value_repository_postgres.py index a753f5781..051cbf4f1 100644 --- a/dashboard/backend/domain/analytics/value_repository_postgres.py +++ b/dashboard/backend/domain/analytics/value_repository_postgres.py @@ -10,33 +10,72 @@ from __future__ import annotations -import json from datetime import date, datetime, timedelta, timezone from typing import Any, Literal, Mapping, Sequence from .repository_common import positive_limit, positive_user_id, utc_iso from .value_repository import ( + _ACTIVATIONS_SQL, + _ACTIVE_DATES_SQL, _ACTIVE_RUN_STATUSES, _ACTIVITY_EVENT_NAMES, + _ATTENTION_SQL, + _COUNT_ACTIVATED_SQL, + _COUNT_SEGMENTS_SQL, + _COUNT_STATES_SQL, + _COUNT_TRANSITIONS_SQL, _DAILY_FACT_COLUMNS, _DAILY_FACT_UPDATES, + _DAYS_WITHOUT_TRANSITIONS_SQL, _EVENTS_FOR_DAY_SQL, + _FACTS_FOR_DATE_SQL, + _GROUP_REPEAT_SQL, + _GROUP_TOTALS_SQL, + _INTERNAL_EXCLUSION_SQL, + _LATEST_FACT_DATE_SQL, + _LEGACY_SNAPSHOTS_SQL, + _LEGACY_UNROLLED_DATES_SQL, + _RECOMPUTE_EPOCH, _RECOMPUTE_EVENTS_SQL, _RECOMPUTE_FACTS_SQL, + _REPLACE_TRANSITIONS_DELETE_SQL, + _REQUEUE_RECOMPUTE_SQL, _TERMINAL_RUN_STATUSES, _TRANSITION_COLUMNS, _TRANSITION_UPDATES, + _TRANSITIONS_FOR_DATE_SQL, + _UPSERT_TRANSITIONS_SQL, + _USER_TRANSITIONS_SQL, _RECENT_FACTS_SQL, _activity_update_values, + _backfill_transition_statements, + _attention_candidate, _day_bounds_iso, _events_totals_from_row, _fact_from_row, + _fact_state_count, _fact_values, + _group_totals, + _id_chunks, + _latest_fact_date, + _legacy_read_dates, + _legacy_snapshot, + _segment_day_count, + _transition_count, + _transition_row, + _user_page_row, + _users_page_sql, + _validate_date_window, + _validate_page_window, + _validate_day, _recompute_days, _transition_values, _activity_from_row, _population_operational_signals, _recent_totals_from_row, + _ROLLUP_COLUMNS, + _lifecycle_rollup_payloads, + _replaced_lifecycle_metrics, LIFECYCLE_ROLLUP_METRICS, LIFECYCLE_SEGMENTS, MAX_USER_BATCH, @@ -44,17 +83,21 @@ CommercialValueFact, CurrentOperationalFacts, ActivityUpdate, + AttentionCandidate, DayEventTotals, + FactStateCount, + GroupFactTotals, + LegacyLifecycleHistory, LifecycleTransitionRow, ProjectionJob, RecentFactTotals, + SegmentDayCount, + TransitionCount, UserActivity, UserDailyFact, - UserLifecycleDailySnapshot, - UserValueSnapshot, - _current_snapshot_from_row, + UserPageQuery, + UserPageRow, _ids, - _legacy_seed, _object_value, _optional_timestamp, _projection_job_name, @@ -71,6 +114,13 @@ ) +def _postgres_table_exists(cur: Any, table: str) -> bool: + """Locale-proof: ``to_regclass`` answers NULL for a missing relation, so no + error message (whose text follows ``lc_messages``) is ever parsed.""" + cur.execute("SELECT to_regclass(%s) IS NOT NULL AS present", (table,)) + return bool(_row_value(cur.fetchone(), "present")) + + class PostgresValueAnalyticsStore: """PostgreSQL value projection storage. @@ -129,186 +179,6 @@ def __init__( def _analytics_connection(self): return self.analytics_base._get_connection() - def upsert_current_snapshot(self, snapshot: UserValueSnapshot) -> UserValueSnapshot: - def evidence(value: Sequence[str]) -> str: - return json.dumps( - list(value), - separators=(",", ":"), - ensure_ascii=True, - ) - - legacy_status, legacy_reason_code, legacy_reason = _legacy_seed(snapshot) - values = ( - snapshot.user_id, - legacy_status, - legacy_reason_code, - legacy_reason, - "[]", - snapshot.lifecycle_segment, - snapshot.lifecycle_reason_code, - snapshot.lifecycle_reason, - evidence(snapshot.lifecycle_evidence), - snapshot.operational_state, - snapshot.operational_reason_code, - snapshot.operational_reason, - evidence(snapshot.operational_evidence), - utc_iso(snapshot.activated_at) if snapshot.activated_at else None, - ( - utc_iso(snapshot.last_meaningful_activity_at) - if snapshot.last_meaningful_activity_at - else None - ), - snapshot.inactive_days, - snapshot.active_days_30d, - snapshot.successful_backtests_30d, - utc_iso(snapshot.calculated_at), - ) - columns = """ - user_id, status, reason_code, human_readable_reason, - evidence_event_ids_json, lifecycle_segment, lifecycle_reason_code, - lifecycle_reason, lifecycle_evidence_json, operational_state, - operational_reason_code, operational_reason, - operational_evidence_json, activated_at, - last_meaningful_activity_at, inactive_days, active_days_30d, - successful_backtests_30d, calculated_at - """ - updates = """ - lifecycle_segment=excluded.lifecycle_segment, - lifecycle_reason_code=excluded.lifecycle_reason_code, - lifecycle_reason=excluded.lifecycle_reason, - lifecycle_evidence_json=excluded.lifecycle_evidence_json, - operational_state=excluded.operational_state, - operational_reason_code=excluded.operational_reason_code, - operational_reason=excluded.operational_reason, - operational_evidence_json=excluded.operational_evidence_json, - activated_at=excluded.activated_at, - last_meaningful_activity_at=excluded.last_meaningful_activity_at, - inactive_days=excluded.inactive_days, - active_days_30d=excluded.active_days_30d, - successful_backtests_30d=excluded.successful_backtests_30d, - calculated_at=excluded.calculated_at - """ - with self._analytics_connection() as conn: - with conn.cursor() as cur: - cur.execute( - f""" - INSERT INTO user_analytics_snapshots ({columns}) - VALUES ({", ".join(["%s"] * len(values))}) - ON CONFLICT(user_id) DO UPDATE SET {updates} - """, - values, - ) - return snapshot - - def get_current_snapshot(self, user_id: int) -> UserValueSnapshot | None: - subject = positive_user_id(user_id) - with self._analytics_connection() as conn: - with conn.cursor() as cur: - cur.execute( - "SELECT * FROM user_analytics_snapshots WHERE user_id=%s", - (subject,), - ) - row = cur.fetchone() - return _current_snapshot_from_row(row) - - def list_current_snapshots( - self, - user_ids: Sequence[int], - ) -> dict[int, UserValueSnapshot]: - ids = _ids(user_ids) - if not ids: - return {} - result: dict[int, UserValueSnapshot] = {} - for offset in range(0, len(ids), MAX_USER_BATCH): - chunk = ids[offset : offset + MAX_USER_BATCH] - with self._analytics_connection() as conn: - with conn.cursor() as cur: - cur.execute( - """ - SELECT * - FROM user_analytics_snapshots - WHERE user_id = ANY(%s) AND lifecycle_segment IS NOT NULL - ORDER BY user_id - """, - [chunk], - ) - rows = cur.fetchall() - for row in rows: - user_id = int(_row_value(row, "user_id")) - snapshot = _current_snapshot_from_row(row) - if snapshot is not None: - result[user_id] = snapshot - return result - - def upsert_daily_snapshot( - self, - snapshot: UserLifecycleDailySnapshot, - ) -> UserLifecycleDailySnapshot: - values = ( - snapshot.snapshot_date.isoformat(), - snapshot.user_id, - snapshot.lifecycle_segment, - snapshot.lifecycle_reason_code, - snapshot.data_quality, - utc_iso(snapshot.calculated_at), - ) - sql = """ - INSERT INTO user_lifecycle_daily_snapshots ( - snapshot_date, user_id, lifecycle_segment, - lifecycle_reason_code, data_quality, calculated_at - ) VALUES (%s, %s, %s, %s, %s, %s) - ON CONFLICT(snapshot_date, user_id) DO UPDATE SET - lifecycle_segment=excluded.lifecycle_segment, - lifecycle_reason_code=excluded.lifecycle_reason_code, - data_quality=excluded.data_quality, - calculated_at=excluded.calculated_at - """ - with self._analytics_connection() as conn: - with conn.cursor() as cur: - cur.execute(sql, values) - return snapshot - - def list_daily_snapshots( - self, - *, - start: date, - end: date, - user_ids: Sequence[int] | None = None, - ) -> list[UserLifecycleDailySnapshot]: - if end <= start: - raise ValueError("end must be later than start") - ids = _ids(user_ids) if user_ids is not None else None - if ids == []: - return [] - params: list[Any] = [start.isoformat(), end.isoformat()] - clause = "" - if ids: - clause = " AND user_id = ANY(%s)" - params.append(ids) - sql = f""" - SELECT * - FROM user_lifecycle_daily_snapshots - WHERE snapshot_date >= %s - AND snapshot_date < %s - {clause} - ORDER BY snapshot_date, user_id - """ - with self._analytics_connection() as conn: - with conn.cursor() as cur: - cur.execute(sql, params) - rows = cur.fetchall() - return [ - UserLifecycleDailySnapshot( - snapshot_date=date.fromisoformat(str(_row_value(row, "snapshot_date"))), - user_id=int(_row_value(row, "user_id")), - lifecycle_segment=_row_value(row, "lifecycle_segment"), - lifecycle_reason_code=_row_value(row, "lifecycle_reason_code"), - data_quality=_row_value(row, "data_quality"), - calculated_at=_timestamp(_row_value(row, "calculated_at")), - ) - for row in rows - ] - def replace_lifecycle_rollups( self, day: date, @@ -318,72 +188,11 @@ def replace_lifecycle_rollups( ) -> None: """Replace only lifecycle aggregates, preserving other daily metrics.""" - values = list(rows) - columns = ( - "rollup_date", - "metric_name", - "event_name", - "billing_mode", - "provider_id", - "model_id", - "outcome", - "error_category", - "user_state", - "value_count", - "value_sum_micro", - "updated_at", + payloads = _lifecycle_rollup_payloads( + day, list(rows), replace_transitions=replace_transitions ) - payloads = [] - for row in values: - if ( - row.rollup_date != day - or row.metric_name not in LIFECYCLE_ROLLUP_METRICS - ): - raise ValueError("invalid lifecycle rollup row") - if row.metric_name == "lifecycle_segment_count": - valid_dimensions = ( - not row.event_name and row.user_state in LIFECYCLE_SEGMENTS - ) - else: - valid_dimensions = ( - replace_transitions - and row.event_name in LIFECYCLE_SEGMENTS - and row.user_state in LIFECYCLE_SEGMENTS - and row.event_name != row.user_state - ) - unused_dimensions = ( - row.billing_mode, - row.provider_id, - row.model_id, - row.error_category, - ) - if ( - not valid_dimensions - or any(unused_dimensions) - or row.outcome not in {"complete", "partial"} - or row.value_sum_micro != 0 - ): - raise ValueError("invalid lifecycle rollup dimensions") - payloads.append( - ( - row.rollup_date.isoformat(), - row.metric_name, - row.event_name, - row.billing_mode, - row.provider_id, - row.model_id, - row.outcome, - row.error_category, - row.user_state, - row.value_count, - row.value_sum_micro, - utc_iso(row.updated_at), - ) - ) - placeholders = ", ".join(["%s"] * len(columns)) - metrics = ["lifecycle_segment_count"] - if replace_transitions: - metrics.append("lifecycle_transition") + placeholders = ", ".join(["%s"] * len(_ROLLUP_COLUMNS)) + metrics = _replaced_lifecycle_metrics(replace_transitions=replace_transitions) metric_placeholders = ", ".join(["%s"] * len(metrics)) with self._analytics_connection() as conn: with conn.cursor() as cur: @@ -398,63 +207,64 @@ def replace_lifecycle_rollups( if payloads: cur.executemany( f""" - INSERT INTO analytics_daily_rollups ({', '.join(columns)}) + INSERT INTO analytics_daily_rollups ({', '.join(_ROLLUP_COLUMNS)}) VALUES ({placeholders}) """, payloads, ) - def list_expiring_daily_dates( - self, - *, - before: date, - limit: int, - ) -> list[date]: + def list_expiring_fact_dates(self, *, before: date, limit: int) -> list[date]: + """See the SQLite twin.""" if not isinstance(before, date) or isinstance(before, datetime): raise ValueError("before must be a date") page_size = positive_limit(limit, maximum=1000) sql = """ - SELECT DISTINCT snapshot_date - FROM user_lifecycle_daily_snapshots - WHERE snapshot_date < %s + SELECT snapshot_date FROM ( + SELECT DISTINCT snapshot_date FROM user_daily_facts + WHERE snapshot_date < %s + UNION + SELECT DISTINCT snapshot_date FROM lifecycle_transitions + WHERE snapshot_date < %s + ) AS expiring ORDER BY snapshot_date LIMIT %s """ with self._analytics_connection() as conn: with conn.cursor() as cur: - cur.execute(sql, (before.isoformat(), page_size)) + cur.execute(sql, (before.isoformat(), before.isoformat(), page_size)) rows = cur.fetchall() return [ date.fromisoformat(str(_row_value(row, "snapshot_date"))) for row in rows ] - def delete_daily_snapshots_for_date(self, day: date) -> int: + def expire_facts_for_date(self, day: date) -> int: + """See the SQLite twin. One connection, so one transaction.""" if not isinstance(day, date) or isinstance(day, datetime): raise ValueError("day must be a date") with self._analytics_connection() as conn: with conn.cursor() as cur: - cur.execute( - """ - DELETE FROM user_lifecycle_daily_snapshots - WHERE snapshot_date = %s - RETURNING user_id - """, - (day.isoformat(),), - ) - return len(cur.fetchall()) + deleted = 0 + for table in ("user_daily_facts", "lifecycle_transitions"): + cur.execute( + f"DELETE FROM {table} WHERE snapshot_date = %s RETURNING user_id", + (day.isoformat(),), + ) + deleted += len(cur.fetchall()) + return deleted - def has_daily_before(self, before: date) -> bool: + def has_facts_before(self, before: date) -> bool: + """See the SQLite twin.""" if not isinstance(before, date) or isinstance(before, datetime): raise ValueError("before must be a date") sql = """ - SELECT 1 - FROM user_lifecycle_daily_snapshots - WHERE snapshot_date < %s + SELECT 1 FROM user_daily_facts WHERE snapshot_date < %s + UNION ALL + SELECT 1 FROM lifecycle_transitions WHERE snapshot_date < %s LIMIT 1 """ with self._analytics_connection() as conn: with conn.cursor() as cur: - cur.execute(sql, (before.isoformat(),)) + cur.execute(sql, (before.isoformat(), before.isoformat())) row = cur.fetchone() return row is not None @@ -547,7 +357,8 @@ def list_activity( return result def seed_activity_from_snapshots(self, *, now: datetime) -> int: - """See the SQLite twin.""" + """See the SQLite twin. Absence is ``to_regclass`` IS NULL, which + resolves through ``search_path`` and reads no localized message text.""" stamp = utc_iso(_utc(now, "now")) sql = """ INSERT INTO user_activity ( @@ -576,9 +387,39 @@ def seed_activity_from_snapshots(self, *, now: datetime) -> int: """ with self._analytics_connection() as conn: with conn.cursor() as cur: + if not _postgres_table_exists(cur, "user_analytics_snapshots"): + return 0 cur.execute(sql, (stamp,)) return max(0, int(cur.rowcount)) + def list_legacy_lifecycle_history(self) -> LegacyLifecycleHistory: + """See the SQLite twin.""" + metrics = sorted(LIFECYCLE_ROLLUP_METRICS) + with self._analytics_connection() as conn: + with conn.cursor() as cur: + if not _postgres_table_exists(cur, "user_lifecycle_daily_snapshots"): + return LegacyLifecycleHistory() + cur.execute( + _LEGACY_UNROLLED_DATES_SQL.format( + metrics=", ".join("%s" for _ in metrics) + ), + metrics, + ) + days = tuple( + date.fromisoformat(str(_row_value(row, "snapshot_date"))) + for row in cur.fetchall() + ) + if not days: + return LegacyLifecycleHistory() + cur.execute( + _LEGACY_SNAPSHOTS_SQL.format(date_clause="snapshot_date = ANY(%s)"), + (_legacy_read_dates(days),), + ) + rows = cur.fetchall() + return LegacyLifecycleHistory( + days=days, snapshots=tuple(_legacy_snapshot(row) for row in rows) + ) + def sum_recent_facts( self, user_ids: Sequence[int] | None, @@ -607,6 +448,172 @@ def sum_recent_facts( for row in rows } + # -- Fact-table reads (design §6.10). One statement each, never per user. -- + + def count_states_for_date(self, day: date) -> list[FactStateCount]: + """See the SQLite twin.""" + params = (_validate_day(day),) + with self._analytics_connection() as conn: + with conn.cursor() as cur: + cur.execute(_COUNT_STATES_SQL.format(p="%s"), params) + rows = cur.fetchall() + return [_fact_state_count(row) for row in rows] + + def count_segments_by_date( + self, *, start: date, end: date + ) -> list[SegmentDayCount]: + """See the SQLite twin.""" + params = _validate_date_window(start, end) + with self._analytics_connection() as conn: + with conn.cursor() as cur: + cur.execute(_COUNT_SEGMENTS_SQL.format(p="%s"), params) + rows = cur.fetchall() + return [_segment_day_count(row) for row in rows] + + def count_transitions(self, *, start: date, end: date) -> list[TransitionCount]: + """See the SQLite twin.""" + params = _validate_date_window(start, end) + with self._analytics_connection() as conn: + with conn.cursor() as cur: + cur.execute(_COUNT_TRANSITIONS_SQL.format(p="%s"), params) + rows = cur.fetchall() + return [_transition_count(row) for row in rows] + + def list_attention_candidates( + self, *, day: date, window_start: date, limit: int + ) -> list[AttentionCandidate]: + """See the SQLite twin.""" + day_iso = _validate_day(day) + if not isinstance(window_start, date) or isinstance(window_start, datetime): + raise ValueError("window_start must be a date") + if window_start > day: + raise ValueError("window_start must not be after day") + params = (window_start.isoformat(), day_iso, day_iso, positive_limit(limit)) + with self._analytics_connection() as conn: + with conn.cursor() as cur: + cur.execute(_ATTENTION_SQL.format(p="%s"), params) + rows = cur.fetchall() + return [_attention_candidate(row) for row in rows] + + def list_users_page( + self, + *, + query: UserPageQuery, + day: date, + limit: int, + offset: int, + ) -> tuple[list[UserPageRow], int]: + """See the SQLite twin.""" + page_size, start = _validate_page_window(limit, offset) + page_sql, count_sql, page_params, count_params = _users_page_sql( + query, day=day, limit=page_size, offset=start, p="%s" + ) + with self._analytics_connection() as conn: + with conn.cursor() as cur: + cur.execute(page_sql, page_params) + rows = cur.fetchall() + cur.execute(count_sql, count_params) + total_row = cur.fetchone() + return [_user_page_row(row) for row in rows], int( + _row_value(total_row, "total", 0) or 0 + ) + + def latest_fact_date(self, *, on_or_before: date) -> date | None: + """See the SQLite twin.""" + with self._analytics_connection() as conn: + with conn.cursor() as cur: + cur.execute( + _LATEST_FACT_DATE_SQL.format(p="%s"), (_validate_day(on_or_before),) + ) + row = cur.fetchone() + return _latest_fact_date(row) + + def sum_facts_by_user_group( + self, *, start: date, end: date + ) -> dict[str, GroupFactTotals]: + """See the SQLite twin.""" + params = _validate_date_window(start, end) + with self._analytics_connection() as conn: + with conn.cursor() as cur: + cur.execute(_GROUP_TOTALS_SQL.format(p="%s"), params) + totals = cur.fetchall() + cur.execute(_GROUP_REPEAT_SQL.format(p="%s"), params) + repeats = cur.fetchall() + return _group_totals(totals, repeats) + + def list_active_dates( + self, user_ids: Sequence[int], *, start: date, end: date + ) -> dict[int, list[date]]: + """See the SQLite twin.""" + chunks = _id_chunks(user_ids) + window = _validate_date_window(start, end) + result: dict[int, list[date]] = { + user_id: [] for chunk in chunks for user_id in chunk + } + with self._analytics_connection() as conn: + with conn.cursor() as cur: + for chunk in chunks: + cur.execute( + _ACTIVE_DATES_SQL.format( + p="%s", user_clause=" AND user_id = ANY(%s)" + ), + [*window, chunk], + ) + for row in cur.fetchall(): + result[int(_row_value(row, "user_id"))].append( + date.fromisoformat(str(_row_value(row, "snapshot_date"))) + ) + return result + + def list_activations( + self, *, start: datetime, end: datetime, include_internal: bool + ) -> dict[int, datetime]: + """See the SQLite twin.""" + window_start, window_end = _validate_window(start, end) + sql = _ACTIVATIONS_SQL.format( + p="%s", internal="" if include_internal else _INTERNAL_EXCLUSION_SQL + ) + with self._analytics_connection() as conn: + with conn.cursor() as cur: + cur.execute(sql, (utc_iso(window_start), utc_iso(window_end))) + rows = cur.fetchall() + return { + int(_row_value(row, "user_id")): _timestamp(_row_value(row, "activated_at")) + for row in rows + } + + def count_activated_users(self, *, include_internal: bool) -> int: + """See the SQLite twin.""" + sql = _COUNT_ACTIVATED_SQL.format( + internal="" if include_internal else _INTERNAL_EXCLUSION_SQL + ) + with self._analytics_connection() as conn: + with conn.cursor() as cur: + cur.execute(sql) + row = cur.fetchone() + return int(_row_value(row, "activated", 0) or 0) + + def list_user_transitions( + self, user_id: int, *, start: date, end: date + ) -> list[LifecycleTransitionRow]: + """See the SQLite twin.""" + subject = positive_user_id(user_id) + window = _validate_date_window(start, end) + with self._analytics_connection() as conn: + with conn.cursor() as cur: + cur.execute(_USER_TRANSITIONS_SQL.format(p="%s"), (subject, *window)) + rows = cur.fetchall() + return [_transition_row(row) for row in rows] + + def list_transitions_for_date(self, day: date) -> list[LifecycleTransitionRow]: + """See the SQLite twin.""" + params = (_validate_day(day),) + with self._analytics_connection() as conn: + with conn.cursor() as cur: + cur.execute(_TRANSITIONS_FOR_DATE_SQL.format(p="%s"), params) + rows = cur.fetchall() + return [_transition_row(row) for row in rows] + def aggregate_events_for_day(self, day: date) -> dict[int, DayEventTotals]: """See the SQLite twin.""" start, end = _day_bounds_iso(day) @@ -694,14 +701,62 @@ def append_lifecycle_transitions( cur.executemany(sql, [_transition_values(row) for row in rows]) return len(rows) + def replace_lifecycle_transitions( + self, day: date, rows: Sequence[LifecycleTransitionRow] + ) -> int: + """See the SQLite twin.""" + day_iso = _validate_day(day) + if any(row.snapshot_date != day for row in rows): + raise ValueError("every transition must belong to day") + with self._analytics_connection() as conn: + with conn.cursor() as cur: + cur.execute(_REPLACE_TRANSITIONS_DELETE_SQL.format(p="%s"), (day_iso,)) + if rows: + cur.executemany( + _UPSERT_TRANSITIONS_SQL.format(p="%s"), + [_transition_values(row) for row in rows], + ) + return len(rows) + + def requeue_days_for_recompute(self, days: Sequence[date]) -> int: + """See the SQLite twin.""" + values = sorted({_validate_day(day) for day in days}) + if not values: + return 0 + sql = _REQUEUE_RECOMPUTE_SQL.format(p="%s", days=", ".join("%s" for _ in values)) + with self._analytics_connection() as conn: + with conn.cursor() as cur: + cur.execute(sql, [_RECOMPUTE_EPOCH, *values]) + return max(0, int(cur.rowcount)) + + def backfill_transitions_from_history(self, *, now: datetime) -> int: + """See the SQLite twin.""" + stamp = utc_iso(_utc(now, "now")) + with self._analytics_connection() as conn: + with conn.cursor() as cur: + if not _postgres_table_exists(cur, "user_lifecycle_daily_snapshots"): + return 0 + cur.execute(_DAYS_WITHOUT_TRANSITIONS_SQL) + days = [ + str(_row_value(row, "snapshot_date")) for row in cur.fetchall() + ] + if not days: + return 0 + inserted = 0 + for sql, params in _backfill_transition_statements( + p="%s", + prev="to_char(cur.snapshot_date::date - 1, 'YYYY-MM-DD')", + days=days, + ): + cur.execute(sql, [stamp, *params]) + inserted += max(0, int(cur.rowcount)) + return inserted + def list_facts_for_date(self, day: date) -> list[UserDailyFact]: """See the SQLite twin.""" with self._analytics_connection() as conn: with conn.cursor() as cur: - cur.execute( - "SELECT * FROM user_daily_facts WHERE snapshot_date = %s ORDER BY user_id", - (day.isoformat(),), - ) + cur.execute(_FACTS_FOR_DATE_SQL.format(p="%s"), (_validate_day(day),)) rows = cur.fetchall() return [_fact_from_row(row) for row in rows] @@ -722,29 +777,6 @@ def list_days_needing_recompute(self, *, since: date, until: date) -> list[date] fact_rows = cur.fetchall() return _recompute_days(event_rows, fact_rows) - def copy_daily_snapshot_history(self, *, since: date, now: datetime) -> int: - """See the SQLite twin. ``active`` is BOOLEAN here.""" - stamp = utc_iso(_utc(now, "now")) - sql = """ - INSERT INTO user_daily_facts ( - snapshot_date, user_id, lifecycle_segment, lifecycle_reason_code, - operational_state, operational_reason_code, tier, user_group, active, - runs_requested, runs_completed, runs_failed, runs_cancelled, - operator_cost_micro, own_spend_micro, data_quality, calculated_at - ) - SELECT s.snapshot_date, s.user_id, s.lifecycle_segment, - s.lifecycle_reason_code, 'healthy', NULL, 'unpaid', - users.user_group, FALSE, 0, 0, 0, 0, 0, 0, 'partial', %s - FROM user_lifecycle_daily_snapshots AS s - JOIN users ON users.id = s.user_id - WHERE s.snapshot_date >= %s - ON CONFLICT(snapshot_date, user_id) DO NOTHING - """ - with self._analytics_connection() as conn: - with conn.cursor() as cur: - cur.execute(sql, (stamp, since.isoformat())) - return max(0, int(cur.rowcount)) - def list_operational_signals( self, user_ids: Sequence[int], diff --git a/dashboard/backend/tests/domain/analytics/test_backfill.py b/dashboard/backend/tests/domain/analytics/test_backfill.py index 78f37d4e1..e3ebdf488 100644 --- a/dashboard/backend/tests/domain/analytics/test_backfill.py +++ b/dashboard/backend/tests/domain/analytics/test_backfill.py @@ -195,7 +195,6 @@ def test_backfill_uses_only_authoritative_candidates(tmp_path): now=NOW, source=source, store=store, - recalculate_snapshot=lambda *_args, **_kwargs: None, rebuild_rollup=lambda *_args, **_kwargs: None, ) events = store.list_user_events(1, limit=50)["items"] @@ -222,7 +221,6 @@ def test_backfill_ids_are_deterministic_and_idempotent(tmp_path): now=NOW, source=source, store=store, - recalculate_snapshot=lambda *_args, **_kwargs: None, rebuild_rollup=lambda *_args, **_kwargs: None, ) first_event_id = store.list_user_events(1, limit=10)["items"][0].event_id @@ -230,7 +228,6 @@ def test_backfill_ids_are_deterministic_and_idempotent(tmp_path): now=NOW, source=source, store=store, - recalculate_snapshot=lambda *_args, **_kwargs: None, rebuild_rollup=lambda *_args, **_kwargs: None, ) second_event_id = store.list_user_events(1, limit=10)["items"][0].event_id @@ -271,7 +268,6 @@ def test_backfill_treats_existing_live_source_event_as_duplicate(tmp_path): ] ), store=store, - recalculate_snapshot=lambda *_args, **_kwargs: None, rebuild_rollup=lambda *_args, **_kwargs: None, ) @@ -282,7 +278,6 @@ def test_backfill_treats_existing_live_source_event_as_duplicate(tmp_path): def test_backfill_enforces_window_dry_run_and_bulk_repairs(tmp_path): store = _analytics_store(tmp_path) - snapshots = [] rollup_days = [] source = StaticSource( [ @@ -317,14 +312,12 @@ def test_backfill_enforces_window_dry_run_and_bulk_repairs(tmp_path): source=source, store=store, dry_run=True, - recalculate_snapshot=lambda user_id, **_kwargs: snapshots.append(user_id), rebuild_rollup=lambda day, **_kwargs: rollup_days.append(day), ) written = backfill_analytics( now=NOW, source=source, store=store, - recalculate_snapshot=lambda user_id, **_kwargs: snapshots.append(user_id), rebuild_rollup=lambda day, **_kwargs: rollup_days.append(day), ) @@ -333,7 +326,7 @@ def test_backfill_enforces_window_dry_run_and_bulk_repairs(tmp_path): assert written.inserted == 2 assert written.skipped_outside_window == 1 assert written.skipped_unmapped_owner == 2 - assert snapshots == [1, 2] + assert written.affected_users == 2 assert rollup_days == [(NOW - timedelta(days=1)).date()] diff --git a/dashboard/backend/tests/domain/analytics/test_daily_facts.py b/dashboard/backend/tests/domain/analytics/test_daily_facts.py index 3413853ae..b65156c4b 100644 --- a/dashboard/backend/tests/domain/analytics/test_daily_facts.py +++ b/dashboard/backend/tests/domain/analytics/test_daily_facts.py @@ -447,3 +447,159 @@ def test_the_retention_coordinator_runs_on_every_tick(daily_fixture): daily_fixture.run(now=NOW + timedelta(minutes=5)) # idle tick assert daily_fixture.retention.calls == 2 + + +def test_a_recompute_that_undoes_a_move_removes_its_transition(daily_fixture): + """Review #2: the writer replaces the day's log. Upserting only the users + who moved kept the first run's transition for a user the recompute put + back on the previous day's segment -- and rollup_lifecycle_day bakes the + log into the permanent rollups at expiry.""" + daily_fixture.seed_previous_segment("growing") + daily_fixture.run() + assert [row["to_segment"] for row in daily_fixture.transitions_for(daily_fixture.at_risk_id)] == [ + "at_risk" + ] + + # The previous day turns out to have been at_risk too (its own recompute), + # and a late event makes D due for one. + daily_fixture.seed_previous_segment("at_risk") + daily_fixture.append_event( + daily_fixture.active_id, + event_name="backtest_completed", + occurred_at=datetime(2026, 9, 11, 23, 59, tzinfo=timezone.utc), + received_at=NOW + timedelta(minutes=5), + index="late-undo", + ) + report = daily_fixture.run(now=NOW + timedelta(days=1)) + + assert D in report.recomputed_dates + assert [ + row for row in daily_fixture.transitions_for(daily_fixture.at_risk_id) + if row["snapshot_date"] == D.isoformat() + ] == [] + + +def _fact_row(user_id, day, segment, **overrides): + values = dict( + snapshot_date=day, + user_id=user_id, + lifecycle_segment=segment, + lifecycle_reason_code="growing_activated_below_core_threshold", + operational_state="healthy", + operational_reason_code=None, + tier="unpaid", + user_group="unknown", + active=False, + data_quality="complete", + calculated_at=NOW, + ) + values.update(overrides) + return UserDailyFact(**values) + + +def test_recomputing_a_day_rederives_the_next_days_log(daily_fixture): + """A transition's from_segment is the previous day's segment, so a + recomputed day must rebuild its successor's log even though the + successor's facts did not change.""" + from dashboard.backend.domain.analytics.daily_facts import ( + _rederive_next_day_transitions, + ) + from dashboard.backend.domain.analytics.value_repository import LifecycleTransitionRow + + store = daily_fixture.store + user_id = daily_fixture.at_risk_id + next_day = D + timedelta(days=1) + store.upsert_daily_facts([_fact_row(user_id, next_day, "at_risk")]) + store.append_lifecycle_transitions( + [ + LifecycleTransitionRow( + user_id=user_id, + snapshot_date=next_day, + from_segment="growing", + to_segment="at_risk", + inactive_days=8, + created_at=NOW, + ) + ] + ) + + # D now reads at_risk as well: the move on D+1 never happened. + assert _rederive_next_day_transitions( + D, [_fact_row(user_id, D, "at_risk")], store=store, now=NOW + ) == 0 + assert store.list_transitions_for_date(next_day) == [] + + # D reads new: D+1's move is new -> at_risk. + assert _rederive_next_day_transitions( + D, [_fact_row(user_id, D, "new")], store=store, now=NOW + ) == 1 + [row] = store.list_transitions_for_date(next_day) + assert (row.from_segment, row.to_segment) == ("new", "at_risk") + + +def test_rederive_keeps_inactive_days_from_the_later_days_own_evaluation(daily_fixture): + from dashboard.backend.domain.analytics.daily_facts import ( + _rederive_next_day_transitions, + ) + from dashboard.backend.domain.analytics.value_repository import LifecycleTransitionRow + + store = daily_fixture.store + user_id = daily_fixture.at_risk_id + next_day = D + timedelta(days=1) + store.upsert_daily_facts([_fact_row(user_id, next_day, "at_risk")]) + store.append_lifecycle_transitions( + [ + LifecycleTransitionRow( + user_id=user_id, + snapshot_date=next_day, + from_segment="growing", + to_segment="at_risk", + inactive_days=8, + created_at=NOW, + ) + ] + ) + + _rederive_next_day_transitions(D, [_fact_row(user_id, D, "core")], store=store, now=NOW) + + [row] = store.list_transitions_for_date(next_day) + assert (row.from_segment, row.inactive_days) == ("core", 8) + + +def test_rederive_is_a_no_op_when_the_next_day_has_no_facts(daily_fixture): + from dashboard.backend.domain.analytics.daily_facts import ( + _rederive_next_day_transitions, + ) + + assert _rederive_next_day_transitions( + D, [_fact_row(daily_fixture.at_risk_id, D, "new")], store=daily_fixture.store, now=NOW + ) == 0 + + +def test_a_failed_recompute_stays_due_until_it_succeeds(daily_fixture): + """The facts upsert refreshes calculated_at before later steps run, and a + day whose facts are newer than its newest event leaves the recompute list. + A failed late-arrival recompute is therefore re-queued explicitly, or its + partial rows would stand for good.""" + daily_fixture.run() + daily_fixture.append_event( + daily_fixture.active_id, + event_name="backtest_completed", + occurred_at=datetime(2026, 9, 11, 23, 59, tzinfo=timezone.utc), + received_at=NOW + timedelta(minutes=5), + index="late-failing", + ) + daily_fixture.break_step("ledger") + + failed = daily_fixture.run(now=NOW + timedelta(days=1)) + + assert D not in failed.recomputed_dates + assert daily_fixture.fact_for(daily_fixture.active_id).data_quality == "partial" + assert daily_fixture.store.list_days_needing_recompute(since=D, until=D) == [D] + + daily_fixture.repair_step("ledger") + retried = daily_fixture.run(now=NOW + timedelta(days=1, minutes=5)) + + assert D in retried.recomputed_dates + assert daily_fixture.fact_for(daily_fixture.active_id).data_quality == "complete" + assert daily_fixture.store.list_days_needing_recompute(since=D, until=D) == [] diff --git a/dashboard/backend/tests/domain/analytics/test_daily_job.py b/dashboard/backend/tests/domain/analytics/test_daily_job.py index ccea126e7..6bbdc24cc 100644 --- a/dashboard/backend/tests/domain/analytics/test_daily_job.py +++ b/dashboard/backend/tests/domain/analytics/test_daily_job.py @@ -109,8 +109,51 @@ def test_startup_starts_the_worker_and_leaves_the_reaper_to_heartbeats(): assert source.count("start_daily_facts_worker()") == 1 assert "register_reaper_sweep(analytics_retention_coordinator.run_if_due)" not in source assert "register_reaper_sweep(run_daily_facts" not in source - # The throttled snapshot repairs stay on the reaper until PR B deletes them. - assert "register_reaper_sweep(run_analytics_maintenance)" in source + # PR B deleted the throttled snapshot repairs along with maintenance.py. + assert "run_analytics_maintenance" not in source + + +def test_the_worker_is_the_only_caller_of_the_startup_migration(): + """Review #11: app.py ran the migration synchronously inside the async + startup hook -- blocking the event loop for the seed, the legacy rollup and + up to a 5 s lock wait on the drop -- and the worker then ran it again. The + worker's prepare step is now the one caller.""" + import ast + import textwrap + + tree = ast.parse(textwrap.dedent(inspect.getsource(app_module.startup_event))) + called = { + inner.func.id + for inner in ast.walk(tree) + if isinstance(inner, ast.Call) and isinstance(inner.func, ast.Name) + } + assert "run_startup_migrations" not in called + assert "start_daily_facts_worker" in called + assert "run_startup_migrations" in inspect.getsource(daily_job._default_prepare) + + +def test_a_failing_prepare_is_retried_before_each_tick_until_it_succeeds(capsys): + """The drop can lose its lock to the outgoing instance on a deploy; one + failure must not leave the legacy tables in place until the next boot.""" + stop = threading.Event() + calls: list[str] = [] + + def prepare(): + calls.append("prepare") + if calls.count("prepare") < 3: + raise RuntimeError("lock timeout") + + def tick(): + calls.append("tick") + if calls.count("tick") >= 4: + stop.set() + + daily_job.start_daily_facts_worker(0.01, stop, tick=tick, prepare=prepare).join(5) + + # prepare (fails), prepare (fails) + tick, prepare (succeeds) + tick, tick, tick. + assert calls[:5] == ["prepare", "prepare", "tick", "prepare", "tick"] + assert calls.count("prepare") == 3 + assert capsys.readouterr().out.count("analytics.facts_migration_failed") == 2 def test_the_job_owns_the_retention_coordinator_now(): diff --git a/dashboard/backend/tests/domain/analytics/test_fact_reads.py b/dashboard/backend/tests/domain/analytics/test_fact_reads.py new file mode 100644 index 000000000..b6bf1d7b2 --- /dev/null +++ b/dashboard/backend/tests/domain/analytics/test_fact_reads.py @@ -0,0 +1,626 @@ +"""Set-based reads over user_daily_facts, lifecycle_transitions and user_activity. + +Every read here is one statement over the whole population (or one page of +it). The SQLite cases run always; ``assert_fact_read_contract`` is also driven +by the Postgres twin from test_repository_postgres.py under @pg_only. +""" + +from __future__ import annotations + +from datetime import date, datetime, timedelta, timezone + +import pytest + +from dashboard.backend.domain.analytics.repository import AnalyticsStore +from dashboard.backend.domain.analytics.value_repository import ( + MAX_USER_BATCH, + LifecycleTransitionRow, + UserDailyFact, + UserPageQuery, + ValueAnalyticsStore, +) +from dashboard.backend.users import UserStore + + +UTC = timezone.utc +NOW = datetime(2026, 9, 15, 12, 0, tzinfo=UTC) +YESTERDAY = date(2026, 9, 14) + + +def fact(user_id: int, day: date, **overrides) -> UserDailyFact: + values = dict( + snapshot_date=day, + user_id=user_id, + lifecycle_segment="growing", + lifecycle_reason_code="growing_activated_below_core_threshold", + operational_state="healthy", + operational_reason_code="no_supported_issue", + tier="unpaid", + user_group="unknown", + active=True, + runs_requested=1, + runs_completed=1, + runs_failed=0, + runs_cancelled=0, + operator_cost_micro=0, + own_spend_micro=0, + data_quality="complete", + calculated_at=datetime.combine(day + timedelta(days=1), datetime.min.time(), tzinfo=UTC), + ) + values.update(overrides) + return UserDailyFact(**values) + + +def transition(user_id: int, day: date, from_segment: str, to_segment: str, **overrides): + values = dict( + user_id=user_id, + snapshot_date=day, + from_segment=from_segment, + to_segment=to_segment, + inactive_days=0, + data_quality="complete", + created_at=datetime.combine(day + timedelta(days=1), datetime.min.time(), tzinfo=UTC), + ) + values.update(overrides) + return LifecycleTransitionRow(**values) + + +def seed_population(store, create_user, *, admin_id: int) -> dict[str, int]: + """Five users plus an admin. Returns the ids by role in the scenario. + + ``create_user(key, email, group) -> int`` makes one account on whichever + dialect is under test: the SQLite fixture goes through ``UserStore``, the + Postgres contract fixture (a stub ``users`` table with no default id) + inserts explicit ids. Every assertion below counts only this population, + so pre-seeded accounts on either tier contribute nothing. + """ + + ids = {} + for key, email, group in ( + ("blocked", "blocked@example.test", "partner"), + ("attention", "attention@example.test", "partner"), + ("core", "core@example.test", "organic"), + ("dormant", "dormant@example.test", "unknown"), + ("fresh", "fresh@example.test", "organic"), + ): + ids[key] = create_user(key, email, group) + ids["admin"] = admin_id + two_days_ago = YESTERDAY - timedelta(days=1) + store.upsert_daily_facts( + [ + fact(ids["blocked"], YESTERDAY, operational_state="blocked", + operational_reason_code="billing_lane_unavailable", + lifecycle_segment="at_risk", user_group="partner", + runs_requested=2, runs_completed=0, runs_failed=2, active=True), + fact(ids["blocked"], two_days_ago, operational_state="blocked", + operational_reason_code="billing_lane_unavailable", + lifecycle_segment="growing", user_group="partner", + runs_requested=1, runs_completed=0, runs_failed=1), + fact(ids["attention"], YESTERDAY, operational_state="needs_attention", + operational_reason_code="three_consecutive_failed_runs", + lifecycle_segment="growing", user_group="partner", + runs_requested=3, runs_completed=0, runs_failed=3, tier="starter"), + fact(ids["core"], YESTERDAY, lifecycle_segment="core", user_group="organic", + runs_requested=4, runs_completed=4, operator_cost_micro=250_000, + own_spend_micro=100_000, tier="invested"), + fact(ids["core"], two_days_ago, lifecycle_segment="growing", user_group="organic", + runs_requested=1, runs_completed=1, operator_cost_micro=50_000), + fact(ids["dormant"], YESTERDAY, lifecycle_segment="dormant", active=False, + runs_requested=0, runs_completed=0, data_quality="partial", + operational_reason_code=None), + ] + ) + store.append_lifecycle_transitions( + [ + transition(ids["blocked"], YESTERDAY, "growing", "at_risk", inactive_days=8), + transition(ids["core"], YESTERDAY, "growing", "core"), + ] + ) + for key, activated_days_ago, active_days_ago in ( + ("blocked", 20, 9), + ("core", 30, 1), + ("attention", 3, 1), + ("admin", 40, 1), + ): + store.record_activity( + ids[key], + occurred_at=NOW - timedelta(days=activated_days_ago), + activating=True, + now=NOW, + ) + store.record_activity( + ids[key], + occurred_at=NOW - timedelta(days=active_days_ago), + activating=False, + now=NOW, + ) + return ids + + +def assert_fact_read_contract(store, ids: dict[str, int]) -> None: + two_days_ago = YESTERDAY - timedelta(days=1) + + # I5: the latest landed day at or before a bound, one MAX() read. + assert store.latest_fact_date(on_or_before=YESTERDAY + timedelta(days=5)) == YESTERDAY + assert store.latest_fact_date(on_or_before=YESTERDAY) == YESTERDAY + assert store.latest_fact_date(on_or_before=two_days_ago) == two_days_ago + assert store.latest_fact_date(on_or_before=two_days_ago - timedelta(days=1)) is None + + states = store.count_states_for_date(YESTERDAY) + by_state = {} + by_reason = {} + by_segment = {} + for row in states: + by_state[row.operational_state] = by_state.get(row.operational_state, 0) + row.users + by_reason[row.operational_reason_code] = by_reason.get(row.operational_reason_code, 0) + row.users + by_segment[row.lifecycle_segment] = by_segment.get(row.lifecycle_segment, 0) + row.users + assert by_state == {"blocked": 1, "needs_attention": 1, "healthy": 2} + assert by_reason["billing_lane_unavailable"] == 1 + assert by_segment == {"at_risk": 1, "growing": 1, "core": 1, "dormant": 1} + assert any(row.partial_rows == 1 for row in states if row.lifecycle_segment == "dormant") + # A NULL reason code (history-copy rows) is a value, not an error. + assert None in by_reason + + segments = store.count_segments_by_date(start=two_days_ago, end=YESTERDAY + timedelta(days=1)) + assert {(row.snapshot_date, row.lifecycle_segment, row.users) for row in segments} == { + (two_days_ago, "growing", 2), + (YESTERDAY, "at_risk", 1), + (YESTERDAY, "growing", 1), + (YESTERDAY, "core", 1), + (YESTERDAY, "dormant", 1), + } + + transitions = store.count_transitions(start=YESTERDAY, end=YESTERDAY + timedelta(days=1)) + assert {(t.from_segment, t.to_segment, t.users) for t in transitions} == { + ("growing", "at_risk", 1), + ("growing", "core", 1), + } + + attention = store.list_attention_candidates( + day=YESTERDAY, window_start=YESTERDAY - timedelta(days=29), limit=10 + ) + assert [c.user_id for c in attention] == [ids["attention"], ids["blocked"]] + assert attention[0].recent_failures == 3 + assert attention[1].recent_failures == 3 # 2 yesterday + 1 the day before + assert attention[1].recent_runs == 3 + assert attention[1].operational_reason_code == "billing_lane_unavailable" + assert attention[0].email == "attention@example.test" + assert store.list_attention_candidates( + day=YESTERDAY, window_start=YESTERDAY - timedelta(days=29), limit=1 + )[0].user_id == ids["attention"] + + groups = store.sum_facts_by_user_group(start=two_days_ago, end=YESTERDAY + timedelta(days=1)) + assert groups["organic"].total_runs == 5 + assert groups["organic"].operator_cost_micro == 300_000 + assert groups["organic"].successful_run_users == 1 + assert groups["organic"].repeat_users == 1 # core completed on two days + assert groups["partner"].total_runs == 6 + assert groups["partner"].successful_run_users == 0 + assert groups["partner"].repeat_users == 0 + assert "internal" not in groups + # The dormant user's partial row is the window's only one (I2). + assert groups["unknown"].partial_rows == 1 + assert groups["organic"].partial_rows == 0 + assert groups["partner"].partial_rows == 0 + + active = store.list_active_dates( + [ids["core"], ids["dormant"], ids["blocked"]], + start=two_days_ago, + end=YESTERDAY + timedelta(days=1), + ) + assert active[ids["core"]] == [two_days_ago, YESTERDAY] + assert active[ids["dormant"]] == [] + assert active[ids["blocked"]] == [two_days_ago, YESTERDAY] + + activations = store.list_activations( + start=NOW - timedelta(days=35), end=NOW, include_internal=False + ) + assert set(activations) == {ids["blocked"], ids["core"], ids["attention"]} + assert activations[ids["core"]] == NOW - timedelta(days=30) + with_admin = store.list_activations( + start=NOW - timedelta(days=45), end=NOW, include_internal=True + ) + assert ids["admin"] in with_admin + + assert store.count_activated_users(include_internal=False) == 3 + assert store.count_activated_users(include_internal=True) == 4 + + mine = store.list_user_transitions( + ids["blocked"], start=two_days_ago, end=YESTERDAY + timedelta(days=1) + ) + assert [(t.from_segment, t.to_segment) for t in mine] == [("growing", "at_risk")] + assert store.list_user_transitions( + ids["blocked"], start=two_days_ago, end=YESTERDAY + ) == [] + + for_day = store.list_transitions_for_date(YESTERDAY) + assert {t.user_id for t in for_day} == {ids["blocked"], ids["core"]} + assert store.list_transitions_for_date(two_days_ago) == [] + + +def assert_read_time_exclusion_contract(store, ids: dict[str, int], *, exclude, promote) -> None: + """Review #7: excluding an account, or promoting it to admin, takes effect + on the next read of rows already written -- as it did when every read went + through ``_eligible_users`` -- not only on facts written from tomorrow. + + Asserted as deltas, so whatever else a tier's fixture seeded on the same + days contributes nothing. Mutates the population: run it last. + """ + two_days_ago = YESTERDAY - timedelta(days=1) + window = dict(start=two_days_ago, end=YESTERDAY + timedelta(days=1)) + + def state_total() -> int: + return sum(row.users for row in store.count_states_for_date(YESTERDAY)) + + def segment_total() -> int: + return sum(row.users for row in store.count_segments_by_date(**window)) + + def transition_total() -> int: + return sum(row.users for row in store.count_transitions(**window)) + + def group_runs() -> dict[str, int]: + return { + group: totals.total_runs + for group, totals in store.sum_facts_by_user_group(**window).items() + } + + states, segments, transitions, runs = ( + state_total(), segment_total(), transition_total(), group_runs() + ) + exclude(ids["blocked"]) + promote(ids["core"]) + + # blocked: rows on both days, one transition, 3 partner runs. + # core: rows on both days, one transition, 5 organic runs. + assert state_total() == states - 2 + assert segment_total() == segments - 4 + assert transition_total() == transitions - 2 + after = group_runs() + assert after.get("partner", 0) == runs["partner"] - 3 + assert after.get("organic", 0) == runs["organic"] - 5 + candidates = { + candidate.user_id + for candidate in store.list_attention_candidates( + day=YESTERDAY, window_start=YESTERDAY - timedelta(days=29), limit=10 + ) + } + assert ids["blocked"] not in candidates + assert ids["attention"] in candidates + # The expiry rollup reads these two, so the permanent record drops them too. + day_facts = {row.user_id for row in store.list_facts_for_date(YESTERDAY)} + assert ids["attention"] in day_facts + assert day_facts.isdisjoint({ids["blocked"], ids["core"]}) + assert {row.user_id for row in store.list_transitions_for_date(YESTERDAY)}.isdisjoint( + {ids["blocked"], ids["core"]} + ) + + # The job's whole-day replace removes only what the job evaluates: the + # hidden row of an excluded account survives it, so an exclusion made by + # mistake can be undone without losing that account's movement. + store.replace_lifecycle_transitions(YESTERDAY, []) + exclude(ids["blocked"], excluded=False) + assert {row.user_id for row in store.list_transitions_for_date(YESTERDAY)} == { + ids["blocked"] + } + + +def _page_query(**overrides) -> UserPageQuery: + values = dict( + q=None, + lifecycle_segment=None, + operational_state=None, + commercial_tier=None, + user_group=None, + activated=None, + last_meaningful_activity_from=None, + last_meaningful_activity_to=None, + priority=False, + include_internal=False, + ) + values.update(overrides) + return UserPageQuery(**values) + + +def assert_users_page_contract(store, ids: dict[str, int]) -> None: + rows, total = store.list_users_page( + query=_page_query(), day=YESTERDAY, limit=100, offset=0 + ) + assert total == 5 + assert [row.user_id for row in rows] == sorted( + ids[key] for key in ("blocked", "attention", "core", "dormant", "fresh") + ) + fresh = next(row for row in rows if row.user_id == ids["fresh"]) + assert fresh.lifecycle_segment is None + assert fresh.operational_state == "healthy" + assert fresh.tier == "unpaid" + assert fresh.activated_at is None + # A NULL stored reason code (history-copy row) reaches the service as healthy. + dormant = next(row for row in rows if row.user_id == ids["dormant"]) + assert dormant.operational_reason_code == "no_supported_issue" + + rows, total = store.list_users_page( + query=_page_query(include_internal=True), day=YESTERDAY, limit=100, offset=0 + ) + assert total == 6 + assert ids["admin"] in {row.user_id for row in rows} + + rows, total = store.list_users_page( + query=_page_query(), day=YESTERDAY, limit=2, offset=2 + ) + assert total == 5 + every_id = sorted( + ids[key] for key in ("blocked", "attention", "core", "dormant", "fresh") + ) + assert [row.user_id for row in rows] == every_id[2:4] + + rows, _total = store.list_users_page( + query=_page_query(priority=True), day=YESTERDAY, limit=100, offset=0 + ) + assert [row.user_id for row in rows] == [ids["blocked"], ids["attention"]] + + rows, total = store.list_users_page( + query=_page_query(user_group="partner", commercial_tier="starter"), + day=YESTERDAY, limit=100, offset=0, + ) + assert total == 1 and rows[0].user_id == ids["attention"] + + rows, total = store.list_users_page( + query=_page_query(operational_state="blocked"), day=YESTERDAY, limit=100, offset=0 + ) + assert [row.user_id for row in rows] == [ids["blocked"]] + assert rows[0].operational_reason_code == "billing_lane_unavailable" + + rows, total = store.list_users_page( + query=_page_query(lifecycle_segment="core"), day=YESTERDAY, limit=100, offset=0 + ) + assert [row.user_id for row in rows] == [ids["core"]] + + rows, total = store.list_users_page( + query=_page_query(activated=False), day=YESTERDAY, limit=100, offset=0 + ) + assert {row.user_id for row in rows} == {ids["dormant"], ids["fresh"]} + + rows, total = store.list_users_page( + query=_page_query( + last_meaningful_activity_from=NOW - timedelta(days=2), + last_meaningful_activity_to=NOW, + ), + day=YESTERDAY, limit=100, offset=0, + ) + assert {row.user_id for row in rows} == {ids["core"], ids["attention"]} + + rows, total = store.list_users_page( + query=_page_query(q="ATTEN"), day=YESTERDAY, limit=100, offset=0 + ) + assert [row.user_id for row in rows] == [ids["attention"]] + rows, total = store.list_users_page( + query=_page_query(q="%"), day=YESTERDAY, limit=100, offset=0 + ) + assert total == 0 # escaped: no literal percent sign in any email or name + + +def assert_priority_sort_contract(store, create_user) -> None: + """The priority order, pinned key by key: group, tier, inactivity, id. + + Users are created in an order that differs from the expected one, and the + lower groups carry the better tier, so no key passes by accident. Only + these users' relative order is asserted, which keeps the check valid on a + store that already holds the contract population. + """ + + # key, fact overrides, days since last meaningful activity (None = no row) + specs = [ + ("pr_onboard_high", dict(lifecycle_segment="onboarding", tier="high_value"), 1), + ("pr_at_risk", dict(lifecycle_segment="at_risk"), 1), + ("pr_attention", dict(operational_state="needs_attention", lifecycle_segment="at_risk", + operational_reason_code="three_consecutive_failed_runs"), 1), + ("pr_blocked_recent", dict(operational_state="blocked", + operational_reason_code="billing_lane_unavailable"), 1), + ("pr_blocked_tie_a", dict(operational_state="blocked", + operational_reason_code="billing_lane_unavailable"), 5), + ("pr_blocked_tie_b", dict(operational_state="blocked", + operational_reason_code="billing_lane_unavailable"), 5), + ("pr_blocked_old", dict(operational_state="blocked", + operational_reason_code="billing_lane_unavailable"), 20), + ("pr_blocked_never", dict(operational_state="blocked", + operational_reason_code="billing_lane_unavailable"), None), + ("pr_blocked_starter", dict(operational_state="blocked", tier="starter", + operational_reason_code="billing_lane_unavailable"), 1), + ("pr_blocked_high", dict(operational_state="blocked", tier="high_value", + operational_reason_code="billing_lane_unavailable"), 1), + ] + ids = {} + for key, overrides, idle_days in specs: + user_id = create_user(key, f"{key}@example.test", "unknown") + ids[key] = user_id + store.upsert_daily_facts([fact(user_id, YESTERDAY, **overrides)]) + if idle_days is not None: + store.record_activity( + user_id, occurred_at=NOW - timedelta(days=idle_days), activating=False, now=NOW + ) + assert ids["pr_blocked_tie_a"] < ids["pr_blocked_tie_b"] + + rows, _total = store.list_users_page( + query=_page_query(priority=True), day=YESTERDAY, limit=100, offset=0 + ) + + mine = {user_id: key for key, user_id in ids.items()} + assert [mine[row.user_id] for row in rows if row.user_id in mine] == [ + "pr_blocked_high", # group blocked, tier high_value + "pr_blocked_starter", # tier starter + "pr_blocked_never", # unpaid; no activity row sorts first + "pr_blocked_old", # then the longest idle + "pr_blocked_tie_a", # equal idle time falls to the lower id + "pr_blocked_tie_b", + "pr_blocked_recent", + "pr_attention", # group needs_attention, though its segment is at_risk + "pr_at_risk", + "pr_onboard_high", # last group, despite the best tier + ] + + +def _sqlite_fixture(tmp_path): + db_path = tmp_path / "fact-reads.db" + users = UserStore(db_path=db_path) + admin = users.create_user("admin@example.test", "Admin", "SecurePass1!") + users.apply_admin_patch(int(admin["id"]), role="admin") + analytics = AnalyticsStore(db_path=db_path) + store = ValueAnalyticsStore( + analytics, + credits_base=object(), + provider_base=object(), + agent_base=object(), + run_base=object(), + ) + return store, users, int(admin["id"]) + + +def _sqlite_user_factory(users): + def create_user(key: str, email: str, group: str) -> int: + user = users.create_user(email, key.title(), "SecurePass1!") + users.apply_admin_patch(int(user["id"]), user_group=group) + return int(user["id"]) + + return create_user + + +def test_sqlite_fact_reads_follow_the_contract(tmp_path): + store, users, admin_id = _sqlite_fixture(tmp_path) + ids = seed_population(store, _sqlite_user_factory(users), admin_id=admin_id) + assert_fact_read_contract(store, ids) + + +def test_fact_reads_are_one_statement_per_call(tmp_path): + """No read here may issue a per-user statement (design §6.11 rule 6).""" + + store, users, admin_id = _sqlite_fixture(tmp_path) + ids = seed_population(store, _sqlite_user_factory(users), admin_id=admin_id) + statements: list[str] = [] + original = store.analytics_base._get_connection + + from contextlib import contextmanager + + @contextmanager + def tracing_connection(): + with original() as conn: + conn.set_trace_callback( + lambda statement: statements.append(statement) + if statement.lstrip().upper().startswith("SELECT") + else None + ) + yield conn + + store.analytics_base._get_connection = tracing_connection + store.count_states_for_date(YESTERDAY) + store.count_segments_by_date(start=YESTERDAY - timedelta(days=30), end=YESTERDAY + timedelta(days=1)) + store.count_transitions(start=YESTERDAY - timedelta(days=30), end=YESTERDAY + timedelta(days=1)) + store.list_attention_candidates(day=YESTERDAY, window_start=YESTERDAY - timedelta(days=29), limit=10) + store.list_active_dates(list(ids.values()), start=YESTERDAY - timedelta(days=30), end=YESTERDAY + timedelta(days=1)) + store.list_activations(start=NOW - timedelta(days=60), end=NOW, include_internal=False) + store.count_activated_users(include_internal=False) + store.list_user_transitions(ids["core"], start=YESTERDAY - timedelta(days=30), end=YESTERDAY + timedelta(days=1)) + store.list_transitions_for_date(YESTERDAY) + assert len(statements) == 9 + statements.clear() + store.sum_facts_by_user_group(start=YESTERDAY - timedelta(days=30), end=YESTERDAY + timedelta(days=1)) + assert len(statements) == 2 + + +def test_list_active_dates_spans_more_than_one_id_batch(tmp_path): + """Retention cohorts can exceed MAX_USER_BATCH; the read chunks, not raises.""" + + store, users, admin_id = _sqlite_fixture(tmp_path) + ids = seed_population(store, _sqlite_user_factory(users), admin_id=admin_id) + padding = list(range(10_000, 10_000 + MAX_USER_BATCH + 20)) + window = dict(start=YESTERDAY - timedelta(days=1), end=YESTERDAY + timedelta(days=1)) + + result = store.list_active_dates([*padding, ids["core"]], **window) + + assert len(result) == len(padding) + 1 + assert result[ids["core"]] == [YESTERDAY - timedelta(days=1), YESTERDAY] + assert all(result[user_id] == [] for user_id in padding) + assert store.list_active_dates([], **window) == {} + + +def test_fact_reads_reject_malformed_windows(tmp_path): + store, _users, _admin_id = _sqlite_fixture(tmp_path) + with pytest.raises(ValueError): + store.count_states_for_date(datetime(2026, 9, 14, tzinfo=UTC)) + with pytest.raises(ValueError): + store.count_segments_by_date(start=YESTERDAY, end=YESTERDAY) + with pytest.raises(ValueError): + store.count_transitions(start=YESTERDAY, end=YESTERDAY - timedelta(days=1)) + with pytest.raises(ValueError): + store.list_attention_candidates( + day=YESTERDAY, window_start=YESTERDAY + timedelta(days=1), limit=5 + ) + with pytest.raises(ValueError): + store.list_attention_candidates(day=YESTERDAY, window_start=YESTERDAY, limit=0) + with pytest.raises(ValueError): + store.list_user_transitions(0, start=YESTERDAY, end=YESTERDAY + timedelta(days=1)) + with pytest.raises(ValueError): + store.list_transitions_for_date("2026-09-14") + + +def test_sqlite_users_page_follows_the_contract(tmp_path): + store, users, admin_id = _sqlite_fixture(tmp_path) + ids = seed_population(store, _sqlite_user_factory(users), admin_id=admin_id) + assert_users_page_contract(store, ids) + + +def test_sqlite_users_page_priority_sort_follows_the_contract(tmp_path): + store, users, _admin_id = _sqlite_fixture(tmp_path) + assert_priority_sort_contract(store, _sqlite_user_factory(users)) + + +def test_sqlite_fact_reads_apply_exclusions_at_read_time(tmp_path): + store, users, admin_id = _sqlite_fixture(tmp_path) + ids = seed_population(store, _sqlite_user_factory(users), admin_id=admin_id) + + assert_read_time_exclusion_contract( + store, + ids, + exclude=lambda user_id, excluded=True: store.analytics_base.set_subject_exclusion( + user_id, excluded=excluded, actor_user_id=admin_id, reason="test account" + ), + promote=lambda user_id: users.apply_admin_patch(user_id, role="admin"), + ) + + +def test_sqlite_transition_log_is_replaced_per_day(tmp_path): + """Review #2: a recompute that puts a user back on the previous day's + segment must remove the first run's transition, not keep it.""" + store, users, admin_id = _sqlite_fixture(tmp_path) + ids = seed_population(store, _sqlite_user_factory(users), admin_id=admin_id) + + assert_transition_replace_contract(store, ids) + + +def assert_transition_replace_contract(store, ids: dict[str, int]) -> None: + """Uses its own day and users, so it composes with any other contract.""" + day = YESTERDAY - timedelta(days=5) + store.append_lifecycle_transitions( + [ + transition(ids["attention"], day, "new", "onboarding"), + transition(ids["dormant"], day, "growing", "at_risk"), + ] + ) + assert {row.user_id for row in store.list_transitions_for_date(day)} == { + ids["attention"], ids["dormant"] + } + + written = store.replace_lifecycle_transitions( + day, [transition(ids["dormant"], day, "growing", "dormant", inactive_days=12)] + ) + + assert written == 1 + [kept] = store.list_transitions_for_date(day) + assert (kept.user_id, kept.to_segment, kept.inactive_days) == ( + ids["dormant"], "dormant", 12 + ) + assert store.replace_lifecycle_transitions(day, []) == 0 + assert store.list_transitions_for_date(day) == [] + with pytest.raises(ValueError): + store.replace_lifecycle_transitions( + day, [transition(ids["dormant"], day - timedelta(days=1), "new", "core")] + ) diff --git a/dashboard/backend/tests/domain/analytics/test_facts_migration.py b/dashboard/backend/tests/domain/analytics/test_facts_migration.py index 05d01bf8a..575629913 100644 --- a/dashboard/backend/tests/domain/analytics/test_facts_migration.py +++ b/dashboard/backend/tests/domain/analytics/test_facts_migration.py @@ -1,139 +1,510 @@ -"""Existing lifecycle history is carried into the fact table as labelled partial rows.""" +"""The last read of the five-state snapshot table, then its drop (design SS11).""" from __future__ import annotations +import inspect +import sqlite3 from datetime import date, datetime, timedelta, timezone +import pytest + +from dashboard.backend.domain.analytics import repository, repository_postgres from dashboard.backend.domain.analytics.facts_migration import ( - HISTORY_COPY_DAYS, StartupMigrationReport, - migrate_lifecycle_history, run_startup_migrations, ) +from dashboard.backend.domain.analytics.metrics import ACTIVATION_EVENTS from dashboard.backend.domain.analytics.repository import AnalyticsStore +from dashboard.backend.domain.analytics.rollups import AnalyticsRollupStore, DailyRollup from dashboard.backend.domain.analytics.value_repository import ( UserDailyFact, - UserLifecycleDailySnapshot, build_value_analytics_store, ) -from dashboard.backend.tests.domain.analytics.test_value_repository import ( - _value_snapshot, -) from dashboard.backend.users import UserStore NOW = datetime(2026, 9, 12, 0, 5, tzinfo=timezone.utc) +ACTIVATED_AT = datetime(2026, 3, 1, 9, 0, tzinfo=timezone.utc) +LAST_ACTIVITY_AT = datetime(2026, 9, 11, 18, 0, tzinfo=timezone.utc) +LEGACY_TABLES = ("user_analytics_snapshots", "user_lifecycle_daily_snapshots") +# The legacy daily table as main declared it (minus the users FK). +LEGACY_DAILY_DDL = """ + CREATE TABLE user_lifecycle_daily_snapshots ( + snapshot_date TEXT NOT NULL, + user_id INTEGER NOT NULL, + lifecycle_segment TEXT NOT NULL, + lifecycle_reason_code TEXT NOT NULL, + data_quality TEXT NOT NULL, + calculated_at TEXT NOT NULL, + PRIMARY KEY (snapshot_date, user_id) + ) +""" +# Two uncovered legacy days (07-11, 07-12) and one the facts already cover +# (07-13). ``(snapshot_date, user_id, lifecycle_segment, data_quality)``. +UNCOVERED_DAYS = (date(2026, 7, 11), date(2026, 7, 12)) +COVERED_DAY = date(2026, 7, 13) +LEGACY_DAILY_ROWS = ( + ("2026-07-11", 101, "new", "complete"), + ("2026-07-11", 102, "core", "complete"), + ("2026-07-12", 101, "growing", "complete"), + ("2026-07-12", 102, "core", "complete"), + ("2026-07-12", 103, "new", "partial"), + ("2026-07-13", 101, "core", "complete"), +) + + +def legacy_lifecycle_rollups(day: date) -> list[tuple]: + """What main's ``rollup_lifecycle_day`` wrote for the fixture's two days.""" + return { + UNCOVERED_DAYS[0]: [ + ("lifecycle_segment_count", "", "core", 1, "complete"), + ("lifecycle_segment_count", "", "new", 1, "complete"), + ], + UNCOVERED_DAYS[1]: [ + ("lifecycle_segment_count", "", "core", 1, "complete"), + ("lifecycle_segment_count", "", "growing", 1, "complete"), + ("lifecycle_segment_count", "", "new", 1, "partial"), + ("lifecycle_transition", "new", "growing", 1, "complete"), + ], + }[day] + + +def _rollup_shape(rows) -> list[tuple]: + return sorted( + (row.metric_name, row.event_name, row.user_state, row.value_count, row.outcome) + for row in rows + ) + + +def covering_fact(user_id: int, day: date = COVERED_DAY) -> UserDailyFact: + return UserDailyFact( + snapshot_date=day, + user_id=user_id, + lifecycle_segment="core", + lifecycle_reason_code="x", + operational_state="healthy", + tier="unpaid", + user_group="organic", + data_quality="complete", + calculated_at=NOW, + ) + +def create_legacy_snapshot_tables(analytics, rows=(), daily_rows=()) -> None: + """Recreate the legacy tables the DDL no longer declares, as prod still has them. -class MigrationFixture: - def __init__(self, tmp_path): - path = tmp_path / "migration.db" - users = UserStore(db_path=path) - self.user_id = int(users.create_user("m@example.test", "M", "SecurePass1!")["id"]) - users.apply_admin_patch(self.user_id, user_group="partner") - self.analytics = AnalyticsStore(db_path=path) - self.store = build_value_analytics_store( - self.analytics, - credits_base=object(), - provider_base=object(), - agent_base=object(), - run_base=object(), + ``rows`` holds ``(user_id, activated_at, last_meaningful_activity_at)``. + Only the columns the seed reads plus the NOT NULL ones the five-state + writer filled are declared; the seed never reads anything else. The daily + table carries main's columns, since the legacy-day rollup reads them. + """ + with analytics._get_connection() as conn: + conn.execute( + """ + CREATE TABLE user_analytics_snapshots ( + user_id INTEGER PRIMARY KEY, + status TEXT NOT NULL, + reason_code TEXT NOT NULL, + human_readable_reason TEXT NOT NULL, + evidence_event_ids_json TEXT NOT NULL DEFAULT '[]', + calculated_at TEXT NOT NULL, + activated_at TEXT, + last_meaningful_activity_at TEXT + ) + """ ) - self.legacy_dates = [ - date(2026, 8, 20), - date(2026, 9, 1), - date(2026, 9, 10), - ] - for day in self.legacy_dates + [date(2026, 6, 1)]: # June is older than 56 days - self.store.upsert_daily_snapshot( - UserLifecycleDailySnapshot( - snapshot_date=day, - user_id=self.user_id, - lifecycle_segment="growing", - lifecycle_reason_code="growing_activated_below_core_threshold", - data_quality="complete", - calculated_at=datetime.combine( - day + timedelta(days=1), datetime.min.time(), tzinfo=timezone.utc - ), - ) + conn.execute(LEGACY_DAILY_DDL) + for snapshot_date, user_id, segment, quality in daily_rows: + conn.execute( + "INSERT INTO user_lifecycle_daily_snapshots VALUES (?, ?, ?, 'x', ?, ?)", + (snapshot_date, user_id, segment, quality, NOW.isoformat()), ) - self.legacy_row_count = len(self.legacy_dates) - - def seed_complete_fact(self, day): - self.store.upsert_daily_facts( - [ - UserDailyFact( - snapshot_date=day, - user_id=self.user_id, - lifecycle_segment="core", - lifecycle_reason_code="core_repeated_value", - operational_state="healthy", - operational_reason_code=None, - tier="starter", - user_group="partner", - active=True, - runs_completed=3, - data_quality="complete", - calculated_at=NOW, - ) - ] + for user_id, activated_at, last_activity_at in rows: + conn.execute( + "INSERT INTO user_analytics_snapshots (user_id, status, reason_code, " + "human_readable_reason, calculated_at, activated_at, " + "last_meaningful_activity_at) VALUES (?, 'active', 'x', 'x', ?, ?, ?)", + ( + user_id, + NOW.isoformat(), + activated_at.isoformat() if activated_at else None, + last_activity_at.isoformat() if last_activity_at else None, + ), + ) + + +def _table_names(analytics) -> set[str]: + with analytics._get_connection() as conn: + return { + row[0] + for row in conn.execute( + "SELECT name FROM sqlite_master WHERE type = 'table'" + ).fetchall() + } + + +def _fixture(tmp_path): + path = tmp_path / "migration.db" + users = UserStore(db_path=path) + user_id = int(users.create_user("m@example.test", "M", "SecurePass1!")["id"]) + analytics = AnalyticsStore(db_path=path) + value_store = build_value_analytics_store( + analytics, + credits_base=object(), + provider_base=object(), + agent_base=object(), + run_base=object(), + ) + return user_id, analytics, value_store + + +def test_reseed_then_drop_carries_activation_across_and_removes_the_tables(tmp_path): + """Design SS11: the seed runs again immediately before the drop, because the + legacy row kept being maintained between PR A and PR B.""" + user_id, analytics, value_store = _fixture(tmp_path) + create_legacy_snapshot_tables( + analytics, [(user_id, ACTIVATED_AT, LAST_ACTIVITY_AT)] + ) + + report = run_startup_migrations(value_store=value_store, now=NOW) + + assert report == StartupMigrationReport(activity_seeded=1) + activity = value_store.get_activity(user_id) + assert activity.activated_at == ACTIVATED_AT + assert activity.last_meaningful_activity_at == LAST_ACTIVITY_AT + assert not set(LEGACY_TABLES) & _table_names(analytics) + + +def test_second_boot_after_the_drop_is_a_clean_no_op(tmp_path): + """run_startup_migrations runs twice per boot (app startup and the worker's + prepare) and on every boot after: once the table is gone the seed reads + nothing and the drop is IF EXISTS.""" + user_id, analytics, value_store = _fixture(tmp_path) + create_legacy_snapshot_tables( + analytics, [(user_id, ACTIVATED_AT, LAST_ACTIVITY_AT)] + ) + run_startup_migrations(value_store=value_store, now=NOW) + + again = run_startup_migrations(value_store=value_store, now=NOW + timedelta(hours=1)) + analytics.drop_legacy_snapshot_tables() + + assert again == StartupMigrationReport(activity_seeded=0) + assert value_store.seed_activity_from_snapshots(now=NOW) == 0 + assert value_store.get_activity(user_id).activated_at == ACTIVATED_AT + + +def test_fresh_install_boots_without_the_legacy_tables(tmp_path): + _user_id, analytics, value_store = _fixture(tmp_path) + + assert not set(LEGACY_TABLES) & _table_names(analytics) + assert run_startup_migrations(value_store=value_store, now=NOW) == ( + StartupMigrationReport(activity_seeded=0) + ) + + +def test_seed_still_raises_on_errors_other_than_a_missing_table(tmp_path): + """Only the dropped table is tolerated; a broken legacy row must stay loud.""" + _user_id, analytics, value_store = _fixture(tmp_path) + with analytics._get_connection() as conn: + # The table exists but lacks the columns the seed reads. + conn.execute("CREATE TABLE user_analytics_snapshots (user_id INTEGER PRIMARY KEY)") + + with pytest.raises(sqlite3.OperationalError, match="no such column"): + value_store.seed_activity_from_snapshots(now=NOW) + + +def test_a_failed_seed_keeps_both_legacy_tables(tmp_path): + """The drop is irreversible, so it must never run after a seed that did not + finish -- not from a ``finally``, not behind a swallowed exception.""" + _user_id, analytics, value_store = _fixture(tmp_path) + with analytics._get_connection() as conn: + # Present but lacking the columns the seed reads: a real failure, not + # the tolerated missing table. + conn.execute("CREATE TABLE user_analytics_snapshots (user_id INTEGER PRIMARY KEY)") + conn.execute( + "CREATE TABLE user_lifecycle_daily_snapshots (" + "snapshot_date TEXT NOT NULL, user_id INTEGER NOT NULL)" ) - def fact_for(self, day): - return {row.user_id: row for row in self.store.list_facts_for_date(day)}[self.user_id] + with pytest.raises(sqlite3.OperationalError, match="no such column"): + run_startup_migrations(value_store=value_store, now=NOW) + + assert set(LEGACY_TABLES) <= _table_names(analytics) + +def test_fresh_schema_declares_no_snapshot_tables(): + for ddl in (repository.ANALYTICS_SQLITE_DDL, repository_postgres.ANALYTICS_POSTGRES_DDL): + for table in LEGACY_TABLES: + assert table not in ddl + for init_schema in ( + repository.AnalyticsStore._init_schema, + repository_postgres.PostgresAnalyticsStore._init_schema, + ): + assert "user_analytics_snapshots" not in inspect.getsource(init_schema) -def test_history_is_copied_as_partial(tmp_path): - """Movement charts keep their history; the missing columns stay honest.""" - fixture = MigrationFixture(tmp_path) - copied = migrate_lifecycle_history(value_store=fixture.store, now=NOW) +def _lifecycle_rollups(analytics, day: date) -> list[tuple]: + return _rollup_shape( + AnalyticsRollupStore(analytics).list_rollups(start=day, end=day + timedelta(days=1)) + ) + + +def test_uncovered_legacy_days_are_rolled_up_before_the_drop(tmp_path): + """I1: legacy days with no fact row survive the drop as anonymous lifecycle + rollups, in the shape main's rollup_lifecycle_day wrote; a covered day is + left to its facts, and the second boot is a no-op.""" + user_id, analytics, value_store = _fixture(tmp_path) + create_legacy_snapshot_tables(analytics, daily_rows=LEGACY_DAILY_ROWS) + value_store.upsert_daily_facts([covering_fact(user_id)]) + + report = run_startup_migrations(value_store=value_store, now=NOW) + + assert report.legacy_days_rolled_up == 2 + for day in UNCOVERED_DAYS: + assert _lifecycle_rollups(analytics, day) == legacy_lifecycle_rollups(day) + assert _lifecycle_rollups(analytics, COVERED_DAY) == [] + assert not set(LEGACY_TABLES) & _table_names(analytics) + + again = run_startup_migrations(value_store=value_store, now=NOW) + assert again == StartupMigrationReport() + for day in UNCOVERED_DAYS: + assert _lifecycle_rollups(analytics, day) == legacy_lifecycle_rollups(day) + + +def test_legacy_rollup_never_overwrites_a_day_that_has_lifecycle_rollups(tmp_path): + """A day already rolled up (an earlier boot whose drop failed, or another + writer) keeps what it has; other metric families on any day are untouched.""" + user_id, analytics, value_store = _fixture(tmp_path) + create_legacy_snapshot_tables(analytics, daily_rows=LEGACY_DAILY_ROWS) + value_store.upsert_daily_facts([covering_fact(user_id)]) + existing = DailyRollup( + rollup_date=UNCOVERED_DAYS[0], + metric_name="lifecycle_segment_count", + outcome="complete", + user_state="dormant", + value_count=9, + updated_at=NOW, + ) + value_store.replace_lifecycle_rollups(UNCOVERED_DAYS[0], [existing]) + AnalyticsRollupStore(analytics).replace_day( + UNCOVERED_DAYS[1], + [ + DailyRollup( + rollup_date=UNCOVERED_DAYS[1], + metric_name="daily_active_users", + value_count=4, + updated_at=NOW, + ) + ], + ) + + report = run_startup_migrations(value_store=value_store, now=NOW) + + assert report.legacy_days_rolled_up == 1 + assert _lifecycle_rollups(analytics, UNCOVERED_DAYS[0]) == [ + ("lifecycle_segment_count", "", "dormant", 9, "complete") + ] + day_two = _lifecycle_rollups(analytics, UNCOVERED_DAYS[1]) + assert ("daily_active_users", "", "", 4, "") in day_two + # The day was rolled up (it holds daily_active_users) before the funnel's + # activation_users metric existed, so the activation backfill fills it -- + # with zeros, since this fixture recorded no raw events that day. + assert sorted(row for row in day_two if row[0] == "activation_users") == sorted( + ("activation_users", event_name, "", 0, "") + for event_name in ACTIVATION_EVENTS + ) + assert [ + row for row in day_two if row[0] not in {"daily_active_users", "activation_users"} + ] == legacy_lifecycle_rollups(UNCOVERED_DAYS[1]) + + +def test_a_failed_legacy_rollup_keeps_both_legacy_tables(tmp_path): + """The rollup is the last read of the daily table: it failing must skip the + drop exactly as a failed seed does.""" + _user_id, analytics, value_store = _fixture(tmp_path) + create_legacy_snapshot_tables( + analytics, + # A five-state label, not a lifecycle segment: the rollup refuses it. + daily_rows=[("2026-07-11", 101, "active", "complete")], + ) + + with pytest.raises(ValueError): + run_startup_migrations(value_store=value_store, now=NOW) + + assert set(LEGACY_TABLES) <= _table_names(analytics) + assert _lifecycle_rollups(analytics, UNCOVERED_DAYS[0]) == [] + + +def test_missing_legacy_table_is_detected_from_the_catalogue_not_message_text(): + """Error text follows the server's lc_messages; a non-English Postgres would + turn the tolerated missing table into a boot-time failure every boot.""" + from dashboard.backend.domain.analytics import value_repository, value_repository_postgres + + for store in ( + value_repository.ValueAnalyticsStore, + value_repository_postgres.PostgresValueAnalyticsStore, + ): + for method in ( + store.seed_activity_from_snapshots, + store.list_legacy_lifecycle_history, + store.backfill_transitions_from_history, + ): + source = inspect.getsource(method) + assert "str(exc)" not in source and "except" not in source, method + assert "_table_exists(" in source, method + assert "to_regclass" in inspect.getsource(value_repository_postgres._postgres_table_exists) + assert "sqlite_master" in inspect.getsource(value_repository._sqlite_table_exists) + + +# Review #1: the PR A history copy filled user_daily_facts for 07-29..09-22 +# and wrote no transition log, and the legacy rollup skips any day a fact row +# covers, so those days lost their movement once _history stopped deriving it +# from the legacy rows. +HISTORY_COPIED_DAYS = (date(2026, 7, 29), date(2026, 7, 30)) +HISTORY_JOB_DAYS = (date(2026, 7, 31), date(2026, 8, 1)) + + +def history_fact(user_id: int, day: date, segment: str, quality: str = "complete"): + return UserDailyFact( + snapshot_date=day, + user_id=user_id, + lifecycle_segment=segment, + lifecycle_reason_code="x", + operational_state="healthy", + tier="unpaid", + user_group="organic", + data_quality=quality, + calculated_at=NOW, + ) + + +def seed_transition_history(value_store, first: int, second: int, insert_legacy) -> None: + """A legacy day, two copied fact days without a log, two job days with one. + + ``insert_legacy(rows)`` writes ``(snapshot_date, user_id, segment, + quality)`` rows into ``user_lifecycle_daily_snapshots`` on the dialect + under test. + """ + from dashboard.backend.domain.analytics.value_repository import LifecycleTransitionRow + + insert_legacy( + [ + ("2026-07-28", first, "core", "complete"), + # Same segment on the first copied day: no move to derive. + ("2026-07-28", second, "new", "complete"), + ] + ) + copied, job = HISTORY_COPIED_DAYS, HISTORY_JOB_DAYS + value_store.upsert_daily_facts( + [ + history_fact(first, copied[0], "at_risk"), + history_fact(second, copied[0], "new"), + history_fact(first, copied[1], "at_risk"), + history_fact(second, copied[1], "growing", "partial"), + history_fact(first, job[0], "at_risk"), + history_fact(second, job[0], "growing"), + history_fact(first, job[1], "dormant"), + history_fact(second, job[1], "growing"), + ] + ) + # The job's own log for its second day, with the inactive_days only the + # job can compute; the first job day had no movement and has no log. + value_store.append_lifecycle_transitions( + [ + LifecycleTransitionRow( + user_id=first, + snapshot_date=job[1], + from_segment="at_risk", + to_segment="dormant", + inactive_days=20, + created_at=NOW, + ) + ] + ) + + +def assert_transition_history_backfilled(value_store, first: int, second: int) -> None: + def log(day): + return { + (row.user_id, row.from_segment, row.to_segment, row.inactive_days, row.data_quality) + for row in value_store.list_transitions_for_date(day) + } + + copied, job = HISTORY_COPIED_DAYS, HISTORY_JOB_DAYS + # The first copied day's predecessor exists only in the legacy table. + assert log(copied[0]) == {(first, "core", "at_risk", 0, "complete")} + assert log(copied[1]) == {(second, "new", "growing", 0, "partial")} + # A job day with no movement derives none; a job day's log is untouched. + assert log(job[0]) == set() + assert log(job[1]) == {(first, "at_risk", "dormant", 20, "complete")} + + +def test_copied_fact_days_get_their_transition_log_before_the_drop(tmp_path): + first, analytics, value_store = _fixture(tmp_path) + second = int( + UserStore(db_path=tmp_path / "migration.db").create_user( + "second@example.test", "S", "SecurePass1!" + )["id"] + ) + create_legacy_snapshot_tables(analytics) + + def insert_legacy(rows): + with analytics._get_connection() as conn: + for snapshot_date, user_id, segment, quality in rows: + conn.execute( + "INSERT INTO user_lifecycle_daily_snapshots VALUES (?, ?, ?, 'x', ?, ?)", + (snapshot_date, user_id, segment, quality, NOW.isoformat()), + ) + + seed_transition_history(value_store, first, second, insert_legacy) - assert copied == fixture.legacy_row_count - for day in fixture.legacy_dates: - row = fixture.fact_for(day) - assert row.data_quality == "partial" - assert row.lifecycle_segment == "growing" - assert row.runs_completed == 0 - assert row.operator_cost_micro == 0 - assert row.tier == "unpaid" - assert row.operational_state == "healthy" - assert row.operational_reason_code is None - assert row.user_group == "partner" # read from users at the copy (D9) - assert row.active is False - assert fixture.store.list_facts_for_date(date(2026, 6, 1)) == [] - assert (NOW.date() - timedelta(days=HISTORY_COPY_DAYS)) > date(2026, 6, 1) + report = run_startup_migrations(value_store=value_store, now=NOW) + assert report.transitions_backfilled == 2 + assert_transition_history_backfilled(value_store, first, second) + assert not set(LEGACY_TABLES) & _table_names(analytics) + # The step goes with the table: a later boot re-derives nothing. + assert run_startup_migrations(value_store=value_store, now=NOW).transitions_backfilled == 0 + assert_transition_history_backfilled(value_store, first, second) -def test_migration_is_idempotent(tmp_path): - fixture = MigrationFixture(tmp_path) - migrate_lifecycle_history(value_store=fixture.store, now=NOW) - second = migrate_lifecycle_history(value_store=fixture.store, now=NOW) +def test_a_failed_transition_backfill_skips_the_drop(tmp_path, monkeypatch): + _user_id, analytics, value_store = _fixture(tmp_path) + create_legacy_snapshot_tables(analytics) - assert second == 0 + def explode(*, now): + raise RuntimeError("backfill failed") + monkeypatch.setattr(value_store, "backfill_transitions_from_history", explode) -def test_migration_never_overwrites_a_complete_row(tmp_path): - """A day the new job already computed wins over a migrated stub.""" - fixture = MigrationFixture(tmp_path) - fixture.seed_complete_fact(date(2026, 9, 10)) + with pytest.raises(RuntimeError): + run_startup_migrations(value_store=value_store, now=NOW) + assert set(LEGACY_TABLES) <= _table_names(analytics) - migrate_lifecycle_history(value_store=fixture.store, now=NOW) - row = fixture.fact_for(date(2026, 9, 10)) - assert row.data_quality == "complete" - assert row.lifecycle_segment == "core" - assert row.tier == "starter" +def test_a_failing_drop_does_not_block_the_activation_backfill(tmp_path, monkeypatch): + """The backfill shares nothing with the drop's chain: a drop that keeps + losing its lock must not also keep the funnel unavailable. The drop's + error is still raised afterwards, so the worker retries.""" + _user_id, analytics, value_store = _fixture(tmp_path) + create_legacy_snapshot_tables(analytics) + day = NOW.date() - timedelta(days=3) + AnalyticsRollupStore(analytics).replace_day( + day, + [DailyRollup(rollup_date=day, metric_name="daily_active_users", value_count=1, updated_at=NOW)], + ) + def lock_timeout(): + raise RuntimeError("lock timeout") -def test_startup_migrations_seed_activity_and_copy_history(tmp_path): - fixture = MigrationFixture(tmp_path) - fixture.store.upsert_current_snapshot(_value_snapshot(fixture.user_id)) + monkeypatch.setattr(analytics, "drop_legacy_snapshot_tables", lock_timeout) - report = run_startup_migrations(value_store=fixture.store, now=NOW) - again = run_startup_migrations(value_store=fixture.store, now=NOW) + with pytest.raises(RuntimeError, match="lock timeout"): + run_startup_migrations(value_store=value_store, now=NOW) - assert report == StartupMigrationReport(activity_seeded=1, history_copied=3) - assert again == StartupMigrationReport(activity_seeded=1, history_copied=0) - assert fixture.store.get_activity(fixture.user_id).activated_at is not None + filled = { + row.event_name + for row in AnalyticsRollupStore(analytics).list_rollups(start=day, end=day + timedelta(days=1)) + if row.metric_name == "activation_users" + } + assert filled == set(ACTIVATION_EVENTS) diff --git a/dashboard/backend/tests/domain/analytics/test_instrumentation.py b/dashboard/backend/tests/domain/analytics/test_instrumentation.py index b10d530ca..b80bc8d19 100644 --- a/dashboard/backend/tests/domain/analytics/test_instrumentation.py +++ b/dashboard/backend/tests/domain/analytics/test_instrumentation.py @@ -2,12 +2,11 @@ from __future__ import annotations -import inspect from datetime import datetime, timezone import pytest -from dashboard.backend.domain.analytics import instrumentation, states +from dashboard.backend.domain.analytics import instrumentation from dashboard.backend.domain.analytics.models import AppendEventResult @@ -26,7 +25,6 @@ def try_record_server_event(self, **kwargs): def _service(monkeypatch) -> RecordingService: service = RecordingService() monkeypatch.setattr(instrumentation, "get_analytics_service", lambda: service) - monkeypatch.setattr(instrumentation, "_snapshot_recalculator", lambda _user_id: None) return service @@ -158,136 +156,13 @@ def test_unsupported_event_name_is_contained(monkeypatch, capsys): assert "category=ValueError" in capsys.readouterr().out -def test_stored_event_recalculates_snapshot_best_effort(monkeypatch): - _service(monkeypatch) - users = [] - monkeypatch.setattr( - instrumentation, +def test_instrumentation_carries_no_snapshot_recalculator(): + """PR B deleted the snapshot model; #522 made the recalculator inert and + this removes it, so no entry point can re-arm a per-event rebuild.""" + for name in ( + "register_snapshot_recalculator", + "_recalculate_snapshot", "_snapshot_recalculator", - lambda user_id: users.append(user_id), - ) - - instrumentation.emit_agent_event( - event_name="agent_created", - user_id=7, - agent_id="agent-1", - occurred_at=NOW, - ) - - assert users == [7] - - -def test_projecting_service_is_not_recalculated_by_instrumentation(monkeypatch): - class ProjectingService: - project_snapshots = True - - def try_record_server_event(self, **kwargs): - return AppendEventResult.model_construct(event=None, created=True) - - users = [] - monkeypatch.setattr( - instrumentation, - "get_analytics_service", - lambda: ProjectingService(), - ) - monkeypatch.setattr( - instrumentation, - "_snapshot_recalculator", - lambda user_id: users.append(user_id), - ) - - instrumentation.emit_agent_event( - event_name="agent_created", - user_id=7, - agent_id="agent-1", - occurred_at=NOW, - ) - - assert users == [] - - -def test_snapshot_failure_never_escapes_or_logs_exception_text( - monkeypatch, - capsys, -): - _service(monkeypatch) - - def fail(_user_id): - raise RuntimeError("snapshot-secret-canary") - - monkeypatch.setattr(instrumentation, "_snapshot_recalculator", fail) - - instrumentation.emit_agent_event( - event_name="agent_created", - user_id=7, - agent_id="agent-1", - occurred_at=NOW, - ) - - output = capsys.readouterr().out - assert "snapshot-secret-canary" not in output - assert "category=RuntimeError" in output - - -class NonProjectingService: - project_snapshots = False - - def try_record_server_event(self, **kwargs): - return AppendEventResult.model_construct(event=None, created=True) - - -def _forbid_snapshot_rebuild(monkeypatch) -> None: - """Make every function in states.py raise, and the recalculator unset. - - Patching every function, not only recalculate_user_snapshots, keeps the - guard independent of how a restored fallback would spell its call: a lazy - import, a module-level import, or a different entry point into states. - """ - - def forbidden(*_args, **_kwargs): - raise AssertionError("synchronous snapshot rebuild") - - for name, value in vars(states).items(): - if inspect.isfunction(value) and value.__module__ == states.__name__: - monkeypatch.setattr(states, name, forbidden) - monkeypatch.setattr(instrumentation, name, forbidden, raising=False) - monkeypatch.setattr(instrumentation, "_snapshot_recalculator", None) - - -@pytest.mark.parametrize("event_name", sorted(instrumentation.SNAPSHOT_RELEVANT_EVENTS)) -def test_unregistered_recalculator_is_inert(monkeypatch, event_name): - """Called directly, because _emit swallows exceptions (#522).""" - _forbid_snapshot_rebuild(monkeypatch) - - instrumentation._recalculate_snapshot(7, event_name) - - -def test_unregistered_recalculator_never_rebuilds_snapshots(monkeypatch, capsys): - """The dashboard backtest child's exact state (#522). - - Nothing registers a recalculator in any process, and the live analytics - singleton does not project. Before #522 that combination fell back to - states.recalculate_user_snapshots on every snapshot-relevant event -- - ~5s each, ~31s of every model call. _emit swallows the AssertionError a - fallback would raise, so the test reads its WARNING line instead. - """ - _forbid_snapshot_rebuild(monkeypatch) - monkeypatch.setattr(instrumentation, "get_analytics_service", NonProjectingService) - - for event_name in ("credits_reserved", "credits_settled", "model_usage_recorded"): - instrumentation.emit_resource_event( - event_name=event_name, - user_id=7, - source_record_type="llm_reservation", - source_record_id=f"reservation-{event_name}", - properties={}, - occurred_at=NOW, - ) - instrumentation.emit_agent_event( - event_name="agent_created", - user_id=7, - agent_id="agent-1", - occurred_at=NOW, - ) - - assert "instrumentation_failed" not in capsys.readouterr().out + "SNAPSHOT_RELEVANT_EVENTS", + ): + assert not hasattr(instrumentation, name) diff --git a/dashboard/backend/tests/domain/analytics/test_lifecycle.py b/dashboard/backend/tests/domain/analytics/test_lifecycle.py index af687ea80..d03513020 100644 --- a/dashboard/backend/tests/domain/analytics/test_lifecycle.py +++ b/dashboard/backend/tests/domain/analytics/test_lifecycle.py @@ -16,6 +16,9 @@ calculate_operational_state, commercial_tier, is_lifecycle_activity, + operational_reason, + operational_result_from_code, + resolve_group_badge, ) from dashboard.backend.domain.analytics.models import ( EVENT_GROUP_BY_NAME, @@ -330,3 +333,67 @@ def test_rule_calculations_reject_naive_calculation_times(): ) with pytest.raises(ValueError, match="activated_at must include a timezone"): activation_cohort_week(datetime(2026, 9, 3, 12, 0)) + + +@pytest.mark.parametrize( + "role,user_group,tier,expected", + [ + ("admin", "internal", "high_value", "admin"), + ("admin", "unknown", "unpaid", "admin"), + ("user", "partner", "unpaid", "partner"), + ("user", "competition", "invested", "competition"), + ("user", "unknown", "starter", "paid"), + ("user", "unknown", "high_value", "paid"), + ("user", "unknown", "unpaid", "free"), + ], +) +def test_group_badge_precedence(role, user_group, tier, expected): + assert resolve_group_badge(role=role, user_group=user_group, tier=tier) == expected + + +def test_operational_reason_text_is_the_rule_table_copy(): + assert operational_reason("account_restricted") == ( + "The Credits account is restricted from model spending." + ) + assert operational_reason("no_supported_issue") == ( + "No supported current operational issue was detected." + ) + + +def test_unknown_operational_reason_code_is_display_safe(): + """A code the rule table does not know must not raise on a read path.""" + + text = operational_reason("synthetic_unknown_code") + assert text == "An operational issue was recorded." + assert "synthetic" not in text + + +def test_operational_result_from_code_carries_static_evidence(): + as_of = datetime(2026, 9, 14, 0, 0, tzinfo=timezone.utc) + blocked = operational_result_from_code("billing_lane_unavailable", as_of=as_of) + attention = operational_result_from_code( + "three_consecutive_failed_runs", as_of=as_of + ) + healthy = operational_result_from_code("no_supported_issue", as_of=as_of) + + assert blocked.state == "blocked" + assert blocked.evidence == ("Usable billing lanes: 0.",) + assert attention.state == "needs_attention" + # No static evidence line exists for this code; the reason text stands in. + assert attention.evidence == ( + "At least three consecutive terminal runs failed within 24 hours.", + ) + assert healthy.state == "healthy" + assert healthy.calculated_at == as_of + + +def test_history_copy_rows_with_no_code_rebuild_as_healthy(): + """History-copy rows store state 'healthy' with a NULL code (plan amendment, Task 1).""" + + as_of = datetime(2026, 9, 14, 0, 0, tzinfo=timezone.utc) + result = operational_result_from_code(None, as_of=as_of) + + assert result.state == "healthy" + assert result.reason_code == "no_supported_issue" + assert result.reason == "No supported current operational issue was detected." + assert result.evidence == ("All supported operational checks passed.",) diff --git a/dashboard/backend/tests/domain/analytics/test_lifecycle_backfill.py b/dashboard/backend/tests/domain/analytics/test_lifecycle_backfill.py deleted file mode 100644 index f2f38c685..000000000 --- a/dashboard/backend/tests/domain/analytics/test_lifecycle_backfill.py +++ /dev/null @@ -1,336 +0,0 @@ -"""Lifecycle history reconstruction is bounded, resumable, and point-in-time.""" - -from __future__ import annotations - -import sqlite3 -from datetime import date, datetime, timedelta, timezone - -import pytest - -from dashboard.backend.domain.analytics.lifecycle_backfill import ( - BACKFILL_JOB_NAME, - LifecycleBackfillEvidence, - LifecycleBackfillSource, - LifecycleBackfillUser, - backfill_lifecycle_history, - run_lifecycle_backfill_batch, -) -from dashboard.backend.domain.analytics.repository import AnalyticsStore -from dashboard.backend.domain.analytics.service import AnalyticsService -from dashboard.backend.domain.analytics.value_repository import ( - UserLifecycleDailySnapshot, - ValueAnalyticsStore, -) - - -UTC = timezone.utc -NOW = datetime(2026, 8, 27, 12, 0, tzinfo=UTC) - - -class FakeSource: - def __init__(self): - self.users = [ - LifecycleBackfillUser( - user_id=1, - created_at=datetime(2026, 8, 13, tzinfo=UTC), - ) - ] - self.evidence = {1: []} - self.complete_from = date.min - - def list_users(self, *, after_user_id, limit): - return tuple(user for user in self.users if user.user_id > after_user_id)[ - :limit - ] - - def list_evidence(self, user_id, *, start, end): - return tuple( - item - for item in self.evidence.get(user_id, ()) - if start <= item.occurred_at < end - ) - - def quality_for(self, snapshot_date, *, required_from): - del required_from - return "complete" if snapshot_date >= self.complete_from else "partial" - - -def _store(tmp_path, user_ids=(1, 2)): - path = tmp_path / "lifecycle-backfill.db" - with sqlite3.connect(path) as conn: - conn.execute( - """ - CREATE TABLE users ( - id INTEGER PRIMARY KEY, - email TEXT NOT NULL, - display_name TEXT NOT NULL, - password_hash TEXT NOT NULL, - role TEXT NOT NULL, - created_at TEXT NOT NULL - ) - """ - ) - conn.executemany( - "INSERT INTO users VALUES (?, ?, ?, 'x', 'user', ?)", - [ - ( - user_id, - f"user-{user_id}@example.test", - f"User {user_id}", - "2026-07-01T00:00:00+00:00", - ) - for user_id in user_ids - ], - ) - return ValueAnalyticsStore( - AnalyticsStore(path), - credits_base=object(), - provider_base=object(), - agent_base=object(), - run_base=object(), - ) - - -def test_backfill_does_not_use_a_later_success_for_an_earlier_day(tmp_path): - source = FakeSource() - source.evidence[1].append( - LifecycleBackfillEvidence( - event_name="backtest_completed", - occurred_at=datetime(2026, 8, 20, 12, 0, tzinfo=UTC), - ) - ) - store = _store(tmp_path, user_ids=(1,)) - - backfill_lifecycle_history( - start=date(2026, 8, 18), - end=date(2026, 8, 22), - batch_size=50, - now=NOW, - source=source, - store=store, - ) - - snapshots = { - row.snapshot_date: row - for row in store.list_daily_snapshots( - start=date(2026, 8, 18), - end=date(2026, 8, 22), - user_ids=[1], - ) - } - assert snapshots[date(2026, 8, 19)].lifecycle_segment == "onboarding" - assert snapshots[date(2026, 8, 20)].lifecycle_segment == "growing" - - -def test_backfill_resumes_without_duplicate_daily_rows(tmp_path): - source = FakeSource() - source.users.append( - LifecycleBackfillUser( - user_id=2, - created_at=datetime(2026, 7, 2, tzinfo=UTC), - ) - ) - source.evidence[2] = [] - store = _store(tmp_path) - start = date(2026, 8, 18) - end = date(2026, 8, 20) - - first = backfill_lifecycle_history( - start=start, - end=end, - batch_size=1, - now=NOW, - source=source, - store=store, - ) - second = backfill_lifecycle_history( - start=start, - end=end, - batch_size=50, - cursor=first.next_cursor, - now=NOW, - source=source, - store=store, - ) - - rows = store.list_daily_snapshots(start=start, end=end) - assert first.complete is False - assert second.complete is True - assert len({(row.snapshot_date, row.user_id) for row in rows}) == len(rows) == 4 - - -def test_untrustworthy_source_horizon_stays_partial(tmp_path): - source = FakeSource() - source.users[0] = LifecycleBackfillUser( - user_id=1, - created_at=datetime(2026, 7, 1, tzinfo=UTC), - ) - source.complete_from = date(2026, 8, 10) - store = _store(tmp_path, user_ids=(1,)) - - report = backfill_lifecycle_history( - start=date(2026, 8, 1), - end=date(2026, 8, 12), - batch_size=100, - now=NOW, - source=source, - store=store, - ) - - assert report.partial_days == 9 - rows = store.list_daily_snapshots( - start=date(2026, 8, 1), - end=date(2026, 8, 12), - ) - assert {row.data_quality for row in rows[:9]} == {"partial"} - assert {row.data_quality for row in rows[9:]} == {"complete"} - - -def test_backfill_rejects_more_than_eight_weeks_and_wrong_window_cursor(tmp_path): - source = FakeSource() - source.users.append( - LifecycleBackfillUser( - user_id=2, - created_at=datetime(2026, 8, 13, tzinfo=UTC), - ) - ) - source.evidence[2] = [] - store = _store(tmp_path, user_ids=(1,)) - start = date(2026, 8, 1) - end = date(2026, 8, 3) - first = backfill_lifecycle_history( - start=start, - end=end, - batch_size=1, - now=NOW, - source=source, - store=store, - ) - - with pytest.raises(ValueError, match="cursor window"): - backfill_lifecycle_history( - start=start, - end=end.replace(day=4), - cursor=first.next_cursor, - now=NOW, - source=source, - store=store, - ) - with pytest.raises(ValueError, match="at most 56 days"): - backfill_lifecycle_history( - start=date(2026, 6, 1), - end=date(2026, 8, 1), - now=NOW, - source=source, - store=store, - ) - - -def test_deployment_backfill_persists_cursor_and_becomes_a_noop(tmp_path): - source = FakeSource() - source.users.append( - LifecycleBackfillUser( - user_id=2, - created_at=datetime(2026, 8, 13, tzinfo=UTC), - ) - ) - source.evidence[2] = [] - store = _store(tmp_path) - - first = run_lifecycle_backfill_batch( - now=NOW, - batch_size=1, - source=source, - store=store, - ) - running_job = store.get_projection_job(BACKFILL_JOB_NAME) - second = run_lifecycle_backfill_batch( - now=NOW, - batch_size=50, - source=source, - store=store, - ) - complete_job = store.get_projection_job(BACKFILL_JOB_NAME) - third = run_lifecycle_backfill_batch( - now=NOW, - batch_size=50, - source=source, - store=store, - ) - - assert first.complete is False - assert running_job.status == "running" - assert running_job.window_end - running_job.window_start == timedelta(days=56) - assert second.complete is True - assert complete_job.status == "complete" - assert third.processed_users == 0 - assert third.written_rows == 0 - - -def test_partial_backfill_does_not_downgrade_a_complete_daily_snapshot(tmp_path): - source = FakeSource() - source.complete_from = date.max - store = _store(tmp_path, user_ids=(1,)) - snapshot_date = date(2026, 8, 20) - complete = UserLifecycleDailySnapshot( - snapshot_date=snapshot_date, - user_id=1, - lifecycle_segment="growing", - lifecycle_reason_code="growing_activated_below_core_threshold", - data_quality="complete", - calculated_at=NOW, - ) - store.upsert_daily_snapshot(complete) - - report = backfill_lifecycle_history( - start=snapshot_date, - end=snapshot_date + timedelta(days=1), - now=NOW, - source=source, - store=store, - ) - - assert report.written_rows == 0 - assert store.list_daily_snapshots( - start=snapshot_date, - end=snapshot_date + timedelta(days=1), - user_ids=[1], - ) == [complete] - - -def test_sqlite_source_lists_only_eligible_users_and_safe_evidence(tmp_path): - store = _store(tmp_path) - analytics = store.analytics_base - with analytics._get_connection() as conn: - conn.execute("UPDATE users SET role = 'admin' WHERE id = 2") - AnalyticsService(analytics).record_server_event( - event_name="backtest_completed", - user_id=1, - source_event_id="run:completed:source-test", - occurred_at=datetime(2026, 8, 20, tzinfo=UTC), - received_at=datetime(2026, 8, 20, tzinfo=UTC), - ) - source = LifecycleBackfillSource( - analytics_base=analytics, - value_store=store, - complete_from=date(2026, 8, 1), - ) - - users = source.list_users(after_user_id=0, limit=10) - evidence = source.list_evidence( - 1, - start=datetime(2026, 8, 19, tzinfo=UTC), - end=datetime(2026, 8, 21, tzinfo=UTC), - ) - - assert [user.user_id for user in users] == [1] - assert [(item.event_name, item.occurred_at) for item in evidence] == [ - ("backtest_completed", datetime(2026, 8, 20, tzinfo=UTC)) - ] - assert ( - source.quality_for( - date(2026, 8, 1), - required_from=date(2026, 7, 3), - ) - == "complete" - ) diff --git a/dashboard/backend/tests/domain/analytics/test_read_budget.py b/dashboard/backend/tests/domain/analytics/test_read_budget.py index 5c020f786..f01579f23 100644 --- a/dashboard/backend/tests/domain/analytics/test_read_budget.py +++ b/dashboard/backend/tests/domain/analytics/test_read_budget.py @@ -42,7 +42,7 @@ # value store 12 claim, aggregate_events, record_activity_batch x2, # list_operational_signals, list_activity, sum_recent_facts, # upsert_daily_facts, list_facts_for_date, -# append_lifecycle_transitions, list_days_needing_recompute, +# replace_lifecycle_transitions, list_days_needing_recompute, # complete_projection_day # analytics 2 list_excluded_user_ids (inside rollup_day), list_daily_subjects # credits 3 aggregate_ledger_for_day, get_balance_projections, diff --git a/dashboard/backend/tests/domain/analytics/test_repository_contract.py b/dashboard/backend/tests/domain/analytics/test_repository_contract.py index 98134d1b3..87ac614fa 100644 --- a/dashboard/backend/tests/domain/analytics/test_repository_contract.py +++ b/dashboard/backend/tests/domain/analytics/test_repository_contract.py @@ -26,10 +26,9 @@ decode_event_cursor, ) from dashboard.backend.domain.analytics.service import AnalyticsService -from dashboard.backend.domain.analytics.states import ( - AnalyticsStateStore, - UserAnalyticsSnapshot, - recalculate_user_snapshot, +from dashboard.backend.domain.analytics.value_repository import ( + UserDailyFact, + build_value_analytics_store, ) from dashboard.backend.users import UserStore @@ -217,6 +216,29 @@ def get_user_admin(self, user_id): return self.user if user_id == self.user["id"] else None +class _HealthyOperationalBase: + """Credits + provider stand-in: a positive balance and one enabled + platform lane, so the profile's live operational axis reads healthy + without the process-wide credits/provider singletons.""" + + def get_balance_projections(self, user_ids): + return {user_id: {"total_available_micro": 1_000_000} for user_id in user_ids} + + def get_account_billing_state(self, user_id): + return {"account_status": "active", "restriction_reason": None} + + def list_user_credentials(self, user_id): + return [] + + def list_all_providers(self): + return [{"provider_id": "openrouter", "status": "enabled", "platform_enabled": True}] + + +class _NoAgents: + def list_agents(self, *, owner_user_id): + return [] + + def assert_pr2_query_contract(store, user_id): events = AnalyticsService(store) common = { @@ -279,11 +301,18 @@ def assert_pr2_query_contract(store, user_id): occurred_at=NOW - timedelta(minutes=59), **{key: value for key, value in common.items() if key != "source_record_type"}, ) - state_store = AnalyticsStateStore(store) - recalculate_user_snapshot(user_id, now=NOW, store=state_store) + operational_base = _HealthyOperationalBase() + value_store = build_value_analytics_store( + analytics_base=store, + credits_base=operational_base, + provider_base=operational_base, + agent_base=_NoAgents(), + run_base=object(), + ) service = AnalyticsQueryService( store=store, user_store=ContractUsers(user_id), + value_store=value_store, ) overview = service.get_overview( @@ -294,23 +323,38 @@ def assert_pr2_query_contract(store, user_id): now=NOW, ) - profile = service.get_user_profile(user_id=user_id, now=NOW) activity = service.get_user_activity( user_id=user_id, section="runs", limit=10, cursor=None, ) - state_store.upsert_snapshot( - UserAnalyticsSnapshot( - user_id=user_id, - status="needs_attention", - reason_code="invalid_default_credential", - human_readable_reason="The default model credential is invalid.", - evidence_event_ids=[completed.event_id], - calculated_at=NOW, - ) + value_store.upsert_daily_facts( + [ + UserDailyFact( + snapshot_date=NOW.date() - timedelta(days=1), + user_id=user_id, + lifecycle_segment="growing", + lifecycle_reason_code="growing_activated_below_core_threshold", + operational_state="needs_attention", + operational_reason_code="invalid_default_credential", + tier="unpaid", + user_group="unknown", + active=True, + runs_requested=2, + runs_completed=1, + runs_failed=1, + runs_cancelled=0, + operator_cost_micro=0, + own_spend_micro=100, + data_quality="complete", + calculated_at=NOW, + ) + ] ) + # Run and spend figures come from the facts as of yesterday; tokens and + # the footprint from the one timeline page. + profile = service.get_user_profile(user_id=user_id, now=NOW) attention_overview = service.get_overview( filters=AnalyticsMetricFilters( start=NOW - timedelta(days=30), @@ -323,7 +367,9 @@ def assert_pr2_query_contract(store, user_id): assert overview.failed_runs == 1 assert overview.platform_model_cost_usd == 0.25 assert profile.state.status == "active" - assert completed.event_id in profile.state.evidence_event_ids + assert profile.state.evidence_event_ids == [] + assert completed.event_id in [item.event_id for item in profile.recent_footprint] + assert profile.run_summary == {"requested": 2, "completed": 1, "failed": 1, "cancelled": 0} assert profile.input_tokens == 120 assert profile.output_tokens == 30 assert profile.credits_debited_micro == 100 @@ -348,97 +394,12 @@ def test_sqlite_schema_contains_all_foundation_tables(sqlite_contract): assert { "analytics_events", "analytics_daily_rollups", - "user_analytics_snapshots", "analytics_subject_settings", "admin_analytics_access_log", } <= names - - -def test_sqlite_schema_adds_user_value_projection_storage(sqlite_contract): - store, _admin_id, _user_id = sqlite_contract - with store._get_connection() as conn: - names = { - row[0] - for row in conn.execute( - "SELECT name FROM sqlite_master WHERE type = 'table'" - ).fetchall() - } - snapshot_columns = { - row[1] - for row in conn.execute( - "PRAGMA table_info(user_analytics_snapshots)" - ).fetchall() - } - - assert { - "user_lifecycle_daily_snapshots", - "analytics_projection_jobs", - } <= names - assert { - "status", - "lifecycle_segment", - "operational_state", - "activated_at", - "last_meaningful_activity_at", - "inactive_days", - "active_days_30d", - "successful_backtests_30d", - } <= snapshot_columns - - -def test_sqlite_user_value_migration_is_idempotent(sqlite_contract): - store, _admin_id, _user_id = sqlite_contract - - store._init_schema() - store._init_schema() - - with store._get_connection() as conn: - names = [ - row[1] - for row in conn.execute( - "PRAGMA table_info(user_analytics_snapshots)" - ).fetchall() - ] - assert len(names) == len(set(names)) - - -def test_sqlite_migrates_existing_legacy_snapshot_table(tmp_path): - db_path = tmp_path / "legacy-value-analytics.db" - UserStore(db_path=db_path).create_user( - "legacy-value-user@example.test", - "Legacy Value User", - "SecurePass1!", - ) - with sqlite3.connect(db_path) as conn: - conn.execute( - """ - CREATE TABLE user_analytics_snapshots ( - user_id INTEGER PRIMARY KEY, - status TEXT NOT NULL, - reason_code TEXT NOT NULL, - human_readable_reason TEXT NOT NULL, - evidence_event_ids_json TEXT NOT NULL DEFAULT '[]', - calculated_at TEXT NOT NULL - ) - """ - ) - - store = AnalyticsStore(db_path=db_path) - store._init_schema() - - with store._get_connection() as conn: - columns = { - row[1] - for row in conn.execute( - "PRAGMA table_info(user_analytics_snapshots)" - ).fetchall() - } - assert { - "lifecycle_segment", - "operational_state", - "inactive_days", - "successful_backtests_30d", - } <= columns + # The five-state snapshot tables are gone (PR B); a fresh schema never + # creates them and facts_migration drops them from an existing one. + assert not {"user_analytics_snapshots", "user_lifecycle_daily_snapshots"} & names def test_sqlite_accepts_provider_quota_exhausted_category(sqlite_contract): diff --git a/dashboard/backend/tests/domain/analytics/test_repository_postgres.py b/dashboard/backend/tests/domain/analytics/test_repository_postgres.py index 294b714f4..348c4fae0 100644 --- a/dashboard/backend/tests/domain/analytics/test_repository_postgres.py +++ b/dashboard/backend/tests/domain/analytics/test_repository_postgres.py @@ -5,7 +5,7 @@ import inspect import os import uuid -from datetime import timedelta +from datetime import datetime, timedelta, timezone from urllib.parse import parse_qsl, urlencode, urlsplit, urlunsplit import psycopg @@ -18,13 +18,17 @@ from dashboard.backend.domain.analytics.repository_postgres import ( PostgresAnalyticsStore, ) -from dashboard.backend.domain.analytics.rollups import AnalyticsRollupStore, DailyRollup +from dashboard.backend.domain.analytics.rollups import ( + AnalyticsRollupStore, + DailyRollup, + rollup_day, + rollup_lifecycle_day, +) from dashboard.backend.domain.analytics.value_repository import ( ActivityUpdate, LifecycleTransitionRow, ProjectionJob, UserDailyFact, - UserLifecycleDailySnapshot, ) from dashboard.backend.domain.analytics.value_repository_postgres import ( PostgresValueAnalyticsStore, @@ -45,7 +49,6 @@ SyntheticCreditsStore, SyntheticProviderStore, SyntheticRunStore, - _value_snapshot, ) @@ -171,25 +174,6 @@ def public_methods(cls): ) -def test_postgres_ddl_declares_user_value_projection_storage(): - ddl = pg_module.ANALYTICS_POSTGRES_DDL - assert "CREATE TABLE IF NOT EXISTS user_lifecycle_daily_snapshots" in ddl - assert "CREATE TABLE IF NOT EXISTS analytics_projection_jobs" in ddl - for column in ( - "lifecycle_segment", - "operational_state", - "activated_at", - "last_meaningful_activity_at", - "inactive_days", - "active_days_30d", - "successful_backtests_30d", - ): - assert column in ddl or f'"{column}"' in ddl - assert "ADD COLUMN IF NOT EXISTS" in inspect.getsource( - PostgresAnalyticsStore._init_schema - ) - - @pg_only def test_postgres_runs_shared_event_contracts(postgres_contract_store): store, _admin_id, user_id = postgres_contract_store @@ -217,11 +201,11 @@ def test_postgres_runs_pr2_query_contract(postgres_contract_store): @pg_only -def test_postgres_user_value_projection_round_trip( +def test_postgres_projection_job_and_lifecycle_rollups_round_trip( postgres_contract_store, tmp_path, ): - analytics, _admin_id, user_id = postgres_contract_store + analytics, _admin_id, _user_id = postgres_contract_store value_store = PostgresValueAnalyticsStore( analytics, SyntheticCreditsStore(tmp_path / "postgres-value-credits.db"), @@ -229,15 +213,6 @@ def test_postgres_user_value_projection_round_trip( agent_base=SyntheticAgentStore(), run_base=SyntheticRunStore({}), ) - snapshot = _value_snapshot(user_id) - daily = UserLifecycleDailySnapshot( - snapshot_date=NOW.date(), - user_id=user_id, - lifecycle_segment="core", - lifecycle_reason_code="core_repeated_value", - data_quality="complete", - calculated_at=NOW, - ) job = ProjectionJob( job_name="postgres-lifecycle-backfill", window_start=NOW.date(), @@ -246,17 +221,9 @@ def test_postgres_user_value_projection_round_trip( updated_at=NOW, ) - value_store.upsert_current_snapshot(snapshot) - value_store.upsert_daily_snapshot(daily) + value_store.save_projection_job(job) value_store.save_projection_job(job) - assert value_store.get_current_snapshot(user_id) == snapshot - assert value_store.list_current_snapshots([user_id]) == {user_id: snapshot} - assert value_store.list_daily_snapshots( - start=NOW.date(), - end=NOW.date() + timedelta(days=1), - user_ids=[user_id], - ) == [daily] assert value_store.get_projection_job(job.job_name) == job value_store.replace_lifecycle_rollups( @@ -277,12 +244,402 @@ def test_postgres_user_value_projection_round_trip( end=NOW.date() + timedelta(days=1), ) assert lifecycle_rollups[0].metric_name == "lifecycle_segment_count" - assert value_store.list_expiring_daily_dates( - before=NOW.date() + timedelta(days=1), - limit=10, - ) == [NOW.date()] - assert value_store.delete_daily_snapshots_for_date(NOW.date()) == 1 - assert value_store.has_daily_before(NOW.date() + timedelta(days=1)) is False + + +@pg_only +def test_postgres_reseed_then_drop_legacy_snapshot_tables( + postgres_contract_store, +): + """The prod path of design SS11: re-seed from the live legacy row, drop the + two tables, and boot cleanly again once they are gone (UndefinedTable).""" + from dashboard.backend.domain.analytics.facts_migration import ( + StartupMigrationReport, + run_startup_migrations, + ) + from dashboard.backend.tests.domain.analytics.test_facts_migration import ( + ACTIVATED_AT, + LAST_ACTIVITY_AT, + LEGACY_TABLES, + ) + + analytics, _admin_id, user_id = postgres_contract_store + value_store = PostgresValueAnalyticsStore( + analytics, + credits_base=object(), + provider_base=object(), + agent_base=object(), + run_base=object(), + ) + with analytics._get_connection() as conn: + with conn.cursor() as cur: + cur.execute( + """ + CREATE TABLE user_analytics_snapshots ( + user_id INTEGER PRIMARY KEY REFERENCES users(id) ON DELETE CASCADE, + status TEXT NOT NULL, + calculated_at TEXT NOT NULL, + activated_at TEXT, + last_meaningful_activity_at TEXT + ) + """ + ) + cur.execute( + "CREATE TABLE user_lifecycle_daily_snapshots (" + "snapshot_date TEXT NOT NULL, user_id INTEGER NOT NULL)" + ) + cur.execute( + "INSERT INTO user_analytics_snapshots (user_id, status, calculated_at, " + "activated_at, last_meaningful_activity_at) VALUES (%s, 'active', %s, %s, %s)", + (user_id, NOW.isoformat(), ACTIVATED_AT.isoformat(), LAST_ACTIVITY_AT.isoformat()), + ) + + def legacy_tables_present() -> set[str]: + with analytics._get_connection() as conn: + with conn.cursor() as cur: + cur.execute( + "SELECT table_name FROM information_schema.tables " + "WHERE table_schema = current_schema() AND table_name = ANY(%s)", + (list(LEGACY_TABLES),), + ) + return {row["table_name"] for row in cur.fetchall()} # dict_row pool + + assert legacy_tables_present() == set(LEGACY_TABLES) + + report = run_startup_migrations(value_store=value_store, now=NOW) + + assert report == StartupMigrationReport(activity_seeded=1) + activity = value_store.get_activity(user_id) + assert activity.activated_at == ACTIVATED_AT + assert activity.last_meaningful_activity_at == LAST_ACTIVITY_AT + assert legacy_tables_present() == set() + + # Second boot: the seed hits UndefinedTable and reads nothing; the drop is + # IF EXISTS. The failed SELECT must not poison the pooled connection. + analytics.drop_legacy_snapshot_tables() + assert value_store.seed_activity_from_snapshots(now=NOW) == 0 + assert run_startup_migrations(value_store=value_store, now=NOW) == ( + StartupMigrationReport(activity_seeded=0) + ) + assert value_store.get_activity(user_id).activated_at == ACTIVATED_AT + + +@pg_only +def test_postgres_rolls_up_uncovered_legacy_days_before_the_drop( + postgres_contract_store, +): + """I1 on the prod dialect: uncovered legacy days become anonymous lifecycle + rollups, a covered day is left alone, the drop follows, and the second boot + (to_regclass IS NULL) is a no-op.""" + from dashboard.backend.domain.analytics.facts_migration import ( + StartupMigrationReport, + run_startup_migrations, + ) + from dashboard.backend.tests.domain.analytics.test_facts_migration import ( + COVERED_DAY, + LEGACY_DAILY_DDL, + LEGACY_DAILY_ROWS, + UNCOVERED_DAYS, + _rollup_shape, + covering_fact, + legacy_lifecycle_rollups, + ) + + analytics, _admin_id, user_id = postgres_contract_store + value_store = PostgresValueAnalyticsStore( + analytics, + credits_base=object(), + provider_base=object(), + agent_base=object(), + run_base=object(), + ) + with analytics._get_connection() as conn: + with conn.cursor() as cur: + cur.execute( + "CREATE TABLE user_analytics_snapshots (user_id INTEGER PRIMARY KEY, " + "activated_at TEXT, last_meaningful_activity_at TEXT)" + ) + cur.execute(LEGACY_DAILY_DDL) + for snapshot_date, legacy_user, segment, quality in LEGACY_DAILY_ROWS: + cur.execute( + "INSERT INTO user_lifecycle_daily_snapshots " + "VALUES (%s, %s, %s, 'x', %s, %s)", + (snapshot_date, legacy_user, segment, quality, NOW.isoformat()), + ) + value_store.upsert_daily_facts([covering_fact(user_id)]) + + def lifecycle_rollups(day): + return _rollup_shape( + AnalyticsRollupStore(analytics).list_rollups( + start=day, end=day + timedelta(days=1) + ) + ) + + report = run_startup_migrations(value_store=value_store, now=NOW) + + assert report == StartupMigrationReport(legacy_days_rolled_up=2) + for day in UNCOVERED_DAYS: + assert lifecycle_rollups(day) == legacy_lifecycle_rollups(day) + assert lifecycle_rollups(COVERED_DAY) == [] + assert value_store.list_legacy_lifecycle_history().days == () + assert run_startup_migrations(value_store=value_store, now=NOW) == ( + StartupMigrationReport() + ) + + +@pg_only +def test_postgres_copied_fact_days_get_their_transition_log(postgres_contract_store): + """Review #1 on the prod dialect: the day arithmetic and the legacy-source + join are the twin's own SQL, so the SQLite run says nothing about them.""" + from dashboard.backend.domain.analytics.facts_migration import ( + run_startup_migrations, + ) + from dashboard.backend.tests.domain.analytics.test_facts_migration import ( + LEGACY_DAILY_DDL, + assert_transition_history_backfilled, + seed_transition_history, + ) + + analytics, _admin_id, first = postgres_contract_store + value_store = PostgresValueAnalyticsStore( + analytics, + credits_base=object(), + provider_base=object(), + agent_base=object(), + run_base=object(), + ) + second = 900 + with analytics._get_connection() as conn: + with conn.cursor() as cur: + cur.execute( + "INSERT INTO users (id, email, display_name, password_hash, role, " + "created_at) VALUES (%s, 'second@example.test', 'S', 'x', 'user', " + "'2026-07-01T00:00:00+00:00')", + (second,), + ) + cur.execute( + "CREATE TABLE user_analytics_snapshots (user_id INTEGER PRIMARY KEY, " + "activated_at TEXT, last_meaningful_activity_at TEXT)" + ) + cur.execute(LEGACY_DAILY_DDL) + + def insert_legacy(rows): + with analytics._get_connection() as conn: + with conn.cursor() as cur: + for snapshot_date, user_id, segment, quality in rows: + cur.execute( + "INSERT INTO user_lifecycle_daily_snapshots " + "VALUES (%s, %s, %s, 'x', %s, %s)", + (snapshot_date, user_id, segment, quality, NOW.isoformat()), + ) + + seed_transition_history(value_store, first, second, insert_legacy) + + report = run_startup_migrations(value_store=value_store, now=NOW) + + assert report.transitions_backfilled == 2 + assert_transition_history_backfilled(value_store, first, second) + assert run_startup_migrations(value_store=value_store, now=NOW).transitions_backfilled == 0 + + +@pg_only +def test_postgres_activation_backfill_fills_rolled_up_days(postgres_contract_store): + """Review #5 on the prod dialect: substr() over the TEXT occurred_at and + the BOOLEAN excluded column are where the twins could diverge.""" + analytics, _admin_id, user_id = postgres_contract_store + rollups = AnalyticsRollupStore(analytics) + day = NOW.date() - timedelta(days=3) + at = datetime.combine(day, datetime.min.time(), tzinfo=timezone.utc) + timedelta(hours=9) + analytics.append_event( + event_record( + user_id, + event_name="agent_created", + event_group="agent", + event_source="server", + source_event_id="agent:agent_created:activation-backfill-pg", + source_record_type="agent", + source_record_id="activation-backfill-pg", + page_view=None, + session_id=None, + device_category=None, + browser_family=None, + outcome=None, + occurred_at=at, + received_at=at + timedelta(seconds=1), + ) + ) + rollup_day(day, store=rollups) + with analytics._get_connection() as conn: + with conn.cursor() as cur: + cur.execute( + "DELETE FROM analytics_daily_rollups WHERE metric_name = 'activation_users'" + ) + + assert rollups.backfill_activation_users(since=day, until=day + timedelta(days=1)) == (day,) + counts = { + row.event_name: row.value_count + for row in rollups.list_rollups(start=day, end=day + timedelta(days=1)) + if row.metric_name == "activation_users" + } + assert counts == { + "account_signed_up": 0, + "credential_verified": 0, + "agent_created": 1, + "backtest_completed": 0, + } + assert rollups.backfill_activation_users(since=day, until=day + timedelta(days=1)) == () + + +@pg_only +def test_postgres_failed_seed_keeps_both_legacy_tables(postgres_contract_store): + from dashboard.backend.domain.analytics.facts_migration import ( + run_startup_migrations, + ) + from dashboard.backend.tests.domain.analytics.test_facts_migration import ( + LEGACY_TABLES, + ) + + analytics, _admin_id, _user_id = postgres_contract_store + value_store = PostgresValueAnalyticsStore( + analytics, + credits_base=object(), + provider_base=object(), + agent_base=object(), + run_base=object(), + ) + with analytics._get_connection() as conn: + with conn.cursor() as cur: + # Lacks activated_at: UndefinedColumn, not the tolerated UndefinedTable. + cur.execute("CREATE TABLE user_analytics_snapshots (user_id INTEGER PRIMARY KEY)") + cur.execute( + "CREATE TABLE user_lifecycle_daily_snapshots (" + "snapshot_date TEXT NOT NULL, user_id INTEGER NOT NULL)" + ) + + with pytest.raises(psycopg.errors.UndefinedColumn): + run_startup_migrations(value_store=value_store, now=NOW) + + with analytics._get_connection() as conn: + with conn.cursor() as cur: + cur.execute( + "SELECT table_name FROM information_schema.tables " + "WHERE table_schema = current_schema() AND table_name = ANY(%s)", + (list(LEGACY_TABLES),), + ) + present = {row["table_name"] for row in cur.fetchall()} + assert present == set(LEGACY_TABLES) + + +@pg_only +def test_postgres_fact_expiry_and_long_term_rollups_round_trip(postgres_contract_store, tmp_path): + from dashboard.backend.tests.domain.analytics.test_fact_reads import fact, transition + + analytics, _admin_id, user_id = postgres_contract_store + value_store = PostgresValueAnalyticsStore( + analytics, + SyntheticCreditsStore(tmp_path / "postgres-expiry-credits.db"), + provider_base=SyntheticProviderStore({}, []), + agent_base=SyntheticAgentStore(), + run_base=SyntheticRunStore({}), + ) + old, recent = NOW.date() - timedelta(days=200), NOW.date() - timedelta(days=1) + value_store.upsert_daily_facts( + [ + fact(user_id, old, tier="starter", user_group="partner", runs_completed=3, operator_cost_micro=90_000), + fact(user_id, recent), + ] + ) + value_store.append_lifecycle_transitions([transition(user_id, old, "onboarding", "growing")]) + before = NOW.date() - timedelta(days=180) + + # Rollup first (the long-term families), then expire the day. + rollups = rollup_lifecycle_day(old, store=value_store) + stored = AnalyticsRollupStore(analytics).list_rollups(start=old, end=old + timedelta(days=1)) + assert {(r.metric_name, r.user_state, r.value_count, r.value_sum_micro) for r in stored} >= { + ("runs_by_tier", "starter", 3, 0), + ("operator_cost_by_user_group", "partner", 1, 90_000), + ("own_spend_by_tier", "starter", 1, 0), + } + assert {r.metric_name for r in rollups} == {r.metric_name for r in stored} + + assert value_store.list_expiring_fact_dates(before=before, limit=10) == [old] + assert value_store.has_facts_before(before) is True + assert value_store.expire_facts_for_date(old) == 2 # one fact row + one transition + assert value_store.list_facts_for_date(old) == [] + assert value_store.list_transitions_for_date(old) == [] + assert value_store.list_facts_for_date(recent) != [] # inside the window, untouched + assert value_store.list_expiring_fact_dates(before=before, limit=10) == [] + assert value_store.has_facts_before(before) is False + + # A re-run finds no facts and leaves the day's rollups alone. + assert rollup_lifecycle_day(old, store=value_store) == () + again = AnalyticsRollupStore(analytics).list_rollups(start=old, end=old + timedelta(days=1)) + assert again == stored + + +@pg_only +def test_postgres_fact_reads_follow_the_contract(postgres_contract_store): + from dashboard.backend.tests.domain.analytics.test_fact_reads import ( + assert_fact_read_contract, + assert_priority_sort_contract, + assert_read_time_exclusion_contract, + assert_transition_replace_contract, + assert_users_page_contract, + seed_population, + ) + + store, admin_id, _user_id = postgres_contract_store + value_store = PostgresValueAnalyticsStore( + analytics_base=store, + credits_base=object(), + provider_base=object(), + agent_base=object(), + run_base=object(), + ) + next_id = iter(range(100, 200)) + with store._get_connection() as conn: + with conn.cursor() as cur: + # The fixture's users table is a stub with no default id and no + # user_group column; the users page filters and returns it. The + # pre-seeded plain user is not part of the seeded population the + # contract counts, so drop it rather than loosen every total. + cur.execute( + "ALTER TABLE users ADD COLUMN IF NOT EXISTS user_group TEXT " + "NOT NULL DEFAULT 'unknown'" + ) + cur.execute("DELETE FROM users WHERE id = 2") + + def create_user(key: str, email: str, group: str) -> int: + user_id = next(next_id) + with store._get_connection() as conn: + with conn.cursor() as cur: + cur.execute( + "INSERT INTO users (id, email, display_name, password_hash, " + "role, created_at, user_group) VALUES (%s, %s, %s, 'x', " + "'user', '2026-08-01T00:00:00+00:00', %s)", + (user_id, email, key.title(), group), + ) + return user_id + + ids = seed_population(value_store, create_user, admin_id=admin_id) + assert_fact_read_contract(value_store, ids) + assert_users_page_contract(value_store, ids) + assert_priority_sort_contract(value_store, create_user) + + def promote(user_id: int) -> None: + with store._get_connection() as conn: + with conn.cursor() as cur: + cur.execute("UPDATE users SET role = 'admin' WHERE id = %s", (user_id,)) + + # Last: it mutates the population the contracts above count. + assert_read_time_exclusion_contract( + value_store, + ids, + exclude=lambda user_id, excluded=True: store.set_subject_exclusion( + user_id, excluded=excluded, actor_user_id=admin_id, reason="test account" + ), + promote=promote, + ) + assert_transition_replace_contract(value_store, ids) @pg_only diff --git a/dashboard/backend/tests/domain/analytics/test_retention.py b/dashboard/backend/tests/domain/analytics/test_retention.py index 0ad207961..c6600d16c 100644 --- a/dashboard/backend/tests/domain/analytics/test_retention.py +++ b/dashboard/backend/tests/domain/analytics/test_retention.py @@ -19,10 +19,7 @@ AnalyticsRetentionService, ) from dashboard.backend.domain.analytics.repository_common import utc_iso -from dashboard.backend.domain.analytics.value_repository import ( - UserLifecycleDailySnapshot, - ValueAnalyticsStore, -) +from dashboard.backend.domain.analytics.value_repository import ValueAnalyticsStore from dashboard.backend.users import UserStore @@ -224,15 +221,6 @@ def test_sqlite_retention_is_bounded_and_preserves_aggregates(tmp_path): """, (utc_iso(NOW),), ) - conn.execute( - """ - INSERT INTO user_analytics_snapshots ( - user_id, status, reason_code, human_readable_reason, - evidence_event_ids_json, calculated_at - ) VALUES (?, 'active', 'recent_activity', 'Recent activity.', '[]', ?) - """, - (int(subject["id"]), utc_iso(NOW)), - ) class RecordingDeleteStore: def __init__(self, wrapped): @@ -268,93 +256,69 @@ def delete_expired(self, **kwargs): assert conn.execute( "SELECT value_count FROM analytics_daily_rollups" ).fetchone()[0] == 7 - assert conn.execute( - "SELECT status FROM user_analytics_snapshots" - ).fetchone()[0] == "active" -def test_lifecycle_history_is_aggregated_before_user_rows_are_deleted(tmp_path): +def test_facts_and_transitions_are_aggregated_before_user_rows_are_deleted(tmp_path): + from dashboard.backend.tests.domain.analytics.test_fact_reads import fact, transition + db_path = tmp_path / "lifecycle-retention.db" users = UserStore(db_path=db_path) - subject = users.create_user( - "lifecycle-retention@example.test", - "Lifecycle Retention", - "SecurePass1!", - ) + subject = users.create_user("lifecycle-retention@example.test", "Lifecycle Retention", "SecurePass1!") + user_id = int(subject["id"]) analytics = AnalyticsStore(db_path=db_path) values = ValueAnalyticsStore( - analytics, - credits_base=object(), - provider_base=object(), - agent_base=object(), - run_base=object(), + analytics, credits_base=object(), provider_base=object(), agent_base=object(), run_base=object() ) cutoff = NOW.date() - timedelta(days=RAW_EVENT_RETENTION_DAYS) - calculated_at = NOW - timedelta(days=RAW_EVENT_RETENTION_DAYS) - for snapshot_date, segment in ( - (cutoff - timedelta(days=2), "onboarding"), - (cutoff - timedelta(days=1), "growing"), - (cutoff, "growing"), - ): - values.upsert_daily_snapshot( - UserLifecycleDailySnapshot( - snapshot_date=snapshot_date, - user_id=int(subject["id"]), - lifecycle_segment=segment, - lifecycle_reason_code=f"{segment}_reason", - data_quality="complete", - calculated_at=calculated_at, - ) - ) + values.upsert_daily_facts( + [ + fact(user_id, cutoff - timedelta(days=2), lifecycle_segment="onboarding", tier="starter", user_group="partner", runs_completed=0), + fact(user_id, cutoff - timedelta(days=1), lifecycle_segment="growing", tier="starter", user_group="partner", runs_completed=3, operator_cost_micro=90_000), + fact(user_id, cutoff, lifecycle_segment="growing", tier="starter", user_group="partner"), + ] + ) + values.append_lifecycle_transitions( + [ + transition(user_id, cutoff - timedelta(days=1), "onboarding", "growing"), + transition(user_id, cutoff + timedelta(days=3), "growing", "at_risk", inactive_days=8), + ] + ) with analytics._get_connection() as conn: conn.execute( - """ - INSERT INTO analytics_daily_rollups ( - rollup_date, metric_name, value_count, updated_at - ) VALUES (?, 'completed_runs', 3, ?) - """, - ((cutoff - timedelta(days=1)).isoformat(), utc_iso(calculated_at)), + "INSERT INTO analytics_daily_rollups (rollup_date, metric_name, value_count, updated_at) VALUES (?, 'completed_runs', 3, ?)", + ((cutoff - timedelta(days=1)).isoformat(), utc_iso(NOW)), ) - result = AnalyticsRetentionService( - store=analytics, - value_store=values, - batch_size=10, - max_batches=10, - ).run_once(NOW) + service = AnalyticsRetentionService(store=analytics, value_store=values, batch_size=10, max_batches=10) + result = service.run_once(NOW) - assert result.lifecycle_rows_deleted == 2 + assert result.lifecycle_rows_deleted == 3 # two fact rows + one transition assert result.has_more_lifecycle_rows is False - assert values.list_daily_snapshots( - start=cutoff - timedelta(days=3), - end=cutoff + timedelta(days=1), - ) == [ - UserLifecycleDailySnapshot( - snapshot_date=cutoff, - user_id=int(subject["id"]), - lifecycle_segment="growing", - lifecycle_reason_code="growing_reason", - data_quality="complete", - calculated_at=calculated_at, - ) - ] - with analytics._get_connection() as conn: - rows = conn.execute( - """ - SELECT metric_name, event_name, user_state, value_count - FROM analytics_daily_rollups - WHERE rollup_date = ? - ORDER BY metric_name - """, - ((cutoff - timedelta(days=1)).isoformat(),), - ).fetchall() - assert ("completed_runs", "", "", 3) in [tuple(row) for row in rows] - assert ( - "lifecycle_transition", - "onboarding", - "growing", - 1, - ) in [tuple(row) for row in rows] + assert [row.snapshot_date for row in values.list_facts_for_date(cutoff)] == [cutoff] + assert values.list_facts_for_date(cutoff - timedelta(days=1)) == [] + assert values.list_transitions_for_date(cutoff - timedelta(days=1)) == [] + assert len(values.list_transitions_for_date(cutoff + timedelta(days=3))) == 1 # inside the window, untouched + + def day_rows(): + with analytics._get_connection() as conn: + return { + tuple(row) + for row in conn.execute( + "SELECT metric_name, event_name, user_state, value_count, value_sum_micro FROM analytics_daily_rollups WHERE rollup_date = ?", + ((cutoff - timedelta(days=1)).isoformat(),), + ).fetchall() + } + + rows = day_rows() + assert ("completed_runs", "", "", 3, 0) in rows + assert ("lifecycle_transition", "onboarding", "growing", 1, 0) in rows + assert ("runs_by_tier", "", "starter", 3, 0) in rows + assert ("operator_cost_by_user_group", "", "partner", 1, 90_000) in rows + + # A re-run finds nothing left to expire and must leave that history alone. + again = service.run_once(NOW) + assert again.lifecycle_rows_deleted == 0 + assert day_rows() == rows def test_the_retention_sweep_never_touches_user_activity(tmp_path): diff --git a/dashboard/backend/tests/domain/analytics/test_rollups.py b/dashboard/backend/tests/domain/analytics/test_rollups.py index 0d10ccb6f..8dc87e93f 100644 --- a/dashboard/backend/tests/domain/analytics/test_rollups.py +++ b/dashboard/backend/tests/domain/analytics/test_rollups.py @@ -13,10 +13,7 @@ rollup_day, ) from dashboard.backend.domain.analytics.service import AnalyticsService -from dashboard.backend.domain.analytics.value_repository import ( - UserLifecycleDailySnapshot, - ValueAnalyticsStore, -) +from dashboard.backend.domain.analytics.value_repository import ValueAnalyticsStore def _store(tmp_path): @@ -235,66 +232,305 @@ def test_rollup_survives_a_stored_property_this_build_does_not_know(tmp_path): ) -def test_lifecycle_rollup_is_bounded_and_preserves_other_metrics(tmp_path): +def test_lifecycle_rollup_reads_facts_and_transitions_and_writes_tier_and_group_families(tmp_path): + from dashboard.backend.tests.domain.analytics.test_fact_reads import fact, transition + analytics, rollups = _store(tmp_path) values = ValueAnalyticsStore( - analytics, - credits_base=object(), - provider_base=object(), - agent_base=object(), - run_base=object(), + analytics, credits_base=object(), provider_base=object(), agent_base=object(), run_base=object() ) day = date(2026, 8, 25) updated_at = datetime(2026, 8, 26, tzinfo=timezone.utc) rollups.replace_day( - day, + day, [DailyRollup(rollup_date=day, metric_name="completed_runs", value_count=7, updated_at=updated_at)] + ) + values.upsert_daily_facts( [ - DailyRollup( - rollup_date=day, - metric_name="completed_runs", - value_count=7, - updated_at=updated_at, - ) - ], + fact(1, day, lifecycle_segment="growing", tier="starter", user_group="organic", + runs_completed=2, operator_cost_micro=100_000, own_spend_micro=5_000), + fact(2, day, lifecycle_segment="onboarding", tier="unpaid", user_group="organic", + runs_completed=0, operator_cost_micro=0, own_spend_micro=0, data_quality="partial"), + ] ) - for snapshot_date, user_id, segment in ( - (day - timedelta(days=1), 1, "new"), - (day - timedelta(days=1), 2, "onboarding"), - (day, 1, "growing"), - (day, 2, "onboarding"), - ): - values.upsert_daily_snapshot( - UserLifecycleDailySnapshot( - snapshot_date=snapshot_date, - user_id=user_id, - lifecycle_segment=segment, - lifecycle_reason_code=f"{segment}_reason", - data_quality="complete", - calculated_at=updated_at, - ) - ) + values.append_lifecycle_transitions([transition(1, day, "new", "growing")]) first = rollup_lifecycle_day(day, store=values) second = rollup_lifecycle_day(day, store=values) stored = rollups.list_rollups(start=day, end=day + timedelta(days=1)) assert first == second - assert any( - row.metric_name == "completed_runs" and row.value_count == 7 for row in stored + assert any(row.metric_name == "completed_runs" and row.value_count == 7 for row in stored) + assert {(r.user_state, r.value_count, r.outcome) for r in stored if r.metric_name == "lifecycle_segment_count"} == { + ("growing", 1, "complete"), ("onboarding", 1, "partial") + } + assert {(r.event_name, r.user_state, r.value_count) for r in stored if r.metric_name == "lifecycle_transition"} == { + ("new", "growing", 1) + } + assert {(r.user_state, r.value_count) for r in stored if r.metric_name == "runs_by_tier"} == {("starter", 2), ("unpaid", 0)} + assert {(r.user_state, r.value_count) for r in stored if r.metric_name == "runs_by_user_group"} == {("organic", 2)} + assert {(r.user_state, r.value_count, r.value_sum_micro) for r in stored if r.metric_name == "operator_cost_by_tier"} == { + ("starter", 1, 100_000), ("unpaid", 1, 0) + } + assert {(r.user_state, r.value_sum_micro) for r in stored if r.metric_name == "own_spend_by_user_group"} == {("organic", 5_000)} + assert all("user_id" not in row.model_dump() for row in first) + + rollup_day(day, store=rollups) + rebuilt = rollups.list_rollups(start=day, end=day + timedelta(days=1)) + for name in ("lifecycle_transition", "lifecycle_segment_count", "runs_by_tier", "own_spend_by_user_group"): + assert any(row.metric_name == name for row in rebuilt), name + + +def test_lifecycle_rollup_of_a_day_with_no_facts_keeps_what_was_already_written(tmp_path): + """A retention re-run after a day's facts expired must not blank its history.""" + from dashboard.backend.tests.domain.analytics.test_fact_reads import fact + + analytics, rollups = _store(tmp_path) + values = ValueAnalyticsStore( + analytics, credits_base=object(), provider_base=object(), agent_base=object(), run_base=object() + ) + day = date(2026, 8, 25) + values.upsert_daily_facts([fact(1, day, tier="starter", user_group="organic", runs_completed=2)]) + written = rollup_lifecycle_day(day, store=values) + assert written + + assert values.expire_facts_for_date(day) == 1 + assert rollup_lifecycle_day(day, store=values) == () + + stored = rollups.list_rollups(start=day, end=day + timedelta(days=1)) + assert {(r.metric_name, r.user_state, r.value_count) for r in stored} == { + (r.metric_name, r.user_state, r.value_count) for r in written + } + + +def test_replace_lifecycle_rollups_rejects_malformed_dimension_rows(tmp_path): + import pytest + + analytics, _rollups = _store(tmp_path) + values = ValueAnalyticsStore( + analytics, credits_base=object(), provider_base=object(), agent_base=object(), run_base=object() ) + day = date(2026, 8, 25) + at = datetime(2026, 8, 26, tzinfo=timezone.utc) + + def row(**overrides): + fields = dict( + rollup_date=day, metric_name="own_spend_by_tier", user_state="starter", + outcome="complete", value_count=1, value_sum_micro=5, updated_at=at, + ) + fields.update(overrides) + return DailyRollup(**fields) + + values.replace_lifecycle_rollups(day, [row()]) # the well-formed shape is accepted + for bad in ( + row(user_state=""), # a family needs its dimension value + row(event_name="growing"), # event_name is not part of this shape + row(value_sum_micro=-1), # cost sums are non-negative + row(metric_name="runs_by_tier", value_sum_micro=5), # runs families carry a count only + row(outcome="unknown"), + row(metric_name="completed_runs"), # not a lifecycle-owned metric + ): + with pytest.raises(ValueError): + values.replace_lifecycle_rollups(day, [bad]) + + +def test_rollup_day_writes_distinct_activation_users_per_event(tmp_path): + analytics, rollups = _store(tmp_path) + service = AnalyticsService(analytics) + day = date(2026, 8, 25) + for index, (user_id, name, record_type) in enumerate( + ( + (1, "account_signed_up", "user"), + (2, "account_signed_up", "user"), + (1, "agent_created", "agent"), + (1, "agent_created", "agent"), + (1, "backtest_completed", "run"), + ) + ): + service.record_server_event( + event_name=name, + user_id=user_id, + source_event_id=f"{record_type}:{name}:{index}", + source_record_type=record_type, + source_record_id=str(index), + occurred_at=datetime(2026, 8, 25, 10 + index, 0, tzinfo=timezone.utc), + ) + + rollup_day(day, store=rollups) + stored = rollups.list_rollups(start=day, end=day + timedelta(days=1)) + + # Written zero or not: the zero row is how the funnel tells "nobody + # verified a credential that day" from "rolled up before the metric". assert { - (row.user_state, row.value_count) + (row.event_name, row.value_count) for row in stored - if row.metric_name == "lifecycle_segment_count" - } == {("growing", 1), ("onboarding", 1)} - assert { - (row.event_name, row.user_state, row.value_count) + if row.metric_name == "activation_users" + } == { + ("account_signed_up", 2), + ("credential_verified", 0), + ("agent_created", 1), + ("backtest_completed", 1), + } + + +def test_rollup_day_counts_failure_categories_as_events_only(tmp_path): + """The overview sums ``event_count`` per category (#647); the per-user + ``affected_users`` family it replaced is no longer written.""" + + analytics, rollups = _store(tmp_path) + service = AnalyticsService(analytics) + day = date(2026, 8, 25) + for index, user_id in enumerate((1, 1, 2)): + service.record_server_event( + event_name="backtest_failed", + user_id=user_id, + source_event_id=f"run:backtest_failed:{index}", + source_record_type="run", + source_record_id=str(index), + outcome="failed", + error_category="provider_timeout", + occurred_at=datetime(2026, 8, 25, 10 + index, 0, tzinfo=timezone.utc), + ) + + rollup_day(day, store=rollups) + stored = rollups.list_rollups(start=day, end=day + timedelta(days=1)) + + assert sum( + row.value_count for row in stored - if row.metric_name == "lifecycle_transition" - } == {("new", "growing", 1)} - assert all("user_id" not in row.model_dump() for row in first) + if row.metric_name == "event_count" and row.error_category == "provider_timeout" + ) == 3 + assert not any(row.metric_name == "affected_users" for row in stored) + + +def _record(service, user_id, name, record_type, occurred_at, index): + service.record_server_event( + event_name=name, + user_id=user_id, + source_event_id=f"{record_type}:{name}:{index}", + source_record_type=record_type, + source_record_id=str(index), + occurred_at=occurred_at, + ) + +def _drop_activation_rows(analytics): + """What a day rolled up before PR B added the metric looks like.""" + with analytics._get_connection() as conn: + conn.execute( + "DELETE FROM analytics_daily_rollups WHERE metric_name = 'activation_users'" + ) + + +def test_activation_backfill_fills_rolled_up_days_that_predate_the_metric(tmp_path): + analytics, rollups = _store(tmp_path) + service = AnalyticsService(analytics) + first, second = date(2026, 8, 24), date(2026, 8, 25) + for index, (user_id, name, record_type, day) in enumerate( + ( + (1, "account_signed_up", "user", first), + (2, "account_signed_up", "user", first), + (1, "agent_created", "agent", second), + (1, "agent_created", "agent", second), + ) + ): + _record( + service, + user_id, + name, + record_type, + datetime.combine(day, datetime.min.time(), tzinfo=timezone.utc) + + timedelta(hours=10 + index), + index, + ) + rollup_day(first, store=rollups) + rollup_day(second, store=rollups) + expected = { + (row.rollup_date, row.event_name, row.value_count) + for row in rollups.list_rollups(start=first, end=second + timedelta(days=1)) + if row.metric_name == "activation_users" + } + _drop_activation_rows(analytics) + + filled = rollups.backfill_activation_users(since=first, until=second + timedelta(days=1)) + + assert filled == (first, second) + after = rollups.list_rollups(start=first, end=second + timedelta(days=1)) + # Byte-for-byte what rollup_day itself writes, zero rows included. + assert { + (row.rollup_date, row.event_name, row.value_count) + for row in after + if row.metric_name == "activation_users" + } == expected + assert (second, "agent_created", 1) in expected + # Idempotent: a filled day no longer matches. + assert rollups.backfill_activation_users( + since=first, until=second + timedelta(days=1) + ) == () + + +def test_activation_backfill_never_writes_a_day_that_was_not_rolled_up(tmp_path): + """A day with no rollup row is unknown, not zero: its raw events may have + expired, so the backfill must not mint a zero funnel for it.""" + analytics, rollups = _store(tmp_path) + service = AnalyticsService(analytics) + day = date(2026, 8, 25) + _record( + service, + 1, + "account_signed_up", + "user", + datetime(2026, 8, 25, 10, tzinfo=timezone.utc), + 0, + ) + + assert rollups.backfill_activation_users(since=day, until=day + timedelta(days=1)) == () + assert rollups.list_rollups(start=day, end=day + timedelta(days=1)) == [] + + +def test_activation_backfill_leaves_out_excluded_and_admin_accounts(tmp_path): + analytics, rollups = _store(tmp_path) + service = AnalyticsService(analytics) + day = date(2026, 8, 25) + for index, user_id in enumerate((1, 2)): + _record( + service, + user_id, + "account_signed_up", + "user", + datetime(2026, 8, 25, 10 + index, tzinfo=timezone.utc), + index, + ) rollup_day(day, store=rollups) - rebuilt = rollups.list_rollups(start=day, end=day + timedelta(days=1)) - assert any(row.metric_name == "lifecycle_transition" for row in rebuilt) - assert any(row.metric_name == "lifecycle_segment_count" for row in rebuilt) + _drop_activation_rows(analytics) + analytics.set_subject_exclusion(2, excluded=True, actor_user_id=1, reason="test account") + + rollups.backfill_activation_users(since=day, until=day + timedelta(days=1)) + + counts = { + row.event_name: row.value_count + for row in rollups.list_rollups(start=day, end=day + timedelta(days=1)) + if row.metric_name == "activation_users" + } + assert counts["account_signed_up"] == 1 + + +def test_lifecycle_rollup_warns_when_a_day_has_transitions_but_no_facts(tmp_path, capsys): + """Unreachable while both expire together; if it happens, the lost movement + must reach the service log rather than vanish.""" + from dashboard.backend.tests.domain.analytics.test_fact_reads import transition + + analytics, rollups = _store(tmp_path) + values = ValueAnalyticsStore( + analytics, credits_base=object(), provider_base=object(), agent_base=object(), run_base=object() + ) + day = date(2026, 8, 25) + values.append_lifecycle_transitions([transition(1, day, "new", "growing")]) + + assert rollup_lifecycle_day(day, store=values) == () + + out = capsys.readouterr().out + assert ( + "WARNING: analytics.rollup.lifecycle_day_without_facts " + "day=2026-08-25 transitions=1" + ) in out + assert rollups.list_rollups(start=day, end=day + timedelta(days=1)) == [] diff --git a/dashboard/backend/tests/domain/analytics/test_service.py b/dashboard/backend/tests/domain/analytics/test_service.py index 72b40defc..80e35c620 100644 --- a/dashboard/backend/tests/domain/analytics/test_service.py +++ b/dashboard/backend/tests/domain/analytics/test_service.py @@ -208,23 +208,23 @@ def test_subject_exclusion_and_profile_access_require_admin_actor(): assert access["subject_user_id"] == 42 -def test_built_singleton_does_not_synchronously_project_snapshots(monkeypatch): +def test_the_activity_upsert_is_the_only_per_event_projection(monkeypatch): + """PR B deleted the per-event snapshot recompute and its flag: an accepted + lifecycle event advances the activity clock and nothing else.""" + import inspect + from dashboard.backend.domain.analytics import service as service_module - from dashboard.backend.domain.analytics import states as states_module store = RecordingStore() monkeypatch.setattr(service_module, "analytics_store", store) - calls = [] + built = service_module._build_analytics_service() + activity = [] monkeypatch.setattr( - states_module, - "recalculate_user_snapshots", - lambda *args, **kwargs: calls.append((args, kwargs)), + built, + "_try_record_activity", + lambda **kwargs: activity.append(kwargs["user_id"]), ) - built = service_module._build_analytics_service() - - assert built.project_snapshots is False - result = built.record_server_event( event_name="backtest_completed", user_id=42, @@ -236,7 +236,12 @@ def test_built_singleton_does_not_synchronously_project_snapshots(monkeypatch): ) assert result.created is True - assert calls == [] + assert activity == [42] + parameters = inspect.signature(AnalyticsService).parameters + assert "project_snapshots" not in parameters + assert "state_store" not in parameters + assert not hasattr(built, "project_snapshots") + assert not hasattr(AnalyticsService, "_try_recalculate_snapshots") def test_built_singleton_maintains_user_activity(monkeypatch): @@ -247,7 +252,6 @@ def test_built_singleton_maintains_user_activity(monkeypatch): built = service_module._build_analytics_service() assert built.maintain_activity is True - assert built.project_snapshots is False def test_maintaining_activity_requires_a_value_store(): diff --git a/dashboard/backend/tests/domain/analytics/test_states.py b/dashboard/backend/tests/domain/analytics/test_states.py deleted file mode 100644 index e51ce5e69..000000000 --- a/dashboard/backend/tests/domain/analytics/test_states.py +++ /dev/null @@ -1,671 +0,0 @@ -"""Explainable Analytics user-state precedence and snapshot persistence.""" - -from __future__ import annotations - -import sqlite3 -from datetime import datetime, timedelta, timezone - -import pytest - -from dashboard.backend.domain.analytics.repository import AnalyticsStore -from dashboard.backend.domain.analytics.service import AnalyticsService -from dashboard.backend.domain.analytics.states import ( - AnalyticsStateStore, - UserAnalyticsSnapshot, - calculate_user_value_snapshot, - calculate_user_state, - recalculate_user_snapshot, - repair_stale_snapshots, - repair_stale_value_snapshots, -) -from dashboard.backend.domain.analytics.value_repository import ( - CurrentOperationalFacts, - UserValueSnapshot, - ValueAnalyticsStore, -) - - -NOW = datetime(2026, 8, 26, 12, 0, tzinfo=timezone.utc) - - -class RecordingValueStore: - def __init__(self): - self.activities = () - self.snapshots = {} - self.daily = [] - self.operational = CurrentOperationalFacts(user_id=1) - self.upsert_count = 0 - - @property - def current(self): - return self.snapshots.get(1) - - @current.setter - def current(self, value): - if value is None: - self.snapshots.pop(1, None) - else: - self.snapshots[1] = value - - def list_credit_activity(self, user_ids, *, start, end): - return {user_id: self.activities for user_id in user_ids} - - def get_current_snapshot(self, user_id): - return self.snapshots.get(user_id) - - def get_operational_facts(self, user_id, *, now): - return self.operational.model_copy(update={"user_id": user_id}) - - def upsert_current_snapshot(self, snapshot): - self.upsert_count += 1 - self.snapshots[snapshot.user_id] = snapshot - return snapshot - - def upsert_daily_snapshot(self, snapshot): - self.daily = [ - row - for row in self.daily - if (row.snapshot_date, row.user_id) - != (snapshot.snapshot_date, snapshot.user_id) - ] - self.daily.append(snapshot) - return snapshot - - -class FailingValueStore(RecordingValueStore): - def list_credit_activity(self, user_ids, *, start, end): - raise RuntimeError("private projection failure") - - -class SelectiveFailingValueStore(RecordingValueStore): - def list_credit_activity(self, user_ids, *, start, end): - if user_ids == [1]: - raise RuntimeError("private first-user failure") - return super().list_credit_activity(user_ids, start=start, end=end) - - -def _fixture(tmp_path, *, created_at=NOW - timedelta(days=1)): - path = tmp_path / "states.db" - with sqlite3.connect(path) as conn: - conn.execute( - """ - CREATE TABLE users ( - id INTEGER PRIMARY KEY, - email TEXT NOT NULL, - display_name TEXT NOT NULL, - password_hash TEXT NOT NULL, - role TEXT NOT NULL, - created_at TEXT NOT NULL - ) - """ - ) - conn.execute( - "INSERT INTO users VALUES (1, 'user@example.test', 'User', 'x', 'user', ?)", - (created_at.isoformat(),), - ) - analytics = AnalyticsStore(path) - state_store = AnalyticsStateStore(analytics) - return ( - AnalyticsService( - analytics, - state_store=state_store, - value_store=RecordingValueStore(), - project_snapshots=True, - ), - state_store, - ) - - -def _run(service, name, index, at, *, error_category=None): - return service.record_server_event( - event_name=name, - user_id=1, - source_event_id=f"run:{name}:run-{index}", - source_record_type="run", - source_record_id=f"run-{index}", - correlation_id=f"run-{index}", - error_category=error_category, - occurred_at=at, - ).event - - -def test_blocked_wins_over_needs_attention(tmp_path): - service, store = _fixture(tmp_path) - for index in range(3): - _run( - service, - "backtest_failed", - index, - NOW - timedelta(hours=index + 2), - error_category="internal_error", - ) - blocked = service.record_server_event( - event_name="safe_error_recorded", - user_id=1, - source_event_id="resource:safe_error_recorded:run-blocked:credits_unavailable", - source_record_type="run", - source_record_id="run-blocked", - error_category="credits_unavailable", - occurred_at=NOW - timedelta(hours=1), - ).event - - snapshot = calculate_user_state(1, now=NOW, store=store) - - assert snapshot.status == "blocked" - assert snapshot.reason_code == "billing_lane_unavailable" - assert snapshot.evidence_event_ids == [blocked.event_id] - - -def test_new_user_without_run_is_onboarding(tmp_path): - _service, store = _fixture(tmp_path) - - snapshot = calculate_user_state(1, now=NOW, store=store) - - assert snapshot.status == "onboarding" - assert snapshot.reason_code == "no_successful_run" - - -def test_three_newest_terminal_failures_need_attention(tmp_path): - service, store = _fixture(tmp_path) - failures = [ - _run( - service, - "backtest_failed", - index, - NOW - timedelta(hours=index + 1), - error_category="internal_error", - ) - for index in range(3) - ] - - snapshot = calculate_user_state(1, now=NOW, store=store) - - assert snapshot.status == "needs_attention" - assert snapshot.reason_code == "three_consecutive_failed_runs" - assert set(snapshot.evidence_event_ids) == {event.event_id for event in failures} - - -def test_completed_or_cancelled_run_breaks_failure_sequence(tmp_path): - service, store = _fixture(tmp_path) - _run(service, "backtest_failed", 1, NOW - timedelta(hours=1)) - _run(service, "backtest_cancelled", 2, NOW - timedelta(hours=2)) - _run(service, "backtest_failed", 3, NOW - timedelta(hours=3)) - _run(service, "backtest_failed", 4, NOW - timedelta(hours=4)) - - snapshot = calculate_user_state(1, now=NOW, store=store) - - assert snapshot.status == "onboarding" - - -def test_dormant_and_active_use_thirty_day_activity(tmp_path): - service, store = _fixture(tmp_path, created_at=NOW - timedelta(days=60)) - _run(service, "backtest_completed", 1, NOW - timedelta(days=45)) - - dormant = calculate_user_state(1, now=NOW, store=store) - _run(service, "backtest_completed", 2, NOW - timedelta(days=1)) - active = calculate_user_state(1, now=NOW, store=store) - - assert dormant.status == "dormant" - assert dormant.reason_code == "no_meaningful_activity_30d" - assert active.status == "active" - - -def test_recalculate_upserts_one_current_snapshot(tmp_path): - service, store = _fixture(tmp_path) - first = recalculate_user_snapshot(1, now=NOW, store=store) - _run(service, "backtest_completed", 1, NOW - timedelta(hours=1)) - second = recalculate_user_snapshot(1, now=NOW, store=store) - - assert first.status == "onboarding" - assert second.status == "active" - assert store.get_snapshot(1) == second - - -def test_value_snapshot_keeps_lifecycle_and_operational_axes_separate(tmp_path): - service, state_store = _fixture( - tmp_path, - created_at=NOW - timedelta(days=20), - ) - _run(service, "backtest_completed", 1, NOW - timedelta(days=9)) - value_store = RecordingValueStore() - value_store.operational = CurrentOperationalFacts( - user_id=1, - account_restricted=True, - ) - - snapshot = calculate_user_value_snapshot( - 1, - now=NOW, - state_store=state_store, - value_store=value_store, - ) - - assert snapshot.lifecycle_segment == "at_risk" - assert snapshot.operational_state == "blocked" - assert snapshot.lifecycle_reason_code == "at_risk_previously_activated" - assert snapshot.operational_reason_code == "account_restricted" - - -def test_value_snapshot_counts_distinct_utc_activity_days(tmp_path): - service, state_store = _fixture( - tmp_path, - created_at=NOW - timedelta(days=20), - ) - for index, days_ago in enumerate((4, 3, 2), start=1): - _run(service, "backtest_completed", index, NOW - timedelta(days=days_ago)) - _run(service, "agent_updated", 4, NOW - timedelta(days=2, hours=1)) - value_store = RecordingValueStore() - value_store.activities = ( - NOW - timedelta(days=1), - NOW - timedelta(days=1, hours=1), - ) - - snapshot = calculate_user_value_snapshot( - 1, - now=NOW, - state_store=state_store, - value_store=value_store, - ) - - assert snapshot.lifecycle_segment == "core" - assert snapshot.active_days_30d == 4 - assert snapshot.successful_backtests_30d == 3 - assert snapshot.last_meaningful_activity_at == NOW - timedelta(days=1) - - -def test_legacy_snapshot_fields_remain_unchanged(tmp_path): - service, state_store = _fixture(tmp_path) - recalculate_user_snapshot(1, now=NOW, store=state_store) - _run(service, "backtest_completed", 1, NOW - timedelta(hours=1)) - legacy = state_store.get_snapshot(1) - - calculate_user_value_snapshot( - 1, - now=NOW, - state_store=state_store, - value_store=RecordingValueStore(), - ) - - assert state_store.get_snapshot(1) == legacy - - -def test_relevant_event_recalculates_value_projection(tmp_path): - _unused_service, state_store = _fixture(tmp_path) - value_store = RecordingValueStore() - value_store.analytics_base = state_store.base_store - service = AnalyticsService( - state_store.base_store, - state_store=state_store, - value_store=value_store, - project_snapshots=True, - ) - - result = service.record_server_event( - event_name="backtest_completed", - user_id=1, - source_event_id="run:completed:synthetic-1", - occurred_at=NOW, - received_at=NOW, - ) - - assert result.created is True - persisted = ValueAnalyticsStore( - state_store.base_store, - credits_base=object(), - provider_base=object(), - agent_base=object(), - run_base=object(), - ).get_current_snapshot(1) - assert persisted is not None - assert persisted.lifecycle_segment == "growing" - assert persisted.operational_state == "healthy" - assert persisted.calculated_at == NOW - legacy = state_store.get_snapshot(1) - assert legacy.status == "active" - assert legacy.calculated_at == persisted.calculated_at - assert value_store.upsert_count == 0 - - -def test_projection_failure_does_not_reject_an_accepted_event(tmp_path, capsys): - _unused_service, state_store = _fixture(tmp_path) - service = AnalyticsService( - state_store.base_store, - state_store=state_store, - value_store=FailingValueStore(), - project_snapshots=True, - ) - - result = service.record_server_event( - event_name="backtest_requested", - user_id=1, - source_event_id="run:requested:synthetic-2", - occurred_at=NOW, - received_at=NOW, - ) - - assert result.created is True - output = capsys.readouterr().out - assert "analytics.value_projection_failed" in output - assert "event=backtest_requested" in output - assert "category=RuntimeError" in output - assert "private projection failure" not in output - assert state_store.get_snapshot(1) is None - - -def test_projection_is_disabled_by_default(tmp_path): - _unused_service, state_store = _fixture(tmp_path) - value_store = RecordingValueStore() - service = AnalyticsService( - state_store.base_store, - state_store=state_store, - value_store=value_store, - ) - - result = service.record_server_event( - event_name="backtest_completed", - user_id=1, - source_event_id="run:completed:projection-disabled", - occurred_at=NOW, - received_at=NOW, - ) - - assert result.created is True - assert value_store.current is None - assert state_store.get_snapshot(1) is None - - -@pytest.mark.parametrize( - "event_name", - ["authenticated_session_started", "credential_revoked", "agent_deleted"], -) -def test_irrelevant_server_event_does_not_recalculate_value_projection( - tmp_path, - event_name, -): - _unused_service, state_store = _fixture(tmp_path) - value_store = RecordingValueStore() - service = AnalyticsService( - state_store.base_store, - state_store=state_store, - value_store=value_store, - project_snapshots=True, - ) - - result = service.record_server_event( - event_name=event_name, - user_id=1, - source_event_id=f"event:{event_name}:synthetic-1", - occurred_at=NOW, - received_at=NOW, - ) - - assert result.created is True - assert value_store.current is None - assert state_store.get_snapshot(1) is None - - -@pytest.mark.parametrize( - "event_name", - ["credential_saved", "account_signed_up", "safe_error_recorded"], -) -def test_only_supported_relevant_event_groups_trigger_projection(tmp_path, event_name): - _unused_service, state_store = _fixture(tmp_path) - value_store = RecordingValueStore() - service = AnalyticsService( - state_store.base_store, - state_store=state_store, - value_store=value_store, - project_snapshots=True, - ) - - result = service.record_server_event( - event_name=event_name, - user_id=1, - source_event_id=f"event:{event_name}:synthetic", - occurred_at=NOW, - received_at=NOW, - ) - - assert result.created is True - assert value_store.upsert_count == 1 - - -def test_idempotent_event_replay_does_not_recalculate_projection(tmp_path): - _unused_service, state_store = _fixture(tmp_path) - value_store = RecordingValueStore() - service = AnalyticsService( - state_store.base_store, - state_store=state_store, - value_store=value_store, - project_snapshots=True, - ) - kwargs = { - "event_name": "agent_created", - "user_id": 1, - "source_event_id": "agent:created:synthetic", - "occurred_at": NOW, - "received_at": NOW, - } - - first = service.record_server_event(**kwargs) - replay = service.record_server_event(**kwargs) - - assert first.created is True - assert replay.created is False - assert value_store.upsert_count == 1 - - -def test_cleared_current_account_is_not_blocked_by_an_old_error(tmp_path): - service, state_store = _fixture(tmp_path) - service.record_server_event( - event_name="safe_error_recorded", - user_id=1, - source_event_id="run:safe-error:credits", - source_record_type="run", - source_record_id="run-credits", - error_category="credits_unavailable", - occurred_at=NOW - timedelta(hours=2), - received_at=NOW - timedelta(hours=2), - ) - value_store = RecordingValueStore() - - snapshot = calculate_user_value_snapshot( - 1, - now=NOW, - state_store=state_store, - value_store=value_store, - ) - - assert snapshot.operational_state == "healthy" - assert snapshot.operational_reason_code == "no_supported_issue" - assert state_store.get_snapshot(1).status == "blocked" - - -def test_activation_timestamp_survives_raw_event_retention(tmp_path): - _service, state_store = _fixture( - tmp_path, - created_at=NOW - timedelta(days=300), - ) - first_success = NOW - timedelta(days=200) - value_store = RecordingValueStore() - value_store.current = UserValueSnapshot( - user_id=1, - lifecycle_segment="dormant", - lifecycle_reason_code="dormant_previously_activated", - lifecycle_reason="Previously activated and inactive.", - operational_state="healthy", - operational_reason_code="no_supported_issue", - operational_reason="No supported current operational issue was detected.", - activated_at=first_success, - last_meaningful_activity_at=first_success, - inactive_days=199, - active_days_30d=0, - successful_backtests_30d=0, - calculated_at=NOW - timedelta(days=1), - ) - - snapshot = calculate_user_value_snapshot( - 1, - now=NOW, - state_store=state_store, - value_store=value_store, - ) - - assert snapshot.activated_at == first_success - assert snapshot.last_meaningful_activity_at == first_success - assert snapshot.lifecycle_reason_code == "dormant_previously_activated" - - -def test_future_previous_projection_timestamps_are_ignored(tmp_path): - _service, state_store = _fixture( - tmp_path, - created_at=NOW - timedelta(days=10), - ) - value_store = RecordingValueStore() - value_store.current = UserValueSnapshot( - user_id=1, - lifecycle_segment="growing", - lifecycle_reason_code="growing_activated_below_core_threshold", - lifecycle_reason="Activated.", - operational_state="healthy", - operational_reason_code="no_supported_issue", - operational_reason="No supported current operational issue was detected.", - activated_at=NOW + timedelta(days=1), - last_meaningful_activity_at=NOW + timedelta(days=2), - inactive_days=0, - active_days_30d=1, - successful_backtests_30d=1, - calculated_at=NOW + timedelta(days=2), - ) - - snapshot = calculate_user_value_snapshot( - 1, - now=NOW, - state_store=state_store, - value_store=value_store, - ) - - assert snapshot.activated_at is None - assert snapshot.last_meaningful_activity_at is None - assert snapshot.lifecycle_reason_code == "at_risk_never_activated" - - -def test_repair_stale_snapshots_uses_a_24_hour_default_window(tmp_path): - _service, state_store = _fixture(tmp_path) - with state_store.base_store._get_connection() as conn: - conn.execute( - "INSERT INTO users VALUES (2, 'second@example.test', 'Second', 'x', 'user', ?)", - ((NOW - timedelta(days=2)).isoformat(),), - ) - state_store.upsert_snapshot( - UserAnalyticsSnapshot( - user_id=1, - status="onboarding", - reason_code="no_successful_run", - human_readable_reason="The user has not completed a successful backtest yet.", - calculated_at=NOW - timedelta(hours=23), - ) - ) - state_store.upsert_snapshot( - UserAnalyticsSnapshot( - user_id=2, - status="onboarding", - reason_code="no_successful_run", - human_readable_reason="The user has not completed a successful backtest yet.", - calculated_at=NOW - timedelta(hours=25), - ) - ) - - repaired = repair_stale_snapshots(now=NOW, limit=10, store=state_store) - - assert repaired == 1 - assert state_store.get_snapshot(1).calculated_at == NOW - timedelta(hours=23) - assert state_store.get_snapshot(2).calculated_at == NOW - - -def test_utc_day_transition_repairs_current_and_daily_value_snapshots(tmp_path): - transition_now = datetime(2026, 8, 27, 0, 5, tzinfo=timezone.utc) - _service, state_store = _fixture( - tmp_path, - created_at=transition_now - timedelta(days=1), - ) - state_store.upsert_snapshot( - UserAnalyticsSnapshot( - user_id=1, - status="onboarding", - reason_code="no_successful_run", - human_readable_reason="The user has not completed a successful backtest yet.", - calculated_at=transition_now - timedelta(minutes=6), - ) - ) - value_store = RecordingValueStore() - - repaired = repair_stale_value_snapshots( - now=transition_now, - limit=10, - state_store=state_store, - value_store=value_store, - ) - - assert repaired == 1 - assert value_store.current is not None - assert len(value_store.daily) == 1 - assert value_store.daily[0].snapshot_date == transition_now.date() - assert value_store.daily[0].data_quality == "complete" - - repair_stale_value_snapshots( - now=transition_now, - limit=10, - state_store=state_store, - value_store=value_store, - ) - assert len(value_store.daily) == 1 - - -def test_value_snapshot_repair_isolates_one_user_failure(tmp_path, capsys): - _service, state_store = _fixture( - tmp_path, - created_at=NOW - timedelta(days=1), - ) - with state_store.base_store._get_connection() as conn: - conn.execute( - "INSERT INTO users VALUES (2, 'second@example.test', 'Second', 'x', 'user', ?)", - ((NOW - timedelta(days=1)).isoformat(),), - ) - value_store = SelectiveFailingValueStore() - - repaired = repair_stale_value_snapshots( - now=NOW, - limit=10, - state_store=state_store, - value_store=value_store, - ) - - assert repaired == 1 - assert state_store.get_snapshot(1) is None - assert state_store.get_snapshot(2) is not None - assert value_store.get_current_snapshot(2) is not None - output = capsys.readouterr().out - assert "analytics.value_snapshot_repair_failed" in output - assert "category=RuntimeError" in output - assert "private first-user failure" not in output - - -def test_a_restricted_account_is_blocked_like_an_empty_balance(tmp_path): - service, store = _fixture(tmp_path) - _run(service, "backtest_failed", 0, NOW - timedelta(hours=2), error_category="internal_error") - blocked = service.record_server_event( - event_name="safe_error_recorded", - user_id=1, - source_event_id="resource:safe_error_recorded:run-restricted:account_restricted", - source_record_type="run", - source_record_id="run-restricted", - error_category="account_restricted", - occurred_at=NOW - timedelta(hours=1), - ).event - - snapshot = calculate_user_state(1, now=NOW, store=store) - - assert snapshot.status == "blocked" - assert snapshot.reason_code == "billing_lane_unavailable" - assert snapshot.evidence_event_ids == [blocked.event_id] diff --git a/dashboard/backend/tests/domain/analytics/test_user_activity.py b/dashboard/backend/tests/domain/analytics/test_user_activity.py index 49f3799d5..bc00e377c 100644 --- a/dashboard/backend/tests/domain/analytics/test_user_activity.py +++ b/dashboard/backend/tests/domain/analytics/test_user_activity.py @@ -12,8 +12,8 @@ from dashboard.backend.domain.analytics.value_repository import ( build_value_analytics_store, ) -from dashboard.backend.tests.domain.analytics.test_value_repository import ( - _value_snapshot, +from dashboard.backend.tests.domain.analytics.test_facts_migration import ( + create_legacy_snapshot_tables, ) @@ -191,9 +191,9 @@ def test_seed_copies_the_legacy_timestamps_and_is_idempotent(tmp_path, monkeypat last year. Without this seed every existing user's activation date would be lost on the night PR B drops ``user_analytics_snapshots``. """ - _service, _store, value_store = _fixture(tmp_path, monkeypatch) - legacy = _value_snapshot(1) # activated NOW-20d, last activity NOW-2d - value_store.upsert_current_snapshot(legacy) + _service, store, value_store = _fixture(tmp_path, monkeypatch) + activated_at, last_activity_at = NOW - timedelta(days=20), NOW - timedelta(days=2) + create_legacy_snapshot_tables(store, [(1, activated_at, last_activity_at)]) first = value_store.seed_activity_from_snapshots(now=NOW) seeded = value_store.get_activity(1) @@ -201,8 +201,8 @@ def test_seed_copies_the_legacy_timestamps_and_is_idempotent(tmp_path, monkeypat reseeded = value_store.get_activity(1) assert first == 1 - assert seeded.activated_at == legacy.activated_at - assert seeded.last_meaningful_activity_at == legacy.last_meaningful_activity_at + assert seeded.activated_at == activated_at + assert seeded.last_meaningful_activity_at == last_activity_at assert second == 1 # the upsert touches the row again ... assert reseeded.activated_at == seeded.activated_at # ... and changes nothing assert reseeded.last_meaningful_activity_at == seeded.last_meaningful_activity_at @@ -210,8 +210,10 @@ def test_seed_copies_the_legacy_timestamps_and_is_idempotent(tmp_path, monkeypat def test_seed_never_regresses_a_row_ingestion_already_advanced(tmp_path, monkeypatch): """MIN on activation, MAX on activity: the seed can only fill gaps.""" - service, _store, value_store = _fixture(tmp_path, monkeypatch) - value_store.upsert_current_snapshot(_value_snapshot(1)) # NOW-20d / NOW-2d + service, store, value_store = _fixture(tmp_path, monkeypatch) + create_legacy_snapshot_tables( + store, [(1, NOW - timedelta(days=20), NOW - timedelta(days=2))] + ) _emit(service, "earlier", NOW - timedelta(days=30), name="backtest_completed") _emit(service, "today", NOW) diff --git a/dashboard/backend/tests/domain/analytics/test_user_metrics.py b/dashboard/backend/tests/domain/analytics/test_user_metrics.py new file mode 100644 index 000000000..345d3ec3c --- /dev/null +++ b/dashboard/backend/tests/domain/analytics/test_user_metrics.py @@ -0,0 +1,127 @@ +"""One metric struct, two audiences (design §6.13).""" + +from __future__ import annotations + +import json +import sqlite3 +from datetime import timedelta + +import pytest + +from dashboard.backend.domain.analytics.query_service import ( + AnalyticsQueryService, + UserMetricsAdmin, + UserMetricsShared, +) +from dashboard.backend.domain.analytics.repository import AnalyticsStore +from dashboard.backend.domain.analytics.value_repository import ValueAnalyticsStore +from dashboard.backend.tests.domain.analytics.test_fact_reads import NOW, YESTERDAY, fact +from dashboard.backend.tests.domain.analytics.test_value_repository import ( + SyntheticAgentStore, + SyntheticCreditsStore, + SyntheticProviderStore, + SyntheticRunStore, +) +from dashboard.backend.users import UserStore + + +_ADMIN_ONLY = {"active_days_30d", "lifecycle", "operational", "user_group", "group_badge", "access_log"} +_SHARED = {"user_id", "facts_as_of", "runs_7d", "runs_30d", "own_spend_micro", "operator_cost_micro", "tier", "credit_balance_micro", "data_quality"} + + +@pytest.fixture +def metrics_fixture(tmp_path): + db_path = tmp_path / "metrics.db" + users = UserStore(db_path=db_path) + admin = users.create_user("admin@example.test", "Admin", "SecurePass1!") + users.apply_admin_patch(int(admin["id"]), role="admin") + subject = users.create_user("subject@example.test", "Subject", "SecurePass1!") + users.apply_admin_patch(int(subject["id"]), user_group="partner") + # create_user stamps the real clock, which is later than the fixed NOW; + # lifecycle evidence may not postdate its as_of. + with sqlite3.connect(db_path) as conn: + conn.execute( + "UPDATE users SET created_at = ?", + ((NOW - timedelta(days=90)).isoformat(),), + ) + analytics = AnalyticsStore(db_path=db_path) + credits = SyntheticCreditsStore(tmp_path / "credits.db") + value_store = ValueAnalyticsStore( + analytics, + credits, + # One enabled platform lane and a positive balance: operationally healthy. + provider_base=SyntheticProviderStore( + {}, [{"provider_id": "openrouter", "status": "enabled", "platform_enabled": True}] + ), + agent_base=SyntheticAgentStore(), + run_base=SyntheticRunStore({}), + ) + user_id = int(subject["id"]) + value_store.record_activity(user_id, occurred_at=NOW - timedelta(days=20), activating=True, now=NOW) + value_store.record_activity(user_id, occurred_at=NOW - timedelta(days=1), activating=False, now=NOW) + value_store.upsert_daily_facts( + [fact(user_id, YESTERDAY - timedelta(days=offset), runs_requested=2, runs_completed=1, + runs_failed=1, operator_cost_micro=100_000, own_spend_micro=40_000) + for offset in range(10)] + ) + analytics.record_admin_access(int(admin["id"]), user_id, "overview") + service = AnalyticsQueryService(store=analytics, user_store=users, value_store=value_store) + return service, user_id + + +def test_the_user_audience_never_sees_admin_fields(metrics_fixture): + service, user_id = metrics_fixture + + metrics = service.get_user_metrics(user_id, audience="user", now=NOW) + + assert type(metrics) is UserMetricsShared + # Absent, not null -- on every serialisation path, including the one + # FastAPI uses, because the model has no such fields at all. + for payload in (metrics.model_dump(), json.loads(metrics.model_dump_json())): + assert _ADMIN_ONLY.isdisjoint(payload) + assert set(payload) == _SHARED + + +def test_both_audiences_agree_on_the_shared_numbers(metrics_fixture): + service, user_id = metrics_fixture + + admin = service.get_user_metrics(user_id, audience="admin", now=NOW).model_dump() + user = service.get_user_metrics(user_id, audience="user", now=NOW).model_dump() + + for field in _SHARED: + assert admin[field] == user[field], field + assert admin["runs_7d"] == {"requested": 14, "completed": 7, "failed": 7, "cancelled": 0} + assert admin["runs_30d"] == {"requested": 20, "completed": 10, "failed": 10, "cancelled": 0} + assert admin["operator_cost_micro"] == 1_000_000 + assert admin["own_spend_micro"] == 400_000 + assert admin["facts_as_of"] == YESTERDAY + assert admin["credit_balance_micro"] == 1_000_000 # SyntheticCreditsStore default + assert admin["data_quality"] == "partial" # 10 of the 30 trailing days have a row + + +def test_the_admin_audience_sees_every_field(metrics_fixture): + service, user_id = metrics_fixture + + metrics = service.get_user_metrics(user_id, audience="admin", now=NOW) + payload = metrics.model_dump() + + assert type(metrics) is UserMetricsAdmin + assert _ADMIN_ONLY <= set(payload) + assert payload["active_days_30d"] == 10 + assert payload["lifecycle"]["segment"] == "core" + assert payload["operational"]["state"] == "healthy" + assert payload["user_group"] == "partner" + assert payload["group_badge"] == "partner" + assert payload["access_log"] == ["overview"] + + +def test_get_user_metrics_rejects_an_unknown_audience(metrics_fixture): + service, user_id = metrics_fixture + with pytest.raises(ValueError): + service.get_user_metrics(user_id, audience="public", now=NOW) + + +def test_get_user_metrics_raises_lookup_error_for_an_unknown_user(metrics_fixture): + service, _user_id = metrics_fixture + with pytest.raises(LookupError): + service.get_user_metrics(404, audience="admin", now=NOW) diff --git a/dashboard/backend/tests/domain/analytics/test_value_queries.py b/dashboard/backend/tests/domain/analytics/test_value_queries.py index 48f65451d..80bc7ed97 100644 --- a/dashboard/backend/tests/domain/analytics/test_value_queries.py +++ b/dashboard/backend/tests/domain/analytics/test_value_queries.py @@ -2,7 +2,7 @@ from __future__ import annotations -from datetime import date, datetime, timedelta, timezone +from datetime import date, datetime, time, timedelta, timezone from types import SimpleNamespace import pytest @@ -12,21 +12,33 @@ AnalyticsStateSummary, AnalyticsUserProfile, ) +from dashboard.backend.domain.analytics.rollups import DailyRollup from dashboard.backend.domain.analytics.value_queries import ( - _MAX_HISTORY_SCAN_DAYS, - _MOVEMENT_WINDOWS, + OperationalAnalyticsResponse, + UserGroupSummary, UserValueFilters, ValueAnalyticsQueryService, ) from dashboard.backend.domain.analytics.value_repository import ( CommercialValueFact, - UserLifecycleDailySnapshot, - UserValueSnapshot, + CurrentOperationalFacts, + FactStateCount, + GroupFactTotals, + LifecycleTransitionRow, + RecentFactTotals, + SegmentDayCount, + TransitionCount, + UserActivity, + UserDailyFact, + UserPageQuery, + UserPageRow, ) +from dashboard.backend.domain.user_groups import USER_GROUPS UTC = timezone.utc NOW = datetime(2026, 9, 3, 12, 0, tzinfo=UTC) +YESTERDAY = NOW.date() - timedelta(days=1) def _user(user_id: int, user_group: str = "unknown") -> dict[str, object]: @@ -39,34 +51,6 @@ def _user(user_id: int, user_group: str = "unknown") -> dict[str, object]: } -def _snapshot( - user_id: int, - *, - lifecycle: str = "core", - operational: str = "healthy", - activated_at: datetime | None = None, - inactive_days: int = 2, -) -> UserValueSnapshot: - activated = activated_at if activated_at is not None else NOW - timedelta(days=60) - return UserValueSnapshot( - user_id=user_id, - lifecycle_segment=lifecycle, - lifecycle_reason_code=f"{lifecycle}_synthetic_reason", - lifecycle_reason=f"Synthetic {lifecycle} lifecycle reason.", - lifecycle_evidence=("Synthetic lifecycle evidence.",), - operational_state=operational, - operational_reason_code=f"{operational}_synthetic_reason", - operational_reason=f"Synthetic {operational} operational reason.", - operational_evidence=("Synthetic operational evidence.",), - activated_at=(None if lifecycle == "onboarding" else activated), - last_meaningful_activity_at=NOW - timedelta(days=inactive_days), - inactive_days=inactive_days, - active_days_30d=3 if lifecycle == "core" else 1, - successful_backtests_30d=3 if lifecycle == "core" else 0, - calculated_at=NOW, - ) - - def _commercial( user_id: int, purchased: int = 0, @@ -92,20 +76,43 @@ def _commercial( ) -def _daily( - user_id: int, - day: date, - lifecycle: str = "core", - quality: str = "complete", -) -> UserLifecycleDailySnapshot: - return UserLifecycleDailySnapshot( +def _fact(user_id: int, day: date, **overrides) -> UserDailyFact: + values = dict( snapshot_date=day, user_id=user_id, - lifecycle_segment=lifecycle, - lifecycle_reason_code=f"{lifecycle}_synthetic_reason", - data_quality=quality, + lifecycle_segment="core", + lifecycle_reason_code="core_repeated_value", + operational_state="healthy", + operational_reason_code="no_supported_issue", + tier="unpaid", + user_group="unknown", + active=True, + runs_requested=1, + runs_completed=1, + runs_failed=0, + runs_cancelled=0, + operator_cost_micro=0, + own_spend_micro=0, + data_quality="complete", calculated_at=NOW, ) + values.update(overrides) + return UserDailyFact(**values) + + +def _activity( + user_id: int, *, activated_days_ago: int | None = 60, active_days_ago: int = 2 +) -> UserActivity: + return UserActivity( + user_id=user_id, + activated_at=( + None + if activated_days_ago is None + else NOW - timedelta(days=activated_days_ago) + ), + last_meaningful_activity_at=NOW - timedelta(days=active_days_ago), + updated_at=NOW, + ) class FakeUserStore: @@ -126,27 +133,222 @@ def list_excluded_user_ids(self, *, include_admin_accounts): class FakeValueStore: - def __init__(self, *, snapshots, commercial, daily=(), credit_activity=None, ledger_days=()): - self.snapshots = dict(snapshots) - self.commercial = dict(commercial) - self.daily = list(daily) + """Fact-backed fake: answers every fact/activity read from in-memory rows.""" + + def __init__( + self, + *, + facts=(), + transitions=(), + activity=None, + commercial=None, + page_rows=(), + operational_facts=None, + internal_ids=(), + ledger_days=(), + credit_activity=None, + ): + self.facts = list(facts) + # user_id -> [datetime]: ledger movements, which retention counts as + # activity alongside the fact ``active`` flag. self.credit_activity = dict(credit_activity or {}) + self.transitions = list(transitions) + self.activity = dict(activity or {}) + self.commercial = dict(commercial or {}) + self.page_rows = list(page_rows) + self.operational_facts = dict(operational_facts or {}) + # UserActivity is frozen and carries no role, so "internal" is a set of ids. + self.internal_ids = set(internal_ids) # user_id -> [LedgerDayTotal]; None stands for an unconfigured reader. self.ledger_days = None if ledger_days is None else dict(ledger_days) self.commercial_windows = [] - self.daily_windows = [] self.ledger_calls = [] + self.page_calls = [] + self.recent_windows = [] + + # -- fact-table reads -- + def count_states_for_date(self, day): + cells = {} + for row in self.facts: + if row.snapshot_date != day: + continue + key = (row.operational_state, row.operational_reason_code, row.lifecycle_segment) + users, partial = cells.get(key, (0, 0)) + cells[key] = (users + 1, partial + (row.data_quality == "partial")) + return [ + FactStateCount( + operational_state=key[0], + operational_reason_code=key[1], + lifecycle_segment=key[2], + users=users, + partial_rows=partial, + ) + for key, (users, partial) in sorted( + cells.items(), key=lambda item: tuple(str(v) for v in item[0]) + ) + ] + + def count_segments_by_date(self, *, start, end): + cells = {} + for row in self.facts: + if not start <= row.snapshot_date < end: + continue + key = (row.snapshot_date, row.lifecycle_segment) + users, partial = cells.get(key, (0, 0)) + cells[key] = (users + 1, partial + (row.data_quality == "partial")) + return [ + SegmentDayCount( + snapshot_date=key[0], + lifecycle_segment=key[1], + users=users, + partial_rows=partial, + ) + for key, (users, partial) in sorted(cells.items()) + ] + + def count_transitions(self, *, start, end): + cells = {} + for row in self.transitions: + if not start <= row.snapshot_date < end: + continue + key = (row.from_segment, row.to_segment) + users, partial = cells.get(key, (0, 0)) + cells[key] = (users + 1, partial + (row.data_quality == "partial")) + return [ + TransitionCount( + from_segment=key[0], to_segment=key[1], users=users, partial_rows=partial + ) + for key, (users, partial) in sorted(cells.items()) + ] - def list_current_snapshots(self, user_ids): + def list_attention_candidates(self, *, day, window_start, limit): + return [] + + def sum_facts_by_user_group(self, *, start, end): + totals = {} + completion_days = {} + for row in self.facts: + if not start <= row.snapshot_date < end: + continue + group = totals.setdefault( + row.user_group, + {"total_runs": 0, "operator_cost_micro": 0, "successful": set(), "partial": 0}, + ) + group["partial"] += row.data_quality == "partial" + group["total_runs"] += row.runs_requested + group["operator_cost_micro"] += row.operator_cost_micro + if row.runs_completed > 0: + group["successful"].add(row.user_id) + completion_days.setdefault((row.user_group, row.user_id), set()).add( + row.snapshot_date + ) return { - user_id: self.snapshots[user_id] + name: GroupFactTotals( + total_runs=values["total_runs"], + operator_cost_micro=values["operator_cost_micro"], + successful_run_users=len(values["successful"]), + repeat_users=sum( + 1 + for (group, _user), days in completion_days.items() + if group == name and len(days) >= 2 + ), + partial_rows=values["partial"], + ) + for name, values in totals.items() + } + + def list_active_dates(self, user_ids, *, start, end): + result = {user_id: [] for user_id in user_ids} + for row in sorted(self.facts, key=lambda r: (r.user_id, r.snapshot_date)): + if row.user_id in result and row.active and start <= row.snapshot_date < end: + result[row.user_id].append(row.snapshot_date) + return result + + def list_credit_activity(self, user_ids, *, start, end): + return { + user_id: tuple( + stamp + for stamp in self.credit_activity.get(user_id, ()) + if start <= stamp < end + ) for user_id in user_ids - if user_id in self.snapshots } - def get_current_snapshot(self, user_id): - return self.snapshots.get(user_id) + def list_activations(self, *, start, end, include_internal): + return { + user_id: activity.activated_at + for user_id, activity in self.activity.items() + if activity.activated_at is not None + and start <= activity.activated_at < end + and (include_internal or user_id not in self.internal_ids) + } + + def count_activated_users(self, *, include_internal): + return sum( + 1 + for user_id, activity in self.activity.items() + if activity.activated_at is not None + and (include_internal or user_id not in self.internal_ids) + ) + + def list_user_transitions(self, user_id, *, start, end): + return [ + row + for row in self.transitions + if row.user_id == user_id and start <= row.snapshot_date < end + ] + + def list_transitions_for_date(self, day): + return [row for row in self.transitions if row.snapshot_date == day] + + def latest_fact_date(self, *, on_or_before): + days = [row.snapshot_date for row in self.facts if row.snapshot_date <= on_or_before] + return max(days) if days else None + def list_users_page(self, *, query, day, limit, offset): + self.page_calls.append((query, day, limit, offset)) + rows = list(self.page_rows) + return rows[offset : offset + limit], len(rows) + + # -- activity clock and trailing windows -- + def get_activity(self, user_id): + return self.activity.get(user_id) + + def list_activity(self, user_ids): + return { + user_id: self.activity[user_id] + for user_id in user_ids + if user_id in self.activity + } + + def sum_recent_facts(self, user_ids, *, start, end): + self.recent_windows.append((start, end)) + result = {} + for user_id in user_ids: + rows = [ + r for r in self.facts if r.user_id == user_id and start <= r.snapshot_date <= end + ] + if not rows: + continue + active_dates = [r.snapshot_date for r in rows if r.active] + result[user_id] = RecentFactTotals( + active_days=len(active_dates), + successful_backtests=sum(r.runs_completed for r in rows), + runs_requested=sum(r.runs_requested for r in rows), + runs_completed=sum(r.runs_completed for r in rows), + runs_failed=sum(r.runs_failed for r in rows), + runs_cancelled=sum(r.runs_cancelled for r in rows), + operator_cost_micro=sum(r.operator_cost_micro for r in rows), + own_spend_micro=sum(r.own_spend_micro for r in rows), + days_present=len(rows), + last_active_date=max(active_dates) if active_dates else None, + ) + return result + + def get_operational_facts(self, user_id, *, now): + return self.operational_facts[user_id] + + # -- commercial and ledger -- def list_commercial_values(self, user_ids, *, start, end): self.commercial_windows.append((start, end)) return { @@ -155,16 +357,6 @@ def list_commercial_values(self, user_ids, *, start, end): if user_id in self.commercial } - def list_daily_snapshots(self, *, start, end, user_ids=None): - self.daily_windows.append((start, end)) - selected = None if user_ids is None else set(user_ids) - return [ - row - for row in self.daily - if start <= row.snapshot_date < end - and (selected is None or row.user_id in selected) - ] - def list_ledger_days(self, user_ids, *, start, end): self.ledger_calls.append((list(user_ids), start, end)) if self.ledger_days is None: @@ -176,50 +368,28 @@ def list_ledger_days(self, user_ids, *, start, end): if start.date() <= row.day < end.date() ] - def list_credit_activity(self, user_ids, *, start, end): - return { - user_id: tuple( - value - for value in self.credit_activity.get(user_id, ()) - if start <= value < end - ) - for user_id in user_ids - } - class FakeRollups: - def __init__(self, events=(), rollups=()): - self.events = list(events) + def __init__(self, rollups=()): self.rollups = list(rollups) def list_events(self, *, start, end, include_internal, user_id=None): - assert include_internal is True - return [ - event - for event in self.events - if start <= event.occurred_at < end - and (user_id is None or event.user_id == user_id) - ] + raise AssertionError("this read must not scan raw events") def list_rollups(self, *, start, end): return [row for row in self.rollups if start <= row.rollup_date < end] class FakeQueryStore: - def __init__(self, *, events=(), rollups=(), legacy_status=None): - self.rollups = FakeRollups(events, rollups) - self.legacy_status = dict(legacy_status or {}) - - def list_snapshots(self): - return { - user_id: SimpleNamespace(status=status) - for user_id, status in self.legacy_status.items() - } + def __init__(self, *, rollups=()): + self.rollups = FakeRollups(rollups) class FakeLegacyService: - def __init__(self, availability=None): + def __init__(self, availability=None, *, operational_facts=None): self.profile_windows = [] + self.operational_facts = dict(operational_facts or {}) + self.raise_lookup = False self.overview_calls = 0 self.platform_cost_filters = [] self.availability = availability or {"growth": True, "friction": True} @@ -246,7 +416,28 @@ def get_overview(self, *, filters, now): top_failure_categories=[], ) - def get_user_profile(self, *, user_id, now, start=None, end=None): + def get_user_axes(self, user_id, *, now): + from dashboard.backend.domain.analytics.lifecycle import ( + LifecycleInputs, OperationalSignals, calculate_lifecycle, calculate_operational_state, + ) + if self.raise_lookup: + raise LookupError("Analytics user was not found") + facts = self.operational_facts.get(user_id) + operational = calculate_operational_state( + OperationalSignals(**facts.model_dump()) if facts else OperationalSignals(user_id=user_id), now + ) + lifecycle = calculate_lifecycle( + LifecycleInputs(user_id=user_id, created_at=now - timedelta(days=90), + first_successful_backtest_at=now - timedelta(days=60), + last_meaningful_activity_at=now - timedelta(days=2), + active_days_30d=3, successful_backtests_30d=3), + now, + ) + return lifecycle, operational + + def get_user_profile(self, *, user_id, now, start=None, end=None, axes=None): + if self.raise_lookup: + raise LookupError("Analytics user was not found") self.profile_windows.append((start, end)) return AnalyticsUserProfile( user_id=user_id, @@ -269,354 +460,513 @@ def get_user_profile(self, *, user_id, now, start=None, end=None): def _service( *, - snapshots, + facts=(), + transitions=(), + activity=None, commercial=None, - daily=(), - events=(), - user_groups=None, + page_rows=(), + operational_facts=None, + internal_ids=(), + users=(), rollups=(), excluded=(), legacy_availability=None, ledger_days=(), ): - facts = commercial or {user_id: _commercial(user_id) for user_id in snapshots} - group_values = dict(user_groups or {}) + user_ids = [user_id for user_id, _group in users] value_store = FakeValueStore( - snapshots=snapshots, - commercial=facts, - daily=daily, + facts=facts, + transitions=transitions, + activity=activity, + commercial=commercial or {user_id: _commercial(user_id) for user_id in user_ids}, + page_rows=page_rows, + operational_facts=operational_facts, + internal_ids=internal_ids, ledger_days=ledger_days, ) - legacy_service = FakeLegacyService(legacy_availability) + legacy_service = FakeLegacyService( + legacy_availability, operational_facts=operational_facts + ) service = ValueAnalyticsQueryService( store=FakeBaseStore(excluded), - user_store=FakeUserStore( - [_user(user_id, group_values.get(user_id, "unknown")) for user_id in snapshots] - ), + user_store=FakeUserStore([_user(user_id, group) for user_id, group in users]), value_store=value_store, - query_store=FakeQueryStore(events=events, rollups=rollups), + query_store=FakeQueryStore(rollups=rollups), legacy_service=legacy_service, ) return service, value_store, legacy_service -def _metric_event( - user_id: int, - event_name: str, - occurred_at: str | datetime, - *, - billing_mode: str | None = None, - properties: dict[str, object] | None = None, -): - at = ( - datetime.fromisoformat(occurred_at) - if isinstance(occurred_at, str) - else occurred_at - ) - return SimpleNamespace( - user_id=user_id, - event_name=event_name, - occurred_at=at, - billing_mode=billing_mode, - properties=properties or {}, +def _retention_service(): + # Cohort week Monday 2026-08-03: users 1 and 2 activated that week; user 3 in the next. + activity = { + 1: _activity(1, activated_days_ago=(NOW.date() - date(2026, 8, 4)).days), + 2: _activity(2, activated_days_ago=(NOW.date() - date(2026, 8, 7)).days), + 3: _activity(3, activated_days_ago=(NOW.date() - date(2026, 8, 12)).days), + 9: _activity(9, activated_days_ago=(NOW.date() - date(2026, 8, 5)).days), + } + facts = [] + # Every day from the cohort start through the latest ``now`` the tests use + # has at least one row, so no week is partial for lack of data. + day = date(2026, 8, 3) + while day <= RETENTION_LAST_FACT_DAY: + facts.append(_fact(3, day, active=False, runs_completed=0)) + day += timedelta(days=1) + # Week 1 (2026-08-10..16): user 1 active, user 2 not. Week 2 (08-17..23): both. + # Week 4 (08-31..09-06): user 2 only. + facts += [ + _fact(1, date(2026, 8, 11)), + _fact(1, date(2026, 8, 18)), + _fact(2, date(2026, 8, 20)), + _fact(2, date(2026, 9, 1)), + ] + return _service( + facts=facts, + activity=activity, + internal_ids={9}, + users=((1, "organic"), (2, "organic"), (3, "organic"), (9, "internal")), + )[0] + + +# Week 4 of the 2026-08-03 cohort ends 2026-09-07, so it is mature from 09-07 on. +RETENTION_NOW = datetime(2026, 9, 10, 12, 0, tzinfo=UTC) +RETENTION_LAST_FACT_DAY = RETENTION_NOW.date() - timedelta(days=1) + + +def test_retention_cells_come_from_activation_weeks_and_active_fact_days(): + service = _retention_service() + + response = service.get_retention( + start=date(2026, 8, 1), end=date(2026, 8, 20), now=RETENTION_NOW ) + first = next(c for c in response.cohorts if c.cohort_week == date(2026, 8, 3)) + assert first.activated_users == 2 # user 9 is internal + assert (first.week_1.retained_users, first.week_1.eligible_users, first.week_1.rate) == (1, 2, 0.5) + assert (first.week_2.retained_users, first.week_2.eligible_users) == (2, 2) + assert (first.week_4.retained_users, first.week_4.eligible_users, first.week_4.rate) == (1, 2, 0.5) + second = next(c for c in response.cohorts if c.cohort_week == date(2026, 8, 10)) + assert second.activated_users == 1 + assert second.week_1.retained_users == 0 + assert response.summary_week_1.rate == 1 / 3 + assert response.availability.status == "ready" + -def _activity(user_id: int, occurred_at: datetime): - return SimpleNamespace( - user_id=user_id, - event_name="agent_updated", - occurred_at=occurred_at, +def test_retention_counts_a_ledger_movement_as_activity(): + """Review #9: main unioned list_credit_activity into retention; the fact + ``active`` flag is set from analytics events alone, so a cohort member + whose only week-1 activity was a Credits purchase read as churned.""" + service = _retention_service() + # User 2 had no active fact day in week 1 (2026-08-10..16). + service.value_store.credit_activity[2] = [datetime(2026, 8, 12, 15, 0, tzinfo=UTC)] + + response = service.get_retention( + start=date(2026, 8, 1), end=date(2026, 8, 20), now=RETENTION_NOW ) + first = next(c for c in response.cohorts if c.cohort_week == date(2026, 8, 3)) + assert (first.week_1.retained_users, first.week_1.eligible_users) == (2, 2) -def test_date_filter_changes_history_not_current_lifecycle_identity(): - snapshots = { - 1: _snapshot(1), - 2: _snapshot(2, lifecycle="at_risk", inactive_days=12), - } - daily = [ - _daily(1, date(2026, 8, 25), "growing"), - _daily(2, date(2026, 8, 25), "onboarding"), - _daily(1, date(2026, 9, 2), "core"), - _daily(2, date(2026, 9, 2), "at_risk"), - ] - service, _value_store, _legacy = _service(snapshots=snapshots, daily=daily) - short = service.get_lifecycle( - start=date(2026, 9, 1), - end=date(2026, 9, 4), - now=NOW, +def test_retention_cohort_membership_covers_the_window_end_week(): + # end falls mid-week: the 08-17 cohort starts before it and must still be read. + service = _retention_service() + service.value_store.activity[4] = _activity( + 4, activated_days_ago=(NOW.date() - date(2026, 8, 23)).days ) - long = service.get_lifecycle( - start=date(2026, 8, 24), - end=date(2026, 9, 4), - now=NOW, + + response = service.get_retention( + start=date(2026, 8, 1), end=date(2026, 8, 19), now=RETENTION_NOW ) - assert short.headline == long.headline - assert short.segment_counts == long.segment_counts - assert short.weekly_segments != long.weekly_segments + assert [c.cohort_week for c in response.cohorts] == [ + date(2026, 8, 3), + date(2026, 8, 10), + date(2026, 8, 17), + ] + assert response.cohorts[-1].activated_users == 1 -@pytest.mark.parametrize( - ("movement_range", "granularity"), - [("5d", "day"), ("1w", "day"), ("1m", "week"), ("1y", "month")], -) -def test_lifecycle_movement_returns_selected_range_and_granularity( - movement_range, granularity -): - snapshots = {1: _snapshot(1)} - daily = [ - _daily(1, date(2025, 10, 1) + timedelta(days=offset), "core") - for offset in range(365) - ] - service, _value_store, _legacy = _service(snapshots=snapshots, daily=daily) +def test_retention_marks_immature_weeks_unavailable_not_zero(): + service = _retention_service() + now = datetime(2026, 8, 25, 12, 0, tzinfo=UTC) - response = service.get_lifecycle( - start=date(2026, 4, 5), - end=date(2026, 10, 1), - movement_range=movement_range, - now=datetime(2026, 10, 1, 12, tzinfo=UTC), - ) - - assert response.movement_range == movement_range - assert response.movement_granularity == granularity - starts = [point.period_start for point in response.movement_segments] - assert starts - assert starts == sorted(set(starts)) - # Derived from the window rather than written down: a calendar bucket count - # depends on where the window falls in the calendar, so a hard-coded ceiling - # holds only for the dates this case happens to pick. - window_days = _MOVEMENT_WINDOWS[movement_range][0] - period_days = {"day": 1, "week": 7, "month": 28}[granularity] - assert len(starts) <= window_days // period_days + 2 - assert all( - date(2026, 10, 1) - timedelta(days=window_days) <= day < date(2026, 10, 1) - for day in starts - ) - - -def test_retention_uses_nulls_for_immature_cells_and_weighted_mature_summary(): - first_week = date(2026, 7, 6) - second_week = date(2026, 7, 13) - immature_week = date(2026, 8, 24) - snapshots = { - 1: _snapshot(1, activated_at=datetime(2026, 7, 6, 10, tzinfo=UTC)), - 2: _snapshot(2, activated_at=datetime(2026, 7, 7, 10, tzinfo=UTC)), - 3: _snapshot(3, activated_at=datetime(2026, 7, 13, 10, tzinfo=UTC)), - 4: _snapshot(4, activated_at=datetime(2026, 8, 24, 10, tzinfo=UTC)), - } - daily = [] - for cohort, members in ((first_week, (1, 2)), (second_week, (3,))): - for target_week in (1, 2, 4): - target_start = cohort + timedelta(days=target_week * 7) - daily.extend( - _daily(user_id, target_start + timedelta(days=day)) - for user_id in members - for day in range(7) - ) - events = [ - _activity(1, datetime(2026, 7, 14, 10, tzinfo=UTC)), - _activity(3, datetime(2026, 7, 21, 10, tzinfo=UTC)), + response = service.get_retention(start=date(2026, 8, 1), end=date(2026, 8, 20), now=now) + + first = next(c for c in response.cohorts if c.cohort_week == date(2026, 8, 3)) + assert first.week_2.mature is True + assert first.week_4.mature is False + assert first.week_4.retained_users is None and first.week_4.rate is None + assert response.summary_week_4.mature is False + + +def test_retention_week_with_a_missing_or_partial_fact_day_is_partial(): + service = _retention_service() + service.value_store.facts = [ + row for row in service.value_store.facts if row.snapshot_date != date(2026, 8, 12) ] - service, _value_store, _legacy = _service( - snapshots=snapshots, - daily=daily, - events=events, + service.value_store.facts.append( + _fact(3, date(2026, 8, 19), active=False, data_quality="partial") ) response = service.get_retention( - start=date(2026, 7, 1), - end=date(2026, 9, 1), - now=NOW, + start=date(2026, 8, 1), end=date(2026, 8, 20), now=RETENTION_NOW ) - assert response.summary_week_1.mature is True - assert response.summary_week_1.retained_users == 2 - assert response.summary_week_1.eligible_users == 3 - assert response.summary_week_1.rate == pytest.approx(2 / 3) - immature = next( - cohort for cohort in response.cohorts if cohort.cohort_week == immature_week + first = next(c for c in response.cohorts if c.cohort_week == date(2026, 8, 3)) + assert first.week_1.data_quality == "partial" # 08-12 has no rows + assert first.week_2.data_quality == "partial" # 08-19 carries a partial row + assert first.week_4.data_quality == "complete" + assert response.availability.status == "partial" + + +def test_retention_include_internal_does_not_deflate_rates(): + """I4: facts hold no internal users, so an internal activation admitted to a + cohort would count as eligible and never retained.""" + service = _retention_service() + window = dict(start=date(2026, 8, 1), end=date(2026, 8, 20), now=RETENTION_NOW) + + default = service.get_retention(**window) + internal = service.get_retention(include_internal=True, **window) + + assert internal.cohorts == default.cohorts + assert internal.summary_week_1 == default.summary_week_1 + assert internal.summary_week_4 == default.summary_week_4 + assert next( + c for c in internal.cohorts if c.cohort_week == date(2026, 8, 3) + ).activated_users == 2 # user 9 is internal + + +def test_retention_reads_no_raw_events(): + # FakeRollups.list_events raises; a passing call proves no raw read happened. + _retention_service().get_retention( + start=date(2026, 8, 1), end=date(2026, 8, 20), now=RETENTION_NOW ) - assert immature.week_1.mature is False - assert immature.week_1.retained_users is None - assert immature.week_1.eligible_users is None - assert immature.week_1.rate is None -def test_missing_or_partial_retention_history_propagates_partial_quality(): - cohort = date(2026, 7, 6) - snapshots = { - 1: _snapshot(1, activated_at=datetime(2026, 7, 6, 10, tzinfo=UTC)), - 2: _snapshot(2, activated_at=datetime(2026, 7, 7, 10, tzinfo=UTC)), - } - target_start = cohort + timedelta(days=7) - daily = [_daily(1, target_start + timedelta(days=day)) for day in range(7)] - daily.extend( - _daily(2, target_start + timedelta(days=day), quality="partial") - for day in range(6) +def _page_row(user_id: int, **overrides) -> UserPageRow: + values = dict( + user_id=user_id, email=f"value-{user_id}@example.test", display_name=f"Value User {user_id}", + role="user", user_group="unknown", created_at=NOW - timedelta(days=90), + activated_at=NOW - timedelta(days=60), last_meaningful_activity_at=NOW - timedelta(days=2), + lifecycle_segment="core", operational_state="healthy", + operational_reason_code="no_supported_issue", tier="unpaid", data_quality="complete", ) - service, _value_store, _legacy = _service(snapshots=snapshots, daily=daily) + values.update(overrides) + return UserPageRow(**values) - response = service.get_retention( - start=date(2026, 7, 1), - end=date(2026, 7, 20), - now=NOW, + +def _utc_midnight(day: date) -> datetime: + return datetime.combine(day, time.min, tzinfo=UTC) + + +def test_users_list_reads_one_sql_page_and_computes_live_segments(): + rows = [ + _page_row(1, operational_state="blocked", operational_reason_code="account_restricted", tier="invested"), + _page_row(2, lifecycle_segment="growing", last_meaningful_activity_at=NOW - timedelta(days=9)), + ] + facts = [_fact(1, YESTERDAY - timedelta(days=offset)) for offset in range(3)] + activity = {1: _activity(1), 2: _activity(2, active_days_ago=9)} + service, value_store, _legacy = _service( + page_rows=rows, facts=facts, activity=activity, + commercial={1: _commercial(1, 6_000_000), 2: _commercial(2)}, ) - assert response.cohorts[0].week_1.data_quality == "partial" - assert response.summary_week_1.data_quality == "partial" - assert response.availability.status == "partial" + response = service.list_users(filters=UserValueFilters(priority=True), limit=25, offset=0, now=NOW) + + assert response.total == 2 + first, second = response.items + assert first.user_id == 1 + assert first.lifecycle.segment == "core" # live: 3 active days, 3 successes + assert first.operational.state == "blocked" # yesterday's fact row + assert first.operational.reason == "The Credits account is restricted from model spending." + assert first.commercial_tier == "invested" + assert first.lifetime_net_purchased_micro == 6_000_000 + assert first.priority_group == "blocked" + assert second.lifecycle.segment == "at_risk" # live: 9 idle days, no fact rows + assert second.priority_group == "none" # yesterday's segment was growing + query, day, limit, offset = value_store.page_calls[0] + assert (day, limit, offset) == (YESTERDAY, 25, 0) + assert query.priority is True + assert value_store.recent_windows == [(YESTERDAY - timedelta(days=29), YESTERDAY)] + assert value_store.commercial_windows == [ + (_utc_midnight(YESTERDAY), _utc_midnight(YESTERDAY + timedelta(days=1))) + ] -def test_priority_order_is_group_value_inactivity_then_user_id(): - snapshots = { - 10: _snapshot(10, operational="blocked", inactive_days=3), - 11: _snapshot(11, operational="blocked", inactive_days=20), - 12: _snapshot(12, operational="blocked", inactive_days=8), - 13: _snapshot(13, operational="blocked", inactive_days=8), - 20: _snapshot(20, operational="needs_attention"), - 30: _snapshot(30, lifecycle="at_risk", inactive_days=15), - 40: _snapshot(40, lifecycle="onboarding", inactive_days=4), - 50: _snapshot(50, lifecycle="core"), - } - commercial = { - 10: _commercial(10, 6_000_000), - 11: _commercial(11, 4_000_000), - 12: _commercial(12, 3_000_000), - 13: _commercial(13, 3_000_000), - 20: _commercial(20, 30_000_000), - 30: _commercial(30, 30_000_000), - 40: _commercial(40, 30_000_000), - 50: _commercial(50, 30_000_000), - } +def test_users_list_passes_every_filter_to_sql_unchanged(): + service, value_store, _legacy = _service(page_rows=[], activity={}) + filters = UserValueFilters( + q="Ada", lifecycle_segment="core", operational_state="healthy", commercial_tier="starter", + user_group="partner", activated=True, last_meaningful_activity_from=NOW - timedelta(days=3), + last_meaningful_activity_to=NOW, priority=False, include_internal=True, + ) + + response = service.list_users(filters=filters, limit=10, offset=20, now=NOW) + + assert response.total == 0 and response.items == [] + query, _day, limit, offset = value_store.page_calls[0] + assert (limit, offset) == (10, 20) + assert query == UserPageQuery( + q="Ada", lifecycle_segment="core", operational_state="healthy", commercial_tier="starter", + user_group="partner", activated=True, last_meaningful_activity_from=NOW - timedelta(days=3), + last_meaningful_activity_to=NOW, priority=False, include_internal=True, + ) + + +def test_users_list_row_without_a_fact_row_reads_as_healthy_unpaid(): service, _value_store, _legacy = _service( - snapshots=snapshots, - commercial=commercial, + page_rows=[_page_row( + 7, lifecycle_segment=None, data_quality=None, activated_at=None, + last_meaningful_activity_at=None, + )], + activity={}, commercial={}, ) - page = service.list_users( - filters=UserValueFilters(priority=True), - limit=25, - offset=0, - now=NOW, + response = service.list_users(filters=UserValueFilters(), limit=25, offset=0, now=NOW) + + item = response.items[0] + assert item.operational.state == "healthy" + assert item.commercial_tier == "unpaid" + assert item.lifetime_net_purchased_micro == 0 + assert item.lifecycle.segment == "dormant" # created 90 days ago, no activity row + assert item.priority_group == "none" + + +def test_users_list_falls_back_to_the_latest_landed_fact_day_and_warns(capsys): + """I5: before the night's facts land, or while the job fails, the page joins + the latest landed day and says so, instead of reading everyone as healthy.""" + stale_day = YESTERDAY - timedelta(days=1) + service, value_store, _legacy = _service( + facts=[_fact(1, stale_day)], activity={1: _activity(1)}, commercial={}, + ) + blocked = _page_row( + 1, operational_state="blocked", operational_reason_code="account_restricted" ) - assert [item.user_id for item in page.items] == [10, 11, 12, 13, 20, 30, 40] + def page(*, query, day, limit, offset): + value_store.page_calls.append((query, day, limit, offset)) + # Only the stale day carries the fact join; yesterday has no rows. + return ([blocked], 1) if day == stale_day else ([_page_row(1)], 1) + value_store.list_users_page = page -def test_user_list_uses_injected_utc_day_for_commercial_window(): - service, value_store, _legacy = _service(snapshots={1: _snapshot(1)}) + response = service.list_users(filters=UserValueFilters(), limit=25, offset=0, now=NOW) - service.list_users( - filters=UserValueFilters(), - limit=25, - offset=0, - now=NOW, + assert value_store.page_calls[0][1] == stale_day + item = response.items[0] + assert item.operational.state == "blocked" + assert item.operational.calculated_at == _utc_midnight(YESTERDAY) + assert ( + "WARNING: analytics.read.facts_stale " + f"latest={stale_day.isoformat()} expected={YESTERDAY.isoformat()}" + ) in capsys.readouterr().out + + # Once per stale day, not once per request: a failed job would otherwise + # write the line on every page load and filter click all day. + service.list_users(filters=UserValueFilters(q="a"), limit=25, offset=25, now=NOW) + assert "facts_stale" not in capsys.readouterr().out + + +def test_users_list_with_no_facts_at_all_keeps_yesterday_and_warns(capsys): + service, value_store, _legacy = _service(page_rows=[_page_row(1)], activity={}, commercial={}) + + response = service.list_users(filters=UserValueFilters(), limit=25, offset=0, now=NOW) + + assert value_store.page_calls[0][1] == YESTERDAY + assert response.items[0].operational.calculated_at == _utc_midnight(NOW.date()) + assert ( + "WARNING: analytics.read.facts_stale " + f"latest=none expected={YESTERDAY.isoformat()}" + ) in capsys.readouterr().out + + +def test_users_list_with_yesterdays_facts_landed_is_silent(capsys): + service, value_store, _legacy = _service( + page_rows=[_page_row(1)], facts=[_fact(1, YESTERDAY)], activity={}, commercial={}, ) - assert value_store.commercial_windows == [ - ( - datetime(2026, 8, 4, tzinfo=UTC), - datetime(2026, 9, 4, tzinfo=UTC), - ) + service.list_users(filters=UserValueFilters(), limit=25, offset=0, now=NOW) + + assert value_store.page_calls[0][1] == YESTERDAY + assert "facts_stale" not in capsys.readouterr().out + + +def test_users_list_priority_groups_are_a_pure_function_of_state_and_segment(): + # The store maps a NULL code to no_supported_issue before the service sees it. + rows = [ + _page_row(1), + _page_row(2, operational_state="needs_attention", operational_reason_code="three_consecutive_failed_runs", lifecycle_segment="at_risk"), + _page_row(3, lifecycle_segment="at_risk"), + _page_row(4, lifecycle_segment="onboarding"), ] + service, _value_store, _legacy = _service(page_rows=rows, activity={}, commercial={}) + items = service.list_users(filters=UserValueFilters(), limit=25, offset=0, now=NOW).items -def test_group_summary_has_six_rows_zeroes_and_fixed_order(): - service, _value_store, _legacy = _service( - snapshots={1: _snapshot(1)}, - user_groups={1: "organic"}, - events=[ - _metric_event( - 1, - "backtest_requested", - "2026-09-01T01:00:00+00:00", - ), - _metric_event( - 1, - "backtest_completed", - "2026-09-01T02:00:00+00:00", - ), - _metric_event( - 1, - "backtest_completed", - "2026-09-03T02:00:00+00:00", - ), - ], + assert items[0].operational.state == "healthy" + assert items[0].operational.reason_code == "no_supported_issue" + assert [item.priority_group for item in items] == [ + "none", "needs_attention", "healthy_at_risk", "healthy_onboarding" + ] + + +def _groups_service(**overrides): + facts = [ + _fact(1, YESTERDAY, user_group="organic", runs_requested=3, runs_completed=2, operator_cost_micro=200_000), + _fact(1, YESTERDAY - timedelta(days=1), user_group="organic", runs_requested=1, runs_completed=1, operator_cost_micro=50_000), + _fact(2, YESTERDAY, user_group="organic", runs_requested=2, runs_completed=0, runs_failed=2), + _fact(3, YESTERDAY, user_group="partner", runs_requested=1, runs_completed=1), + _fact(4, YESTERDAY - timedelta(days=20), user_group="partner", runs_requested=9, runs_completed=9), + # The job writes a row per eligible user per day, active or not, so + # every landed day has rows; user 5's idle days stand for that here. + *(_fact(5, YESTERDAY - timedelta(days=offset), user_group="unknown") for offset in range(7)), + ] + values = dict( + facts=facts, + users=((1, "organic"), (2, "organic"), (3, "partner"), (4, "partner"), (5, "unknown")), + commercial={1: _commercial(1, 6_000_000), 2: _commercial(2), 3: _commercial(3, 1), 4: _commercial(4), 5: _commercial(5)}, + ) + values.update(overrides) + return _service(**values)[0:2] + + +def test_group_summary_has_six_rows_in_fixed_order_with_facts_totals(): + service, _store = _groups_service() + + response = service.get_groups( + start=YESTERDAY - timedelta(days=6), end=YESTERDAY + timedelta(days=1), now=NOW ) - result = service.get_groups( - date(2026, 9, 1), date(2026, 9, 4), now=NOW + assert [row.group for row in response.groups] == list(USER_GROUPS) + by_group = {row.group: row for row in response.groups} + assert (by_group["organic"].users, by_group["organic"].total_runs) == (2, 6) + assert by_group["organic"].atl_cost_micro_usd == 250_000 + assert by_group["organic"].successful_run_users == 1 + assert by_group["organic"].repeat_users == 1 + assert by_group["organic"].paid_users == 1 + assert (by_group["partner"].users, by_group["partner"].total_runs) == (2, 1) # user 4's row is outside the window + assert by_group["partner"].paid_users == 1 + assert by_group["unknown"].users == 1 + assert by_group["internal"] == UserGroupSummary( + group="internal", label="Internal", users=0, successful_run_users=0, + repeat_users=0, total_runs=0, atl_cost_micro_usd=0, paid_users=0, + ) + assert response.availability.status == "ready" + assert response.availability.coverage_end == YESTERDAY + + +def test_group_filter_zeroes_every_other_group(): + service, _store = _groups_service() + + response = service.get_groups( + start=YESTERDAY - timedelta(days=6), end=YESTERDAY + timedelta(days=1), + user_group="partner", now=NOW, + ) + + by_group = {row.group: row for row in response.groups} + assert response.selected_user_group == "partner" + assert by_group["partner"].total_runs == 1 + assert by_group["organic"].total_runs == 0 + assert by_group["organic"].users == 0 + assert by_group["organic"].paid_users == 0 + + +def test_group_summary_is_partial_when_the_fact_read_fails(capsys): + service, store = _groups_service() + + def explode(**_kwargs): + raise RuntimeError("private detail") + + store.sum_facts_by_user_group = explode + + response = service.get_groups( + start=YESTERDAY - timedelta(days=6), end=YESTERDAY + timedelta(days=1), now=NOW ) - assert [row.group for row in result.groups] == [ - "internal", - "invited", - "organic", - "competition", - "partner", - "unknown", + assert response.availability.available is True + assert response.availability.status == "partial" + assert all(row.total_runs == 0 for row in response.groups) + out = capsys.readouterr().out + assert "WARNING: analytics.read.groups_facts_failed category=RuntimeError" in out + assert "private detail" not in out + + +def test_group_summary_is_partial_when_any_fact_row_in_the_window_is_partial(): + """I2: the PR A history copy wrote zero runs as partial rows; a window + reaching them must not label those zeros ready.""" + partial_day = YESTERDAY - timedelta(days=2) + facts = [ + row + for row in _groups_service()[1].facts + if (row.user_id, row.snapshot_date) != (5, partial_day) + ] + [ + _fact(5, partial_day, user_group="unknown", + runs_requested=0, runs_completed=0, data_quality="partial"), ] - organic = result.groups[2] - assert organic.users == 1 - assert organic.successful_run_users == 1 - assert organic.repeat_users == 1 - assert organic.total_runs == 1 - assert result.groups[0].users == 0 - assert result.availability.status == "ready" + service, _store = _groups_service(facts=facts) + response = service.get_groups( + start=YESTERDAY - timedelta(days=6), end=YESTERDAY + timedelta(days=1), now=NOW + ) -def test_group_summary_cost_excludes_byok_and_paid_uses_lifetime_credits(): - service, _value_store, _legacy = _service( - snapshots={1: _snapshot(1), 2: _snapshot(2)}, - user_groups={1: "partner", 2: "organic"}, - commercial={ - 1: _commercial(1, 5_000_000), - 2: _commercial(2, 0), - }, - events=[ - _metric_event( - 1, - "model_usage_recorded", - "2026-09-01T01:00:00+00:00", - billing_mode="platform_credits", - properties={"cost_micro_usd": 1234}, - ), - _metric_event( - 1, - "model_usage_recorded", - "2026-09-01T02:00:00+00:00", - billing_mode="byok", - properties={"cost_micro_usd": 9999}, - ), - ], + assert response.availability.available is True + assert response.availability.status == "partial" + assert response.availability.coverage_end == YESTERDAY + # Outside the selected group the partial row does not downgrade the answer. + filtered = service.get_groups( + start=YESTERDAY - timedelta(days=6), end=YESTERDAY + timedelta(days=1), + user_group="organic", now=NOW, ) + assert filtered.availability.status == "ready" - result = service.get_groups( - date(2026, 9, 1), date(2026, 9, 2), now=NOW + +def test_group_summary_clamps_coverage_to_yesterday_for_a_window_ending_today(): + service, _store = _groups_service() + + response = service.get_groups( + start=YESTERDAY - timedelta(days=6), end=NOW.date() + timedelta(days=1), now=NOW ) - partner = next(row for row in result.groups if row.group == "partner") - organic = next(row for row in result.groups if row.group == "organic") - assert partner.atl_cost_micro_usd == 1234 - assert partner.paid_users == 1 - assert organic.atl_cost_micro_usd == 0 - assert organic.paid_users == 0 + assert response.availability.status == "ready" + assert response.availability.coverage_end == YESTERDAY + +def test_group_summary_does_not_read_an_unlanded_window_as_zero_runs(): + service, store = _groups_service() + store.sum_facts_by_user_group = lambda **_kwargs: pytest.fail("no day has landed") -def test_user_group_filter_applies_to_priority_users(): - service, _value_store, _legacy = _service( - snapshots={1: _snapshot(1, operational="blocked"), 2: _snapshot(2)}, - user_groups={1: "partner", 2: "organic"}, + response = service.get_groups( + start=NOW.date(), end=NOW.date() + timedelta(days=1), now=NOW + ) + + assert response.availability.available is True + assert response.availability.status == "partial" + assert response.availability.coverage_end is None + assert {row.group: row.users for row in response.groups}["organic"] == 2 + + +def test_group_summary_reads_a_past_day_without_facts_as_unknown_not_zero(): + """Review #4: start < today is not "landed". Before the night's job lands + (or when it fails outright) yesterday has no fact rows, and summing it as + zero runs under "ready" is the answer the landed check exists to prevent.""" + facts = [row for row in _groups_service()[1].facts if row.snapshot_date != YESTERDAY] + service, _store = _groups_service(facts=facts) + + response = service.get_groups( + start=YESTERDAY - timedelta(days=6), end=YESTERDAY + timedelta(days=1), now=NOW ) - filters = UserValueFilters(user_group="partner", priority=True) - page = service.list_users(filters=filters, limit=25, offset=0, now=NOW) + assert response.availability.status == "partial" + assert response.availability.coverage_start == YESTERDAY - timedelta(days=6) + assert response.availability.coverage_end == YESTERDAY - timedelta(days=1) - assert filters.user_group == "partner" - assert [item.user_id for item in page.items] == [1] + only_yesterday = service.get_groups( + start=YESTERDAY, end=YESTERDAY + timedelta(days=1), now=NOW + ) + assert only_yesterday.availability.status == "partial" + assert only_yesterday.availability.coverage_end is None + assert all(row.total_runs == 0 for row in only_yesterday.groups) def test_commercial_response_keeps_revenue_usage_grants_cost_and_balances_separate(): @@ -640,7 +990,7 @@ def test_commercial_response_keeps_revenue_usage_grants_cost_and_balances_separa ), } service, _value_store, _legacy = _service( - snapshots={1: _snapshot(1), 2: _snapshot(2)}, + users=((1, "unknown"), (2, "unknown")), commercial=facts, ) @@ -674,7 +1024,7 @@ def test_commercial_platform_cost_is_unknown_not_zero_when_its_read_failed(): behind the figure failed, the field is None so the page draws a dash, and the section is partial.""" service, _value_store, _legacy = _service( - snapshots={1: _snapshot(1)}, + users=((1, "unknown"),), legacy_availability={"growth": False, "friction": True}, ) @@ -688,7 +1038,7 @@ def test_commercial_reads_the_platform_cost_alone_not_a_whole_overview(): """The figure is one headline of the overview; building the whole overview for it scanned the range's raw events, rollups, snapshots and users a second time on every page load.""" - service, _value_store, legacy = _service(snapshots={1: _snapshot(1)}) + service, _value_store, legacy = _service(users=((1, "unknown"),)) service.get_commercial( start=date(2026, 8, 1), end=date(2026, 9, 1), include_internal=True, now=NOW @@ -710,7 +1060,7 @@ def test_commercial_series_sum_across_users_and_match_their_headlines(): from dashboard.backend.domain.credits.repository_common import LedgerDayTotal service, value_store, _legacy = _service( - snapshots={1: _snapshot(1), 2: _snapshot(2)}, + users=((1, "unknown"), (2, "unknown")), ledger_days={ 1: [ LedgerDayTotal(day=date(2026, 8, 3), purchased_micro=5_000_000, refunded_micro=2_000_000, consumed_micro=100_000), @@ -738,7 +1088,7 @@ def test_commercial_series_sum_across_users_and_match_their_headlines(): def test_commercial_series_chunk_user_ids_like_the_headline(): user_ids = range(1, 1202) service, value_store, _legacy = _service( - snapshots={user_id: _snapshot(user_id) for user_id in user_ids}, + users=tuple((user_id, "unknown") for user_id in user_ids), ) service.get_commercial(start=date(2026, 8, 1), end=date(2026, 9, 1), now=NOW) @@ -748,7 +1098,7 @@ def test_commercial_series_chunk_user_ids_like_the_headline(): def test_commercial_series_failure_is_partial_and_logged_not_an_empty_ready(capsys): service, _value_store, _legacy = _service( - snapshots={1: _snapshot(1)}, + users=((1, "unknown"),), ledger_days=None, ) @@ -767,7 +1117,8 @@ def test_operational_growth_figures_are_unknown_not_zero_when_their_read_failed( read. When it failed they are None, not 0 -- which is the real answer "no runs" -- even though the overview may have computed some before failing.""" service, _value_store, _legacy = _service( - snapshots={1: _snapshot(1)}, + facts=[_fact(1, YESTERDAY)], + users=((1, "unknown"),), legacy_availability={"growth": False, "friction": True}, ) @@ -783,220 +1134,451 @@ def test_operational_growth_figures_are_unknown_not_zero_when_their_read_failed( ) == (None, None, None, None, None, None) assert response.availability.status == "partial" - service, _value_store, _legacy = _service(snapshots={1: _snapshot(1)}) + service, _value_store, _legacy = _service(facts=[_fact(1, YESTERDAY)], users=((1, "unknown"),)) response = service.get_operational(start=date(2026, 8, 1), end=date(2026, 9, 1), now=NOW) assert (response.completed_runs, response.platform_model_cost_micro_usd) == (9, 250_000) -def test_missing_operational_subsection_is_reported_as_partial(): - service, _value_store, _legacy = _service( - snapshots={1: _snapshot(1)}, - legacy_availability={"growth": True, "friction": False}, +def _operational_service(**overrides): + facts = [ + _fact(1, YESTERDAY, operational_state="blocked", operational_reason_code="billing_lane_unavailable"), + _fact(2, YESTERDAY, operational_state="blocked", operational_reason_code="billing_lane_unavailable"), + _fact(3, YESTERDAY, operational_state="needs_attention", operational_reason_code="invalid_default_credential"), + _fact(4, YESTERDAY), + _fact(5, YESTERDAY - timedelta(days=1), operational_state="blocked", operational_reason_code="account_restricted"), + ] + values = dict(facts=facts, users=((1, "unknown"), (2, "unknown"), (3, "unknown"), (4, "unknown"))) + values.update(overrides) + return _service(**values)[0] + + +def test_operational_state_counts_are_yesterdays_facts(): + service = _operational_service() + + response = service.get_operational( + start=YESTERDAY - timedelta(days=6), end=YESTERDAY + timedelta(days=1), now=NOW + ) + + assert response.operational_state_counts == {"blocked": 2, "needs_attention": 1, "healthy": 1} + assert response.completed_runs == 9 # FakeLegacyService + assert response.availability.status == "ready" + + +def test_operational_state_counts_fall_back_to_the_latest_landed_day(): + """Review #10: /users falls back to the latest landed day before the + night's job lands; /operational read 0 blocked / 0 needing attention + until then. Both now read the same day, and this one says it is stale.""" + stale_day = YESTERDAY - timedelta(days=1) + service = _operational_service( + facts=[ + _fact(1, stale_day, operational_state="blocked", operational_reason_code="billing_lane_unavailable"), + _fact(2, stale_day), + ] ) response = service.get_operational( - start=date(2026, 8, 1), - end=date(2026, 9, 1), - now=NOW, + start=YESTERDAY - timedelta(days=6), end=YESTERDAY + timedelta(days=1), now=NOW + ) + + assert response.operational_state_counts == {"blocked": 1, "needs_attention": 0, "healthy": 1} + assert (response.availability.status, response.availability.stale) == ("partial", True) + assert (response.availability.coverage_start, response.availability.coverage_end) == ( + stale_day, stale_day + ) + + +def test_operational_is_building_when_no_fact_day_has_landed(): + service = _operational_service(facts=[]) + + response = service.get_operational( + start=YESTERDAY - timedelta(days=6), end=YESTERDAY + timedelta(days=1), now=NOW + ) + + assert set(response.operational_state_counts.values()) == {0} + assert response.availability.status == "building" + + +def test_operational_is_partial_when_the_overview_growth_panel_is_down(): + service = _operational_service(legacy_availability={"growth": False, "friction": True}) + + response = service.get_operational( + start=YESTERDAY - timedelta(days=6), end=YESTERDAY + timedelta(days=1), now=NOW ) assert response.availability.available is True assert response.availability.status == "partial" -def test_profile_includes_selected_period_and_lifecycle_transition(): - start = date(2026, 8, 1) - end = date(2026, 9, 1) - snapshots = {1: _snapshot(1)} - daily = [ - _daily(1, start - timedelta(days=1), "growing"), - _daily(1, start, "core"), +def test_missing_operational_subsection_is_reported_as_partial(): + service = _operational_service(legacy_availability={"growth": True, "friction": False}) + + response = service.get_operational( + start=YESTERDAY - timedelta(days=6), end=YESTERDAY + timedelta(days=1), now=NOW + ) + + assert response.availability.available is True + assert response.availability.status == "partial" + + +def test_operational_is_building_before_the_first_fact_day(): + service = _operational_service(facts=[]) + + response = service.get_operational( + start=YESTERDAY - timedelta(days=6), end=YESTERDAY + timedelta(days=1), now=NOW + ) + + assert response.operational_state_counts == {"blocked": 0, "needs_attention": 0, "healthy": 0} + assert response.availability.status == "building" + + +def test_null_operational_reason_codes_count_by_state_but_are_not_reasons(): + facts = [ + _fact(1, YESTERDAY, operational_state="healthy", operational_reason_code=None), + _fact(2, YESTERDAY, operational_state="blocked", operational_reason_code=None), + _fact(3, YESTERDAY, operational_state="blocked", operational_reason_code="billing_lane_unavailable"), ] - service, _value_store, legacy = _service(snapshots=snapshots, daily=daily) + service = _operational_service( + facts=facts, users=((1, "unknown"), (2, "unknown"), (3, "unknown")) + ) - profile = service.get_user_profile( - user_id=1, - start=start, - end=end, - now=NOW, + response = service.get_operational( + start=YESTERDAY - timedelta(days=6), end=YESTERDAY + timedelta(days=1), now=NOW ) + reasons = service.top_operational_reasons(day=YESTERDAY) + + assert response.operational_state_counts == {"blocked": 2, "needs_attention": 0, "healthy": 1} + assert [(r.reason_code, r.users) for r in reasons] == [("billing_lane_unavailable", 1)] + + +def test_top_operational_reasons_rank_blocking_reasons_by_users(): + service = _operational_service() + + reasons = service.top_operational_reasons(day=YESTERDAY) + + assert [(r.reason_code, r.users) for r in reasons] == [ + ("billing_lane_unavailable", 2), + ("invalid_default_credential", 1), + ] + assert reasons[0].reason == "No usable model billing lane is available." + assert service.top_operational_reasons(day=YESTERDAY, limit=1)[0].reason_code == "billing_lane_unavailable" + assert "top_operational_reasons" not in OperationalAnalyticsResponse.model_fields + + +def _day_start(day: date) -> datetime: + return datetime.combine(day, time.min, tzinfo=UTC) + + +def test_value_profile_combines_live_axes_facts_and_user_transitions(): + transitions = [ + LifecycleTransitionRow(user_id=1, snapshot_date=YESTERDAY, from_segment="growing", to_segment="core", + inactive_days=0, data_quality="complete", created_at=NOW), + # Another user's move on the same day stays off this profile. + LifecycleTransitionRow(user_id=2, snapshot_date=YESTERDAY, from_segment="growing", to_segment="at_risk", + inactive_days=8, data_quality="complete", created_at=NOW), + ] + service, _value_store, legacy = _service( + facts=[_fact(1, YESTERDAY - timedelta(days=offset)) for offset in range(3)], + transitions=transitions, activity={1: _activity(1)}, commercial={1: _commercial(1, 6_000_000)}, + operational_facts={1: CurrentOperationalFacts(user_id=1, account_restricted=True)}, + ) + + profile = service.get_user_profile(user_id=1, start=YESTERDAY - timedelta(days=6), end=YESTERDAY + timedelta(days=1), now=NOW) - assert profile.selected_period_start == start - assert profile.selected_period_end == end assert profile.lifecycle.segment == "core" - assert profile.operational.state == "healthy" + assert profile.operational.state == "blocked" + assert profile.operational.reason_code == "account_restricted" + assert profile.commercial.commercial_tier == "invested" + assert (profile.selected_period_start, profile.selected_period_end) == (YESTERDAY - timedelta(days=6), YESTERDAY + timedelta(days=1)) assert [ - (row.from_segment, row.to_segment) - for row in profile.recent_lifecycle_transitions - ] == [("growing", "core")] - assert legacy.profile_windows == [ - (datetime(2026, 8, 1, tzinfo=UTC), datetime(2026, 9, 1, tzinfo=UTC)) - ] + (t.from_segment, t.to_segment, t.users, t.period_start, t.period_end) + for t in profile.recent_lifecycle_transitions + ] == [("growing", "core", 1, YESTERDAY, YESTERDAY + timedelta(days=1))] + assert legacy.profile_windows == [(_day_start(YESTERDAY - timedelta(days=6)), _day_start(YESTERDAY + timedelta(days=1)))] -def test_profile_never_uses_anonymous_transition_rollups(): - start = date(2026, 8, 1) +def test_value_profile_never_uses_anonymous_transition_rollups(): aggregate = SimpleNamespace( - rollup_date=start, + rollup_date=YESTERDAY, metric_name="lifecycle_transition", event_name="growing", user_state="core", value_count=99, outcome="complete", ) - service, _value_store, _legacy = _service( - snapshots={1: _snapshot(1)}, - rollups=[aggregate], - ) + service, _value_store, _legacy = _service(users=((1, "unknown"),), rollups=[aggregate]) profile = service.get_user_profile( - user_id=1, - start=start, - end=date(2026, 9, 1), - now=NOW, + user_id=1, start=YESTERDAY - timedelta(days=6), end=YESTERDAY + timedelta(days=1), now=NOW ) assert profile.recent_lifecycle_transitions == [] +def test_value_profile_raises_lookup_error_for_an_unknown_user(): + service, _value_store, legacy = _service(activity={}, commercial={}) + legacy.raise_lookup = True + + with pytest.raises(LookupError): + service.get_user_profile(user_id=404, start=YESTERDAY - timedelta(days=6), end=YESTERDAY + timedelta(days=1), now=NOW) + + def test_internal_accounts_are_excluded_unless_explicitly_included(): - snapshots = {1: _snapshot(1), 2: _snapshot(2)} - service, _value_store, _legacy = _service( - snapshots=snapshots, - excluded={2}, + service, value_store, _legacy = _service(page_rows=[_page_row(1)], activity={}, commercial={}) + + service.list_users(filters=UserValueFilters(), limit=25, offset=0, now=NOW) + service.list_users(filters=UserValueFilters(include_internal=True), limit=25, offset=0, now=NOW) + + # The exclusion is a predicate of the SQL page; the service only forwards the flag. + assert [call[0].include_internal for call in value_store.page_calls] == [False, True] + + +def _lifecycle_service(**overrides): + facts = [ + _fact(1, YESTERDAY, lifecycle_segment="core"), + _fact(2, YESTERDAY, lifecycle_segment="at_risk", active=False), + _fact(3, YESTERDAY, lifecycle_segment="onboarding", runs_completed=0), + _fact(1, YESTERDAY - timedelta(days=1), lifecycle_segment="growing"), + _fact( + 2, + YESTERDAY - timedelta(days=1), + lifecycle_segment="growing", + data_quality="partial", + ), + _fact( + 3, YESTERDAY - timedelta(days=1), lifecycle_segment="new", runs_completed=0 + ), + ] + transitions = [ + LifecycleTransitionRow( + user_id=1, snapshot_date=YESTERDAY, from_segment="growing", + to_segment="core", inactive_days=0, data_quality="complete", + created_at=NOW, + ), + LifecycleTransitionRow( + user_id=2, snapshot_date=YESTERDAY, from_segment="growing", + to_segment="at_risk", inactive_days=8, data_quality="partial", + created_at=NOW, + ), + ] + activity = { + 1: _activity(1), + 2: _activity(2, active_days_ago=8), + 3: _activity(3, activated_days_ago=None), + 9: _activity(9), + } + values = dict( + facts=facts, + transitions=transitions, + activity=activity, + internal_ids={9}, + users=((1, "organic"), (2, "partner"), (3, "unknown"), (9, "internal")), + commercial={ + 1: _commercial(1, 6_000_000), + 2: _commercial(2), + 3: _commercial(3), + 9: _commercial(9), + }, ) + values.update(overrides) + return _service(**values)[0] - external = service.list_users( - filters=UserValueFilters(), - limit=25, - offset=0, - now=NOW, + +def test_lifecycle_counts_are_yesterdays_facts_and_headline_reads_activity(): + service = _lifecycle_service() + + response = service.get_lifecycle( + start=YESTERDAY - timedelta(days=6), end=YESTERDAY + timedelta(days=1), now=NOW ) - all_users = service.list_users( - filters=UserValueFilters(include_internal=True), - limit=25, - offset=0, - now=NOW, + + assert response.segment_counts == { + "new": 0, "onboarding": 1, "growing": 0, "core": 1, "at_risk": 1, "dormant": 0 + } + assert response.headline.core_users == 1 + assert response.headline.at_risk_users == 1 + assert response.headline.activated_users == 2 # user 9 is internal + assert response.headline.paid_users == 1 + current = response.availability["current"] + assert (current.status, current.coverage_start, current.coverage_end) == ( + "ready", YESTERDAY, YESTERDAY ) - assert [item.user_id for item in external.items] == [1] - assert [item.user_id for item in all_users.items] == [1, 2] +def test_lifecycle_current_falls_back_to_the_latest_landed_day_before_yesterdays_facts_land(): + """Review #10: until the night's job lands, the headline reads the day + /users falls back to, labelled stale, instead of 0 core / 0 at risk.""" + stale_day = YESTERDAY - timedelta(days=1) + service = _lifecycle_service(facts=[_fact(1, stale_day, lifecycle_segment="core")]) -def _transition_rollup(day: date, count: int, outcome: str = "complete"): - return SimpleNamespace( - rollup_date=day, - metric_name="lifecycle_transition", - event_name="growing", - user_state="core", - value_count=count, - outcome=outcome, + response = service.get_lifecycle( + start=YESTERDAY - timedelta(days=6), end=YESTERDAY + timedelta(days=1), now=NOW ) + assert response.headline.core_users == 1 + current = response.availability["current"] + assert (current.status, current.stale) == ("partial", True) + assert (current.coverage_start, current.coverage_end) == (stale_day, stale_day) -def test_lifecycle_transitions_ignore_rollups_outside_the_requested_window(): - """A long movement range widens the scan; it must not widen the totals. - `transitions` is stamped with the requested period, so a rollup from before - `start` that is summed in reports itself as having happened inside a window - it predates. - """ - start = date(2026, 9, 1) - end = date(2026, 10, 1) - daily = [ - _daily(1, start + timedelta(days=offset)) - for offset in range(30) - if start + timedelta(days=offset) != date(2026, 9, 10) - ] - service, _value_store, _legacy = _service( - snapshots={1: _snapshot(1)}, - daily=daily, - rollups=[ - _transition_rollup(date(2026, 9, 10), 2), - _transition_rollup(date(2026, 1, 15), 97), - ], +def test_lifecycle_current_is_building_when_no_fact_day_has_landed(): + service = _lifecycle_service(facts=[]) + + response = service.get_lifecycle( + start=YESTERDAY - timedelta(days=6), end=YESTERDAY + timedelta(days=1), now=NOW ) + assert set(response.segment_counts.values()) == {0} + assert response.availability["current"].status == "building" + assert response.availability["current"].stale is False + + +def test_lifecycle_include_internal_counts_internal_activations(): + service = _lifecycle_service() + response = service.get_lifecycle( - start=start, - end=end, - movement_range="1y", - now=datetime(2026, 10, 1, 12, tzinfo=UTC), + start=YESTERDAY - timedelta(days=6), + end=YESTERDAY + timedelta(days=1), + include_internal=True, + now=NOW, ) - assert [ - (row.from_segment, row.to_segment, row.users) for row in response.transitions - ] == [("growing", "core", 2)] + assert response.headline.activated_users == 3 + +def test_lifecycle_movement_and_transitions_come_from_facts_and_the_transition_log(): + service = _lifecycle_service() -def test_lifecycle_coverage_reports_the_requested_window_not_the_movement_history(): - """Coverage describes the window the admin asked for. + response = service.get_lifecycle( + start=YESTERDAY - timedelta(days=6), + end=YESTERDAY + timedelta(days=1), + movement_range="5d", + now=NOW, + ) - The movement chart needs a wider scan than the filter range, but - `availability.history` is what the Lifecycle distribution card renders, so - it must keep describing `start..end`. - """ - start = date(2026, 9, 1) - end = date(2026, 10, 1) - daily = [_daily(1, start + timedelta(days=offset)) for offset in range(30)] - daily.append(_daily(1, date(2025, 12, 1), quality="partial")) - service, _value_store, _legacy = _service(snapshots={1: _snapshot(1)}, daily=daily) + assert response.movement_granularity == "day" + assert [point.period_start for point in response.movement_segments] == [ + YESTERDAY - timedelta(days=1), + YESTERDAY, + ] + assert response.movement_segments[0].segment_counts["growing"] == 2 + assert response.movement_segments[0].data_quality == "partial" + assert response.movement_segments[1].segment_counts["core"] == 1 + assert { + (t.from_segment, t.to_segment, t.users, t.data_quality) + for t in response.transitions + } == { + ("growing", "core", 1, "complete"), + ("growing", "at_risk", 1, "partial"), + } + assert response.availability["history"].status == "partial" + assert response.availability["history"].coverage_start == YESTERDAY - timedelta( + days=1 + ) + + +def test_lifecycle_history_is_ready_through_yesterday_for_the_default_window(): + """The router's default ``end`` is today + 1, and facts exist only through + yesterday: a fully covered window must not read as incomplete for lacking + a row that cannot exist yet.""" + start = YESTERDAY - timedelta(days=6) + service = _lifecycle_service( + facts=[_fact(1, start + timedelta(days=offset)) for offset in range(7)], + transitions=[], + ) response = service.get_lifecycle( - start=start, - end=end, - movement_range="1y", - now=datetime(2026, 10, 1, 12, tzinfo=UTC), + start=start, end=NOW.date() + timedelta(days=1), now=NOW ) history = response.availability["history"] - assert history.coverage_start == start - assert history.coverage_end == end - timedelta(days=1) - assert history.status == "ready" + assert (history.status, history.coverage_start, history.coverage_end) == ( + "ready", start, YESTERDAY + ) -def test_lifecycle_movement_buckets_never_start_before_the_selected_window(): - """Calendar bucketing must not label a point outside the chart's own range.""" - start = date(2026, 9, 15) - end = date(2026, 10, 15) - window_start = end - timedelta(days=365) - daily = [_daily(1, window_start + timedelta(days=offset)) for offset in range(366)] - service, _value_store, _legacy = _service(snapshots={1: _snapshot(1)}, daily=daily) +def test_lifecycle_history_falls_back_to_rollups_for_days_without_facts(): + old_day = YESTERDAY - timedelta(days=3) + rollups = [ + DailyRollup( + rollup_date=old_day, metric_name="lifecycle_segment_count", + user_state="core", outcome="complete", value_count=7, updated_at=NOW, + ), + DailyRollup( + rollup_date=old_day, metric_name="lifecycle_transition", + event_name="onboarding", user_state="growing", outcome="complete", + value_count=2, updated_at=NOW, + ), + ] + service = _lifecycle_service(rollups=rollups) response = service.get_lifecycle( - start=start, - end=end, - movement_range="1y", - now=datetime(2026, 10, 15, 12, tzinfo=UTC), + start=YESTERDAY - timedelta(days=6), end=YESTERDAY + timedelta(days=1), now=NOW ) - assert response.movement_segments - assert min(point.period_start for point in response.movement_segments) >= window_start + assert any( + point.period_start == old_day and point.segment_counts["core"] == 7 + for point in response.movement_segments + ) + assert any( + t.from_segment == "onboarding" and t.to_segment == "growing" and t.users == 2 + for t in response.transitions + ) + # I4: facts are internal-free just as the rollups are, so the internal + # toggle has nothing to protect here and must not drop rollup-backed days. + internal = service.get_lifecycle( + start=YESTERDAY - timedelta(days=6), + end=YESTERDAY + timedelta(days=1), + include_internal=True, + now=NOW, + ) + assert internal.movement_segments == response.movement_segments + assert internal.transitions == response.transitions + assert internal.weekly_segments == response.weekly_segments -@pytest.mark.parametrize("movement_range", sorted(_MOVEMENT_WINDOWS)) -def test_lifecycle_daily_scan_stays_within_the_derived_history_bound(movement_range): - """Pins the widest span of per-user rows one request may read. - Not a regression test -- `_validate_dates` already bounds the filter range, - so the scan is bounded today. It is a drift guard: the movement window is - read from a table, and a longer entry added to that table would widen this - scan for every eligible user with nothing else noticing. - """ - start = date(2026, 9, 1) - end = date(2026, 10, 1) - service, value_store, _legacy = _service( - snapshots={1: _snapshot(1)}, - daily=[_daily(1, start + timedelta(days=offset)) for offset in range(30)], +def test_lifecycle_reads_no_raw_events(): + service = _lifecycle_service() + # FakeRollups.list_events raises. get_lifecycle swallows a failed read into + # an unavailable section, so the proof is that every section is available. + response = service.get_lifecycle( + start=YESTERDAY - timedelta(days=6), end=YESTERDAY + timedelta(days=1), now=NOW ) + assert all(section.available for section in response.availability.values()) + + +def test_lifecycle_failed_activation_read_is_partial_not_a_ready_zero(capsys): + service = _lifecycle_service() + + def fail(*, include_internal): + raise RuntimeError("user_activity unavailable") - service.get_lifecycle( - start=start, - end=end, - movement_range=movement_range, - now=datetime(2026, 10, 1, 12, tzinfo=UTC), + service.value_store.count_activated_users = fail + + response = service.get_lifecycle( + start=YESTERDAY - timedelta(days=6), end=YESTERDAY + timedelta(days=1), now=NOW + ) + + assert response.headline.activated_users == 0 + assert response.availability["current"].status == "partial" + assert response.segment_counts["core"] == 1 # the rest is still served + out = capsys.readouterr().out + assert "WARNING: analytics.read.lifecycle_activation_failed category=RuntimeError" in out + assert "user_activity unavailable" not in out + + +def test_lifecycle_failed_history_read_is_logged(capsys): + service = _lifecycle_service() + + def fail(**_kwargs): + raise RuntimeError("private detail") + + service.value_store.count_segments_by_date = fail + + response = service.get_lifecycle( + start=YESTERDAY - timedelta(days=6), end=YESTERDAY + timedelta(days=1), now=NOW ) - scan_start, scan_end = value_store.daily_windows[-1] - assert scan_end == end - assert scan_start >= end - timedelta(days=_MAX_HISTORY_SCAN_DAYS + 1) + assert response.availability["history"].status == "unavailable" + out = capsys.readouterr().out + assert "WARNING: analytics.read.lifecycle_history_failed category=RuntimeError" in out + assert "private detail" not in out diff --git a/dashboard/backend/tests/domain/analytics/test_value_repository.py b/dashboard/backend/tests/domain/analytics/test_value_repository.py index 64caa04b0..10cccd575 100644 --- a/dashboard/backend/tests/domain/analytics/test_value_repository.py +++ b/dashboard/backend/tests/domain/analytics/test_value_repository.py @@ -6,14 +6,8 @@ from datetime import date, datetime, timedelta, timezone from dashboard.backend.domain.analytics.repository import AnalyticsStore -from dashboard.backend.domain.analytics.states import ( - AnalyticsStateStore, - UserAnalyticsSnapshot, -) from dashboard.backend.domain.analytics.value_repository import ( ProjectionJob, - UserLifecycleDailySnapshot, - UserValueSnapshot, ValueAnalyticsStore, ) from dashboard.backend.users import UserStore @@ -218,26 +212,6 @@ def list_runs(self, agent_id): return list(self.runs.get(agent_id, ())) -def _value_snapshot(user_id: int) -> UserValueSnapshot: - return UserValueSnapshot( - user_id=user_id, - lifecycle_segment="core", - lifecycle_reason_code="core_repeated_value", - lifecycle_reason="The user repeatedly completed valuable work.", - lifecycle_evidence=("3 active days in the trailing 30 days",), - operational_state="healthy", - operational_reason_code="no_supported_issue", - operational_reason="No supported current operational issue was detected.", - operational_evidence=("All supported operational checks passed.",), - activated_at=NOW - timedelta(days=20), - last_meaningful_activity_at=NOW - timedelta(days=2), - inactive_days=2, - active_days_30d=3, - successful_backtests_30d=4, - calculated_at=NOW, - ) - - def _stores(tmp_path): analytics_path = tmp_path / "analytics.db" user = UserStore(db_path=analytics_path).create_user( @@ -260,103 +234,25 @@ def _value_store(analytics, credits): ) -def test_current_projection_round_trips_without_overwriting_legacy_state(tmp_path): - user_id, analytics, credits = _stores(tmp_path) - legacy = UserAnalyticsSnapshot( - user_id=user_id, - status="needs_attention", - reason_code="invalid_default_credential", - human_readable_reason="The default model credential is invalid.", - evidence_event_ids=["synthetic-event"], - calculated_at=NOW - timedelta(minutes=1), - ) - AnalyticsStateStore(analytics).upsert_snapshot(legacy) - store = _value_store(analytics, credits) - - expected = _value_snapshot(user_id) - store.upsert_current_snapshot(expected) - - assert store.get_current_snapshot(user_id) == expected - persisted_legacy = AnalyticsStateStore(analytics).get_snapshot(user_id) - assert persisted_legacy is not None - assert persisted_legacy.status == legacy.status - assert persisted_legacy.reason_code == legacy.reason_code - assert persisted_legacy.human_readable_reason == legacy.human_readable_reason - assert persisted_legacy.evidence_event_ids == legacy.evidence_event_ids - - -def test_current_projections_are_loaded_in_one_batched_query(tmp_path): - user_id, analytics, credits = _stores(tmp_path) - second = UserStore(db_path=analytics.db_path).create_user( - "second-value-user@example.test", - "Second Value User", - "SecurePass1!", - ) - second_id = int(second["id"]) - store = _value_store(analytics, credits) - first_snapshot = _value_snapshot(user_id) - second_snapshot = _value_snapshot(second_id).model_copy( - update={"lifecycle_segment": "growing"} - ) - store.upsert_current_snapshot(first_snapshot) - store.upsert_current_snapshot(second_snapshot) - - snapshots = store.list_current_snapshots([second_id, user_id, second_id]) - - assert snapshots == { - user_id: first_snapshot, - second_id: second_snapshot, - } - - -def test_daily_snapshot_and_projection_job_upserts_are_idempotent(tmp_path): - user_id, analytics, credits = _stores(tmp_path) +def test_projection_job_upsert_is_idempotent(tmp_path): + _user_id, analytics, credits = _stores(tmp_path) store = _value_store(analytics, credits) - first = UserLifecycleDailySnapshot( - snapshot_date=NOW.date(), - user_id=user_id, - lifecycle_segment="growing", - lifecycle_reason_code="growing_activated_below_core_threshold", - data_quality="partial", - calculated_at=NOW - timedelta(minutes=1), - ) - replacement = first.model_copy( - update={ - "lifecycle_segment": "core", - "lifecycle_reason_code": "core_repeated_value", - "data_quality": "complete", - "calculated_at": NOW, - } - ) - - store.upsert_daily_snapshot(first) - store.upsert_daily_snapshot(replacement) - rows = store.list_daily_snapshots( - start=NOW.date(), - end=NOW.date() + timedelta(days=1), - user_ids=[user_id], - ) - - assert rows == [replacement] - assert ( - store.list_daily_snapshots( - start=NOW.date(), - end=NOW.date() + timedelta(days=1), - user_ids=[], - ) - == [] - ) - job = ProjectionJob( job_name="lifecycle-backfill", window_start=date(2026, 7, 6), window_end=date(2026, 8, 30), cursor="user:40", status="running", - updated_at=NOW, + updated_at=NOW - timedelta(minutes=1), + ) + replacement = job.model_copy( + update={"cursor": None, "status": "complete", "updated_at": NOW} ) + store.save_projection_job(job) - assert store.get_projection_job(job.job_name) == job + store.save_projection_job(replacement) + + assert store.get_projection_job(job.job_name) == replacement def test_commercial_value_keeps_revenue_grants_usage_and_balance_separate(tmp_path): diff --git a/dashboard/backend/tests/fixtures/admin_analytics/operational.json b/dashboard/backend/tests/fixtures/admin_analytics/operational.json index 87c447942..61a6a8cd8 100644 --- a/dashboard/backend/tests/fixtures/admin_analytics/operational.json +++ b/dashboard/backend/tests/fixtures/admin_analytics/operational.json @@ -14,11 +14,11 @@ "top_failure_categories": [ { "error_category": "credential_invalid", - "affected_users": 4 + "event_count": 4 }, { "error_category": "provider_timeout", - "affected_users": 2 + "event_count": 2 } ], "availability": { diff --git a/dashboard/backend/tests/fixtures/admin_analytics/overview.json b/dashboard/backend/tests/fixtures/admin_analytics/overview.json index 3d0d30cb1..8d61142c1 100644 --- a/dashboard/backend/tests/fixtures/admin_analytics/overview.json +++ b/dashboard/backend/tests/fixtures/admin_analytics/overview.json @@ -22,21 +22,14 @@ "agent_created": 59, "backtest_completed": 50 }, - "user_state_counts": { - "blocked": 2, - "needs_attention": 3, - "dormant": 8, - "onboarding": 11, - "active": 37 - }, "top_failure_categories": [ { "error_category": "credential_invalid", - "affected_users": 4 + "event_count": 4 }, { "error_category": "provider_timeout", - "affected_users": 2 + "event_count": 2 } ], "users_needing_attention": [ diff --git a/dashboard/backend/tests/fixtures/admin_analytics/overview_partial_error.json b/dashboard/backend/tests/fixtures/admin_analytics/overview_partial_error.json index db06a0fa8..e12119297 100644 --- a/dashboard/backend/tests/fixtures/admin_analytics/overview_partial_error.json +++ b/dashboard/backend/tests/fixtures/admin_analytics/overview_partial_error.json @@ -16,17 +16,10 @@ "agent_created": 59, "backtest_completed": 50 }, - "user_state_counts": { - "blocked": 2, - "needs_attention": 3, - "dormant": 8, - "onboarding": 11, - "active": 37 - }, "top_failure_categories": [ { "error_category": "credential_invalid", - "affected_users": 4 + "event_count": 4 } ], "users_needing_attention": [], diff --git a/dashboard/backend/tests/fixtures/admin_analytics/target/operational.json b/dashboard/backend/tests/fixtures/admin_analytics/target/operational.json index e09e9233e..2708c7c3f 100644 --- a/dashboard/backend/tests/fixtures/admin_analytics/target/operational.json +++ b/dashboard/backend/tests/fixtures/admin_analytics/target/operational.json @@ -14,11 +14,11 @@ "top_failure_categories": [ { "error_category": "credential_invalid", - "affected_users": 4 + "event_count": 4 }, { "error_category": "provider_timeout", - "affected_users": 2 + "event_count": 2 } ], "top_operational_reasons": [ diff --git a/dashboard/backend/tests/test_admin_analytics_api.py b/dashboard/backend/tests/test_admin_analytics_api.py index 139c80b21..a778d0dca 100644 --- a/dashboard/backend/tests/test_admin_analytics_api.py +++ b/dashboard/backend/tests/test_admin_analytics_api.py @@ -16,6 +16,7 @@ from dashboard.backend.app import app from dashboard.backend.domain.analytics.metrics import AnalyticsMetricFilters from dashboard.backend.domain.analytics.query_service import ( + AnalyticsOverview, AnalyticsQueryService, get_analytics_query_service, get_value_analytics_query_service, @@ -32,10 +33,6 @@ FrontendAnalyticsEvent, RequestAnalyticsContext, ) -from dashboard.backend.domain.analytics.states import ( - AnalyticsStateStore, - recalculate_user_snapshot, -) from dashboard.backend.domain.analytics.value_queries import ( CommercialAnalyticsResponse, GroupAnalyticsResponse, @@ -45,8 +42,10 @@ RetentionAnalyticsResponse, SectionAvailability, UserGroupSummary, + UserValueFilters, ValueUserProfile, ) +from dashboard.backend.domain.analytics.value_repository import UserDailyFact from dashboard.backend.domain.user_groups import ( DEFAULT_USER_GROUP, USER_GROUP_LABELS, @@ -214,7 +213,6 @@ def _fixture(tmp_path): analytics, AnalyticsService(analytics), AnalyticsRollupStore(analytics), - AnalyticsStateStore(analytics), QueryUsers(users), ) @@ -232,7 +230,7 @@ def _event(service, name, at, source_id, **kwargs): def test_query_service_merges_completed_rollups_with_current_raw_day(tmp_path): - analytics, events, rollups, _states, users = _fixture(tmp_path) + analytics, events, rollups, users = _fixture(tmp_path) yesterday = NOW.date() - timedelta(days=1) rollups.replace_day( yesterday, @@ -282,7 +280,7 @@ def test_query_service_merges_completed_rollups_with_current_raw_day(tmp_path): def test_billing_lane_mix_counts_model_calls_from_rollups_and_today_honouring_filters(tmp_path): - analytics, events, rollups, _states, users = _fixture(tmp_path) + analytics, events, rollups, users = _fixture(tmp_path) yesterday = NOW.date() - timedelta(days=1) stamp = datetime.combine(NOW.date(), datetime.min.time(), tzinfo=timezone.utc) @@ -366,7 +364,7 @@ def test_billing_lane_mix_carries_each_lane_in_credits_honouring_filters(tmp_pat platform_model_cost_usd / byok_estimated_cost_usd rollups, today from raw events. The platform lane sums to the headline under every filter, and the BYOK estimate never enters it.""" - analytics, events, rollups, _states, users = _fixture(tmp_path) + analytics, events, rollups, users = _fixture(tmp_path) yesterday = NOW.date() - timedelta(days=1) stamp = datetime.combine(NOW.date(), datetime.min.time(), tzinfo=timezone.utc) @@ -469,7 +467,7 @@ def test_failed_raw_event_read_marks_growth_unavailable_not_a_rollup_only_range( raw events. When that read failed, growth stayed available and published the rollups alone as the whole range -- yesterday's cost as the range's, today's calls missing from every lane -- with status "ready".""" - analytics, events, rollups, _states, users = _fixture(tmp_path) + analytics, events, rollups, users = _fixture(tmp_path) yesterday = NOW.date() - timedelta(days=1) stamp = datetime.combine(NOW.date(), datetime.min.time(), tzinfo=timezone.utc) rollups.replace_day( @@ -511,7 +509,7 @@ def test_billing_lane_mix_counts_unpriced_byok_calls_honouring_filters(tmp_path) and are all unpriced; with one, #572 priced an unlisted model at the $1/$5 fallback, so the estimate stands only for a listed model. Deciding that per day instead made the unfiltered total and a per-model filter disagree.""" - analytics, events, rollups, _states, users = _fixture(tmp_path) + analytics, events, rollups, users = _fixture(tmp_path) stamp = datetime.combine(NOW.date(), datetime.min.time(), tzinfo=timezone.utc) counted, legacy, legacy_priced = (NOW.date() - timedelta(days=n) for n in (1, 2, 3)) listed = "openai/gpt-5.5" @@ -612,7 +610,7 @@ def test_activity_reports_atl_cost_only_for_byok_usage(tmp_path): """The activity API's cost_micro_usd is ATL cost. A BYOK call costs ATL nothing -- including one from the PR #572 era, whose event carried the list-price estimate in that very field.""" - analytics, events, _rollups, _states, users = _fixture(tmp_path) + analytics, events, _rollups, users = _fixture(tmp_path) for index, (billing_mode, properties) in enumerate( ( ("platform_credits", {"cost_micro_usd": 1_250}), @@ -645,7 +643,7 @@ def test_billing_lane_mix_skips_a_day_with_no_calls_and_no_cost(tmp_path): """Every rolled-up day carries a platform cost total, zero or not. A zero day is not activity: listing it would replace the panel's empty state with flat lines.""" - analytics, _events, rollups, _states, users = _fixture(tmp_path) + analytics, _events, rollups, users = _fixture(tmp_path) yesterday = NOW.date() - timedelta(days=1) stamp = datetime.combine(NOW.date(), datetime.min.time(), tzinfo=timezone.utc) rollups.replace_day( @@ -674,69 +672,59 @@ def test_billing_lane_mix_skips_a_day_with_no_calls_and_no_cost(tmp_path): assert overview.billing_lane_mix == [] -def test_user_list_and_profile_are_display_safe(tmp_path): - analytics, events, _rollups, states, users = _fixture(tmp_path) - _event( - events, - "account_signed_up", - NOW - timedelta(days=20), - "account:account_signed_up:1", - source_record_type="user", - source_record_id="1", - ) - success = _event( - events, - "backtest_completed", - NOW - timedelta(hours=1), - "run:backtest_completed:run-1", - correlation_id="run-1", - outcome="succeeded", +def test_profile_is_display_safe_and_reads_facts_not_the_event_window(tmp_path): + analytics, events, _rollups, users = _fixture(tmp_path) + from dashboard.backend.domain.analytics.value_repository import ValueAnalyticsStore + from dashboard.backend.tests.domain.analytics.test_fact_reads import fact + from dashboard.backend.tests.domain.analytics.test_value_repository import ( + SyntheticAgentStore, SyntheticCreditsStore, SyntheticProviderStore, SyntheticRunStore, ) - _event( - events, - "model_usage_recorded", - NOW - timedelta(minutes=50), - "resource:model_usage_recorded:run-1:0", - correlation_id="run-1", - provider_id="openrouter", - model_id="openai/gpt-5.5", - billing_mode="platform_credits", - outcome="succeeded", - properties={ - "input_tokens": 120, - "output_tokens": 30, - "cost_micro_usd": 250_000, - }, + + value_store = ValueAnalyticsStore( + analytics, SyntheticCreditsStore(tmp_path / "credits.db"), + # One enabled platform lane and a positive balance: operationally healthy. + provider_base=SyntheticProviderStore( + {}, [{"provider_id": "openrouter", "status": "enabled", "platform_enabled": True}] + ), + agent_base=SyntheticAgentStore(), run_base=SyntheticRunStore({}), ) - _event( - events, - "credits_settled", - NOW - timedelta(minutes=49), - "resource:credits_settled:reservation-1:grant", - source_record_type="credit_reservation", - source_record_id="reservation-1", - correlation_id="run-1", - billing_mode="platform_credits", - properties={"amount_micro": 100, "bucket": "grant"}, + yesterday = NOW.date() - timedelta(days=1) + value_store.record_activity(1, occurred_at=NOW - timedelta(hours=1), activating=True, now=NOW) + value_store.upsert_daily_facts( + [fact(1, yesterday, runs_requested=3, runs_completed=2, runs_failed=1, operator_cost_micro=250_000, own_spend_micro=100)] ) - recalculate_user_snapshot(1, now=NOW, store=states) - service = AnalyticsQueryService(store=analytics, user_store=users) + _event(events, "model_usage_recorded", NOW - timedelta(minutes=50), "resource:model_usage_recorded:run-1:0", + correlation_id="run-1", provider_id="openrouter", model_id="openai/gpt-5.5", + billing_mode="platform_credits", outcome="succeeded", + properties={"input_tokens": 120, "output_tokens": 30, "cost_micro_usd": 250_000}) + service = AnalyticsQueryService(store=analytics, user_store=users, value_store=value_store) + + def refuse_window_scan(**_kwargs): + raise AssertionError("the profile must not scan the event window") + service.query_store.rollups.list_events = refuse_window_scan + recorder = _WindowRecordingQueryStore(service.query_store) + service.query_store = recorder - profile = service.get_user_profile(user_id=1, now=NOW) + profile = service.get_user_profile(user_id=1, now=NOW, start=NOW - timedelta(days=30), end=NOW) serialized = profile.model_dump(mode="json") assert profile.state.status == "active" - assert success.event_id in profile.state.evidence_event_ids - assert profile.input_tokens == 120 + assert profile.state.evidence_event_ids == [] + assert profile.run_summary == {"requested": 3, "completed": 2, "failed": 1, "cancelled": 0} assert profile.platform_model_cost_usd == 0.25 assert profile.credits_debited_micro == 100 + assert profile.input_tokens == 120 + assert profile.default_provider == "openrouter" + assert profile.activation_milestones["backtest_completed"] == NOW - timedelta(hours=1) + assert profile.last_meaningful_activity == NOW - timedelta(hours=1) + assert recorder.windows == [] # no list_metric_events call: the only raw read is one timeline page assert "properties" not in str(serialized) assert "session_id" not in str(serialized) def test_activity_sections_page_independently_and_hide_session_ids(tmp_path): - analytics, events, _rollups, _states, users = _fixture(tmp_path) + analytics, events, _rollups, users = _fixture(tmp_path) for index in range(3): _event( events, @@ -786,6 +774,7 @@ def test_activity_sections_page_independently_and_hide_session_ids(tmp_path): section="sessions", limit=2, cursor=None, + now=NOW, ) assert len(first_runs.items) == 2 @@ -796,8 +785,47 @@ def test_activity_sections_page_independently_and_hide_session_ids(tmp_path): assert "session_id" not in str(sessions.model_dump(mode="json")) -def test_overview_marks_only_failed_rollup_panel_unavailable(tmp_path, monkeypatch): - analytics, events, _rollups, _states, users = _fixture(tmp_path) +def test_sessions_section_is_bounded_to_thirty_days(tmp_path): + analytics, events, _rollups, users = _fixture(tmp_path) + context = RequestAnalyticsContext( + country_code="US", device_category="desktop", browser_family="Chrome" + ) + for days_ago in (2, 45): + occurred_at = NOW - timedelta(days=days_ago) + events.accept_frontend_event( + user={"id": 1}, + payload=FrontendAnalyticsEvent( + event_id=str(uuid4()), + schema_version=1, + event_name="page_viewed", + session_id=str(uuid4()), + occurred_at=occurred_at, + page_view="agents", + properties={}, + ), + context=context, + # Ingest rejects occurred_at older than received_at - 24h. + received_at=occurred_at, + ) + service = AnalyticsQueryService(store=analytics, user_store=users) + + sessions = service.get_user_activity( + user_id=1, section="sessions", limit=10, cursor=None, now=NOW + ) + + assert len(sessions.items) == 1 + assert sessions.items[0].occurred_at == NOW - timedelta(days=2) + import inspect + + from dashboard.backend.domain.analytics.query_service import AnalyticsQueryStore + + assert "occurred_at >=" in inspect.getsource(AnalyticsQueryStore.list_session_rows) + + +def test_overview_marks_rollup_backed_panels_unavailable_when_rollups_fail( + tmp_path, monkeypatch, capsys +): + analytics, events, _rollups, users = _fixture(tmp_path) _event( events, "backtest_completed", @@ -806,25 +834,648 @@ def test_overview_marks_only_failed_rollup_panel_unavailable(tmp_path, monkeypat outcome="succeeded", ) service = AnalyticsQueryService(store=analytics, user_store=users) + # Yesterday's facts have landed, so attention has a day to read. + service.value_store.upsert_daily_facts([_fact(1, NOW.date() - timedelta(days=1))]) monkeypatch.setattr( service.query_store.rollups, "list_rollups", lambda **_kwargs: (_ for _ in ()).throw(RuntimeError("private detail")), ) + overview = service.get_overview( + now=NOW, + filters=AnalyticsMetricFilters(start=NOW - timedelta(days=1), end=NOW), + ) + + assert overview.active_users_7d is None + assert overview.completed_runs is None + assert overview.availability["snapshot"].available is False + assert overview.availability["growth"].available is False + assert overview.availability["funnel"].available is False + assert overview.availability["growth"].error_code == "temporarily_unavailable" + # M1: friction (top_failure_categories) is computed from the funnel's + # reads, so it goes down with them; attention reads facts and stays up. + assert overview.availability["friction"].available is False + assert overview.availability["attention"].available is True + # M2: the swallowed failure reaches the log, by class name only. + out = capsys.readouterr().out + assert "WARNING: analytics.read.overview_rollups_failed category=RuntimeError" in out + assert "private detail" not in out + + +class _WindowRecordingQueryStore: + """Wraps AnalyticsQueryStore and records every raw-event read window.""" + + def __init__(self, inner): + self.inner = inner + self.windows = [] + + def list_metric_events(self, *, start, end, include_internal): + self.windows.append((start, end)) + return self.inner.list_metric_events( + start=start, end=end, include_internal=include_internal + ) + + def __getattr__(self, name): + return getattr(self.inner, name) + + +def test_overview_reads_raw_events_for_the_current_utc_day_only(tmp_path): + """Design §6.15 item 2: one bounded one-day scan, never the filter window.""" + + analytics, _events, _rollups, users = _fixture(tmp_path) + service = AnalyticsQueryService(store=analytics, user_store=users) + recorder = _WindowRecordingQueryStore(service.query_store) + service.query_store = recorder + + service.get_overview( + now=NOW, + filters=AnalyticsMetricFilters(start=NOW - timedelta(days=30), end=NOW), + ) + + assert len(recorder.windows) == 1 + start, end = recorder.windows[0] + assert start == datetime.combine(NOW.date(), datetime.min.time(), tzinfo=timezone.utc) + assert end == NOW + + +def test_overview_skips_the_raw_scan_for_a_window_that_ends_before_today(tmp_path): + analytics, _events, _rollups, users = _fixture(tmp_path) + service = AnalyticsQueryService(store=analytics, user_store=users) + recorder = _WindowRecordingQueryStore(service.query_store) + service.query_store = recorder + overview = service.get_overview( now=NOW, filters=AnalyticsMetricFilters( - start=NOW - timedelta(days=1), + start=NOW - timedelta(days=10), + end=datetime.combine(NOW.date(), datetime.min.time(), tzinfo=timezone.utc), + ), + ) + + assert recorder.windows == [] + # Nothing to read is not a failed read. + assert overview.availability["growth"].available is True + assert overview.availability["funnel"].available is True + + +def test_overview_trailing_metrics_and_funnel_come_from_rollups(tmp_path): + analytics, events, rollups, users = _fixture(tmp_path) + yesterday = NOW.date() - timedelta(days=1) + stamp = datetime.combine(NOW.date(), datetime.min.time(), tzinfo=timezone.utc) + + def rollup(metric_name, value_count, **dimensions): + return DailyRollup( + rollup_date=yesterday, + metric_name=metric_name, + value_count=value_count, + updated_at=stamp, + **dimensions, + ) + + rollups.replace_day( + yesterday, + [ + rollup("rolling_active_users_7d", 9), + rollup("mature_signup_cohort_users", 4), + rollup("first_success_within_7d_users", 1), + rollup("users_with_first_success", 8), + rollup("users_with_repeat_success_24h_30d", 2), + rollup("activation_users", 5, event_name="account_signed_up"), + rollup("activation_users", 2, event_name="backtest_completed"), + rollup( + "event_count", + 3, + event_name="backtest_failed", + outcome="failed", + error_category="provider_timeout", + ), + rollup("input_tokens", 1000), + rollup("output_tokens", 250), + ], + ) + _event( + events, + "backtest_completed", + NOW - timedelta(minutes=2), + "run:backtest_completed:today", + outcome="succeeded", + ) + _event( + events, + "backtest_failed", + NOW - timedelta(minutes=1), + "run:backtest_failed:today-fail", + source_record_id="today-fail", + outcome="failed", + error_category="provider_timeout", + ) + service = AnalyticsQueryService(store=analytics, user_store=users) + + overview = service.get_overview( + now=NOW, + filters=AnalyticsMetricFilters( + start=datetime.combine(yesterday, datetime.min.time(), tzinfo=timezone.utc), end=NOW, ), ) - assert overview.active_users_7d == 1 - assert overview.completed_runs is None + assert overview.active_users_7d == 9 + assert overview.first_success_conversion == 0.25 + assert overview.repeat_run_rate == 0.25 + assert overview.activation_funnel == { + "account_signed_up": 5, + "credential_verified": 0, + "agent_created": 0, + "backtest_completed": 3, + } + assert [ + (c.error_category, c.event_count) for c in overview.top_failure_categories + ] == [("provider_timeout", 4)] + assert overview.input_tokens == 1000 + assert overview.output_tokens == 250 assert overview.availability["snapshot"].available is True - assert overview.availability["growth"].available is False - assert overview.availability["growth"].error_code == "temporarily_unavailable" + assert overview.availability["funnel"].available is True + + +def test_overview_failure_categories_count_events_filtered_or_not(tmp_path): + """Failure categories sum ``event_count`` rows plus today's events, with + or without a dimension filter (#647). The per-category ``affected_users`` + rows older rollups still hold are not read.""" + + analytics, events, rollups, users = _fixture(tmp_path) + yesterday = NOW.date() - timedelta(days=1) + stamp = datetime.combine(NOW.date(), datetime.min.time(), tzinfo=timezone.utc) + + def failed_rollup(model_id, count): + return DailyRollup( + rollup_date=yesterday, + metric_name="event_count", + event_name="backtest_failed", + provider_id="openrouter", + model_id=model_id, + outcome="failed", + error_category="provider_timeout", + value_count=count, + updated_at=stamp, + ) + + rollups.replace_day( + yesterday, + [ + failed_rollup("a", 3), + failed_rollup("b", 7), + DailyRollup( + rollup_date=yesterday, + metric_name="affected_users", + error_category="provider_timeout", + value_count=2, + updated_at=stamp, + ), + ], + ) + for index in range(2): + _event( + events, + "backtest_failed", + NOW - timedelta(minutes=index + 1), + f"run:backtest_failed:today-{index}", + source_record_id=f"today-{index}", + provider_id="openrouter", + model_id="a", + outcome="failed", + error_category="provider_timeout", + ) + service = AnalyticsQueryService(store=analytics, user_store=users) + start = datetime.combine(yesterday, datetime.min.time(), tzinfo=timezone.utc) + + def categories(**filters): + overview = service.get_overview( + now=NOW, + filters=AnalyticsMetricFilters(start=start, end=NOW, **filters), + ) + return [ + (c.error_category, c.event_count) + for c in overview.top_failure_categories + ] + + # Unfiltered: 3 + 7 yesterday, 2 today; the legacy row adds nothing. + assert categories() == [("provider_timeout", 12)] + # Filtered: 3 yesterday on model a, 2 today. + assert categories(model_id="a") == [("provider_timeout", 5)] + + +def _fact(user_id, day, **overrides): + values = { + "snapshot_date": day, + "user_id": user_id, + "lifecycle_segment": "growing", + "lifecycle_reason_code": "growing_activated_below_core_threshold", + "operational_state": "healthy", + "operational_reason_code": "no_supported_issue", + "tier": "unpaid", + "user_group": "unknown", + "active": True, + "data_quality": "complete", + "calculated_at": NOW, + } + values.update(overrides) + return UserDailyFact(**values) + + +def test_overview_attention_comes_from_yesterdays_facts(tmp_path): + """Design D20: the overview carries no five-state counts; attention lists + yesterday's blocked / needs-attention users with their 30-day run totals + and their current activity clock.""" + + analytics, _events, _rollups, users = _fixture(tmp_path) + service = AnalyticsQueryService(store=analytics, user_store=users) + yesterday = NOW.date() - timedelta(days=1) + with analytics._get_connection() as conn: + conn.executemany( + "INSERT INTO users VALUES (?, ?, ?, 'x', 'user', ?)", + [ + (user_id, f"u{user_id}@example.test", f"U{user_id}", NOW.isoformat()) + for user_id in (3, 4, 5) + ], + ) + last_seen = NOW - timedelta(days=3) + service.value_store.record_activity( + 1, occurred_at=last_seen, activating=False, now=last_seen + ) + service.value_store.upsert_daily_facts( + [ + _fact( + 1, + yesterday, + operational_state="needs_attention", + operational_reason_code="invalid_default_credential", + runs_requested=2, + runs_failed=1, + ), + # Ten days earlier, inside the 30-day window: adds to the totals. + _fact(1, yesterday - timedelta(days=10), runs_requested=3, runs_failed=2), + _fact( + 3, + yesterday, + operational_state="blocked", + operational_reason_code="account_restricted", + ), + _fact(4, yesterday, lifecycle_segment="dormant", + lifecycle_reason_code="dormant_previously_activated"), + _fact(5, yesterday, lifecycle_segment="onboarding", + lifecycle_reason_code="onboarding_signed_up_not_activated"), + # Today's row (if one existed) is not yesterday's population. + _fact(5, NOW.date(), operational_state="blocked", + operational_reason_code="account_restricted"), + ] + ) + + overview = service.get_overview( + now=NOW, + filters=AnalyticsMetricFilters(start=NOW - timedelta(days=1), end=NOW), + ) + + assert [ + (item.user_id, item.status, item.reason_code, item.recent_runs, item.recent_failures) + for item in overview.users_needing_attention + ] == [ + (1, "needs_attention", "invalid_default_credential", 5, 3), + (3, "blocked", "account_restricted", 0, 0), + ] + first = overview.users_needing_attention[0] + assert first.human_readable_reason == "The default model credential is invalid." + assert first.last_meaningful_activity == last_seen + assert first.display_name == "One" + assert overview.users_needing_attention[1].last_meaningful_activity is None + assert overview.availability["friction"].available is True + assert overview.availability["attention"].available is True + + +def test_overview_attention_falls_back_to_the_latest_landed_fact_day(tmp_path, capsys): + """Review #3: before the night's job lands (or all day, when it fails) + yesterday has no rows, and [] marked available read as "nobody needs + attention". The list joins the latest landed day, as /users does.""" + analytics, _events, _rollups, users = _fixture(tmp_path) + service = AnalyticsQueryService(store=analytics, user_store=users) + landed = NOW.date() - timedelta(days=2) + service.value_store.upsert_daily_facts( + [ + _fact( + 1, + landed, + operational_state="blocked", + operational_reason_code="account_restricted", + ) + ] + ) + + overview = service.get_overview( + now=NOW, filters=AnalyticsMetricFilters(start=NOW - timedelta(days=1), end=NOW) + ) + + assert [item.user_id for item in overview.users_needing_attention] == [1] + assert overview.availability["attention"].available is True + assert "WARNING: analytics.read.facts_stale" in capsys.readouterr().out + + +def test_overview_attention_is_unavailable_when_no_fact_day_has_landed(tmp_path): + analytics, _events, _rollups, users = _fixture(tmp_path) + service = AnalyticsQueryService(store=analytics, user_store=users) + + overview = service.get_overview( + now=NOW, filters=AnalyticsMetricFilters(start=NOW - timedelta(days=1), end=NOW) + ) + + assert overview.users_needing_attention == [] + assert overview.availability["attention"].available is False + + +def test_overview_attention_refuses_a_fact_day_older_than_the_nightly_gap(tmp_path): + """The attention panel has no as-of field: after days of failed jobs, an + old blocked list must read unavailable, not current.""" + analytics, _events, _rollups, users = _fixture(tmp_path) + service = AnalyticsQueryService(store=analytics, user_store=users) + service.value_store.upsert_daily_facts( + [ + _fact( + 1, + NOW.date() - timedelta(days=3), + operational_state="blocked", + operational_reason_code="account_restricted", + ) + ] + ) + + overview = service.get_overview( + now=NOW, filters=AnalyticsMetricFilters(start=NOW - timedelta(days=1), end=NOW) + ) + + assert overview.users_needing_attention == [] + assert overview.availability["attention"].available is False + + +def _midnight(day): + return datetime.combine(day, datetime.min.time(), tzinfo=timezone.utc) + + +def test_overview_funnel_is_unavailable_while_a_rolled_up_day_lacks_activation_rows(tmp_path): + """Review #5: a day rolled up before activation_users existed summed as + zero under an available funnel. Until the backfill fills it, the funnel + says unavailable; friction, from the same reads, stays up.""" + analytics, _events, rollups, users = _fixture(tmp_path) + yesterday = NOW.date() - timedelta(days=1) + service = AnalyticsQueryService(store=analytics, user_store=users) + filters = AnalyticsMetricFilters(start=_midnight(yesterday), end=NOW) + pre_metric_day = [ + DailyRollup( + rollup_date=yesterday, + metric_name="daily_active_users", + value_count=3, + updated_at=_midnight(NOW.date()), + ) + ] + rollups.replace_day(yesterday, pre_metric_day) + + stale = service.get_overview(now=NOW, filters=filters) + + assert stale.availability["funnel"].available is False + assert stale.activation_funnel == {} + assert stale.availability["friction"].available is True + + rollups.replace_day( + yesterday, + pre_metric_day + + [ + DailyRollup( + rollup_date=yesterday, + metric_name="activation_users", + event_name=event_name, + value_count=2 if event_name == "account_signed_up" else 0, + updated_at=_midnight(NOW.date()), + ) + for event_name in ("account_signed_up", "credential_verified", "agent_created", "backtest_completed") + ], + ) + + filled = service.get_overview(now=NOW, filters=filters) + + assert filled.availability["funnel"].available is True + assert filled.activation_funnel["account_signed_up"] == 2 + + +def test_overview_include_internal_does_not_add_internal_accounts_to_today(tmp_path): + """Review #6: completed days come from rollups written without internal + accounts, so today's raw slice must leave them out too, whatever the + flag; mixing them produced figures that matched neither setting.""" + analytics, events, _rollups, users = _fixture(tmp_path) + for index, user_id in enumerate((1, 2)): # user 2 is the admin account + events.record_server_event( + event_name="backtest_completed", + user_id=user_id, + source_event_id=f"run:backtest_completed:today-{index}", + source_record_type="run", + source_record_id=f"today-{index}", + outcome="succeeded", + occurred_at=NOW - timedelta(minutes=5 + index), + ) + service = AnalyticsQueryService(store=analytics, user_store=users) + start = _midnight(NOW.date()) + + internal_free = service.get_overview( + now=NOW, filters=AnalyticsMetricFilters(start=start, end=NOW) + ) + widened = service.get_overview( + now=NOW, filters=AnalyticsMetricFilters(start=start, end=NOW, include_internal=True) + ) + + assert internal_free.completed_runs == 1 + assert widened.completed_runs == internal_free.completed_runs + assert widened.daily_completed_runs == internal_free.daily_completed_runs + assert widened.filters.include_internal is True # accepted and echoed + + +def test_platform_cost_read_alone_also_leaves_internal_accounts_out_of_today(tmp_path): + """The figure /commercial publishes is the overview's, read alone; it + must not widen today's slice under include_internal either.""" + analytics, events, _rollups, users = _fixture(tmp_path) + for index, user_id in enumerate((1, 2)): # user 2 is the admin account + events.record_server_event( + event_name="model_usage_recorded", + user_id=user_id, + source_event_id=f"resource:model_usage_recorded:today-{index}:0", + source_record_type="resource", + source_record_id=f"today-{index}", + correlation_id=f"today-{index}", + provider_id="openrouter", + model_id="openai/gpt-5.5", + billing_mode="platform_credits", + outcome="succeeded", + properties={"input_tokens": 10, "output_tokens": 5, "cost_micro_usd": 300_000}, + occurred_at=NOW - timedelta(minutes=5 + index), + ) + service = AnalyticsQueryService(store=analytics, user_store=users) + start = _midnight(NOW.date()) + + def cost(include_internal): + return service.get_platform_model_cost_micro( + filters=AnalyticsMetricFilters(start=start, end=NOW, include_internal=include_internal), + now=NOW, + ) + + assert cost(False) == 300_000 + assert cost(True) == cost(False) + + +def test_profile_milestones_are_first_occurrences_not_the_newest_page(tmp_path): + """Review #8: min() over the newest-100 timeline page dropped a first + verification older than the user's last hundred events.""" + analytics, events, _rollups, users = _fixture(tmp_path) + first_verified = NOW - timedelta(days=40) + events.record_server_event( + event_name="credential_verified", + user_id=1, + source_event_id="credential:c-1:verified:first", + source_record_type="credential", + source_record_id="c-1", + occurred_at=first_verified, + ) + events.record_server_event( + event_name="credential_verified", + user_id=1, + source_event_id="credential:c-1:verified:again", + source_record_type="credential", + source_record_id="c-1", + occurred_at=NOW - timedelta(days=2), + ) + for index in range(101): + _event( + events, + "backtest_requested", + NOW - timedelta(hours=1, minutes=index), + f"run:backtest_requested:page-{index}", + source_record_id=f"page-{index}", + ) + from dashboard.backend.domain.analytics.value_repository import ValueAnalyticsStore + from dashboard.backend.tests.domain.analytics.test_value_repository import ( + SyntheticAgentStore, SyntheticCreditsStore, SyntheticProviderStore, SyntheticRunStore, + ) + + value_store = ValueAnalyticsStore( + analytics, SyntheticCreditsStore(tmp_path / "credits.db"), + provider_base=SyntheticProviderStore( + {}, [{"provider_id": "openrouter", "status": "enabled", "platform_enabled": True}] + ), + agent_base=SyntheticAgentStore(), run_base=SyntheticRunStore({}), + ) + service = AnalyticsQueryService(store=analytics, user_store=users, value_store=value_store) + + profile = service.get_user_profile( + user_id=1, now=NOW, start=NOW - timedelta(days=90), end=NOW + ) + + assert profile.activation_milestones["credential_verified"] == first_verified + assert "agent_created" not in profile.activation_milestones + # The selected range bounds it, as on main. + narrow = service.get_user_profile( + user_id=1, now=NOW, start=NOW - timedelta(days=10), end=NOW + ) + assert narrow.activation_milestones["credential_verified"] == NOW - timedelta(days=2) + + +def test_a_session_crossing_the_30_day_cutoff_is_read_whole(tmp_path): + """Review #13: the window selects sessions, not events. A session that + began before the cutoff and continued after it keeps all its events; a + session wholly before the cutoff stays out (the design's 30-day window).""" + analytics, events, _rollups, users = _fixture(tmp_path) + cutoff = NOW - timedelta(days=30) + context = RequestAnalyticsContext( + country_code="US", device_category="desktop", browser_family="Chrome" + ) + straddling, older = str(uuid4()), str(uuid4()) + for session_id, name, at in ( + (straddling, "page_viewed", cutoff - timedelta(minutes=5)), + (straddling, "session_heartbeat", cutoff + timedelta(minutes=5)), + (older, "page_viewed", cutoff - timedelta(days=1)), + ): + events.accept_frontend_event( + user={"id": 1}, + payload=FrontendAnalyticsEvent( + event_id=str(uuid4()), + schema_version=1, + event_name=name, + session_id=session_id, + occurred_at=at, + page_view="agents", + properties=({} if name == "page_viewed" else {"visible_ms": 500}), + ), + context=context, + received_at=at + timedelta(minutes=1), + ) + service = AnalyticsQueryService(store=analytics, user_store=users) + + page = service.get_user_activity( + user_id=1, section="sessions", limit=10, cursor=None, now=NOW + ) + + assert [item.session_event_count for item in page.items] == [2] + assert page.items[0].visible_ms == 500 + + +def test_overview_marks_fact_panels_unavailable_when_fact_reads_fail(tmp_path, capsys): + analytics, _events, _rollups, users = _fixture(tmp_path) + + class BrokenFacts: + def count_states_for_date(self, _day): + raise AssertionError("friction availability no longer probes the facts") + + def latest_fact_date(self, *, on_or_before): + return on_or_before + + def list_attention_candidates(self, **_kwargs): + raise RuntimeError("private detail") + + service = AnalyticsQueryService( + store=analytics, user_store=users, value_store=BrokenFacts() + ) + + overview = service.get_overview( + now=NOW, + filters=AnalyticsMetricFilters(start=NOW - timedelta(days=1), end=NOW), + ) + + assert overview.users_needing_attention == [] + # M1: friction is served from the funnel's reads, so it follows them. + assert overview.availability["friction"].available is True + assert overview.availability["attention"].available is False + assert overview.availability["growth"].available is True + # M2: the swallowed failure reaches the log, by class name only. + out = capsys.readouterr().out + assert "WARNING: analytics.read.overview_attention_failed category=RuntimeError" in out + assert "private detail" not in out + + +def test_query_service_builds_its_value_store_only_on_first_use(tmp_path, monkeypatch): + """The module-level service is built at import; an eager default would pull + the credits / provider / agent / run singletons onto the boot path.""" + + from dashboard.backend.domain.analytics import query_service as module + + analytics, _events, _rollups, users = _fixture(tmp_path) + built = [] + real = module.build_value_analytics_store + + def recording(**kwargs): + built.append(kwargs) + return real(**kwargs) + + monkeypatch.setattr(module, "build_value_analytics_store", recording) + service = AnalyticsQueryService(store=analytics, user_store=users) + assert built == [] + + first = service.value_store + assert service.value_store is first + assert built == [{"analytics_base": analytics}] @pytest.fixture @@ -864,8 +1515,6 @@ def admin_analytics_api(monkeypatch): outcome="succeeded", occurred_at=at - timedelta(minutes=index + 1), ) - state_store = AnalyticsStateStore(analytics) - recalculate_user_snapshot(subject["id"], now=at, store=state_store) query_service = AnalyticsQueryService(store=analytics, user_store=users) value_query_service = FixtureValueQueryService(int(subject["id"])) monkeypatch.setattr(users_module, "user_store", users) @@ -1097,7 +1746,6 @@ def test_admin_user_list_accepts_documented_filters(admin_analytics_api): "/api/admin/analytics/users", params={ "q": "Subject", - "status": "active", "lifecycle_segment": "core", "operational_state": "healthy", "commercial_tier": "invested", @@ -1125,7 +1773,6 @@ def test_admin_user_list_accepts_documented_filters(admin_analytics_api): assert filters.commercial_tier == "invested" assert filters.user_group == "partner" assert filters.activated is True - assert filters.legacy_status == "active" def test_admin_analytics_maps_not_found_and_cursor_errors_safely( @@ -1232,3 +1879,21 @@ def test_value_routes_reject_unknown_duplicate_and_unsafe_queries( assert response.status_code == 422 assert response.json() == {"detail": "Invalid Analytics query."} assert "synthetic secret" not in response.text + + +def test_overview_has_no_five_state_field_and_users_rejects_status(admin_analytics_api): + """Design D20: the one permitted contract break before PR D.""" + api = admin_analytics_api + + overview = api["client"].get("/api/admin/analytics/overview", headers=api["admin_headers"]) + users = api["client"].get( + "/api/admin/analytics/users", params={"status": "active"}, headers=api["admin_headers"] + ) + + assert overview.status_code == 200, overview.text + assert "user_state_counts" not in overview.json() + assert set(overview.json()["availability"]) == {"snapshot", "growth", "funnel", "friction", "attention"} + assert users.status_code == 422 + assert users.json() == {"detail": "Invalid Analytics query."} + assert "user_state_counts" not in AnalyticsOverview.model_fields + assert "legacy_status" not in UserValueFilters.model_fields diff --git a/dashboard/backend/tests/test_admin_analytics_frontend.py b/dashboard/backend/tests/test_admin_analytics_frontend.py index c906e5066..4a65b55cf 100644 --- a/dashboard/backend/tests/test_admin_analytics_frontend.py +++ b/dashboard/backend/tests/test_admin_analytics_frontend.py @@ -169,7 +169,7 @@ def test_app_lifecycle_and_cache_versions_are_wired(): 'src="js/admin-shell.js?v=8"', 'src="js/credit-format.js?v=1"', 'src="js/admin-live.js?v=1"', - 'src="js/admin-overview.js?v=9"', + 'src="js/admin-overview.js?v=10"', 'src="js/admin-users.js?v=1"', 'src="js/admin-providers.js?v=1"', ): diff --git a/dashboard/backend/tests/test_admin_overview_frontend.py b/dashboard/backend/tests/test_admin_overview_frontend.py index c36cd8241..13636bcb7 100644 --- a/dashboard/backend/tests/test_admin_overview_frontend.py +++ b/dashboard/backend/tests/test_admin_overview_frontend.py @@ -504,7 +504,7 @@ def test_health_detail_has_no_affected_users_column(): ) assert result["h1"] == "System health" assert result["metrics"] == ["10", "80%", "40", "2"] - assert result["headers"] == [["Failure category", "Count"], ["Reason", "State", "Users"]] + assert result["headers"] == [["Failure category", "Events"], ["Reason", "State", "Users"]] assert result["rows"] == [ [["Credential Invalid", "4"], ["Provider Timeout", "2"]], [["No Usable Billing Lane", "Blocked", "2"], ["Credential Invalid", "Needs attention", "4"]], diff --git a/dashboard/backend/tests/test_admin_page_shell.py b/dashboard/backend/tests/test_admin_page_shell.py index 5d0bffa4c..e1148bd39 100644 --- a/dashboard/backend/tests/test_admin_page_shell.py +++ b/dashboard/backend/tests/test_admin_page_shell.py @@ -22,7 +22,7 @@ # The app chrome's ticker (D5 layout pass) rides after the formatters. "js/admin-ticker.js?v=2", "js/admin-live.js?v=1", - "js/admin-overview.js?v=9", + "js/admin-overview.js?v=10", "js/admin-users.js?v=1", "js/admin-providers.js?v=1", # The absorbed credits console (design N2/PR2). diff --git a/dashboard/backend/tests/test_analytics_integration.py b/dashboard/backend/tests/test_analytics_integration.py index 008fc3ccb..d7e370639 100644 --- a/dashboard/backend/tests/test_analytics_integration.py +++ b/dashboard/backend/tests/test_analytics_integration.py @@ -24,12 +24,10 @@ AnalyticsService, get_analytics_service, ) -from dashboard.backend.domain.analytics.states import ( - AnalyticsStateStore, - recalculate_user_snapshot, - recalculate_user_snapshots, +from dashboard.backend.domain.analytics.value_repository import ( + UserDailyFact, + ValueAnalyticsStore, ) -from dashboard.backend.domain.analytics.value_repository import ValueAnalyticsStore from dashboard.backend.domain.analytics.value_queries import ValueAnalyticsQueryService from dashboard.backend.domain.credits.repository import CreditsStore from dashboard.backend.domain.runs import service as run_service @@ -210,12 +208,15 @@ def test_synthetic_acceptance_scenario_has_no_real_credentials(monkeypatch): ) store = AnalyticsStore(path) event_service = AnalyticsService(store) - query_service = AnalyticsQueryService(store=store, user_store=users) - state_store = AnalyticsStateStore(store) value_store = ValueAnalyticsStore( analytics_base=store, credits_base=CreditsStore(path), ) + # Both services read the same value store: the profile's live axes and + # fact sums go through the legacy service's. + query_service = AnalyticsQueryService( + store=store, user_store=users, value_store=value_store + ) value_query_service = ValueAnalyticsQueryService( store=store, user_store=users, @@ -230,16 +231,6 @@ def test_synthetic_acceptance_scenario_has_no_real_credentials(monkeypatch): "get_analytics_service", lambda: event_service, ) - monkeypatch.setattr( - instrumentation, - "_snapshot_recalculator", - lambda user_id: recalculate_user_snapshots( - user_id, - now=now, - state_store=state_store, - value_store=value_store, - ), - ) app.dependency_overrides[get_analytics_query_service] = lambda: query_service app.dependency_overrides[get_value_analytics_query_service] = ( lambda: value_query_service @@ -339,6 +330,33 @@ def test_synthetic_acceptance_scenario_has_no_real_credentials(monkeypatch): occurred_at=now, ) + # The profile's run and spend figures are "as of yesterday" (design + # §6.3): today's settle above reaches them once the daily job writes + # today's facts, so the scenario carries yesterday's row the job wrote. + value_store.upsert_daily_facts( + [ + UserDailyFact( + snapshot_date=now.date() - timedelta(days=1), + user_id=subject["id"], + lifecycle_segment="growing", + lifecycle_reason_code="growing_activated_below_core_threshold", + operational_state="healthy", + operational_reason_code="no_supported_issue", + tier="unpaid", + user_group="unknown", + active=True, + runs_requested=2, + runs_completed=1, + runs_failed=1, + runs_cancelled=0, + operator_cost_micro=420_000, + own_spend_micro=420, + data_quality="complete", + calculated_at=now, + ) + ] + ) + token = users.create_session(admin["id"]) headers = {"Authorization": f"Bearer {token}"} try: @@ -375,6 +393,17 @@ def test_synthetic_acceptance_scenario_has_no_real_credentials(monkeypatch): "platform_credits": 1, } assert profile["credits_debited_micro"] == 420 - assert profile["state"]["status"] == "active" - assert profile["state"]["evidence_event_ids"] + assert profile["run_summary"] == { + "requested": 2, + "completed": 1, + "failed": 1, + "cancelled": 0, + } + # The state summary follows the live axes: the synthetic subject has + # no Credits balance and no stored credential (the credential events + # above are instrumentation only), so no billing lane is usable. + assert profile["operational"]["reason_code"] == "billing_lane_unavailable" + assert profile["state"]["status"] == "blocked" + assert profile["state"]["reason_code"] == "billing_lane_unavailable" + assert profile["state"]["evidence_event_ids"] == [] assert "api_key" not in str(profile) diff --git a/dashboard/backend/tests/test_analytics_maintenance.py b/dashboard/backend/tests/test_analytics_maintenance.py deleted file mode 100644 index 2391d1198..000000000 --- a/dashboard/backend/tests/test_analytics_maintenance.py +++ /dev/null @@ -1,69 +0,0 @@ -"""Analytics maintenance is bounded, idempotent, and safe to register.""" - -from __future__ import annotations - -import inspect -from datetime import date, datetime, timezone -from pathlib import Path -from types import SimpleNamespace - -import dashboard.backend.app as app_module -import dashboard.backend.domain.analytics.maintenance as maintenance - - -NOW = datetime(2026, 8, 26, 12, 0, tzinfo=timezone.utc) - - -def test_maintenance_repairs_bounded_batches_and_nothing_else(): - repair_limits = [] - value_repair_limits = [] - repair_order = [] - - def repair(**kwargs): - repair_order.append("legacy") - repair_limits.append(kwargs["limit"]) - return 25 - - def repair_values(**kwargs): - repair_order.append("value") - value_repair_limits.append(kwargs["limit"]) - return 12 - - report = maintenance.run_analytics_maintenance( - now=NOW, - snapshot_limit=250, - repair_snapshots=repair, - repair_value_snapshots=repair_values, - ) - - assert report.repaired_snapshots == 25 - assert report.repaired_value_snapshots == 12 - assert report.failures == 0 - assert repair_limits == [100] - assert value_repair_limits == [100] - assert repair_order == ["value", "legacy"] - - -def test_maintenance_isolates_repair_failures(): - def fail_repair(**_kwargs): - raise RuntimeError("private user detail") - - def fail_value_repair(**_kwargs): - raise RuntimeError("private value projection detail") - - report = maintenance.run_analytics_maintenance( - now=NOW, - repair_snapshots=fail_repair, - repair_value_snapshots=fail_value_repair, - ) - - assert report.repaired_snapshots == 0 - assert report.repaired_value_snapshots == 0 - assert report.failures == 2 - - -def test_maintenance_no_longer_owns_rollups_or_the_lifecycle_backfill(): - """Design SS6.9 step 1 / D23: the daily job owns rollup_day now, so the - reaper tick must not be able to write the same rollup rows.""" - parameters = inspect.signature(maintenance.run_analytics_maintenance).parameters - source = inspect.getsource(maintenance) diff --git a/dashboard/backend/tests/test_architecture_boundaries.py b/dashboard/backend/tests/test_architecture_boundaries.py index d73e23b7d..cc27529dd 100644 --- a/dashboard/backend/tests/test_architecture_boundaries.py +++ b/dashboard/backend/tests/test_architecture_boundaries.py @@ -366,6 +366,39 @@ def test_deleted_shims_absent_from_production_and_scripts(): assert offenders == [], f"deleted shim paths still imported: {offenders}" +_DELETED_ANALYTICS_MODULES = ( + "dashboard.backend.domain.analytics.states", + "dashboard.backend.domain.analytics.lifecycle_backfill", + "dashboard.backend.domain.analytics.maintenance", +) + + +@pytest.mark.parametrize("module_name", _DELETED_ANALYTICS_MODULES) +def test_deleted_five_state_module_is_not_importable(module_name): + """Design §13 row B: the five-state model left in PR B. These are plain + modules, not package directories, so a leftover __pycache__ cannot make + the import succeed -- unlike the deleted shim *packages* in _DELETED_SHIMS, + whose stale .pyc dirs resolve as PEP-420 namespace packages. Python never + imports a sourceless .pyc from __pycache__.""" + with pytest.raises(ModuleNotFoundError): + importlib.import_module(module_name) + + +def test_app_no_longer_registers_analytics_maintenance_with_the_reaper(): + source = (Path(__file__).resolve().parents[1] / "app.py").read_text(encoding="utf-8") + assert "run_analytics_maintenance" not in source + assert "analytics.maintenance_registration_failed" not in source + + +def test_rollup_day_writes_no_five_state_rows(): + import inspect + + from dashboard.backend.domain.analytics import rollups + + assert "user_state_count" not in inspect.getsource(rollups.rollup_day) + assert "state_counts" not in inspect.signature(rollups.rollup_day).parameters + + # --------------------------------------------------------------------------- # Chat / Discord import safety: no secrets, no network at import time # --------------------------------------------------------------------------- @@ -557,34 +590,30 @@ def test_portfolio_manager_construction_always_declares_settlement(): # --------------------------------------------------------------------------- _ANALYTICS = _BACKEND / "domain" / "analytics" -_EVENT_READ_NAMES = {"list_events", "list_user_events", "list_metric_events"} +_EVENT_READ_NAMES = { + "list_events", + "list_user_events", + "list_metric_events", + "first_event_times", +} # Files that may call a raw-event reader, each with the reason. Rules 3-5: the # paginated timeline and the overview's one-day scan are the only permitted raw # reads; everything else reads rollups or user_daily_facts. Every entry must -# still contain a hit (the stale check below), so PR B removes the last two as -# it deletes the files and narrows the others as it moves the reads. +# still contain a hit (the stale check below); PR B removed the five-state +# calculator's and the lifecycle backfill's entries with their files. _EVENT_READ_ALLOWLIST = { "dashboard/backend/domain/analytics/rollups.py": ( - "defines AnalyticsRollupStore.list_events; rollup_day / rollup_current_day " - "are the day rollups and the overview's current-day scan (rule 5), " - "narrowed to one UTC day in PR B" + "defines AnalyticsRollupStore.list_events; rollup_day is the day " + "rollup (rule 5), its read bounded to the day plus its 30-day trailing " + "context. (The startup migration's activation_users backfill is one " + "grouped SQL read here too, outside this name-based check.)" ), "dashboard/backend/domain/analytics/query_service.py": ( "the paginated timeline (rule 3) and the overview's current-day read " - "(rule 5); PR B narrows the overview read and gives sessions a 30-day window" - ), - "dashboard/backend/domain/analytics/value_queries.py": ( - "the retention grid's per-activation-week scan and the one-user profile " - "scan; PR B moves both onto user_daily_facts / user_activity" - ), - "dashboard/backend/domain/analytics/states.py": ( - "legacy five-state calculator: full-history per-user reads. DELETED IN " - "PR B (design SS13 row B); allowlisted, not fixed, because PR A creates " - "and PR B drops" - ), - "dashboard/backend/domain/analytics/lifecycle_backfill.py": ( - "historical eight-week reconstruction, per-user history reads. DELETED " - "IN PR B; the copy in facts_migration.py carries the history now" + "(rule 5), bounded to one UTC day; the sessions read has a 30-day " + "window; first_event_times is the profile's grouped MIN() for its two " + "activation milestones, at most two rows (design SS6.12, amended " + "2026-10-16)" ), } _OWN_CONNECTION_RECEIVERS = {"self", "self.analytics_base", "self.base_store"} diff --git a/dashboard/backend/tests/test_backtest_db_postgres.py b/dashboard/backend/tests/test_backtest_db_postgres.py index bf9cfe65a..1213d5c47 100644 --- a/dashboard/backend/tests/test_backtest_db_postgres.py +++ b/dashboard/backend/tests/test_backtest_db_postgres.py @@ -882,3 +882,33 @@ def test_update_run_baselines_coalesce_preserves_unset_field_postgres(pg_backtes run = pg_backtest_db.get_run("run-base") assert run["baseline_djia_run_id"] == "djia-1" # untouched by the second call assert run["baseline_buyhold_run_id"] == "bh-1" + + +@pg_only +def test_operator_cost_excludes_byok_runs_postgres(pg_backtest_db): + """I3 on the prod dialect: the twin of test_run_history_aggregates.""" + from datetime import date + + from dashboard.backend.tests.test_run_history_aggregates import BILLING_CASES + + for run_id, owner, cost, metadata in BILLING_CASES: + pg_backtest_db.insert_run( + run_id=run_id, session_id="s1", agent_name="Agent", mode="backtest", + start_date="2026-09-01", end_date="2026-09-02", initial_equity=1000.0, + est_cost_usd=cost, owner_user_id=owner, metadata=metadata, + ) + pg_backtest_db.insert_run( + run_id="bad", session_id="s1", agent_name="Agent", mode="backtest", + start_date="2026-09-01", end_date="2026-09-02", initial_equity=1000.0, + est_cost_usd=0.5, owner_user_id=8, + ) + with pg_backtest_db._get_connection() as conn: + with conn.cursor() as cur: + cur.execute("UPDATE agent_runs SET metadata = '{not json' WHERE run_id = 'bad'") + cur.execute("UPDATE agent_runs SET updated_at = '2026-09-11 15:30:00'") + + assert pg_backtest_db.aggregate_operator_cost_for_day(date(2026, 9, 11)) == { + 4: 1_000_000, + 5: 500_000, + 8: 500_000, + } diff --git a/dashboard/backend/tests/test_run_history_aggregates.py b/dashboard/backend/tests/test_run_history_aggregates.py index afbcc9ee0..8793555d0 100644 --- a/dashboard/backend/tests/test_run_history_aggregates.py +++ b/dashboard/backend/tests/test_run_history_aggregates.py @@ -7,7 +7,7 @@ from dashboard.backend.database import BacktestDatabase -def _seed(db, run_id, *, owner, cost, day): +def _seed(db, run_id, *, owner, cost, day, metadata=None): db.insert_run( run_id=run_id, session_id="session-1", @@ -18,6 +18,7 @@ def _seed(db, run_id, *, owner, cost, day): initial_equity=100000.0, est_cost_usd=cost, owner_user_id=owner, + metadata=metadata, ) conn = db._get_connection() try: @@ -51,3 +52,44 @@ def test_a_negative_stored_cost_cannot_violate_the_fact_check(tmp_path): _seed(db, "neg", owner=3, cost=-0.75, day="2026-09-11") assert db.aggregate_operator_cost_for_day(date(2026, 9, 11)) == {3: 0} + + +def _billing(mode): + return {"llm_execution": {"billing_mode": mode, "model_id": "m"}} + + +# One shape per case, shared with the Postgres twin's test. +BILLING_CASES = ( + # (run_id, owner, cost, metadata) + ("byok", 4, 2.0, _billing("byok")), + ("platform", 4, 0.75, _billing("platform_credits")), + ("no-execution", 4, 0.25, {"llm_max_output_tokens": 2000}), + ("no-metadata", 5, 0.5, None), + ("byok-only", 6, 1.0, _billing("byok")), +) + + +def test_operator_cost_excludes_byok_runs(tmp_path): + """I3: BYOK spend is the user's own key, not ATL platform model cost. Runs + that record no billing lane keep counting (rule-based runs cost 0 anyway).""" + db = BacktestDatabase(tmp_path / "runs.db") + for run_id, owner, cost, metadata in BILLING_CASES: + _seed(db, run_id, owner=owner, cost=cost, day="2026-09-11", metadata=metadata) + + assert db.aggregate_operator_cost_for_day(date(2026, 9, 11)) == { + 4: 1_000_000, + 5: 500_000, + } + + +def test_operator_cost_tolerates_malformed_metadata(tmp_path): + db = BacktestDatabase(tmp_path / "runs.db") + _seed(db, "bad", owner=8, cost=0.5, day="2026-09-11") + conn = db._get_connection() + try: + conn.execute("UPDATE agent_runs SET metadata = '{not json' WHERE run_id = 'bad'") + conn.commit() + finally: + conn.close() + + assert db.aggregate_operator_cost_for_day(date(2026, 9, 11)) == {8: 500_000} diff --git a/dashboard/backend/tests/test_store_twin_parity.py b/dashboard/backend/tests/test_store_twin_parity.py index a2ede8935..f01e3944e 100644 --- a/dashboard/backend/tests/test_store_twin_parity.py +++ b/dashboard/backend/tests/test_store_twin_parity.py @@ -275,20 +275,6 @@ def test_every_postgres_twin_module_is_registered(): "`%s` query in the file would reach sqlite3. None is a missing " "extraction: this file *is* the extraction." ), - "dashboard/backend/domain/analytics/states.py": ( - "AnalyticsStateStore dialect-branches over the already-twinned " - "AnalyticsStore/PostgresAnalyticsStore base_store for the legacy " - "user_analytics_snapshots table, not a store of its own -- the " - "branched methods also read users (get_user; that table belongs to " - "another registered twin, UserStore/PostgresUserStore, so this is a " - "cross-domain read, not a missing extraction) and, via " - "list_stale_user_ids, analytics_subject_settings and " - "user_lifecycle_daily_snapshots. Admin layer redesign PR A/PR B " - "(docs/superpowers/specs/2026-09-15-admin-layer-redesign-design.md " - "§6.5) deletes the five-state columns and repair path this class " - "serves, which removes or shrinks this branch -- tracked there, not " - "in PR T." - ), "dashboard/backend/domain/analytics/rollups.py": ( "AnalyticsRollupStore dialect-branches over the already-twinned " "AnalyticsStore/PostgresAnalyticsStore base_store for " @@ -306,12 +292,6 @@ def test_every_postgres_twin_module_is_registered(): "of its own; that surface is rewritten in admin layer redesign PR A/" "PR B (design doc §6.10), which is where this branch is next touched." ), - "dashboard/backend/domain/analytics/lifecycle_backfill.py": ( - "LifecycleBackfillSource dialect-branches over the already-twinned " - "analytics_base for the historical 8-week lifecycle reconstruction " - "job. Not a store of its own; admin layer redesign PR A's daily job " - "and migration (design doc §6.9) replace this reconstruction path." - ), } @@ -513,8 +493,8 @@ def _name(raw: str) -> str: _NO_OWN_DDL_TWINS: dict[str, str] = { "PostgresValueAnalyticsStore": ( "ValueAnalyticsStore/PostgresValueAnalyticsStore own no CREATE TABLE " - "or CREATE INDEX: both read and write user_analytics_snapshots, " - "user_lifecycle_daily_snapshots and analytics_projection_jobs " + "or CREATE INDEX: both read and write user_daily_facts, " + "user_activity, lifecycle_transitions and analytics_projection_jobs " "through an injected analytics_base connection, and that base is " "AnalyticsStore/PostgresAnalyticsStore -- already a registered pair " "above, whose own schema-column test covers those tables." diff --git a/dashboard/frontend/admin.html b/dashboard/frontend/admin.html index 25f4fae0a..6ad5e4988 100644 --- a/dashboard/frontend/admin.html +++ b/dashboard/frontend/admin.html @@ -18,7 +18,7 @@ - + diff --git a/dashboard/frontend/js/admin-overview.js b/dashboard/frontend/js/admin-overview.js index e45ab662f..595bdb739 100644 --- a/dashboard/frontend/js/admin-overview.js +++ b/dashboard/frontend/js/admin-overview.js @@ -695,8 +695,8 @@ // Rollups are anonymous (design D13, §8.2): the failure table carries only // a category and its count, with no per-user breakdown column. section(root, 'Failure categories', 'Display-safe categories from the daily rollups', table( - ['Failure category', 'Count'], - (operational?.top_failure_categories || []).map((failure) => [s.humanize(failure.error_category), s.formatNumber(failure.affected_users)]), + ['Failure category', 'Events'], + (operational?.top_failure_categories || []).map((failure) => [s.humanize(failure.error_category), s.formatNumber(failure.event_count)]), 'No failure categories in this range.' )); section(root, 'What is blocking users?', 'Operational reasons as of yesterday UTC', table( diff --git a/dashboard/scripts/backfill_analytics.py b/dashboard/scripts/backfill_analytics.py index e4da83144..b43303568 100644 --- a/dashboard/scripts/backfill_analytics.py +++ b/dashboard/scripts/backfill_analytics.py @@ -30,6 +30,12 @@ def _before(value: str) -> datetime: def _parser() -> argparse.ArgumentParser: parser = argparse.ArgumentParser( description="Backfill safe server-authoritative Analytics history.", + epilog=( + "Inserted events update the raw event log and the day rollups only. " + "They do not update user_activity (activation dates, last meaningful " + "activity) or user_daily_facts; the daily job's late-arrival sweep " + "recomputes facts for the most recent days only." + ), ) parser.add_argument( "--days", @@ -45,7 +51,7 @@ def _parser() -> argparse.ArgumentParser: parser.add_argument( "--dry-run", action="store_true", - help="Validate and count without writing events, snapshots, or rollups.", + help="Validate and count without writing events or rollups.", ) return parser diff --git a/dashboard/scripts/benchmark_analytics_queries.py b/dashboard/scripts/benchmark_analytics_queries.py index 23ca91042..477917bb4 100644 --- a/dashboard/scripts/benchmark_analytics_queries.py +++ b/dashboard/scripts/benchmark_analytics_queries.py @@ -135,25 +135,19 @@ def _seed(path: Path, *, users: int, events_per_user: int, days: int) -> None: ) conn.executemany( """ - INSERT INTO user_analytics_snapshots ( - user_id, status, reason_code, human_readable_reason, - evidence_event_ids_json, calculated_at - ) VALUES (?, ?, ?, ?, '[]', ?) + INSERT INTO user_daily_facts ( + snapshot_date, user_id, lifecycle_segment, lifecycle_reason_code, + operational_state, operational_reason_code, tier, active, + data_quality, calculated_at + ) VALUES (?, ?, 'core', 'core_repeated_value', ?, ?, 'unpaid', 1, + 'complete', ?) """, ( ( + (BENCHMARK_NOW.date() - timedelta(days=1)).isoformat(), user_id, - "needs_attention" if user_id % 20 == 0 else "active", - ( - "invalid_default_credential" - if user_id % 20 == 0 - else "recent_successful_run_and_activity" - ), - ( - "The default model credential is invalid." - if user_id % 20 == 0 - else "A successful run and recent meaningful activity are present." - ), + "needs_attention" if user_id % 20 == 0 else "healthy", + "invalid_default_credential" if user_id % 20 == 0 else None, now_iso, ) for user_id in range(1, users + 1) diff --git a/docs/superpowers/specs/2026-09-15-admin-layer-redesign-design.md b/docs/superpowers/specs/2026-09-15-admin-layer-redesign-design.md index 955123c6c..564879ad4 100644 --- a/docs/superpowers/specs/2026-09-15-admin-layer-redesign-design.md +++ b/docs/superpowers/specs/2026-09-15-admin-layer-redesign-design.md @@ -224,7 +224,7 @@ The 09-12 document ranked cost control **first**. *Why it moved:* that ranking w ### Metrics -**D13 — `top_failure_categories` is kept and served from `analytics_daily_rollups` on the `error_category` dimension.** *Why:* the 09-12 rewrite silently deleted it (zero mentions across its 4,930 lines of spec and plan), while both frontends render it. The rollups already key on `error_category`, so keeping it costs no schema change and no raw-event read. +**D13 — `top_failure_categories` is kept and served from `analytics_daily_rollups` on the `error_category` dimension.** *Why:* the 09-12 rewrite silently deleted it (zero mentions across its 4,930 lines of spec and plan), while both frontends render it. The rollups already key on `error_category`, so keeping it costs no schema change and no raw-event read. (**Amended 2026-10-16**, #647.) Each category counts **events**, from `event_count` rows plus today's raw slice, with or without a billing/provider/model filter. The rollups are anonymous, so a distinct-user count over a range would need a raw read or a rollup keyed by user. Summing the per-day distinct-user rows counts user-days, and those rows had no dimensions, so a filtered view counted events anyway. The field is `event_count`, renamed from `affected_users`, and `rollup_day` no longer writes the `affected_users` family. PR B makes this one contract and frontend change, to `admin-overview.js` (one field and the column header), in the same commit as its only reader. **D14 — Billing-lane mix (BYOK vs platform credits) is kept, from the rollups' `billing_mode` dimension.** Same reasoning as D13. @@ -474,7 +474,7 @@ Pinned by a test that wraps **every store the job touches** — analytics, credi | daily job, per UTC day | a constant number of queries, independent of user count, none parameterised by a single user id | | daily job, per UTC day (**Amended**, new) | wall-clock under a fixed bound at 200 users, and the 400-user run no more than 2.5× the 200-user run, so a constant-count step that scans an unindexed table or hides a quadratic fold is caught | | admin overview page load | at most 1 raw-event query, windowed to the current UTC day | -| admin user profile load | at most 1 raw-event query (timeline page) | +| admin user profile load | at most 1 raw-event query (timeline page), plus (**Amended 2026-10-16**) one grouped `MIN(occurred_at)` returning at most 2 rows for the `credential_verified` / `agent_created` milestones — first occurrences the newest-100 page cannot answer (PR B review #8). Moving them onto `user_activity` would drop it; not done in PR B because it touches the ingestion hot path | The PR A plan fixes the daily-job constant from the final step list and the test pins that exact number. The test fails on the first user-id-parameterised query inside the daily job regardless of count, and it fails if doubling the synthetic user count changes the query count at all.