Skip to content

metrics: shared v_* views + north_star.sql (one definition of sessions, actives, shares) - #393

Merged
ohld merged 2 commits into
productionfrom
feat/metrics-views
Oct 31, 2026
Merged

ohld merged 2 commits into
productionfrom
feat/metrics-views

Conversation

@ohld

@ohld ohld commented Oct 31, 2026 •

Copy link
Copy Markdown
Member

Why

Today there are ~4 session definitions in docs/analyst/ (reaction gaps vs send gaps, with or without push rows, >= 2 memes or not), and readouts silently disagree. This adds ONE metrics layer that agents and experiment readouts share.

What

One Alembic migration (a7c3e9f1b5d2, on top of e5a7c9b1d3f4): plain views + one table. SET lock_timeout = '5s', CREATE OR REPLACE VIEW / CREATE TABLE IF NOT EXISTS, so it is safe to re-run.

  • analytics_excluded_user(user_id, reason, created_at): automated accounts, stored as data so user IDs never enter the repo. The migration leaves it empty.
  • v_metric_user: segment (excluded / moderator / admin / user), signup, acquisition channel (the first /start deep link within a day of signup: organic / share / channel / vcru / ...; unknown before the log existed), languages.
  • v_feed_session: feed sends only (recommended_by NOT LIKE 'broadcast%'); a new session starts after a 30-min gap. Columns: memes_sent, reactions, likes, started_by_push.
  • v_user_day: memes_sent / reactions / likes (feed + push), feed sessions, and shares (distinct non-self clicks on the user's m_/s_ links).
  • v_new_user: got_first_meme, first_tap, first_like (all on the first meme), reached_5_reactions (within 7d), d1/d7 return (NULL until observable), blocked_within_24h.
  • v_experiment_user: one row per experiment_assignment, with outcomes since assignment (reactions/likes per week, memes per session) for user-level-mean readouts.
  • docs/analyst/north_star.sql: 4 queries over the last 28 days in 7-day buckets. All of them drop segment='excluded'. Q2 and Q3 also drop moderators and admins, because their low_sent_pool review work inflates reactions and session length.
  • docs/analyst/excluded-users-detector.sql: finds users with >= 50 push reactions where >= 50% of them fall in one 10-second bin of (reacted_at - sent_at).
  • Docs: analyst README (the views are the definitions), SPEC north star link, 2 lines in AGENTS.md. explore_budget_v1 keeps its frozen readout SQL; a note in its experiment file asks for a v_experiment_user cross-check.

Performance (prod, read-only, views inlined as NOT MATERIALIZED CTEs, 28-day window)

query execution
Q1 new users by channel 0.12 s
Q2 reactions per active 8.7 s
Q3 median memes/session 2.7 s
Q4 shares + invites per 1k 8.7 s

Filter on the views' date columns with plain bounds (CURRENT_DATE - 28). With DATE_TRUNC(now()) bounds the filter is evaluated per row, and Q2/Q4 took about 25 s. Session-start detection uses LATERAL ... LIMIT 1 so the planner keeps per-row index probes; with NOT EXISTS it hashed the whole table and took more than 150 s.

Tests

tests/test_metrics_views.py builds a tiny fixture (3 users, feed + push sends, share deep links, one excluded user, one assignment) and checks every view. Upgrade, re-run after stamp and downgrade were checked manually against a local DB (not in the test).

Deploy

  1. Apply the migration on prod BEFORE merge. It is views only, so it is safe for the old code.
  2. Then the operator runs docs/analyst/excluded-users-detector.sql, reviews it (today it returns 1 candidate), and inserts the rows:
    INSERT INTO analytics_excluded_user (user_id, reason) <query> ON CONFLICT (user_id) DO NOTHING;

Review fixes

  • started_by_push now requires the push to have been opened within 15 min of delivery. Before, idle users always had a push as the last chat message, so about 90% of returns looked push-driven.
  • reached_5_reactions is NULL until 8 days after signup.
  • super_user gets its own segment.
  • Documented: v_user_day attributes reactions to the send day; the v_experiment_user window is open-ended, so bound it in readouts.

ohld added 2 commits October 31, 2026 20:42
One definition of feed session (30-min gap on feed sends; broadcast% is push),
actives, non-self shares, new-user funnel and per-assignment outcomes, as plain
views. analytics_excluded_user holds automated accounts (filled by an operator
from excluded-users-detector.sql). north_star.sql is the canonical readout.
… guard reached_5 for young users; super_user segment
@ohld
ohld merged commit 8e1c0c6 into production Oct 31, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant