Skip to content

crossposting: one scoring module with named factors; LIVE picks per channel - #392

Merged
ohld merged 2 commits into
productionfrom
refactor/crosspost-scoring
Oct 31, 2026
Merged

ohld merged 2 commits into
productionfrom
refactor/crosspost-scoring

Conversation

@ohld

@ohld ohld commented Oct 31, 2026

Copy link
Copy Markdown
Member

What

All crosspost scores now live in one module, src/crossposting/scoring.py:

  • FACTORS: named factor functions (row, channel) -> float | None: the v2 parts, like_volume_factor, v5_base, content_mult_v1, src_fwd_mult.
  • SCORES: maps each score name to the factors it multiplies: v2, v4, v5, v5c, v5f.
  • PRESELECT / LIVE: which score picks the top-5 from the pool, and which score picks the meme from that top-5, per channel. Both are set to today's behaviour (RU v4, EN v2).
  • SCORE_VERSION: maps score name to the int stored in crossposting.score_version / decision_log.score_version.

To promote a shadow score, edit LIVE (for example LIVE["tgchannelru"] = "v5f", which gives score_version 7). If the shadow inputs fail to load, the pick falls back to PRESELECT.

SQL (_POOL_QUERY) now only filters and fetches the eligible pool. Ordering and the pick happen in Python. Fetching the whole pool (~128k RU / 45k EN rows) took 1.3 s on prod. The share-click counts moved into the shadow-inputs query, so they are log-only and can no longer block a post.

Deleted: shadow_score.py, _compute_score_breakdown (the "must stay in sync" copy), both SQL ORDER BY formulas, attach_v5_ranks, shadow_version, and the CROSSPOST_RU_MEME_LIKE_VOLUME_ENABLED setting. LIVE is now the switch. The setting is env-only and pydantic ignores unknown env vars, so removing it is safe.

Decision log

Each candidate now carries factors, scores, ranks, picks (each score's would-be pick) and picked_by. It also still emits the legacy flat keys that the readouts read (final_score, lr_factor, like_volume_*, score_v5/v5c/v5f, shadow_v5*_rank, shadow_v5*_pick_meme_id, content_mult_v1, src_fwd_pct, rank). The docs/analyst/crossposting-*.sql files therefore work on old and new rows without changes. Caveat, unchanged from before: shadow scores are ranked only within the production top-5.

Two existing quirks, kept so the picks don't change (product call needed)

The replay found that the old SQL ORDER BY did not match the logged breakdown:

  1. The impression penalty has always been 0. CASE WHEN raw_impr_rank <= 1 THEN 1 ELSE :impr_penalty END gets typed as integer, so asyncpg sent 0 instead of 0.8 (RU) / 0.5 (EN). Any meme with raw_impr_rank > 1 scored 0. Prod confirms it: across the last 40 decisions, every top-5 candidate has raw_impr_rank 0 or 1.
  2. Sources without enough history get a 0.5 multiplier. GREATEST(0.5, NULL) = 0.5, so a source with fewer than 5 posts got src_quality_mult 0.5, not the neutral 1.0 that the spec intended and the log claimed.

The code keeps both (impr_penalty: 0.0, src_quality_mult returns 0.5) with comments. final_score / impr_factor / src_quality_mult on new rows now show the values that actually ranked the memes. Fixing either quirk changes the picks, so it should be its own experiment.

Proof of zero pick change

  • Local replay (scratch script: old service.py from origin/production vs this branch, same DB). 120 random fixture sets (60 per channel) covering missing src quality, impression ranks 0/1/2/5/99999, captions, age thresholds, nmemes_sent 0/1/n, invited_count > 10, videos (excluded), a source posted within 24h (excluded), NULL/present content tags and exact ties. Result: same pick, same top-5 order, same score_version and same pool size in 120/120.
  • Prod decision-log replay (read-only). Re-ranked the last 40 logged top-5s with scoring.py: 40/40 picks and 40/40 top-5 orders match.
  • Prod current-pool replay (read-only). The old SQL top-5 vs scoring.py over the full current pool (127,532 RU / 45,074 EN): same pick and same top-5 on both channels.
  • Also ran the new shadow-inputs query on prod (read-only, 5 ids): it works.

Tests

tests/crossposting, tests/flows/crossposting and tests/test_crossposting_meme.py pass (42). ruff check and ruff format are clean.

ohld added 2 commits October 31, 2026 20:36
…hannel)

src/crossposting/scoring.py holds FACTORS, SCORES, PRESELECT, LIVE and
SCORE_VERSION. SQL only fetches the eligible pool; Python preselects the
top-5, attaches shadow inputs and picks with LIVE. Promoting a shadow
score is now a one-line LIVE edit.

Picks are unchanged. The old SQL ORDER BY had two quirks that the logged
breakdown hid; both are kept on purpose:
- :impr_penalty was bound into an integer CASE, so memes with
  raw_impr_rank > 1 always scored 0 (not 0.8 RU / 0.5 EN);
- sources without enough history got src mult 0.5 (GREATEST ignores
  NULL), not the logged 1.0.

Removes shadow_score.py, the duplicated breakdown, and the
CROSSPOST_RU_MEME_LIKE_VOLUME_ENABLED setting (LIVE is the switch).
Decision log keeps the legacy flat keys the readout SQL reads and adds
factors/scores/ranks/picks/picked_by dicts.
@ohld
ohld merged commit 2a9bcae into production Oct 31, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant