Skip to content

fix(web): fetch the preceding period so Explorer stat trends stop reading +100% - #708

Merged
dash0-dev[bot] merged 16 commits into
mainfrom
fix/explorer-stats-trend-window
Oct 21, 2026
Merged

dash0-dev[bot] merged 16 commits into
mainfrom
fix/explorer-stats-trend-window

Conversation

@mthines

@mthines mthines commented Oct 20, 2026 •

Copy link
Copy Markdown
Owner

Every stat card in the Lore Explorer's Activity panel showed a +100% trend at every range.

The trend chip compares the charted window against the equal-length period before it (computeRangeTrends / computeCountTrend tally 2 × count buckets). But fetchExplorerStats only asked the server for the selected window, so the comparison half was always empty, and pctChange(n, 0) returns +100% for any n.

Fix: activity and read-activity are now each fetched a second time, over the grid's comparison period (comparisonWindow in lib/aggregations.ts):

  • The charted span, one grid length earlier: today until 13:00 is compared with yesterday until 13:00, so both sides cover the same length and the same time of day, and a daily rhythm doesn't read as a decline.
  • The charted span stops at now: a preset ends partway through its last bucket and today's heatmap cell ends tomorrow, so neither is compared against more time than the chart covers.
  • Both halves split at the first charted bucket (chartStart), so an unaligned range can't leak earlier rows into the first bar and nothing is counted twice, not even against the Storybook mocks, which ignore the bounds.

The selection fetch and the Archived/Expired usage summary are unchanged. The cost is two extra small requests per header load.

Headlines: stat-card figures of 10,000 or more now abbreviate (12.3K, 1.57M, via formatHeadline) so the trend chip keeps its room beside them; the exact value stays in the hover title and the screen-reader announcement. The Storybook visual baselines are refreshed.

Tests: explorer-stats.trend.spec.ts drives fetchExplorerStats against a fake server that filters by created_at the way the RPCs do. It pins:

  • a uniform rate reads 0% on the 24h/7d/30d presets and on today's heatmap cell
  • a 09:00–18:00 daily rhythm reads 0% on today's cell and the 7d preset
  • an unaligned 10:30→12:00 range counts no pre-window rows
  • usage stays on the selection

ExplorerStatsTrend.test.stories.tsx is the end-to-end repro. It renders the Activity panel against MSW handlers that honour the window and serve a uniform rate, and imports nothing this PR adds. On main it fails on the chip itself (+1200%–+5800% where 0% is honest); with the fix it passes.

Docs: tags.mdx and docs/key-files.md describe the headline abbreviation. The trend fix restores documented behaviour ("vs. the preceding period of the same length").

🧪 UI verification spec — run against the preview deployment

Target: preview
Format: intent

Spec 1: The Activity panel's trend chips compare against the preceding period

Changed: the trend chips on the Stat charts cards (Memories written, Memories retrieved, Memories opened, Scopes active) in the Lore Explorer's Activity panel
Start: /lore
Preconditions:

  • The signed-in account has written and read memories in both the last 7 days and the 7 days before that
    Steps:
  1. Open the Lore Explorer and make sure the Activity panel shows the Stat charts view
  2. [must-follow] Select the 7d range in the Activity panel
  3. Read the trend chip on each of the four cards
  4. [must-follow] Select the 30d range and read the four trend chips again
    Expected:
  • At 7d, at least one of the four trend chips shows a value other than +100%
  • At 30d, at least one of the four trend chips shows a value other than +100%
  • Each card's headline number and sparkbar still describe only the selected range (the caption reads "in the last 7 days" / "in the last 30 days")
  • The Memories archived card still shows no trend chip
    Hints:
  • Signed-in check: {role: "link", name: "User settings for"}
  • Range buttons sit in the Activity panel header: 24h, 7d, 30d, All
    Out of scope:
  • The exact percentages, which depend on the account's data

…ading +100%

The stat cards' trend chip compares the selected window against the
equal-length window before it, but the activity and read-activity queries
only fetched the selected window. The comparison half was always empty, so
pctChange(n, 0) printed +100% on every card for every range.

Widen the two series fetches by one window length (comparisonSince); the
usage summary (Archived/Expired, no trend) stays on the selected window.
…er since

Reaching one window further back with a single request let rows from before
an unaligned absolute since into the first charted bar, inflating the
headline. Fetch the grid's comparison buckets as their own request
(comparisonWindow) and keep the selection's fetch unchanged.
…both halves

A preset's charted half ends at now, mid-bucket, so a comparison of count
whole buckets was longer and read a flat account as a decline. Size it by
what the chart actually covers. Clip the comparison rows to before the first
charted bucket too, so a backend ignoring the bounds (the Storybook mocks)
cannot count a charted bar twice.
@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

The dashboard preview for this PR — redeployed on each push that changes the web app.

Project Deployment Actions Updated
lorekit Ready Ready Preview Oct 21, 2026 3:43pm UTC

Preview always points at this PR's latest commit; Deployment is the immutable build of 47f0dc6. · Comment /web-preview to force a redeploy. · Workflow logs

@mthines

mthines commented Oct 20, 2026 •

Copy link
Copy Markdown
Owner Author

✅ No issues found

Fixes the Explorer trend chips reading +100% by fetching the preceding period, and abbreviates large headlines. This push only regenerated visual baselines, which clears the stale LorePage screenshot.

Progress: open review threads 1 → 1 → 0 → 0 → 0 across the last 5 reviews

Review details
Gate Details
Description vs. code ✅ The description matches what the diff does.
Prior review feedback ✅ Earlier review comments are resolved.
Documentation ✅ The change is documented well enough to follow.
Self-review signals ✅ No debug logs, leftover TODOs, or unreviewed stubs.
Code review ✅ The multi-lens review found no blocking issues.
Run Details
Mode incremental · 0 lines in delta · depth checkout
Anomaly ⚠️ no authored code since 97d86d0, only regenerated PNG baselines and an empty CI commit, so no finder or lens ran; the baselines and the carried LorePage finding were checked directly
Checked 1 possible issue → 0 confirmed → 0 posted
Reach 9 changed exports · 9 dependent files · 0 checked · 9 not checked
fetchExplorerStats — 1 consumer file · 0 checked
statsWindow — 4 consumer files · 0 checked
StatCard — 2 consumer files · 0 checked
useExplorerStats — 2 consumer files · 0 checked
ExplorerStats — 1 consumer file · 0 checked
+4 more
Found produced 1 → posted inline 0 · cleared 0 · carried forward 0 · deferred 0 · below-bar 0
Verified LorePage Default baseline now byte-identical to main (5359ebf, 960x1793), clearing the carried finding; Storybook visual job passes at 47f0dc6; the other 11 PNGs differ from main by at most 1.2% px, under the 2% pixelmatch allowance
Memories 166 indexed · 2 used (1 hotspot · 1 lesson)

Memories used

  • lesson reviewer-lessons::zero-delta-run-has-no-gate6-carry-forward-channel-into-finalize — zero-delta handling: carried finding re-checked before Gate 6, mode zero-delta, no routed tier, one disclosed RUN_ANOMALY
  • hotspot hotspot::packages/web/src/app/(dashboard)/lore/__screenshots__/LorePage.stories.tsx/Default-1-chromium-linux.png — pointer to the carried finding's file (re-verified: head blob equals main)

Nothing to report — standards (skipped), measurability (skipped), integrations (skipped), severity, 0 files skipped.

pr-reviewer · commit 47f0dc6 · no code changes since 97d86d0, gate checks only · how these findings are produced · updated Oct 21, 2026 3:51pm UTC

Comment thread packages/web/src/lib/queries/explorer-stats.trend.spec.ts Outdated
Renders the Activity panel against MSW handlers that honour since/until and
serve a uniform rate, and asserts the trend chip reads 0%. Imports nothing
this PR adds, so on main it fails on the trend itself (+1200%..+5800%).
@mthines

mthines commented Oct 20, 2026

Copy link
Copy Markdown
Owner Author

Local verify — ⚠️ verified with open items

The fix works on a real stack: at 24h, 7d, 30d and All, every trend chip matches the change computed independently from the API responses, and the header makes exactly one selection request and one comparison request per series. No code changes were needed. One low-severity design trade-off is open (F-1): a failed comparison request blanks the whole header. Two findings were already on main before this PR and are out of scope.

PR #708 · no ticket
Commit verified 05237c38 (branch fix/explorer-stats-trend-window)
Stack web (next dev, webpack) → production edge functions · command, serving this checkout
Telemetry dash0-mcp · dataset default (RUM service.name=web, edge service.name=api) · 1 loop(s)
Run 2026-10-20T10:38:25Z → 2026-10-20T10:55Z

Acceptance criteria

ID Criterion Result Evidence
AC-1 Each trend chip equals the period-over-period change computed independently from the responses (24h / 7d / 30d / All) pass S-1, S-2, S-3; oracle match on all four ranges (e.g. 24h Written 1 vs 12 → −92%; 30d 38 vs 123 → −69%)
AC-2 The headline equals the selection's charted buckets; the Archived card has no chip pass S-1–S-3; headlines match the oracle; Archived hasChip=false on every probe
AC-3 One selection + one comparison request per series; the comparison ends at the first charted bucket and covers the charted span; all 2xx pass RUM: 2× POST /activity, 2× GET /read-activity, 1× GET /usage per load, all 200; e.g. 7d comparison 2026-10-07T13:07Z → 2026-10-14T00:00Z, the same length as the selection, ending where the chart starts

Edge cases

ID Probe Result Evidence
EC-1 Absolute day range (3 days) pass Comparison 10-09 → 10-13 sits right before 10-13 → 10-17; chips match the oracle
EC-2 Drilled single hour pass One bucket shows no chip (StatCard needs ≥2 points; behaviour unchanged from main); requests still well-formed
EC-3 Unaligned ISO range 10:30 → 12:00 pass S-4; comparison 08:30 → 10:00; no rows from before the window counted
EC-4 Scope selected pass All 4 series requests carry scope=repo::mthines/lorekit; chips match the oracle
EC-5 Comparison request fails (injected 500) pass (see F-1) S-5; the documented error message, no uncaught error, no blank page
EC-6 Rapid toggling 24h → 7d → 30d → 24h pass Superseded requests cancelled (RUM: no status); final chips match the final range
EC-7 375px viewport at 7d pass S-6; no horizontal overflow
Not applicable (2)
  • Permissions / content / size: the diff touches no input, authorization, or rendering of user text.
  • Empty window: covered by the unit spec and Storybook (0 vs 0 → 0%); the live account always has activity.

Findings

ID Severity Kind What happened Status
F-1 low ux If only a comparison request fails, the whole stats header shows its error state, so the selection figures go with the chips open (design trade-off)
F-2 medium crash pre-existing: next dev --turbopack fails on /lore with "Export ApiToken doesn't exist in target module" (type re-export from a 'use server' file, lib/tokens.ts, since #697). Webpack dev and the production build are unaffected open, out of scope
F-3 low performance pre-existing: RUM reports CLS 0.34 on /lore loads. Not caused by this diff (no layout change); not compared against main open, out of scope

Fixes

No code changes.

Screenshots

S-1 — AC-1: 24h chips
S-2 — AC-1: 7d chips
S-3 — AC-1: 30d chips
S-4 — EC-3: unaligned range
S-5 — EC-5: comparison request fails
S-6 — EC-7: 375px

Under the hood

  • AC-1/AC-2: browser → POST /functions/v1/memories/activity ×2 + GET /read-activity ×2 + GET /usage → 200 each, ~0.5–0.9 s, run in parallel. Edge spans continue the browser's trace IDs; no ERROR span on faas.name=memories across the run.
  • AC-3: the comparison request's window always ends at the chart's first bucket and has the same length as the charted span (7d: 10-07T13:07 → 10-14T00:00; 30d: 08-22 → 09-21).
  • The only 5xx in the run: 6× POST /activity 500 at 1–4 ms, the Playwright-injected EC-5 failure. The only browser.error: F-2, from before the stack switched off Turbopack.

Open and escalated

  • F-1 — what a comparison failure should do. Today Promise.all turns any one failed request into the header's error state.
    • Option (a): keep as is. It is graceful, and it is how a failure of the selection request already behaves.
    • Option (b): on a comparison failure, keep the figures and hide only the chips. Falling back to "no comparison data" without hiding the chips would bring back the +100% bug.
    • Recommendation: (a) for this PR; (b) as a follow-up if partial failures turn out to be common.
  • F-2: a separate fix, e.g. import the types with import type in their consumers instead of re-exporting them from the 'use server' module.

Lessons

  • local-verify-lessons::turbopack-dev-use-server-type-export (repo scope)
  • local-verify-lessons::networkidle-never-settles (repo scope)

Generated by /local-verify · full artifacts in .agent/fix/explorer-stats-trend-window/local-verify/

mthines and others added 3 commits October 20, 2026 13:08
…d chip keeps its room

A 1,566,285 headline pushed the trend chip beside it off the card. Headlines now
render exact and grouped below 10,000 and compact to three significant digits
above it (12.3K, 883K, 1.57M), so they never exceed five characters. The exact
figure stays on hover and in AnimatedNumber's announced twin.

Co-Authored-By: Claude <[email protected]>
Regenerated by the Update visual baselines workflow for PR #708.
@github-actions

Copy link
Copy Markdown
Contributor

✅ Update visual baselines: committed 36 updated baseline(s) as 06ac47b830f9a8858e52a503f12f9a4ca9e359e9.

ℹ️ This commit was pushed with GITHUB_TOKEN, so web-test will not re-run automatically — push a follow-up commit if you need CI to re-verify.

The baseline workflow regenerated every stale screenshot on main. Restore the
ones outside this PR's --changed story set so the diff stays scoped; the
remaining updates are stories CI runs here (AnimatedNumber importers) whose
baselines had drifted on main.

Co-Authored-By: Claude <[email protected]>
@mthines
mthines marked this pull request as ready for review October 20, 2026 12:41
@dash0-dev

dash0-dev Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Darkplane Automerge did not approve this pull request. Score 65.81 of 100, threshold 75. Blast radius 80, Evidence quality 90, Change footprint 27.

Darkplane Automerge did not approve this pull request. It scored 65.81 of 100, and 75 was required.

  • 🔴 Change footprint scored 27.
    This is a hand-written logic change to the Explorer stats data path. explorer-stats.ts now makes two requests per series and clips the rows at chartStart, aggregations.ts gains 51 lines, and format-number.ts changes the shared COMPACT formatter to three significant digits, which also affects formatCompact. It is well covered, with about 411 lines of new or extended tests, and the rest is small UI wiring plus regenerated screenshots and one-line doc edits. The change is still novel, not mechanical.
The other 2 criteria
  • 🟢 Blast radius scored 80.
    The change is confined to the web package's Explorer stats path: explorer-stats.ts, aggregations.ts, ExplorerStats.tsx, StatCard.tsx and format-number.ts. All of it is client-side display code with no schema or migration changes. It adds new specs and stories and refreshes some screenshots. Main costs: Explorer stats now issue two extra preceding-period activity requests per refresh, and StatCard figure formatting changes everywhere it is used. A plain revert undoes it.
  • 🟢 Evidence quality scored 90.
    The change adds explorer-stats.trend.spec.ts (fake server filtering by created_at, 0% on 24h/7d/30d and today's cell), ExplorerStatsTrend.test.stories.tsx (end-to-end repro that fails on main), and new format-number.spec.ts cases. CI is green for Typecheck/Test/Lint, Storybook interaction and visual tests, Playwright e2e and the Vercel preview. Bot review comments were all addressed, and a local-verify run against a real stack matched an independent calculation, with one low-severity open item (a failed comparison request blanks the header). The reviews are bot and self comments, with no independent human approval.

An AI evaluator produced these scores from the diff, the repository history, and the pull request text. The pull request text is author-controlled, so treat the scores as signals rather than proof. Reviewed for commit 47f0dc6.

Was this decision right? Give feedback

Comment thread packages/web/src/lib/format-number.ts Outdated
Addresses @dash0-dev's comment: #708 (comment)
Refs: #708

Co-authored-by: dash0-dev[bot] <257284812+dash0-dev[bot]@users.noreply.github.com>
Comment thread packages/web/src/lib/aggregations.ts Outdated
…r it

Addresses @dash0-dev's comment: #708 (comment)
Refs: #708

Co-authored-by: dash0-dev[bot] <257284812+dash0-dev[bot]@users.noreply.github.com>
Comment thread packages/web/src/lib/aggregations.ts Outdated
… earlier

The comparison sat directly before the chart, so today's cell weighed this
morning against yesterday afternoon and read identical 09-18 days as -43%.
It is now the charted span moved back a whole grid length, with the chart
start carried explicitly for the row split. Presets move too: on a daily
rhythm 7d goes from -5% to 0% and 30d from -1% to 0%; 24h and uniform
rates are unchanged.

Addresses the pr-reviewer comment: #708 (comment)
Refs: #708

Co-authored-by: dash0-dev[bot] <257284812+dash0-dev[bot]@users.noreply.github.com>
Comment thread packages/web/src/lib/queries/explorer-stats.ts Outdated
…halves

Addresses the pr-reviewer comment: #708 (comment)
Refs: #708

Co-authored-by: dash0-dev[bot] <257284812+dash0-dev[bot]@users.noreply.github.com>
…end-window

# Conflicts:
#	packages/web/src/app/(dashboard)/lore/__screenshots__/LorePage.stories.tsx/Default-1-chromium-linux.png
#	packages/web/src/components/lore/__screenshots__/DuplicateClustersSidebar.stories.tsx/Default-1-chromium-linux.png
#	packages/web/src/components/lore/__screenshots__/DuplicateClustersSidebar.stories.tsx/Saturated-Window-1-chromium-linux.png
@mthines

mthines commented Oct 21, 2026

Copy link
Copy Markdown
Owner Author

/update-baselines

Regenerated by the Update visual baselines workflow for PR #708.
@github-actions

Copy link
Copy Markdown
Contributor

✅ Update visual baselines: committed 15 updated baseline(s) as 9b74f60fa921f8f6a077c0794fd1d8e2ad105494.

ℹ️ This commit was pushed with GITHUB_TOKEN, so web-test will not re-run automatically — push a follow-up commit if you need CI to re-verify.

The baselines commit was pushed with GITHUB_TOKEN, which does not trigger workflow runs.

Co-authored-by: dash0-dev[bot] <257284812+dash0-dev[bot]@users.noreply.github.com>
@dash0-dev
dash0-dev Bot merged commit d259a85 into main Oct 21, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant