diff --git a/docs/feature-flags.md b/docs/feature-flags.md index c1d3af27e..457e9b12b 100644 --- a/docs/feature-flags.md +++ b/docs/feature-flags.md @@ -56,9 +56,9 @@ it can never ship a flag that silently resolves to `undefined`. > **Every `example-*` key in this document is illustrative, not a real flag.** > Grep for one and you will find only this file. The registry holds exactly the -> flags that gate live behaviour — today three rollout gates, all plain booleans: -> `retention-policies`, `lore-explorer-instruments` and -> `lore-explorer-duplicate-clusters`. It has also held entries since removed: the +> flags that gate live behaviour — today four rollout gates, all plain booleans: +> `retention-policies`, `lore-explorer-instruments`, +> `lore-explorer-duplicate-clusters` and `insights-visual`. It has also held entries since removed: the > rollout gate for the `/insights` route (deleted once that page became > unconditional — see docs/decisions.md) and five demonstration entries (an > A/B experiment, two unread toggles, and one each to exercise the `string` diff --git a/docs/key-files.md b/docs/key-files.md index 53ec2b960..1e77071a3 100644 --- a/docs/key-files.md +++ b/docs/key-files.md @@ -93,6 +93,7 @@ else. Grouped loosely by subsystem in migration/PR order. | `supabase/functions/memories/handlers/clusters.ts` | `GET /memories/clusters` — near-duplicate clusters for the Explorer's Duplicate Clusters panel. **Two-phase by design** (the `GET /relevant` precedent): Postgres SELECTs the newest `CANDIDATE_LIMIT` visible rows, TypeScript does the set-relative work, so there is no plpgsql clustering and no migration. Read-only: it never merges, edits or deletes. The response echoes `candidate_limit` beside `candidates` so a caller can detect a saturated window (`candidates === candidate_limit`) instead of hardcoding the cap — the panel's footnote and the "run `lorekit dedupe` for the whole store" pointer both hang off that. Takes the reject-only `parseScopeFilter` (a scope filter IS the question here, and `memories.scope` is stored verbatim), pinned per handler by `scope-filter-validation.spec.ts`. REST-only (`telemetry-vocabulary.ts`), reporting as `memory.clusters` | | `packages/web/src/lib/duplicate-clusters-view.ts` | The pure half of Duplicate Clusters: stable `clusterId`/`memberLabel` identities, `findCluster` (resolve the held selection by VALUE so a refetch cannot silently swap what you are reading), `sizeLabel`, the three honest labels `similarityLabel` ("linked at …", never "every pair is at least"), `recurrenceLabel` (a partial class match is labelled partial) and `clustersSummary` + `windowSaturated` (an empty result over a FULL window is not a clean bill of health), and `lessonEntryFromClusterMember` (turns a cluster member into the `LessonEntry` shape `seedOptimisticLesson` writes into the shared `['lesson-by-ref', scope, key]` cache, the mechanism behind showing cluster members immediately on selection). Co-located `.spec.ts` | | `packages/web/src/components/lore/DuplicateClustersPanel.tsx` (+ `.on.tsx`, `.off.tsx`) and `DuplicateClustersSidebarPanel.tsx` (+ `.on.tsx`, `.off.tsx`) | Two independent `lore-explorer-duplicate-clusters` gates — one per render site (trigger, sidebar) — and the repo's LIVE example of the copy-and-suffix convention (`packages/feature-flags/CLAUDE.md`; the previous example was deleted for gating nothing). Each resolver dispatches on the variant KEY — never the resolved value — and its `default` arm lands on `off`, so a stale override cookie or a renamed variant fails closed on a surface that issues a quadratic-in-the-worst-case read. Two properties worth preserving on both: the `off` arm is a whole component returning `null` (retiring the flag is "delete these files", a complete instruction), and the ARM is separate from the IMPLEMENTATION (`.on.tsx` renders `DuplicateClusters.tsx` / `DuplicateClustersSidebar.tsx`, which own the behaviour, stories and pixel baselines) — so the flag's blast radius stays small files and `LoreExplorer.tsx` carries no flag read at all | +| `packages/web/src/components/insights/InsightsView.tsx` (+ `.on.tsx`, `.off.tsx`) and `components/insights/visual/` | The `insights-visual` resolver (copy-and-suffix, `default` lands on `off`) and the visual `/insights` page it gates. `off` renders the original `InsightsPage` unchanged; `on` renders `InsightsVisualPage`: ONE page-wide window, the Learn → Recall → Promote → Prune strip, the Pulse timeline (written / chosen / bulk as small multiples on a shared axis — never stacked, chosen would be a sliver), the recall flow (client → tool → scope type, calls in every band) and runs as a weekday × hour rhythm plus swimlanes. Pure models with specs: `lib/insights-loop.ts` (stage figures, pulse buckets, the `max_rows` truncation check), `lib/flow-layout.ts` (hand-rolled layered layout, one scale for every column), `lib/recall-flow.ts`, `lib/runs-rhythm.ts` (runs, not hourly activity, because hourly rows per scope pass PostgREST's 1000-row cap silently). Data: `lib/queries/insights-loop.ts` — existing routes only, current + preceding window | | `packages/web/src/components/lore/DuplicateClusters.tsx` | The compact TRIGGER — a single-row toolbar button, not the panel itself (see `docs/decisions.md`: an Explorer PANEL entry point, not an instrument — a grouping over lesson bodies is not a `?filters=` dimension). Opens COLLAPSED and the disclosure (`PREFERENCE_KEYS.explorerClustersOpen`, lifted to `LoreExplorer`) IS the React Query `enabled` flag it shares with `DuplicateClustersSidebar`, so a folded trigger issues no clustering request at all — the load-bearing part, pinned by the `FoldedMeansNotFetched` interaction test. Only renders the summary badge once opened at least once this session | | `packages/web/src/components/lore/DuplicateClustersSidebar.tsx` | The non-modal SIDEBAR the trigger reveals — a flex sibling of the Explorer's results column (never `position:fixed`, no backdrop), so a reader keeps clicking lesson rows while it is open. One `role="radiogroup"` of clusters; picking one hands the whole `DuplicateCluster` back to `LoreExplorer` BY VALUE (re-picking the held one clears the selection), which is what swaps the Explorer's own results list to that cluster's members (`renderResults`) instead of a nested detail pane — opening a member is the same click as opening any other lesson. Read-only, and its footnote says so plus which question a saturated window answered | | `packages/web/src/lib/explorer-instruments.ts` | The pure model behind the Explorer's instrument panel, and the place the rule lives: **an instrument is a filter INPUT, not a view** — every selection it makes is written to the same `?filters=` / `?range=` state the menu and the date picker write, so the list stays the single output. The test that keeps analytics on `/insights` rather than creeping in here: *can you click it and end up holding lore?* `MATRIX_AXES` is derived from `FILTER_FIELDS` (minus `pr`, whose per-value cardinality makes a grid of one-cell columns), so a dimension the bar can filter by is exactly one the matrix can pivot on. `buildMatrixGrid` decides every visible property of the grid — which values appear, in what order, how many were dropped; `heatStep` roots the ratio because memory counts are heavily skewed and a linear ramp paints the whole tail the same near-black; `dayIndexAt` and `brushRange` are the whole of a pointer gesture's correctness | diff --git a/docs/otel.md b/docs/otel.md index e551fdb8f..be24f8597 100644 --- a/docs/otel.md +++ b/docs/otel.md @@ -1046,14 +1046,16 @@ Three of these are load-bearing in a way a name does not convey: | Event | Emitted when | Attributes | |-------|--------------|------------| -| `insights.range_changed` | One of the page's two independent windows moved | `lorekit.insights.range.section` (`agent-activity`/`scope-consumption`), `.preset`, `.span_days` (optional) | +| `insights.range_changed` | An Insights window moved (`page` = the visual page's one window) | `lorekit.insights.range.section` (`agent-activity`/`scope-consumption`/`page`), `.preset`, `.span_days` (optional) | | `insights.utility_quadrant_selected` | A utility quadrant was opened or closed | `lorekit.insights.utility.quadrant`, `.selected` | | `insights.utility_prompt_copied` | A grooming prompt reached the clipboard | `lorekit.insights.utility.quadrant`, `.entry_count`, `.succeeded` | | `insights.scope_opened` | A row handed off to the Explorer | `lorekit.insights.scope.type`, `.source` | +| `insights.recall_flow_view_changed` | The visual page's recall flow swapped between chart and table | `lorekit.insights.recall_flow.view` (`chart`/`table`) | `insights.range_changed` names its SECTION because the page deliberately carries two independent windows (see `InsightsPage.tsx`); a bare preset could not say -which one a reader moved. `insights.scope_opened` is the only evidence that the +which one a reader moved. The flag-gated visual page (`insights-visual`) has a +single window, reported as `page`. `insights.scope_opened` is the only evidence that the leaderboard's and the grid's "pick a row to open it in the Explorer" promise is ever taken. diff --git a/packages/feature-flags/generated/flags.manifest.json b/packages/feature-flags/generated/flags.manifest.json index d80360004..02bf2cc31 100644 --- a/packages/feature-flags/generated/flags.manifest.json +++ b/packages/feature-flags/generated/flags.manifest.json @@ -54,6 +54,24 @@ "lore-explorer", "rollout" ] + }, + { + "key": "insights-visual", + "description": "Insights — the visual first version of `/insights`: ONE page-wide window, the Learn → Recall → Promote → Prune loop strip, the Pulse timeline (lessons written and records read on a shared time axis, chosen vs bulk), the recall flow (client → tool → scope type), and runs drawn as a weekday × hour rhythm plus swimlanes — with the existing 'earning its place' grid, runs list and operational health kept below. Gates the whole page body through the `InsightsView` resolver; with it off `/insights` renders exactly as before. Reads only routes that already ship (`/memories/activity`, `/read-activity`, `/usage`, `/usage/runs`, `/utility`), so there is no backend prerequisite and flipping it is a pure UI decision.", + "type": "boolean", + "variants": { + "off": false, + "on": true + }, + "defaultVariant": "off", + "experiment": null, + "owner": "@lorekit/web", + "tags": [ + "dashboard", + "web", + "insights", + "rollout" + ] } ] } diff --git a/packages/feature-flags/src/generated/flags.generated.ts b/packages/feature-flags/src/generated/flags.generated.ts index 49c5dbcb9..84c2ee4f8 100644 --- a/packages/feature-flags/src/generated/flags.generated.ts +++ b/packages/feature-flags/src/generated/flags.generated.ts @@ -8,13 +8,15 @@ export type FlagKey = | 'retention-policies' | 'lore-explorer-instruments' - | 'lore-explorer-duplicate-clusters'; + | 'lore-explorer-duplicate-clusters' + | 'insights-visual'; /** Flag key -> its evaluated value type. */ export interface FlagValueMap { 'retention-policies': boolean; 'lore-explorer-instruments': boolean; 'lore-explorer-duplicate-clusters': boolean; + 'insights-visual': boolean; } /** The value type `evaluateFlag(key, ...)` resolves to for a given key. */ @@ -24,5 +26,6 @@ export type FlagValue = FlagValueMap[K]; export const FLAG_KEYS: readonly FlagKey[] = [ "retention-policies", "lore-explorer-instruments", - "lore-explorer-duplicate-clusters" + "lore-explorer-duplicate-clusters", + "insights-visual" ]; diff --git a/packages/feature-flags/src/registry.ts b/packages/feature-flags/src/registry.ts index 2898bb62a..b215f83ea 100644 --- a/packages/feature-flags/src/registry.ts +++ b/packages/feature-flags/src/registry.ts @@ -78,6 +78,16 @@ const REGISTRY_INPUT = [ owner: '@lorekit/web', tags: ['dashboard', 'web', 'lore-explorer', 'rollout'], }, + { + key: 'insights-visual', + description: + "Insights — the visual first version of `/insights`: ONE page-wide window, the Learn → Recall → Promote → Prune loop strip, the Pulse timeline (lessons written and records read on a shared time axis, chosen vs bulk), the recall flow (client → tool → scope type), and runs drawn as a weekday × hour rhythm plus swimlanes — with the existing 'earning its place' grid, runs list and operational health kept below. Gates the whole page body through the `InsightsView` resolver; with it off `/insights` renders exactly as before. Reads only routes that already ship (`/memories/activity`, `/read-activity`, `/usage`, `/usage/runs`, `/utility`), so there is no backend prerequisite and flipping it is a pure UI decision.", + type: 'boolean', + variants: { off: false, on: true }, + defaultVariant: 'off', + owner: '@lorekit/web', + tags: ['dashboard', 'web', 'insights', 'rollout'], + }, ] satisfies unknown[]; function parseRegistry(input: unknown): readonly FlagDefinition[] { diff --git a/packages/web/src/app/(dashboard)/insights/page.tsx b/packages/web/src/app/(dashboard)/insights/page.tsx index 7fd6f90e0..171196682 100644 --- a/packages/web/src/app/(dashboard)/insights/page.tsx +++ b/packages/web/src/app/(dashboard)/insights/page.tsx @@ -1,6 +1,7 @@ import type { Metadata } from 'next'; import { Suspense } from 'react'; -import { InsightsPage, InsightsPageSkeleton } from '@/components/insights/InsightsPage'; +import { InsightsPageSkeleton } from '@/components/insights/InsightsPage'; +import { InsightsView } from '@/components/insights/InsightsView'; export const metadata: Metadata = { title: 'Insights' }; @@ -13,12 +14,16 @@ export const metadata: Metadata = { title: 'Insights' }; * * Suspense is required, not decorative: InsightsPage's RunsList/queries read * client-side state that opts the subtree out of static rendering. + * + * `InsightsView` is the `insights-visual` flag resolver: off renders the + * original `InsightsPage`, on renders the visual page. One skeleton serves both + * arms — they share the same outer layout. */ export default function Page() { return (
}> - +
); diff --git a/packages/web/src/components/dashboard/StatCard.tsx b/packages/web/src/components/dashboard/StatCard.tsx index a4733c3fe..1a7c28d75 100644 --- a/packages/web/src/components/dashboard/StatCard.tsx +++ b/packages/web/src/components/dashboard/StatCard.tsx @@ -56,16 +56,32 @@ export function UnitTag({ label }: { label: string }) { * assistive tech — the same two-node pattern `AnimatedNumber` uses, for the same * reason. Below the threshold there is no twin, so nothing changes for the * overwhelming majority of chips. + * + * ## `tone="neutral"` when up is not "good" + * + * Green-for-up says more is better. For a figure where neither direction is a + * verdict — more pruning, more reads — `neutral` keeps the arrow and the sign + * (direction stays readable without colour) and drops the green/red judgment. */ -export function TrendChip({ changePct, title }: { changePct: number; title: string }) { +export function TrendChip({ + changePct, + title, + tone = 'semantic', +}: { + changePct: number; + title: string; + tone?: 'semantic' | 'neutral'; +}) { const dir = changePct > 0 ? 'up' : changePct < 0 ? 'down' : 'flat'; const Icon = dir === 'up' ? TrendingUp : dir === 'down' ? TrendingDown : Minus; const color = - dir === 'up' - ? 'text-[var(--color-success)]' - : dir === 'down' - ? 'text-[var(--color-error)]' - : 'text-[var(--color-content-tertiary)]'; + tone === 'neutral' && dir !== 'flat' + ? 'text-[var(--color-content-secondary)]' + : dir === 'up' + ? 'text-[var(--color-success)]' + : dir === 'down' + ? 'text-[var(--color-error)]' + : 'text-[var(--color-content-tertiary)]'; const abbreviated = isPercentDeltaAbbreviated(changePct); const exact = `${changePct > 0 ? '+' : ''}${changePct}%`; diff --git a/packages/web/src/components/insights/InsightsView.off.tsx b/packages/web/src/components/insights/InsightsView.off.tsx new file mode 100644 index 000000000..b79fb81e3 --- /dev/null +++ b/packages/web/src/components/insights/InsightsView.off.tsx @@ -0,0 +1,14 @@ +'use client'; + +/** + * The `off` arm of `insights-visual` — the original Insights page, unchanged. + * + * A whole component rather than an inlined `` in the resolver, + * so retiring the flag is "delete this file" — a complete instruction. + */ + +import { InsightsPage } from './InsightsPage'; + +export function InsightsViewOff() { + return ; +} diff --git a/packages/web/src/components/insights/InsightsView.on.tsx b/packages/web/src/components/insights/InsightsView.on.tsx new file mode 100644 index 000000000..dd7e3b8f8 --- /dev/null +++ b/packages/web/src/components/insights/InsightsView.on.tsx @@ -0,0 +1,15 @@ +'use client'; + +/** + * The `on` arm of `insights-visual` — the visual page. + * + * Deliberately thin: the behaviour, stories and pixel baselines live in + * `visual/InsightsVisualPage.tsx`, so retiring the flag never touches the + * component under test. + */ + +import { InsightsVisualPage } from './visual/InsightsVisualPage'; + +export function InsightsViewOn() { + return ; +} diff --git a/packages/web/src/components/insights/InsightsView.test.stories.tsx b/packages/web/src/components/insights/InsightsView.test.stories.tsx new file mode 100644 index 000000000..15f9004f1 --- /dev/null +++ b/packages/web/src/components/insights/InsightsView.test.stories.tsx @@ -0,0 +1,60 @@ +import type { Meta, StoryObj } from '@storybook/react'; +import { expect, waitFor, within } from 'storybook/test'; +import { InsightsView } from './InsightsView'; +import { FROZEN_NOW } from '@/mocks/memories'; +import { withQueryClient, withFrozenClock, withFlagVariants } from '@/mocks/decorators'; +import { visualHandlers } from './visual/insights-visual.fixtures'; + +/** + * The FLAG gate on the `/insights` page body — `InsightsView` is the + * copy-and-suffix resolver for `insights-visual`. + * + * What is being guarded: the flag ships OFF, so the off arm must be the + * original page, untouched, and an unknown variant (a stale override cookie, + * a renamed variant) must land on it too. Each arm is identified by a control + * only it renders: the original page's "Agent activity time range" picker, and + * the visual page's "The learning loop" strip. + */ +const meta: Meta = { + title: 'Pages/Insights/InsightsView/Tests', + component: InsightsView, + tags: ['test'], + parameters: { + chromatic: { disableSnapshot: true }, + layout: 'fullscreen', + // A superset of both arms' endpoints, so either page renders against data. + msw: { handlers: visualHandlers() }, + nextjs: { appDirectory: true }, + }, + decorators: [withFrozenClock(FROZEN_NOW), withQueryClient], +}; + +export default meta; +type Story = StoryObj; + +async function expectOriginalPage(canvasElement: HTMLElement) { + const canvas = within(canvasElement); + await waitFor(() => expect(canvas.getByRole('radiogroup', { name: 'Agent activity time range' })).toBeInTheDocument()); + await expect(canvas.queryByRole('region', { name: 'The learning loop' })).toBeNull(); + await expect(canvas.queryByRole('radiogroup', { name: 'Insights time range' })).toBeNull(); +} + +export const OffRendersTheOriginalPage: Story = { + decorators: [withFlagVariants({ 'insights-visual': 'off' })], + play: async ({ canvasElement }) => expectOriginalPage(canvasElement), +}; + +export const OnRendersTheVisualPage: Story = { + decorators: [withFlagVariants({ 'insights-visual': 'on' })], + play: async ({ canvasElement }) => { + const canvas = within(canvasElement); + await waitFor(() => expect(canvas.getByRole('region', { name: 'The learning loop' })).toBeInTheDocument()); + await expect(canvas.getByRole('radiogroup', { name: 'Insights time range' })).toBeInTheDocument(); + await expect(canvas.queryByRole('radiogroup', { name: 'Agent activity time range' })).toBeNull(); + }, +}; + +export const UnknownVariantFallsBackToTheOriginalPage: Story = { + decorators: [withFlagVariants({ 'insights-visual': 'bogus' })], + play: async ({ canvasElement }) => expectOriginalPage(canvasElement), +}; diff --git a/packages/web/src/components/insights/InsightsView.tsx b/packages/web/src/components/insights/InsightsView.tsx new file mode 100644 index 000000000..20f4993ec --- /dev/null +++ b/packages/web/src/components/insights/InsightsView.tsx @@ -0,0 +1,35 @@ +'use client'; + +/** + * The flag resolver for the `/insights` page body (`insights-visual`). + * + * The route imports only this file, so `page.tsx` carries no flag read and + * nothing to untangle when the rollout ends — the copy-and-suffix convention + * (`packages/feature-flags/CLAUDE.md`, live example + * `components/lore/DuplicateClustersPanel{,.on,.off}.tsx`). Each arm is a whole + * standalone component; this dispatches on the variant KEY, never the value. + * + * Retiring the flag: promote `.on`'s body here (or re-export + * `InsightsVisualPage`), delete `.off`, delete this `switch`, drop the registry + * entry. The original `InsightsPage` can go too once `InsightsPageSkeleton` + * (still imported by `page.tsx` as the Suspense fallback) has moved out of it. + * + * `default` is OFF: an unknown variant (a stale override cookie, a renamed + * variant) must land on the page every reader already knows, matching the + * registry's `defaultVariant: 'off'` so the two cannot disagree. + */ + +import { useFeatureFlagVariant } from '@/components/providers/FeatureFlagsProvider'; +import { InsightsViewOff } from './InsightsView.off'; +import { InsightsViewOn } from './InsightsView.on'; + +export function InsightsView() { + const variant = useFeatureFlagVariant('insights-visual'); + switch (variant) { + case 'on': + return ; + case 'off': + default: + return ; + } +} diff --git a/packages/web/src/components/insights/visual/InsightsVisualPage.stories.tsx b/packages/web/src/components/insights/visual/InsightsVisualPage.stories.tsx new file mode 100644 index 000000000..a9f9c8706 --- /dev/null +++ b/packages/web/src/components/insights/visual/InsightsVisualPage.stories.tsx @@ -0,0 +1,39 @@ +import type { Meta, StoryObj } from '@storybook/react'; +import { InsightsVisualPage } from './InsightsVisualPage'; +import { FROZEN_NOW } from '@/mocks/memories'; +import { withQueryClient, withFrozenClock } from '@/mocks/decorators'; +import { emptyVisualHandlers, visualHandlers } from './insights-visual.fixtures'; + +/** + * Full-page visual-regression stories for the visual `/insights` page — the + * `on` arm of `insights-visual`. Every section fetches over TanStack Query → a + * REST endpoint MSW mocks with deterministic fixtures + * (`insights-visual.fixtures.ts`), so this stories the real page. + */ +const meta: Meta = { + title: 'Pages/Insights/Visual', + component: InsightsVisualPage, + parameters: { + layout: 'fullscreen', + msw: { handlers: visualHandlers() }, + // RangePicker and RunsList need the App Router context. + nextjs: { appDirectory: true }, + }, + decorators: [withFrozenClock(FROZEN_NOW), withQueryClient], + render: () => ( +
+ +
+ ), +}; + +export default meta; +type Story = StoryObj; + +/** A busy week: writes, chosen and bulk reads, three kinds of runs, and a quieter previous week. */ +export const Default: Story = {}; + +/** An account with no activity — every section renders its empty state rather than empty axes. */ +export const Empty: Story = { + parameters: { msw: { handlers: emptyVisualHandlers() } }, +}; diff --git a/packages/web/src/components/insights/visual/InsightsVisualPage.test.stories.tsx b/packages/web/src/components/insights/visual/InsightsVisualPage.test.stories.tsx new file mode 100644 index 000000000..a32a99154 --- /dev/null +++ b/packages/web/src/components/insights/visual/InsightsVisualPage.test.stories.tsx @@ -0,0 +1,110 @@ +import type { Meta, StoryObj } from '@storybook/react'; +import { expect, userEvent, waitFor, within } from 'storybook/test'; +import { InsightsVisualPage } from './InsightsVisualPage'; +import { FROZEN_NOW } from '@/mocks/memories'; +import { withQueryClient, withFrozenClock } from '@/mocks/decorators'; +import { visualHandlers } from './insights-visual.fixtures'; + +/** + * Interaction tests for the visual `/insights` page. `/Tests` namespace, + * `test`-tagged, and `chromatic.disableSnapshot` so the visual `afterEach` + * skips these while the `play` functions still run in the browser. + */ +const meta: Meta = { + title: 'Pages/Insights/Visual/Tests', + component: InsightsVisualPage, + tags: ['test'], + parameters: { + chromatic: { disableSnapshot: true }, + layout: 'fullscreen', + msw: { handlers: visualHandlers() }, + nextjs: { appDirectory: true }, + }, + decorators: [withFrozenClock(FROZEN_NOW), withQueryClient], +}; + +export default meta; +type Story = StoryObj; + +export const RendersTheLoopAndEverySection: Story = { + play: async ({ canvasElement, step }) => { + const canvas = within(canvasElement); + await step('one page-wide window, not two', async () => { + await expect(canvas.getByRole('heading', { name: 'Insights', level: 1 })).toBeInTheDocument(); + await expect(canvas.getAllByRole('radiogroup', { name: /time range/i })).toHaveLength(1); + await expect(canvas.getByRole('radiogroup', { name: 'Insights time range' })).toBeInTheDocument(); + }); + await step('the four loop stages render in order', async () => { + const loop = await waitFor(() => canvas.getByRole('region', { name: 'The learning loop' })); + const stages = within(loop).getAllByRole('listitem').map((li) => li.textContent ?? ''); + await expect(stages).toHaveLength(4); + ['Learn', 'Recall', 'Promote', 'Prune'].forEach((name, i) => expect(stages[i]).toContain(name)); + }); + await step('every section heading renders', async () => { + for (const heading of [ + 'Pulse', + 'Recall flow', + 'Runs', + 'Is this lore earning its place?', + 'Where it is read', + 'Operational health', + ]) { + await waitFor(() => expect(canvas.getByRole('heading', { name: heading })).toBeInTheDocument()); + } + }); + await step('Prune states its window total and that it has no daily series yet', async () => { + // Fixture: 3 archived + 2 expired in the current window. + await waitFor(() => expect(canvas.getByText('3 archived · 2 expired', { exact: false })).toBeVisible()); + await expect(canvas.getByText(/not recorded per day yet/)).toBeVisible(); + }); + }, +}; + +export const RecallFlowSwapsToATable: Story = { + play: async ({ canvasElement, step }) => { + const page = within(canvasElement); + const section = await waitFor(() => { + const heading = page.getByRole('heading', { name: 'Recall flow' }); + const container = heading.closest('section'); + if (!container) throw new Error('Recall flow heading is not inside a section'); + return container; + }); + const canvas = within(section); + const toggle = await waitFor(() => canvas.getByRole('button', { name: 'View as table' })); + await step('the chart is the default, the table is one click away', async () => { + await userEvent.click(toggle); + await expect(canvas.getByRole('button', { name: 'View as chart' })).toBeInTheDocument(); + }); + await step('the table carries the exact calls the bands were drawn from', async () => { + const table = canvas.getByRole('table'); + // Fixture: mcp · memory.list · repo = 640 calls, the largest row. + const [, firstRow] = within(table).getAllByRole('row'); + if (!firstRow) throw new Error('the table rendered no data rows'); + await expect(firstRow).toHaveTextContent('mcp'); + await expect(firstRow).toHaveTextContent('memory.list'); + await expect(firstRow).toHaveTextContent('640'); + // memory.write is not a recall, so it never appears. + await expect(within(table).queryByText('memory.write')).toBeNull(); + // Successful reads only: 640 + 210 + 380 + 44 + 90 + 30 + 120 = 1,514. The fixture's + // 6 failed memory.read calls would make it 1,520 — a failed read recalled nothing. + await expect(canvas.getByText('1,514 successful lore-read calls')).toBeInTheDocument(); + // A call with no scope type is named "unattributed", not folded into "other". + await expect(within(table).getByText('unattributed')).toBeInTheDocument(); + await expect(within(table).queryByText('other')).toBeNull(); + }); + }, +}; + +export const BelowTheFoldSectionsLoadOnScroll: Story = { + play: async ({ canvasElement, step }) => { + const canvas = within(canvasElement); + const heading = await waitFor(() => canvas.getByRole('heading', { name: 'Runs' })); + await step('the runs panel mounts once its section is scrolled near the viewport', async () => { + heading.scrollIntoView(); + // Fixture: 5 local sessions, 4 CI jobs, 2 PR automations, 1 unlabelled run. + await waitFor(() => expect(canvas.getByText('Local sessions')).toBeInTheDocument()); + await expect(canvas.getByText('CI jobs')).toBeInTheDocument(); + await expect(canvas.getByText('PR automations')).toBeInTheDocument(); + }); + }, +}; diff --git a/packages/web/src/components/insights/visual/InsightsVisualPage.tsx b/packages/web/src/components/insights/visual/InsightsVisualPage.tsx new file mode 100644 index 000000000..8fbdfce34 --- /dev/null +++ b/packages/web/src/components/insights/visual/InsightsVisualPage.tsx @@ -0,0 +1,297 @@ +'use client'; + +/** + * InsightsVisualPage — the visual first version of `/insights`, rendered when + * `insights-visual` is on (see `InsightsView.tsx`). With it off the page is the + * original `InsightsPage`, untouched. + * + * ## The bill leads, then the loop, then when it moves + * + * 0. **The cost line** (`LoreCostHeadline`) stays first, per the decision + * that `/insights` leads with the bill (`docs/decisions.md`): everything + * below it is a volume, and a volume alone reads as health. + * 1. **The loop** — Learn → Recall → Promote → Prune as one strip + * (`LoopStageStrip`): is the loop turning, and where has it stalled? + * 2. **Pulse** — written, chosen and bulk on one time axis (`PulseTimeline`). + * 3. **Recall flow** — who asked, through which tool, from which scope type. + * 4. **Runs** — when work starts (weekday × hour) and its shape (swimlanes), + * with the existing audit-trail list kept below for the drill-down. + * 5. **Is this lore earning its place?** — the existing utility grid, + * unchanged. It is the one section readers interact with today (quadrant + * picks and copied groom prompts), so the visual page keeps it whole + * rather than re-drawing it. + * 6. **Where it is read** and **Operational health** — the existing scope + * leaderboard and diagnostics, now on the page's window. + * + * ## ONE window + * + * The original page carries two independent range controls. Here everything + * windowed follows one control, so the strip, the timeline, the flow, the runs + * and the diagnostics can never describe different periods. The census and the + * cost headline are the exceptions and caption their own periods ("all-time", + * "the last 30 days"), as they already do. Bounded presets only — a delta + * needs a preceding window of equal length, and "all time" has none. + * + * While a new range loads, the previous result stays on screen, dimmed and + * `aria-busy`, and every grid is drawn from the window that result was fetched + * for (`data.window`), never from the newly selected one. + * + * ## Load fan-out is bounded + * + * Above the fold the page issues the loop fetch (activity, read-activity and + * usage for the window and the one before it) and ONE `/memories/utility` + * request shared by the cost line, the strip and the grid. Runs, the run list + * and the scope leaderboard mount only when scrolled near (`Deferred`): every + * one of these hits the same database, and firing all of them at once on each + * load or range switch pushed the slowest aggregate past its statement timeout. + */ + +import { useMemo, useState } from 'react'; +import { Activity, Grid2x2, Layers, PlayCircle, Stethoscope, Workflow } from 'lucide-react'; +import { HealthSummary } from '@/components/dashboard/HealthSummary'; +import { UsageHealth } from '@/components/dashboard/UsageHealth'; +import { LoreCostHeadline } from '@/components/lore/LoreCostHeadline'; +import { LoreUtilityGrid } from '@/components/lore/LoreUtilityGrid'; +import { ScopeConsumption } from '@/components/lore/ScopeConsumption'; +import { RunsList } from '@/components/settings/RunsList'; +import { RangePicker } from '@/components/ui/RangePicker'; +import { track } from '@/lib/analytics/track'; +import { rangeTelemetry } from '@/lib/analytics/lore-events'; +import { loopStages, pulseBuckets, pulseTotals, rawTotals } from '@/lib/insights-loop'; +import { useInsightsLoop } from '@/lib/queries/insights-loop'; +import { useLoreUtilitySummary } from '@/lib/queries/lore-utility'; +import { excludeDashboardReads, failuresByToolOutcome } from '@/lib/usage-health'; +import { rangeCaption, resolveRange, type RangePreset, type TimeRange } from '@/lib/time-range'; +import { LoopStageStrip } from './LoopStageStrip'; +import { PulseTimeline } from './PulseTimeline'; +import { RecallFlowChart } from './RecallFlowChart'; +import { RunsPanel } from './RunsPanel'; +import { viewerTimeZone } from './format'; +import { useInView } from './use-in-view'; + +const PRESETS: readonly RangePreset[] = ['24h', '7d', '30d', '90d']; +const DEFAULT_RANGE: TimeRange = { preset: '7d' }; + +function Section({ + id, + icon: Icon, + title, + description, + children, +}: { + id?: string; + icon: typeof Activity; + title: string; + description: string; + children: React.ReactNode; +}) { + return ( +
+
+

+ + {title} +

+

{description}

+
+ {children} +
+ ); +} + +function Card({ children }: { children: React.ReactNode }) { + return
{children}
; +} + +function Skeleton({ className = 'h-32' }: { className?: string }) { + return ( +
+ ); +} + +function Notice({ children }: { children: React.ReactNode }) { + return ( +

+ {children} +

+ ); +} + +/** + * Mounts `children` only once the placeholder has come near the viewport, so + * below-the-fold sections do not fetch on load or on every range switch — see + * `use-in-view.ts` for why the burst matters. + */ +function Deferred({ children, placeholderClassName = 'h-40' }: { children: React.ReactNode; placeholderClassName?: string }) { + const { ref, inView } = useInView(); + if (inView) return <>{children}; + return
; +} + +const LOAD_ERROR = 'Failed to load usage data. Please refresh the page to try again.'; + +export function InsightsVisualPage() { + // Mount-stable clock: the picker's label and the resolved window describe the same instant. + const nowIso = useMemo(() => new Date().toISOString(), []); + const timeZone = useMemo(() => viewerTimeZone(), []); + const [range, setRange] = useState(DEFAULT_RANGE); + const window = useMemo( + () => resolveRange(range, nowIso) ?? resolveRange(DEFAULT_RANGE, nowIso) ?? { from: nowIso, to: nowIso }, + [range, nowIso], + ); + const caption = useMemo(() => rangeCaption(range, nowIso), [range, nowIso]); + + function commitRange(next: TimeRange) { + const { preset, spanDays } = rangeTelemetry(next, nowIso); + track({ name: 'insights.range_changed', section: 'page', preset, ...(spanDays === undefined ? {} : { spanDays }) }); + setRange(next); + } + + // Above the fold: the loop data and ONE utility request, shared with the cost + // line and the utility grid. Runs and the scope leaderboard load on scroll. + const loop = useInsightsLoop(window); + const census = useLoreUtilitySummary(); + + // Gridded from the window the rows were FETCHED for — see the module docblock. + const pulse = useMemo( + () => (loop.data ? pulseBuckets(loop.data.window, loop.data.unit, loop.data.current.writes, loop.data.current.reads) : []), + [loop.data], + ); + const stages = useMemo(() => { + if (!loop.data) return null; + const { current, previous, currentCapped, previousCapped } = loop.data; + return loopStages( + { totals: rawTotals(current.writes, current.reads), summary: current.usage.summary }, + { totals: rawTotals(previous.writes, previous.reads), summary: previous.usage.summary }, + census.data?.census ?? null, + { current: currentCapped, previous: previousCapped }, + ); + }, [loop.data, census.data]); + const updating = loop.isPlaceholderData; + + const data = loop.data; + const pulseIsEmpty = useMemo(() => { + const t = pulseTotals(pulse); + return t.written + t.chosen + t.bulk === 0; + }, [pulse]); + + const usageRows = useMemo(() => loop.data?.current.usage.by_tool ?? [], [loop.data]); + const agentRows = useMemo(() => excludeDashboardReads(usageRows), [usageRows]); + const agentPreviousRows = useMemo(() => excludeDashboardReads(loop.data?.previous.usage.by_tool ?? []), [loop.data]); + const failures = useMemo(() => failuresByToolOutcome(agentRows), [agentRows]); + + return ( +
+
+
+

Insights

+

+ How your lore is learned, recalled, promoted and pruned — and when it moves. +

+
+ +
+ + {/* The bill leads — see the module docblock. Its own 30-day window, captioned in its sentence. */} + + +
+ {loop.isLoading ? ( + + ) : loop.isError || !data || !stages ? ( + {LOAD_ERROR} + ) : ( + + )} + +
+ {loop.isLoading ? ( + + ) : loop.isError || !data ? ( + {LOAD_ERROR} + ) : pulseIsEmpty ? ( + Nothing was written or read in {caption}. + ) : ( + + + + )} +
+ +
+ {loop.isLoading ? : loop.isError ? {LOAD_ERROR} : } +
+ +
+ + + + + + + + +
+ +
+ + + +
+ +
+ + + + + +
+ +
+ {loop.isLoading ? ( + + ) : loop.isError ? ( + {LOAD_ERROR} + ) : ( + <> + + {agentRows.length > 0 && } + + )} +
+
+
+ ); +} diff --git a/packages/web/src/components/insights/visual/LoopStageStrip.tsx b/packages/web/src/components/insights/visual/LoopStageStrip.tsx new file mode 100644 index 000000000..a37ca7fdf --- /dev/null +++ b/packages/web/src/components/insights/visual/LoopStageStrip.tsx @@ -0,0 +1,167 @@ +'use client'; + +/** + * LoopStageStrip — the header of the visual Insights page: the four stages of + * the loop LoreKit runs (Learn → Recall → Promote → Prune), each as a figure, a + * period delta where one exists, and a sparkbar where a bucketed series exists. + * + * Every figure's source and its limits are in `lib/insights-loop.ts`. Two of + * them are shown as limits rather than papered over: + * + * - **Promote** is all-time (the census reads lifetime counters), so it has + * no delta chip and says "all-time" in place of the window caption. + * - **Prune** has window totals but no bucketed series yet, so it draws no + * bars and says so — a flat empty sparkbar would read as "nothing pruned + * on any day", which is a claim the data does not make. + * + * Read left to right, the strip answers "is the loop turning": lessons are + * being learned, recalled, promoted and pruned — or one stage has stalled. + */ + +import { ArrowRight, BookOpenCheck, Brain, Scissors, TrendingUp } from 'lucide-react'; +import { AnimatedNumber } from '@/components/ui/AnimatedNumber'; +import { TrendChip } from '@/components/dashboard/StatCard'; +import { Sparkbar, type SparkPoint } from '@/components/dashboard/Sparkbar'; +import { chosenShare, type LoopStages, type PulseBucket } from '@/lib/insights-loop'; +import { formatShare } from '@/lib/lore-cost'; +import type { BucketUnit } from '@/lib/time-range'; +import { formatBucketLabel } from './format'; + +interface LoopStageStripProps { + stages: LoopStages; + /** The census request failed — Promote says so instead of "loading" forever. */ + censusFailed?: boolean; + pulse: readonly PulseBucket[]; + unit: BucketUnit; + /** e.g. "the last 7 days". */ + rangeCaption: string; + timeZone: string; +} + +function Stage({ + icon: Icon, + name, + verb, + figure, + unitLabel, + changePct, + caption, + children, +}: { + icon: typeof Brain; + name: string; + verb: string; + figure: number | null; + unitLabel: string; + changePct?: number | null; + caption: string; + children?: React.ReactNode; +}) { + return ( +
  • +
    +

    + + {name} +

    + {changePct !== undefined && changePct !== null && ( + // Neutral: more learning, recall or pruning is not good or bad on its own. + + )} +
    +

    + + {figure === null ? '—' : } + + {unitLabel} +

    +

    + {verb} · {caption} +

    +
    {children}
    +
  • + ); +} + +export function LoopStageStrip({ stages, censusFailed = false, pulse, unit, rangeCaption, timeZone }: LoopStageStripProps) { + const label = (start: string) => formatBucketLabel(start, unit, timeZone); + const writePoints: SparkPoint[] = pulse.map((b) => ({ label: label(b.start), value: b.written })); + const readPoints: SparkPoint[] = pulse.map((b) => ({ label: label(b.start), value: b.chosen + b.bulk })); + const share = formatShare(chosenShare(stages)); + // Learn and Recall come from a series that hit the row cap: say they are lower bounds. + const incomplete = stages.seriesIncomplete ? ( +

    At least this many — the series hit the 1,000-row cap.

    + ) : null; + const promoteVerb = stages.promote + ? `${stages.promote.specialist} specialist${stages.promote.specialist === 1 ? '' : 's'} could reach further` + : censusFailed + ? 'Census unavailable — refresh to retry' + : 'Loading census'; + + return ( +
    +
      + + {incomplete} + + + + + {incomplete} + + + + + + Candidates for a permanent rule + + + + + +

      + Window total only — archives and expiries are not recorded per day yet. +

      +
      +
    +
    + ); +} diff --git a/packages/web/src/components/insights/visual/PulseTimeline.tsx b/packages/web/src/components/insights/visual/PulseTimeline.tsx new file mode 100644 index 000000000..6899143d6 --- /dev/null +++ b/packages/web/src/components/insights/visual/PulseTimeline.tsx @@ -0,0 +1,159 @@ +'use client'; + +/** + * PulseTimeline — when memory moves: lessons written, records chosen on + * purpose, and records delivered in bulk, as three rows on ONE shared time + * axis with one cursor across all three. + * + * ## Small multiples, not a stacked or mirrored bar + * + * Bulk deliveries outnumber deliberate fetches by orders of magnitude, so a + * stacked read bar would draw "chosen" as an invisible sliver, and mirroring + * writes against reads would put two very different scales on one axis — a + * dual axis in disguise. Three rows, each with its OWN scale and its maximum + * printed beside it, keep every series readable and every scale stated. The + * shared column is what lets a reader compare the three at one moment. + * + * Colour is never the only encoding: each row is also named and positioned. + * + * ## One cursor, one readout + * + * A column spans all three rows, so hovering (or arrowing to) a bucket shows + * every figure for that bucket in one portaled readout — the same contract as + * `Sparkbar` (see `use-chart-cursor.ts`). + */ + +import { useId } from 'react'; +import { AlertTriangle } from 'lucide-react'; +import { AnchoredTooltip } from '@/components/ui/AnchoredTooltip'; +import { pulseTotals, type PulseBucket } from '@/lib/insights-loop'; +import type { BucketUnit } from '@/lib/time-range'; +import { formatBucketLabel, formatCompact } from './format'; +import { useChartCursor } from './use-chart-cursor'; + +interface PulseTimelineProps { + buckets: readonly PulseBucket[]; + /** e.g. "the last 7 days" — the totals in the text alternative are for the whole window. */ + rangeCaption: string; + unit: BucketUnit; + timeZone: string; + /** The series reached the row cap — newest buckets may be missing. */ + capped: boolean; +} + +const ROWS = [ + { key: 'written', label: 'Written', hint: 'new lessons', color: 'bg-[var(--color-accent)]', height: 'h-12' }, + { key: 'chosen', label: 'Chosen', hint: 'records fetched on purpose', color: 'bg-[var(--color-success)]', height: 'h-12' }, + { key: 'bulk', label: 'Bulk', hint: 'records riding along in list/search', color: 'bg-[var(--color-content-tertiary)]', height: 'h-16' }, +] as const; + +type RowKey = (typeof ROWS)[number]['key']; + +export function PulseTimeline({ buckets, rangeCaption, unit, timeZone, capped }: PulseTimelineProps) { + const cursor = useChartCursor(buckets.length); + const readoutId = `pulse-readout-${useId().replace(/:/g, '')}`; + const totals = pulseTotals(buckets); + const max: Record = { + written: Math.max(1, ...buckets.map((b) => b.written)), + chosen: Math.max(1, ...buckets.map((b) => b.chosen)), + bulk: Math.max(1, ...buckets.map((b) => b.bulk)), + }; + const label = (b: PulseBucket) => formatBucketLabel(b.start, unit, timeZone); + const activeBucket = cursor.active === null ? null : buckets[cursor.active]; + const tickIndexes = buckets.length > 2 ? [0, Math.floor((buckets.length - 1) / 2), buckets.length - 1] : buckets.map((_, i) => i); + + if (buckets.length === 0) return null; + + return ( +
    + {capped && ( +

    + + This window returned the server's 1,000-row maximum, so the newest buckets may be missing. Pick a + shorter range for a complete picture. +

    + )} + +
    + {/* Row labels — the series are named by position, not only by colour. */} +
    + {ROWS.map((row) => ( +
    + {row.label} + max {formatCompact(max[row.key])} +
    + ))} +
    + +
    + {buckets.map((b, i) => { + const isActive = cursor.active === i; + return ( +
    + {ROWS.map((row) => { + const value = b[row.key]; + const pct = value > 0 ? Math.max((value / max[row.key]) * 100, 8) : 0; + return ( +
    + {value === 0 && } +
    +
    + ); + })} +
    + ); + })} +
    +
    + + {/* Three ticks — first, middle, last — each centred under ITS bar, so an even + bucket count does not put the middle tick on the gap between two bars. */} +
    + {tickIndexes.map((i, n) => ( + + {buckets[i] ? label(buckets[i]) : ''} + + ))} +
    + + + {activeBucket && ( + + {label(activeBucket)} + {ROWS.map((row) => ( + + + {activeBucket[row.key].toLocaleString('en-US')} + {' '} + {row.hint} + + ))} + + )} + +
    + ); +} diff --git a/packages/web/src/components/insights/visual/RecallFlowChart.tsx b/packages/web/src/components/insights/visual/RecallFlowChart.tsx new file mode 100644 index 000000000..9f9cdbeaa --- /dev/null +++ b/packages/web/src/components/insights/visual/RecallFlowChart.tsx @@ -0,0 +1,292 @@ +'use client'; + +/** + * RecallFlowChart — who recalled lore, through which read tool, from which + * kind of scope, as a three-column flow (`lib/recall-flow.ts` builds the + * nodes, `lib/flow-layout.ts` lays them out). + * + * Hovering or focusing a node highlights every band touching it and states its + * figure in a live line under the chart; "View as table" swaps the drawing for + * the exact rows it was built from — the accessible equivalent a flow chart + * needs, and the precise numbers a reader may want. + * + * ## Labels live in gutters + * + * The outer columns' labels sit in a gutter OUTSIDE their nodes (left of the + * left column, right of the right column), never on top of the bands, where + * their counts used to disappear into the band colour. The middle column has + * bands on both sides, so its labels carry a halo in the card's background + * colour instead. + * + * ## Two colour families, one per half + * + * Client → tool bands take their CLIENT's colour (warm and mixed hues); tool → + * scope bands take their TOOL's colour (a blue family no client uses). Sharing + * one palette across both halves would paint a client and a tool the same hue, + * inviting exactly the misreading the caption warns against — that a band + * leaving a tool is the same calls that entered it from that client. + */ + +import { useEffect, useMemo, useState } from 'react'; +import { Table2, Workflow } from 'lucide-react'; +import { track } from '@/lib/analytics/track'; +import { layoutFlow } from '@/lib/flow-layout'; +import { OTHER, RECALL_FLOW_COLUMNS, UNATTRIBUTED, recallFlow } from '@/lib/recall-flow'; +import type { UsageStatRow } from '@lorekit/schemas/usage'; +import { formatCompact } from './format'; + +/** Fallback before the first measurement, and the floor below which labels would collide. */ +const DEFAULT_WIDTH = 720; +const MIN_WIDTH = 320; +const HEIGHT = 240; +const HEADER_HEIGHT = 20; +const NODE_WIDTH = 10; +const LABEL_GAP = 6; +/** Approximate advance of one 11px character — enough to truncate a label before it leaves its gutter. */ +const CHAR_WIDTH = 6.4; + +/** Named clients, in node order. All token-derived; none is blue, which is the tools' family. */ +const CLIENT_COLORS = [ + 'var(--color-accent)', + 'var(--color-success)', + 'var(--color-scope-global)', + 'color-mix(in oklab, var(--color-error) 70%, var(--color-scope-global))', +]; +/** The four lore-read tools, in node order: shades of one blue, distinct from every client hue. */ +const TOOL_COLORS = [ + 'var(--color-info)', + 'color-mix(in oklab, var(--color-info) 55%, white)', + 'color-mix(in oklab, var(--color-info) 60%, black)', + 'color-mix(in oklab, var(--color-info) 55%, var(--color-content-tertiary))', +]; +const UNATTRIBUTED_COLOR = 'var(--color-content-secondary)'; +const OTHER_COLOR = 'var(--color-content-tertiary)'; +const SCOPE_NODE_COLOR = 'var(--color-content-secondary)'; + +/** + * The chart's own pixel width, tracked with a ResizeObserver. + * + * Laid out at the real width rather than scaled through a fixed viewBox: a + * scaled SVG shrinks its 11px labels to ~5px on a phone, which is the + * "chart that ignores the canvas" failure. Measuring keeps text at its size. + * + * A callback ref (element in state), not a ref object: the measured element + * unmounts while the table is shown, and an observer attached once on mount + * would keep watching the detached node. + */ +function useMeasuredWidth() { + const [element, setElement] = useState(null); + const [width, setWidth] = useState(DEFAULT_WIDTH); + useEffect(() => { + if (!element || typeof ResizeObserver === 'undefined') return; + const observer = new ResizeObserver(([entry]) => { + if (entry) setWidth(Math.max(MIN_WIDTH, Math.round(entry.contentRect.width))); + }); + observer.observe(element); + return () => observer.disconnect(); + }, [element]); + return { ref: setElement, width }; +} + +/** Gutter for the outer labels: a sixth of the width, within readable bounds. */ +function gutterFor(width: number): number { + return Math.min(132, Math.max(76, Math.round(width * 0.16))); +} + +/** Truncate `label` so `label value` fits in `maxPx`, keeping the value whole. */ +function fitLabel(label: string, value: string, maxPx: number): string { + const maxChars = Math.floor(maxPx / CHAR_WIDTH); + if (label.length + 1 + value.length <= maxChars) return label; + return `${label.slice(0, Math.max(1, maxChars - value.length - 2))}…`; +} + +/** Colour per node id for one column: named values from `palette` in order, catch-alls neutral. */ +function columnColors(ids: readonly string[], palette: readonly string[]): Map { + let next = 0; + return new Map( + ids.map((id) => { + const key = id.slice(id.indexOf(':') + 1); + if (key === OTHER) return [id, OTHER_COLOR]; + if (key === UNATTRIBUTED) return [id, UNATTRIBUTED_COLOR]; + return [id, palette[next++] ?? OTHER_COLOR]; + }), + ); +} + +interface RecallFlowChartProps { + rows: readonly UsageStatRow[]; + rangeCaption: string; +} + +export function RecallFlowChart({ rows, rangeCaption }: RecallFlowChartProps) { + const flow = useMemo(() => recallFlow(rows), [rows]); + const { ref: measureRef, width } = useMeasuredWidth(); + const gutter = gutterFor(width); + const innerWidth = width - 2 * gutter; + const layout = useMemo( + () => layoutFlow(flow.nodes, flow.links, { width: innerWidth, height: HEIGHT, nodeWidth: NODE_WIDTH, nodePadding: 12 }), + [flow, innerWidth], + ); + const [focused, setFocused] = useState(null); + const [asTable, setAsTable] = useState(false); + + if (flow.totalCalls === 0) { + return ( +

    + No successful lore reads in {rangeCaption}. +

    + ); + } + + const clientIds = layout.nodes.filter((n) => n.layer === 0).map((n) => n.id); + const toolIds = layout.nodes.filter((n) => n.layer === 1).map((n) => n.id); + const nodeColor = new Map([...columnColors(clientIds, CLIENT_COLORS), ...columnColors(toolIds, TOOL_COLORS)]); + // Every band takes its SOURCE node's colour: a client's on the left, a tool's on the right. + const bandColor = (source: string) => nodeColor.get(source) ?? OTHER_COLOR; + const touches = (l: { source: string; target: string }) => focused !== null && (l.source === focused || l.target === focused); + const focusedNode = layout.nodes.find((n) => n.id === focused) ?? null; + const middleX = (innerWidth - NODE_WIDTH) / 2; + + return ( +
    +
    + + {flow.totalCalls.toLocaleString('en-US')} successful lore-read calls + + +
    + + {asTable ? ( +
    + + + + + + + + + + + + {flow.rows.map((r) => ( + + + + + + + ))} + +
    Lore-read calls by client, tool and scope type, {rangeCaption}
    Who askedRead toolScope typeCalls
    {r.client}{r.tool}{r.scopeType}{r.calls.toLocaleString('en-US')}
    +
    + ) : ( +
    + + {/* Column names, each aligned to its node column. */} + + + {RECALL_FLOW_COLUMNS[0]} + + + {RECALL_FLOW_COLUMNS[1]} + + + {RECALL_FLOW_COLUMNS[2]} + + + + + + {layout.links.map((l) => ( + ${l.target}`} + d={l.path} + style={{ stroke: bandColor(l.source) }} + strokeWidth={Math.max(1, l.width)} + strokeOpacity={focused === null ? 0.4 : touches(l) ? 0.8 : 0.08} + className="transition-[stroke-opacity] duration-150" + /> + ))} + + {layout.nodes.map((n) => { + const value = formatCompact(n.value); + // Outer columns label into their gutter; the middle one labels over the bands, haloed. + const side = n.layer === 0 ? 'left' : n.layer === 2 ? 'right' : 'middle'; + const x = side === 'left' ? n.x0 - LABEL_GAP : n.x1 + LABEL_GAP; + // Middle labels start at middleX + NODE_WIDTH + LABEL_GAP and must stop a gap before the right nodes at 2 * middleX. + const maxPx = side === 'middle' ? middleX - NODE_WIDTH - LABEL_GAP * 2 : gutter - LABEL_GAP * 2; + const label = fitLabel(n.label, value, maxPx); + return ( + setFocused(n.id)} + onPointerLeave={() => setFocused((f) => (f === n.id ? null : f))} + onFocus={() => setFocused(n.id)} + onBlur={() => setFocused((f) => (f === n.id ? null : f))} + className="cursor-default outline-none [&:focus-visible>rect]:stroke-[var(--color-accent)]" + > + + + {label} + {value} + + + ); + })} + + +
    + )} + +

    + {focusedNode && !asTable + ? `${focusedNode.label}: ${focusedNode.value.toLocaleString('en-US')} of ${flow.totalCalls.toLocaleString('en-US')} lore-read calls` + : `Calls, ${rangeCaption}. Left bands are coloured by who asked, right bands by read tool — each half is its own pairing, so a band into a tool and a band out of it are not the same calls continuing.`} +

    +
    + ); +} diff --git a/packages/web/src/components/insights/visual/RunsPanel.tsx b/packages/web/src/components/insights/visual/RunsPanel.tsx new file mode 100644 index 000000000..b54d6d9a0 --- /dev/null +++ b/packages/web/src/components/insights/visual/RunsPanel.tsx @@ -0,0 +1,58 @@ +'use client'; + +/** + * RunsPanel — the runs rhythm and swimlanes, with their own paginated fetch. + * + * Its own component so the page can mount it only once the Runs section is + * near the viewport (`use-in-view.ts`): `useWindowRuns` reads up to five pages, + * and none of it is visible on load. + */ + +import { useWindowRuns, MAX_RUN_PAGES } from '@/lib/queries/insights-loop'; +import type { AbsoluteRange } from '@/lib/time-range'; +import { RunsRhythm } from './RunsRhythm'; +import { RunsSwimlanes } from './RunsSwimlanes'; + +interface RunsPanelProps { + window: AbsoluteRange; + rangeCaption: string; + timeZone: string; +} + +const NOTICE = + 'rounded-xl border border-[var(--color-border)] bg-[var(--color-bg-raised)] p-4 text-xs text-[var(--color-content-tertiary)]'; + +export function RunsPanel({ window, rangeCaption, timeZone }: RunsPanelProps) { + const runs = useWindowRuns(window); + const data = runs.data; + + if (runs.isLoading) { + return ( +
    + ); + } + if (runs.isError) return

    Failed to load runs. Please refresh the page to try again.

    ; + if (!data || data.runs.length === 0) return

    No runs recorded in {rangeCaption}.

    ; + + return ( +
    +
    + + {/* Drawn on the window these runs were fetched for, not the newly selected one. */} + +
    + {data.truncated && ( +

    + Showing the {data.runs.length.toLocaleString('en-US')} most recent runs ({MAX_RUN_PAGES} pages), so the start + of the window may look emptier than it was — pick a shorter range to see them all. +

    + )} +
    + ); +} diff --git a/packages/web/src/components/insights/visual/RunsRhythm.tsx b/packages/web/src/components/insights/visual/RunsRhythm.tsx new file mode 100644 index 000000000..e927d159b --- /dev/null +++ b/packages/web/src/components/insights/visual/RunsRhythm.tsx @@ -0,0 +1,129 @@ +'use client'; + +/** + * RunsRhythm — when your agents start work with lore: runs started per local + * weekday × hour (`lib/runs-rhythm.ts#rhythmGrid`). + * + * Built from runs rather than the hourly activity series for the reason in + * that module's docblock (the hourly series would hit the row cap silently); + * the caption states the coverage that choice costs — calls without a run id + * are not placed. + * + * One tab stop; arrow keys move the cell in two dimensions, Home/End jump to + * the row's ends, Escape clears. The readout is one live line beneath the grid + * rather than a floating panel, because a 7 × 24 grid of 168 small cells has + * no edge a floating panel could avoid covering. + */ + +import { useState, type KeyboardEvent } from 'react'; +import { WEEKDAYS, rhythmGrid } from '@/lib/runs-rhythm'; +import type { UsageRun } from '@lorekit/schemas/usage'; + +interface RunsRhythmProps { + runs: readonly UsageRun[]; + timeZone: string; +} + +const HOUR_TICKS = [0, 6, 12, 18]; + +function hourLabel(hour: number): string { + return `${String(hour).padStart(2, '0')}:00`; +} + +export function RunsRhythm({ runs, timeZone }: RunsRhythmProps) { + const grid = rhythmGrid(runs, timeZone); + const [cell, setCell] = useState<{ d: number; h: number } | null>(null); + + function onKeyDown(e: KeyboardEvent) { + if (e.key === 'Escape') { + setCell(null); + return; + } + const current = cell ?? { d: 0, h: 9 }; + const moves: Record = { + ArrowRight: { d: current.d, h: Math.min(23, current.h + 1) }, + ArrowLeft: { d: current.d, h: Math.max(0, current.h - 1) }, + ArrowDown: { d: Math.min(6, current.d + 1), h: current.h }, + ArrowUp: { d: Math.max(0, current.d - 1), h: current.h }, + Home: { d: current.d, h: 0 }, + End: { d: current.d, h: 23 }, + }; + const next = moves[e.key]; + if (!next) return; + e.preventDefault(); + setCell(cell === null ? current : next); + } + + // The busiest cell, for the summary a screen reader hears first. + let peak: { d: number; h: number; n: number } | null = null; + for (const [d, row] of grid.cells.entries()) { + for (const [h, n] of row.entries()) { + if (n > 0 && (peak === null || n > peak.n)) peak = { d, h, n }; + } + } + const activeCount = cell ? (grid.cells[cell.d]?.[cell.h] ?? 0) : null; + + return ( +
    +
    +
    + {WEEKDAYS.map((d) => ( + + {d} + + ))} +
    +
    setCell(null)} + aria-label={ + peak + ? `${grid.total} runs. Busiest: ${WEEKDAYS[peak.d]} at ${hourLabel(peak.h)} with ${peak.n}. Use the arrow keys to read each hour.` + : 'No runs in this window.' + } + className="grid min-w-0 flex-1 grid-cols-[repeat(24,minmax(0,1fr))] grid-rows-7 gap-[2px] rounded-sm focus-visible:outline-none focus-visible:ring-1 focus-visible:ring-[var(--color-accent)]" + > + {grid.cells.map((row, d) => + row.map((n, h) => { + const isActive = cell?.d === d && cell?.h === h; + // Opacity floor so a single run is visible against an empty cell. + const opacity = n === 0 ? 0 : 0.2 + 0.8 * (n / Math.max(1, grid.max)); + return ( +
    { + if (e.pointerType !== 'touch') setCell({ d, h }); + }} + onPointerLeave={(e) => { + if (e.pointerType !== 'touch') setCell((c) => (c?.d === d && c?.h === h ? null : c)); + }} + onClick={() => setCell((c) => (c?.d === d && c?.h === h ? null : { d, h }))} + > +
    +
    + ); + }), + )} +
    +
    +
    + {HOUR_TICKS.map((h) => ( + {hourLabel(h)} + ))} +
    +

    + Fewer + + More — up to {grid.max} run{grid.max === 1 ? '' : 's'} in one hour +

    +

    + {cell && activeCount !== null + ? `${WEEKDAYS[cell.d]} ${hourLabel(cell.h)}–${hourLabel((cell.h + 1) % 24)}: ${activeCount} run${activeCount === 1 ? '' : 's'} started` + : `Runs started, by your local weekday and hour (${timeZone}).`} +

    +
    + ); +} diff --git a/packages/web/src/components/insights/visual/RunsSwimlanes.tsx b/packages/web/src/components/insights/visual/RunsSwimlanes.tsx new file mode 100644 index 000000000..b2fa4e2ca --- /dev/null +++ b/packages/web/src/components/insights/visual/RunsSwimlanes.tsx @@ -0,0 +1,140 @@ +'use client'; + +/** + * RunsSwimlanes — every run in the window on one time axis, one lane per + * session kind (local session, CI job, PR automation), from + * `lib/runs-rhythm.ts#swimlanes`. + * + * Bar LENGTH is the run's duration (first to last call); bar THICKNESS is how + * many records it read, on a capped scale so one huge CI job does not flatten + * every other run into a hairline. A run shorter than a pixel still gets a + * visible minimum width — "it happened" is the information there. + * + * Overlapping runs stack into sub-rows (`packRows`), so 26 concurrent local + * sessions read as 26 bars rather than one thick line; a lane is as tall as the + * rows it needs, capped at `MAX_LANE_ROWS`. + * + * Same interaction contract as the other charts (`use-chart-cursor.ts`): one + * tab stop walks every run in time order across lanes, and one portaled + * readout names it. The audit-trail list below this chart keeps the drill-down + * into what a run did; this view is for seeing the shape of the work. + */ + +import { useId } from 'react'; +import { AnchoredTooltip } from '@/components/ui/AnchoredTooltip'; +import { RUN_LANE_LABEL, swimlanes } from '@/lib/runs-rhythm'; +import type { AbsoluteRange } from '@/lib/time-range'; +import type { UsageRun } from '@lorekit/schemas/usage'; +import { useChartCursor } from './use-chart-cursor'; + +interface RunsSwimlanesProps { + runs: readonly UsageRun[]; + window: AbsoluteRange; + timeZone: string; +} + +/** One sub-row's height in px; thickness is scaled inside it so stacked bars never touch. */ +const ROW_HEIGHT = 14; +const LANE_PADDING = 6; +const MIN_THICKNESS = 3; +const MAX_THICKNESS = ROW_HEIGHT - 4; +const MIN_BAR_WIDTH_PX = 6; + +function laneHeight(rows: number): number { + return rows * ROW_HEIGHT + LANE_PADDING; +} + +export function RunsSwimlanes({ runs, window, timeZone }: RunsSwimlanesProps) { + const layout = swimlanes(runs, window); + // Time order across lanes, so arrow keys move through the day, not lane by lane. + const ordered = layout.lanes.flatMap((l) => l.runs).sort((a, b) => a.x0 - b.x0); + const indexOf = new Map(ordered.map((r, i) => [r.run.correlation_id + r.run.first_seen, i])); + const cursor = useChartCursor(ordered.length); + const readoutId = `swimlane-readout-${useId().replace(/:/g, '')}`; + const active = cursor.active === null ? null : ordered[cursor.active]; + + const time = new Intl.DateTimeFormat('en-GB', { weekday: 'short', day: 'numeric', month: 'short', hour: '2-digit', minute: '2-digit', hourCycle: 'h23', timeZone }); + const thickness = (records: number) => + layout.maxRecordsRead === 0 + ? MIN_THICKNESS + : MIN_THICKNESS + (MAX_THICKNESS - MIN_THICKNESS) * Math.sqrt(records / layout.maxRecordsRead); + + if (ordered.length === 0) return null; + + return ( +
    +
    +
    + {layout.lanes.map((l) => ( + + {RUN_LANE_LABEL[l.lane]} + {l.runs.length} + + ))} +
    +
    + {layout.lanes.map((l) => ( +
    + {l.runs.map((r) => { + const i = indexOf.get(r.run.correlation_id + r.run.first_seen) ?? -1; + const h = thickness(r.run.records_read); + const isActive = cursor.active === i; + return ( +
    + ); + })} +
    + ))} +
    +
    +
    + {time.format(new Date(window.from))} + {time.format(new Date(window.to))} +
    +

    + Length = first to last call. Thickness = records read (square-root scale, thickest{' '} + {layout.maxRecordsRead.toLocaleString('en-US')}). +

    + + + {active && ( + + {active.run.correlation_id} + + {RUN_LANE_LABEL[active.lane]} · {time.format(new Date(active.run.first_seen))} →{' '} + {time.format(new Date(active.run.last_seen))} + + + {active.run.records_read.toLocaleString('en-US')} records read · {active.run.write_events.toLocaleString('en-US')}{' '} + writes · {active.run.distinct_scopes} scope{active.run.distinct_scopes === 1 ? '' : 's'} + + + )} + +
    + ); +} diff --git a/packages/web/src/components/insights/visual/__screenshots__/InsightsVisualPage.stories.tsx/Default-1-chromium-linux.png b/packages/web/src/components/insights/visual/__screenshots__/InsightsVisualPage.stories.tsx/Default-1-chromium-linux.png new file mode 100644 index 000000000..168370da5 Binary files /dev/null and b/packages/web/src/components/insights/visual/__screenshots__/InsightsVisualPage.stories.tsx/Default-1-chromium-linux.png differ diff --git a/packages/web/src/components/insights/visual/__screenshots__/InsightsVisualPage.stories.tsx/Empty-1-chromium-linux.png b/packages/web/src/components/insights/visual/__screenshots__/InsightsVisualPage.stories.tsx/Empty-1-chromium-linux.png new file mode 100644 index 000000000..b532cb368 Binary files /dev/null and b/packages/web/src/components/insights/visual/__screenshots__/InsightsVisualPage.stories.tsx/Empty-1-chromium-linux.png differ diff --git a/packages/web/src/components/insights/visual/format.ts b/packages/web/src/components/insights/visual/format.ts new file mode 100644 index 000000000..ca6e08112 --- /dev/null +++ b/packages/web/src/components/insights/visual/format.ts @@ -0,0 +1,44 @@ +/** + * Label formatting shared by the visual Insights charts. + * + * Day buckets are UTC days on the server (`date_trunc` on UTC), so they are + * labelled in UTC: labelling the 00:00Z bucket in a US zone would call it the + * previous day. Hour buckets are instants, so they are labelled in the + * viewer's zone, where "14:00" means their afternoon. + */ + +import type { BucketUnit } from '@/lib/time-range'; + +const dayFormat = new Intl.DateTimeFormat('en-GB', { weekday: 'short', day: 'numeric', month: 'short', timeZone: 'UTC' }); +const hourFormats = new Map(); + +function hourFormat(timeZone: string): Intl.DateTimeFormat { + let formatter = hourFormats.get(timeZone); + if (!formatter) { + formatter = new Intl.DateTimeFormat('en-GB', { weekday: 'short', hour: '2-digit', minute: '2-digit', hourCycle: 'h23', timeZone }); + hourFormats.set(timeZone, formatter); + } + return formatter; +} + +export function formatBucketLabel(start: string, unit: BucketUnit, timeZone: string): string { + const date = new Date(start); + if (Number.isNaN(date.getTime())) return start; + return unit === 'day' ? dayFormat.format(date) : hourFormat(timeZone).format(date); +} + +const compact = new Intl.NumberFormat('en-US', { notation: 'compact', maximumFractionDigits: 1 }); + +/** `840`, `1.2K`, `18K` — for tick labels and readouts where width matters. */ +export function formatCompact(n: number): string { + return n < 1000 ? String(n) : compact.format(n); +} + +/** The viewer's IANA time zone, falling back to UTC where the runtime has none. */ +export function viewerTimeZone(): string { + try { + return Intl.DateTimeFormat().resolvedOptions().timeZone || 'UTC'; + } catch { + return 'UTC'; + } +} diff --git a/packages/web/src/components/insights/visual/insights-visual.fixtures.ts b/packages/web/src/components/insights/visual/insights-visual.fixtures.ts new file mode 100644 index 000000000..b7b112d5e --- /dev/null +++ b/packages/web/src/components/insights/visual/insights-visual.fixtures.ts @@ -0,0 +1,189 @@ +/** + * Deterministic MSW handlers for the visual Insights page stories. + * + * The shared `memoryHandlers()` read-activity fixture carries no `read_kind`, + * which the Pulse timeline (correctly) counts as bulk — so the "Chosen" row + * would always be empty. These handlers return typed reads, writes and runs + * derived from the bucket's day only (never a clock or a PRNG), and make the + * PREVIOUS window quieter than the current one so the strip's deltas render. + */ + +import { http, HttpResponse } from 'msw'; +import type { UsageRun, UsageStatRow } from '@lorekit/schemas/usage'; +import { FROZEN_NOW, memoryHandlers, utilityHandlers } from '@/mocks/memories'; +import { DAY_MS, HOUR_MS } from '@/lib/time-range'; +const FROZEN = Date.parse(FROZEN_NOW); +/** Requests whose window starts before this are the "previous" window of the default 7d range. */ +const CURRENT_FROM = FROZEN - 7 * DAY_MS; + +const WRITES = [2, 0, 3, 1, 4, 0, 2, 1]; +const BULK = [120, 80, 210, 150, 260, 40, 90, 180]; +const CHOSEN = [2, 0, 4, 1, 6, 0, 1, 3]; +const SCOPES = ['repo::mthines/lorekit', 'global', 'branch::mthines/lorekit::feat/insights'] as const; + +function bucketsIn(since: string | null, until: string | null, unit: string | null): number[] { + const step = unit === 'hour' ? HOUR_MS : DAY_MS; + const from = Math.floor(Date.parse(since ?? new Date(FROZEN - 7 * DAY_MS).toISOString()) / step) * step; + const to = Date.parse(until ?? FROZEN_NOW); + const out: number[] = []; + for (let t = from; t < to; t += step) out.push(t); + return out; +} + +/** Scales a window down to 60% when it is the previous window, so deltas are positive. */ +function scaleFor(since: string | null): number { + return since !== null && Date.parse(since) < CURRENT_FROM - HOUR_MS ? 0.6 : 1; +} + +function row(overrides: Partial): UsageStatRow { + return { + tool_name: 'memory.list', + outcome: 'ok', + scope_type: 'repo', + client: 'mcp', + kind: 'lesson', + host: 'claude', + event_count: 1, + record_count: 1, + total_duration_ms: 100, + ...overrides, + }; +} + +export const VISUAL_BY_TOOL: UsageStatRow[] = [ + row({ client: 'mcp', tool_name: 'memory.list', scope_type: 'repo', event_count: 640, record_count: 9_200, total_duration_ms: 96_000 }), + row({ client: 'mcp', tool_name: 'memory.search', scope_type: 'global', event_count: 210, record_count: 2_400, total_duration_ms: 63_000 }), + row({ client: 'cli', tool_name: 'memory.list', scope_type: 'branch', event_count: 380, record_count: 1_100, total_duration_ms: 41_000, host: 'aw' }), + row({ client: 'cli', tool_name: 'memory.read', scope_type: 'repo', event_count: 44, record_count: 44, total_duration_ms: 3_900, host: 'aw' }), + row({ client: 'dashboard', tool_name: 'memory.read', scope_type: 'project', event_count: 90, record_count: 90, total_duration_ms: 8_100, host: 'dashboard' }), + row({ client: 'api', tool_name: 'memory.search', scope_type: 'repo', event_count: 30, record_count: 300, total_duration_ms: 9_000 }), + // memory.search carries a scopes[] array, so no single scope type is recorded — the "unattributed" bucket. + row({ client: 'mcp', tool_name: 'memory.search', scope_type: null, event_count: 120, record_count: 1_500, total_duration_ms: 30_000 }), + row({ client: 'mcp', tool_name: 'memory.write', scope_type: 'repo', event_count: 26, record_count: 26, total_duration_ms: 5_200 }), + row({ client: 'mcp', tool_name: 'memory.read', outcome: 'error', scope_type: 'branch', event_count: 6, record_count: 0, total_duration_ms: 600 }), +]; + +function at(daysAgo: number, hourUtc: number, minutes = 0): string { + const midnight = Math.floor(FROZEN / DAY_MS) * DAY_MS; + return new Date(midnight - daysAgo * DAY_MS + hourUtc * HOUR_MS + minutes * 60_000).toISOString(); +} + +function run(id: string, kind: string | null, daysAgo: number, hour: number, minutes: number, records: number, writes = 0): UsageRun { + const first = at(daysAgo, hour); + return { + correlation_id: id, + session_kind: kind, + first_seen: first, + last_seen: new Date(Date.parse(first) + minutes * 60_000).toISOString(), + read_events: Math.max(1, Math.round(records / 20)), + records_read: records, + write_events: writes, + distinct_scopes: 1 + (records % 3), + total_duration_ms: minutes * 600, + }; +} + +export const VISUAL_RUNS: UsageRun[] = [ + run('session:3f2a', 'local', 6, 9, 150, 420, 2), + run('session:7c1d', 'local', 5, 13, 90, 260, 1), + run('session:a91e', 'local', 4, 9, 200, 610, 3), + run('session:c044', 'local', 2, 10, 60, 180), + run('session:e5b7', 'local', 1, 14, 120, 300, 1), + run('ci:build-1182', 'ci', 6, 2, 12, 90), + run('ci:build-1190', 'ci', 4, 2, 14, 95), + run('ci:build-1201', 'ci', 2, 2, 11, 88), + run('ci:build-1214', 'ci', 0, 2, 13, 92), + run('pr:mthines/lorekit#712', 'pr', 3, 11, 40, 340, 2), + run('pr:mthines/lorekit#715', 'pr', 1, 9, 35, 290, 1), + run('hook:unlabelled', null, 0, 8, 20, 60), +]; + +export function visualHandlers() { + // MSW resolves in list order — every override precedes `...memoryHandlers()`. + return [ + http.get('*/functions/v1/memories/activity', ({ request }) => { + const url = new URL(request.url); + const since = url.searchParams.get('since'); + const unit = url.searchParams.get('bucket'); + const scale = scaleFor(since); + const buckets = bucketsIn(since, url.searchParams.get('until'), unit).flatMap((t, i) => { + const count = Math.round((WRITES[i % WRITES.length] ?? 0) * scale); + return count === 0 ? [] : [{ bucket: new Date(t).toISOString(), scope: SCOPES[i % SCOPES.length] ?? 'global', count }]; + }); + return HttpResponse.json({ bucket: unit ?? 'day', since, until: url.searchParams.get('until'), buckets }); + }), + http.get('*/functions/v1/memories/read-activity', ({ request }) => { + const url = new URL(request.url); + const since = url.searchParams.get('since'); + const unit = url.searchParams.get('bucket'); + const scope = url.searchParams.get('scope'); + const scale = scaleFor(since); + const buckets = bucketsIn(since, url.searchParams.get('until'), unit) + .flatMap((t, i) => { + const bucket = new Date(t).toISOString(); + const bulk = Math.round((BULK[i % BULK.length] ?? 0) * scale); + const chosen = Math.round((CHOSEN[i % CHOSEN.length] ?? 0) * scale); + return [ + { bucket, scope: SCOPES[0], read_kind: 'bulk' as const, count: Math.round(bulk * 0.6) }, + { bucket, scope: SCOPES[1], read_kind: 'bulk' as const, count: Math.round(bulk * 0.3) }, + { bucket, scope: null, read_kind: 'bulk' as const, count: bulk - Math.round(bulk * 0.6) - Math.round(bulk * 0.3) }, + { bucket, scope: SCOPES[2], read_kind: 'targeted' as const, count: chosen }, + ]; + }) + .filter((cell) => cell.count > 0 && (scope === null || cell.scope === scope)); + return HttpResponse.json({ bucket: unit ?? 'day', since, until: url.searchParams.get('until'), buckets }); + }), + http.get('*/functions/v1/memories/usage', ({ request }) => { + const url = new URL(request.url); + const since = url.searchParams.get('since'); + const previous = scaleFor(since) < 1; + return HttpResponse.json({ + range: { since, until: url.searchParams.get('until') }, + correlation_id: null, + summary: { + total_events: 1_426, reads: 1_394, writes: 26, other: 6, + records_read: 13_134, + archived: previous ? 1 : 3, + expired: previous ? 0 : 2, + by_outcome: { ok: 1_420, error: 6 }, + }, + by_tool: previous ? VISUAL_BY_TOOL.map((r) => ({ ...r, event_count: Math.round(r.event_count * 0.6) })) : VISUAL_BY_TOOL, + by_scope_type: [], + }); + }), + http.get('*/functions/v1/memories/usage/runs', ({ request }) => { + const url = new URL(request.url); + return HttpResponse.json({ + range: { since: url.searchParams.get('since') ?? new Date(FROZEN - 90 * DAY_MS).toISOString(), until: FROZEN_NOW }, + runs: VISUAL_RUNS, + next_cursor: null, + }); + }), + ...utilityHandlers(), + ...memoryHandlers(), + ]; +} + +/** An account with no activity at all — every section renders its empty state. */ +export function emptyVisualHandlers() { + const empty = (url: URL) => ({ bucket: url.searchParams.get('bucket') ?? 'day', since: url.searchParams.get('since'), until: url.searchParams.get('until'), buckets: [] }); + return [ + http.get('*/functions/v1/memories/activity', ({ request }) => HttpResponse.json(empty(new URL(request.url)))), + http.get('*/functions/v1/memories/read-activity', ({ request }) => HttpResponse.json(empty(new URL(request.url)))), + http.get('*/functions/v1/memories/usage', ({ request }) => { + const url = new URL(request.url); + return HttpResponse.json({ + range: { since: url.searchParams.get('since'), until: url.searchParams.get('until') }, + correlation_id: null, + summary: { total_events: 0, reads: 0, writes: 0, other: 0, records_read: 0, archived: 0, expired: 0, by_outcome: {} }, + by_tool: [], + by_scope_type: [], + }); + }), + http.get('*/functions/v1/memories/usage/runs', () => + HttpResponse.json({ range: { since: null, until: FROZEN_NOW }, runs: [], next_cursor: null }), + ), + ...utilityHandlers([]), + ...memoryHandlers([]), + ]; +} diff --git a/packages/web/src/components/insights/visual/use-chart-cursor.ts b/packages/web/src/components/insights/visual/use-chart-cursor.ts new file mode 100644 index 000000000..ef065cb15 --- /dev/null +++ b/packages/web/src/components/insights/visual/use-chart-cursor.ts @@ -0,0 +1,83 @@ +'use client'; + +/** + * The active-item cursor every chart on the visual Insights page shares — the + * same interaction contract `Sparkbar` established, factored out so the Pulse + * timeline and the runs swimlanes behave identically: + * + * - ONE tab stop per chart; arrow keys / Home / End walk the items + * (`nextIndex`), Escape clears, blur clears. + * - Pointer hover activates (not on touch); a tap toggles and persists, and a + * pointer-down outside the chart dismisses it. + * - Item ELEMENTS are kept (not rects) so the portaled readout can re-measure + * its anchor on scroll. + */ + +import { useCallback, useEffect, useRef, useState, type KeyboardEvent, type PointerEvent } from 'react'; +import { nextIndex } from '@/components/dashboard/Sparkbar'; + +export function useChartCursor(count: number) { + const [active, setActive] = useState(null); + const rootRef = useRef(null); + const itemRefs = useRef<(Element | null)[]>([]); + + const dismiss = useCallback(() => setActive(null), []); + + useEffect(() => { + function onDown(e: globalThis.PointerEvent) { + if (rootRef.current && !rootRef.current.contains(e.target as Node)) setActive(null); + } + document.addEventListener('pointerdown', onDown); + return () => document.removeEventListener('pointerdown', onDown); + }, []); + + // A window change can shrink the series under an active index. + useEffect(() => { + setActive((a) => (a !== null && a >= count ? null : a)); + }, [count]); + + const onKeyDown = useCallback( + (e: KeyboardEvent) => { + if (e.key === 'Escape') { + setActive(null); + return; + } + const next = nextIndex(active, e.key, count); + if (next === null) return; + e.preventDefault(); + setActive(next); + }, + [active, count], + ); + + /** Pointer + click handlers for item `i`. */ + const itemHandlers = useCallback( + (i: number) => ({ + onPointerEnter: (e: PointerEvent) => { + if (e.pointerType !== 'touch') setActive(i); + }, + onPointerLeave: (e: PointerEvent) => { + if (e.pointerType !== 'touch') setActive((a) => (a === i ? null : a)); + }, + onClick: () => setActive((a) => (a === i ? null : i)), + }), + [], + ); + + const itemRef = useCallback( + (i: number) => (el: Element | null) => { + itemRefs.current[i] = el; + }, + [], + ); + + return { + active, + dismiss, + rootRef, + anchor: active === null ? null : (itemRefs.current[active] ?? null), + itemRef, + itemHandlers, + rootProps: { tabIndex: 0, onKeyDown, onBlur: dismiss }, + }; +} diff --git a/packages/web/src/components/insights/visual/use-in-view.ts b/packages/web/src/components/insights/visual/use-in-view.ts new file mode 100644 index 000000000..fafd5b569 --- /dev/null +++ b/packages/web/src/components/insights/visual/use-in-view.ts @@ -0,0 +1,41 @@ +'use client'; + +/** + * Whether an element has come within `rootMargin` of the viewport — ONCE. + * + * The visual Insights page defers its below-the-fold sections (runs, the scope + * leaderboard) until a reader scrolls near them, so a page load or a range + * switch does not fire every query on the page at the same moment: the API's + * slowest aggregates share one database, and a burst of a dozen concurrent + * requests is what pushed the delivery-cost query past its statement timeout. + * + * Latching (never flipping back to false) keeps a section's data and scroll + * height stable once it has loaded. Without IntersectionObserver (old browsers, + * some test environments) it reports in-view immediately — deferral is an + * optimisation, never a reason for content to be missing. + */ + +import { useEffect, useState } from 'react'; + +export function useInView(rootMargin = '300px') { + const [element, setElement] = useState(null); + const [inView, setInView] = useState(false); + + useEffect(() => { + if (inView || !element) return; + if (typeof IntersectionObserver === 'undefined') { + setInView(true); + return; + } + const observer = new IntersectionObserver( + (entries) => { + if (entries.some((e) => e.isIntersecting)) setInView(true); + }, + { rootMargin }, + ); + observer.observe(element); + return () => observer.disconnect(); + }, [element, inView, rootMargin]); + + return { ref: setElement, inView }; +} diff --git a/packages/web/src/components/lore/LoreCostHeadline.tsx b/packages/web/src/components/lore/LoreCostHeadline.tsx index 8b95e4bef..853bb3c48 100644 --- a/packages/web/src/components/lore/LoreCostHeadline.tsx +++ b/packages/web/src/components/lore/LoreCostHeadline.tsx @@ -27,32 +27,26 @@ * census that never moved. */ -import { useMemo } from 'react'; import { Coins } from 'lucide-react'; -import { useLoreUtility } from '@/lib/queries/lore-utility'; -import { readDeliveryCost, CHARS_PER_TOKEN_ESTIMATE } from '@/lib/lore-cost'; +import { useLoreUtilitySummary } from '@/lib/queries/lore-utility'; +import { readDeliveryCost, CHARS_PER_TOKEN_ESTIMATE, COST_WINDOW_DAYS } from '@/lib/lore-cost'; /** - * The window, in days. - * - * A rolling 30 days rather than the calendar month the phrase "this month" - * suggests: on the 2nd of a month the calendar figure is two days of data - * under a label a reader will compare against last month's thirty, and a bill - * that collapses at the start of every month is a bill nobody trusts. The - * caption says "the last 30 days" so the label matches the arithmetic. + * The window is `COST_WINDOW_DAYS` (30), rolling rather than the calendar month + * "this month" suggests: on the 2nd of a month the calendar figure is two days + * of data under a label a reader will compare against last month's thirty, and + * a bill that collapses at the start of every month is a bill nobody trusts. + * The caption says "the last 30 days" so the label matches the arithmetic. The + * `since` is shared with the utility grid (`useLoreUtilitySummary`) so the two + * read ONE request. */ -const WINDOW_DAYS = 30; +const WINDOW_DAYS = COST_WINDOW_DAYS; export function LoreCostHeadline() { - // Mount-stable, so the query key and the caption describe one instant. - const since = useMemo( - () => new Date(Date.now() - WINDOW_DAYS * 86_400_000).toISOString(), - [], - ); // No `until`: the cost sum's upper bound is exclusive of the whole day it // names (its source is a daily rollup), so passing `now` would silently drop // today. Omitting it is how a caller says "through right now". - const { data, isLoading, isError } = useLoreUtility({ since }); + const { data, isLoading, isError } = useLoreUtilitySummary(); if (isError) { // Not folded into an empty state: "0 tokens delivered" is a claim, and a diff --git a/packages/web/src/components/lore/LoreUtilityGrid.tsx b/packages/web/src/components/lore/LoreUtilityGrid.tsx index 778d15788..d7d0e1c0f 100644 --- a/packages/web/src/components/lore/LoreUtilityGrid.tsx +++ b/packages/web/src/components/lore/LoreUtilityGrid.tsx @@ -37,7 +37,7 @@ import { useState } from 'react'; import Link from 'next/link'; import { Copy, Check, ArrowUpRight } from 'lucide-react'; import { ScopeBadge } from '@/components/memory/ScopeBadge'; -import { useLoreUtility, useLoreUtilityRows } from '@/lib/queries/lore-utility'; +import { useLoreUtilitySummary, useLoreUtilityRows } from '@/lib/queries/lore-utility'; import { LESSON_UTILITY_META, formatPullThrough, type LessonUtility } from '@/lib/lesson-utility'; import { groomPrompt } from '@/lib/lore-utility-prompt'; import { track } from '@/lib/analytics/track'; @@ -85,7 +85,8 @@ export function LoreUtilityGrid() { } // Account-wide: the page it lives on has no scope filter, and the hooks take // one only because the route does. - const census = useLoreUtility({}); + // The census is window-independent; sharing the cost line's key makes the two one request. + const census = useLoreUtilitySummary(); const rows = useLoreUtilityRows(selected); if (census.isError) { diff --git a/packages/web/src/components/ui/__screenshots__/Badge.stories.tsx/Default-1-chromium-linux.png b/packages/web/src/components/ui/__screenshots__/Badge.stories.tsx/Default-1-chromium-linux.png index fcfc23f3e..8ce4fd923 100644 Binary files a/packages/web/src/components/ui/__screenshots__/Badge.stories.tsx/Default-1-chromium-linux.png and b/packages/web/src/components/ui/__screenshots__/Badge.stories.tsx/Default-1-chromium-linux.png differ diff --git a/packages/web/src/components/ui/__screenshots__/Badge.stories.tsx/Playground-1-chromium-linux.png b/packages/web/src/components/ui/__screenshots__/Badge.stories.tsx/Playground-1-chromium-linux.png index a80091ca2..f6c58cb96 100644 Binary files a/packages/web/src/components/ui/__screenshots__/Badge.stories.tsx/Playground-1-chromium-linux.png and b/packages/web/src/components/ui/__screenshots__/Badge.stories.tsx/Playground-1-chromium-linux.png differ diff --git a/packages/web/src/lib/analytics/lore-events.ts b/packages/web/src/lib/analytics/lore-events.ts index c758e5f7f..316fd5a7c 100644 --- a/packages/web/src/lib/analytics/lore-events.ts +++ b/packages/web/src/lib/analytics/lore-events.ts @@ -166,8 +166,12 @@ export type MemoryMutationOutcome = 'success' | 'error'; /** Archive is a toggle, so the direction is part of the fact. */ export type MemoryArchiveAction = 'archive' | 'restore'; -/** Insights has two independent windows; a bare preset would not say which moved. */ -export type InsightsRangeSection = 'agent-activity' | 'scope-consumption'; +/** + * Which Insights window moved. The original page has two independent windows, + * so a bare preset would not say which; the flag-gated visual page + * (`insights-visual`) has ONE, reported as `page`. + */ +export type InsightsRangeSection = 'agent-activity' | 'scope-consumption' | 'page'; /** Where a click into the Explorer came from on the Insights page. */ export type InsightsScopeSource = 'scope-consumption' | 'utility-row'; diff --git a/packages/web/src/lib/analytics/track.spec.ts b/packages/web/src/lib/analytics/track.spec.ts index da6bf69b2..d7b052dac 100644 --- a/packages/web/src/lib/analytics/track.spec.ts +++ b/packages/web/src/lib/analytics/track.spec.ts @@ -514,4 +514,11 @@ describe('track — insights events', () => { 'lorekit.insights.scope.source': 'scope-consumption', }); }); + + it('says which view the recall flow swapped to', () => { + track({ name: 'insights.recall_flow_view_changed', view: 'table' }); + expect(lastCall().attributes).toEqual({ + 'lorekit.insights.recall_flow.view': 'table', + }); + }); }); diff --git a/packages/web/src/lib/analytics/track.ts b/packages/web/src/lib/analytics/track.ts index 59d801f39..cef96c724 100644 --- a/packages/web/src/lib/analytics/track.ts +++ b/packages/web/src/lib/analytics/track.ts @@ -57,10 +57,12 @@ * - `lore.results_paged` — "Load more" appended a page. * * ### Insights (`insights.*`) - * - `insights.range_changed` — one of the page's TWO independent windows moved. + * - `insights.range_changed` — one of the page's TWO independent windows moved + * (`section: 'page'` — the visual page's single window). * - `insights.utility_quadrant_selected` — a utility quadrant was opened or closed. * - `insights.utility_prompt_copied` — a grooming prompt reached the clipboard. * - `insights.scope_opened` — a row handed off to the Explorer. + * - `insights.recall_flow_view_changed` — the visual page's recall flow swapped between chart and table. * * ## PII / cardinality * Command ids are a fixed enum EXCEPT the dynamic "Open Lesson…" children, whose @@ -343,7 +345,7 @@ export type AnalyticsEvent = // ── Insights ────────────────────────────────────────────────────────────── | { name: 'insights.range_changed'; - /** The page carries TWO independent windows; a bare preset would not say which moved. */ + /** The original page carries TWO independent windows; a bare preset would not say which moved. The visual page's one window is `page`. */ section: InsightsRangeSection; preset: RangeTelemetryPreset; spanDays?: number; @@ -366,7 +368,8 @@ export type AnalyticsEvent = */ succeeded: boolean; } - | { name: 'insights.scope_opened'; scopeType: ScopeSelectionType; source: InsightsScopeSource }; + | { name: 'insights.scope_opened'; scopeType: ScopeSelectionType; source: InsightsScopeSource } + | { name: 'insights.recall_flow_view_changed'; view: 'chart' | 'table' }; /** * Dynamic lesson commands embed `scope::key` in their id (user content + @@ -590,6 +593,8 @@ function toAttributes(event: AnalyticsEvent): Record { 'lorekit.insights.scope.type': event.scopeType, 'lorekit.insights.scope.source': event.source, }; + case 'insights.recall_flow_view_changed': + return { 'lorekit.insights.recall_flow.view': event.view }; } } diff --git a/packages/web/src/lib/flow-layout.spec.ts b/packages/web/src/lib/flow-layout.spec.ts new file mode 100644 index 000000000..4a7b84737 --- /dev/null +++ b/packages/web/src/lib/flow-layout.spec.ts @@ -0,0 +1,68 @@ +import { describe, expect, it } from 'vitest'; +import { layoutFlow, type FlowNodeInput } from './flow-layout'; + +const nodes: FlowNodeInput[] = [ + { id: 'a', label: 'a', layer: 0 }, + { id: 'b', label: 'b', layer: 0 }, + { id: 'm', label: 'm', layer: 1 }, + { id: 'x', label: 'x', layer: 2 }, + { id: 'y', label: 'y', layer: 2 }, +]; + +describe('layoutFlow', () => { + const layout = layoutFlow( + nodes, + [ + { source: 'a', target: 'm', value: 30 }, + { source: 'b', target: 'm', value: 10 }, + { source: 'm', target: 'x', value: 25 }, + { source: 'm', target: 'y', value: 15 }, + ], + { width: 300, height: 100, nodeWidth: 10, nodePadding: 10 }, + ); + const byId = new Map(layout.nodes.map((n) => [n.id, n])); + + it('gives a pass-through node its conserved flow and places columns across the width', () => { + expect(byId.get('m')?.value).toBe(40); + expect(byId.get('a')?.x0).toBe(0); + expect(byId.get('m')?.x0).toBe(145); + expect(byId.get('x')?.x1).toBe(300); + }); + + it('uses one scale for every column, picked from the fullest', () => { + // Columns 0 and 2 each hold 40 across two nodes (one 10px gap) → 90px / 40. + const scale = 90 / 40; + const height = (id: string) => { + const n = byId.get(id); + return n ? n.y1 - n.y0 : NaN; + }; + expect(height('m')).toBeCloseTo(40 * scale); + expect(height('a')).toBeCloseTo(30 * scale); + for (const link of layout.links) expect(link.width).toBeCloseTo(link.value * scale); + }); + + it('stacks bands inside their nodes without overlap', () => { + const [upper, lower] = layout.links.filter((l) => l.target === 'm').sort((p, q) => p.ty - q.ty); + const m = byId.get('m'); + expect(upper && m && upper.ty - upper.width / 2).toBeCloseTo(m?.y0 ?? NaN); + expect(lower && m && lower.ty + lower.width / 2).toBeCloseTo(m?.y1 ?? NaN); + }); + + it('drops zero, negative and dangling links, and nodes left without flow', () => { + const sparse = layoutFlow( + nodes, + [ + { source: 'a', target: 'm', value: 5 }, + { source: 'b', target: 'm', value: 0 }, + { source: 'm', target: 'ghost', value: 5 }, + ], + { width: 100, height: 50 }, + ); + expect(sparse.nodes.map((n) => n.id).sort()).toEqual(['a', 'm']); + expect(sparse.links).toHaveLength(1); + }); + + it('returns an empty layout when there is nothing to draw', () => { + expect(layoutFlow(nodes, [], { width: 100, height: 50 })).toEqual({ nodes: [], links: [] }); + }); +}); diff --git a/packages/web/src/lib/flow-layout.ts b/packages/web/src/lib/flow-layout.ts new file mode 100644 index 000000000..32ade5f47 --- /dev/null +++ b/packages/web/src/lib/flow-layout.ts @@ -0,0 +1,172 @@ +/** + * A small, pure layered-flow (Sankey) layout: nodes in fixed columns, link + * bands whose thickness is proportional to their value. + * + * ## Why hand-rolled rather than `d3-sankey` + * + * The recall flow is three fixed columns (client → tool → scope type) with a + * handful of nodes each — no cycles, no column assignment to infer, no + * iterative relaxation worth running. What `d3-sankey` adds over this is + * exactly the part this chart does not need, and every other chart in the + * dashboard is hand-rolled SVG over a pure layout (`Sparkbar`, + * `ContributionHeatmap`), so this keeps the same shape: geometry here, tested; + * rendering and interaction in the component. + * + * ## One scale for the whole chart + * + * Every band shares ONE value→pixel scale, picked from the fullest column, so a + * band is the same thickness at both ends and two bands anywhere on the chart + * compare by thickness alone. Per-column scales would let a column with less + * total flow look as full as one with more — the Sankey equivalent of a + * truncated axis. + * + * ## Order is the caller's + * + * Nodes are stacked in the order given within their column (the caller sorts, + * typically by value with "other" last). Bands leave a node ordered by their + * target's position and arrive ordered by their source's, which is what keeps + * crossings to the ones the data actually implies. + */ + +export interface FlowNodeInput { + id: string; + label: string; + /** 0-based column. */ + layer: number; +} + +export interface FlowLinkInput { + source: string; + target: string; + value: number; +} + +export interface FlowNode extends FlowNodeInput { + /** `max(inflow, outflow)` — a pass-through node's flow, a leaf's only side. */ + value: number; + x0: number; + x1: number; + y0: number; + y1: number; +} + +export interface FlowLink extends FlowLinkInput { + /** Band thickness in px. */ + width: number; + /** Band centre where it leaves its source. */ + sy: number; + /** Band centre where it reaches its target. */ + ty: number; + /** SVG path for the band's centre line — stroke it at `width`. */ + path: string; +} + +export interface FlowLayout { + nodes: FlowNode[]; + links: FlowLink[]; +} + +export interface FlowLayoutOptions { + width: number; + height: number; + nodeWidth?: number; + nodePadding?: number; +} + +/** + * Lay out `nodes` and `links` in a `width` × `height` box. + * + * Total: links with a non-positive value or an unknown endpoint are dropped, + * nodes left with no flow are dropped, and an input with nothing to draw + * returns an empty layout rather than NaN geometry. + */ +export function layoutFlow( + nodes: readonly FlowNodeInput[], + links: readonly FlowLinkInput[], + { width, height, nodeWidth = 12, nodePadding = 10 }: FlowLayoutOptions, +): FlowLayout { + const known = new Set(nodes.map((n) => n.id)); + const usable = links.filter((l) => l.value > 0 && known.has(l.source) && known.has(l.target)); + + const inflow = new Map(); + const outflow = new Map(); + for (const l of usable) { + outflow.set(l.source, (outflow.get(l.source) ?? 0) + l.value); + inflow.set(l.target, (inflow.get(l.target) ?? 0) + l.value); + } + + const valued = nodes + .map((n) => ({ ...n, value: Math.max(inflow.get(n.id) ?? 0, outflow.get(n.id) ?? 0) })) + .filter((n) => n.value > 0); + if (valued.length === 0) return { nodes: [], links: [] }; + + const layerCount = Math.max(...valued.map((n) => n.layer)) + 1; + const columns: (typeof valued)[] = Array.from({ length: layerCount }, () => []); + for (const n of valued) columns[n.layer]?.push(n); + + // One scale for every column — see the module docblock. + const scale = Math.min( + ...columns + .filter((col) => col.length > 0) + .map((col) => { + const total = col.reduce((sum, n) => sum + n.value, 0); + return Math.max(0, height - nodePadding * (col.length - 1)) / total; + }), + ); + + const step = layerCount > 1 ? (width - nodeWidth) / (layerCount - 1) : 0; + const laidOut = new Map(); + columns.forEach((col, layer) => { + const used = col.reduce((sum, n) => sum + n.value * scale, 0) + nodePadding * Math.max(0, col.length - 1); + // Centre each column vertically, so a column with less flow than the + // fullest one sits in the middle rather than hugging the top edge. + let y = (height - used) / 2; + const x0 = layer * step; + for (const n of col) { + const h = n.value * scale; + laidOut.set(n.id, { ...n, x0, x1: x0 + nodeWidth, y0: y, y1: y + h }); + y += h + nodePadding; + } + }); + + // Resolve both endpoints once, so the placement below never re-looks-up a node. + const placed = usable.flatMap((link) => { + const source = laidOut.get(link.source); + const target = laidOut.get(link.target); + return source && target ? [{ link, source, target, width: link.value * scale, sy: 0, ty: 0 }] : []; + }); + + // Leave each source ordered by target position, arrive ordered by source position. + const sourceOffset = new Map(); + for (const band of [...placed].sort((a, b) => a.source.y0 - b.source.y0 || a.target.y0 - b.target.y0)) { + const offset = sourceOffset.get(band.source.id) ?? 0; + band.sy = band.source.y0 + offset + band.width / 2; + sourceOffset.set(band.source.id, offset + band.width); + } + const targetOffset = new Map(); + for (const band of [...placed].sort((a, b) => a.target.y0 - b.target.y0 || a.source.y0 - b.source.y0)) { + const offset = targetOffset.get(band.target.id) ?? 0; + band.ty = band.target.y0 + offset + band.width / 2; + targetOffset.set(band.target.id, offset + band.width); + } + + const flowLinks: FlowLink[] = placed.map(({ link, source, target, width: w, sy, ty }) => { + const x0 = source.x1; + const x1 = target.x0; + const xm = (x0 + x1) / 2; + return { + ...link, + width: w, + sy, + ty, + path: `M${round(x0)},${round(sy)}C${round(xm)},${round(sy)} ${round(xm)},${round(ty)} ${round(x1)},${round(ty)}`, + }; + }); + + return { nodes: [...laidOut.values()], links: flowLinks }; +} + +/** Two decimals keep the path string short without visible error. */ +function round(n: number): number { + return Math.round(n * 100) / 100; +} diff --git a/packages/web/src/lib/insights-loop.spec.ts b/packages/web/src/lib/insights-loop.spec.ts new file mode 100644 index 000000000..1d7910c4c --- /dev/null +++ b/packages/web/src/lib/insights-loop.spec.ts @@ -0,0 +1,148 @@ +import { describe, expect, it } from 'vitest'; +import type { ActivityBucket, ReadActivityBucket } from '@lorekit/schemas/memory'; +import { + ROW_CAP, + bucketStarts, + chosenShare, + isAtRowCap, + loopStages, + periodChange, + pulseBuckets, + pulseTotals, + rawTotals, +} from './insights-loop'; + +const write = (bucket: string, count: number, scope = 'global'): ActivityBucket => ({ bucket, scope, count }); +const read = (bucket: string, count: number, read_kind?: 'targeted' | 'bulk', scope: string | null = 'global'): ReadActivityBucket => ({ + bucket, + scope, + count, + ...(read_kind ? { read_kind } : {}), +}); + +describe('isAtRowCap', () => { + it('flags a response at or above the PostgREST row cap', () => { + expect(isAtRowCap(ROW_CAP - 1)).toBe(false); + expect(isAtRowCap(ROW_CAP)).toBe(true); + expect(isAtRowCap(ROW_CAP + 5)).toBe(true); + }); +}); + +describe('bucketStarts', () => { + it('covers every UTC day bucket the window touches, including a partial first day', () => { + expect(bucketStarts({ from: '2026-06-12T14:00:00.000Z', to: '2026-06-15T12:00:00.000Z' }, 'day')).toEqual([ + '2026-06-12T00:00:00.000Z', + '2026-06-13T00:00:00.000Z', + '2026-06-14T00:00:00.000Z', + '2026-06-15T00:00:00.000Z', + ]); + }); + + it('steps by the hour for hourly buckets', () => { + expect(bucketStarts({ from: '2026-06-15T09:30:00.000Z', to: '2026-06-15T12:00:00.000Z' }, 'hour')).toEqual([ + '2026-06-15T09:00:00.000Z', + '2026-06-15T10:00:00.000Z', + '2026-06-15T11:00:00.000Z', + ]); + }); + + it('returns nothing for an inverted or unparseable window', () => { + expect(bucketStarts({ from: '2026-06-15T12:00:00.000Z', to: '2026-06-15T09:00:00.000Z' }, 'hour')).toEqual([]); + expect(bucketStarts({ from: 'nope', to: '2026-06-15T09:00:00.000Z' }, 'day')).toEqual([]); + }); +}); + +describe('pulseBuckets', () => { + const window = { from: '2026-06-13T00:00:00.000Z', to: '2026-06-16T00:00:00.000Z' }; + + it('sums writes across scopes and splits reads by kind, keeping empty buckets as zeros', () => { + const buckets = pulseBuckets( + window, + 'day', + [write('2026-06-13T00:00:00.000Z', 2, 'global'), write('2026-06-13T00:00:00.000Z', 3, 'repo::a/b')], + [ + read('2026-06-15T00:00:00.000Z', 40, 'bulk'), + read('2026-06-15T00:00:00.000Z', 4, 'targeted'), + read('2026-06-15T00:00:00.000Z', 6, 'bulk', null), + ], + ); + expect(buckets).toEqual([ + { start: '2026-06-13T00:00:00.000Z', written: 5, chosen: 0, bulk: 0 }, + { start: '2026-06-14T00:00:00.000Z', written: 0, chosen: 0, bulk: 0 }, + { start: '2026-06-15T00:00:00.000Z', written: 0, chosen: 4, bulk: 46 }, + ]); + }); + + it('counts an untyped (pre-00080) read row as bulk, never as a deliberate fetch', () => { + const [bucket] = pulseBuckets(window, 'day', [], [read('2026-06-13T00:00:00.000Z', 9)]); + expect(bucket).toMatchObject({ chosen: 0, bulk: 9 }); + }); + + it('ignores rows outside the window rather than misplacing them', () => { + const buckets = pulseBuckets(window, 'day', [write('2026-06-01T00:00:00.000Z', 7)], []); + expect(pulseTotals(buckets).written).toBe(0); + }); + + it('bars sum to the totals the strip states', () => { + const writes = [write('2026-06-13T00:00:00.000Z', 1), write('2026-06-14T00:00:00.000Z', 2)]; + const reads = [read('2026-06-14T00:00:00.000Z', 3, 'targeted'), read('2026-06-15T00:00:00.000Z', 5, 'bulk')]; + expect(pulseTotals(pulseBuckets(window, 'day', writes, reads))).toEqual(rawTotals(writes, reads)); + }); +}); + +describe('periodChange', () => { + it('omits a delta when the previous window is empty', () => { + expect(periodChange(0, 0)).toBeNull(); + expect(periodChange(12, 0)).toBeNull(); + }); + + it('is a rounded percentage otherwise', () => { + expect(periodChange(15, 10)).toBe(50); + expect(periodChange(5, 10)).toBe(-50); + }); +}); + +describe('loopStages', () => { + const current = { totals: { written: 12, chosen: 6, bulk: 294 }, summary: { archived: 3, expired: 2 } }; + const previous = { totals: { written: 8, chosen: 2, bulk: 148 }, summary: { archived: 0, expired: 0 } }; + const census = { 'load-bearing': 4, specialist: 7, 'noise-tax': 30, dormant: 11, unproven: 40 }; + + it('states each stage from its own source', () => { + expect(loopStages(current, previous, census)).toEqual({ + learn: { created: 12, changePct: 50 }, + recall: { records: 300, chosen: 6, changePct: 100 }, + promote: { loadBearing: 4, specialist: 7 }, + prune: { archived: 3, expired: 2, changePct: null }, + seriesIncomplete: false, + }); + }); + + it('drops the Learn and Recall deltas when the current series hit the row cap, but keeps Prune', () => { + const stages = loopStages(current, { ...previous, summary: { archived: 1, expired: 0 } }, census, { + current: true, + previous: false, + }); + expect(stages.learn.changePct).toBeNull(); + expect(stages.recall.changePct).toBeNull(); + expect(stages.prune.changePct).toBe(400); + expect(stages.seriesIncomplete).toBe(true); + }); + + it('drops the Learn and Recall deltas but does not mark the figures incomplete when only the previous series hit the cap', () => { + const stages = loopStages(current, previous, census, { current: false, previous: true }); + expect(stages.seriesIncomplete).toBe(false); + expect(stages.learn.changePct).toBeNull(); + expect(stages.recall.changePct).toBeNull(); + }); + + it('leaves Promote absent while the census has not loaded', () => { + expect(loopStages(current, previous, null).promote).toBeNull(); + }); +}); + +describe('chosenShare', () => { + it('is the targeted share of recalled records, or null with no reads', () => { + expect(chosenShare({ recall: { records: 200, chosen: 5, changePct: null } })).toBe(0.025); + expect(chosenShare({ recall: { records: 0, chosen: 0, changePct: null } })).toBeNull(); + }); +}); diff --git a/packages/web/src/lib/insights-loop.ts b/packages/web/src/lib/insights-loop.ts new file mode 100644 index 000000000..8d3d3ed03 --- /dev/null +++ b/packages/web/src/lib/insights-loop.ts @@ -0,0 +1,232 @@ +/** + * The pure model behind the visual Insights page (`insights-visual`): the + * Learn → Recall → Promote → Prune strip and the Pulse timeline. + * + * ## One window, and the figures it can honestly carry + * + * The strip states four stages of the loop LoreKit runs, and each figure comes + * from the ONE source that can carry it — never a second source that happens to + * be nearby: + * + * - **Learn** — lessons CREATED in the window (`/memories/activity`). Not + * `memory.write` calls: a re-write of an existing key is also a write, and + * no endpoint buckets re-writes over time yet, so mixing the two would make + * the headline disagree with the bars drawn under it. + * - **Recall** — memory RECORDS read (`/memories/read-activity`), split into + * `targeted` (an agent chose this lesson) and `bulk` (it rode along in a + * list/search page). Records, not calls — the additive figure a bar can sum. + * - **Promote** — the `load-bearing` and `specialist` counts of the utility + * census. ALL-TIME by construction (it reads lifetime counters), so it has + * no period delta and the strip says so rather than inventing one. + * - **Prune** — `memory.archive` calls plus TTL-expired records in the window + * (`/memories/usage` summary). Window totals only: there is no bucketed + * lifecycle series yet, so this stage draws no bars. + * + * ## The row cap is a real ceiling, so it is checked rather than assumed away + * + * The activity series arrive as one row per `(bucket, scope[, read_kind])`, and + * PostgREST returns at most `max_rows` (1000, `supabase/config.toml`) rows from + * the RPC — with no error. A long window over many scopes can reach it, and + * because the RPC orders by bucket ascending, the rows it drops are the NEWEST + * ones: the chart would show a cliff to zero at "now". {@link isAtRowCap} lets + * the page say "this may be incomplete" instead of drawing that cliff as fact. + * + * Pure and total: every function answers for an empty account without a + * division by zero and without a caller-side guard. + */ + +import type { ActivityBucket, ReadActivityBucket, UtilityCensus } from '@lorekit/schemas/memory'; +import type { UsageSummary } from '@lorekit/schemas/usage'; +import { pctChange } from '@/lib/aggregations'; +import { DAY_MS, HOUR_MS, type AbsoluteRange, type BucketUnit } from '@/lib/time-range'; + +/** PostgREST's `max_rows` for this project (`supabase/config.toml`). */ +export const ROW_CAP = 1000; + +/** + * Whether a series response may have been truncated by {@link ROW_CAP}. + * + * `>=`, not `===`: the cap is a ceiling the server enforces, so a count at or + * above it means "we cannot tell what was dropped". Exactly-1000 genuine rows is + * indistinguishable from a truncated response, and saying "may be incomplete" + * about a complete one is the cheaper mistake. + */ +export function isAtRowCap(rowCount: number): boolean { + return rowCount >= ROW_CAP; +} + +function unitMs(unit: BucketUnit): number { + return unit === 'hour' ? HOUR_MS : DAY_MS; +} + +/** + * Every bucket start (UTC, ISO) whose bucket intersects `window`. + * + * Anchored the way the server anchors its buckets (`date_trunc` on UTC), so a + * server row lands on exactly one of these. The first and last buckets are + * usually PARTIAL: the server filters `>= since`, so a 7-day window opened at + * 14:00 yields 8 day bars whose first holds only that day's afternoon. + */ +export function bucketStarts(window: AbsoluteRange, unit: BucketUnit): string[] { + const step = unitMs(unit); + const from = Date.parse(window.from); + const to = Date.parse(window.to); + if (Number.isNaN(from) || Number.isNaN(to) || to <= from) return []; + const starts: string[] = []; + for (let t = Math.floor(from / step) * step; t < to; t += step) { + starts.push(new Date(t).toISOString()); + } + return starts; +} + +/** One bucket of the Pulse timeline. */ +export interface PulseBucket { + /** UTC start of the bucket, ISO. */ + start: string; + /** Lessons created in the bucket, summed across scopes. */ + written: number; + /** Records an agent fetched on purpose (`memory.read`). */ + chosen: number; + /** Records that rode along in a list/search page. */ + bulk: number; +} + +/** + * The Pulse series: one entry per bucket of the window, zeros included. + * + * Zeros are kept because a gap IS the information on a time axis — a missing + * Tuesday must render as an empty Tuesday, not as Monday sitting next to + * Wednesday. A `read_kind` absent from the row (a pre-00080 response) counts as + * `bulk`: the conservative reading, since `chosen` is a claim that an agent + * deliberately reached for the lesson and an untyped row cannot support it. + */ +export function pulseBuckets( + window: AbsoluteRange, + unit: BucketUnit, + writes: readonly ActivityBucket[], + reads: readonly ReadActivityBucket[], +): PulseBucket[] { + const step = unitMs(unit); + const buckets = bucketStarts(window, unit).map((start) => ({ start, written: 0, chosen: 0, bulk: 0 })); + const index = new Map(buckets.map((b, i) => [Date.parse(b.start), i])); + const slot = (iso: string): PulseBucket | undefined => { + const t = Date.parse(iso); + if (Number.isNaN(t)) return undefined; + const i = index.get(Math.floor(t / step) * step); + return i === undefined ? undefined : buckets[i]; + }; + + for (const row of writes) { + const bucket = slot(row.bucket); + if (bucket) bucket.written += row.count; + } + for (const row of reads) { + const bucket = slot(row.bucket); + if (!bucket) continue; + if (row.read_kind === 'targeted') bucket.chosen += row.count; + else bucket.bulk += row.count; + } + return buckets; +} + +/** Sums of a Pulse series — the totals the strip's headline figures state. */ +export interface PulseTotals { + written: number; + chosen: number; + bulk: number; +} + +export function pulseTotals(buckets: readonly PulseBucket[]): PulseTotals { + return buckets.reduce( + (acc, b) => ({ written: acc.written + b.written, chosen: acc.chosen + b.chosen, bulk: acc.bulk + b.bulk }), + { written: 0, chosen: 0, bulk: 0 }, + ); +} + +/** + * Totals for the PREVIOUS window, straight from the raw rows. + * + * No bucket grid is needed — the previous window only ever feeds a delta — so + * this skips {@link pulseBuckets} and sums every row the server returned for + * that window. + */ +export function rawTotals(writes: readonly ActivityBucket[], reads: readonly ReadActivityBucket[]): PulseTotals { + let written = 0; + let chosen = 0; + let bulk = 0; + for (const row of writes) written += row.count; + for (const row of reads) { + if (row.read_kind === 'targeted') chosen += row.count; + else bulk += row.count; + } + return { written, chosen, bulk }; +} + +export interface LoopWindowInput { + totals: PulseTotals; + /** The `/usage` summary for the same window — the source of the Prune stage. */ + summary: Pick; +} + +export interface LoopStages { + learn: { created: number; changePct: number | null }; + recall: { records: number; chosen: number; changePct: number | null }; + promote: { loadBearing: number; specialist: number } | null; + prune: { archived: number; expired: number; changePct: number | null }; + /** The CURRENT window's series hit the row cap — Learn and Recall are lower bounds. */ + seriesIncomplete: boolean; +} + +/** + * A period delta, or `null` when there is nothing to compare against. + * + * `null` when BOTH windows are empty (0 → 0 is not "flat", it is no evidence) + * and when the previous window is empty but this one is not: `pctChange` calls + * that +100%, which a reader takes as "doubled" when it really means "started". + * A chip is omitted rather than shown misleading. + */ +export function periodChange(current: number, previous: number): number | null { + if (previous === 0) return null; + return pctChange(current, previous); +} + +/** + * The four stage figures, each from the one source that can carry it (see the module docblock). + * + * `truncation` — which window's activity series hit the row cap. Only the + * CURRENT window's cap makes the stated Learn and Recall figures lower bounds + * (`seriesIncomplete`); the previous window only ever feeds a delta. A cap on + * EITHER window drops the Learn and Recall deltas: a delta against a figure + * that may be missing rows is not a comparison. + */ +export function loopStages( + current: LoopWindowInput, + previous: LoopWindowInput, + census: UtilityCensus | null, + truncation: { current: boolean; previous: boolean } = { current: false, previous: false }, +): LoopStages { + const records = current.totals.chosen + current.totals.bulk; + const previousRecords = previous.totals.chosen + previous.totals.bulk; + const pruned = current.summary.archived + current.summary.expired; + const previousPruned = previous.summary.archived + previous.summary.expired; + const eitherTruncated = truncation.current || truncation.previous; + const delta = (now: number, before: number) => (eitherTruncated ? null : periodChange(now, before)); + return { + learn: { created: current.totals.written, changePct: delta(current.totals.written, previous.totals.written) }, + recall: { records, chosen: current.totals.chosen, changePct: delta(records, previousRecords) }, + promote: census ? { loadBearing: census['load-bearing'], specialist: census.specialist } : null, + prune: { archived: current.summary.archived, expired: current.summary.expired, changePct: periodChange(pruned, previousPruned) }, + seriesIncomplete: truncation.current, + }; +} + +/** + * Share of recalled records that were a deliberate fetch, or `null` with no reads. + * + * The same selection-not-influence caveat as the cost headline applies: a + * lesson injected at session start never needs a second fetch to be acted on, + * so a low share is "rarely looked up on purpose", not "wasted". + */ +export function chosenShare(stages: Pick): number | null { + return stages.recall.records === 0 ? null : stages.recall.chosen / stages.recall.records; +} diff --git a/packages/web/src/lib/lore-cost.spec.ts b/packages/web/src/lib/lore-cost.spec.ts index f08a3cb0f..1d0bd4b7a 100644 --- a/packages/web/src/lib/lore-cost.spec.ts +++ b/packages/web/src/lib/lore-cost.spec.ts @@ -1,5 +1,5 @@ import { describe, it, expect } from 'vitest'; -import { formatTokenVolume, formatShare, readDeliveryCost, CHARS_PER_TOKEN_ESTIMATE } from './lore-cost'; +import { formatTokenVolume, formatShare, readDeliveryCost, costWindowSince, CHARS_PER_TOKEN_ESTIMATE } from './lore-cost'; import type { UtilityCost } from '@lorekit/schemas/memory'; const cost = (over: Partial = {}): UtilityCost => ({ @@ -111,3 +111,15 @@ describe('CHARS_PER_TOKEN_ESTIMATE', () => { expect(CHARS_PER_TOKEN_ESTIMATE).toBe(4); }); }); + +describe('costWindowSince', () => { + it('is 30 days back, floored to the start of the UTC hour', () => { + expect(costWindowSince(Date.parse('2026-10-21T08:47:13.250Z'))).toBe('2026-09-21T08:00:00.000Z'); + }); + + it('is the same value for every instant inside one hour, so callers share one query key', () => { + expect(costWindowSince(Date.parse('2026-10-21T08:00:00.000Z'))).toBe( + costWindowSince(Date.parse('2026-10-21T08:59:59.999Z')), + ); + }); +}); diff --git a/packages/web/src/lib/lore-cost.ts b/packages/web/src/lib/lore-cost.ts index c79608dff..1c3d710bb 100644 --- a/packages/web/src/lib/lore-cost.ts +++ b/packages/web/src/lib/lore-cost.ts @@ -27,6 +27,28 @@ import { CHARS_PER_TOKEN_ESTIMATE, type UtilityCost } from '@lorekit/schemas/mem export { CHARS_PER_TOKEN_ESTIMATE }; +/** The cost line's window, in days — "the last 30 days", rolling, not a calendar month. */ +export const COST_WINDOW_DAYS = 30; + +const HOUR_MS = 3_600_000; + +/** + * The `since` every `/memories/utility` census caller sends: 30 days back, + * floored to the start of the UTC hour. + * + * ONE value so the cost line, the utility grid and the Insights strip share ONE + * request (same query key). Every `/utility` call runs the census AND the + * delivery-cost aggregate server-side, and the cost aggregate is the slowest + * query on the page — a census-only caller that omitted `since` used to pay + * for the ALL-HISTORY variant of it just to read the census. Flooring to the + * hour keeps the key stable across renders and remounts; the window it names + * is at most an hour longer than 30 days, well inside the estimate's precision. + */ +export function costWindowSince(nowMs: number): string { + const since = nowMs - COST_WINDOW_DAYS * 86_400_000; + return new Date(Math.floor(since / HOUR_MS) * HOUR_MS).toISOString(); +} + /** * `602M`, `1.2M`, `84K`, `840`. * diff --git a/packages/web/src/lib/queries/insights-loop.ts b/packages/web/src/lib/queries/insights-loop.ts new file mode 100644 index 000000000..43212bd0f --- /dev/null +++ b/packages/web/src/lib/queries/insights-loop.ts @@ -0,0 +1,156 @@ +'use client'; + +/** + * Data for the visual Insights page (`insights-visual`): the loop strip, the + * Pulse timeline and the recall flow share ONE fetch over ONE window, and the + * runs views share a second, paginated one. + * + * ## One window, plus the one before it + * + * Every windowed figure on the page follows the page's single range control, so + * the strip, the timeline and the flow can never describe different periods. + * The immediately preceding window of equal length is fetched beside it + * (`priorWindow`, shared with `insights-usage.ts`) because a period delta needs + * the actual previous period, not a slice of a wider one. + * + * No new API surface: `/activity`, `/read-activity` and `/usage` all accept + * explicit `since`/`until`, and the census comes from the same cached + * `useLoreUtilitySummary()` request the cost line and the utility grid share. + * + * ## The row cap is reported, not hidden + * + * `currentCapped` / `previousCapped` say whether either series of that window + * reached PostgREST's row cap (see `isAtRowCap`). The page then says the + * figures may be incomplete instead of drawing the cliff a truncated series + * would show at "now", and drops deltas that would compare a partial window. + * + * ## The window travels with the data + * + * Both hooks keep the previous result on screen while a new range loads + * (`keepPreviousData`). Each result therefore carries the `window` it was + * fetched for, and the page draws grids from THAT window — never re-gridding + * last week's rows onto this month's axis as if the empty days were fact. + */ + +import { useQuery, keepPreviousData } from '@tanstack/react-query'; +import type { ActivityBucket, ReadActivityBucket } from '@lorekit/schemas/memory'; +import type { UsageRun, UsageStatsResponse } from '@lorekit/schemas/usage'; +import { browserAccessToken } from '@/lib/api/session-browser'; +import { activityRequest, readActivityRequest, usageRequest, usageRunsRequest } from '@/lib/api/memories'; +import { isAtRowCap } from '@/lib/insights-loop'; +import { priorWindow } from '@/lib/queries/insights-usage'; +import { bucketPlanFor, type AbsoluteRange, type BucketUnit } from '@/lib/time-range'; + +export interface InsightsLoopWindow { + writes: ActivityBucket[]; + reads: ReadActivityBucket[]; + usage: UsageStatsResponse; +} + +export interface InsightsLoopData { + /** The window these rows were fetched for. */ + window: AbsoluteRange; + unit: BucketUnit; + current: InsightsLoopWindow; + previous: InsightsLoopWindow; + /** A current-window series reached the row cap — its newest buckets may be missing. */ + currentCapped: boolean; + /** A previous-window series reached the row cap — a delta against it is unreliable. */ + previousCapped: boolean; +} + +async function fetchWindow( + token: string, + window: AbsoluteRange, + unit: BucketUnit, + signal?: AbortSignal, +): Promise { + const params = { bucket: unit, since: window.from, until: window.to }; + const [activity, readActivity, usage] = await Promise.all([ + activityRequest(token, params, signal), + readActivityRequest(token, params, signal), + usageRequest(token, { since: window.from, until: window.to }, signal), + ]); + return { writes: activity.buckets, reads: readActivity.buckets, usage }; +} + +const EMPTY_USAGE: UsageStatsResponse = { + range: { since: null, until: null }, + correlation_id: null, + summary: { total_events: 0, reads: 0, writes: 0, other: 0, records_read: 0, archived: 0, expired: 0, by_outcome: {} }, + by_tool: [], + by_scope_type: [], +}; +const EMPTY_WINDOW: InsightsLoopWindow = { writes: [], reads: [], usage: EMPTY_USAGE }; + +function seriesCapped(w: InsightsLoopWindow): boolean { + return isAtRowCap(w.writes.length) || isAtRowCap(w.reads.length); +} + +async function fetchInsightsLoop(window: AbsoluteRange, signal?: AbortSignal): Promise { + const unit = bucketPlanFor(window).unit; + const token = await browserAccessToken(); + if (!token) { + return { window, unit, current: EMPTY_WINDOW, previous: EMPTY_WINDOW, currentCapped: false, previousCapped: false }; + } + const [current, previous] = await Promise.all([ + fetchWindow(token, window, unit, signal), + fetchWindow(token, priorWindow(window), unit, signal), + ]); + return { window, unit, current, previous, currentCapped: seriesCapped(current), previousCapped: seriesCapped(previous) }; +} + +export function useInsightsLoop(window: AbsoluteRange) { + return useQuery({ + queryKey: ['insights-loop', window.from, window.to], + queryFn: ({ signal }) => fetchInsightsLoop(window, signal), + placeholderData: keepPreviousData, + staleTime: 60_000, + }); +} + +/** + * How many pages of runs the page reads before it stops and says so. + * + * 5 × 100 = 500 runs — far beyond what a weekday × hour grid needs to show a + * rhythm, and a bound on what one page view can cost. `truncated` tells the + * components to caption the cut rather than imply the grid is complete. + */ +export const MAX_RUN_PAGES = 5; +const RUNS_PAGE_SIZE = 100; + +export interface WindowRuns { + /** The window these runs were fetched for. */ + window: AbsoluteRange; + /** Most recent first — the route orders by `last_seen desc`. */ + runs: UsageRun[]; + /** More runs exist past {@link MAX_RUN_PAGES}; the OLDEST part of the window is missing. */ + truncated: boolean; +} + +async function fetchWindowRuns(window: AbsoluteRange, signal?: AbortSignal): Promise { + const token = await browserAccessToken(); + if (!token) return { window, runs: [], truncated: false }; + const runs: UsageRun[] = []; + let cursor: string | null = null; + for (let pageIndex = 0; pageIndex < MAX_RUN_PAGES; pageIndex++) { + const runsPage = await usageRunsRequest( + token, + { since: window.from, until: window.to, limit: RUNS_PAGE_SIZE, ...(cursor ? { cursor } : {}) }, + signal, + ); + runs.push(...runsPage.runs); + cursor = runsPage.next_cursor; + if (!cursor) return { window, runs, truncated: false }; + } + return { window, runs, truncated: true }; +} + +export function useWindowRuns(window: AbsoluteRange) { + return useQuery({ + queryKey: ['insights-window-runs', window.from, window.to], + queryFn: ({ signal }) => fetchWindowRuns(window, signal), + placeholderData: keepPreviousData, + staleTime: 60_000, + }); +} diff --git a/packages/web/src/lib/queries/lore-utility.ts b/packages/web/src/lib/queries/lore-utility.ts index 12231085d..b95e93064 100644 --- a/packages/web/src/lib/queries/lore-utility.ts +++ b/packages/web/src/lib/queries/lore-utility.ts @@ -19,6 +19,7 @@ import { useQuery, keepPreviousData } from '@tanstack/react-query'; import { browserAccessToken } from '@/lib/api/session-browser'; import { utilityRequest } from '@/lib/api/memories'; import type { LessonUtilityName, UtilityResponse } from '@lorekit/schemas/memory'; +import { costWindowSince } from '@/lib/lore-cost'; /** * The shape a signed-out render sees. @@ -61,6 +62,18 @@ export function useLoreUtility(window: { since?: string; until?: string }, scope }); } +/** + * The census plus the cost line's 30-day cost — the ONE utility request the + * cost headline, the utility grid and the Insights loop strip share. + * + * Use this rather than `useLoreUtility({})`: an omitted `since` asks the server + * for the all-history cost, the most expensive variant of its slowest query, + * and a second distinct key doubles the requests. See `costWindowSince`. + */ +export function useLoreUtilitySummary() { + return useLoreUtility({ since: costWindowSince(Date.now()) }); +} + /** * One quadrant's rows. * @@ -69,10 +82,14 @@ export function useLoreUtility(window: { since?: string; until?: string }, scope * clusters sidebar takes (a closed panel issues no request). */ export function useLoreUtilityRows(quadrant: LessonUtilityName | null, scope?: string, limit = 20) { + // The server computes the cost on this call too and it is discarded here, so + // bound it to the shared 30-day window rather than letting an omitted `since` + // request the all-history variant on every quadrant click. + const since = costWindowSince(Date.now()); return useQuery({ - queryKey: ['lore-utility', 'rows', quadrant, scope ?? null, limit], + queryKey: ['lore-utility', 'rows', quadrant, scope ?? null, limit, since], queryFn: ({ signal }) => - fetchUtility({ quadrant: quadrant as LessonUtilityName, ...(scope ? { scope } : {}), limit }, signal), + fetchUtility({ quadrant: quadrant as LessonUtilityName, ...(scope ? { scope } : {}), limit, since }, signal), enabled: quadrant !== null, placeholderData: keepPreviousData, staleTime: 90_000, diff --git a/packages/web/src/lib/recall-flow.spec.ts b/packages/web/src/lib/recall-flow.spec.ts new file mode 100644 index 000000000..7cfc56164 --- /dev/null +++ b/packages/web/src/lib/recall-flow.spec.ts @@ -0,0 +1,96 @@ +import { describe, expect, it } from 'vitest'; +import type { UsageStatRow } from '@lorekit/schemas/usage'; +import { recallFlow } from './recall-flow'; + +function row(overrides: Partial): UsageStatRow { + return { + tool_name: 'memory.list', + outcome: 'ok', + scope_type: 'repo', + client: 'mcp', + kind: 'lesson', + host: 'claude', + event_count: 1, + record_count: 1, + total_duration_ms: 10, + ...overrides, + }; +} + +describe('recallFlow', () => { + it('keeps only the four lore-read tools and counts calls, not records', () => { + const flow = recallFlow([ + row({ tool_name: 'memory.list', event_count: 10, record_count: 600 }), + row({ tool_name: 'memory.write', event_count: 99 }), + row({ tool_name: 'memory.scopes', event_count: 50 }), + row({ tool_name: 'org.list', event_count: 50 }), + ]); + expect(flow.totalCalls).toBe(10); + expect(flow.rows).toEqual([{ client: 'mcp', tool: 'memory.list', scopeType: 'repo', calls: 10 }]); + }); + + it('counts only successful calls — a failed read recalled nothing', () => { + const flow = recallFlow([ + row({ tool_name: 'memory.read', event_count: 5 }), + row({ tool_name: 'memory.read', outcome: 'error', event_count: 9 }), + ]); + expect(flow.totalCalls).toBe(5); + }); + + it('names a missing client or scope type "unattributed", and an unknown scope type "other"', () => { + const flow = recallFlow([ + row({ client: null, scope_type: null, event_count: 3 }), + row({ client: 'mcp', scope_type: 'not-a-scope-type', event_count: 2 }), + ]); + expect(flow.rows).toEqual([ + { client: 'unattributed', tool: 'memory.list', scopeType: 'unattributed', calls: 3 }, + { client: 'mcp', tool: 'memory.list', scopeType: 'other', calls: 2 }, + ]); + }); + + it('never folds "unattributed" into "other", and ranks it after the named values', () => { + const flow = recallFlow( + [ + row({ scope_type: 'repo', event_count: 9 }), + row({ scope_type: 'global', event_count: 8 }), + row({ scope_type: 'branch', event_count: 7 }), + row({ scope_type: null, event_count: 1 }), + ], + 1, + ); + const scopes = flow.nodes.filter((n) => n.layer === 2).map((n) => n.label); + expect(scopes).toEqual(['repo', 'unattributed', 'other']); + }); + + it('conserves flow through the middle column', () => { + const flow = recallFlow([ + row({ client: 'mcp', tool_name: 'memory.list', scope_type: 'repo', event_count: 7 }), + row({ client: 'cli', tool_name: 'memory.list', scope_type: 'global', event_count: 3 }), + row({ client: 'dashboard', tool_name: 'memory.read', scope_type: 'branch', event_count: 2 }), + ]); + const into = (id: string) => flow.links.filter((l) => l.target === id).reduce((s, l) => s + l.value, 0); + const out = (id: string) => flow.links.filter((l) => l.source === id).reduce((s, l) => s + l.value, 0); + expect(into('t:memory.list')).toBe(10); + expect(out('t:memory.list')).toBe(10); + expect(into('t:memory.read')).toBe(out('t:memory.read')); + }); + + it('folds clients past the top N into "other", listed last', () => { + const flow = recallFlow( + [ + row({ client: 'a', event_count: 9 }), + row({ client: 'b', event_count: 8 }), + row({ client: 'c', event_count: 1 }), + row({ client: 'd', event_count: 1 }), + ], + 2, + ); + const clients = flow.nodes.filter((n) => n.layer === 0).map((n) => n.label); + expect(clients).toEqual(['a', 'b', 'other']); + expect(flow.rows.find((r) => r.client === 'other')?.calls).toBe(2); + }); + + it('is empty for an account with no reads', () => { + expect(recallFlow([])).toEqual({ nodes: [], links: [], rows: [], totalCalls: 0 }); + }); +}); diff --git a/packages/web/src/lib/recall-flow.ts b/packages/web/src/lib/recall-flow.ts new file mode 100644 index 000000000..8233ef474 --- /dev/null +++ b/packages/web/src/lib/recall-flow.ts @@ -0,0 +1,172 @@ +/** + * The recall flow: which surface recalled lore (client), through which read + * tool, from which kind of scope — as the nodes and links a layered flow chart + * draws (`lib/flow-layout.ts`). + * + * ## Calls, in every band + * + * A flow chart is only honest when every band is the same unit. `/usage` rows + * carry both `event_count` (calls) and `record_count` (records the calls + * returned); one `memory.list` is one call and possibly hundreds of records, so + * mixing them would make the bands disagree with their own nodes. Calls are + * used throughout, because "who is asking" is a question about calls. + * + * ## The four lore-read tools, successful calls only + * + * The same tool set `/memories/read-activity` counts — `memory.read`, + * `memory.list`, `memory.search`, `memory.list_archived`. `memory.scopes`, + * `memory.usage` and the `org.*` reads are catalogue lookups, not recall. Only + * `outcome === 'ok'` counts: a failed read recalled nothing (failures are the + * operational-health section's job). This is NOT the same population as the + * Pulse timeline: that series counts records and excludes dashboard reads, + * this one counts calls and includes them. + * + * ## Dashboard reads are kept + * + * Unlike the verdict banner, which excludes your own browsing, this chart's job + * is to show WHO reads — the dashboard is one of the answers, the same call + * `AgentBreakdown` makes. + * + * ## "unattributed" is a bucket, "other" is a fold + * + * A call with no `scope_type` (mostly `memory.search`, whose `scopes[]` array + * has no single value to record) and a call with no `client` are both + * UNATTRIBUTED — a real, honestly-named answer, shown and never folded away, + * the same rule `ScopeConsumption` follows. "other" means only what it says: a + * scope type outside the known vocabulary, or a value past the top N. + * + * ## Read as two pairings, never as paths + * + * The rows are aggregates per (client, tool, scope_type), and the chart draws + * client → tool and tool → scope type as two separate pairings. A band entering + * a tool and a band leaving it are NOT the same calls continuing; the middle + * node conserves the total, nothing more. The component's caption says so. + */ + +import type { UsageStatRow } from '@lorekit/schemas/usage'; +import { bucketScopeType } from '@/lib/usage-health'; +import type { FlowLinkInput, FlowNodeInput } from '@/lib/flow-layout'; + +export const RECALL_TOOLS: ReadonlySet = new Set([ + 'memory.read', + 'memory.list', + 'memory.search', + 'memory.list_archived', +]); + +/** Named nodes per column before the rest fold into "other". Keeps the palette ≤ 6 hues. */ +export const RECALL_FLOW_TOP_N = 5; + +export const RECALL_FLOW_COLUMNS = ['Who asked', 'Read tool', 'Scope type'] as const; + +export interface RecallFlowRow { + client: string; + tool: string; + scopeType: string; + calls: number; +} + +export interface RecallFlow { + nodes: FlowNodeInput[]; + links: FlowLinkInput[]; + /** The aggregated rows the chart was built from — the "view as table" source. */ + rows: RecallFlowRow[]; + totalCalls: number; +} + +export const OTHER = 'other'; +export const UNATTRIBUTED = 'unattributed'; + +/** `null` client is a genuinely unattributed call — named, not dropped. */ +function clientLabel(client: string | null | undefined): string { + return client ?? UNATTRIBUTED; +} + +/** `null` scope type is unattributed; a non-null value outside the vocabulary is "other". */ +function scopeTypeLabel(scopeType: string | null): string { + return scopeType === null ? UNATTRIBUTED : bucketScopeType(scopeType); +} + +/** Keep the `limit` largest named keys, plus "unattributed" always; everything else folds into "other". */ +function keptKeys(totals: Map, limit: number): Set { + const named = [...totals.entries()] + .filter(([key]) => key !== OTHER && key !== UNATTRIBUTED) + .sort((a, b) => b[1] - a[1] || a[0].localeCompare(b[0])) + .slice(0, limit) + .map(([key]) => key); + return new Set([...named, UNATTRIBUTED]); +} + +function addTo(map: Map, key: string, value: number): void { + map.set(key, (map.get(key) ?? 0) + value); +} + +/** Named values by flow, then "unattributed", then "other" — the two catch-alls never outrank a named value. */ +function rank(key: string): number { + if (key === OTHER) return 2; + if (key === UNATTRIBUTED) return 1; + return 0; +} + +function columnNodes(totals: Map, layer: number, prefix: string): FlowNodeInput[] { + return [...totals.entries()] + .sort((a, b) => rank(a[0]) - rank(b[0]) || b[1] - a[1] || a[0].localeCompare(b[0])) + .map(([key]) => ({ id: `${prefix}:${key}`, label: key, layer })); +} + +export function recallFlow(usageRows: readonly UsageStatRow[], topN = RECALL_FLOW_TOP_N): RecallFlow { + const raw: RecallFlowRow[] = usageRows + .filter((r) => RECALL_TOOLS.has(r.tool_name) && r.outcome === 'ok' && r.event_count > 0) + .map((r) => ({ + client: clientLabel(r.client), + tool: r.tool_name, + scopeType: scopeTypeLabel(r.scope_type), + calls: r.event_count, + })); + + const clientTotals = new Map(); + const scopeTotals = new Map(); + for (const r of raw) { + addTo(clientTotals, r.client, r.calls); + addTo(scopeTotals, r.scopeType, r.calls); + } + const keepClients = keptKeys(clientTotals, topN); + const keepScopes = keptKeys(scopeTotals, topN); + + // Re-aggregate after folding, so the table and the chart share one row set. + const folded = new Map(); + for (const r of raw) { + const client = keepClients.has(r.client) ? r.client : OTHER; + const scopeType = keepScopes.has(r.scopeType) ? r.scopeType : OTHER; + const key = `${client}\u0000${r.tool}\u0000${scopeType}`; + const existing = folded.get(key); + if (existing) existing.calls += r.calls; + else folded.set(key, { client, tool: r.tool, scopeType, calls: r.calls }); + } + const rows = [...folded.values()].sort((a, b) => b.calls - a.calls); + + const clients = new Map(); + const tools = new Map(); + const scopes = new Map(); + const clientTool = new Map(); + const toolScope = new Map(); + for (const r of rows) { + addTo(clients, r.client, r.calls); + addTo(tools, r.tool, r.calls); + addTo(scopes, r.scopeType, r.calls); + addTo(clientTool, `c:${r.client}\u0000t:${r.tool}`, r.calls); + addTo(toolScope, `t:${r.tool}\u0000s:${r.scopeType}`, r.calls); + } + + const links: FlowLinkInput[] = [...clientTool, ...toolScope].map(([key, value]) => { + const [source = '', target = ''] = key.split('\u0000'); + return { source, target, value }; + }); + + return { + nodes: [...columnNodes(clients, 0, 'c'), ...columnNodes(tools, 1, 't'), ...columnNodes(scopes, 2, 's')], + links, + rows, + totalCalls: rows.reduce((sum, r) => sum + r.calls, 0), + }; +} diff --git a/packages/web/src/lib/runs-rhythm.spec.ts b/packages/web/src/lib/runs-rhythm.spec.ts new file mode 100644 index 000000000..e6333f030 --- /dev/null +++ b/packages/web/src/lib/runs-rhythm.spec.ts @@ -0,0 +1,126 @@ +import { describe, expect, it } from 'vitest'; +import type { UsageRun } from '@lorekit/schemas/usage'; +import { MAX_LANE_ROWS, localParts, packRows, rhythmGrid, runLane, swimlanes } from './runs-rhythm'; + +function run(overrides: Partial): UsageRun { + return { + correlation_id: 'session:1', + session_kind: 'local', + first_seen: '2026-06-15T09:00:00.000Z', + last_seen: '2026-06-15T10:00:00.000Z', + read_events: 1, + records_read: 10, + write_events: 0, + distinct_scopes: 1, + total_duration_ms: 100, + ...overrides, + }; +} + +describe('localParts', () => { + it('places an instant in the given time zone, Monday-first', () => { + // 2026-06-15 is a Monday. 23:30 UTC is already Tuesday 01:30 in Copenhagen (UTC+2). + expect(localParts('2026-06-15T23:30:00.000Z', 'UTC')).toEqual({ weekday: 0, hour: 23 }); + expect(localParts('2026-06-15T23:30:00.000Z', 'Europe/Copenhagen')).toEqual({ weekday: 1, hour: 1 }); + }); + + it('returns null for an unparseable instant', () => { + expect(localParts('not-a-date', 'UTC')).toBeNull(); + }); +}); + +describe('rhythmGrid', () => { + it('counts runs by local start hour, not by every hour they were active', () => { + const grid = rhythmGrid( + [ + run({ first_seen: '2026-06-15T09:10:00.000Z', last_seen: '2026-06-15T12:00:00.000Z' }), + run({ first_seen: '2026-06-15T09:50:00.000Z' }), + run({ first_seen: '2026-06-17T14:00:00.000Z' }), + ], + 'UTC', + ); + expect(grid.cells[0]?.[9]).toBe(2); + expect(grid.cells[0]?.[10]).toBe(0); + expect(grid.cells[2]?.[14]).toBe(1); + expect(grid).toMatchObject({ max: 2, total: 3 }); + }); + + it('is all zeros for no runs', () => { + expect(rhythmGrid([], 'UTC')).toMatchObject({ max: 0, total: 0 }); + }); +}); + +describe('runLane', () => { + it('maps a null or unknown session kind to the unlabelled lane', () => { + expect(runLane('ci')).toBe('ci'); + expect(runLane(null)).toBe('unknown'); + expect(runLane('cron')).toBe('unknown'); + }); +}); + +describe('swimlanes', () => { + const window = { from: '2026-06-15T00:00:00.000Z', to: '2026-06-16T00:00:00.000Z' }; + + it('places runs as fractions of the window, only in lanes that have runs, in lane order', () => { + const result = swimlanes( + [ + run({ session_kind: 'pr', first_seen: '2026-06-15T12:00:00.000Z', last_seen: '2026-06-15T18:00:00.000Z', records_read: 40 }), + run({ session_kind: 'local', first_seen: '2026-06-15T06:00:00.000Z', last_seen: '2026-06-15T06:00:00.000Z' }), + ], + window, + ); + expect(result.lanes.map((l) => l.lane)).toEqual(['local', 'pr']); + expect(result.lanes[1]?.runs[0]).toMatchObject({ x0: 0.5, x1: 0.75 }); + expect(result.lanes[0]?.runs[0]).toMatchObject({ x0: 0.25, x1: 0.25 }); + expect(result.maxRecordsRead).toBe(40); + }); + + it('clamps a run that reaches outside the window instead of dropping it', () => { + const result = swimlanes( + [run({ first_seen: '2026-06-14T20:00:00.000Z', last_seen: '2026-06-15T06:00:00.000Z' })], + window, + ); + expect(result.lanes[0]?.runs[0]).toMatchObject({ x0: 0, x1: 0.25 }); + }); + + it('stacks overlapping runs into sub-rows and reports how many rows the lane needs', () => { + const result = swimlanes( + [ + run({ correlation_id: 'a', first_seen: '2026-06-15T01:00:00.000Z', last_seen: '2026-06-15T20:00:00.000Z' }), + run({ correlation_id: 'b', first_seen: '2026-06-15T02:00:00.000Z', last_seen: '2026-06-15T03:00:00.000Z' }), + run({ correlation_id: 'c', first_seen: '2026-06-15T06:00:00.000Z', last_seen: '2026-06-15T07:00:00.000Z' }), + ], + window, + ); + const lane = result.lanes[0]; + expect(lane?.rows).toBe(2); + const rowOf = (id: string) => lane?.runs.find((r) => r.run.correlation_id === id)?.row; + expect(rowOf('a')).toBe(0); + // b overlaps a, so it goes below; c starts after b ends, so it reuses b's row. + expect(rowOf('b')).toBe(1); + expect(rowOf('c')).toBe(1); + }); + + it('returns no lanes for a degenerate window', () => { + expect(swimlanes([run({})], { from: window.to, to: window.from }).lanes).toEqual([]); + }); +}); + +describe('packRows', () => { + const r = (x0: number, x1: number) => ({ run: run({}), lane: 'local' as const, x0, x1 }); + + it('caps a lane at MAX_LANE_ROWS, overlapping in the row that frees up first', () => { + const many = Array.from({ length: MAX_LANE_ROWS + 2 }, (_, i) => r(0.1 + i * 0.001, 0.9 - i * 0.01)); + const packed = packRows(many); + expect(Math.max(...packed.map((p) => p.row))).toBe(MAX_LANE_ROWS - 1); + }); + + it('counts an instant run as its drawn width, so two near-simultaneous bars do not overlap on screen', () => { + const packed = packRows([r(0.5, 0.5), r(0.505, 0.505)]); + expect(packed.map((p) => p.row)).toEqual([0, 1]); + }); + + it('reuses a row once its last run has ended', () => { + expect(packRows([r(0.1, 0.2), r(0.3, 0.4)]).map((p) => p.row)).toEqual([0, 0]); + }); +}); diff --git a/packages/web/src/lib/runs-rhythm.ts b/packages/web/src/lib/runs-rhythm.ts new file mode 100644 index 000000000..d99cb000f --- /dev/null +++ b/packages/web/src/lib/runs-rhythm.ts @@ -0,0 +1,196 @@ +/** + * Runs as time: a weekday × hour rhythm and swimlanes per session kind, both + * from `GET /memories/usage/runs`. + * + * ## Why runs, and not the hourly activity series + * + * A "when do my agents work" heatmap needs hourly data across the whole window. + * `/memories/activity` and `/read-activity` can return hourly buckets, but one + * row per (hour, scope[, read_kind]) — a week of hours across a dozen scopes is + * thousands of rows, past PostgREST's 1000-row `max_rows`, and the RPC drops the + * newest rows silently. `/usage/runs` is keyset-paginated instead, so every page + * is complete and the page count is the only bound, which the caller states. + * + * The cost of that choice is coverage: a run is a `correlation_id`, which the + * LoreKit hooks and MCP clients stamp on their calls. A call that carries none + * is in the activity series but not here. The components caption this. + * + * ## The viewer's time zone, injected + * + * Buckets are placed in the time zone passed in (the browser's, in the page), + * never the server's UTC: "agents mostly work at 14:00" is a statement about + * the reader's day. The zone is a parameter rather than read inside so the + * specs are deterministic on any machine. + */ + +import type { UsageRun } from '@lorekit/schemas/usage'; +import type { AbsoluteRange } from '@/lib/time-range'; + +/** Monday-first, matching the contribution heatmap's rows. */ +export const WEEKDAYS = ['Mon', 'Tue', 'Wed', 'Thu', 'Fri', 'Sat', 'Sun'] as const; + +const WEEKDAY_INDEX: Record = { Mon: 0, Tue: 1, Wed: 2, Thu: 3, Fri: 4, Sat: 5, Sun: 6 }; + +const partsFormatters = new Map(); + +function formatterFor(timeZone: string): Intl.DateTimeFormat { + let formatter = partsFormatters.get(timeZone); + if (!formatter) { + formatter = new Intl.DateTimeFormat('en-US', { timeZone, weekday: 'short', hour: 'numeric', hourCycle: 'h23' }); + partsFormatters.set(timeZone, formatter); + } + return formatter; +} + +/** Weekday (0 = Monday) and hour (0–23) of an instant in `timeZone`, or `null` if unparseable. */ +export function localParts(iso: string, timeZone: string): { weekday: number; hour: number } | null { + const t = Date.parse(iso); + if (Number.isNaN(t)) return null; + const parts = formatterFor(timeZone).formatToParts(new Date(t)); + const weekday = WEEKDAY_INDEX[parts.find((p) => p.type === 'weekday')?.value ?? '']; + const hour = Number(parts.find((p) => p.type === 'hour')?.value); + if (weekday === undefined || Number.isNaN(hour)) return null; + return { weekday, hour: hour % 24 }; +} + +export interface RhythmGrid { + /** `cells[weekday][hour]` — runs that STARTED in that local hour. */ + cells: number[][]; + max: number; + total: number; +} + +/** + * Runs started per local weekday × hour. + * + * Counted by start (`first_seen`) rather than spread across every hour a run + * was active: a run is one decision to start work, and spreading a three-hour + * CI job over three cells would weight long runs as if they were three runs. + */ +export function rhythmGrid(runs: readonly UsageRun[], timeZone: string): RhythmGrid { + const cells = WEEKDAYS.map(() => Array.from({ length: 24 }, () => 0)); + let total = 0; + for (const run of runs) { + const parts = localParts(run.first_seen, timeZone); + const row = parts ? cells[parts.weekday] : undefined; + if (!parts || !row) continue; + row[parts.hour] = (row[parts.hour] ?? 0) + 1; + total += 1; + } + const max = Math.max(0, ...cells.flat()); + return { cells, max, total }; +} + +/** The lanes, in display order. `unknown` covers a null or unrecognised `session_kind`. */ +export const RUN_LANES = ['local', 'ci', 'pr', 'unknown'] as const; +export type RunLane = (typeof RUN_LANES)[number]; + +export const RUN_LANE_LABEL: Record = { + local: 'Local sessions', + ci: 'CI jobs', + pr: 'PR automations', + unknown: 'Unlabelled', +}; + +export function runLane(sessionKind: string | null): RunLane { + return sessionKind === 'local' || sessionKind === 'ci' || sessionKind === 'pr' ? sessionKind : 'unknown'; +} + +export interface SwimlaneRun { + run: UsageRun; + lane: RunLane; + /** Start and end as fractions of the window, clamped to [0, 1]. */ + x0: number; + x1: number; + /** Sub-row inside the lane (0 = top), so overlapping runs sit side by side, not on top of each other. */ + row: number; +} + +export interface SwimlaneLane { + lane: RunLane; + runs: SwimlaneRun[]; + /** How many sub-rows the lane needs — its drawn height. */ + rows: number; +} + +export interface Swimlanes { + lanes: SwimlaneLane[]; + /** The largest `records_read` of any placed run — the bar-thickness scale. */ + maxRecordsRead: number; +} + +/** + * The most sub-rows one lane may grow to. Past it, a run goes into the row that + * frees up first and overlaps there — a 40-row lane would push the rest of the + * page off screen to show runs that are better read in the list below. + */ +export const MAX_LANE_ROWS = 6; + +/** + * The narrowest a run counts as when packing, as a fraction of the window. + * + * Bars have a minimum drawn width (a run shorter than a pixel still has to be + * seen), so two instant runs a hair apart would overlap on screen while their + * real intervals do not. Packing by this floor keeps what is DRAWN from + * overlapping, not just what is measured. + */ +export const MIN_PACKED_SPAN = 0.012; + +/** + * Assign each run a sub-row: greedy interval packing, earliest start first, into + * the first row whose last run has ended. Runs are placed in time order, so the + * result is stable for a given input. + */ +export function packRows(runs: readonly Omit[], maxRows = MAX_LANE_ROWS): SwimlaneRun[] { + const rowEnds: number[] = []; + return [...runs] + .sort((a, b) => a.x0 - b.x0 || a.x1 - b.x1) + .map((bar) => { + const end = Math.max(bar.x1, bar.x0 + MIN_PACKED_SPAN); + let row = rowEnds.findIndex((rowEnd) => rowEnd <= bar.x0); + if (row === -1 && rowEnds.length < maxRows) row = rowEnds.length; + // Every row is busy and the lane is full: overlap in the row that frees up first. + if (row === -1) row = rowEnds.indexOf(Math.min(...rowEnds)); + rowEnds[row] = Math.max(rowEnds[row] ?? 0, end); + return { ...bar, row }; + }); +} + +/** + * Place runs on the window's time axis, one lane per session kind. + * + * Only lanes with runs are returned, so an account that never runs CI does not + * carry an empty "CI jobs" row. A run reaching outside the window is clamped to + * its edge rather than dropped: it did touch lore inside the window. + */ +export function swimlanes(runs: readonly UsageRun[], window: AbsoluteRange): Swimlanes { + const from = Date.parse(window.from); + const span = Date.parse(window.to) - from; + let maxRecordsRead = 0; + if (Number.isNaN(from) || !(span > 0)) return { lanes: [], maxRecordsRead }; + + const clamp = (v: number) => Math.min(1, Math.max(0, v)); + const byLane = new Map[]>(); + for (const run of runs) { + const start = Date.parse(run.first_seen); + const end = Date.parse(run.last_seen); + if (Number.isNaN(start) || Number.isNaN(end)) continue; + const x0 = clamp((start - from) / span); + const x1 = clamp((Math.max(start, end) - from) / span); + const lane = runLane(run.session_kind); + const list = byLane.get(lane) ?? []; + list.push({ run, lane, x0, x1 }); + byLane.set(lane, list); + maxRecordsRead = Math.max(maxRecordsRead, run.records_read); + } + + return { + lanes: RUN_LANES.flatMap((lane) => { + const unpacked = byLane.get(lane); + if (!unpacked) return []; + const laneRuns = packRows(unpacked); + return [{ lane, runs: laneRuns, rows: Math.max(1, ...laneRuns.map((bar) => bar.row + 1)) }]; + }), + maxRecordsRead, + }; +}