diff --git a/docs/key-files.md b/docs/key-files.md index 971f92927..918f04ef9 100644 --- a/docs/key-files.md +++ b/docs/key-files.md @@ -186,8 +186,10 @@ else. Grouped loosely by subsystem in migration/PR order. | `packages/mcp-core/src/memory/read-refs.ts` (+ its edge mirror) | The pure core behind batch `memory.read` (`refs`), shared by MCP's `toolRead` and REST's `handleRead`. Import-free by construction — it takes only already-PARSED `{scope,key}` pairs, never re-deriving the `scope::key` grammar `parseMemoryRefs` alone owns. `groupRefsByScope` collapses N refs into one `{scope, keys}` group per DISTINCT scope (plan D5) — the transport issues exactly one `.eq('scope', s).in('key', keys)` query per group, awaited concurrently, instead of one query per ref. `isQueryableKey` rejects a key carrying a comma/paren/quote/backslash BEFORE it reaches the query, since postgrest-js's `.in()` filter quotes a comma/paren but does not escape a quote or backslash inside the value — a bad ref becomes a `missing` entry rather than a malformed filter list. `missingRefs` reports every unresolved ref as its own caller-supplied `scope::key` string, in REQUEST order — but it only ever sees what `parseMemoryRefs` RETURNED, so a ref dropped as malformed or truncated past the 32-ref cap reaches neither `entries` nor `missing` (the gap is visible as `lorekit.refs.requested` vs `lorekit.refs.count`); matching stays against the VERBATIM stored scope, deliberately diverging from the singular `memory.read` path's `validateScope` normalisation, because the transports differ in what they hand `memory_write` and no reader can infer how a row was cased — see `docs/decisions.md#memoryread-batching-is-conditional-verbatim-scoped-and-inflates-a-signal-it-doesnt-count` | | `supabase/migrations/00108_retention_conditions_on_aggregates.sql` | Gives `lorekit_memory_facets` (24→31 args), `lorekit_memory_activity` (26→31) and `lorekit_memory_pivot` (27→34) the five retention thresholds `lorekit_memory_list` has had since 00092, plus a `created_since`/`created_until` pair on facets and pivot (activity's own `since`/`until` already bound `created_at`). **The bug it fixes is a DISAGREEMENT, not a missing feature**: only the list applied the thresholds, so `max_opened_count => 0` narrowed the Explorer's rows and left its facet counts, stat cards and matrix counting the un-narrowed population — every number on the page describing a different set from the one below it. Introduces `lorekit_match_retention`, ONE inlinable row test now composed by all four readers, so a fourth inline copy cannot drift (00101 records that exact drift happening once already between two copies). It takes CUTOFF INSTANTS rather than day counts, which is what keeps it `immutable` and index-usable — each caller resolves its `now()`-relative bounds once per query in a declare block — and it holds 00100's never-opened rule (`coalesce(last_opened_at, created_at)`) in one place. **Applied in the `base` CTE's WHERE, never as an `ok_*` flag**, so a threshold narrows even the SELF-EXCLUDED dimension: self-exclusion exists so the dimension you are standing in still lists what you could switch to, and switching `host` does not stop you looking at lore older than 30 days. The three aggregates are DROPPED at their exact old signatures and recreated (`create or replace` cannot change a signature, and leaving both overloads makes every PostgREST named-arg call ambiguous — 00092's footgun); `lorekit_memory_list` is a true `create or replace` at its verbatim 00105 signature, so its grants persist. Pinned by §108 | | `supabase/migrations/00113_explorer_reverse_activity_filters_and_opened_sort.sql` | The Explorer's REVERSE directions: `opened_within_days` and the `min_seen_count`/`min_read_count`/`min_opened_count` floors, the mirror of four of the five retention thresholds, for "which lore IS used?" rather than "what can I archive?". Widens `lorekit_match_retention` 10→14 args (the 10-arg form is dropped LAST, after every caller is recreated), appends the four to `_facets`/`_activity`/`_pivot` and, with a `p_cursor_count`, to `lorekit_memory_list`, which also accepts `p_sort => 'opened_count'` (the format() splices `$44` or `$29` as the keyset placeholder). `opened_within_days` does NOT age a never-opened lesson from `created_at` — the opposite of 00100's rule for `unseen_days`, on purpose. READ-only: no groom RPC or `GroomConditionsSchema` change. Restores the `anon` revoke on `lorekit_memory_list` that 00105's drop-and-recreate lost. Pinned by §113; §108's helper-ACL check now derives the signature | +| `supabase/migrations/00115_explorer_recent_read_sort.sql` | The Explorer's **Recent read** order: `lorekit_memory_list(p_sort => 'last_opened_at')` orders by the last DELIBERATE agent read (00099's column, not the bulk-moved `last_read_at`), `desc NULLS LAST` with `id desc` breaking ties, so never-opened lessons trail as a tail rather than leading or vanishing (the total is unchanged). Same signature as 00114 (`create or replace`): the nullable keyset rides in `p_cursor_ts`/`p_cursor_id`, where a NULL timestamp with a set id means "inside the never-opened tail". The keyset predicate and ORDER BY are built per sort in the declare block. Pinned by §115 | | `supabase/migrations/00114_exclude_tags_dimension.sql` | `p_exclude_tags text[]` appended to `lorekit_memory_list`/`_facets`/`_activity`/`_pivot` (each dropped at its exact 00113 signature first): an always-applied "carries none of these labels" conjunct, independent of `p_tags_mode`, reusing `lorekit_match_tags(…, 'none')` — so "tagged a AND NOT tagged b" is expressible. In `_facets` it is an `ok_tag_exclusion` flag the tag facet NEVER self-excludes (an excluded label stays listed at `count = 0`); `_pivot` narrows `base`; the list binds it as `$45`. The dashboard sends it only when a tag include is also active — a lone exclusion still travels as `tags_mode=none`, and retention policies never gain it. Pinned by §114 | | `packages/mcp-core/src/rest/count-cursor.ts` | The `sort=opened_count` keyset cursor (`{ id, opened_count }`, base64url) and its `buildCountPage`. Separate from `_shared/api/paginate.ts` because that cursor carries a timestamp and is byte-mirrored into the MCP function, which has no count sort. Mirrored verbatim into `supabase/functions/_shared/api/count-cursor.ts`, guarded by `edge-parity.spec.ts`; tested by `count-cursor.spec.ts` | +| `packages/mcp-core/src/rest/last-opened-cursor.ts` | The `sort=last_opened_at` keyset cursor (`{ id, last_opened_at }`, base64url, the timestamp NULLABLE because a page can end inside the never-opened tail) and its `buildLastOpenedPage`. Its own codec for `count-cursor.ts`'s reason: `paginate.ts` rejects a missing timestamp and is byte-mirrored into the MCP function. Requires the `last_opened_at` KEY, so it rejects the date and count cursors (and they reject it). Mirrored verbatim into `supabase/functions/_shared/api/last-opened-cursor.ts`, guarded by `edge-parity.spec.ts`; tested by `last-opened-cursor.spec.ts` | | `supabase/functions/_shared/api/retention.ts` | 00108's fix one layer up: `retentionFrom` picks the five thresholds off a validated query or body, `retentionRpcParams` maps them to the `p_*` arguments all four RPCs share. Used by `list.ts` and the three aggregate handlers, so a route cannot be given the parameter and then forget to forward it. `?? null` and never a truthiness check — `max_opened_count => 0` is the whole point of 00105, so a falsy test turns the most useful threshold in the set into no filter at all | | `packages/web/src/lib/retention-filter.ts` → `retentionConditionsToAggregateBody` | The client half. Delegates to `retentionConditionsToListBody` (the field names are identical by construction — one shared `retentionConditionBodyFields` spread in the schemas) but is typed against `ListFacetsBody`, so a schema that stops carrying one of the five is a TYPE error rather than a threshold the aggregate silently ignores. Its spec reads each field back off the PARSED schema output rather than asserting `safeParse(...).success`: these schemas are not `.strict()`, so zod strips an unknown key and still reports success — a success-only check would pass against the pre-00108 routes and prove nothing | | `packages/cli/src/core/lessons.mjs` → `citationHint` | The Stop-hook ask. Reads the session's already-recorded shown-set and names those `scope::key` refs, capped at `CITED_HINT_MAX` (8) with a `(+N more)` tail — an agent picking from a list is a different task from one recalling a convention. Empty when nothing was injected (with no candidates the ask is an instruction to invent references). Refs are taken in INJECTION order, so the SessionStart set leads | diff --git a/docs/otel.md b/docs/otel.md index be24f8597..15b5d15a0 100644 --- a/docs/otel.md +++ b/docs/otel.md @@ -1015,7 +1015,7 @@ attribute here. | `lore.memory_archive_toggled` | A memory was archived or restored | `lorekit.lore.memory.archive_action`, `.archive_outcome` | | `lore.filter_changed` | Any filter-bar mutation, age/activity thresholds included | `lorekit.lore.filter.action`, `.field` (optional), `.operator` (optional), `.count`, `.retention_count` | | `lore.status_changed` | Active / Archived / Expiring | `lorekit.lore.status` | -| `lore.sort_changed` | The list order was picked (Newest / Most chosen) | `lorekit.lore.sort` (`created_at`/`opened_count`) | +| `lore.sort_changed` | The list order was picked (Newest / Most chosen / Recent read) | `lorekit.lore.sort` (`created_at`/`opened_count`/`last_opened_at`) | | `lore.search_committed` | The debounced search settled | `lorekit.lore.search.length` | | `lore.scope_selected` | A scope chip or Browse-all row was picked | `lorekit.lore.scope.type`, `.source` (`strip`/`browse-all`) | | `lore.range_changed` | Any of the FIVE controls that write `?range=` | `lorekit.lore.range.preset`, `.source`, `.span_days` (omitted when unbounded) | diff --git a/packages/cli/src/shared/deeplink-pure.mjs b/packages/cli/src/shared/deeplink-pure.mjs index 786a84edc..dbeae8efb 100644 --- a/packages/cli/src/shared/deeplink-pure.mjs +++ b/packages/cli/src/shared/deeplink-pure.mjs @@ -50,7 +50,8 @@ export const LORE_PARAM_DEFAULTS = { // five retention thresholds plus, since 00113, their four reverse directions). // `null` means no narrowing; absent from the URL when unset. retention: null, - // 'created_at' | 'opened_count' | null — the list order (00113). `null` is + // 'created_at' | 'opened_count' | 'last_opened_at' | null — the list order + // (00113, 00115). `null` is // Newest, the default, so an unsorted link carries no `sort` param. sort: null, // 'active' | 'archived' | 'expiring' | null — the Explorer's Status control. diff --git a/packages/cli/src/shared/mirror-pairs.mjs b/packages/cli/src/shared/mirror-pairs.mjs index ef44caf82..c342f3f6d 100644 --- a/packages/cli/src/shared/mirror-pairs.mjs +++ b/packages/cli/src/shared/mirror-pairs.mjs @@ -76,6 +76,7 @@ export const mirrorPairs = [ { core: 'packages/mcp-core/src/limits/memory-quota.ts', edge: 'supabase/functions/_shared/limits/memory-quota.ts', driftChecked: true }, { core: 'packages/mcp-core/src/rest/cors-origins.ts', edge: 'supabase/functions/_shared/api/cors-origins.ts', driftChecked: true }, { core: 'packages/mcp-core/src/rest/count-cursor.ts', edge: 'supabase/functions/_shared/api/count-cursor.ts', driftChecked: true }, + { core: 'packages/mcp-core/src/rest/last-opened-cursor.ts', edge: 'supabase/functions/_shared/api/last-opened-cursor.ts', driftChecked: true }, { core: 'packages/mcp-core/src/scope/scope-type-attribute.ts', edge: 'supabase/functions/_shared/scope/scope-type-attribute.ts', driftChecked: true }, // Which row an UNSCOPED read resolves to. Import-free on both sides (it only // parses the scope prefix and compares strings), so the byte comparison diff --git a/packages/mcp-core/src/rest/last-opened-cursor.spec.ts b/packages/mcp-core/src/rest/last-opened-cursor.spec.ts new file mode 100644 index 000000000..9d7c8edcf --- /dev/null +++ b/packages/mcp-core/src/rest/last-opened-cursor.spec.ts @@ -0,0 +1,90 @@ +import { describe, it, expect } from 'vitest'; +import { + buildLastOpenedPage, + decodeLastOpenedCursor, + encodeLastOpenedCursor, +} from './last-opened-cursor.ts'; +import { decodeCountCursor, encodeCountCursor } from './count-cursor.ts'; + +const ID = '0f8fad5b-d9cb-469f-a165-70867728950e'; +const ID2 = '7c9e6679-7425-40de-944b-e07fc1f90ae7'; +const TS = '2026-10-20T09:15:00.123+00:00'; + +/** A `paginate.ts`-shaped date cursor, built the way that module builds one. */ +function dateCursor(id: string, field: 'updated_at' | 'created_at', ts: string): string { + return btoa(JSON.stringify({ id, [field]: ts })).replace(/\+/g, '-').replace(/\//g, '_').replace(/=/g, ''); +} + +describe('last_opened_at cursor codec', () => { + it('round-trips an id and a timestamp', () => { + expect(decodeLastOpenedCursor(encodeLastOpenedCursor(ID, TS))).toEqual({ id: ID, ts: TS }); + }); + + it('keeps a NULL timestamp — a page that ended inside the never-opened tail is a real boundary', () => { + expect(decodeLastOpenedCursor(encodeLastOpenedCursor(ID, null))).toEqual({ id: ID, ts: null }); + }); + + it('emits URL-safe text with no padding', () => { + expect(encodeLastOpenedCursor(ID, TS)).toMatch(/^[A-Za-z0-9_-]+$/); + expect(encodeLastOpenedCursor(ID, null)).toMatch(/^[A-Za-z0-9_-]+$/); + }); + + it.each([ + ['an updated_at date cursor', dateCursor(ID, 'updated_at', TS)], + ['a created_at date cursor', dateCursor(ID, 'created_at', TS)], + ['an opened_count cursor', encodeCountCursor(ID, 3)], + ])('refuses %s, so flipping sort mid-pagination restarts at page one', (_label, raw) => { + expect(decodeLastOpenedCursor(raw)).toBeNull(); + }); + + it('is refused by the count decoder in turn', () => { + expect(decodeCountCursor(encodeLastOpenedCursor(ID, TS))).toBeNull(); + expect(decodeCountCursor(encodeLastOpenedCursor(ID, null))).toBeNull(); + }); + + it.each([ + ['not base64 at all', '%%%'], + ['valid base64, not JSON', btoa('hello')], + ['JSON, not an object', btoa('42')], + ['JSON null', btoa('null')], + ['the key absent (an id alone is not this cursor)', btoa(JSON.stringify({ id: ID }))], + ['non-uuid id', btoa(JSON.stringify({ id: 'abc', last_opened_at: TS }))], + ['numeric timestamp', btoa(JSON.stringify({ id: ID, last_opened_at: 1_760_000_000 }))], + ['free-text timestamp', btoa(JSON.stringify({ id: ID, last_opened_at: 'yesterday' }))], + ['a timestamp carrying a PostgREST operator', btoa(JSON.stringify({ id: ID, last_opened_at: ',id.gt.0' }))], + ])('fails closed on %s', (_label, raw) => { + expect(decodeLastOpenedCursor(raw)).toBeNull(); + }); +}); + +describe('buildLastOpenedPage', () => { + const rows = [ + { id: ID, last_opened_at: TS }, + { id: ID2, last_opened_at: null }, + { id: '00000000-0000-4000-8000-000000000001', last_opened_at: null }, + ]; + + it('reports more pages from the overflow row and cursors on the LAST returned row', () => { + const page = buildLastOpenedPage(rows, 1); + expect(page.entries).toHaveLength(1); + expect(page.hasMore).toBe(true); + expect(decodeLastOpenedCursor(page.nextCursor ?? '')).toEqual({ id: ID, ts: TS }); + }); + + it('mints a NULL-timestamp cursor when the page ends inside the never-opened tail', () => { + const page = buildLastOpenedPage(rows, 2); + expect(page.hasMore).toBe(true); + expect(decodeLastOpenedCursor(page.nextCursor ?? '')).toEqual({ id: ID2, ts: null }); + }); + + it('has no cursor on the last page', () => { + const page = buildLastOpenedPage(rows, 5); + expect(page.hasMore).toBe(false); + expect(page.nextCursor).toBeNull(); + }); + + it('reads a missing column as never opened rather than minting an undecodable cursor', () => { + const page = buildLastOpenedPage([{ id: ID }, { id: ID2 }], 1); + expect(decodeLastOpenedCursor(page.nextCursor ?? '')).toEqual({ id: ID, ts: null }); + }); +}); diff --git a/packages/mcp-core/src/rest/last-opened-cursor.ts b/packages/mcp-core/src/rest/last-opened-cursor.ts new file mode 100644 index 000000000..390a47e48 --- /dev/null +++ b/packages/mcp-core/src/rest/last-opened-cursor.ts @@ -0,0 +1,84 @@ +/** + * The keyset cursor for the `sort=last_opened_at` list — "Recent read" + * (migration 00115): the lessons an agent deliberately read most recently, + * first, and the never-opened ones after them. + * + * Pure and shared (mirrored into `supabase/functions/_shared/api/last-opened-cursor.ts`, + * guarded by `edge-parity.spec.ts`) so the codec is unit-tested here and the + * deployed copy cannot drift from what was tested. + * + * Why it is not one more case in `_shared/api/paginate.ts`: that cursor's + * timestamp is never null (both date sorts are NOT NULL columns), its decoder + * rejects a missing one, and it is byte-mirrored into the MCP function, which + * has no such sort. `last_opened_at` is NULL for a lesson nothing ever opened, + * and a page can end inside that never-opened tail — so this cursor must carry + * a null timestamp, and say so explicitly: `{ id, last_opened_at: null }`. + * + * The three formats cannot be mistaken for each other. `paginate.ts`'s decoder + * finds neither `updated_at` nor `created_at` here, `count-cursor.ts`'s finds + * no integer `opened_count`, and this decoder requires the `last_opened_at` + * KEY to be present (with a timestamp or `null`) — so a caller who flips + * `sort` mid-pagination restarts at page one rather than receiving a page cut + * at the wrong place. + */ + +/** + * The decoded keyset position: the last row's id and its `last_opened_at` — + * `null` when that row was never opened, i.e. the page ended inside the tail. + */ +export interface LastOpenedCursor { + id: string; + ts: string | null; +} + +/** A cursor's `id` is a `memories` primary key, which is a `uuid`. */ +const UUID_RE = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i; + +/** The same leading shape `paginate.ts` requires of its timestamp. */ +const TS_RE = /^\d{4}-\d{2}-\d{2}[T ]\d{2}:\d{2}:\d{2}/; + +/** base64url without padding — the same alphabet `paginate.ts` uses. */ +export function encodeLastOpenedCursor(id: string, ts: string | null): string { + return btoa(JSON.stringify({ id, last_opened_at: ts })) + .replace(/\+/g, '-') + .replace(/\//g, '_') + .replace(/=/g, ''); +} + +/** + * Decode a Recent read cursor, or `null` for anything that is not one. + * + * Fails CLOSED: the values reach the RPC as typed parameters (`uuid`, + * `timestamptz`), so a malformed one would be a 500 rather than an injection — + * and the right answer to a cursor that does not parse is the first page, the + * same contract `paginate.ts` and `count-cursor.ts` keep. + */ +export function decodeLastOpenedCursor(raw: string): LastOpenedCursor | null { + try { + const b64 = raw.replace(/-/g, '+').replace(/_/g, '/'); + const pad = (4 - (b64.length % 4)) % 4; + const parsed: unknown = JSON.parse(atob(b64 + '='.repeat(pad))); + if (!parsed || typeof parsed !== 'object' || !('last_opened_at' in parsed)) return null; + const { id, last_opened_at: ts } = parsed as { id?: unknown; last_opened_at?: unknown }; + if (typeof id !== 'string' || !UUID_RE.test(id)) return null; + if (ts !== null && (typeof ts !== 'string' || !TS_RE.test(ts))) return null; + return { id, ts }; + } catch { + return null; + } +} + +/** + * The page and its next cursor, from `limit + 1` rows — the overflow row is how + * `hasMore` is known, exactly as `paginate.ts`'s `buildPage` reads it. + */ +export function buildLastOpenedPage( + rows: T[], + limit: number, +): { entries: T[]; hasMore: boolean; nextCursor: string | null } { + const hasMore = rows.length > limit; + const entries = hasMore ? rows.slice(0, limit) : rows; + const last = entries[entries.length - 1]; + const nextCursor = hasMore && last ? encodeLastOpenedCursor(last.id, last.last_opened_at ?? null) : null; + return { entries, hasMore, nextCursor }; +} diff --git a/packages/schemas/src/domain/memory-body.spec.ts b/packages/schemas/src/domain/memory-body.spec.ts index c0cfd54a0..a8dd5510e 100644 --- a/packages/schemas/src/domain/memory-body.spec.ts +++ b/packages/schemas/src/domain/memory-body.spec.ts @@ -55,6 +55,15 @@ describe('ListMemoriesBodySchema', () => { expect(ListMemoriesBodySchema.safeParse({ sort: 'read_count' }).success).toBe(false); }); + it('accepts the last_opened_at ("Recent read") sort on both transports (00115)', () => { + expect(ListMemoriesBodySchema.parse({ sort: 'last_opened_at' }).sort).toBe('last_opened_at'); + expect(ListMemoriesQuerySchema.parse({ sort: 'last_opened_at' }).sort).toBe('last_opened_at'); + // `last_read_at` is the BROADER column (bulk list/search pages move it) and + // is deliberately not an order: "Recent read" means a deliberate read. + expect(ListMemoriesBodySchema.safeParse({ sort: 'last_read_at' }).success).toBe(false); + expect(ListMemoriesQuerySchema.safeParse({ sort: 'last_read_at' }).success).toBe(false); + }); + it('coerces the reverse directions on the query transport like every other threshold', () => { const r = ListMemoriesQuerySchema.parse({ sort: 'opened_count', min_opened_count: '10', opened_within_days: '7' }); expect(r.sort).toBe('opened_count'); diff --git a/packages/schemas/src/domain/memory.ts b/packages/schemas/src/domain/memory.ts index e5e6cefc2..75a1c2666 100644 --- a/packages/schemas/src/domain/memory.ts +++ b/packages/schemas/src/domain/memory.ts @@ -130,8 +130,15 @@ export const TimestampFilterSchema = z * deliberate-fetch counter, `id` breaking ties. It pages through a cursor of * its own (`count-cursor.ts`), because a keyset bound on an integer column * cannot ride in the timestamp cursor the two date sorts share. + * + * `last_opened_at` (migration 00115) is "recently read first": descending on + * the last DELIBERATE agent read (00099 — a targeted `memory.read` / CLI + * `read`, never a bulk list/search page nor a human opening the dashboard), + * `id` breaking ties, and NULLS LAST, so lessons nothing ever opened trail the + * list rather than lead it. Its cursor (`last-opened-cursor.ts`) carries a + * NULLABLE timestamp, because a page can end inside that never-opened tail. */ -export const MemorySortSchema = z.enum(['updated_at', 'created_at', 'opened_count']); +export const MemorySortSchema = z.enum(['updated_at', 'created_at', 'opened_count', 'last_opened_at']); export type MemorySort = z.infer; /** diff --git a/packages/schemas/src/llms/template.md b/packages/schemas/src/llms/template.md index 7c32317de..de35d81d1 100644 --- a/packages/schemas/src/llms/template.md +++ b/packages/schemas/src/llms/template.md @@ -279,6 +279,18 @@ curl -H "Authorization: Bearer lk_ro_…" \ min_opened_count=5&sort=opened_count" ``` +`sort=last_opened_at` orders by the last DELIBERATE read instead — an agent's `memory.read` or +the CLI's `read`, never a bulk `memory.list`/`memory.search` page nor a human opening the +dashboard — most recent first (ties by id). Memories nothing has ever opened are not dropped: +they follow every opened one, so `total` is the same as for any other sort. Its `nextCursor` is +only valid for that sort. This is the Lore Explorer's **Recent read** order. + +```bash +# What agents are reading right now, most recent first +curl -H "Authorization: Bearer lk_ro_…" \ + "https://pqokxlhvnosogizsjztg.supabase.co/functions/v1/memories?sort=last_opened_at" +``` + ### Finding lore that is about to expire `?expiring_within_days=N` (1–365) narrows the list to memories whose TTL runs out soon — those diff --git a/packages/web/public/llms.txt b/packages/web/public/llms.txt index 439af2669..965c8b653 100644 --- a/packages/web/public/llms.txt +++ b/packages/web/public/llms.txt @@ -822,6 +822,18 @@ curl -H "Authorization: Bearer lk_ro_…" \ min_opened_count=5&sort=opened_count" ``` +`sort=last_opened_at` orders by the last DELIBERATE read instead — an agent's `memory.read` or +the CLI's `read`, never a bulk `memory.list`/`memory.search` page nor a human opening the +dashboard — most recent first (ties by id). Memories nothing has ever opened are not dropped: +they follow every opened one, so `total` is the same as for any other sort. Its `nextCursor` is +only valid for that sort. This is the Lore Explorer's **Recent read** order. + +```bash +# What agents are reading right now, most recent first +curl -H "Authorization: Bearer lk_ro_…" \ + "https://pqokxlhvnosogizsjztg.supabase.co/functions/v1/memories?sort=last_opened_at" +``` + ### Finding lore that is about to expire `?expiring_within_days=N` (1–365) narrows the list to memories whose TTL runs out soon — those diff --git a/packages/web/src/components/lore/LoreExplorer.tsx b/packages/web/src/components/lore/LoreExplorer.tsx index 2208225c4..3c25c5e87 100644 --- a/packages/web/src/components/lore/LoreExplorer.tsx +++ b/packages/web/src/components/lore/LoreExplorer.tsx @@ -54,8 +54,9 @@ * least N times", "opened within N days"); those reverse directions are * Explorer-only, so "Create retention policy" is hidden while one is set. * - `sort` param: the list order (`lib/explorer-sort.ts`) — Newest (absent, - * the default) or Most chosen (`opened_count`, migration 00113). Shareable; - * an unknown value falls back to Newest. + * the default), Most chosen (`opened_count`, migration 00113) or Recent read + * (`last_opened_at`, migration 00115). Shareable; an unknown value falls back + * to Newest. * - `owner` param: the superseded ownership shorthand from the old * client-side owner bar. Still READ so old links (and pre-change accept-invite * deep links) land; never written. `resolveFilters` folds a `'personal'` value diff --git a/packages/web/src/content/docs/grooming.mdx b/packages/web/src/content/docs/grooming.mdx index 41fdd7caf..656a2a999 100644 --- a/packages/web/src/content/docs/grooming.mdx +++ b/packages/web/src/content/docs/grooming.mdx @@ -183,6 +183,14 @@ filters, so a sorted view is shareable; **Newest** is the default. Over the REST API the same order is `sort=opened_count` on `GET /memories` and `POST /memories/list`. +To see what agents are reading **right now**, set **Sort** to **Recent read**: +the lessons an agent most recently fetched on purpose come first. Only a +deliberate read counts — an agent's `memory.read` or the CLI's `read` — so a +lesson that merely appeared in a `memory.list` or `memory.search` page, or that +you opened in the dashboard, does not move up. Lessons no agent has ever read +stay in the list, after every read one. Over the REST API the same order is +`sort=last_opened_at`. + diff --git a/packages/web/src/content/docs/self-improving-workflows.mdx b/packages/web/src/content/docs/self-improving-workflows.mdx index bf07ebd86..3ff54397d 100644 --- a/packages/web/src/content/docs/self-improving-workflows.mdx +++ b/packages/web/src/content/docs/self-improving-workflows.mdx @@ -266,8 +266,9 @@ Where you see it: times; until then it is **Too new to judge**. - **The [Lore Explorer](/lore) detail sheet** shows Recurrence, Delivered, Chosen, and Cited side by side. -- **The Explorer list** sorts by **Most chosen**, and its **Chosen** filter - finds lessons chosen N times or fewer. +- **The Explorer list** sorts by **Most chosen** or **Recent read** (the last + deliberate read first), and its **Chosen** filter finds lessons chosen N times + or fewer. - **The CLI** previews the lessons nothing has ever chosen: ```bash diff --git a/packages/web/src/lib/analytics/track.ts b/packages/web/src/lib/analytics/track.ts index cef96c724..7af4d3f5a 100644 --- a/packages/web/src/lib/analytics/track.ts +++ b/packages/web/src/lib/analytics/track.ts @@ -46,7 +46,7 @@ * - `lore.memory_archive_toggled` — a memory was archived or restored. * - `lore.filter_changed` — any filter-bar mutation, thresholds included. * - `lore.status_changed` — Active / Archived / Expiring. - * - `lore.sort_changed` — the list order (Newest / Most chosen). + * - `lore.sort_changed` — the list order (Newest / Most chosen / Recent read). * - `lore.search_committed` — the debounced search settled. * - `lore.scope_selected` — a scope chip (or Browse-all row) was picked. * - `lore.range_changed` — any of the FIVE controls that write `?range=`. diff --git a/packages/web/src/lib/explorer-sort.spec.ts b/packages/web/src/lib/explorer-sort.spec.ts index 6e5c8f0d9..5ce4273c5 100644 --- a/packages/web/src/lib/explorer-sort.spec.ts +++ b/packages/web/src/lib/explorer-sort.spec.ts @@ -12,10 +12,11 @@ describe('resolveExplorerSort', () => { it('keeps a known order', () => { expect(resolveExplorerSort('opened_count')).toBe('opened_count'); expect(resolveExplorerSort('created_at')).toBe('created_at'); + expect(resolveExplorerSort('last_opened_at')).toBe('last_opened_at'); }); it('falls back to Newest for an absent, unknown or non-string value', () => { - for (const raw of [null, undefined, '', 'read_count', 'updated_at', 42, {}]) { + for (const raw of [null, undefined, '', 'read_count', 'last_read_at', 'updated_at', 42, {}]) { expect(resolveExplorerSort(raw)).toBe(DEFAULT_EXPLORER_SORT); } }); @@ -25,6 +26,7 @@ describe('explorerSortParamValue', () => { it('drops the param for the default and keeps it otherwise', () => { expect(explorerSortParamValue('created_at')).toBeNull(); expect(explorerSortParamValue('opened_count')).toBe('opened_count'); + expect(explorerSortParamValue('last_opened_at')).toBe('last_opened_at'); }); }); @@ -40,4 +42,12 @@ describe('EXPLORER_SORTS', () => { it('labels every option', () => { for (const sort of EXPLORER_SORTS) expect(EXPLORER_SORT_LABELS[sort].length).toBeGreaterThan(0); }); + + it('offers Recent read, ordered by the last DELIBERATE read', () => { + // `last_opened_at`, not `last_read_at`: a bulk list/search page bumps the + // latter, and "Recent read" means a lesson an agent reached for. + expect(EXPLORER_SORTS).toContain('last_opened_at'); + expect(EXPLORER_SORTS).not.toContain('last_read_at'); + expect(EXPLORER_SORT_LABELS.last_opened_at).toBe('Recent read'); + }); }); diff --git a/packages/web/src/lib/explorer-sort.ts b/packages/web/src/lib/explorer-sort.ts index 628839589..bce2e9edd 100644 --- a/packages/web/src/lib/explorer-sort.ts +++ b/packages/web/src/lib/explorer-sort.ts @@ -1,7 +1,7 @@ /** * The Lore Explorer's list order — the `?sort=` URL param. * - * Two orders, both descending: + * Three orders, all descending: * * - `created_at` — **Newest**, the default and the order the list has always * had (`listMemories` used to hard-code it so a backdated migration lands at @@ -10,6 +10,15 @@ * deliberately fetched the most, first. It is the direct answer to "which * lore is actually used?", where a `Chosen: N times or more` threshold makes * the reader guess an N first. + * - `last_opened_at` — **Recent read**, migration 00115: the lessons an agent + * deliberately read most recently, first — "what is being used right now?", + * which a lifetime count cannot tell apart from what was used last spring. + * A deliberate read only (00099): a lesson that merely rode along in a bulk + * list/search page, or that a human opened in this dashboard, does not move. + * Never-read lessons trail the list; they are not dropped from it. + * + * `last_read_at` is not offered: bulk pages bump it, so as an order it would + * mostly re-state "was in a recent listing". * * `updated_at` is a legal API sort but is not offered: a re-write bumps it, so * as a list order it mostly re-states recurrence, which `Recurrence` already @@ -18,7 +27,7 @@ * Pure, so the URL handling is unit-tested without a browser. */ -export const EXPLORER_SORTS = ['created_at', 'opened_count'] as const; +export const EXPLORER_SORTS = ['created_at', 'opened_count', 'last_opened_at'] as const; export type ExplorerSort = (typeof EXPLORER_SORTS)[number]; /** The order an absent `?sort=` means. */ @@ -28,6 +37,7 @@ export const DEFAULT_EXPLORER_SORT: ExplorerSort = 'created_at'; export const EXPLORER_SORT_LABELS: Record = { created_at: 'Newest', opened_count: 'Most chosen', + last_opened_at: 'Recent read', }; /** diff --git a/packages/web/src/lib/memory-list.ts b/packages/web/src/lib/memory-list.ts index d7c21561a..705197241 100644 --- a/packages/web/src/lib/memory-list.ts +++ b/packages/web/src/lib/memory-list.ts @@ -60,7 +60,8 @@ export interface MemoryFilters { retentionConditions?: RetentionConditions; /** * The list order (`lib/explorer-sort.ts`) — `created_at` (Newest, the - * default) or `opened_count` (Most chosen, migration 00113). A cursor is only + * default), `opened_count` (Most chosen, migration 00113) or `last_opened_at` + * (Recent read, migration 00115). A cursor is only * valid for the order that minted it; the route restarts at page one if the * two disagree, and the Explorer's query key changes with the sort anyway. */ @@ -105,7 +106,8 @@ const MAX_PAGE_SIZE = 100; * Ordering defaults to `created_at desc` — `sort: 'created_at'` — not the * route's `updated_at` default: a memory migrated with a backdated `created_at` * belongs at its original position in the Explorer, which is the order the list - * has always been in. `filters.sort` swaps in `opened_count` ("Most chosen"). + * has always been in. `filters.sort` swaps in `opened_count` ("Most chosen") or + * `last_opened_at` ("Recent read"). * * A body rather than a query string: the filter bar's dimensions are unbounded, * and the query transport caps each one at 2048 characters — the ceiling that diff --git a/supabase/byod/bootstrap.sql b/supabase/byod/bootstrap.sql index f082b4dfb..a6c413e65 100644 --- a/supabase/byod/bootstrap.sql +++ b/supabase/byod/bootstrap.sql @@ -2,7 +2,7 @@ -- -- ⚠ GENERATED FILE — DO NOT EDIT BY HAND. -- Generated by `node supabase/byod/generate.mjs` from the --- 109 migrations in supabase/migrations/ (newest: 00114_exclude_tags_dimension.sql). +-- 110 migrations in supabase/migrations/ (newest: 00115_explorer_recent_read_sort.sql). -- CI fails when this file and the migrations disagree; edit a migration (or add -- one) and regenerate instead. -- @@ -23345,6 +23345,340 @@ insert into supabase_migrations.schema_migrations (version, name) values ('00114', 'exclude_tags_dimension') on conflict (version) do nothing; +-- ───────────────────────────────────────────────────────────────────────────── +-- 00115_explorer_recent_read_sort.sql +-- ───────────────────────────────────────────────────────────────────────────── + +-- ═════════════════════════════════════════════════════════════════════════ +-- 00115 — the "Recent read" sort: `lorekit_memory_list(p_sort => 'last_opened_at')`. +-- +-- THE GAP. The Explorer can order by Newest (`created_at`) and Most chosen +-- (`opened_count`, 00113), but not by "what did an agent read lately?". The +-- count answers "which lore is used, ever"; it cannot tell a lesson that was +-- fetched forty times last spring from one fetched twice this morning. +-- +-- THE COLUMN. `last_opened_at` (00099), not `last_read_at`. `last_read_at` +-- moves on EVERY read — a bulk `memory.list`/`.search` page a lesson merely +-- rode along in, and a human opening the dashboard's detail sheet, included. +-- `last_opened_at` moves only on a TARGETED read from an agent surface +-- (`mcp`/`cli`): an agent deliberately reaching for this one lesson. "Read, +-- not listed" is exactly that column. +-- +-- ── DESIGN ─────────────────────────────────────────────────────────────── +-- +-- NULLS LAST. `last_opened_at` is NULL for a lesson nothing has ever opened, +-- and Postgres puts NULLs FIRST under `desc` — the default ordering would have +-- led "Recent read" with the never-read lessons. The order is therefore +-- `last_opened_at desc nulls last, id desc`, and the never-opened lessons form +-- a tail ordered by id alone. +-- +-- A SORT, NOT A FILTER. The never-opened tail stays in the result, so +-- `total_count` is the same under every order — a sort that dropped rows would +-- make the count above the list describe a different population than the +-- unsorted one (00094's contract). "Only lessons opened lately" is already a +-- filter: `p_opened_within_days` (00113). +-- +-- THE KEYSET. The cursor is `(last_opened_at, id)` with a NULLABLE first half, +-- so "no cursor" can no longer be read off `p_cursor_ts is null` the way the +-- date sorts read it — a NULL `p_cursor_ts` with a set `p_cursor_id` means +-- "the previous page ended inside the never-opened tail". Under this sort: +-- +-- p_cursor_id is null → first page +-- p_cursor_ts is null → tail only: last_opened_at is null and id < cursor +-- otherwise → the rest of the opened run (strictly +-- older, or equal with a smaller id), +-- then the whole tail +-- +-- NO NEW PARAMETER, NO NEW SIGNATURE. The timestamp half rides in the existing +-- `p_cursor_ts`, the id in `p_cursor_id`; `p_cursor_count` (00113) is not read. +-- The argument list is 00114's exactly, so this is a `create or replace` — +-- grants and the single-overload invariant (§108/§113/§114) are untouched. The +-- revoke/grant pair is restated anyway, so this file alone documents the ACL. +-- +-- The three existing orders are unchanged: their keyset predicate and ORDER BY +-- are the 00113 text, now built in the declare block instead of inline. +-- +-- §115 in `migrations.test.sql` is the executable proof. +-- ═════════════════════════════════════════════════════════════════════════ + +create or replace function lorekit_memory_list( + p_user_id uuid, + p_archived boolean default false, + p_scope text default null, + p_key text default null, + p_key_prefix text default null, + p_q text default null, + p_created_since timestamptz default null, + p_created_until timestamptz default null, + p_expires_after timestamptz default null, + p_expires_on_or_before timestamptz default null, + p_tags text[] default null, + p_tags_mode text default 'any', + p_source_agent text[] default null, + p_source_agent_mode text default 'in', + p_trigger text[] default null, + p_trigger_mode text default 'in', + p_kind text[] default null, + p_kind_mode text default 'in', + p_host text[] default null, + p_host_mode text default 'in', + p_origin_repo text[] default null, + p_origin_repo_mode text default 'in', + p_origin_branch text[] default null, + p_origin_branch_mode text default 'in', + p_origin_pr text[] default null, + p_origin_pr_mode text default 'in', + p_owner text[] default null, + p_owner_mode text default 'in', + p_sort text default 'updated_at', + p_cursor_ts timestamptz default null, + p_cursor_id uuid default null, + p_limit integer default 51, + p_key_scopes text[] default '{}', + p_key_org_access text default 'all', + p_key_org_ids uuid[] default '{}', + p_min_age_days integer default null, + p_unseen_days integer default null, + p_max_seen_count integer default null, + p_max_read_count integer default null, + p_max_opened_count integer default null, + -- 00113: the Explorer's reverse directions, appended and null-defaulted. + p_opened_within_days integer default null, + p_min_seen_count integer default null, + p_min_read_count integer default null, + p_min_opened_count integer default null, + -- 00113: the keyset cursor for `p_sort => 'opened_count'`. An integer sort + -- needs an integer cursor; `p_cursor_ts` keeps serving the timestamp sorts — + -- since 00115 including the nullable `last_opened_at`. + p_cursor_count integer default null, + -- 00114: labels the row must carry NONE of — an always-applied conjunct, + -- independent of p_tags/p_tags_mode, so "tagged a AND NOT tagged b" is + -- expressible. Appended, null-defaulted: null = not filtered. + p_exclude_tags text[] default null +) +returns table ( + id uuid, + scope text, + key text, + value text, + tags text[], + source_agent text, + trigger text, + created_at timestamptz, + updated_at timestamptz, + expires_at timestamptz, + archived_at timestamptz, + origin_repo text, + origin_branch text, + origin_commit text, + origin_pr integer, + kind text, + host text, + seen_count integer, + read_count integer, + opened_count integer, + last_read_at timestamptz, + last_opened_at timestamptz, + org_id uuid, + created_by uuid, + updated_by uuid, + org_name text, + org_slug text, + total_count integer +) +language plpgsql +stable +security definer +set search_path = public +as $$ +#variable_conflict use_column +declare + v_actor uuid := case + when auth.role() = 'service_role' then coalesce(p_user_id, auth.uid()) + else auth.uid() + end; + v_origin_pr integer[] := ( + select array_agg(x::integer) + from unnest(coalesce(p_origin_pr, '{}'::text[])) as x + where x ~ '^0*[0-9]{1,9}$' + ); + -- 00113: `opened_count` joins the two date sorts; 00115: `last_opened_at`. + -- Anything else still falls back to `updated_at`, so an unknown value cannot + -- reach the format() below. + v_sort text := case + when p_sort in ('created_at', 'opened_count', 'last_opened_at') then p_sort else 'updated_at' + end; + -- Which bound parameter carries the keyset value: the integer cursor ($44) + -- for the count sort, the timestamp cursor ($29) for the others. A fixed + -- placeholder chosen here, never caller text, so splicing it is safe. + v_cursor text := case when v_sort = 'opened_count' then '$44' else '$29' end; + -- The keyset predicate and the ORDER BY, per sort. Built from fixed text and + -- `v_sort` (whitelisted above, quoted with %I) only. + -- + -- 00115: `last_opened_at` is NULLABLE, so it cannot use the shared shape — + -- NULLs would sort first under `desc`, and `col < cursor` is never true for + -- a NULL row, so the never-opened tail would be unreachable past page one. + -- See the header for the three cursor states. + v_keyset text := case + when v_sort = 'last_opened_at' then + $k$( + $30 is null + or case + when $29 is null then m.last_opened_at is null and m.id < $30 + else m.last_opened_at < $29 + or (m.last_opened_at = $29 and m.id < $30) + or m.last_opened_at is null + end + )$k$ + else format( + '(%2$s is null or m.%1$I < %2$s or (m.%1$I = %2$s and m.id < $30))', + v_sort, v_cursor) + end; + v_order text := case + when v_sort = 'last_opened_at' then 'm.last_opened_at desc nulls last, m.id desc' + else format('m.%I desc, m.id desc', v_sort) + end; + -- Resolved once per call so lorekit_match_retention stays immutable (00108). + v_created_cutoff timestamptz := case + when p_min_age_days is null then null else now() - (p_min_age_days * interval '1 day') + end; + v_unseen_cutoff timestamptz := case + when p_unseen_days is null then null else now() - (p_unseen_days * interval '1 day') + end; + -- 00113: the "opened within N days" lower bound, resolved once per call for + -- the same reason as the two cutoffs above. + v_opened_since timestamptz := case + when p_opened_within_days is null then null else now() - (p_opened_within_days * interval '1 day') + end; +begin + return query execute format($q$ + select + m.id, m.scope, m.key, m.value, m.tags, m.source_agent, m.trigger, + m.created_at, m.updated_at, m.expires_at, m.archived_at, + m.origin_repo, m.origin_branch, m.origin_commit, m.origin_pr, + m.kind, m.host, m.seen_count, + m.read_count, m.opened_count, m.last_read_at, m.last_opened_at, + m.org_id, m.created_by, m.updated_by, + o.name as org_name, o.slug as org_slug, + (count(*) over ())::integer as total_count + from memories m + left join orgs o on o.id = m.org_id + where ( + ($1 is null and auth.role() = 'service_role') + or m.user_id = $1 + or m.org_id in (select lorekit_member_org_ids($1)) + ) + and lorekit_api_token_scope_allowed($32, m.scope) + and lorekit_api_token_org_allowed($33, $34, m.org_id) + and ( + case + when $2 then m.archived_at is not null + else m.archived_at is null + and (m.expires_at is null or m.expires_at > now()) + end + ) + and ($3 is null or m.scope = $3) + and ($4 is null or m.key = $4) + and ($5 is null or m.key ilike $5 || '%%') + and ($6 is null or m.key ilike '%%' || $6 || '%%' or m.value ilike '%%' || $6 || '%%') + and ($7 is null or m.created_at >= $7) + and ($8 is null or m.created_at < $8) + and ($9 is null or m.expires_at > $9) + and ($10 is null or m.expires_at <= $10) + and lorekit_match_tags(m.tags, $11, $12) + and lorekit_match_tags(m.tags, $45, 'none') + and lorekit_match_text(m.source_agent, $13, $14) + and lorekit_match_text(m.trigger, $15, $16) + and lorekit_match_text(m.kind, $17, $18) + and lorekit_match_text(m.host, $19, $20) + and lorekit_match_text(m.origin_repo, $21, $22) + and lorekit_match_text(m.origin_branch, $23, $24) + and lorekit_match_int (m.origin_pr, $25, $26) + and ($27 is null or case coalesce($28, 'in') + when 'nin' then ( + (case when m.org_id is null then 'personal' else o.slug end) is not null + and (case when m.org_id is null then 'personal' else o.slug end) <> all($27) + ) + else ( + ('personal' = any($27) and m.org_id is null) + or (m.org_id is not null and o.slug = any($27)) + ) + end) + and lorekit_match_retention( + m.created_at, m.last_opened_at, m.seen_count, m.read_count, m.opened_count, + $35, $36, $37, $38, $39, + $40, $41, $42, $43 + ) + and %1$s + order by %2$s + limit $31 + $q$, v_keyset, v_order) + using + v_actor, coalesce(p_archived, false), p_scope, p_key, p_key_prefix, p_q, + p_created_since, p_created_until, p_expires_after, p_expires_on_or_before, + p_tags, p_tags_mode, + p_source_agent, p_source_agent_mode, + p_trigger, p_trigger_mode, + p_kind, p_kind_mode, + p_host, p_host_mode, + p_origin_repo, p_origin_repo_mode, + p_origin_branch, p_origin_branch_mode, + v_origin_pr, p_origin_pr_mode, + p_owner, p_owner_mode, + p_cursor_ts, p_cursor_id, + greatest(coalesce(p_limit, 51), 1), + coalesce(p_key_scopes, '{}'::text[]), + coalesce(p_key_org_access, 'all'), + coalesce(p_key_org_ids, '{}'::uuid[]), + v_created_cutoff, v_unseen_cutoff, p_max_seen_count, p_max_read_count, + p_max_opened_count, + v_opened_since, p_min_seen_count, p_min_read_count, p_min_opened_count, + p_cursor_count, + p_exclude_tags; +end; +$$; + +revoke execute on function lorekit_memory_list( + uuid, boolean, text, text, text, text, timestamptz, timestamptz, timestamptz, + timestamptz, text[], text, text[], text, text[], text, text[], text, text[], + text, text[], text, text[], text, text[], text, text[], text, text, + timestamptz, uuid, integer, text[], text, uuid[], integer, integer, integer, + integer, integer, + integer, integer, integer, integer, integer, + text[]) from public, anon; +grant execute on function lorekit_memory_list( + uuid, boolean, text, text, text, text, timestamptz, timestamptz, timestamptz, + timestamptz, text[], text, text[], text, text[], text, text[], text, text[], + text, text[], text, text[], text, text[], text, text[], text, text, + timestamptz, uuid, integer, text[], text, uuid[], integer, integer, integer, + integer, integer, + integer, integer, integer, integer, integer, + text[]) to authenticated, service_role; + +comment on function lorekit_memory_list( + uuid, boolean, text, text, text, text, timestamptz, timestamptz, timestamptz, + timestamptz, text[], text, text[], text, text[], text, text[], text, text[], + text, text[], text, text[], text, text[], text, text[], text, text, + timestamptz, uuid, integer, text[], text, uuid[], integer, integer, integer, + integer, integer, + integer, integer, integer, integer, integer, + text[]) is + 'The keyset-paginated memory list behind GET /memories and POST /memories/list. + Sorts by updated_at (default), created_at, opened_count (00113) or + last_opened_at (00115) — each descending with id as the tie-break; + last_opened_at is NULLS LAST, so never-opened lessons trail. p_cursor_ts + carries the timestamp sorts'' keyset value (NULL under last_opened_at with a + set p_cursor_id = inside the never-opened tail) and p_cursor_count the count + sort''s. Applies the five retention thresholds and the Explorer''s reverse + directions through the shared lorekit_match_retention, so its total_count + agrees with _facets, _activity and _pivot under the same parameters. + p_exclude_tags (00114): the row carries none of these labels, ANDed with + whatever p_tags/p_tags_mode ask.'; +; +insert into supabase_migrations.schema_migrations (version, name) +values ('00115', 'explorer_recent_read_sort') +on conflict (version) do nothing; + -- ───────────────────────────────────────────────────────────────────────────── -- storage-overlay.sql — BYOD only: drop foreign keys into auth.users -- ───────────────────────────────────────────────────────────────────────────── diff --git a/supabase/functions/_shared/api/last-opened-cursor.ts b/supabase/functions/_shared/api/last-opened-cursor.ts new file mode 100644 index 000000000..390a47e48 --- /dev/null +++ b/supabase/functions/_shared/api/last-opened-cursor.ts @@ -0,0 +1,84 @@ +/** + * The keyset cursor for the `sort=last_opened_at` list — "Recent read" + * (migration 00115): the lessons an agent deliberately read most recently, + * first, and the never-opened ones after them. + * + * Pure and shared (mirrored into `supabase/functions/_shared/api/last-opened-cursor.ts`, + * guarded by `edge-parity.spec.ts`) so the codec is unit-tested here and the + * deployed copy cannot drift from what was tested. + * + * Why it is not one more case in `_shared/api/paginate.ts`: that cursor's + * timestamp is never null (both date sorts are NOT NULL columns), its decoder + * rejects a missing one, and it is byte-mirrored into the MCP function, which + * has no such sort. `last_opened_at` is NULL for a lesson nothing ever opened, + * and a page can end inside that never-opened tail — so this cursor must carry + * a null timestamp, and say so explicitly: `{ id, last_opened_at: null }`. + * + * The three formats cannot be mistaken for each other. `paginate.ts`'s decoder + * finds neither `updated_at` nor `created_at` here, `count-cursor.ts`'s finds + * no integer `opened_count`, and this decoder requires the `last_opened_at` + * KEY to be present (with a timestamp or `null`) — so a caller who flips + * `sort` mid-pagination restarts at page one rather than receiving a page cut + * at the wrong place. + */ + +/** + * The decoded keyset position: the last row's id and its `last_opened_at` — + * `null` when that row was never opened, i.e. the page ended inside the tail. + */ +export interface LastOpenedCursor { + id: string; + ts: string | null; +} + +/** A cursor's `id` is a `memories` primary key, which is a `uuid`. */ +const UUID_RE = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i; + +/** The same leading shape `paginate.ts` requires of its timestamp. */ +const TS_RE = /^\d{4}-\d{2}-\d{2}[T ]\d{2}:\d{2}:\d{2}/; + +/** base64url without padding — the same alphabet `paginate.ts` uses. */ +export function encodeLastOpenedCursor(id: string, ts: string | null): string { + return btoa(JSON.stringify({ id, last_opened_at: ts })) + .replace(/\+/g, '-') + .replace(/\//g, '_') + .replace(/=/g, ''); +} + +/** + * Decode a Recent read cursor, or `null` for anything that is not one. + * + * Fails CLOSED: the values reach the RPC as typed parameters (`uuid`, + * `timestamptz`), so a malformed one would be a 500 rather than an injection — + * and the right answer to a cursor that does not parse is the first page, the + * same contract `paginate.ts` and `count-cursor.ts` keep. + */ +export function decodeLastOpenedCursor(raw: string): LastOpenedCursor | null { + try { + const b64 = raw.replace(/-/g, '+').replace(/_/g, '/'); + const pad = (4 - (b64.length % 4)) % 4; + const parsed: unknown = JSON.parse(atob(b64 + '='.repeat(pad))); + if (!parsed || typeof parsed !== 'object' || !('last_opened_at' in parsed)) return null; + const { id, last_opened_at: ts } = parsed as { id?: unknown; last_opened_at?: unknown }; + if (typeof id !== 'string' || !UUID_RE.test(id)) return null; + if (ts !== null && (typeof ts !== 'string' || !TS_RE.test(ts))) return null; + return { id, ts }; + } catch { + return null; + } +} + +/** + * The page and its next cursor, from `limit + 1` rows — the overflow row is how + * `hasMore` is known, exactly as `paginate.ts`'s `buildPage` reads it. + */ +export function buildLastOpenedPage( + rows: T[], + limit: number, +): { entries: T[]; hasMore: boolean; nextCursor: string | null } { + const hasMore = rows.length > limit; + const entries = hasMore ? rows.slice(0, limit) : rows; + const last = entries[entries.length - 1]; + const nextCursor = hasMore && last ? encodeLastOpenedCursor(last.id, last.last_opened_at ?? null) : null; + return { entries, hasMore, nextCursor }; +} diff --git a/supabase/functions/_shared/schemas/memory.ts b/supabase/functions/_shared/schemas/memory.ts index bef73bef6..53dfd299c 100644 --- a/supabase/functions/_shared/schemas/memory.ts +++ b/supabase/functions/_shared/schemas/memory.ts @@ -135,8 +135,15 @@ export const TimestampFilterSchema = z * deliberate-fetch counter, `id` breaking ties. It pages through a cursor of * its own (`count-cursor.ts`), because a keyset bound on an integer column * cannot ride in the timestamp cursor the two date sorts share. + * + * `last_opened_at` (migration 00115) is "recently read first": descending on + * the last DELIBERATE agent read (00099 — a targeted `memory.read` / CLI + * `read`, never a bulk list/search page nor a human opening the dashboard), + * `id` breaking ties, and NULLS LAST, so lessons nothing ever opened trail the + * list rather than lead it. Its cursor (`last-opened-cursor.ts`) carries a + * NULLABLE timestamp, because a page can end inside that never-opened tail. */ -export const MemorySortSchema = z.enum(['updated_at', 'created_at', 'opened_count']); +export const MemorySortSchema = z.enum(['updated_at', 'created_at', 'opened_count', 'last_opened_at']); export type MemorySort = z.infer; /** diff --git a/supabase/functions/memories/CLAUDE.md b/supabase/functions/memories/CLAUDE.md index 9dd128adb..85dbbad29 100644 --- a/supabase/functions/memories/CLAUDE.md +++ b/supabase/functions/memories/CLAUDE.md @@ -353,6 +353,17 @@ table constraint). The list also takes `sort=opened_count` (most chosen first, so `paginate.ts` and its MCP byte-mirror stay untouched, and each decoder rejects the other's format so a sort flip restarts at page one. §113 pins all of it. +**`sort=last_opened_at` ("Recent read", 00115)** orders by the last deliberate agent read +(00099's column — targeted AND from `mcp`/`cli`, never a bulk page or the dashboard), `desc NULLS +LAST`, `id desc` breaking ties. It is a sort, not a filter: never-opened lessons stay in the result +as an id-ordered tail, so `total` matches every other order. The column is nullable, so it pages +through a third codec, `_shared/api/last-opened-cursor.ts` (mirrored from +`mcp-core/src/rest/last-opened-cursor.ts`), whose `last_opened_at` may be `null`: a page that ended +inside the tail. The RPC reuses `p_cursor_ts`/`p_cursor_id` — under this sort a NULL `p_cursor_ts` +with a set `p_cursor_id` means "inside the tail", so the handler must pass the id whenever a cursor +decoded. Same signature as 00114 (`create or replace`). §115 pins the order, the total and a page +walk across the tie and the opened→never-opened boundary. + They differ from a dimension in one way that matters when reading a count: a threshold is applied in the RPC's `base` CTE rather than as a per-dimension `ok_*` flag, so it narrows **every** facet including the SELF-EXCLUDED one. That is the correct asymmetry, not an diff --git a/supabase/functions/memories/handlers/list.ts b/supabase/functions/memories/handlers/list.ts index 86667458a..7aa7feedf 100644 --- a/supabase/functions/memories/handlers/list.ts +++ b/supabase/functions/memories/handlers/list.ts @@ -6,6 +6,7 @@ import { validateOptionalBody, validateQuery } from '../../_shared/api/validate. import { parseScopeFilter } from '../../_shared/scope/scope.ts'; import { buildPage, decodeCursor } from '../../_shared/api/paginate.ts'; import { buildCountPage, decodeCountCursor } from '../../_shared/api/count-cursor.ts'; +import { buildLastOpenedPage, decodeLastOpenedCursor } from '../../_shared/api/last-opened-cursor.ts'; import { createTracedClient } from '../../_shared/telemetry/otel.ts'; import type { Span } from '../../_shared/telemetry/otel.ts'; import { @@ -182,14 +183,18 @@ async function respondWithPage( // so it is ignored (→ first page) rather than silently mis-paging. Unchanged // contract; only the place the two halves are applied has moved. // - // Two cursor FORMATS since 00113: the date sorts keep `paginate.ts`'s - // timestamp cursor, and `opened_count` uses `count-cursor.ts`'s integer one. - // Each decoder rejects the other's format, so the same "wrong sort → first - // page" rule holds across the two families as well as within the dates. + // Three cursor FORMATS: the date sorts keep `paginate.ts`'s timestamp + // cursor, `opened_count` (00113) uses `count-cursor.ts`'s integer one, and + // `last_opened_at` (00115) uses `last-opened-cursor.ts`'s NULLABLE timestamp + // — a page can end inside the never-opened tail. Each decoder rejects the + // others' formats, so the same "wrong sort → first page" rule holds across + // the three families as well as within the dates. const countSort = params.sort === 'opened_count'; - const dateCursor = !countSort && params.cursor ? decodeCursor(params.cursor) : null; + const lastOpenedSort = params.sort === 'last_opened_at'; + const dateCursor = !countSort && !lastOpenedSort && params.cursor ? decodeCursor(params.cursor) : null; const usableCursor = dateCursor && dateCursor.sort === params.sort ? dateCursor : null; const countCursor = countSort && params.cursor ? decodeCountCursor(params.cursor) : null; + const lastOpenedCursor = lastOpenedSort && params.cursor ? decodeLastOpenedCursor(params.cursor) : null; // The asymmetric `(after, on_or_before]` window stays in TypeScript so the // `now`-relative boundary has exactly one implementation (edge-parity.spec.ts @@ -251,8 +256,11 @@ async function respondWithPage( p_key_org_access: keyRestriction(auth)?.orgAccess ?? 'all', p_key_org_ids: keyRestriction(auth)?.orgIds ?? [], p_sort: params.sort, - p_cursor_ts: usableCursor?.ts ?? null, - p_cursor_id: usableCursor?.id ?? countCursor?.id ?? null, + // 00115: under `last_opened_at` a NULL `p_cursor_ts` with a set + // `p_cursor_id` is meaningful — "the last page ended inside the + // never-opened tail" — so the id alone is what tells the RPC a cursor exists. + p_cursor_ts: usableCursor?.ts ?? lastOpenedCursor?.ts ?? null, + p_cursor_id: usableCursor?.id ?? countCursor?.id ?? lastOpenedCursor?.id ?? null, // 00113: the count sort's keyset bound. The RPC reads it only when // `p_sort` is `opened_count`, and `p_cursor_ts` only otherwise. p_cursor_count: countCursor?.count ?? null, @@ -273,7 +281,9 @@ async function respondWithPage( const page = params.sort === 'opened_count' ? buildCountPage(rows, params.limit) - : buildPage(rows, params.limit, params.sort); + : params.sort === 'last_opened_at' + ? buildLastOpenedPage(rows, params.limit) + : buildPage(rows, params.limit, params.sort); // `total_count` is the SAME value on every row of `rows` (a `count(*) over // ()` — see migration 00094), so any row answers it; an empty result has no // row to read it from, and 0 matches is the exact total in that case too. diff --git a/supabase/migrations/00115_explorer_recent_read_sort.sql b/supabase/migrations/00115_explorer_recent_read_sort.sql new file mode 100644 index 000000000..8bfa6d5f7 --- /dev/null +++ b/supabase/migrations/00115_explorer_recent_read_sort.sql @@ -0,0 +1,325 @@ +-- ═════════════════════════════════════════════════════════════════════════ +-- 00115 — the "Recent read" sort: `lorekit_memory_list(p_sort => 'last_opened_at')`. +-- +-- THE GAP. The Explorer can order by Newest (`created_at`) and Most chosen +-- (`opened_count`, 00113), but not by "what did an agent read lately?". The +-- count answers "which lore is used, ever"; it cannot tell a lesson that was +-- fetched forty times last spring from one fetched twice this morning. +-- +-- THE COLUMN. `last_opened_at` (00099), not `last_read_at`. `last_read_at` +-- moves on EVERY read — a bulk `memory.list`/`.search` page a lesson merely +-- rode along in, and a human opening the dashboard's detail sheet, included. +-- `last_opened_at` moves only on a TARGETED read from an agent surface +-- (`mcp`/`cli`): an agent deliberately reaching for this one lesson. "Read, +-- not listed" is exactly that column. +-- +-- ── DESIGN ─────────────────────────────────────────────────────────────── +-- +-- NULLS LAST. `last_opened_at` is NULL for a lesson nothing has ever opened, +-- and Postgres puts NULLs FIRST under `desc` — the default ordering would have +-- led "Recent read" with the never-read lessons. The order is therefore +-- `last_opened_at desc nulls last, id desc`, and the never-opened lessons form +-- a tail ordered by id alone. +-- +-- A SORT, NOT A FILTER. The never-opened tail stays in the result, so +-- `total_count` is the same under every order — a sort that dropped rows would +-- make the count above the list describe a different population than the +-- unsorted one (00094's contract). "Only lessons opened lately" is already a +-- filter: `p_opened_within_days` (00113). +-- +-- THE KEYSET. The cursor is `(last_opened_at, id)` with a NULLABLE first half, +-- so "no cursor" can no longer be read off `p_cursor_ts is null` the way the +-- date sorts read it — a NULL `p_cursor_ts` with a set `p_cursor_id` means +-- "the previous page ended inside the never-opened tail". Under this sort: +-- +-- p_cursor_id is null → first page +-- p_cursor_ts is null → tail only: last_opened_at is null and id < cursor +-- otherwise → the rest of the opened run (strictly +-- older, or equal with a smaller id), +-- then the whole tail +-- +-- NO NEW PARAMETER, NO NEW SIGNATURE. The timestamp half rides in the existing +-- `p_cursor_ts`, the id in `p_cursor_id`; `p_cursor_count` (00113) is not read. +-- The argument list is 00114's exactly, so this is a `create or replace` — +-- grants and the single-overload invariant (§108/§113/§114) are untouched. The +-- revoke/grant pair is restated anyway, so this file alone documents the ACL. +-- +-- The three existing orders are unchanged: their keyset predicate and ORDER BY +-- are the 00113 text, now built in the declare block instead of inline. +-- +-- §115 in `migrations.test.sql` is the executable proof. +-- ═════════════════════════════════════════════════════════════════════════ + +create or replace function lorekit_memory_list( + p_user_id uuid, + p_archived boolean default false, + p_scope text default null, + p_key text default null, + p_key_prefix text default null, + p_q text default null, + p_created_since timestamptz default null, + p_created_until timestamptz default null, + p_expires_after timestamptz default null, + p_expires_on_or_before timestamptz default null, + p_tags text[] default null, + p_tags_mode text default 'any', + p_source_agent text[] default null, + p_source_agent_mode text default 'in', + p_trigger text[] default null, + p_trigger_mode text default 'in', + p_kind text[] default null, + p_kind_mode text default 'in', + p_host text[] default null, + p_host_mode text default 'in', + p_origin_repo text[] default null, + p_origin_repo_mode text default 'in', + p_origin_branch text[] default null, + p_origin_branch_mode text default 'in', + p_origin_pr text[] default null, + p_origin_pr_mode text default 'in', + p_owner text[] default null, + p_owner_mode text default 'in', + p_sort text default 'updated_at', + p_cursor_ts timestamptz default null, + p_cursor_id uuid default null, + p_limit integer default 51, + p_key_scopes text[] default '{}', + p_key_org_access text default 'all', + p_key_org_ids uuid[] default '{}', + p_min_age_days integer default null, + p_unseen_days integer default null, + p_max_seen_count integer default null, + p_max_read_count integer default null, + p_max_opened_count integer default null, + -- 00113: the Explorer's reverse directions, appended and null-defaulted. + p_opened_within_days integer default null, + p_min_seen_count integer default null, + p_min_read_count integer default null, + p_min_opened_count integer default null, + -- 00113: the keyset cursor for `p_sort => 'opened_count'`. An integer sort + -- needs an integer cursor; `p_cursor_ts` keeps serving the timestamp sorts — + -- since 00115 including the nullable `last_opened_at`. + p_cursor_count integer default null, + -- 00114: labels the row must carry NONE of — an always-applied conjunct, + -- independent of p_tags/p_tags_mode, so "tagged a AND NOT tagged b" is + -- expressible. Appended, null-defaulted: null = not filtered. + p_exclude_tags text[] default null +) +returns table ( + id uuid, + scope text, + key text, + value text, + tags text[], + source_agent text, + trigger text, + created_at timestamptz, + updated_at timestamptz, + expires_at timestamptz, + archived_at timestamptz, + origin_repo text, + origin_branch text, + origin_commit text, + origin_pr integer, + kind text, + host text, + seen_count integer, + read_count integer, + opened_count integer, + last_read_at timestamptz, + last_opened_at timestamptz, + org_id uuid, + created_by uuid, + updated_by uuid, + org_name text, + org_slug text, + total_count integer +) +language plpgsql +stable +security definer +set search_path = public +as $$ +#variable_conflict use_column +declare + v_actor uuid := case + when auth.role() = 'service_role' then coalesce(p_user_id, auth.uid()) + else auth.uid() + end; + v_origin_pr integer[] := ( + select array_agg(x::integer) + from unnest(coalesce(p_origin_pr, '{}'::text[])) as x + where x ~ '^0*[0-9]{1,9}$' + ); + -- 00113: `opened_count` joins the two date sorts; 00115: `last_opened_at`. + -- Anything else still falls back to `updated_at`, so an unknown value cannot + -- reach the format() below. + v_sort text := case + when p_sort in ('created_at', 'opened_count', 'last_opened_at') then p_sort else 'updated_at' + end; + -- Which bound parameter carries the keyset value: the integer cursor ($44) + -- for the count sort, the timestamp cursor ($29) for the others. A fixed + -- placeholder chosen here, never caller text, so splicing it is safe. + v_cursor text := case when v_sort = 'opened_count' then '$44' else '$29' end; + -- The keyset predicate and the ORDER BY, per sort. Built from fixed text and + -- `v_sort` (whitelisted above, quoted with %I) only. + -- + -- 00115: `last_opened_at` is NULLABLE, so it cannot use the shared shape — + -- NULLs would sort first under `desc`, and `col < cursor` is never true for + -- a NULL row, so the never-opened tail would be unreachable past page one. + -- See the header for the three cursor states. + v_keyset text := case + when v_sort = 'last_opened_at' then + $k$( + $30 is null + or case + when $29 is null then m.last_opened_at is null and m.id < $30 + else m.last_opened_at < $29 + or (m.last_opened_at = $29 and m.id < $30) + or m.last_opened_at is null + end + )$k$ + else format( + '(%2$s is null or m.%1$I < %2$s or (m.%1$I = %2$s and m.id < $30))', + v_sort, v_cursor) + end; + v_order text := case + when v_sort = 'last_opened_at' then 'm.last_opened_at desc nulls last, m.id desc' + else format('m.%I desc, m.id desc', v_sort) + end; + -- Resolved once per call so lorekit_match_retention stays immutable (00108). + v_created_cutoff timestamptz := case + when p_min_age_days is null then null else now() - (p_min_age_days * interval '1 day') + end; + v_unseen_cutoff timestamptz := case + when p_unseen_days is null then null else now() - (p_unseen_days * interval '1 day') + end; + -- 00113: the "opened within N days" lower bound, resolved once per call for + -- the same reason as the two cutoffs above. + v_opened_since timestamptz := case + when p_opened_within_days is null then null else now() - (p_opened_within_days * interval '1 day') + end; +begin + return query execute format($q$ + select + m.id, m.scope, m.key, m.value, m.tags, m.source_agent, m.trigger, + m.created_at, m.updated_at, m.expires_at, m.archived_at, + m.origin_repo, m.origin_branch, m.origin_commit, m.origin_pr, + m.kind, m.host, m.seen_count, + m.read_count, m.opened_count, m.last_read_at, m.last_opened_at, + m.org_id, m.created_by, m.updated_by, + o.name as org_name, o.slug as org_slug, + (count(*) over ())::integer as total_count + from memories m + left join orgs o on o.id = m.org_id + where ( + ($1 is null and auth.role() = 'service_role') + or m.user_id = $1 + or m.org_id in (select lorekit_member_org_ids($1)) + ) + and lorekit_api_token_scope_allowed($32, m.scope) + and lorekit_api_token_org_allowed($33, $34, m.org_id) + and ( + case + when $2 then m.archived_at is not null + else m.archived_at is null + and (m.expires_at is null or m.expires_at > now()) + end + ) + and ($3 is null or m.scope = $3) + and ($4 is null or m.key = $4) + and ($5 is null or m.key ilike $5 || '%%') + and ($6 is null or m.key ilike '%%' || $6 || '%%' or m.value ilike '%%' || $6 || '%%') + and ($7 is null or m.created_at >= $7) + and ($8 is null or m.created_at < $8) + and ($9 is null or m.expires_at > $9) + and ($10 is null or m.expires_at <= $10) + and lorekit_match_tags(m.tags, $11, $12) + and lorekit_match_tags(m.tags, $45, 'none') + and lorekit_match_text(m.source_agent, $13, $14) + and lorekit_match_text(m.trigger, $15, $16) + and lorekit_match_text(m.kind, $17, $18) + and lorekit_match_text(m.host, $19, $20) + and lorekit_match_text(m.origin_repo, $21, $22) + and lorekit_match_text(m.origin_branch, $23, $24) + and lorekit_match_int (m.origin_pr, $25, $26) + and ($27 is null or case coalesce($28, 'in') + when 'nin' then ( + (case when m.org_id is null then 'personal' else o.slug end) is not null + and (case when m.org_id is null then 'personal' else o.slug end) <> all($27) + ) + else ( + ('personal' = any($27) and m.org_id is null) + or (m.org_id is not null and o.slug = any($27)) + ) + end) + and lorekit_match_retention( + m.created_at, m.last_opened_at, m.seen_count, m.read_count, m.opened_count, + $35, $36, $37, $38, $39, + $40, $41, $42, $43 + ) + and %1$s + order by %2$s + limit $31 + $q$, v_keyset, v_order) + using + v_actor, coalesce(p_archived, false), p_scope, p_key, p_key_prefix, p_q, + p_created_since, p_created_until, p_expires_after, p_expires_on_or_before, + p_tags, p_tags_mode, + p_source_agent, p_source_agent_mode, + p_trigger, p_trigger_mode, + p_kind, p_kind_mode, + p_host, p_host_mode, + p_origin_repo, p_origin_repo_mode, + p_origin_branch, p_origin_branch_mode, + v_origin_pr, p_origin_pr_mode, + p_owner, p_owner_mode, + p_cursor_ts, p_cursor_id, + greatest(coalesce(p_limit, 51), 1), + coalesce(p_key_scopes, '{}'::text[]), + coalesce(p_key_org_access, 'all'), + coalesce(p_key_org_ids, '{}'::uuid[]), + v_created_cutoff, v_unseen_cutoff, p_max_seen_count, p_max_read_count, + p_max_opened_count, + v_opened_since, p_min_seen_count, p_min_read_count, p_min_opened_count, + p_cursor_count, + p_exclude_tags; +end; +$$; + +revoke execute on function lorekit_memory_list( + uuid, boolean, text, text, text, text, timestamptz, timestamptz, timestamptz, + timestamptz, text[], text, text[], text, text[], text, text[], text, text[], + text, text[], text, text[], text, text[], text, text[], text, text, + timestamptz, uuid, integer, text[], text, uuid[], integer, integer, integer, + integer, integer, + integer, integer, integer, integer, integer, + text[]) from public, anon; +grant execute on function lorekit_memory_list( + uuid, boolean, text, text, text, text, timestamptz, timestamptz, timestamptz, + timestamptz, text[], text, text[], text, text[], text, text[], text, text[], + text, text[], text, text[], text, text[], text, text[], text, text, + timestamptz, uuid, integer, text[], text, uuid[], integer, integer, integer, + integer, integer, + integer, integer, integer, integer, integer, + text[]) to authenticated, service_role; + +comment on function lorekit_memory_list( + uuid, boolean, text, text, text, text, timestamptz, timestamptz, timestamptz, + timestamptz, text[], text, text[], text, text[], text, text[], text, text[], + text, text[], text, text[], text, text[], text, text[], text, text, + timestamptz, uuid, integer, text[], text, uuid[], integer, integer, integer, + integer, integer, + integer, integer, integer, integer, integer, + text[]) is + 'The keyset-paginated memory list behind GET /memories and POST /memories/list. + Sorts by updated_at (default), created_at, opened_count (00113) or + last_opened_at (00115) — each descending with id as the tie-break; + last_opened_at is NULLS LAST, so never-opened lessons trail. p_cursor_ts + carries the timestamp sorts'' keyset value (NULL under last_opened_at with a + set p_cursor_id = inside the never-opened tail) and p_cursor_count the count + sort''s. Applies the five retention thresholds and the Explorer''s reverse + directions through the shared lorekit_match_retention, so its total_count + agrees with _facets, _activity and _pivot under the same parameters. + p_exclude_tags (00114): the row carries none of these labels, ANDed with + whatever p_tags/p_tags_mode ask.'; diff --git a/supabase/tests/migrations.test.sql b/supabase/tests/migrations.test.sql index ba23a9fa1..24ee52545 100644 --- a/supabase/tests/migrations.test.sql +++ b/supabase/tests/migrations.test.sql @@ -46,6 +46,9 @@ values ('00000000-0000-0000-0000-000000000000', '00000000-0000-0000-0000-0000000000a7', 'authenticated', 'authenticated', 'lk-mig-a7@test.local', now(), now()), -- §114's own account, for §108's reason: exact counts out of four readers. ('00000000-0000-0000-0000-000000000000', '00000000-0000-0000-0000-0000000000a8', 'authenticated', 'authenticated', 'lk-mig-a8@test.local', now(), now()), + -- §115's own account: its full-order and page-walk assertions compare + -- against the WHOLE (user, scope) population. + ('00000000-0000-0000-0000-000000000000', '00000000-0000-0000-0000-0000000000a9', 'authenticated', 'authenticated', 'lk-mig-a9@test.local', now(), now()), ('00000000-0000-0000-0000-000000000000', '00000000-0000-0000-0000-0000000000b2', 'authenticated', 'authenticated', 'lk-mig-b@test.local', now(), now()), ('00000000-0000-0000-0000-000000000000', '00000000-0000-0000-0000-0000000000c3', 'authenticated', 'authenticated', 'lk-mig-c@test.local', now(), now()), ('00000000-0000-0000-0000-000000000000', '00000000-0000-0000-0000-0000000000d4', 'authenticated', 'authenticated', 'lk-mig-d@test.local', now(), now()), @@ -11066,6 +11069,184 @@ begin end; $$; +-- ── 115. the "Recent read" sort — p_sort => 'last_opened_at' (00115) ──────── +-- 00115 lets the list order by the last DELIBERATE agent read (00099's +-- last_opened_at), not by last_read_at, which bulk list/search pages and the +-- dashboard's detail sheet also move. The column is NULL for a lesson nothing +-- ever opened, so the order is `desc nulls last` and the keyset carries a +-- nullable timestamp. +-- +-- Seven lessons on a fresh account, so every order and count is exact: +-- 115-recent opened 1 hour ago +-- 115-tie-1 opened at T (2 days ago) } the SAME timestamp: id breaks +-- 115-tie-2 opened at T (2 days ago) } the tie +-- 115-old opened 30 days ago +-- 115-listed NEVER opened, but last_read_at = now and read_count 500 — a +-- lesson that rides along in every bulk page +-- 115-never-a never opened +-- 115-never-b never opened +-- +-- AC-1: sort => 'last_opened_at' orders by last_opened_at desc NULLS LAST, +-- id desc — the most recently opened leads, and 115-listed sits in the +-- never-opened tail however recently it was LISTED. +-- AC-2: the sort is not a filter: total_count and the row count equal the +-- unsorted list's. +-- AC-3: walking it two rows at a time from each page's last row — a cursor +-- of (last_opened_at, id) — returns every row exactly once, across the +-- tie, across the opened→never-opened boundary, and inside the tail +-- (where the cursor's timestamp half is NULL). +-- AC-4: p_cursor_count means nothing to this sort and must not narrow it; a +-- NULL p_cursor_ts with a NULL p_cursor_id is the first page. +-- AC-5: the other orders are unchanged — created_at still orders by +-- created_at desc, id desc, and an unknown sort still falls back. +-- AC-6: still exactly one overload, still off the anonymous surface. +do $$ +declare + v_user uuid := '00000000-0000-0000-0000-0000000000a9'; + v_t timestamptz := now() - interval '2 days'; + v_n bigint; + v_total integer; + v_unsorted integer; + v_expected text[]; + v_got text[]; + v_page text[]; + v_walk text[] := '{}'; + v_pages integer := 0; + v_cur_ts timestamptz; + v_cur_id uuid; +begin + set local role service_role; + perform set_config('request.jwt.claims', + format('{"sub":"%s","role":"service_role"}', v_user), true); + + insert into memories (user_id, scope, key, value, created_at, + read_count, opened_count, last_read_at, last_opened_at) values + (v_user, 'global', '115-recent', 'v', now() - interval '50 days', 9, 4, now() - interval '1 hour', now() - interval '1 hour'), + (v_user, 'global', '115-tie-1', 'v', now() - interval '40 days', 7, 2, v_t, v_t), + (v_user, 'global', '115-tie-2', 'v', now() - interval '30 days', 7, 2, v_t, v_t), + (v_user, 'global', '115-old', 'v', now() - interval '60 days', 3, 1, now() - interval '30 days', now() - interval '30 days'), + (v_user, 'global', '115-listed', 'v', now() - interval '20 days', 500, 0, now(), null), + (v_user, 'global', '115-never-a', 'v', now() - interval '10 days', 0, 0, null, null), + (v_user, 'global', '115-never-b', 'v', now() - interval '5 days', 0, 0, null, null); + + -- ── AC-1: the order ───────────────────────────────────────────────────── + select array_agg(key order by last_opened_at desc nulls last, id desc) into v_expected + from memories where user_id = v_user and scope = 'global'; + + select array_agg(l.key order by l.ord), max(l.total_count) into v_got, v_total + from lorekit_memory_list(p_user_id => v_user, p_scope => 'global', + p_sort => 'last_opened_at', p_limit => 100) + with ordinality as l(id, scope, key, value, tags, source_agent, trigger, + created_at, updated_at, expires_at, archived_at, + origin_repo, origin_branch, origin_commit, origin_pr, + kind, host, seen_count, read_count, opened_count, + last_read_at, last_opened_at, org_id, created_by, + updated_by, org_name, org_slug, total_count, ord); + assert v_got = v_expected, + format('115 AC-1a: sort => last_opened_at must order by last_opened_at desc nulls last, ' + 'id desc. expected %s, got %s', v_expected, v_got); + assert v_got[1] = '115-recent', + format('115 AC-1b: the most recently opened lesson must lead, got %s', v_got); + assert v_got[4] = '115-old' and '115-listed' = any(v_got[5:7]), + format('115 AC-1c: never-opened lessons — including one LISTED a moment ago — must ' + 'trail every opened one, got %s', v_got); + assert array_position(v_got, '115-tie-1') between 2 and 3 + and array_position(v_got, '115-tie-2') between 2 and 3, + format('115 AC-1d: the two lessons opened at the same instant sit together, got %s', v_got); + + -- ── AC-2: a sort, not a filter ────────────────────────────────────────── + select max(total_count), count(*) into v_unsorted, v_n + from lorekit_memory_list(p_user_id => v_user, p_scope => 'global', p_limit => 100); + assert v_total = v_unsorted and v_total = 7 and v_n = 7, + format('115 AC-2: total_count must be the unsorted total (7), got sorted=%s unsorted=%s rows=%s', + v_total, v_unsorted, v_n); + + -- ── AC-3: the page walk ───────────────────────────────────────────────── + -- Pages of 2 over 7 rows: [recent, tie-x] ends inside the tie, [tie-y, old] + -- ends on the last OPENED row, [never, never] starts and ends inside the + -- tail (cursor timestamp NULL), [never] is the remainder. + loop + select coalesce(array_agg(l.key order by l.ord), '{}') into v_page + from lorekit_memory_list(p_user_id => v_user, p_scope => 'global', + p_sort => 'last_opened_at', p_limit => 2, + p_cursor_ts => v_cur_ts, p_cursor_id => v_cur_id) + with ordinality as l(id, scope, key, value, tags, source_agent, trigger, + created_at, updated_at, expires_at, archived_at, + origin_repo, origin_branch, origin_commit, origin_pr, + kind, host, seen_count, read_count, opened_count, + last_read_at, last_opened_at, org_id, created_by, + updated_by, org_name, org_slug, total_count, ord); + exit when cardinality(v_page) = 0; + v_walk := v_walk || v_page; + v_pages := v_pages + 1; + select id, last_opened_at into v_cur_id, v_cur_ts + from memories where user_id = v_user and key = v_page[cardinality(v_page)]; + exit when v_pages > 10; + end loop; + assert v_walk = v_expected, + format('115 AC-3a: the page walk must return every row exactly once, in order — no gap, ' + 'no overlap. walk=%s expected=%s', v_walk, v_expected); + assert v_pages = 4, + format('115 AC-3b: 7 rows at 2 per page is 4 pages, got %s', v_pages); + + -- ── AC-4: foreign cursor halves ───────────────────────────────────────── + select count(*) into v_n from lorekit_memory_list( + p_user_id => v_user, p_scope => 'global', p_sort => 'last_opened_at', + p_cursor_count => 0, p_limit => 100); + assert v_n = 7, + format('115 AC-4a: p_cursor_count must be ignored under last_opened_at, got %s rows', v_n); + + select count(*) into v_n from lorekit_memory_list( + p_user_id => v_user, p_scope => 'global', p_sort => 'last_opened_at', + p_cursor_ts => null, p_cursor_id => null, p_limit => 100); + assert v_n = 7, + format('115 AC-4b: no cursor id means the first page, got %s rows', v_n); + + -- ── AC-5: the other orders are unchanged ──────────────────────────────── + select array_agg(key order by created_at desc, id desc) into v_expected + from memories where user_id = v_user and scope = 'global'; + select array_agg(l.key order by l.ord) into v_got + from lorekit_memory_list(p_user_id => v_user, p_scope => 'global', + p_sort => 'created_at', p_limit => 100) + with ordinality as l(id, scope, key, value, tags, source_agent, trigger, + created_at, updated_at, expires_at, archived_at, + origin_repo, origin_branch, origin_commit, origin_pr, + kind, host, seen_count, read_count, opened_count, + last_read_at, last_opened_at, org_id, created_by, + updated_by, org_name, org_slug, total_count, ord); + assert v_got = v_expected, + format('115 AC-5a: created_at must still order by created_at desc, id desc. ' + 'expected %s, got %s', v_expected, v_got); + + select count(*) into v_n from lorekit_memory_list( + p_user_id => v_user, p_scope => 'global', p_sort => 'bogus', p_limit => 100); + assert v_n = 7, format('115 AC-5b: an unknown sort must fall back, not fail, got %s', v_n); + + reset role; + perform set_config('request.jwt.claims', '', true); +end; +$$; + +-- AC-6: the catalog side — 00115 replaced the body, not the signature. +do $$ +declare + v_sig text; +begin + assert (select count(*) from pg_proc p join pg_namespace n on n.oid = p.pronamespace + where n.nspname = 'public' and p.proname = 'lorekit_memory_list') = 1, + '115 AC-6a: lorekit_memory_list must still have exactly one overload'; + + select p.oid::regprocedure::text into v_sig + from pg_proc p join pg_namespace n on n.oid = p.pronamespace + where n.nspname = 'public' and p.proname = 'lorekit_memory_list'; + assert not has_function_privilege('anon', v_sig, 'EXECUTE'), + '115 AC-6b: anon must NOT have EXECUTE on lorekit_memory_list'; + assert has_function_privilege('authenticated', v_sig, 'EXECUTE') + and has_function_privilege('service_role', v_sig, 'EXECUTE'), + '115 AC-6c: authenticated and service_role MUST keep EXECUTE on lorekit_memory_list'; +end; +$$; + rollback; \echo 'migrations.test.sql: all assertions passed'