Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -439,6 +439,7 @@ their rationale inline. **Do not relitigate these.**
- **The Explorer's Duplicate Clusters panel is a PANEL, not an instrument** — [rationale](./docs/decisions.md#dashboard-analytics-reads-stay-rest-only)
- **Dashboard analytics live on one dedicated `/insights` route** — [rationale](./docs/decisions.md#dashboard-analytics-live-on-one-dedicated-insights-route)
- **A COUNT must describe the rows its LIST returns — retention thresholds reach all four readers** — [rationale](./docs/decisions.md#a-count-must-describe-the-rows-its-list-returns)
- **Age/activity conditions read both ways (00113), but the reverse directions are READ-only — never in `GroomConditionsSchema`** — [rationale](./docs/decisions.md#the-ageactivity-conditions-read-both-ways-but-only-one-way-reaches-a-policy)
- **A read must never restamp `updated_at`** — [rationale](./docs/decisions.md#a-read-must-never-restamp-updated_at)
- **Lore value is the RATIO `opened_count / read_count`, and `/insights` leads with the bill** — [rationale](./docs/decisions.md#lore-value-is-a-ratio-and-insights-leads-with-the-bill)
- **A citation is the agent's word, and it is a fact about a RUN** — [rationale](./docs/decisions.md#a-citation-is-the-agents-word-and-it-is-a-fact-about-a-run)
Expand Down
24 changes: 24 additions & 0 deletions docs/decisions.md
Original file line number Diff line number Diff line change
Expand Up @@ -749,6 +749,30 @@ directories (`packages/cli/skill/*`) — the Claude-plugin-vendored mirror under
`scripts/codegen/sync-plugin-skill.mjs --check` (run inside the `plugin` CI job), so the two guards
compose instead of duplicating each other's job.

## The age/activity conditions read both ways, but only one way reaches a policy

The five retention conditions all point the archive way ("N or fewer", "not opened for N days"), so the
Explorer could show the dead weight and never the load-bearing lore. 00113 adds the mirror of four of them
— `opened_within_days`, `min_seen_count`, `min_read_count`, `min_opened_count` — plus `sort=opened_count`.

They are **read conditions only**: they reach the list and the three aggregates (the rule below), and are
deliberately absent from `GroomConditionsSchema`. Archiving "everything chosen at least 10 times" is not a
policy anyone wants, and a schema able to express it is one careless edit from archiving the most useful
lore in the store. The Explorer therefore hides **Create retention policy** while one is set, rather than
saving a policy that would archive a different set from the one on screen
(`retentionConditionsToGroomConditions` drops them as a second line).

In the menu each pair shares ONE row and label, and that row's value list holds BOTH directions as ordinary
rows under two headings; a row holds one direction at a time (setting one clears the other). Two menu rows
per metadata row would double the level-one list for a choice that is a property of the row. A direction
TOGGLE above the value list was tried first and replaced: it was a mode that hid half the values, cost a
click and a Tab stop before they were even visible, and typing could only ever fill the side it was set to.
As rows, typing follows the list: a bare number offers both directions, `>N` / `<N` pick one, and a strict
comparison on a COUNT is stored as the next whole number (`>40` → 41) so the row reads as the filter it
applies. `opened_within_days` does **not** inherit 00100's never-opened fallback: a lesson
nothing ever opened was not "opened recently". Created has no reverse — the date-range picker already
answers "created within".

## A COUNT must describe the rows its LIST returns

The five retention conditions (`min_age_days`, `unseen_days`, `max_seen_count`, `max_read_count`,
Expand Down
2 changes: 2 additions & 0 deletions docs/key-files.md
Original file line number Diff line number Diff line change
Expand Up @@ -177,6 +177,8 @@ else. Grouped loosely by subsystem in migration/PR order.
| `packages/mcp-core/src/scope/memory-ref.spec.ts` | One case table run against THREE implementations — mcp-core's copy, the edge copy (imported directly from `supabase/functions/_shared/`), and the CLI's `resolveScopeArg` — because the grammar has to mean one thing on every surface. It caught the `validateScope` delegation on its first run: the edge copy split the branch-scope case one `::` early |
| `packages/mcp-core/src/memory/read-refs.ts` (+ its edge mirror) | The pure core behind batch `memory.read` (`refs`), shared by MCP's `toolRead` and REST's `handleRead`. Import-free by construction — it takes only already-PARSED `{scope,key}` pairs, never re-deriving the `scope::key` grammar `parseMemoryRefs` alone owns. `groupRefsByScope` collapses N refs into one `{scope, keys}` group per DISTINCT scope (plan D5) — the transport issues exactly one `.eq('scope', s).in('key', keys)` query per group, awaited concurrently, instead of one query per ref. `isQueryableKey` rejects a key carrying a comma/paren/quote/backslash BEFORE it reaches the query, since postgrest-js's `.in()` filter quotes a comma/paren but does not escape a quote or backslash inside the value — a bad ref becomes a `missing` entry rather than a malformed filter list. `missingRefs` reports every unresolved ref as its own caller-supplied `scope::key` string, in REQUEST order — but it only ever sees what `parseMemoryRefs` RETURNED, so a ref dropped as malformed or truncated past the 32-ref cap reaches neither `entries` nor `missing` (the gap is visible as `lorekit.refs.requested` vs `lorekit.refs.count`); matching stays against the VERBATIM stored scope, deliberately diverging from the singular `memory.read` path's `validateScope` normalisation, because the transports differ in what they hand `memory_write` and no reader can infer how a row was cased — see `docs/decisions.md#memoryread-batching-is-conditional-verbatim-scoped-and-inflates-a-signal-it-doesnt-count` |
| `supabase/migrations/00108_retention_conditions_on_aggregates.sql` | Gives `lorekit_memory_facets` (24→31 args), `lorekit_memory_activity` (26→31) and `lorekit_memory_pivot` (27→34) the five retention thresholds `lorekit_memory_list` has had since 00092, plus a `created_since`/`created_until` pair on facets and pivot (activity's own `since`/`until` already bound `created_at`). **The bug it fixes is a DISAGREEMENT, not a missing feature**: only the list applied the thresholds, so `max_opened_count => 0` narrowed the Explorer's rows and left its facet counts, stat cards and matrix counting the un-narrowed population — every number on the page describing a different set from the one below it. Introduces `lorekit_match_retention`, ONE inlinable row test now composed by all four readers, so a fourth inline copy cannot drift (00101 records that exact drift happening once already between two copies). It takes CUTOFF INSTANTS rather than day counts, which is what keeps it `immutable` and index-usable — each caller resolves its `now()`-relative bounds once per query in a declare block — and it holds 00100's never-opened rule (`coalesce(last_opened_at, created_at)`) in one place. **Applied in the `base` CTE's WHERE, never as an `ok_*` flag**, so a threshold narrows even the SELF-EXCLUDED dimension: self-exclusion exists so the dimension you are standing in still lists what you could switch to, and switching `host` does not stop you looking at lore older than 30 days. The three aggregates are DROPPED at their exact old signatures and recreated (`create or replace` cannot change a signature, and leaving both overloads makes every PostgREST named-arg call ambiguous — 00092's footgun); `lorekit_memory_list` is a true `create or replace` at its verbatim 00105 signature, so its grants persist. Pinned by §108 |
| `supabase/migrations/00113_explorer_reverse_activity_filters_and_opened_sort.sql` | The Explorer's REVERSE directions: `opened_within_days` and the `min_seen_count`/`min_read_count`/`min_opened_count` floors, the mirror of four of the five retention thresholds, for "which lore IS used?" rather than "what can I archive?". Widens `lorekit_match_retention` 10→14 args (the 10-arg form is dropped LAST, after every caller is recreated), appends the four to `_facets`/`_activity`/`_pivot` and, with a `p_cursor_count`, to `lorekit_memory_list`, which also accepts `p_sort => 'opened_count'` (the format() splices `$44` or `$29` as the keyset placeholder). `opened_within_days` does NOT age a never-opened lesson from `created_at` — the opposite of 00100's rule for `unseen_days`, on purpose. READ-only: no groom RPC or `GroomConditionsSchema` change. Restores the `anon` revoke on `lorekit_memory_list` that 00105's drop-and-recreate lost. Pinned by §113; §108's helper-ACL check now derives the signature |
| `packages/mcp-core/src/rest/count-cursor.ts` | The `sort=opened_count` keyset cursor (`{ id, opened_count }`, base64url) and its `buildCountPage`. Separate from `_shared/api/paginate.ts` because that cursor carries a timestamp and is byte-mirrored into the MCP function, which has no count sort. Mirrored verbatim into `supabase/functions/_shared/api/count-cursor.ts`, guarded by `edge-parity.spec.ts`; tested by `count-cursor.spec.ts` |
| `supabase/functions/_shared/api/retention.ts` | 00108's fix one layer up: `retentionFrom` picks the five thresholds off a validated query or body, `retentionRpcParams` maps them to the `p_*` arguments all four RPCs share. Used by `list.ts` and the three aggregate handlers, so a route cannot be given the parameter and then forget to forward it. `?? null` and never a truthiness check — `max_opened_count => 0` is the whole point of 00105, so a falsy test turns the most useful threshold in the set into no filter at all |
| `packages/web/src/lib/retention-filter.ts` → `retentionConditionsToAggregateBody` | The client half. Delegates to `retentionConditionsToListBody` (the field names are identical by construction — one shared `retentionConditionBodyFields` spread in the schemas) but is typed against `ListFacetsBody`, so a schema that stops carrying one of the five is a TYPE error rather than a threshold the aggregate silently ignores. Its spec reads each field back off the PARSED schema output rather than asserting `safeParse(...).success`: these schemas are not `.strict()`, so zod strips an unknown key and still reports success — a success-only check would pass against the pre-00108 routes and prove nothing |
| `packages/cli/src/core/lessons.mjs` → `citationHint` | The Stop-hook ask. Reads the session's already-recorded shown-set and names those `scope::key` refs, capped at `CITED_HINT_MAX` (8) with a `(+N more)` tail — an agent picking from a list is a different task from one recalling a convention. Empty when nothing was injected (with no candidates the ask is an instruction to invent references). Refs are taken in INJECTION order, so the SessionStart set leads |
1 change: 1 addition & 0 deletions docs/otel.md
Original file line number Diff line number Diff line change
Expand Up @@ -917,6 +917,7 @@ attribute here.
| `lore.memory_archive_toggled` | A memory was archived or restored | `lorekit.lore.memory.archive_action`, `.archive_outcome` |
| `lore.filter_changed` | Any filter-bar mutation, age/activity thresholds included | `lorekit.lore.filter.action`, `.field` (optional), `.operator` (optional), `.count`, `.retention_count` |
| `lore.status_changed` | Active / Archived / Expiring | `lorekit.lore.status` |
| `lore.sort_changed` | The list order was picked (Newest / Most chosen) | `lorekit.lore.sort` (`created_at`/`opened_count`) |
| `lore.search_committed` | The debounced search settled | `lorekit.lore.search.length` |
| `lore.scope_selected` | A scope chip or Browse-all row was picked | `lorekit.lore.scope.type`, `.source` (`strip`/`browse-all`) |
| `lore.range_changed` | Any of the FIVE controls that write `?range=` | `lorekit.lore.range.preset`, `.source`, `.span_days` (omitted when unbounded) |
Expand Down
15 changes: 10 additions & 5 deletions packages/cli/src/shared/deeplink-pure.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -46,9 +46,13 @@ export const LORE_PARAM_DEFAULTS = {
// "unfiltered".
filters: null,
tags: [], // string[] — legacy label filter (AND across labels); [] means "no filter". Still READ by the app, superseded by `filters`
// RetentionConditions | null — the retention-preview trio (min age / unseen-for
// / seen-at-most). `null` means no narrowing; absent from the URL when unset.
// RetentionConditions | null — the Explorer's age/activity conditions (the
// five retention thresholds plus, since 00113, their four reverse directions).
// `null` means no narrowing; absent from the URL when unset.
retention: null,
// 'created_at' | 'opened_count' | null — the list order (00113). `null` is
// Newest, the default, so an unsorted link carries no `sort` param.
sort: null,
// 'active' | 'archived' | 'expiring' | null — the Explorer's Status control.
// `null`, NOT 'active', is the default for `filters`' reason: the app has to
// tell "absent" from an explicit choice, because an absent `status` falls back
Expand All @@ -62,9 +66,9 @@ export const LORE_PARAM_DEFAULTS = {

// A stable, readable param order (also makes URLs deterministic for tests).
// Mirrors the `useUrlState` call order in `LoreExplorer.tsx` (+ the `lesson`
// param last), so `filters` and `tags` sit between `owner` and `status`, and
// `retention` sits between `tags` and `status`. `scope` precedes `lesson` so a
// lesson link reads `?scope=…&lesson=…`.
// param last), so `filters` and `tags` sit between `owner` and `status`,
// `retention` sits between `tags` and `status`, and `sort` follows `archived`.
// `scope` precedes `lesson` so a lesson link reads `?scope=…&lesson=…`.
const PARAM_ORDER = [
'scope',
'q',
Expand All @@ -75,6 +79,7 @@ const PARAM_ORDER = [
'retention',
'status',
'archived',
'sort',
'lesson',
];

Expand Down
1 change: 1 addition & 0 deletions packages/cli/src/shared/mirror-pairs.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@ export const mirrorPairs = [
{ core: 'packages/mcp-core/src/telemetry/usage-stats.ts', edge: 'supabase/functions/_shared/telemetry/usage-stats.ts', driftChecked: true },
{ core: 'packages/mcp-core/src/limits/expiring-window.ts', edge: 'supabase/functions/_shared/limits/expiring-window.ts', driftChecked: true },
{ core: 'packages/mcp-core/src/rest/cors-origins.ts', edge: 'supabase/functions/_shared/api/cors-origins.ts', driftChecked: true },
{ core: 'packages/mcp-core/src/rest/count-cursor.ts', edge: 'supabase/functions/_shared/api/count-cursor.ts', driftChecked: true },
{ core: 'packages/mcp-core/src/scope/scope-type-attribute.ts', edge: 'supabase/functions/_shared/scope/scope-type-attribute.ts', driftChecked: true },
// Which row an UNSCOPED read resolves to. Import-free on both sides (it only
// parses the scope prefix and compares strings), so the byte comparison
Expand Down
67 changes: 67 additions & 0 deletions packages/mcp-core/src/rest/count-cursor.spec.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
import { describe, it, expect } from 'vitest';
import { buildCountPage, decodeCountCursor, encodeCountCursor } from './count-cursor.ts';

const ID = '0f8fad5b-d9cb-469f-a165-70867728950e';
const ID2 = '7c9e6679-7425-40de-944b-e07fc1f90ae7';

/** A `paginate.ts`-shaped date cursor, built the way that module builds one. */
function dateCursor(id: string, ts: string): string {
return btoa(JSON.stringify({ id, updated_at: ts })).replace(/\+/g, '-').replace(/\//g, '_').replace(/=/g, '');
}

describe('count cursor codec', () => {
it('round-trips an id and a count', () => {
expect(decodeCountCursor(encodeCountCursor(ID, 12))).toEqual({ id: ID, count: 12 });
});

it('keeps a count of 0 — the never-chosen tail is a real page boundary', () => {
expect(decodeCountCursor(encodeCountCursor(ID, 0))).toEqual({ id: ID, count: 0 });
});

it('emits URL-safe text with no padding', () => {
expect(encodeCountCursor(ID, 123_456)).toMatch(/^[A-Za-z0-9_-]+$/);
});

it('refuses a DATE cursor, so flipping sort mid-pagination restarts at page one', () => {
expect(decodeCountCursor(dateCursor(ID, '2026-10-01T00:00:00Z'))).toBeNull();
});

it.each([
['not base64 at all', '%%%'],
['valid base64, not JSON', btoa('hello')],
['JSON, not an object', btoa('42')],
['non-uuid id', btoa(JSON.stringify({ id: 'abc', opened_count: 1 }))],
['string count', btoa(JSON.stringify({ id: ID, opened_count: '1' }))],
['fractional count', btoa(JSON.stringify({ id: ID, opened_count: 1.5 }))],
['negative count', btoa(JSON.stringify({ id: ID, opened_count: -1 }))],
['count past int4', btoa(JSON.stringify({ id: ID, opened_count: 2_147_483_648 }))],
])('fails closed on %s', (_label, raw) => {
expect(decodeCountCursor(raw)).toBeNull();
});
});

describe('buildCountPage', () => {
const rows = [
{ id: ID, opened_count: 9 },
{ id: ID2, opened_count: 3 },
{ id: '00000000-0000-4000-8000-000000000001', opened_count: 3 },
];

it('reports more pages from the overflow row and cursors on the LAST returned row', () => {
const page = buildCountPage(rows, 2);
expect(page.entries).toHaveLength(2);
expect(page.hasMore).toBe(true);
expect(decodeCountCursor(page.nextCursor ?? '')).toEqual({ id: ID2, count: 3 });
});

it('has no cursor on the last page', () => {
const page = buildCountPage(rows, 5);
expect(page.hasMore).toBe(false);
expect(page.nextCursor).toBeNull();
});

it('reads a missing counter as 0 rather than minting an undecodable cursor', () => {
const page = buildCountPage([{ id: ID }, { id: ID2 }], 1);
expect(decodeCountCursor(page.nextCursor ?? '')).toEqual({ id: ID, count: 0 });
});
});
Loading
Loading