Repository navigation
perf(web): fetch the scope tree once on /lore instead of twice - #693
Conversation
Co-authored-by: dash0-dev[bot] <257284812+dash0-dev[bot]@users.noreply.github.com>
useLoreData fetched GET /memories/scopes itself while LorePageClient also mounted useScopeTree, so every /lore load ran the lorekit_memory_scopes RPC twice, concurrently. fetchLoreData now reads the scope tree through the shared ['lore-scopes'] query via queryClient.fetchQuery, which joins an in-flight fetch or serves a fresh cached tree. One shared options object keeps the key and stale time defined once. Co-authored-by: dash0-dev[bot] <257284812+dash0-dev[bot]@users.noreply.github.com>
Co-authored-by: dash0-dev[bot] <257284812+dash0-dev[bot]@users.noreply.github.com>
✅ No issues foundNo code change since the last review at Checked: 0 of 7 dependent files traced Progress: open review threads 1 → 0 → 0 across the last 3 reviews What this change reaches — 2 changed exports · 7 dependent files · 0 checked · 7 not checkedflowchart LR
subgraph pr["Changed in this PR"]
s1["useLoreData<br/>body changed"]:::changed
s2["useFacetCatalog<br/>body changed"]:::changed
end
s1c1["packages/web/src/components/lore/ · 2 files<br/>? not checked"]:::unknown
s1c2["packages/web/src/app/(dashboard)/lore/LorePageClient.tsx<br/>? not checked"]:::unknown
s1c3["packages/web/src/components/command/NavigationCommands.tsx<br/>? not checked"]:::unknown
s1c4["packages/web/src/components/providers/MemorySidebarProvider.tsx<br/>? not checked"]:::unknown
s1c5["packages/web/src/lib/queries/lore-scopes-dedupe.spec.ts<br/>? not checked"]:::unknown
s2c1["packages/web/src/components/lore/LoreExplorer.tsx<br/>? not checked"]:::unknown
s2c2["packages/web/src/components/settings/GroomingRuleBuilder.tsx<br/>? not checked"]:::unknown
s1 -.-> s1c1
s1 -.-> s1c2
s1 -.-> s1c3
s1 -.-> s1c4
s1 -.-> s1c5
s2 -.-> s2c1
s2 -.-> s2c2
classDef changed fill:#eef2ff,stroke:#6366f1,color:#1e1b4b
classDef ok fill:#e7f6ec,stroke:#16a34a,color:#14532d
classDef partial fill:#fef9c3,stroke:#ca8a04,color:#713f12
classDef unknown fill:#f3f4f6,stroke:#9ca3af,stroke-dasharray:4 3,color:#374151
classDef bad fill:#fde8e8,stroke:#dc2626,color:#7f1d1d
classDef warn fill:#fff4e5,stroke:#d97706,color:#78350f
classDef imports fill:#f8fafc,stroke:#64748b,stroke-dasharray:4 3,color:#1e293b
Review details
Found Quality — produced 0 → posted inline 0 · cleared 0 · carried forward 0 · deferred 0 · below-bar 0 Run incremental · 0 lines in delta · tier deep · depth checkout Nothing to report — standards (skipped), optimality (skipped), measurability (skipped), integrations (skipped), severity, 0 files skipped.
|
…ally mount useLoreData first Addresses @dash0-dev[bot]'s comment: #693 (comment) Refs: #693 Co-authored-by: dash0-dev[bot] <257284812+dash0-dev[bot]@users.noreply.github.com>
…seScopeTree keeps its retry policy Addresses @dash0-dev[bot]'s comment: #693 (comment) Refs: #693 Co-authored-by: dash0-dev[bot] <257284812+dash0-dev[bot]@users.noreply.github.com>
|
The dashboard preview for this PR — redeployed on each push that changes the web app.
Warning The preview deploy for Vercel deploy timed out after 15m. A deployment that never becomes Ready is almost always BLOCKED by Vercel — most commonly because the commit author's email can't be matched to a GitHub account with access to the project. Check the commit author and the project's Git deployment-protection settings. A blocked deployment is most often the commit author's email not matching a GitHub account with project access. Fix the cause, then push again or comment |
Summary
Every load of the dashboard's Lore Explorer (
/lore, the default landing route) sentGET /functions/v1/memories/scopestwice at the same time, so thelorekit_memory_scopesPostgres RPC ran twice and the two copies slowed each other down. This affects every signed-in dashboard user on their first page. With this changeuseLoreDatareads the scope tree through the shared['lore-scopes']query, so when both hooks mount together the browser sends one request.Evidence (production telemetry)
CALL lorekit_memory_scopesDB client spans (serviceapi,deployment.environment.name=production) that land in the same second as another call: 45.5% in the current window 2026-10-10T08:33:09Z..2026-10-17T08:33:09Z (111 of 244 calls) vs 36.7% in the baseline 2026-09-19T08:33:09Z..2026-10-10T08:33:09Z (277 of 754 calls). Related:memories.scopesREST p95 is 2.137 s now vs 1.740 s in the baseline. The RPC averages ~640 ms, and 21.7% of calls take over 1 s. Browser→edgeGET /functions/v1/memories/scopesaverages 1059 ms (98 calls, current window).d04200003f11316c908f8bc0b4356136: twolorekit_memory_scopescalls started 5 ms apart (2026-10-11T06:01:32.346Z and .351Z) and each took 4.13 s for 66 rows. Web/loreSSR trace371aa694f8c77fa430b38fa02d9f7b2f.Root cause
LorePageClient.tsx#L10mountsuseScopeTree(), and#L17mountsuseLoreData()next to it.useScopeTree(lore.ts#L316-L324) fetches['lore-scopes']throughfetchScopes(#L153-L157), which callslistScopesRequest.useLoreData(#L361-L369) runsfetchLoreData, which callslistScopesRequestagain on its own (#L341) and builds the same tree withbuildScopeTree(#L352).['lore-scopes']and['lore']), so TanStack Query cannot de-duplicate them. Two identical requests leave at the same moment, which is the same-second pair the signal counts.useLoreDatacannot simply be removed.NavigationCommands,MemoryExpandButton(which reads itsscopes) andMemorySidebarProvideralso use it, so itsscopesfield keeps the same shape and content.Change
packages/web/src/lib/queries/lore.ts: the scope-tree query (key['lore-scopes'],fetchScopes, 90 sstaleTime,retryUnlessSignedOut) is now one sharedqueryOptionsobject.useScopeTreeuses it as is.fetchLoreDatagets theQueryClient(fromuseQueryClient()inuseLoreData). It now gets the scopes half fromqueryClient.fetchQuery(scopeTreeQueryOptions)instead of callinglistScopesRequestitself. In@tanstack/query-core5.101.4 (the version in the lockfile),fetchQueryreturns the cached tree while it is fresh. While a fetch is in flight,Query.fetchreturns that fetch's promise instead of starting a new one. So/loremakes one network call whichever hook starts first.retryoverride on that call. The retry policy is captured by whichever caller starts the shared fetch, so a no-retry override would leave auseScopeTreethat joins a fetch started byuseLoreDatafailing on a single transient error. The accepted cost: during a sustained scopes outage, the['lore']query's own retry re-runs an already-retried, backoff-spaced fetch.['lore-scopes']and['lore'], so after an archive or restore both queries refetch and still share one scopes request.No user-facing docs apply: internal request de-duplication with no observable behaviour change.
Test
packages/web/src/lib/queries/lore-scopes-dedupe.spec.ts. The package has no DOM test environment and no@testing-library/react, so the spec runs the realuseScopeTreeanduseLoreDataas plain functions.useQueryis replaced by a recorder of the options each hook passes,useQueryClientreturns the test client, and the recorded options are fetched through one realQueryClient.listScopesRequest,listMemoriesRequest,activityRequestand the browser-token helper are mocked./memories/scopesis requested exactly once when both hooks mount, in either order (the reverse-order case mountsuseLoreDatafirst and waits until its scopes request is in flight before mountinguseScopeTree), and thatuseLoreData().scopesequals the scope tree. It also checks thatuseLoreDataalone still fetches the tree, and that auseScopeTreejoining a fetch started byuseLoreDatasurvives one transient failure instead of erroring.lore.ts; both mount-order tests failed withexpected "vi.fn()" to be called 1 times, but got 2 times. The transient-failure test failed withError: transientwhile aretry: falseoverride was in place.npx vitest run --root packages/web: 108 files, 1589 tests), the web typecheck (tsc --noEmit -p packages/web/tsconfig.json) passes, and ESLint on both touched files reports 0 problems.How we will know it worked
Re-run the
dup_pctquery above (unit: percent) over a production window after deploy. It should go down from 45.5%. The weekly loop re-measures it on or after 2026-10-24. Some same-second calls will remain: the server-side onboardinggetKeyScopeCatalog()call on/loreand other pages that read scopes also run this RPC, so the number will not reach zero.Opened by the weekly Agent0 telemetry-driven improvement automation. It is merged automatically when the review loop converges with an approving verdict and CI is green (and, for a web UI change, when the ui-verify test passes on this PR's Vercel preview), unless it carries the needs-human-merge label.