Skip to content

feat(api): search an explicit set of projects with one query - #1567

Merged
phernandez merged 1 commit into
feat/1558-scoped-search-3a-vector-scopefrom
feat/1558-scoped-search-3b-scoped-route
Sep 15, 2026
Merged

phernandez merged 1 commit into
feat/1558-scoped-search-3a-vector-scopefrom
feat/1558-scoped-search-3b-scoped-route

Conversation

@phernandez

Copy link
Copy Markdown
Member

Part of #1558. Fifth PR in the stack, on top of #1566 (which is on #1565, #1563, #1562). Closes the API half of the cloud contract in basicmachines-co/basic-memory-cloud#2080.

What changes

Route. QUERY /v2/search/ (and POST /v2/search/ for clients that cannot send QUERY) searches an explicit set of projects in one database with one query. The body is ScopedSearchQuery: the project search body plus project_ids, a required list of internal ids the caller has already authorized. An empty list answers no rows; there is no way to ask for every project. page and page_size are validated (ge=1, page_size <= 1000). The response is the existing SearchResponse. The route sets Cache-Control: no-store: the read cache is keyed by one project's generation, and a set of projects has no single generation to invalidate on.

Composition. create_search_reader(session_maker, scope, app_config) builds a SearchReader over a ProjectScope with no project repository. It resolves the same shared embedding provider, vector adapter, and reranker create_search_repository gives a repository, so one project through this route ranks exactly as its own route does, and several projects are one ranking over the union rather than per-project pages merged afterwards. Whether semantic retrieval is available is decided once from configuration; a semantic query against a reader built without it fails as a disabled feature (400).

Service. ScopedSearchService(session_maker, scope, reader) runs and hydrates a scoped search. Hydration stays inside the scope: owning entities, relation endpoints, project identities, and valid-time assertions are all read under the scope predicate, so a page never names something outside it. It satisfies the hydration protocols the existing helpers already take, so to_search_results and load_temporal_metadata are reused unchanged in shape.

Shared preparation. The query preparation SearchService already had moves to module functions both services call: prepare_search_query, include_legacy_note_type_spellings (now bounded to the scope it serves; a scope cannot learn what another project stores), and relaxed_fts_fallback_eligible. SearchService methods delegate to them; nothing changes for the project route's callers.

Results. Every SearchResult, on both routes, now carries project_id and project_external_id (additive, optional). The project route fills them from its path.

Smaller pieces. Both search routes share one search_error_boundary for the status mapping (400 / 503 / 502 / 400). SemanticSearch returns before embedding when the scope is empty. The sqlite-vec adapter converts a missing enable_load_extension into the same typed dependency error the repository raises for #711, so a scoped vector search on such a host answers 400 rather than 500.

Design notes

  • Identity keys stay (type, id). Search row ids are the entity, observation, and relation primary keys, which are database-wide, so (type, id) is unique across projects in one database and rows carry project_id through every stage. The test corpus uses distinct ids per project the way real data does, rather than the artificial cross-project id collision in feat(core): add explicit multi-project API search #1559's corpus.
  • Milvus is supported. A multi-project scope searches each project's collection and merges (PR 3a), so there is no "unsupported adapter" error on this route.
  • Vector order and pagination stability are unchanged and land in PR 4 with the tenant-snapshot benchmark, for both routes at once.

Tests

tests/api/v2/test_scoped_search_router.py seeds three real projects with real search rows and real vectors (only the embedding provider is a double, and it produces different vectors for different text) and covers, on both backends: one pipeline per mode over scopes of 0, 1, and 2 projects with no writes and one embedding per search; pages as stable slices; stale, re-modelled, and pending manifests excluded; hybrid lexical-only rows under a threshold; note-type expansion bounded to the scope; hydration bounded to the scope; filters with project identity on results; scope shaping the answer; POST as the documented twin; 422 for missing, empty-typed, or invalid scopes and for page=0; temporal filter and explanation within scope; pagination and empty later pages; 400 when semantic is disabled; project identity on the project route; relaxed lexical queries and strict-match suppression across pages; invalid and boolean queries not broadening the lexical channel.

New unit modules cover create_search_reader composition and the sqlite-vec dependency error. The reader gains an empty-scope test.

Check Result
just fast-check passed
new and touched modules 74 passed
tests/repository, tests/services, tests/api, tests/mcp, SQLite 3530 passed, 40 skipped, plus the one telemetry test fixed after that run for the route's new dependency parameter (46 passed on the search route files after the fix)
tests/repository, both search routes, test_search_service, Postgres 1244 passed, 42 skipped
just doctor passed

New route, service, and factory are at 100% line coverage in the unit suites.

Next in the stack

PR 4: vector order per request shape and ANN pre-filter, benchmarked on a tenant snapshot. PR 5: MCP search_all_projects on this route and the scoped reader.

🤖 Generated with Claude Code

https://claude.ai/code/session_019YW9ysxugGGBCNEGzsxtFV

QUERY /v2/search/ (and POST /v2/search/ for clients that cannot send QUERY)
searches an explicit set of projects in one database. The body is the project
search body plus project_ids, a required list of internal ids the caller has
already authorized; an empty list answers no rows and there is no spelling for
every project.

create_search_reader composes a SearchReader over a ProjectScope with no project
repository, resolving the same shared embedding provider, vector adapter, and
reranker a repository gets, so one project through this route ranks exactly as
its own route does and several projects are one ranking over the union.
ScopedSearchService runs the query and hydrates only from projects in scope:
owning entities, relation endpoints, project identities, and valid-time
assertions are all read under the scope predicate.

The query preparation the project service already had (criteria normalization,
legacy note-type expansion, relaxed full-text eligibility) moves to module
functions both services call, with the note-type expansion now bounded to the
scope it serves. Every SearchResult, on both routes, carries project_id and
project_external_id. The two search routes share one error boundary. An empty
scope returns before embedding.

The sqlite-vec adapter converts a missing enable_load_extension into the same
typed dependency error the repository raises, so a scoped vector search on such
a host answers 400 rather than 500.

Part of #1558.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_019YW9ysxugGGBCNEGzsxtFV
Signed-off-by: phernandez <[email protected]>
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@phernandez
phernandez added this pull request to stack #1564 September 15, 2026 22:21
@phernandez
phernandez merged commit 06b5f93 into main Sep 15, 2026
51 of 52 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant