refactor(core): bind vector adapters to the database and search them over a ProjectScope - #1566
Merged
phernandez merged 1 commit intoSep 15, 2026
Conversation
…over a ProjectScope The vector adapters were bound to one project through VectorIndexScope.project_id, which made a multi-project search impossible to express: the reader could only ask the adapter for the project it was built for. The project was never part of the vector's identity (entity ids are database-wide primary keys); it is the partition an operation touches. VectorIndexScope is now the database namespace plus the embedding schema. Every write names its project (upsert, delete, delete_entity, delete_orphans take project_id first) and search takes a ProjectScope, so one sqlite-vec or pgvector adapter answers any set of projects with one statement through the scope's IN predicate. An empty scope returns nothing without touching storage. Milvus keeps a collection per project, validates each collection on its first use instead of in initialize(), and searches the collections in scope, merging by similarity. SemanticSearch passes its own scope to the adapter, so a project repository's vector search is unchanged. The repositories build the scope without a project and the factory no longer takes one. Two dead lookup helpers in the built-in adapters go with the change. Test doubles implement the new signatures; the pgvector, sqlite-vec, and Milvus suites gain multi-project and empty-scope cases. No query behavior changes. Part of #1558. Co-Authored-By: Claude Fable 5.1 <[email protected]> Claude-Session: https://claude.ai/code/session_019YW9ysxugGGBCNEGzsxtFV Signed-off-by: phernandez <[email protected]>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #1558. Fourth PR in the stack, on top of #1565 (which is on #1563, on #1562). First half of PR 3: the adapters have to read a set of projects before a route can build a
SearchReaderover one.What changes
The vector adapters were bound to one project through
VectorIndexScope.project_id, so the reader could only ask an adapter for the project it was built for. The project was never part of a vector's identity (entity ids are database-wide primary keys); it is the partition an operation touches.VectorIndexScopeis now the database namespace plus the embedding schema (namespace,embedding_identity,dimensions).storage_keyis gone.upsert(project_id, records),delete(project_id, records),delete_entity(project_id, entity_id),delete_orphans(project_id, live_keys).search(query, *, limit, projects: ProjectScope). The built-in adapters bind the scope's ids throughProjectScope.predicate, so sqlite-vec and pgvector answer any set of projects with one statement. An empty scope returns nothing without touching storage.initialize()has nothing shared to prepare, so each project's collection is created or validated on first use, once per adapter instance. A search over several projects asks each collection for its own toplimitand merges by similarity.SemanticSearchpasses its scope to the adapter, so a project repository's vector search runs the same statement as before with a scope of one. The repositories build the scope without a project;build_vector_index_scopeandcreate_semantic_vector_indexno longer take one.No query behavior changes.
Tests
Adapter doubles implement the new signatures. New cases:
e.project_id IN (:scope_0, :scope_1)andc.project_id IN (:scope_0, :scope_1); an empty scope makes no statement.initialize()touches nothing; first use creates or validates the collection; empty operations still touch nothing.just fast-checktests/repository, SQLitetests/repository, Postgres (testcontainers)tests/services,tests/api,tests/mcp, SQLitetest-int/semantic/test_milvus_lite.py,test-int/test_embedding_status_vec0.pyjust doctorTrade-off to note
Milvus used to validate the project's collection at startup through
initialize(). With no project on the scope, that validation moves to the first operation against each project. A dimension mismatch on an existing collection now surfaces on the first write or search rather than at boot.Next in the stack
PR 3b adds
QUERY /v2/search/and a scoped service method that builds aSearchReaderover a multi-projectProjectScopewithout a repository.🤖 Generated with Claude Code
https://claude.ai/code/session_019YW9ysxugGGBCNEGzsxtFV