feat(core): add explicit multi-project API search

[phernandez]

Why

Cloud currently repeats retrieval and query embedding for each selected project in a tenant database. This adds an API-only, explicitly scoped read pipeline that ranks the selected projects together and paginates once.

Closes #1558. Companion: Cloud #2080 and corrected contract.

What Changed

• Add read-only MultiProjectSearchRepository, requiring an explicit collection of internal database project IDs. Missing/null scope is invalid; an empty collection returns no rows.
• Add QUERY /v2/search/. Example body: {"project_ids":[1,2],"text":"nebula","retrieval_mode":"hybrid"}. Results include project_id and project_external_id alongside the existing search-result fields.
• Support FTS, sqlite-vec, and pgvector. Unsupported semantic adapters return HTTP 400 with code unsupported_multi_project_vector_adapter; FTS remains usable. Existing single-project Milvus support stays in place.
• Bypass the new route's cache explicitly (Cache-Control: no-store). Existing project-route caches remain unchanged.

Implementation Details

Most moved code is the existing SQLite/Postgres FTS query compilation, extracted for reuse. Project-bound repositories retain indexing, vector initialization/writes, deletion, reconciliation, and existing search execution. Their public signatures, the existing project endpoint, and MCP/CLI defaults and routing are unchanged.

The new reader embeds once and executes a database query over shared storage. Scope, row kind, and row ID remain part of filtering, vector matching, fusion, hydration, and ordering. Vector reads require the configured model/index, dimensions, ready manifests, and matching source hashes. The reader never calls vector adapter initialization, which can rebuild storage.

Vector/hybrid ranking evaluates eligible rows in SQL before pagination, using the existing max(vector, fts) + 0.3 * min(vector, fts) formula with global FTS normalization. This deliberately replaces per-project candidate ordering with database-scope ranking. Scores and candidate membership do not depend on page size. Only page rows and their top five actual matching chunks are hydrated into Python. FTS has exact counts; semantic modes use an extra-row pagination probe with total_is_exact=false. Reranking is not invoked. Guarded natural-language relaxation is selected over the full scope inside SQL before pagination; malformed PostgreSQL strict queries retry with the established safe relaxed renderer without re-embedding. Invalid explicit lexical syntax preserves an empty FTS channel. SQLite missing-package/extension-capability errors return an actionable 400 while FTS remains usable.

Authorization remains Cloud-owned: Core consumes the effective project list and never discovers an unrestricted scope. Cloud can add authorization-aware multi-project caching separately.

Testing

• uv run pytest test-int/test_multi_project_search.py -q -s --no-cov --maxfail=1: 44 passed on SQLite.
• BASIC_MEMORY_TEST_POSTGRES=1 uv run pytest test-int/test_multi_project_search.py -q -s --no-cov --maxfail=1: 42 passed, 2 SQLite-only cases skipped on real Postgres/pgvector.
• uv run pytest tests/repository tests/services/test_search*.py tests/api/v2/test_search*.py tests/api/v2/test_cjk_search_guidance.py -q --no-cov --maxfail=3: 1269 passed, 38 skipped.
• BASIC_MEMORY_TEST_POSTGRES=1 uv run pytest tests/repository/test_search_repository.py tests/repository/test_postgres_search_repository.py tests/repository/test_postgres_search_quoted_queries.py tests/repository/test_search_text_with_metadata_filters.py tests/repository/test_search_file_path_prefix.py tests/repository/test_search_relaxation.py tests/repository/test_vector_temporal_filter.py tests/services/test_search_service_temporal.py tests/api/v2/test_search_router.py tests/api/v2/test_search_router_temporal.py tests/repository/test_milvus_index.py -q --no-cov --maxfail=3: 262 passed, 15 skipped.
• just fast-check, just doctor, git diff --check: passed.

Integration coverage includes one/multiple/empty project sets, invalid scope, colliding paths and composite IDs, mixed result kinds, metadata/note-type/directory/category filters, temporal filtering and hydration, exact counts, stale vector exclusion, stable pagination, cache bypass across scope changes, unsupported adapters, and one embedding/one retrieval statement with no index mutations. Only the embedding provider is deterministic; database/index operations are real.

A local coverage-instrumented run failed before test collection while importing FastMCP/Pydantic on Python 3.14 (Settings: Input should be an instance of Settings), also with COVERAGE_CORE=ctrace. The normal tests and canonical checks pass. No coverage percentage is claimed from that failed run; no dependency or coverage configuration changes were made. CI test recipes also use --no-cov, so passing checks do not establish 100% coverage.

Small-corpus comparison

Three projects, nine result identities, deterministic embeddings; repeated runs use warm storage, not response caching. Both implementations returned the same result-identity set. These measurements are local fixture evidence, not a production latency guarantee.

Backend / mode	Warm per-project baseline	Warm combined	Query embeddings
SQLite vector	6.71 ms	1.94 ms	3 → 1
SQLite hybrid	8.84 ms	2.04 ms	3 → 1
Postgres vector	229.24 ms	32.01 ms	3 → 1
Postgres hybrid	322.52 ms	39.56 ms	3 → 1

Serialized repository-result payloads on this corpus were about 4.5 KB for the baseline and 4.9–5.0 KB combined, including actual vector chunks. Cold/warm measurements are emitted by the comparison integration test.

Risks / Follow-ups

Global ranking changes relevance relative to merging independently ranked project pages. Exact vector scoring and fusion operate over the eligible SQL set, so database work grows with that set; the small fixture does not establish production-scale latency or memory use. Milvus multi-project vector retrieval, cache generation machinery, reranking, Cloud orchestration, and deployment are outside this PR. No merge or deployment is requested.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-09-15T05:30:09.921757Z	0159119	New commits

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 14d9a31f9d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[phernandez]

@codex review

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.