Problem
PostgresSearchRepository._ensure_vector_tables() issues CREATE INDEX IF NOT EXISTS idx_search_vector_chunks_project_entity on first use of each repository instance, even when the shared chunk table and index already exist. The per-instance initialization lock does not protect against ordinary writes from other repositories or processes.
On PostgreSQL17, this existing-index DDL waits behind an uncommitted INSERT. With a200ms lock timeout it raises LockNotAvailable; the identical statement succeeds immediately after the writer rolls back. This is a reproduced runtime initialization defect, not a claim that a particular historical production transaction held the lock.
Minimal reproduction
Create search_vector_chunks(project_id int, entity_id int) and its index on (project_id, entity_id). ConnectionA begins a transaction and inserts one row without committing. ConnectionB sets lock_timeout='200ms' and executes:
CREATE INDEX IF NOT EXISTS idx_search_vector_chunks_project_entity
ON search_vector_chunks(project_id,entity_id);
It fails with LockNotAvailable although no index creation is needed. Roll back A and repeat B: it succeeds. Verified in a temporary PostgreSQL17 container with cleanup.
Scope and acceptance
- Avoid unnecessary runtime DDL when the required vector schema/index already exists, using explicit catalog/schema checks or the appropriate migration-owned boundary.
- Preserve supported fresh-database initialization, concurrent initialization, database schema routing, and pgvector initialization.
- Do not mark initialization complete after a failed setup, broaden transactions, add global locks, or merely increase timeout/retry budgets.
- Add a regression through the actual repository initialization path with an existing index and an ordinary concurrent writer. Prove it fails against the old code, and cover missing-index/fresh-schema behavior.
- After Core delivery, update the Cloud dependency pin and verify hosted embedding behavior separately.
Production investigation context
Cloud has observed embedding job timeouts in _ensure_vector_tables, including current-release failures at the index DDL and older fingerprint2054. Some jobs recovered; others exhausted retries. The affected tenant's index exists now. Historical blocking transaction identity is unavailable, so production causal attribution remains incomplete. Exact failed payloads are not always retained; coverage reconciliation must precede recovery. Cloud PR2013 fixes a separate missing-migration request and does not solve this redundant runtime DDL.
The Core checkout is occupied by other integration work; no Core files were changed during this investigation.
Problem
PostgresSearchRepository._ensure_vector_tables()issuesCREATE INDEX IF NOT EXISTS idx_search_vector_chunks_project_entityon first use of each repository instance, even when the shared chunk table and index already exist. The per-instance initialization lock does not protect against ordinary writes from other repositories or processes.On PostgreSQL17, this existing-index DDL waits behind an uncommitted INSERT. With a200ms lock timeout it raises LockNotAvailable; the identical statement succeeds immediately after the writer rolls back. This is a reproduced runtime initialization defect, not a claim that a particular historical production transaction held the lock.
Minimal reproduction
Create
search_vector_chunks(project_id int, entity_id int)and its index on(project_id, entity_id). ConnectionA begins a transaction and inserts one row without committing. ConnectionB setslock_timeout='200ms'and executes:It fails with LockNotAvailable although no index creation is needed. Roll back A and repeat B: it succeeds. Verified in a temporary PostgreSQL17 container with cleanup.
Scope and acceptance
Production investigation context
Cloud has observed embedding job timeouts in
_ensure_vector_tables, including current-release failures at the index DDL and older fingerprint2054. Some jobs recovered; others exhausted retries. The affected tenant's index exists now. Historical blocking transaction identity is unavailable, so production causal attribution remains incomplete. Exact failed payloads are not always retained; coverage reconciliation must precede recovery. Cloud PR2013 fixes a separate missing-migration request and does not solve this redundant runtime DDL.The Core checkout is occupied by other integration work; no Core files were changed during this investigation.