officialCodeWork · officialCodeWork · May 26, 2026 · May 25, 2026
diff --git a/TRACKER.md b/TRACKER.md
@@ -8,7 +8,7 @@
 > Use `Taskfile.yml` for task targets (cross-platform), `Makefile` for Unix convenience.
 > CI matrix always includes `ubuntu-22.04`, `macos-14`, and `windows-latest`.
 
-**Last updated:** 2026-05-26 (Step 2.2 — vector retrieval backends)
+**Last updated:** 2026-05-26 (tracker sync — PR #76)
 **Current phase:** Phase 2 — Retrieval Engine
 **Next action:** Phase 2 Step 2.3 — BM25 / keyword retrieval: Elasticsearch BM25 plugin satisfying the Step 1.1b `KeywordRetrievalBackend` / `KeywordIndexBackend` SPI, plus an in-process `tantivy` fallback for laptops / CI, plus configurable per-field boosting consumed via the existing `FilterExpr` push-down (no new SPI surface). Step 2.2 widened vector-backend coverage (Weaviate / Pinecone / ES dense-vector) on top of the Step 2.1 read-layer contract.
 
@@ -93,7 +93,7 @@
 | Step | Title | Status | Branch | PR | Key Deliverables |
 |------|-------|--------|--------|----|-----------------|
 | 2.1 | Knowledge store read layer | ✅ | `build/phase-2/step-2.1-knowledge-store-read-layer` | [#74](https://github.com/officialCodeWork/AgentContextOS/pull/74) | `FilterExpr` AST + in-memory `evaluate()` reference semantics relocated from `rag-policy` into `rag_core.filter` so SPI signatures can take it directly without inverting the dependency graph (`rag_policy.filter` keeps re-exporting every symbol for back-compat). `VectorRetrievalBackend.retrieve_ids` switched from `dict[str, Any]` to typed `FilterExpr \| None`; `KeywordRetrievalBackend.retrieve_ids` gains the same parameter. New SPI signature gate `test_retrieve_ids_accepts_filter_expr` enforces the shape on every retrieval ABC. Noop stores (`NoopVectorStore`, `NoopKeywordStore`) apply the predicate via `evaluate()` against a per-chunk attribute view — the canonical oracle every translator must agree with. `PgVectorStore`: new `_filter_sql.translate()` produces parameterised `WHERE` fragments, `acl_labels TEXT[]` column + GIN index added (idempotent `ADD COLUMN IF NOT EXISTS` migration on `initialize()`), `bulk_index` persists ACL labels from `Embedding.acl_labels`. `QdrantVectorStore`: new `_filter_qdrant.translate()` builds `Filter(must / must_not / should)` merged with existing tenant/corpus `must` clauses, payload gains `acl_labels` on every write. Unsupported fields raise `RetrievalError` (no silent push-down loss). 75 new tests (16 `evaluate` units, 16 SQL translator units, 11 Qdrant translator units, 5 vector contract push-down, 3 keyword contract push-down, 1 signature gate, schema regen). New docs: [architecture/retrieval-read-layer.md](docs/architecture/retrieval-read-layer.md); `reference/rag-core.md` + `reference/rag-policy.md` updated. `rag-core` 0.13.0 → 0.14.0. Schemas regenerated (`Eq.json`, `AnyIn.json`, `And.json`, `Or.json`, `Not.json`, `TrueExpr.json`). |
-| 2.2 | Vector retrieval backends | ✅ | `build/phase-2/step-2.2-vector-retrieval-backends` | — | Three new vector backends behind their own optional extras: `WeaviateVectorStore` (`[weaviate]`, weaviate-client v4 async; single `RagEmbeddings` collection w/ `tenant_id` filter + UUID5 `(tenant, chunk)` IDs; `text[]` `acl_labels` property pushed down via `_filter_weaviate.translate()` that lazy-imports the SDK and composes `Filter.by_property(...).equal/contains_any` with `&`/`\|`/`~`), `PineconeVectorStore` (`[pinecone]`, PineconeAsyncio v5+; namespace=tenant primary isolation primitive plus belt-and-braces `tenant_id` filter; metadata: `tenant_id`/`chunk_id`/`corpus_id`/`model`/`acl_labels`; `_filter_pinecone.translate()` emits MongoDB-style `$eq`/`$in`/`$ne`/`$nin`/`$and`/`$or` and rewrites `Not` via De Morgan since Pinecone has no `$not`), `ElasticsearchDenseVectorStore` (`[elasticsearch]`, AsyncElasticsearch 8.x w/ `dense_vector` + native kNN; single index, `tenant_id` keyword filter, composite `_id="<tenant>::<chunk>"`; `_filter_elasticsearch.translate()` emits query-DSL `bool.filter`/`terms`/`match_all`/`match_none`). All five backends now consume `IndexHint` via the new shared `_index_hint.select_index_variant()` helper that centralises the ADR-0009 §1 size/recall → variant table — pgvector picks ivfflat vs HNSW DDL at `initialize()` time (with a new `_index_ddl_for` helper; PQ tier falls back to HNSW w/ larger `m` since pgvector has no PQ), Qdrant configures `HnswConfigDiff` + scalar `ScalarQuantization(INT8)` for the PQ tier (new `_qdrant_index_config` helper), Weaviate picks `Configure.VectorIndex.{flat,hnsw,hnsw+pq}` via a `_vector_index_config` helper, ES picks `index=false` (flat) vs HNSW `index_options.{m,ef_construction}` per tier, Pinecone records the advisory variant in logs (managed indexes). Each SDK is imported lazily inside `__init__` so importing the module without the extra still succeeds (constructing the class raises a clear `ImportError` pointing at the extra). 53 new tests: 11 `_index_hint` parametrised tier tests, 12 `_filter_pinecone` translator units, 11 `_filter_elasticsearch` translator units, 13 `_filter_weaviate` translator units (SDK stubbed at `sys.modules` so they run without the extra) + 6 integration suites per backend gated on `pytest.importorskip` + a service-reachability fixture (Weaviate via `/v1/.well-known/ready`, ES via `_cluster/health`, Pinecone via `TEST_PINECONE_API_KEY`). Policy-engine coverage allowlist extended for the three new files (the call site, not the engine, consults `PolicyEngine`). New extras + workspace wiring in `packages/backends/pyproject.toml` (`weaviate`/`pinecone`/`elasticsearch`) and mypy overrides in root `pyproject.toml` so CI without the extras stays green. Cross-platform: all three SDKs ship pure-Python or wheel distributions for ubuntu/macos/windows. Deliberately deferred: `ragctl ann tune` (Step 2.6 — query understanding owns the planner-side tuning surface) and per-corpus `dtype: int8` knob in `rag.yaml` (Step 3.5 — corpus router lands rag.yaml → backend wiring). New docs: [reference/backends.md](docs/reference/backends.md) (Weaviate / Pinecone / ES sections + IndexHint mapping tables per backend), [architecture/vector-backends.md](docs/architecture/vector-backends.md) (IndexHint → variant table, FilterExpr translator contracts, tenant-isolation patterns, extension points). |
+| 2.2 | Vector retrieval backends | ✅ | `build/phase-2/step-2.2-vector-retrieval-backends` | [#76](https://github.com/officialCodeWork/AgentContextOS/pull/76) | Three new vector backends behind their own optional extras: `WeaviateVectorStore` (`[weaviate]`, weaviate-client v4 async; single `RagEmbeddings` collection w/ `tenant_id` filter + UUID5 `(tenant, chunk)` IDs; `text[]` `acl_labels` property pushed down via `_filter_weaviate.translate()` that lazy-imports the SDK and composes `Filter.by_property(...).equal/contains_any` with `&`/`\|`/`~`), `PineconeVectorStore` (`[pinecone]`, PineconeAsyncio v5+; namespace=tenant primary isolation primitive plus belt-and-braces `tenant_id` filter; metadata: `tenant_id`/`chunk_id`/`corpus_id`/`model`/`acl_labels`; `_filter_pinecone.translate()` emits MongoDB-style `$eq`/`$in`/`$ne`/`$nin`/`$and`/`$or` and rewrites `Not` via De Morgan since Pinecone has no `$not`), `ElasticsearchDenseVectorStore` (`[elasticsearch]`, AsyncElasticsearch 8.x w/ `dense_vector` + native kNN; single index, `tenant_id` keyword filter, composite `_id="<tenant>::<chunk>"`; `_filter_elasticsearch.translate()` emits query-DSL `bool.filter`/`terms`/`match_all`/`match_none`). All five backends now consume `IndexHint` via the new shared `_index_hint.select_index_variant()` helper that centralises the ADR-0009 §1 size/recall → variant table — pgvector picks ivfflat vs HNSW DDL at `initialize()` time (with a new `_index_ddl_for` helper; PQ tier falls back to HNSW w/ larger `m` since pgvector has no PQ), Qdrant configures `HnswConfigDiff` + scalar `ScalarQuantization(INT8)` for the PQ tier (new `_qdrant_index_config` helper), Weaviate picks `Configure.VectorIndex.{flat,hnsw,hnsw+pq}` via a `_vector_index_config` helper, ES picks `index=false` (flat) vs HNSW `index_options.{m,ef_construction}` per tier, Pinecone records the advisory variant in logs (managed indexes). Each SDK is imported lazily inside `__init__` so importing the module without the extra still succeeds (constructing the class raises a clear `ImportError` pointing at the extra). 53 new tests: 11 `_index_hint` parametrised tier tests, 12 `_filter_pinecone` translator units, 11 `_filter_elasticsearch` translator units, 13 `_filter_weaviate` translator units (SDK stubbed at `sys.modules` so they run without the extra) + 6 integration suites per backend gated on `pytest.importorskip` + a service-reachability fixture (Weaviate via `/v1/.well-known/ready`, ES via `_cluster/health`, Pinecone via `TEST_PINECONE_API_KEY`). Policy-engine coverage allowlist extended for the three new files (the call site, not the engine, consults `PolicyEngine`). New extras + workspace wiring in `packages/backends/pyproject.toml` (`weaviate`/`pinecone`/`elasticsearch`) and mypy overrides in root `pyproject.toml` so CI without the extras stays green. Cross-platform: all three SDKs ship pure-Python or wheel distributions for ubuntu/macos/windows. Deliberately deferred: `ragctl ann tune` (Step 2.6 — query understanding owns the planner-side tuning surface) and per-corpus `dtype: int8` knob in `rag.yaml` (Step 3.5 — corpus router lands rag.yaml → backend wiring). New docs: [reference/backends.md](docs/reference/backends.md) (Weaviate / Pinecone / ES sections + IndexHint mapping tables per backend), [architecture/vector-backends.md](docs/architecture/vector-backends.md) (IndexHint → variant table, FilterExpr translator contracts, tenant-isolation patterns, extension points). |
 | 2.3 | BM25 / keyword retrieval | ⏳ | — | — | Elasticsearch BM25 plugin; tantivy in-process fallback; field boosting |
 | 2.4 | Graph retrieval | ⏳ | — | — | Neo4j, Memgraph, in-memory NetworkX plugins; multi-hop neighbor expansion |
 | 2.5 | Hybrid RRF fusion | ⏳ | — | — | Reciprocal Rank Fusion across dense+sparse+graph; configurable weights |
@@ -244,6 +244,8 @@
 | [#72](https://github.com/officialCodeWork/AgentContextOS/pull/72) | feat(ingest,gateway): write path + POST /v1/ingest (Step 1.10 — CLI demo) | `build/phase-1/step-1.10-write-path-ingest-api` | ✅ Merged | 2026-05-25 |
 | [#73](https://github.com/officialCodeWork/AgentContextOS/pull/73) | chore(tracker): mark Phase 1 complete (Step 1.10 → ✅; log PRs #71, #72) | `chore/tracker-sync-pr72` | ✅ Merged | 2026-05-25 |
 | [#74](https://github.com/officialCodeWork/AgentContextOS/pull/74) | feat(core,backends): knowledge store read layer + FilterExpr push-down (Step 2.1) | `build/phase-2/step-2.1-knowledge-store-read-layer` | ✅ Merged | 2026-05-25 |
+| [#75](https://github.com/officialCodeWork/AgentContextOS/pull/75) | chore(tracker): sync Step 2.1 → ✅ and log PRs #73, #74 | `chore/tracker-sync-pr74` | ✅ Merged | 2026-05-26 |
+| [#76](https://github.com/officialCodeWork/AgentContextOS/pull/76) | feat(backends): vector retrieval backends — Weaviate / Pinecone / Elasticsearch + IndexHint wiring (Step 2.2) | `build/phase-2/step-2.2-vector-retrieval-backends` | ✅ Merged | 2026-05-26 |
 
 ---