week_6b: Module C (The Librarian) — C.4 envelope emitter + C.0→C.4 pipeline glue - #991
week_6b: Module C (The Librarian) — C.4 envelope emitter + C.0→C.4 pipeline glue#991PRAteek-singHWY wants to merge 17 commits into
Conversation
… once report_retrieval_recall and report_calibration each built the live pipeline (DB + embedding model + cross-encoder) independently, loading it twice and reranking every positive row twice per --use_live_embeddings run. Build it once in main and pass (retriever, reranker) into both reports, matching _build_live_pipeline's stated intent. Behavior-preserving: recall@20 285/292, rerank top-1 220/292, ECE 0.046 PASS unchanged.
…ecision gate C.3 (Week 5) produces one honest, calibrated confidence per chunk; C.4 turns it into the action — auto-link into the OpenCRE graph, or route to a human — which is the accuracy gate of the whole pipeline. - decision_engine.py: pure `decide(confidence, candidates, *, threshold, adversarial, update_ambiguous) -> DecisionResult`. Links the top-1 iff confidence >= threshold AND candidates exist AND no blocking flag; otherwise reviews with a reason_code. Reason precedence NO_CANDIDATES > ADVERSARIAL_FLAG > UPDATE_AMBIGUOUS > BELOW_THRESHOLD. Frozen result, versioned ENGINE_NAME, custom DecisionError — mirrors the C.1/C.2/C.3 model-free seams. Does not import the C.3 scaler (confidence-in -> decision-out), so it is hermetically testable. - decision_engine_test.py: 14 hermetic tests — table-driven over every confidence/flag combination, the inclusive >= boundary, all four reason codes, precedence order, and the input guards. - evaluate_librarian.py: additive report_decision_accuracy — fits T on positive+hard_negative, runs the live C.1->C.4 decision over the golden set, and reports overall agreement plus auto-link recall vs review recall (a single accuracy hides that at tau=0.80 the softmax top-1 mass of a correct-but-close winner is often ~0.5, so correct positives route to review — the safe direction; W7 tunes tau). Informational, not a gate: the SafetyGuard flags are not wired yet, so flag-based reason codes lag until that lands. Emitters (LinkProposal/ReviewItem writers) and the C.0->C.4 pipeline glue follow in a stacked week_6b PR.
|
Warning Review limit reached
Next review available in: 59 minutes You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (6)
Summary by CodeRabbit
WalkthroughAdds temperature-scaling calibration, deterministic decision and envelope generation, an injected librarian pipeline, live calibration and decision evaluation reports, and hermetic tests covering the new behavior. ChangesLibrarian calibration and decision pipeline
Estimated code review effort: 4 (Complex) | ~45 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (3)
application/utils/librarian/pipeline.py (2)
58-73: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winConstructor params
source/retriever/reranker/scalerare untyped.Unlike
threshold: float/pipeline_run_id: strin the same signature, and unlike the fully-typeddecision_engine.py/emitter.py, these duck-typed seams carry no type hints at all. ConsiderProtocolclasses (e.g.RetrieverLike,RerankerLike,ScalerLike) or at minimumAnyannotations formake mypyconsistency.As per coding guidelines, "Run
make mypyfor Python type checking."🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@application/utils/librarian/pipeline.py` around lines 58 - 73, Update the constructor in the pipeline class around __init__ to annotate source, retriever, reranker, and scaler, preferably using appropriate Protocol types such as RetrieverLike, RerankerLike, and ScalerLike; use Any only where no suitable interface exists. Preserve the existing threshold and pipeline_run_id annotations and run make mypy to verify the changes.Source: Coding guidelines
75-104: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick winPer-row failures in
retrieve/rerank/confidence/decide/emitabort the whole run.Only
section_from_queue_rowis guarded with try/except; a single failure from any other stage propagates and aborts the entire batch, discarding all envelopes/stats accumulated so far. This is currently safe with hermetic stubs, but the docstring states these seams are meant to become live DB/embedding/cross-encoder calls — worth hardening before that wiring lands.♻️ Suggested per-row error containment
- audit = self._retriever.retrieve(section.text) - audit = self._reranker.rerank(section.text, audit) - reranked = [c for c in audit.reranked if c.score_rerank is not None] - logits = [float(c.score_rerank) for c in reranked] - cre_ids = [c.cre_id for c in reranked] - confidence = self._scaler.confidence(logits) if logits else 0.0 - - result = decide(confidence, cre_ids, threshold=self._threshold) - envelope = emit(section, audit, result, pipeline_run_id=self._run_id, at=at) - envelopes.append(envelope) - if isinstance(envelope, LinkProposal): - linked += 1 - else: - review += 1 + try: + audit = self._retriever.retrieve(section.text) + audit = self._reranker.rerank(section.text, audit) + reranked = [c for c in audit.reranked if c.score_rerank is not None] + logits = [float(c.score_rerank) for c in reranked] + cre_ids = [c.cre_id for c in reranked] + confidence = self._scaler.confidence(logits) if logits else 0.0 + result = decide(confidence, cre_ids, threshold=self._threshold) + envelope = emit(section, audit, result, pipeline_run_id=self._run_id, at=at) + except Exception: + errored += 1 # new RunStats field + continue + envelopes.append(envelope) + if isinstance(envelope, LinkProposal): + linked += 1 + else: + review += 1🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@application/utils/librarian/pipeline.py` around lines 75 - 104, Update the per-item processing in run so failures from retrieve, rerank, confidence, decide, or emit are contained to that row instead of aborting the batch. Wrap the full processing pipeline after section_from_queue_row in a per-row try/except, increment the appropriate skipped/error statistic for failed rows, and continue processing later items while preserving already-created envelopes and existing successful-row counts.scripts/evaluate_librarian.py (1)
288-307: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick win
report_decision_accuracyre-derives the calibration set and re-fitsT, duplicatingreport_calibration's work.Both functions build the identical
positive+hard_negative(shortlist, label) set and callfit_temperatureon it (lines 234-253 inreport_calibration, lines 293-307 here). Sincemain()calls both sequentially over the sameretriever/reranker, this doubles the live per-rowreranker.rerank()(cross-encoder inference) cost with no behavioral difference — the fittedTwill be identical.Consider having
report_calibrationreturn(status, scaler)and passing the scaler intoreport_decision_accuracy, or extracting the calibration-set-building + fit into one shared helper called once frommain().🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@scripts/evaluate_librarian.py` around lines 288 - 307, The calibration data and temperature scaler are redundantly recomputed in report_decision_accuracy after report_calibration. Refactor report_calibration and main so calibration fitting occurs once, returns its status and fitted scaler, and passes that scaler into report_decision_accuracy; remove the duplicate cal_rows construction, reranker.rerank calls, label validation, and fit_temperature invocation while preserving existing skip/status behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@application/utils/librarian/emitter.py`:
- Around line 52-62: Update _proposed_links so entries used as
ReviewItem.suggested_links are not labeled with AUTO_LINK_TYPE; use the
established distinct suggested-review link type if available, or omit link_type
when constructing ProposedLink. Preserve the existing confidence, rationale, and
CRE ID values.
---
Nitpick comments:
In `@application/utils/librarian/pipeline.py`:
- Around line 58-73: Update the constructor in the pipeline class around
__init__ to annotate source, retriever, reranker, and scaler, preferably using
appropriate Protocol types such as RetrieverLike, RerankerLike, and ScalerLike;
use Any only where no suitable interface exists. Preserve the existing threshold
and pipeline_run_id annotations and run make mypy to verify the changes.
- Around line 75-104: Update the per-item processing in run so failures from
retrieve, rerank, confidence, decide, or emit are contained to that row instead
of aborting the batch. Wrap the full processing pipeline after
section_from_queue_row in a per-row try/except, increment the appropriate
skipped/error statistic for failed rows, and continue processing later items
while preserving already-created envelopes and existing successful-row counts.
In `@scripts/evaluate_librarian.py`:
- Around line 288-307: The calibration data and temperature scaler are
redundantly recomputed in report_decision_accuracy after report_calibration.
Refactor report_calibration and main so calibration fitting occurs once, returns
its status and fitted scaler, and passes that scaler into
report_decision_accuracy; remove the duplicate cal_rows construction,
reranker.rerank calls, label validation, and fit_temperature invocation while
preserving existing skip/status behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yml
Review profile: CHILL
Plan: Pro Plus
Run ID: 7118b7e4-fc18-4e63-bd3f-3ab1008a3300
📒 Files selected for processing (11)
application/tests/librarian/decision_engine_test.pyapplication/tests/librarian/emitter_test.pyapplication/tests/librarian/pipeline_test.pyapplication/tests/librarian/temperature_test.pyapplication/utils/librarian/__init__.pyapplication/utils/librarian/calibration/__init__.pyapplication/utils/librarian/calibration/temperature.pyapplication/utils/librarian/decision_engine.pyapplication/utils/librarian/emitter.pyapplication/utils/librarian/pipeline.pyscripts/evaluate_librarian.py
…string The calibration/__init__.py docstring still described the rejected single-logit `p = sigmoid(z/T)` approach. temperature.py actually calibrates the softmax over the whole shortlist (`p = softmax(logits / T)`, confidence = top-1 mass) — the sigmoid-on-one-logit approach cannot be calibrated by temperature alone. Docstring now matches the implementation.
…l / ReviewItem) The decision engine (week_6) yields a verdict; this turns it into the wire contract Module D consumes. - emitter.py: `emit(section, audit, result, *, pipeline_run_id, at)` dispatches on the verdict — `linked` -> RFC LinkProposal, `review` -> RFC ReviewItem — plus the two explicit builders. Pure and timestamp-injected (no clock read) so it is hermetically testable; only builds the envelope (persistence is W8). Auto-links carry link_type "Automatically linked to" (mirrors cre_defs.LinkTypes) and the calibrated confidence; reviews carry the reason_code and a deterministic review_id derived from the chunk id. update_detection defaults to the declared degraded value (is_update=False) until the SafetyGuard lands. - emitter_test.py: 9 hermetic tests — both envelopes end-to-end, audit passthrough, degraded update_detection, no-candidates review has no suggestions, the verdict/reason guards, and link_type == cre_defs single source of truth. Stacked on week_6. Pipeline glue (C.0->C.4) follows next.
Wires the librarian end to end: section_from_queue_row (C.0) -> retriever (C.1) -> reranker (C.2) -> scaler.confidence (C.3) -> decide + emit (C.4), one envelope per valid knowledge_queue row. - pipeline.py: LibrarianPipeline.run(at=...) -> RunResult(envelopes, RunStats). Every stage is an injected seam (source/retriever/reranker/scaler), so the whole pipeline runs hermetically with stubs. Inherently dry-run — builds envelopes, never persists (queue write-back + graph writes are W8). pipeline_run_id and the timestamp are injected, never read from the clock, so a run is reproducible. Rows rejected at the C.0 boundary (e.g. UNCERTAIN) are skipped and counted, not linked. - pipeline_test.py: 5 hermetic tests — confident row auto-links, low confidence reviews (below_threshold), empty shortlist reviews (no_candidates), UNCERTAIN row skipped at the boundary, and mixed-batch counts. Stacked on the week_6b emitter.
…ions as auto-links
_proposed_links stamped link_type "Automatically linked to" on every ProposedLink,
including a ReviewItem's suggested_links — but those are candidates for a human to
consider, not auto-links. Parametrize the link type: LinkProposal.links use
AUTO_LINK_TYPE, ReviewItem.suggested_links use SUGGESTED_LINK_TYPE ("Related",
mirrors cre_defs.LinkTypes.Related). Test asserts a review suggestion is never
labelled as an auto-link.
6f6c262 to
8d9fda5
Compare
northdpole
left a comment
northdpole
left a comment
There was a problem hiding this comment.
Maintainer review — Module C Week 6b (#991)
Emitter + dry-run pipeline glue look good: type guards, Related vs Automatically linked to, injected at / pipeline_run_id, and hermetic tests are in good shape. No blocking bugs in the unique Week-6b surface.
Stacked on #990 / #974 — rebase as those land. Inline notes are non-blocking for this dry-run PR but matter before W8 persistence.
| cre_ids = [c.cre_id for c in reranked] | ||
| confidence = self._scaler.confidence(logits) if logits else 0.0 | ||
|
|
||
| result = decide(confidence, cre_ids, threshold=self._threshold) |
There was a problem hiding this comment.
Note (pre-W8) — SafetyGuard flags not wired
decide(...) is called without adversarial= / update_ambiguous=, so those reason codes can never fire from this pipeline yet. Fine for dry-run Week 6b, but must be wired before any graph / queue write-back (W8), or auto-links will ignore the safety path the decision engine already supports.
There was a problem hiding this comment.
Agreed, and noted as a W8 blocker rather than a change here. decide(...) gets adversarial= / update_ambiguous= wired in before any graph or queue write-back, so the safety reason codes can actually fire. Leaving it unwired in this dry-run PR since there is no SafetyGuard to feed them yet.
| for item in self._source.items(): | ||
| total += 1 | ||
| try: | ||
| section = section_from_queue_row(item) |
There was a problem hiding this comment.
Note (B↔C integration) — queue row shape still the C mirror
section_from_queue_row / C's KnowledgeQueueItem still expect the flat source_repo / source_path / source_commit_sha mirror. Module B's live knowledge_queue (#989) is a richer row (locator_*, content_hash, provenance columns, …). Already called out in section_validator for W8 — please keep this on the W8 checklist so the dry-run pipeline here does not silently assume the wrong row shape when wired to Postgres.
There was a problem hiding this comment.
Kept on the W8 checklist. Reconciling section_from_queue_row / KnowledgeQueueItem against Module B's live knowledge_queue (#989) is a real schema reconciliation, not a rename: B's row carries locator_*, content_hash, and the provenance columns, where C currently assumes the flat source_repo / source_path / source_commit_sha mirror. The dry-run pipeline here stays on the mirror shape deliberately, and the adapter seam is where the mapping lands once B freezes the table.
…hortlists, dedupe softmax Three nitpicks from the bot review, none of which changed a metric: - evaluate_librarian: the live retrieve+rerank was recomputed per report. The models were already built once in main, but report_retrieval_recall and report_calibration each re-ran the pipeline over the positive slice, so every positive row paid for two cross-encoder passes. live_audits() now retrieves and reranks each row once, keyed by golden row id, and both reports read the same shortlists. Rows without an audit no longer count toward a report's denominator, so the printed fractions cannot divide by unscored rows. - temperature: _softmax_top duplicated the empty-shortlist guard and the softmax already in TemperatureScaler.probabilities. Both now route through one _softmax_at helper, and confidence() derives from probabilities() so the two can never disagree. - temperature is now clean under the --strict mypy the coding guidelines ask for: annotated _paired's return and the bounds tuple, re-asserted the array type over untyped scipy, and hoisted the label conversion out of the fit objective (it was re-validated on every optimiser iteration anyway). Behaviour is unchanged: 113 librarian tests pass and the hermetic harness run still exits 0. (cherry picked from commit 2bbc76e) (cherry picked from commit ae6bb69)
…on gate must fail
report_calibration returned 0 ("skipped") when the live calibration set was
degenerate (single-class labels, empty after dropping empty shortlists, or a
--slice with one class). Under --use_live_embeddings that let a run exit 0 without
the ECE < 0.10 gate ever running, so CI could greenwash a live run in which
calibration was never checked. A skipped gate now returns 1 (fail), with a message
stating the row/class counts, so exit 0 means the gate actually ran and passed.
(cherry picked from commit 62abf2d)
(cherry picked from commit cb5577a)
…he live reports report_decision_accuracy rebuilt the positive + hard_negative calibration set from its own retrieve+rerank pass and fit its own temperature, duplicating what report_calibration had already done a few lines earlier. On a live run that meant a third pipeline pass over the calibration slices and two independent fits of the same T, with nothing guaranteeing the two agreed. Now there is one of each: - calibration_set() extracts the (shortlist, label) derivation, so the C.3 gate and the C.4 report read the same pairs off the same shared audits. - report_calibration returns (status, scaler); report_decision_accuracy takes the fitted scaler instead of fitting its own, so C.4 thresholds on exactly the T the ECE gate measured. - A degenerate set yields (1, None) and C.4 is skipped with a message: no fitted T means no honest confidence to threshold, and the run has already failed. - The live audit set now covers expected-decision rows too. C.4 grades those and they are not confined to the positive/hard_negative slices, so keying them off the calibration slices alone would have dropped them. Adds evaluate_harness_test.py. The live reports only run under --use_live_embeddings, so nothing exercised their wiring — which is exactly the code that has to share one pipeline pass and one T across three reports. A counting stub asserts the pipeline is called once per row and not once per report, that only the two calibration slices enter the fit, and that a degenerate set returns status 1 with no scaler rather than reporting success. Verified the last one fails if the gate is flipped back to 0. 134 librarian tests pass; the hermetic harness run still exits 0. (cherry picked from commit 6ef7865)
…in per-row failures Two nitpicks on pipeline.py, both aimed at the wiring rather than the dry-run behaviour: - The injected seams were the only untyped parameters in the signature, next to an annotated threshold/pipeline_run_id and fully-typed decision_engine/emitter. Each is structurally one method, so they are now Protocols — KnowledgeSource, Retriever, Reranker, Scaler — which keeps stubs as trivial as before while the call sites are checked. pipeline.py is clean under the --strict mypy the coding guidelines ask for; KnowledgeSource.items() is typed to what section_from_queue_row actually accepts, not a looser Mapping. - Only the C.0 boundary was guarded, so a failure from retrieve / rerank / confidence / decide / emit propagated and abandoned the whole batch along with every envelope already built. Those stages are stubs today but become live DB, embedding, and cross-encoder calls in W8, where one timeout must not cost the run. Failures are now contained per row: the row is logged with its chunk and artifact id, counted in a new RunStats.errored, and the run continues. errored is separate from skipped on purpose — a boundary rejection is expected input hygiene, an error is a fault worth chasing. It defaults to 0, so existing RunStats callers are unaffected. Adds five containment tests: each seam failing in isolation, one bad row in a batch of three leaving the other two linked, and errored not being conflated with skipped. Verified all five fail if the containment is removed. 153 librarian tests pass; the hermetic harness run still exits 0.
…he live reports report_decision_accuracy rebuilt the positive + hard_negative calibration set from its own retrieve+rerank pass and fit its own temperature, duplicating what report_calibration had already done a few lines earlier. On a live run that meant a third pipeline pass over the calibration slices and two independent fits of the same T, with nothing guaranteeing the two agreed. Now there is one of each: - calibration_set() extracts the (shortlist, label) derivation, so the C.3 gate and the C.4 report read the same pairs off the same shared audits. - report_calibration returns (status, scaler); report_decision_accuracy takes the fitted scaler instead of fitting its own, so C.4 thresholds on exactly the T the ECE gate measured. - A degenerate set yields (1, None) and C.4 is skipped with a message: no fitted T means no honest confidence to threshold, and the run has already failed. - The live audit set now covers expected-decision rows too. C.4 grades those and they are not confined to the positive/hard_negative slices, so keying them off the calibration slices alone would have dropped them. Adds evaluate_harness_test.py. The live reports only run under --use_live_embeddings, so nothing exercised their wiring — which is exactly the code that has to share one pipeline pass and one T across three reports. A counting stub asserts the pipeline is called once per row and not once per report, that only the two calibration slices enter the fit, and that a degenerate set returns status 1 with no scaler rather than reporting success. Verified the last one fails if the gate is flipped back to 0. 134 librarian tests pass; the hermetic harness run still exits 0.
|
Pushed. Both Seams are typed. They were the only untyped parameters in the signature, next to an annotated Per-row failures are contained. Only the C.0 boundary was guarded, so a failure from Five containment tests added: each seam failing in isolation, one bad row in a batch of three leaving the other two linked, and The two W8 notes on this PR stay open by design, as agreed in the threads above: the SafetyGuard flag wiring, and reconciling the queue row shape against Module B's live 153 librarian tests pass; the hermetic harness run still exits 0. |
Follow-up to the shared-audit refactor. live_audits keys the reranked shortlists by row.id and the reports read them back with audits[row.id], so two rows sharing an id would collapse in that dict: the surviving audit gets reused for the earlier row and its scores are reported against the wrong text. Before the refactor each report recomputed per row, so a duplicate id was harmless; keying by it turned a harmless quirk into a silently wrong number. GoldenDatasetRow only requires an id to be non-empty, so uniqueness is enforced at load time and the harness refuses the file instead of printing a wrong metric. The committed dataset has 319 distinct ids, so nothing changes today; this closes the trap the refactor opened. Adds a test that a forced collision raises and names the offending id. (cherry picked from commit f352dfc) (cherry picked from commit 8bb865a)
Two follow-ups from the bot review of the last push, neither behavioural: - The live-path descriptions still predated C.4. The module docstring claimed the semantic path was stubbed, _build_live_pipeline named only recall and calibration as its consumers, --use_live_embeddings help listed only recall and top-1, and the offline message omitted the decision report. All four now say what the run actually does, including that C.3 is the one live report that sets the exit status (a failed or skipped gate returns nonzero) while C.4 is informational until SafetyGuard and tau tuning land. - evaluate_harness_test bound the calibration status it never asserted (Ruff RUF059). Bound to _status: the test is about the pipeline not being re-run and the scaler coming back, and the gate outcome on stub logits is not a meaningful assertion. ruff check is clean on both files. 136 librarian tests pass; the hermetic harness run still exits 0. (cherry picked from commit 90992f5)
2 participants
Hi @northdpole - Week 6b of Module C, stacked on the Week-6 decision engine. Week 6 produced the verdict (auto-link vs. review); this PR turns that verdict into the wire envelopes Module D consumes, and wires the whole C.0→C.4 pipeline end to end (dry-run).
Overview
Week 6 gave us
decide()→ aDecisionResult. Two things were deliberately left out of that PR to keep it a clean, provable unit: emitting the RFC envelope, and wiring the stages together. This PR adds both.This PR's role:
The emitter (
emitter.py) -emit(section, audit, result, *, pipeline_run_id, at)dispatches on the verdict:linked→ an RFCLinkProposal,review→ an RFCReviewItem. It only builds the envelope (persisting it is W8). Pure and timestamp-injected (no clock read), so every branch is hermetically testable. Auto-links carrylink_type"Automatically linked to" (mirrorscre_defs.LinkTypes) and the calibrated confidence; reviews carry thereason_codeand a deterministicreview_idderived from the chunk id.update_detectiondefaults to the declared-degraded value (is_update=False) until the SafetyGuard lands.The pipeline (
pipeline.py) -LibrarianPipeline.run(at=...)runs C.0→C.4 over a knowledge source:section_from_queue_row(C.0) →retrieve(C.1) →rerank(C.2) →scaler.confidence(C.3) →decide+emit(C.4), one envelope per valid row. Every stage is an injected seam, so the whole pipeline runs hermetically with stubs. Inherently dry-run - builds envelopes, never persists.pipeline_run_idand the timestamp are injected, never read from the clock, so a run is reproducible.Scope: 2 new modules + 2 new tests. No frontend, no migration, no behaviour change to OpenCRE proper.
What changed
emitter.py(new)emit()+build_link_proposal/build_review_item.DecisionResult→ RFCLinkProposal/ReviewItem, snapshotting the chunk (KnowledgeSnapshot) and passing the C.1/C.2RetrievalAuditthrough untouched. Pure, timestamp-injected, customEmitterError; auto-linklink_typemirrorscre_defs.LinkTypes.AutomaticallyLinkedTo; degradedupdate_detectiondefault; deterministicreview_id.pipeline.py(new)LibrarianPipeline.run(at=...) -> RunResult(envelopes, RunStats). Wires all five stages via injected seams (source/retriever/reranker/scaler), inherently dry-run. Rows rejected at the C.0 boundary (e.g.UNCERTAIN) are skipped and counted, not linked.emitter_test.py(new),pipeline_test.py(new)update_detection, no-candidates review has no suggestions, the verdict/reason guards, andlink_type == cre_defssingle source of truth. Pipeline (5): confident row auto-links, low confidence reviews (below_threshold), empty shortlist reviews (no_candidates),UNCERTAINrow skipped at the boundary, and mixed-batch counts.How the pieces connect
flowchart TB row["knowledge_queue row"] subgraph PIPE["LibrarianPipeline.run (this PR)"] c0["C.0 section_from_queue_row"] c1["C.1 retriever.retrieve"] c2["C.2 reranker.rerank"] c3["C.3 scaler.confidence"] c4["C.4 decide()"] emit["emit()"] c0 --> c1 --> c2 --> c3 --> c4 --> emit end row --> c0 emit --> lp["LinkProposal (linked)"] emit --> ri["ReviewItem (review + reason_code)"] skip["UNCERTAIN / invalid row -> skipped, counted"] c0 -.-> skipResults
The emitter and pipeline are pure and dry-run - every branch is covered by the hermetic tests above, no live key required. The end-to-end demo on the golden set (populated envelopes for a full slice) is the midterm deliverable; this PR lands the machinery it runs on.
What is intentionally not here
ood/conformal/update_detector) that would populate theadversarial/update_ambiguousflags and realupdate_detection.cre_maindispatch and persistence / queue write-back / graph writes (W8) - the pipeline stays dry-run; nothing is written to OpenCRE.KnowledgeQueueItemmirror over the golden fixture, not a live connection to Module B.How to verify locally