Add reproducible Context Fabric text benchmark corpus - #2804
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Independent Terra review round 1 at The reviewer independently recomputed all nine source hashes and reference IDs (zero mismatches, 934 bytes), checked byte-preserving attributes, per-kind exact-set scoring, incomplete-prediction rejection, and unavailable metrics when no predictions are provided. No processor or media/runtime measurements are claimed. One nonblocking documentation/validation parity finding is retained on #2319: the README describes validation against the JSON Schema, while the command uses manual validation and the schema test validates the schema itself. Numeric optional Local 21-test, corpus, link and governance evidence remains as recorded. Hosted CI is a separate merge gate. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6c9bec284b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Refs #2319
Context Fabric processors need shared, reproducible fixtures before their quality can be compared. Add the first CF-24A corpus slice: nine synthetic text/transcript fixtures, exact source hashes, reference outputs, licence/method/date metadata, and explicit hostile-injection controls.
The local command validates the corpus and its 16 KiB source-byte budget, then computes precision/recall/F1 by candidate kind when supplied predictions cover every fixture. Without predictions, metrics are unavailable. The example predictions are deliberately imperfect; no processor is executed or measured. Audio, image, PDF and runtime benchmark acceptance remains on #2319.
Validation at
6c9bec284b1feaade2cd49ce3bf9219c0b1d4320:py -3 -B -m unittest discover -s scripts/context_fabric -p "test_*.py": 21 passed.node scripts/check-doc-links.mjs: 664 Markdown files, zero broken links.node scripts/check-docs-governance.mjsandgit diff --check: passed.The testing guide documents the command and its limits. Review-first execution is unchanged. Hosted CI and independent Terra review are separate pending gates. No live provider, external media, or real user data was used.