diff --git a/DESIGN.md b/DESIGN.md index 2f91e95..ddd9726 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -69,7 +69,8 @@ The candidate loop implements monotonic greedy local search: ever reverted: this is candidate rejection, not rollback of an applied state. The historical label `rollback_regression` appears only in v1 archives. The rule is a fixed-tolerance comparison, not a confidence-interval or trust-region barrier. -5. **Held-out Isolation**: the held-out split is measured once, after selection stops, +5. **Held-out Isolation**: the held-out split is scored only after selection stops (once + for the baseline policy and once for the final policy, both in `rsi_framework/core.py`), and never enters generation, scoring, or the rejection rule. ## Scoped Data-Isolation Check diff --git a/README.md b/README.md index 4c99637..cd2e661 100644 --- a/README.md +++ b/README.md @@ -13,7 +13,7 @@ RSI (recursive self-improvement) is the research question behind the name. This - Generates, evaluates, and selects keyword-rule changes on synthetic text. - Checks each proposed change against vocabulary, polarity, and bias rules. - Rejects changes that reduce accuracy on development data. -- Measures held-out data once after selection ends. +- Scores held-out data only after selection ends (once for the baseline policy and once for the final policy). - Writes reproducible run records and checked result summaries. ## Quick start