Repository navigation
research(conversion): test which pairing window to train on - #51
Merged
Merged
Conversation
… training window Fits the equivalence model and flat multiplier on 14/30/60/90-day training pairs with partitions, repeat-swim noise, shrinkage and the ratio screen held as shipped, and scores every fit on one fixed 14-day test set (30-day as a secondary view) with swimmer-clustered bootstrap deltas vs 30 days, the independent-swim decile bias, and a seasonal SCY/LCM order check. Aggregate-only receipt in window-experiment-01.json. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Render the window experiment's receipt as a REPORT table under 7.8: one model per training window, all scored on the same 14-day test pairs. Thirty days stays. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The report's existing window table scores the 30-day model on wider test pairs, so it can't say which window to train on. This PR adds that experiment.
window_experiment.pyfits one Equivalence model per training window (14, 30, 60 and 90 days). Each model uses the same partitions, noise estimates and shrinkage. All four are scored on the same 14-day test pairs, plus the 30-day test pairs as a secondary view. The script reuses the shipped functions rather than copying them, and stops if the snapshot or input hash differs from the shipped receipt. Its 30-day model reproduces the shipped MAE exactly: 2.5095 on 14-day pairs and 2.7711 on 30-day pairs.Result: keep 30 days.
The receipt
window-experiment-01.jsonholds aggregates only. REPORT §7.8 gains a generated table (render_report_tables.py --checkpasses) and a short paragraph that also states the limits: the intervals aren't adjusted for multiple comparisons, and the seasonal swim-order hypothesis isn't established. The shipped model, snapshot and evaluation receipt are unchanged.🤖 Generated with Claude Code