Repository navigation
Fix scoring percentile description; remove speculative event types - #1
Merged
Merged
Conversation
Two participant-facing docs fixes: - Notebook 01's closing cell said the competition converts the realized reaction into a percentile "against the asset's historical distribution". That is wrong and actively confused a participant: per the production scorer (ported verbatim in src/examples/scoring.py, which was already correct), car1 is percentile-ranked in [0, 1] across the PERIOD's (calendar quarter's) cross-section of all scored event outcomes — a cross-sectional rank over the quarter's events, not a rank within the asset's own history. Rewrote the cell to match and point at examples.scoring.percentile_ranks. - Removed FOMC_MEETING / MACRO_DATA_CPI / MACRO_DATA_JOBS (and the speculative UNSCHEDULED_8K sample row) everywhere: the live competition currently emits only EARNINGS_RELEASE, so advertising nonexistent types misleads participants. event_type stays an open string (no enum) so new types never break parsing; the open-string tolerance is still tested via a clearly-fake SOME_FUTURE_TYPE in tests/test_frames.py, which also keeps the empty-focal-assets rendering check. Sample calendar rows replaced with plausible large-cap earnings on the same date range so the quickstart's table and daily bar chart still render across multiple days. Notebook cell ids were renumbered by the repo's nbstripout pre-commit hook.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two participant-facing docs fixes (a participant was actively confused by the first):
Correct the scoring description in
notebooks/01_historical_archive.ipynb. The closing "Where to go from here" cell claimed the reaction is percentiled "against the asset's historical distribution". Per the production scorer —src/examples/scoring.pyis a verbatim port and was already correct; it is the source of truth —car1is converted to a percentile rank in[0, 1]across the period's (calendar quarter's) cross-section of all scored event outcomes, not within the asset's own history. The cell now says exactly that and points atexamples.scoring.percentile_ranks.Remove
FOMC_MEETING/MACRO_DATA_CPI/MACRO_DATA_JOBSmentions. The live competition currently emits onlyEARNINGS_RELEASE; advertising nonexistent types misleads participants.event_typeremains an open string (no enum) so new types never break parsing:src/examples/schemas.pydocstring reworded (open string; onlyEARNINGS_RELEASEemitted today).notebooks/00_api_quickstart.ipynbcalendar intro reworded likewise.data/sample/events_sample.json: macro/FOMC rows replaced with plausible large-cap earnings rows (BA, IBM, CVX) on the same Jan 27 – Feb 4 2026 range, so the quickstart table and stacked daily bar chart still render across multiple days. Judgment call: the speculativeUNSCHEDULED_8Ksample row was replaced too (QCOM) under the same "only earnings are emitted" rationale — shout if you want anUNSCHEDULEDtiming example kept.tests/test_frames.py: the open-string tolerance guarantee is still exercised via a clearly-fakeSOME_FUTURE_TYPE(which also keeps the empty-focal_assetsrendering assertion and the manifest sort-order check), so no test property was weakened.Housekeeping: the repo's
nbstripoutpre-commit hook renumbered notebook cell ids to its sequential form, which inflates the notebook diffs slightly.Verification:
uv run pytest— 28 passed;pre-commit(ruff, ruff-format, nbstripout) clean; both notebooks executed end-to-end withjupyter nbconvert --executein sample mode (noEM_API_KEY) — outputs not committed, matching the nbstripout convention.