Repository navigation
fix(live): NYSE early closes on the live board - #638
Merged
Merged
Conversation
Co-Authored-By: Claude Opus 5.5 <[email protected]>
… closes Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
On an NYSE early close Yahoo returns the 12:30-13:00 half bar as null and sends the closing level as a point stamped 13:00, which an open-stamped reader drops as after-hours. Restamp it into the half bar so the Live board's ^DJI/^GSPC baselines end the day on the official close. Pinned by two recorded ^DJI responses (2025-11-28 early close, 2025-11-26 full). Co-Authored-By: Claude Opus 5.5 <[email protected]>
…4-2035 The rule was pinned to four published years. A list generated once from exchange_calendars 4.13.2 (no new dependency) widens that to 32: 22 years of sessions that happened and 10 years of an independent implementation. States where the rule stops holding (pre-2004 policy) and corrects the post-close bar wording: on SIP the 13:00 bar opens with the closing auction, it is not just thin after-hours prints. Co-Authored-By: Claude Opus 5.5 <[email protected]>
…025-2028 Every trading day, each span the Alpaca loader stamps (1, 5, 60 min): a bar is in session iff it opens at or after 09:30 and closes by that day's close. The pattern is exchange_calendars' TestTradingIndex; the hourly-shape pin is the pandas_market_calendars #138 failure mode (a short session losing bars). Co-Authored-By: Claude Opus 5.5 <[email protected]>
The rules missed the days NYSE closed at short notice, so is_trading_day called 2025-01-09 (President Carter's day of mourning) a session, and so did everything that walks the calendar: backtest window sizing, the default window, the Live board's expected sessions. Dated data beside the rules, as exchange_calendars keeps its XNYS adhoc_holidays: Reagan, Ford, Sandy (2), G. H. W. Bush, Carter. Pinned by sessions per year against exchange_calendars 4.13.2 XNYS, 2004-2035 (generated once offline; not a dependency). Co-Authored-By: Claude Opus 5.5 <[email protected]>
… mark is Research corrections, no behaviour change: - the bars after a 13:00 bell carry real prints (late sessions and Alpaca's half-day extended hours run to 17:00); the filter drops them by stamp; - every US day is marked pre-auction: the auction print opens the dropped bar. Yahoo's index level is the exception on a half day (official close, ~0.02% off a full day's basis), the lesser gap against a stale 11:30 level; - freeze_settle_time settles against the SIP delay clamp; Alpaca can still revise bars after it; - the dense 04:00-19:00 test grids are a superset of Alpaca's output, not its shape; the node-key test names Yahoo's restamped closing level. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Yahoo appends the latest close to every hourly response, stamped at the close itself, so on a full session a fetch that reaches today held a point at 16:00 ET and the restamp moved it onto the 15:30 half bar: today marked on the official close, the same day replayed later marked pre-auction. The restamp now needs is_early_close as well as the close time. Pinned with the recorded response run through a window that keeps the appended point, and with three more recorded early closes (^GSPC 2025-11-28, ^DJI and ^GSPC 2025-12-24) whose last level is the published close. Co-Authored-By: Claude Opus 5.5 <[email protected]>
…elease - UNSCHEDULED_EARLY_CLOSES (empty: none since 2004) joins the rule's candidates ahead of the trading-day filter, so the next short-notice half day is a one-line addition that can shorten a session but never open a closed one. The docstring now says what nothing catches: a missing one, since bars still arrive after the bell. - PUBLISHED_EARLY_CLOSES gains 2022-2024 from NYSE's 2021-12-27 release (checked against the release text), so the hand-copied pin covers 2022-2028, with sources, retrieval date and a re-pin note. - The rules are described as what they are: NYSE Rule 7.2 lists holidays only, early closes are announced year by year, later years extrapolate. Co-Authored-By: Claude Opus 5.5 <[email protected]>
… third check - Six more recorded index responses (^DJI and ^GSPC on 2024-11-29, 2024-12-24 and 2025-07-03), so all five early closes from 2024-11-29 to 2025-12-24 are pinned on both indices, each ending on the published close. The 2024 ones leave Yahoo's 730-day hourly window in late 2026 and could not be recorded after that. - The fixture comment no longer says the window filter always drops Yahoo's appended close; it does unless the window reaches the latest session. - LEAN's market-hours database agrees with every early close and holiday 2004-2028 (diffed offline); it is not the pin because of its Sunday 2017-12-24 early close. Co-Authored-By: Claude Opus 5.5 <[email protected]>
…erved claims Review follow-ups: - If Yahoo ever drops or shifts its 13:00 point, the restamp does nothing and the board's 13:00 node repeats the 11:30 level: plausible and wrong. One flushed ERROR (yahoo.early_close_level_missing) per settled early close with no level at its last half bar; a day within an hour of its close is skipped. Pinned with the recorded response minus that point, and the ten recorded early closes must stay silent. - Docstrings no longer state as observed what nobody has seen on an early close (Alpaca's late-session bars, the 13:00 auction print's bar); the 16:00 case is the measured one. - The market-local-time test now bites: a naive 12:00 stamp must be kept, which reading it as UTC would not. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
…er it Only the open-stamped filter knew the 13:00 close. Aggregation still built a 13:30 bucket from 12:30-12:59 and is_session_close still matched 16:00 only, so on an early close a minute-source run lost its closing decision. day_session_windows is now the per-day owner read by is_in_session, is_session_close and aggregate_bars. Drops REGULAR_CLOSE_TIME (unused; the CodeQL note) and decides US by market name, not by window shape. Conformance T05 now fails as T02 does (D2/D16) instead of losing the order (D19/D8). Co-Authored-By: Claude Opus 5.5 <[email protected]>
- leaderboard/service.py: as-of and daily-window cutoffs use session_close_time(day, "US") instead of a fixed 16:00. - _yahoo.py: accept both New York zone spellings; a missing meta.exchangeTimezoneName logs one ERROR instead of silently skipping the restamp; ERRORs dedupe per (symbol, day) per process. - Live board copy names the 13:00 early close (leaderboard.js v=38). - Tests: early-close daily window, Yahoo zone/dedupe cases, copy guard; is_session_close asserts carry no side effects. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Collaborator
Author
|
Dated post-deploy checks for 2026-11-27 and 2026-12-24 are tracked in #639. |
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Merge and deploy by 2026-11-20: 2026-11-27 (13:00 ET close) is the first NYSE early close the live board has to trade.
Do not merge between 22:15 and 23:30 UTC on a weekday. The merge auto-deploys prod, and the Render restart kills the 22:30 UTC live refresh thread mid-run, so that night's session is billed again on the next night.
nyse_early_closes/is_early_closeinus_market_calendar.py, by rule (day after Thanksgiving; Dec 24 Mon–Thu; Jul 3 when Jul 4 is Tue–Fri). Pinned to NYSE's 2022–2028 releases and to exchange_calendars XNYS for 2004–2035; the XNYS pin is a literal generated offline, not a dependency. DatedUNSCHEDULED_CLOSURES(2025-01-09 and five more since 2004) and an emptyUNSCHEDULED_EARLY_CLOSES. Sessions per year now match XNYS for 2004–2035. The close times live insessions.py, the only moduletest_no_module_restates_the_session_boundslets spell a session bound.day_session_windows(day, market)is the one per-day session source; on a US early close it ends at 13:00. The open-stamped filter (is_in_session), the session-close fill (is_session_close) andaggregate_barsall read it, so an early close buckets 10:30/11:30/12:30/13:00 and fills at 13:00. Bars stamped 13:00 and later are not decision bars: they hold the closing auction, then late sessions to 17:00 (on main, 3 extra billed model calls filled off those prints). Applies to My Agents and protocol runs too. The close-stamped rule (time_in_session) still ends every US day at 16:00 (D19, engine rewrite). Conformance: T05 now fails on D2/D16 instead of D19/D8 (known failures re-recorded); 25 strict xfails, no XPASS._yahoorestamps that point to 12:30, on early closes only, so ^DJI and ^GSPC end the day on the 13:00 node instead of 12:00. Pinned with recorded responses. AcceptsAmerica/New_YorkandUS/Eastern; a missingmeta.exchangeTimezoneNamelogs oneERROR: yahoo.exchange_zone_missing, and missing-level ERRORs dedupe per (symbol, day).leaderboard/service.py's as-of and daily-window cutoffs readsession_close_time(day)too. The Competition board is unaffected: its window has no early close.force=true(live.pyforce_refresh). That replays the month from the 1st and bills it again.Tests:
pytest dashboard/backend/tests -q -p no:cacheprovider(test_report_pdf.pyneedsreportlablocally).🤖 Generated with Claude Code