Repository navigation
feat(backtest): per-bar decision tape - #627
Merged
Merged
Conversation
Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01JpcbUoP1Va89XvhScxMctY
Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01JpcbUoP1Va89XvhScxMctY
Intent + pre-decision state is the engine-independent replay unit; recorded actions are a same-engine reproduction check only. Parent labels trace outcomes per arm; child writes nothing on SystemExit. Market realism is scored by the conformance suite, not retrofitted into the engine being replaced; adjusted US bars with provenance is the one item split out as a follow-up PR. Co-Authored-By: Claude Fable 5.1 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
gate_rewrote no longer raises on a non-numeric model position_size (treated as matching any size). Plan: Task 3 recorder gains record_bar_from so Task 4's payload builders run inside the swallow layer. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Suspend the decision tape recorder after MAX_CONSECUTIVE_STORE_FAILURES (3) failed bars in a row. Swallowing exceptions only bounds a store that fails fast; one that hangs before raising (Neon dial, TCP timeout) would cost that hang on every bar and could push the run into the subprocess timeout. Later bars are counted in the new summary key suspended_skipped without touching the store; a successful write resets the streak. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
- Parent trace label: arms only record it; written once, last in finally, after finalize_run and _finalize_slot, on a daemon thread joined with TRACE_LABEL_DEADLINE_SECONDS. A hung or raising trace store can no longer strand a slot or a reservation. - Intent is an exact copy of the gate's input: present-null fields kept, non-finite numbers and object/array values carry their JSON text under raw[field], non-object entries kept whole; orders_for_replay restores them. bound_payload marks an order unreplayable rather than silently changing it. - gate_rewrote is False on llm_fallback bars; gate_rewrites counts LLM-driven bars only. - Perturbed-engine test maps the divergent curve point to its decision bar via the fill plan, and adds a script whose first fill is on bar 2. Co-Authored-By: Claude Opus 5.5 <[email protected]>
- Parent: label the trace in an inner finally so a raising cleanup step cannot leave it running; a negative returncode (signal the parent did not send) labels run_killed. - Child: trace_lifecycle treats an exception raised while a SystemExit unwinds as the exit and writes nothing. - Tape: rejection reason/strategy_reason truncated and shed before a bar is dropped; load_decision_tape marks each bar complete or not. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
- Trace stays open through both baselines and update_run_baselines, so a baseline failure no longer leaves a `completed` trace. - Tape bars are batched (TAPE_FLUSH_BARS) into one store transaction via a new append_events on both trace-store twins; flushed at loop end and before the trace closes on failure. - Trace store moves to AGENT_RUNS_DATABASE_URL (run data, off the users/content DB). - fail_trace_if_running raises on store errors; the worker and finish_trace_best_effort log ERROR lines instead of failing silently. - Collapsed rejections carry only the collapse key plus this bar's size, not the first rejection's timestamp, size or reasoning. - gate_rewrote ignores sells widened to the sellable position, buys left at size 0, and symbol padding. - Pre-decision snapshots run inside the recorder's swallow layer and are skipped when the recorder is a no-op or suspended; the rejection snapshot covers only the events a bar could collapse into. - Oversized raw model values mark the order unreplayable instead of dropping the bar. - _run_with_deadline is shared by the spend lookup and the trace label. - CodeQL: comments on the empty excepts; test raises go through a helper. Co-Authored-By: Claude Opus 5.5 <[email protected]>
CodeQL py/side-effect-in-assert on the capture tests. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Records every decision bar of a My Agents pipeline backtest into the run's #622 agent trace as a
decision_recorded/execution_resultpair: raw model orders (intent), pre-decision portfolio (state), post-gateactions, fill plan, fills, rejections, driver (llm/llm_fallback/rule_based) and whether the gate rewrote the model. Read back withtraces.service.load_decision_tape(run_id).intent+stateis the replay unit;actionsis the same-engine reproduction check. Evidence (test_decision_tape_engine.py):test_recorded_actions_reproduce_the_run_on_the_same_enginetest_intent_replay_through_the_gate_reproduces_an_llm_run(+..._keeps_null_and_non_finite_fields— intent keeps raw nulls/NaN/containers;orders_for_replayrestores them)test_perturbed_engine_diverges_at_the_first_filled_barBars are written in batches of 25 (
append_events, one transaction, both twins) and flushed at loop end and before a failed trace closes; a SIGTERM'd run loses at most its unflushed buffer. Per-rundecision_tape_summary()reportsgate_rewrites, write failures, oversize skips, dropped bars; the recorder suspends after 3 consecutive store failures. No tape/trace exception reaches the bar loop (test_broken_trace_writes_do_not_change_the_run).Trace close: the child closes it from
trace_lifecycle(constructor → both baselines →update_run_baselines) and writes nothing onSystemExit. The parent labels a still-runningtrace once, after slot/reservation cleanup, on a 10s-bounded thread:run_cancelledrun_timed_outrun_failedrun_killedtrace_close_failedrun_killedA completed trace is never relabelled.
Realism is out of scope. The recorded engine is frictionless, latency-free and on unadjusted bars; the conformance suite (#623) scores realism. Follow-ups are listed in the spec (adjusted bars with provenance first; plus a newly found intraday marking rule to pin before comparing engines).
Trace store moves to
AGENT_RUNS_DATABASE_URL(wasCONTENT_DATABASE_URL, #622): per-bar tape rows are run data and stay off the users/content DB. Traces written since #622 remain in the content DB, unread. Boot line:trace_store backend: postgres (<host>/<db>).Not touched:
portfolio_manager.py,trading/execution.py, step sessions, AI Hedge Fund runtime, leaderboard. Full backend suite: 6575 passed; the only failures are the 7 local-envtest_report_pdfones, same set asmain.🤖 Generated with Claude Code