Repository navigation
Add the earnings-call fact extraction behind each event's disclosure - #4
Merged
Merged
Conversation
Every realized EARNINGS_RELEASE event carries a disclosure item with kind=facts / source=earnings_call whose content is ten sentences distilled from the call. Participants build on those facts without any account of how they are produced -- data/README called the sample "a stand-in for exposition" and schemas.py did not model the disclosure at all. examples/summary.py is an exact semantic port of the production summarizer, the same way scoring.py ports the scorer: the system prompt, user template, per-model thinking configuration (opus-4-5 manual budget; 4-6/4-7/4-8 adaptive at effort=high), the request builders, format_transcript, and both the strict and lenient fact parsers. A request built here is the request the pipeline sends. Two things the port makes visible. The prompt is the entire structure mechanism -- no tools, no JSON schema, no response_format, no temperature -- so the ten-fact shape is requested in prose and validated after the fact. And the transcript renderer prepends only the call's own header title: no ticker, date, fiscal period, estimates or market data, and no truncation or chunking. The summarizer sees the call and nothing else. Extraction can fail. recover_facts applies only non-corrupting repairs -- it never fabricates, splits, or pads to reach ten -- so an artifact can carry facts: null with a parse_note explaining why. Notebook 02 walks that vocabulary explicitly; agents should handle absent facts. Also adds lenient Disclosure/DisclosureItem models, an optional ANTHROPIC_API_KEY path (not a competition credential; only the notebook's last section uses it, and it self-skips without one), and synthetic sample transcript/artifact files -- real transcripts are licensed and are not distributed here. Verified against production by differential: every constant, the prompts, build_request for all four models, format_transcript in both modes, and 420 parser inputs (20 fixed plus 400 seeded random mutations covering fences, truncation, over/under-count, empty facts and mangled tails) match exactly. Note this cannot regenerate a competition disclosure -- different transcript, and thinking makes the call nondeterministic. SUMMARY_MODEL tracks the pipeline's model constant and will need updating when production switches.
pre-commit pinned ruff v0.9.6 while CI runs `uv run ruff` (0.15.20 via uv.lock), so the two disagreed about what passes. The stale hook failed on archive.py with UP038, a rule newer ruff has since removed: the PEP 604 form it suggests for isinstance is slightly slower, isn't recommended practice, and wrongly implies other typing syntaxes work there. Applying it would have made the code worse to satisfy retracted advice, so archive.py is untouched and the rev moves instead. Also switches the hook id from `ruff` to `ruff-check`; `ruff` is now a legacy alias and the run warns about it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ports the production earnings-call summarizer into
examples, the same wayscoring.pyports the scorer — so participants can see where the ten facts on eachdisclosurecome from.What's here
src/examples/summary.py— exact semantic port: system prompt, user template, per-model thinking config (opus-4-5 manual budget; 4-6/4-7/4-8 adaptive ateffort=high), request builders,format_transcript, and both the strict and lenient fact parsers.tests/test_summary.py— 34 tests, including the production parser suite one-for-one.notebooks/02_earnings_call_facts.ipynb— seven sections, runs top-to-bottom with no credentials.data/sample/transcript_sample.json,summary_sample.json— synthetic; real transcripts are licensed and are not distributed here.Disclosure/DisclosureItemmodels (the disclosure was previously unmodeled), and an optionalANTHROPIC_API_KEYpath.What the port makes visible
The prompt is the entire structure mechanism — no tools, no JSON schema, no
response_format, notemperature. The ten-fact shape is requested in prose and validated after the fact.The transcript renderer prepends only the call's own header title: no ticker, date, fiscal period, estimates, or market data, and no truncation or chunking. The summarizer sees the call and nothing else.
Extraction can fail.
recover_factsapplies only non-corrupting repairs — never fabricates, splits, or pads to reach ten — so an artifact can carryfacts: nullwith aparse_note. Agents should handle absent facts.Verification
Differential against
chalicelib.summarize_calls: every constant, both prompts,build_requestfor all four models,format_transcriptin both modes, and 420 parser inputs (20 fixed + 400 seeded random mutations covering fences, truncation, over/under-count, empty facts, mangled tails) match exactly.CI-equivalent locally: ruff, ruff-format, mypy, 73 tests, all 3 notebooks under
nbmakewith no key.Notes
SUMMARY_MODELtracks the pipeline's model constant and will need updating when production switches models.anthropicas a runtime dependency for one optional notebook section; worth deciding whether that should be an optional extra instead.Second commit is unrelated: the pre-commit ruff rev was pinned to v0.9.6 while CI lints with 0.15.20, and the stale hook failed on
archive.pywith UP038 — a rule newer ruff removed because the fix it suggests is slower and not recommended.archive.pyis untouched; the rev moved instead.