Skip to content

feat(event): score first-story detections with FAR and miss rates - #65

Draft
seonghobae wants to merge 1 commit into
mainfrom
agent/event-first-story-detection
Draft

feat(event): score first-story detections with FAR and miss rates#65
seonghobae wants to merge 1 commit into
mainfrom
agent/event-first-story-detection

Conversation

@seonghobae

Copy link
Copy Markdown
Contributor

Summary

#37 merged on protected main (babd591). #45 kept ready (0007) but remains review-blocked (no independent APPROVE; still CONFLICTING after #37). Next no-0008 gap: first-story false-alarm/miss calibration in existing event_core (does not recreate TDT/CHRONOS crates, does not wait on #48/#49/#53, and does not allocate 0008).

  • refuse_first_story_as_instance never promotes a detection to an instance
  • decide_first_story uses an inclusive probability threshold
  • first_story_false_alarm_rate and first_story_miss_rate fail closed on empty, mismatched, or single-class streams
  • Computed RMSE of known first-story targets is lower under calibrated scores than under an always-first detector

Test plan

  • RED: first_story_contract failed with E0432/E0599
  • cargo test -p event_core --offline
  • cargo clippy -p event_core --all-targets --offline -- -D warnings
  • python3 scripts/validate_documentation.py and check_docstrings.py
  • exact-head required Checks (do not treat queued evidence as passed)

Do not self-approve or merge. Prefer merge #45 when exact-head required checks and a qualifying independent APPROVE exist. Do not allocate 0008.

Keep TDT first-story labels distinct from promoted event instances and
require computed false-alarm, miss, and RMSE evidence against known truth.
@coderabbitai

coderabbitai Bot commented Aug 13, 2026

Copy link
Copy Markdown

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 850293a9-905a-4ddc-aadb-e11eb2031425

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant