Skip to content

[belongs in hypatia] OutcomeTracker should record the dispatcher's predicate verdict, not success/failure #647

Open

Description

@hyperpolymath

Filed here because this session could not add hyperpolymath/hypatia. Please transfer it there; the change is in Hypatia.

What

Since #616 and #622, scripts/dispatch-runner.sh computes a predicate verdict for each fix: verified, no_op, incomplete, unverified or failed. It writes that verdict, with before_hits/after_hits, to the outcome JSONL.

When it calls mix hypatia.record_outcome, though, it still collapses the verdict to success/failure/false_positive (record_outcome() → hypatia_outcome). Hypatia's OutcomeTracker.record_outcome/2 therefore still learns from exit codes. The standards ultraplan's Wave 0 item "True outcomes" says it should accept verified only from a predicate or rescan.

Proposed change

  • Hypatia: accept a verdict argument in mix hypatia.record_outcome / OutcomeTracker.record_outcome/2. Treat only verified as positive evidence, no_op as neutral, and unverified as no evidence. Mark the historical log unverified rather than deleting it.
  • gitbot-fleet: pass the verdict through (one argument in hypatia_record_outcome), keeping the collapsed field for older Hypatia versions.

Evidence

  • E1 (docs/experiments/E1-fix-outcome-replay-2026-10-20.adoc): 0 of 130 replayed "success" rows were confirmed by their recipe's own predicate.
  • Current wiring: scripts/dispatch-runner.sh, functions record_outcome and hypatia_record_outcome.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions