Skip to content

docs: clarify evaluation method and measured limitations - #7

Merged
noetion merged 1 commit into
masterfrom
docs/evaluation-overview
Sep 29, 2026
Merged

noetion merged 1 commit into
masterfrom
docs/evaluation-overview

Conversation

@noetion

@noetion noetion commented Sep 29, 2026

Copy link
Copy Markdown
Owner

Purpose

Make the evaluation question, sequence rule, reproducible result, and error analysis easier to follow.

Changes

  • Introduce the experiment and show the measured sequence.
  • Summarise the holdout counts using the existing result fields.
  • Add clone and directory steps to reproduction instructions.
  • Give the provenance finding its own section and link to the error audit.
  • Keep the generated README and its renderer synchronized.

Verification

  • uv run tripwire verify reproduced the committed result.
  • All 37 tests passed.
  • Ruff and mypy passed; source distribution and wheel built successfully.
  • git diff --check passed.

Review scope and limitations

Review the README together with its renderer. Detection logic, corpus, splits, and committed metrics are unchanged. The canonical holdout remains spent; the provenance finding is not presented as a tested successor detector. Local pytest reported a cache-write warning; tests passed.

@noetion

noetion commented Sep 29, 2026

Copy link
Copy Markdown
Owner Author

Reviewed 9720a5a: no blocking findings in the README and renderer diff. The generated summary matches the committed result (TP 6, FP 1, TN 36, FN 11); uv run tripwire verify reproduces it. All 37 tests, Ruff, mypy, and package build passed. GitHub verification passed for push and pull-request runs. Checked rendered sections and links. Corpus, rule logic, splits, and metrics are unchanged; limitations and the spent-holdout qualification remain explicit.

@noetion
noetion merged commit 47c2599 into master Sep 29, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant