Skip to content

Link comparisons to their reports and fix doc paths - #10

Merged
Shuvam1024 merged 1 commit into
mainfrom
shuvam1024/report-links
Oct 7, 2026
Merged

Shuvam1024 merged 1 commit into
mainfrom
shuvam1024/report-links

Conversation

@Shuvam1024

@Shuvam1024 Shuvam1024 commented Oct 7, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • benchmarks/scifact/hybrid-vs-rerank.json is regenerated from the current hybrid and rerank reports. Its candidate run ID now matches rerank-selected.json. Every other comparison under benchmarks/ already matched its reports. Deltas, confidence intervals, and p-values are unchanged. The current compare path also stores the statistics recipe on this newly written file.
  • Signed figures that round to zero keep their sign, so the SciFact chunked-minus-ablation nDCG@10 mean renders as -0.000 instead of +0.000.
  • Links in docs/results.md use ../ so they resolve from docs/. A test checks every relative link in README.md and docs/*.md.
  • The getting-started Docker pointer now goes to the architecture note, because the README no longer has a Docker section.
  • The regression note says reports committed before the rename keep "versions": {"ragbench": ...} alongside the legacy schema IDs.

Test plan

  • ruff check and ruff format --check
  • mypy
  • pytest with coverage (89.09% with the dense extra installed locally)
  • Comparison run IDs match the reports they name
  • Relative documentation links resolve

The SciFact hybrid-versus-rerank file now names the current rerank run.
Deltas, intervals, and p-values are unchanged. A rounded zero keeps its
sign, and documentation links resolve from docs/.
@Shuvam1024
Shuvam1024 marked this pull request as ready for review October 7, 2026 21:12
@Shuvam1024
Shuvam1024 merged commit 31a4af1 into main Oct 7, 2026
5 checks passed
@Shuvam1024
Shuvam1024 deleted the shuvam1024/report-links branch October 7, 2026 21:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant