You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
bench: fresh_clone leaks every upstream fix branch into the cell, so real-issue runs can copy the gold patch #1028
bench/run.sh's fresh_clone does a full clone and then checkout BASE, so every upstream branch — including the fix branch for the very issue under test — sits in the cell's .git. In the 2026-09-14 dev vs #993 vs #1017 grid (four issues on MALIBA-AI/bambara-text-normalization @ 6c978ffa1c, deepseek/deepseek-v4.1-flash), all three binaries found and read them: git show origin/feat/… appears in the transcripts, one dev cell notes its diff "reproduces the gold index line". Issue 21's diff is the upstream patch verbatim in all three runs, issue 20's in two of three. Only issue 22 was uncontaminated. The grid's quality column is therefore not a measurement.
bench/canary/lib/cell.sh already does this right: a fetch-only tree at the base commit, no other refs.
Replication a stranger can run
AFORGE_MODE=chat bench/run.sh <issue 21 of that corpus># then, in the cell:
git -C <cell> branch -r | grep -c feat/ # > 0 today, must be 0
grep -c 'git show origin/'<cell>/transcript.jsonl # > 0 today
Acceptance
End to end first: a cell's git branch -r lists only the base ref and git log --all reaches no commit newer than BASE; a run's transcript contains no git show origin//git log origin/ of a branch that does not exist in the cell. Then fresh_clone shares bench/canary/lib/cell.sh's fetch-only path rather than a second clone recipe.
Evidence: spark:~/af-bench-evidence/REPORT.md and the per-cell transcript.jsonl files; the same table is posted on #993 and #1017.
Activity
erSpell commentedon Sep 27, 2026
I'd like to work on this.