Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/STRATEGY_DEVELOPMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,8 @@ reaches `main` by pull request:
`git add` new files before `pixi run lint`: it checks tracked files only. Before waiting on
a `--reuse` round-robin, read the "N reused ... M to play" line it prints: M much larger
than your own matches means the cache is stale (shared code changed since it was filled).
Stop it and ask for a full refresh on `main` rather than paying for one on your branch.
Let it run anyway (a full round-robin is about 10 minutes on 15 workers): the other
strategies' matches run `main`'s code, so they refill the cache for everyone.
5. Open a draft pull request into `main` with the round-robin result, and the bench result if you
ran one. An agent never merges: a person reviews and merges.

Expand Down
54 changes: 27 additions & 27 deletions docs/strategies.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,36 +55,36 @@ only from `--both-sides`, side/kickoff cells, or `--fuzz-restarts`.

## Latest results

**Smoke round-robin, full catalog** (2026-10-01, `e83a7466`, 22 configs, 231 matches, 65s,
`--strict`, 0 stalls, `replays/tournament_20261001_094103/summary.json`). Points are 3 a win,
1 a draw. 65s matches are mostly draws (152 of 231 here), so the middle of the table is close
to noise; two round-robins of near-identical code rank strategies with Spearman +0.97
(`docs/STRATEGY_DEVELOPMENT.md`, A/B on the scenario bank).
**Smoke round-robin, full catalog** (2026-10-03, v2.0.1, 22 configs, 231 matches, 65s,
`--strict`, 0 stalls, about 9 min wall on 15 workers, `replays/tournament_20261003_092151/summary.json`).
Points are 3 a win, 1 a draw. 65s matches are mostly draws (129 of 231 here), so the middle of
the table is close to noise; two round-robins of near-identical code rank strategies with Spearman
+0.97 (`docs/STRATEGY_DEVELOPMENT.md`, A/B on the scenario bank).

| Strategy | W-D-L | GF-GA | Points per match |
|---|---|---|---|
| `split_shape` | 8-13-0 | 11-1 | 1.76 |
| `clear_danger` | 6-15-0 | 7-0 | 1.57 |
| `overload_press` | 7-11-3 | 9-3 | 1.52 |
| `counter_flow` | 5-16-0 | 6-0 | 1.48 |
| `score_aware_counter_flow` | 5-15-1 | 5-1 | 1.43 |
| `press_and_pass` | 5-14-2 | 5-2 | 1.38 |
| `high_press` | 3-18-0 | 3-0 | 1.29 |
| `low_block` | 4-15-2 | 4-2 | 1.29 |
| `give_and_go_solo` | 5-12-4 | 6-7 | 1.29 |
| `clear_press_plus` | 3-17-1 | 3-1 | 1.24 |
| `counter_press` | 4-14-3 | 5-4 | 1.24 |
| `press_trigger_flow` | 4-14-3 | 5-4 | 1.24 |
| `high_line_zone` | 4-13-4 | 4-4 | 1.19 |
| `tiki_taka_plus` | 3-15-3 | 3-3 | 1.14 |
| `tiki_taka` | 2-17-2 | 2-2 | 1.10 |
| `decoy_and_overload` | 3-13-5 | 3-6 | 1.05 |
| `three_slot` | 1-17-3 | 1-3 | 0.95 |
| `switch_of_play` | 3-9-9 | 3-9 | 0.86 |
| `overload_flow` | 1-13-7 | 2-9 | 0.76 |
| `zone_fluid` | 1-12-8 | 2-10 | 0.71 |
| `shadow_switch` | 2-8-11 | 4-13 | 0.67 |
| `score_aware_zone_flow` | 0-13-8 | 0-9 | 0.62 |
| `counter_flow` | 9-12-0 | 9-0 | 1.86 |
| `clear_danger` | 10-9-2 | 10-2 | 1.86 |
| `split_shape` | 10-8-3 | 12-3 | 1.81 |
| `give_and_go_solo` | 8-11-2 | 9-3 | 1.67 |
| `score_aware_counter_flow` | 6-14-1 | 6-1 | 1.52 |
| `decoy_and_overload` | 7-9-5 | 8-5 | 1.43 |
| `low_block` | 5-14-2 | 5-2 | 1.38 |
| `overload_press` | 4-16-1 | 5-1 | 1.33 |
| `press_and_pass` | 4-16-1 | 4-1 | 1.33 |
| `press_trigger_flow` | 6-10-5 | 6-5 | 1.33 |
| `clear_press_plus` | 4-15-2 | 4-2 | 1.29 |
| `high_press` | 4-14-3 | 4-3 | 1.24 |
| `tiki_taka_plus` | 3-16-2 | 3-2 | 1.19 |
| `counter_press` | 4-11-6 | 4-6 | 1.10 |
| `high_line_zone` | 4-11-6 | 4-7 | 1.10 |
| `tiki_taka` | 2-16-3 | 2-3 | 1.05 |
| `overload_flow` | 4-8-9 | 5-11 | 0.95 |
| `three_slot` | 2-12-7 | 2-7 | 0.86 |
| `score_aware_zone_flow` | 3-7-11 | 4-12 | 0.76 |
| `switch_of_play` | 2-10-9 | 2-10 | 0.76 |
| `shadow_switch` | 0-13-8 | 2-11 | 0.62 |
| `zone_fluid` | 1-6-14 | 1-14 | 0.43 |

The full-match tables below predate the planner, referee and tactic fixes since 2026-09-02.
Treat them as rough ordering, not current truth.
Expand Down
8 changes: 7 additions & 1 deletion utama_core/replay/fingerprint.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@
import hashlib
import json
import os
import re
import sys
from dataclasses import dataclass
from functools import cached_property
Expand All @@ -60,6 +61,10 @@
# Files next to a module that are not data it reads.
_NOT_DATA_SUFFIXES = (".py", ".pyc", ".md")

# The release workflow rewrites `utama_core/__init__.py`'s `__version__` on every merge into main.
# Nothing reads it in a match, and hashing it would invalidate every stored result on each merge.
_VERSION_LINE = re.compile(rb"""^__version__\s*=\s*["'][^"'\n]*["']\s*$""", re.MULTILINE)

# Method names that mutate their receiver; called at import time, they reach outside the module.
_MUTATORS = frozenset(
{
Expand Down Expand Up @@ -195,7 +200,8 @@ def file_digests(self, names: Iterable[str]) -> list[str]:
for name in names:
mod = self.module(name)
if mod is not None and mod.path is not None:
rows.append(f"{mod.path.relative_to(self.root).as_posix()} {_sha(mod.source)}")
code = _VERSION_LINE.sub(b"", mod.source)
rows.append(f"{mod.path.relative_to(self.root).as_posix()} {_sha(code)}")
Comment on lines +203 to +204
return sorted(rows)

def data_files(self, names: Iterable[str]) -> list[str]:
Expand Down
12 changes: 11 additions & 1 deletion utama_core/tests/replay/test_fingerprint.py
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ def build_c_kernel_strategy(ids):
FILES = {
"tools/tournament/__init__.py": "",
"tools/tournament/tournament_lib.py": "from utama_core.run.runner import run\nfrom utama_core.strategy import kernel_strategy\n",
"utama_core/__init__.py": "",
"utama_core/__init__.py": '__version__ = "1.0.0"\n',
"utama_core/run/__init__.py": "",
"utama_core/run/runner.py": "from ..planning import planner\nfrom utama_core.profiles import loader\n\n\ndef run():\n"
" return planner.plan()\n",
Expand Down Expand Up @@ -253,6 +253,16 @@ def test_fingerprints_do_not_depend_on_where_the_checkout_is(repo, tmp_path_fact
assert _fingerprints(other) == _fingerprints(repo)


def test_a_release_version_bump_changes_nothing(repo):
# The release workflow rewrites `__version__` on every merge into main; hashing it emptied
# the whole match cache after each merge.
rel = "utama_core/__init__.py"
assert _changed_after(repo, rel, '__version__ = "1.0.0"', '__version__ = "2.0.1"') == set()
assert _changed_after(repo, rel, '__version__ = "2.0.1"', '__version__ = "2.0.1"\nDEBUG = True') == {"base"} | set(
CONFIGS
)


def test_a_relative_import_resolves(repo):
g = CodeGraph(repo)
assert "utama_core.planning.planner" in g.closure(["utama_core.run.runner"])
Expand Down
Loading