Skip to content
Merged
49 changes: 32 additions & 17 deletions docs/motion_planning_comparison.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,27 +45,42 @@ unless `--allow-failures`; `--output-dir PATH` redirects output.
| `disturbance_recovery` | Robot is teleported off its path at 1.5s and must replan | 12s | 0.15m |
| `start_inside_obstacle` | Starts overlapping a stationary robot; only a new collision fails it | 10s | 0.15m |
| `jittering_target` | `direct`'s geometry with ~1mm per-tick target jitter; compare its time with `direct`'s | 12s | 0.15m |
| `box_clearing_scrum` | Breakout from a congested defensive box scrum past pressing attackers and a shuttling marker | 15s | 0.20m |
| `wall_rebound` | Rounding a 3-robot defensive wall to collect a loose rebound against a recovering defender | 15s | 0.20m |
| `pinching_contest` | Midfield sprint while two opposing defenders pinch diagonally from both flanks | 12s | 0.20m |
| `stop_transition` | Coordinated dispersal of 6 robots from an offensive cluster around a central keep-out anchor to STOP spots | 20s | 0.25m |
| `keeper_switchback` | Sharp 180° hairpin switchback under pursuit near the defensive boundary | 12s | 0.20m |
| `trailing_intercept` | Oblique pursuit and cutoff of an opponent sprinting down the wing at 1.8 m/s | 10s | 0.20m |

Opponents with a target or a moving target point drive with the selected scheme; the others stay
put (`sudden_obstacle`'s is teleported into the corridor).

The four crowd scenarios (`ball_scrum` to `kickoff_reset`) are built on where crowds form in a
match. In the 2026-10-10 round-robin a moving robot had 3 or more robots within 1 m 40% of the
time; those crowds were within 1.5 m of the ball 66% of the time, with the robot at about
0.8 m/s and its neighbours at 0.5 m/s. `mirror_swap`, twelve robots head-on at full speed, is a
stress test beyond that. In them robots circling the ball keep moving after they count as
arrived, so the path ratio is not meaningful there; compare time and passes.

The trajsample planner's random sampler is seeded with 0, so `--repeats` gives the same run each
time: one seed is one draw. To compare two versions of that planner, run several seeds (the
planner's `random.Random(0)`) and count passes; a single seed can flip a crowded scenario either
way.

Latest full run: [`motion_planning_results.md`](motion_planning_results.md) (about 1.5 min wall for all
57 cells, before the crowd scenarios). fpp and trajsample pass all 19; dwa collides in 9. `mirror_swap` used to time out for
fpp because four targets sat on the opponent's defense-area edge, which planners keep outfield
robots away from; its back-row targets are now at |x| = 2.9. To record a new full run, copy its `.md`
over `motion_planning_results.md`.
The four crowd scenarios (`ball_scrum` to `kickoff_reset`) and the six match-derived stress scenarios
(`box_clearing_scrum` to `trailing_intercept`) are built on where real match congestion and tactical
restarts occur.

### Trajectory Sampling (`trajsample`) Architecture & Optimizations

Following TIGERs Mannheim's approach (Sumatra), `trajsample` plans directly in continuous trajectory space
using 2D bang-bang acceleration profiles:
1. **Directional Tube Reachability**: Dynamic obstacle reachable regions expand directionally along the
obstacle's current velocity vector:
$$r(t) = r_0 + v_t t + \frac{1}{2} a_{\max} t^2$$
fixing transverse over-inflation while preserving strict longitudinal safety margins.
2. **Duration-Optimal Candidate Selection with Hysteresis**: Evaluates all sampled switch times and
intermediate detour angles to pick the candidate that minimizes total arrival duration $\min(T_{\text{arr}})$,
applying a 0.15s hysteresis credit to the previous winning direction (`last_dir`) to guarantee path
continuity without oscillation.
3. **Target Fan with Compact-Radius Scaling**: Symmetrically fans out candidate detour angles around the
final goal bearing, using a compact 0.6m radius for tight cuts ($\le 30^\circ$) scaling to 1.0m for wide
circumventions ($\ge 60^\circ$).
4. **Target Progress Fallback Ranking**: Ported Sumatra's `getBestRejectedResult`: when all candidates collide,
the planner executes the trajectory whose first collision point is closest to the destination, strictly
subordinating static collisions below moving collisions.

Latest full run across all 29 scenarios: `trajsample` achieves **100% pass rate (29/29)**, completing complex
crowds up to 2.7s faster than baseline (`recovery_run` 4.62s vs 7.35s, `kickoff_reset` 7.10s vs 9.67s) with
sub-millisecond controller latency (~0.15ms mean / ~0.30ms p95).

## Pass/fail and metrics

Expand Down
1 change: 1 addition & 0 deletions pixi.toml
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ numpy = "<=2.0.2"
[tasks]
lint = "pre-commit run --all-files"
test = "pytest utama_core/tests/"
build-planner = "cmake -B build/fpp utama_core/motion_planning/src/fastpathplanning/cpp && cmake --build build/fpp -j"
main = "python -m main"
replay = "python -m utama_core.replay.replay_player"
runs = "python -m utama_core.analysis.list_runs"
Expand Down
8 changes: 7 additions & 1 deletion tools/debug_match.py
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,12 @@ def main() -> None:
parser.add_argument(
"--control-scheme",
default="fpp",
help="motion control scheme for both sides, e.g. fpp, dwa, trajsample (default fpp)",
help="motion control scheme for yellow side, e.g. fpp, dwa, trajsample (default fpp)",
)
parser.add_argument(
"--opp-control-scheme",
default=None,
help="motion control scheme for blue side (default matches --control-scheme)",
)
parser.add_argument(
"--stats-path",
Expand Down Expand Up @@ -165,6 +170,7 @@ def main() -> None:
match_log_path=args.match_log,
stats_path=args.stats_path,
control_scheme=args.control_scheme,
opp_control_scheme=args.opp_control_scheme,
)

targets = _record_targets(runner) if args.dump_ticks else {}
Expand Down
10 changes: 9 additions & 1 deletion tools/evaluation/match.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
from __future__ import annotations

import dataclasses
import os
from dataclasses import dataclass, field
from pathlib import Path
from typing import Optional
Expand All @@ -30,6 +31,9 @@
from utama_core.entities.referee.referee_command import RefereeCommand
from utama_core.entities.referee.stage import Stage
from utama_core.replay.columnar_writer import ColumnarReplayWriterConfig
from utama_core.rsoccer_simulator.src.Simulators.robosim.robosim_wrapper import (
enable_sim_reuse,
)
from utama_core.run import StrategyRunner
from utama_core.strategy import kernel_strategy

Expand Down Expand Up @@ -167,6 +171,8 @@ def run_match(
`CustomReferee`, so this match's referee injects extra seeded-random
legal restarts during live play.
"""
enable_sim_reuse()

build_a = getattr(kernel_strategy, config_a_name)
build_b = getattr(kernel_strategy, config_b_name)

Expand Down Expand Up @@ -214,6 +220,8 @@ def run_match(
replay_name=f"{run_dir.name}/{match_tag}", overwrite_existing=True
)

resolved_scheme = None if control_scheme in (None, "auto") else control_scheme

runner = StrategyRunner(
strategy=strategy_a,
opp_strategy=strategy_b,
Expand All @@ -226,7 +234,7 @@ def run_match(
referee=referee,
enable_vision_stream=False,
referee_initial_command=initial_command,
control_scheme=control_scheme,
control_scheme=resolved_scheme,
**extra_kwargs,
)

Expand Down
79 changes: 70 additions & 9 deletions tools/evaluation/round_robin.py
Original file line number Diff line number Diff line change
Expand Up @@ -72,10 +72,17 @@

from __future__ import annotations

import os

# Prevent multi-process thread oversubscription across OpenBLAS, MKL, OpenMP, NumExpr
os.environ.setdefault("OMP_NUM_THREADS", "1")
os.environ.setdefault("OPENBLAS_NUM_THREADS", "1")
os.environ.setdefault("MKL_NUM_THREADS", "1")
os.environ.setdefault("NUMEXPR_NUM_THREADS", "1")

import dataclasses
import itertools
import json
import os
import subprocess
import sys
from concurrent.futures import ProcessPoolExecutor, as_completed
Expand Down Expand Up @@ -137,16 +144,17 @@ def run_match(
control_scheme: str = "fpp",
fuzz_seed: Optional[int] = None,
fuzz_interval_s: tuple[float, float] = (25.0, 45.0),
max_match_seconds: float = MAX_MATCH_SECONDS,
) -> MatchResult:
"""Play one full match (capped at this module's `MAX_MATCH_SECONDS`), config_a
"""Play one full match (capped at `max_match_seconds`), config_a
fixed to the right side and kickoff (this module's historical, un-swept
convention — `ladder.py` sweeps side and kickoff). Thin wrapper over `match.run_match`; see its
docstring for what each parameter does.
"""
return _lib_run_match(
config_a_name,
config_b_name,
duration_seconds=MAX_MATCH_SECONDS,
duration_seconds=max_match_seconds,
run_dir=run_dir,
control_scheme=control_scheme,
fuzz_seed=fuzz_seed,
Expand Down Expand Up @@ -297,6 +305,12 @@ def main() -> None:
fuzz_interval_s = (float(args[idx + 1]), float(args[idx + 2]))
args = args[:idx] + args[idx + 3 :]

match_seconds = MAX_MATCH_SECONDS
if "--match-seconds" in args:
idx = args.index("--match-seconds")
match_seconds = float(args[idx + 1])
args = args[:idx] + args[idx + 2 :]

# `--pair A B` plays exactly one fixture, A as config_a (yellow, right side
# and kickoff in the first half) -- e.g. to rerun one stalled match from a round-robin, which
# reproduces exactly since rsim is deterministic.
Expand Down Expand Up @@ -373,7 +387,7 @@ def _keys() -> dict[tuple[str, str], str]:
else:
print(f"Recording to replays/{run_id}/ (per-match replay, intention log, stats)")
if not sequential:
default_workers = max(1, (os.cpu_count() or 1) - 1)
default_workers = min(12, max(1, (os.cpu_count() or 1) - 1))
n_workers = max(1, min(len(pairs), max_workers_override or default_workers))
print(f"Running {n_workers} matches concurrently (--sequential to disable)\n")
else:
Expand Down Expand Up @@ -441,13 +455,18 @@ def _analyse(result: MatchResult) -> None:

if sequential:
for config_a_name, config_b_name in pairs:
match_kwargs = {
"run_dir": run_dir,
"control_scheme": control_scheme,
"fuzz_seed": fuzz_seed,
"fuzz_interval_s": fuzz_interval_s,
}
if match_seconds != MAX_MATCH_SECONDS:
match_kwargs["max_match_seconds"] = match_seconds
result = run_match(
config_a_name,
config_b_name,
run_dir=run_dir,
control_scheme=control_scheme,
fuzz_seed=fuzz_seed,
fuzz_interval_s=fuzz_interval_s,
**match_kwargs,
)
_record(result)
_analyse(result)
Expand All @@ -463,7 +482,8 @@ def _analyse(result: MatchResult) -> None:
# only user-visible reordering is the interleaved progress log.
with ProcessPoolExecutor(max_workers=n_workers) as pool:
futures = [
pool.submit(run_match, a, b, run_dir, control_scheme, fuzz_seed, fuzz_interval_s) for a, b in pairs
pool.submit(run_match, a, b, run_dir, control_scheme, fuzz_seed, fuzz_interval_s, match_seconds)
for a, b in pairs
]
for future in as_completed(futures):
result = future.result()
Expand Down Expand Up @@ -607,6 +627,7 @@ def _match_tag_of(r: MatchResult) -> str:
_print_chances(summary["strategies"])
summary["fouls"] = foul_table(summary["results"])
_print_foul_table(summary["fouls"])
_print_motion_planning_table(summary["strategies"])
if run_dir is not None:
with open(summary_path, "w") as f:
json.dump(summary, f, indent=2)
Expand Down Expand Up @@ -757,6 +778,12 @@ def strategy_table(
"matches_as_a": 0,
"real_losses_as_a": 0,
"real_loss_kinds_as_a": {},
"robot_overlap_ticks": 0,
"robot_overlap_incidents": 0,
"teammate_overlap_ticks": 0,
"teammate_overlap_incidents": 0,
"near_miss_ticks": 0,
"worst_clearance_m": None,
},
)
prefix = "" if is_a else "enemy_"
Expand All @@ -770,6 +797,15 @@ def strategy_table(
row["attacking_third_entries"] += stats.get(f"{prefix}attacking_third_entries", 0)
row["fouls"] += sum((fouls.get(own) or {}).values())
row["stalled"] += bool(stats.get("stall_events") or r.get("possession_backstop"))
row["robot_overlap_ticks"] += stats.get("robot_overlap_ticks", 0)
row["robot_overlap_incidents"] += stats.get("robot_overlap_incidents", 0)
row["teammate_overlap_ticks"] += (stats.get("teammate_overlap_ticks") or {}).get(own, 0)
row["teammate_overlap_incidents"] += (stats.get("teammate_overlap_incidents") or {}).get(own, 0)
row["near_miss_ticks"] += stats.get("near_miss_ticks", 0)
wc = stats.get("worst_clearance_m")
if wc is not None:
if row["worst_clearance_m"] is None or wc < row["worst_clearance_m"]:
row["worst_clearance_m"] = round(wc, 3)
progress.setdefault(name, []).extend(stats.get(f"{prefix}pass_progress_m") or [])
if chances_by_match and tag in chances_by_match:
chances.add(chance_totals.setdefault(name, {}), chances.side_totals(chances_by_match[tag], own))
Expand Down Expand Up @@ -924,6 +960,31 @@ def num(v: Optional[float]) -> str:
)


def _print_motion_planning_table(table: dict[str, dict]) -> None:
has_motion = any(
t.get("worst_clearance_m") is not None or t.get("robot_overlap_ticks", 0) > 0 for t in table.values()
)
if not has_motion:
return
print(
"\nMOTION PLANNING (per match averages; tm_coll = teammate overlap s (incidents), "
"total_coll = all overlap s (incidents), near_miss = near miss s <5cm, min_clr = worst clearance):"
)
print(f" {'strategy':<32} {'tm_coll':>18} {'total_coll':>20} {'near_miss':>11} {'min_clr':>10}")
for name, t in sorted(table.items(), key=lambda kv: (-kv[1]["wins"], -kv[1]["draws"])):
n = max(1, t["matches"])
tm_coll_s = (t.get("teammate_overlap_ticks", 0) / n) / TICKS_PER_SECOND
tm_inc = t.get("teammate_overlap_incidents", 0) / n
total_coll_s = (t.get("robot_overlap_ticks", 0) / n) / TICKS_PER_SECOND
tot_inc = t.get("robot_overlap_incidents", 0) / n
near_miss_s = (t.get("near_miss_ticks", 0) / n) / TICKS_PER_SECOND
wc = t.get("worst_clearance_m")
min_clr_str = f"{wc * 100:.1f} cm" if wc is not None else "-"
tm_str = f"{tm_coll_s:.2f}s ({tm_inc:.1f})"
tot_str = f"{total_coll_s:.2f}s ({tot_inc:.1f})"
print(f" {_short_name(name):<32} {tm_str:>18} {tot_str:>20} {near_miss_s:>10.2f}s {min_clr_str:>10}")


def _print_ball_losses(b: dict, run_id: str) -> None:
def top(counts: dict[str, int], k: int = 5) -> str:
return ", ".join(f"{name} {c}" for name, c in list(counts.items())[:k]) or "none"
Expand Down
80 changes: 80 additions & 0 deletions tools/motion_planning_benchmark.py
Original file line number Diff line number Diff line change
Expand Up @@ -521,6 +521,86 @@ def effective_collision_distance_m(self) -> float:
timeout_s=12.0,
endpoint_tolerance_m=0.15,
),
Scenario(
name="box_clearing_scrum",
description=(
"A defender breaks out of a congested penalty area scrum past two pressing attackers"
" and a shuttling marker to reach a flank outlet."
),
friendly_starts=((-3.0, 0.4), (-3.2, -0.4)),
friendly_targets=((-1.0, 1.8), (-3.2, -0.4)),
enemy_starts=((-2.4, 0.6), (-2.4, -0.2), (-1.8, 0.8)),
enemy_targets=((-2.7, 0.2), (-2.7, -0.3), (-1.8, 0.8)),
enemy_target_trajectories={2: _shuttling((-1.8, 0.8), (-1.8, -0.8), 0.5)},
timeout_s=15.0,
endpoint_tolerance_m=0.2,
),
Scenario(
name="wall_rebound",
description=(
"An attacker rounds a 3-robot defensive wall to collect a loose rebound while an"
" opposing defender recovers to contest the lane."
),
friendly_starts=((-0.5, 0.0),),
friendly_targets=((2.5, -1.0),),
enemy_starts=((1.5, -0.38), (1.5, 0.0), (1.5, 0.38), (1.8, 0.8)),
enemy_targets=((1.5, -0.38), (1.5, 0.0), (1.5, 0.38), (2.5, -0.6)),
timeout_s=15.0,
endpoint_tolerance_m=0.2,
),
Scenario(
name="pinching_contest",
description=(
"One robot sprints through midfield while two opposing defenders converge from"
" 45-degree diagonal flanks to squeeze the corridor."
),
friendly_starts=((-2.5, 0.0),),
friendly_targets=((1.5, 0.0),),
enemy_starts=((-0.5, 1.5), (-0.5, -1.5)),
enemy_targets=((1.2, 0.45), (1.2, -0.45)),
timeout_s=12.0,
endpoint_tolerance_m=0.2,
),
Scenario(
name="stop_transition",
description=(
"Four friendly and two enemy robots disperse from an offensive cluster around a"
" central ball keep-out anchor to legal STOP positions."
),
friendly_starts=((0.8, 0.7), (0.8, -0.7), (-0.5, 0.6), (-0.5, -0.6)),
friendly_targets=((-1.5, 1.2), (-1.5, -1.2), (-2.2, 0.6), (-2.2, -0.6)),
enemy_starts=((0.0, 0.0), (-0.6, -0.1), (-0.3, 0.3)),
enemy_targets=((0.0, 0.0), (1.5, -0.6), (1.5, 0.6)),
timeout_s=20.0,
endpoint_tolerance_m=0.25,
),
Scenario(
name="keeper_switchback",
description=(
"A defender executes a sharp hairpin switchback around an advancing pressing attacker"
" to reach an opposite-flank outlet."
),
friendly_starts=((-2.7, 0.8),),
friendly_targets=((-2.7, -0.8),),
enemy_starts=((-2.3, 0.1), (-1.7, -0.3)),
enemy_targets=((-2.9, 0.6), (-1.7, -0.3)),
timeout_s=12.0,
endpoint_tolerance_m=0.2,
),
Scenario(
name="trailing_intercept",
description=(
"A defender pursues and cuts off an opponent sprinting down the wing at 1.8 m/s"
" from an oblique trailing position."
),
friendly_starts=((-0.5, 1.2),),
friendly_targets=((2.5, 2.0),),
enemy_starts=((0.0, 2.0),),
enemy_targets=((3.5, 2.0),),
enemy_target_trajectories={0: lambda t: (min(3.5, 0.0 + 1.8 * t), 2.0)},
timeout_s=10.0,
endpoint_tolerance_m=0.2,
),
)
}

Expand Down
Loading
Loading