Build and train systems of agents.

Install

Use uv for the main development workflow:

uv sync

Install the training backend you need:

uv sync --extra tinker --extra wandb
# OR
uv sync --extra areal --extra wandb

Install a plugin/environment from its directory:

cd plugins/<plugin-name>
uv sync --extra <backend> --extra wandb

New Environments/Plugins

New environments live as separate python packages in the plugins folder. You can add new environments by following the example of the existing plugins.

Plugins can also be used to extend the functionality of platoon beyond adding new environments. E.g., The OpenHands plugin adds support for the OpenHands agent harness.

Training

Tinker example:

cd plugins/textcraft
uv run python -m platoon.textcraft.train_scripts.tinker.train_tinker \
  --config platoon/textcraft/configs/tinker/textcraft_tinker.yaml

AReaL example:

cd plugins/textcraft

uv run python -m areal.launcher.local \
  platoon/textcraft/train_scripts/areal/train_areal_synth.py \
  --config platoon/textcraft/configs/areal/textcraft_synth_ctx40000_recursive_medium_areal.yaml

Most config values can be overridden from the CLI:

uv run python3 platoon/number_search/train.py \
  --config platoon/number_search/number_search_cispo_areal.yaml \
  trial_name=debug-run \
  train_dataset.batch_size=16

Inference

Standalone inference workflows benchmark an OpenAI-compatible endpoint and write rollouts plus aggregate reports under inference.output_dir.

cd plugins/appworld
uv run python -m platoon.appworld.run_inference \
  --config platoon/appworld/configs/inference/appworld_inference.yaml

Visualization

Use the trajectory visualization CLI to tail, replay, and analyze rollout event logs:

uv run -m platoon.visualization.cli --help

See platoon/visualization/README.md.

Experiment Reproduction

To reproduce experiments for Recursive Agent Optimization (RAO), you may refer to this branch, which is a snapshot of the codebase used for the RAO paper.

Acknowledgements

Parts of platoon's design and optimizations were inspired by many existing great RL frameworks and projects including AReaL, tinker-cookbook and agent-lightning.

Citation

Platoon was originally designed for the paper Recursive Agent Optimization (RAO). Please cite the following if you found platoon to be useful in your work:

@article{gandhi2026rao,
  title   = {Recursive Agent Optimization},
  author  = {Gandhi, Apurva and Chakraborty, Satyaki and Wang, Xiangjun
             and Kumar, Aviral and Neubig, Graham},
  journal = {arXiv preprint arXiv:2605.06639},
  year    = {2026}
}

@misc{gandhi2025platoon,
  author       = {Gandhi, Apurva},
  title        = {{Platoon}: Build and Train Systems of Agents},
  howpublished = {\url{https://github.com/ApGa/platoon}},
  year         = {2025}
}