Skip to content

Robotics: the cap-harness engine, the benchmarks and ASPIRE no longer share code #296

Robotics: the cap-harness engine, the benchmarks and ASPIRE no longer share code

Robotics: the cap-harness engine, the benchmarks and ASPIRE no longer share code #296

Workflow file for this run

name: Tests
# A pull request runs the test groups it touches (`pytest --changed`, tests/README.md); main, a daily run and a
# manual run run the whole suite, which also catches what a selection would miss.
on:
pull_request:
push:
branches: [main]
schedule:
- cron: "0 8 * * *"
workflow_dispatch:
permissions:
contents: read
concurrency:
group: tests-${{ github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
jobs:
test:
runs-on: ubuntu-22.04 # the release checks' runner: the sandbox tests run there
timeout-minutes: 45
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0 # --changed compares with the merge base of the base branch
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install the test requirements
run: |
sudo apt-get update
sudo apt-get install -y bubblewrap strace
python -m venv "$RUNNER_TEMP/test-env"
"$RUNNER_TEMP/test-env/bin/python" -m pip install --upgrade pip
# the methods' dependency groups and the mini-swe-agent extra, so their tests run rather than skip (not
# ace's group: it pulls torch)
"$RUNNER_TEMP/test-env/bin/python" -m pip install -e '.[test,mini-swe-agent]' \
--group gepa --group skillopt --group harness2 --group rrsi --group she
echo "$RUNNER_TEMP/test-env/bin" >> "$GITHUB_PATH"
- name: The test groups this pull request touches
if: github.event_name == 'pull_request'
# not -q: the header says which groups ran and why
run: python -m pytest -rs --changed --base "origin/${{ github.base_ref }}"
- name: The whole suite
if: github.event_name != 'pull_request'
run: python -m pytest -q -rs