Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 31 additions & 10 deletions docs/contributing/env.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,32 +4,53 @@ Environments live in `plugrl-env-client`.

## Where to edit

- Implementation: `plugrl_env_client/envs/...`
- Registration: `plugrl_env_client.utils.registration`
- Implementation: `src/plugrl_env_client/envs/<family>/<family>_env.py`. The
CLI imports every `*_env.py` file under `envs/` at startup.
- Registration: `plugrl_env_client.utils.registration` (`register_env`,
`register_env_config`).
- Optional dependencies: a new extra under
`[project.optional-dependencies]` in `pyproject.toml`, and a check at the top
of the module that raises `ImportError` naming that extra, as
`envs/mujoco/mujoco_env.py` does. The CLI then skips the env with a warning
when the extra is missing, instead of failing to start.

## Checklist

- Implement a `BaseEnv` and a dataclass config.
- Register an env UID so `plugrl-run-env-client <env-id>` works.
- Ensure the env returns observations compatible with the worker recorder.
- Implement a `BaseEnv` and a dataclass config with a default for every field.
- Register an env UID so `plugrl-run-env-client <env-id>` works; the
subcommand is the UID in lowercase.
- Follow the [contract](../env/custom_env.md#contract): set
`single_action_space`, return batched arrays from `step`, honour
`reset_indices`, seed through `seed_rngs`, and never reset inside `step`.
- Return an `Observation` whose image and state arrays all have `num_envs` as
their leading axis. The `Recorder` (`plugrl_env_client.recorder`) slices it
per env to save first and last observations and videos.
- If the task has a notion of success, pass
`best_reward_threshold_for_success` to `register_env`, or the server's
`rollout/success` stays 0.

## Verify

Start a dummy server.
Start a dummy server, in `plugrl-server`. Set `--policy.action-dim` to your
env's action size; for a discrete action, add `--policy.discrete` and set it
to the number of choices.

```bash
plugrl-run-server dummy-policy default dummy default
uv run plugrl-run-server dummy-policy default dummy default --policy.action-dim <action-dim>
```

Start an env client with your env.
Start an env client with your env, in `plugrl-env-client`.

```bash
plugrl-run-env-client <env-id> --num-episodes 1 --server-host 127.0.0.1 --server-port 8000
uv run plugrl-run-env-client <env-id> --server-host 127.0.0.1 --server-port 8000 --num-episodes 3
```

## Troubleshooting

- Env client CLI cannot find env UID: registration module was not imported.
- Env client CLI cannot find the env UID: its module was not imported. Look
for a `Skip loading env module ...` warning at startup.
- `Expected action shape tail ...`: the dummy server's `--policy.action-dim`
does not match the env.
- Env creation fails: check optional dependencies and your config defaults.

## Next steps
Expand Down
33 changes: 23 additions & 10 deletions docs/contributing/env.zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,32 +4,45 @@

## 代码放哪里

- 实现:`plugrl_env_client/envs/...`
- 注册:`plugrl_env_client.utils.registration`
- 实现:`src/plugrl_env_client/envs/<family>/<family>_env.py`。CLI 启动时会 import
`envs/` 下所有 `*_env.py` 文件。
- 注册:`plugrl_env_client.utils.registration`(`register_env`、`register_env_config`)。
- 可选依赖:在 `pyproject.toml` 的 `[project.optional-dependencies]` 下加一个
extra,并在模块开头检查依赖,缺了就抛出写明这个 extra 的 `ImportError`,
和 `envs/mujoco/mujoco_env.py` 一样。这样没装 extra 时,CLI 只是打一条警告
跳过这个环境,而不是整个起不来。

## 清单

- 实现 `BaseEnv` 与配置 dataclass。
- 注册 env UID,让 `plugrl-run-env-client <env-id>` 可用。
- 观测格式与 recorder 兼容。
- 实现 `BaseEnv` 与配置 dataclass,配置的每个字段都要有默认值。
- 注册 env UID,让 `plugrl-run-env-client <env-id>` 可用;子命令是 UID 的小写形式。
- 遵守[约定](../env/custom_env.zh.md#contract):设好 `single_action_space`,
`step` 返回成批的数组,处理 `reset_indices`,通过 `seed_rngs` 播种,
绝不在 `step` 里自己 reset。
- 返回的 `Observation` 里,每个图像和状态数组的第一维都是 `num_envs`。
`Recorder`(`plugrl_env_client.recorder`)会按 env 切开它,保存首末帧观测和视频。
- 任务有"成功"这个概念的话,给 `register_env` 传 `best_reward_threshold_for_success`,
否则 server 的 `rollout/success` 一直是 0。

## 验证

启动 dummy server。
在 `plugrl-server` 里启动 dummy server。把 `--policy.action-dim` 设成你的环境的
动作维数;离散动作的话再加 `--policy.discrete`,并把它设成可选动作的个数。

```bash
plugrl-run-server dummy-policy default dummy default
uv run plugrl-run-server dummy-policy default dummy default --policy.action-dim <action-dim>
```

启动 env client,使用你的 env。
在 `plugrl-env-client` 里用你的 env 启动 env client。

```bash
plugrl-run-env-client <env-id> --num-episodes 1 --server-host 127.0.0.1 --server-port 8000
uv run plugrl-run-env-client <env-id> --server-host 127.0.0.1 --server-port 8000 --num-episodes 3
```

## 常见问题

- CLI 找不到 UID:注册模块没有被 import。
- CLI 找不到 UID:模块没有被 import。看看启动时有没有 `Skip loading env module ...` 警告。
- `Expected action shape tail ...`:dummy server 的 `--policy.action-dim` 和环境不一致。
- 创建 env 失败:检查可选依赖与默认配置。

## 下一步
Expand Down
16 changes: 11 additions & 5 deletions docs/contributing/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,9 @@ cd plugrl-server
uv sync
```

If you are working on the env client, run the same commands in `plugrl-env-client`.
If you are working on the env client, run the same commands in
`plugrl-env-client`, adding the `--extra` flags for the environments you use
(for example `uv sync --extra mujoco`).

Enable pre-commit hooks.

Expand All @@ -21,16 +23,20 @@ uv run pre-commit install

## Verify

Run a minimal end-to-end smoke test.
Run a minimal end-to-end smoke test. Each command runs from inside its own
repository.

```bash
# Terminal 1: plugrl-server
# Terminal 1, in plugrl-server
uv run plugrl-run-server dummy-policy default dummy default

# Terminal 2: plugrl-env-client
uv run plugrl-run-env-client dummy-v1 --num-episodes 1 --server-host 127.0.0.1 --server-port 8000
# Terminal 2, in plugrl-env-client
uv run plugrl-run-env-client dummy-v1 --server-host 127.0.0.1 --server-port 8000 --num-episodes 3
```

The client exits 0 after three episodes. `--server-host 127.0.0.1` is needed:
the client's default, `0.0.0.0`, is not connectable on Windows.

## What to extend

- Environments: env-client-side (`plugrl-env-client`)
Expand Down
14 changes: 9 additions & 5 deletions docs/contributing/index.zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,8 @@ cd plugrl-server
uv sync
```

如果你在改 env client 仓库,把目录换成 `plugrl-env-client`。
如果你在改 env client,就在 `plugrl-env-client` 里跑同样的命令,并加上你要用的
环境对应的 `--extra`(例如 `uv sync --extra mujoco`)。

启用 pre-commit。

Expand All @@ -21,16 +22,19 @@ uv run pre-commit install

## 验证

跑一个最小端到端 smoke test。
跑一个最小端到端 smoke test。每条命令都在它所属的仓库目录里运行。

```bash
# 终端 1:plugrl-server
# Terminal 1, in plugrl-server
uv run plugrl-run-server dummy-policy default dummy default

# 终端 2:plugrl-env-client
uv run plugrl-run-env-client dummy-v1 --num-episodes 1 --server-host 127.0.0.1 --server-port 8000
# Terminal 2, in plugrl-env-client
uv run plugrl-run-env-client dummy-v1 --server-host 127.0.0.1 --server-port 8000 --num-episodes 3
```

客户端跑完三个 episode 后以 0 退出。`--server-host 127.0.0.1` 不能省:客户端的
默认值 `0.0.0.0` 在 Windows 上连不上。

## 扩展点

- 环境:env client 侧,仓库为 `plugrl-env-client`
Expand Down
14 changes: 12 additions & 2 deletions docs/contributing/pre_commit.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,21 @@ uv run pre-commit install
uv run pre-commit run --all-files
```

In `plugrl-env-client`, add the `--extra` flags you use to `uv sync`
(for example `uv sync --extra mujoco`). A plain `uv sync` removes the
packages of every extra you leave out.

## Notes

- The first run may auto-fix files. Stage changes and run again.
- Formatting is handled by `ruff-format`.
- `plugrl-server` requires Python `>=3.11`.
- Linting and formatting are handled by `ruff` and `ruff-format`.
- `plugrl-server` requires Python `>=3.11,<3.14`; `plugrl-env-client`
requires `>=3.10,<3.13`.
- Each `.pre-commit-config.yaml` pins the hooks' Python with
`default_language_version`: `python3.11` in `plugrl-server`, `python3.10` in
`plugrl-env-client`. pre-commit builds the hook environments with the
`.venv`'s own Python if its version matches, and otherwise needs that
version installed somewhere it can find it.

## Next steps

Expand Down
11 changes: 9 additions & 2 deletions docs/contributing/pre_commit.zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,18 @@ uv run pre-commit install
uv run pre-commit run --all-files
```

在 `plugrl-env-client` 里,`uv sync` 要带上你在用的 `--extra`(例如
`uv sync --extra mujoco`)。不带的话,没点名的 extra 装的包都会被删掉。

## 说明

- 第一次跑可能会自动改文件。`git add` 后再跑一次。
- 格式化由 `ruff-format` 负责。
- `plugrl-server` 需要 Python `>=3.11`。
- 代码检查和格式化由 `ruff` 与 `ruff-format` 负责。
- `plugrl-server` 需要 Python `>=3.11,<3.14`;`plugrl-env-client` 需要 `>=3.10,<3.13`。
- 两个仓库的 `.pre-commit-config.yaml` 都用 `default_language_version` 钉死了
hook 用的 Python:`plugrl-server` 是 `python3.11`,`plugrl-env-client` 是
`python3.10`。`.venv` 自己的 Python 版本对得上时,pre-commit 就用它建 hook 的
环境;对不上时,就得另外装好那个版本,并且让 pre-commit 找得到。

## 下一步

Expand Down
Loading
Loading