diff --git a/experiments/e27-robomimic/.gitignore b/experiments/e27-robomimic/.gitignore new file mode 100644 index 0000000..99eef5b --- /dev/null +++ b/experiments/e27-robomimic/.gitignore @@ -0,0 +1,3 @@ +# Checkpoints and tensorboard event files; summarise.py reads out what they hold. +results/*/fpo/ +results/*/dppo/ diff --git a/experiments/e27-robomimic/FINDINGS.md b/experiments/e27-robomimic/FINDINGS.md new file mode 100644 index 0000000..fc3af97 --- /dev/null +++ b/experiments/e27-robomimic/FINDINGS.md @@ -0,0 +1,109 @@ +# E27: the platform's MLP policies run on robomimic + +2026-09-25 · Linux workstation (`guangzhao`), CPU only · two policies, two +algorithms, robomimic's `square` task, three seeds per cell · protocol: +[`PROTOCOL.md`](PROTOCOL.md) · pilots: [`results/pilot.txt`](results/pilot.txt) + +--- + +## The result + +The env client registers a robomimic family and `dppo-policy` ships a +configuration and normalisation for robomimic's `square` task, but nothing +had ever run them: on the GPU cluster the client cannot even import +robomimic. E27 is the robomimic column of E24's matrix. + +**All three cells run end to end, on every seed.** + +| cell | policy · algorithm · task | claim | result | +| --- | --- | --- | --- | +| `fpo-rm` | `fpo-policy` · FPO · square | runs | runs, 3 of 3 (14 min) | +| `fpodppo-rm` | `fpo-policy` · DPPO · square | runs | runs, 3 of 3 (15 min) | +| `dppo-rm` | `dppo-policy` · DPPO · square | runs | runs, 3 of 3 (17 min) | + +**P1 holds, 3 of 3**: on all nine seeds all twenty iterations are logged, +the checkpoint at iteration 20 is written, no server log holds a traceback +and every client exits 0. Each seed ran 81,920 environment steps as 204 +episodes. The only error in any client log is the one declared in advance +(`PROTOCOL.md`, item 3): robosuite freeing its EGL context as the interpreter +shuts down, printed as "Exception ignored in", after the server has ended the +run. + +With E24, the MLP policies' block of the matrix now reads: + +| | HalfCheetah | Hopper | Walker2d | robomimic square | +| --- | --- | --- | --- | --- | +| `fpo-policy` · FPO | learns (E6, E16) | learns (E23) | learns (E24) | **runs (E27)** | +| `fpo-policy` · DPPO | runs (E17, E18) | runs (E24) | runs (E24) | **runs (E27)** | +| `dppo-policy` · DPPO | runs (E24) | runs (E24) | runs (E24) | **runs (E27)** | + +--- + +## Getting it to run found three defects + +Each was fixed, with tests written first, before the registered run: + +1. **The `robomimic` extra could not be installed.** It pinned robomimic + 0.3.0, the last on PyPI, which imports `mujoco_py` unconditionally. + Now robomimic v0.4.0 from its tag, robosuite 1.4.1 and mujoco 2.3.7 + (plugrl-env-client#9). +2. **No episode ever ended.** robomimic's environments never end an episode + themselves and the client neither stopped on success nor at a horizon: + the first pilot finished 0 episodes in 4,096 steps. Episodes now end on + success and at robomimic's rollout horizon, 400 steps for `square` + (plugrl-env-client#9). +3. **`fpo-policy` could not read robomimic's observation.** It read + `states["obs"]` only; robomimic hands over named low-dimensional keys. + `--policy.state-keys` now concatenates them in order and checks the width + (#61). + +--- + +## Nothing was learned, and nothing was expected to be + +Every iteration of every seed has a success rate of 0, a return of 0 and +episodes of exactly 400 steps - the horizon. The shipped `square` metadata +sets `reward_shaping: false`, so the reward is 1 only on success. In 204 +episodes per seed, a randomly initialised policy never assembled the nut +once, and a sparse reward that never fires gives an on-policy algorithm +nothing to follow. + +No cell claimed to learn. `dppo-policy` can load a pretrained actor through +`checkpoint_path`, and DPPO's own robomimic results fine-tune a policy +pretrained on demonstrations; no such checkpoint was given here, so every +cell started from a random initialisation, as in E24. + +--- + +## What this does and does not support + +**Supported:** + +* Both MLP policies run on robomimic's `square` task under DPPO, and + `fpo-policy` runs under FPO - every combination of these policies and + algorithms the code allows, on a fourth task family. +* robomimic now installs and runs through the client, in its own + environment, on a Linux machine with EGL. + +**Not supported:** + +* Anything about learning on robomimic. That needs either a pretrained + actor for `dppo-policy` or a shaped reward, and neither was registered. +* That robomimic runs on the GPU cluster: the fixed extra was installed only + on `guangzhao`. +* Seeded environments. `robomimic-v1` refuses a seed, so the three seeds vary + only the policy's initialisation. + +--- + +## Reproducing + +```bash +OMP_NUM_THREADS=1 bash run.sh # three cells, nine runs at once; about 17 minutes on 24 cores +python summarise.py # P1 cell by cell, then the figures above; writes summary.tsv +``` + +`results/verdicts.txt` is `summarise.py`'s output. `summary.tsv` has one row +per cell and seed in E24's columns, so a coverage figure can read both the +same way. Checkpoints and tensorboard files stay on `guangzhao` +(`.gitignore`). diff --git a/experiments/e27-robomimic/PROTOCOL.md b/experiments/e27-robomimic/PROTOCOL.md new file mode 100644 index 0000000..bc43588 --- /dev/null +++ b/experiments/e27-robomimic/PROTOCOL.md @@ -0,0 +1,111 @@ +# E27 measurement protocol (pre-registered) + +**Written 2026-09-25, after the pilot in `results/pilot.txt` and before any +registered run.** + +This file must not be edited after the first registered data point. Anything +learned afterwards goes in `AMENDMENT.md`, dated. + +--- + +## The question + +The env client registers a robomimic family, and `dppo-policy` ships a +configuration and normalisation for robomimic's square task. Nothing had run +it. On the GPU cluster the client's environments cannot even import it: +robomimic 0.3.0, which the client's `robomimic` extra pins, imports +`mujoco_py` unconditionally, and neither cluster environment has it. E12 +measured what installing it costs, not whether it runs. + +**Do the platform's MLP policies train end to end on robomimic?** Three +cells, the robomimic column of E24's matrix. + +### The environment, and why it is this one + +On a Linux workstation (`guangzhao`), in a client venv of its own: +robomimic **v0.4.0** from its GitHub tag (PyPI stops at 0.3.0; v0.4.0 no +longer imports `mujoco_py` and supports robosuite 1.2 onwards), **robosuite +1.4.1** and **mujoco 2.3.7** - the pairing the GPU cluster already found +necessary, and the one the client's shipped environment metadata was written +for. `egl-probe`, a robomimic dependency, needs CMake to build; it was +installed into the venv, not the system. Before this file was written, the +client's own code built `NutAssemblySquare` from `square-img`, reset it and +stepped it twenty times, and its four low-dimensional keys summed to the 23 +values `dppo-policy`'s square config expects. + +### What this cannot settle + +* Learning. Every cell claims only to run end to end. +* Seeded environments. `robomimic-v1` refuses a seed - it cannot reach the + randomness of the robosuite simulation underneath - so each cell's three + seeds vary the policy's initialisation and not the environment. + +--- + +## Declared in advance: what was already known + +1. **The first pilot** (`results/pilot.txt`): all three cells ran one + iteration and exited 0 - and finished **0 episodes in 4,096 steps**. + robomimic's environments never end an episode themselves (the shipped + metadata sets `ignore_done`); robomimic's own rollouts stop on success or + at a per-task horizon, and the client did neither. A platform defect, + fixed in plugrl-env-client#9 (931ab56) with tests written first: episodes + now end on success and at robomimic's rollout horizon, 400 for this task. + The same PR makes the `robomimic` extra installable. +2. **The second pilot**, on the fixed client: each cell ran one iteration, + finished **10 episodes** in about 4,096 steps - the 400-step horizon - and + exited 0. +3. **An error at exit that is not a failure.** Every client log ends with an + `OpenGL.error.GLError` from `eglMakeCurrent`, raised in robosuite's + `EGLGLContext.__del__` as the interpreter shuts down, after the server has + ended the run. It is printed as "Exception ignored in", changes nothing + the run did, and is not counted against P1, which reads the server's log + and the client's exit code. + +--- + +## Design + +| cell | policy | algorithm | iterations | buffer / batch | seeds | +| --- | --- | --- | --- | --- | --- | +| `fpo-rm` | `fpo-policy`, 23 state values (#61) | `fpo` | 20 | 4,096 / FPO's default | 0, 1, 2 | +| `fpodppo-rm` | `fpo-policy`, 23 state values | `dppo` default | 20 | 4,096 / 256 | 0, 1, 2 | +| `dppo-rm` | `dppo-policy`, robomimic `square` config | `dppo` default | 20 | 1,024 / 256 | 0, 1, 2 | + +Task `square-img` (NutAssemblySquare), one env per client, the agentview +image at 84x84 since no policy reads images. `fpo-policy` executes every +action; `dppo-policy` its chunks of 4, so its buffer is 1,024 entries for +4,096 environment steps, as in E24. DPPO's default variant has no +learning-rate scheduler, so twenty iterations need no warmup. Code: `main` +plus #61 for the server, plugrl-env-client at 931ab56 (#9) for the client. +Every process capped at one thread, as E24 on the same machine. + +--- + +## Predictions, and what falsifies each + +**P1 - every cell runs end to end.** For each cell, on every seed: all +twenty iterations logged, the checkpoint at iteration 20, no traceback in +the server log, and a client that exits 0. + +> Reported cell by cell. A cell that fails is reported as not running, with +> its error, and not rerun until it passes. Falsified, for that cell, by any +> seed failing any part. + +**Reported, not predicted:** each run's first and last-ten returns, episode +lengths and success rates. + +--- + +## Declared deviations allowed in advance + +1. One restart of any run that dies for a reason outside the experiment, + recorded in `AMENDMENT.md`. +2. A failure caused by how a cell was launched rather than what it runs is + fixed and the cell rerun once, recorded in `AMENDMENT.md`. + +--- + +## Reading order + +P1 cell by cell, then the descriptive figures. diff --git a/experiments/e27-robomimic/results/dppo-rm.out b/experiments/e27-robomimic/results/dppo-rm.out new file mode 100644 index 0000000..332b543 --- /dev/null +++ b/experiments/e27-robomimic/results/dppo-rm.out @@ -0,0 +1,12 @@ +cell: dppo-rm = dppo-policy/default x dppo/default x robomimic square +iters: 20 x 1024 batch: 256 replan: 4 seeds: 0 1 2 +server: /home/guangzhao/zuogou/plugrl/e27/../plugrl-server/.venv/bin/python (src /home/guangzhao/zuogou/plugrl/e27/src) +client: /home/guangzhao/zuogou/plugrl/e27/../plugrl-env-client/.venv-robomimic/bin/python +out: /home/guangzhao/zuogou/plugrl/e27/experiments/e27-robomimic/results/dppo-rm +start 2026-09-25 19:09:32 +seed 0 finished rc=0 at 19:26:05 +seed 1 finished rc=0 at 19:26:09 +seed 2 finished rc=0 at 19:26:16 +end 2026-09-25 19:26:16 +failed seeds: 0 +CELL_DONE dppo-rm diff --git a/experiments/e27-robomimic/results/dppo-rm/client-seed0.log b/experiments/e27-robomimic/results/dppo-rm/client-seed0.log new file mode 100644 index 0000000..d41667e --- /dev/null +++ b/experiments/e27-robomimic/results/dppo-rm/client-seed0.log @@ -0,0 +1,82 @@ +2026-09-25 19:09:34.456 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 19:09:34.456 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 19:09:34.456 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 19:09:34.457 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +[robosuite WARNING] No private macro file found! (macros.py:53) +[robosuite WARNING] It is recommended to use a private macro file (macros.py:54) +[robosuite WARNING] To setup, run: python /home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/scripts/setup_macros.py (macros.py:55) +2026-09-25 19:09:35.140 | INFO | __main__:main:83 - Starting env client exp_name=robomimic-v1-nenv1-20260925-190935-13b95ad3 output_dir=runs/robomimic-v1-nenv1-20260925-190935-13b95ad3 recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 19:09:36.195 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9420... +2026-09-25 19:09:36.197 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'DPPOPolicy', 'action_dim': 7, 'action_horizon': 4} +2026-09-25 19:10:06.212 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=3146 infer_calls=787 feedback_calls=786 infer_wait=9.414s infer_obs_pack=0.033s env_step=20.295s feedback_total=0.142s feedback_obs_pack=0.048s feedback_info_pack=0.002s effective_fps=105.28 +2026-09-25 19:10:36.224 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=5181 infer_calls=1296 feedback_calls=1295 infer_wait=25.862s infer_obs_pack=0.054s env_step=33.666s feedback_total=0.236s feedback_obs_pack=0.078s feedback_info_pack=0.004s effective_fps=86.61 +2026-09-25 19:11:15.498 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=8197 infer_calls=2050 feedback_calls=2049 infer_wait=45.210s infer_obs_pack=0.086s env_step=53.310s feedback_total=0.368s feedback_obs_pack=0.123s feedback_info_pack=0.006s effective_fps=82.82 +2026-09-25 19:11:45.510 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=11201 infer_calls=2801 feedback_calls=2800 infer_wait=54.920s infer_obs_pack=0.118s env_step=73.324s feedback_total=0.506s feedback_obs_pack=0.169s feedback_info_pack=0.009s effective_fps=86.92 +2026-09-25 19:12:15.520 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=13233 infer_calls=3309 feedback_calls=3308 infer_wait=71.389s infer_obs_pack=0.139s env_step=86.675s feedback_total=0.597s feedback_obs_pack=0.199s feedback_info_pack=0.011s effective_fps=83.33 +2026-09-25 19:12:45.524 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=16348 infer_calls=4087 feedback_calls=4087 infer_wait=80.916s infer_obs_pack=0.172s env_step=106.864s feedback_total=0.735s feedback_obs_pack=0.245s feedback_info_pack=0.013s effective_fps=86.64 +2026-09-25 19:13:15.526 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=18389 infer_calls=4598 feedback_calls=4597 infer_wait=97.334s infer_obs_pack=0.194s env_step=120.251s feedback_total=0.833s feedback_obs_pack=0.276s feedback_info_pack=0.015s effective_fps=84.12 +2026-09-25 19:13:45.552 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=20485 infer_calls=5122 feedback_calls=5121 infer_wait=113.766s infer_obs_pack=0.216s env_step=133.649s feedback_total=0.925s feedback_obs_pack=0.306s feedback_info_pack=0.016s effective_fps=82.42 +2026-09-25 19:14:15.569 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=23585 infer_calls=5897 feedback_calls=5896 infer_wait=123.311s infer_obs_pack=0.248s env_step=153.829s feedback_total=1.065s feedback_obs_pack=0.353s feedback_info_pack=0.019s effective_fps=84.70 +2026-09-25 19:14:45.574 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=25633 infer_calls=6409 feedback_calls=6408 infer_wait=139.886s infer_obs_pack=0.270s env_step=167.060s feedback_total=1.158s feedback_obs_pack=0.383s feedback_info_pack=0.020s effective_fps=83.12 +2026-09-25 19:15:25.212 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=28677 infer_calls=7170 feedback_calls=7169 infer_wait=159.281s infer_obs_pack=0.302s env_step=187.019s feedback_total=1.291s feedback_obs_pack=0.428s feedback_info_pack=0.023s effective_fps=82.43 +2026-09-25 19:15:55.221 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=31781 infer_calls=7946 feedback_calls=7945 infer_wait=168.789s infer_obs_pack=0.409s env_step=207.157s feedback_total=1.430s feedback_obs_pack=0.475s feedback_info_pack=0.025s effective_fps=84.12 +2026-09-25 19:16:25.231 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=33813 infer_calls=8454 feedback_calls=8453 infer_wait=185.307s infer_obs_pack=0.430s env_step=220.461s feedback_total=1.519s feedback_obs_pack=0.505s feedback_info_pack=0.027s effective_fps=82.93 +2026-09-25 19:17:05.071 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=36869 infer_calls=9218 feedback_calls=9217 infer_wait=204.875s infer_obs_pack=0.462s env_step=240.447s feedback_total=1.655s feedback_obs_pack=0.552s feedback_info_pack=0.029s effective_fps=82.40 +2026-09-25 19:17:35.086 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=39957 infer_calls=9990 feedback_calls=9989 infer_wait=214.359s infer_obs_pack=0.498s env_step=260.685s feedback_total=1.796s feedback_obs_pack=0.598s feedback_info_pack=0.032s effective_fps=83.71 +2026-09-25 19:18:05.091 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=42021 infer_calls=10506 feedback_calls=10505 infer_wait=230.723s infer_obs_pack=0.520s env_step=274.126s feedback_total=1.891s feedback_obs_pack=0.629s feedback_info_pack=0.033s effective_fps=82.84 +2026-09-25 19:18:44.866 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=45061 infer_calls=11266 feedback_calls=11265 infer_wait=250.341s infer_obs_pack=0.552s env_step=294.001s feedback_total=2.026s feedback_obs_pack=0.674s feedback_info_pack=0.036s effective_fps=82.39 +2026-09-25 19:19:14.868 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=48152 infer_calls=12038 feedback_calls=12038 infer_wait=259.689s infer_obs_pack=0.584s env_step=314.365s feedback_total=2.165s feedback_obs_pack=0.720s feedback_info_pack=0.038s effective_fps=83.48 +2026-09-25 19:19:44.879 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=50201 infer_calls=12551 feedback_calls=12550 infer_wait=276.187s infer_obs_pack=0.606s env_step=327.686s feedback_total=2.257s feedback_obs_pack=0.750s feedback_info_pack=0.040s effective_fps=82.74 +2026-09-25 19:20:24.867 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=53253 infer_calls=13314 feedback_calls=13313 infer_wait=295.766s infer_obs_pack=0.639s env_step=347.806s feedback_total=2.392s feedback_obs_pack=0.796s feedback_info_pack=0.042s effective_fps=82.36 +2026-09-25 19:20:54.876 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=56305 infer_calls=14077 feedback_calls=14076 infer_wait=305.250s infer_obs_pack=0.671s env_step=368.047s feedback_total=2.529s feedback_obs_pack=0.842s feedback_info_pack=0.045s effective_fps=83.23 +2026-09-25 19:21:24.880 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=58302 infer_calls=14576 feedback_calls=14575 infer_wait=321.793s infer_obs_pack=0.693s env_step=381.320s feedback_total=2.617s feedback_obs_pack=0.872s feedback_info_pack=0.046s effective_fps=82.53 +2026-09-25 19:21:54.883 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=61331 infer_calls=15333 feedback_calls=15332 infer_wait=331.366s infer_obs_pack=0.725s env_step=401.463s feedback_total=2.751s feedback_obs_pack=0.917s feedback_info_pack=0.049s effective_fps=83.30 +2026-09-25 19:22:24.892 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=63329 infer_calls=15833 feedback_calls=15832 infer_wait=348.001s infer_obs_pack=0.746s env_step=414.646s feedback_total=2.844s feedback_obs_pack=0.948s feedback_info_pack=0.051s effective_fps=82.65 +2026-09-25 19:22:56.710 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=65541 infer_calls=16386 feedback_calls=16385 infer_wait=365.057s infer_obs_pack=0.770s env_step=429.201s feedback_total=2.943s feedback_obs_pack=0.981s feedback_info_pack=0.052s effective_fps=82.13 +2026-09-25 19:23:26.721 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=68705 infer_calls=17177 feedback_calls=17176 infer_wait=374.401s infer_obs_pack=0.803s env_step=449.572s feedback_total=3.085s feedback_obs_pack=1.028s feedback_info_pack=0.055s effective_fps=82.99 +2026-09-25 19:23:56.736 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=70833 infer_calls=17709 feedback_calls=17708 infer_wait=390.664s infer_obs_pack=0.825s env_step=463.119s feedback_total=3.182s feedback_obs_pack=1.059s feedback_info_pack=0.057s effective_fps=82.58 +2026-09-25 19:24:33.295 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=73733 infer_calls=18434 feedback_calls=18433 infer_wait=408.506s infer_obs_pack=0.855s env_step=481.564s feedback_total=3.313s feedback_obs_pack=1.102s feedback_info_pack=0.059s effective_fps=82.45 +2026-09-25 19:25:03.311 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=77161 infer_calls=19291 feedback_calls=19290 infer_wait=416.866s infer_obs_pack=0.891s env_step=502.901s feedback_total=3.479s feedback_obs_pack=1.150s feedback_info_pack=0.062s effective_fps=83.50 +2026-09-25 19:25:33.323 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=79417 infer_calls=19855 feedback_calls=19854 infer_wait=432.244s infer_obs_pack=0.915s env_step=517.302s feedback_total=3.607s feedback_obs_pack=1.183s feedback_info_pack=0.063s effective_fps=83.24 +2026-09-25 19:26:05.408 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=81924 infer_calls=20481 feedback_calls=20481 infer_wait=438.347s infer_obs_pack=0.942s env_step=533.082s feedback_total=3.732s feedback_obs_pack=1.218s feedback_info_pack=0.065s effective_fps=83.93 +2026-09-25 19:26:05.409 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 204/81920 episodes. +Created environment with name NutAssemblySquare +Action size is 7 +Exception ignored in: +Traceback (most recent call last): + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/utils/binding_utils.py", line 199, in __del__ + self.gl_ctx.free() + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/renderers/context/egl_context.py", line 149, in free + EGL.eglMakeCurrent(EGL_DISPLAY, EGL.EGL_NO_SURFACE, EGL.EGL_NO_SURFACE, EGL.EGL_NO_CONTEXT) + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/OpenGL/error.py", line 228, in glCheckError + raise GLError( +OpenGL.error.GLError: GLError( + err = 12289, + baseOperation = eglMakeCurrent, + cArguments = ( + , + , + , + , + ), + result = 0 +) +Exception ignored in: +Traceback (most recent call last): + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/renderers/context/egl_context.py", line 155, in __del__ + self.free() + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/renderers/context/egl_context.py", line 149, in free + EGL.eglMakeCurrent(EGL_DISPLAY, EGL.EGL_NO_SURFACE, EGL.EGL_NO_SURFACE, EGL.EGL_NO_CONTEXT) + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/OpenGL/error.py", line 228, in glCheckError + raise GLError( +OpenGL.error.GLError: GLError( + err = 12289, + baseOperation = eglMakeCurrent, + cArguments = ( + , + , + , + , + ), + result = 0 +) diff --git a/experiments/e27-robomimic/results/dppo-rm/client-seed1.log b/experiments/e27-robomimic/results/dppo-rm/client-seed1.log new file mode 100644 index 0000000..201a3ff --- /dev/null +++ b/experiments/e27-robomimic/results/dppo-rm/client-seed1.log @@ -0,0 +1,82 @@ +2026-09-25 19:09:40.463 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 19:09:40.464 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 19:09:40.466 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 19:09:40.466 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +[robosuite WARNING] No private macro file found! (macros.py:53) +[robosuite WARNING] It is recommended to use a private macro file (macros.py:54) +[robosuite WARNING] To setup, run: python /home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/scripts/setup_macros.py (macros.py:55) +2026-09-25 19:09:41.223 | INFO | __main__:main:83 - Starting env client exp_name=robomimic-v1-nenv1-20260925-190941-6d3d8795 output_dir=runs/robomimic-v1-nenv1-20260925-190941-6d3d8795 recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 19:09:42.337 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9421... +2026-09-25 19:09:42.340 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'DPPOPolicy', 'action_dim': 7, 'action_horizon': 4} +2026-09-25 19:10:12.358 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=3110 infer_calls=778 feedback_calls=777 infer_wait=9.753s infer_obs_pack=0.033s env_step=19.959s feedback_total=0.143s feedback_obs_pack=0.046s feedback_info_pack=0.003s effective_fps=104.05 +2026-09-25 19:10:42.365 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=5115 infer_calls=1279 feedback_calls=1278 infer_wait=26.469s infer_obs_pack=0.054s env_step=33.064s feedback_total=0.232s feedback_obs_pack=0.076s feedback_info_pack=0.004s effective_fps=85.51 +2026-09-25 19:11:12.372 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=8165 infer_calls=2042 feedback_calls=2041 infer_wait=36.273s infer_obs_pack=0.086s env_step=52.978s feedback_total=0.368s feedback_obs_pack=0.122s feedback_info_pack=0.007s effective_fps=91.02 +2026-09-25 19:11:42.378 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=10152 infer_calls=2538 feedback_calls=2538 infer_wait=52.945s infer_obs_pack=0.107s env_step=66.124s feedback_total=0.456s feedback_obs_pack=0.152s feedback_info_pack=0.008s effective_fps=84.86 +2026-09-25 19:12:13.607 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=12293 infer_calls=3074 feedback_calls=3073 infer_wait=69.874s infer_obs_pack=0.129s env_step=80.224s feedback_total=0.549s feedback_obs_pack=0.185s feedback_info_pack=0.010s effective_fps=81.53 +2026-09-25 19:12:43.612 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=15379 infer_calls=3845 feedback_calls=3844 infer_wait=79.642s infer_obs_pack=0.162s env_step=100.166s feedback_total=0.691s feedback_obs_pack=0.231s feedback_info_pack=0.012s effective_fps=85.13 +2026-09-25 19:13:13.621 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=17397 infer_calls=4350 feedback_calls=4349 infer_wait=96.218s infer_obs_pack=0.183s env_step=113.409s feedback_total=0.784s feedback_obs_pack=0.262s feedback_info_pack=0.014s effective_fps=82.61 +2026-09-25 19:13:53.454 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=20485 infer_calls=5122 feedback_calls=5121 infer_wait=115.940s infer_obs_pack=0.216s env_step=133.228s feedback_total=0.925s feedback_obs_pack=0.308s feedback_info_pack=0.017s effective_fps=81.84 +2026-09-25 19:14:23.459 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=23602 infer_calls=5901 feedback_calls=5900 infer_wait=125.556s infer_obs_pack=0.249s env_step=153.327s feedback_total=1.063s feedback_obs_pack=0.355s feedback_info_pack=0.019s effective_fps=84.23 +2026-09-25 19:14:53.476 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=25633 infer_calls=6409 feedback_calls=6408 infer_wait=142.323s infer_obs_pack=0.271s env_step=166.388s feedback_total=1.154s feedback_obs_pack=0.386s feedback_info_pack=0.021s effective_fps=82.65 +2026-09-25 19:15:33.269 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=28677 infer_calls=7170 feedback_calls=7169 infer_wait=161.999s infer_obs_pack=0.303s env_step=186.219s feedback_total=1.291s feedback_obs_pack=0.432s feedback_info_pack=0.023s effective_fps=81.98 +2026-09-25 19:16:03.273 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=31792 infer_calls=7948 feedback_calls=7948 infer_wait=171.582s infer_obs_pack=0.406s env_step=206.280s feedback_total=1.430s feedback_obs_pack=0.479s feedback_info_pack=0.026s effective_fps=83.73 +2026-09-25 19:16:33.275 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=33839 infer_calls=8460 feedback_calls=8459 infer_wait=188.147s infer_obs_pack=0.428s env_step=219.518s feedback_total=1.522s feedback_obs_pack=0.510s feedback_info_pack=0.028s effective_fps=82.61 +2026-09-25 19:17:12.830 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=36869 infer_calls=9218 feedback_calls=9217 infer_wait=207.806s infer_obs_pack=0.460s env_step=239.122s feedback_total=1.661s feedback_obs_pack=0.556s feedback_info_pack=0.030s effective_fps=82.10 +2026-09-25 19:17:42.838 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=39945 infer_calls=9987 feedback_calls=9986 infer_wait=217.419s infer_obs_pack=0.492s env_step=259.230s feedback_total=1.798s feedback_obs_pack=0.603s feedback_info_pack=0.032s effective_fps=83.40 +2026-09-25 19:18:12.843 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=41994 infer_calls=10499 feedback_calls=10498 infer_wait=234.021s infer_obs_pack=0.514s env_step=272.437s feedback_total=1.892s feedback_obs_pack=0.634s feedback_info_pack=0.034s effective_fps=82.53 +2026-09-25 19:18:52.797 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=45061 infer_calls=11266 feedback_calls=11265 infer_wait=253.753s infer_obs_pack=0.547s env_step=292.370s feedback_total=2.030s feedback_obs_pack=0.681s feedback_info_pack=0.037s effective_fps=82.12 +2026-09-25 19:19:22.806 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=48185 infer_calls=12047 feedback_calls=12046 infer_wait=263.294s infer_obs_pack=0.583s env_step=312.542s feedback_total=2.171s feedback_obs_pack=0.728s feedback_info_pack=0.039s effective_fps=83.28 +2026-09-25 19:19:52.806 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=50249 infer_calls=12563 feedback_calls=12562 infer_wait=279.789s infer_obs_pack=0.604s env_step=325.854s feedback_total=2.265s feedback_obs_pack=0.760s feedback_info_pack=0.041s effective_fps=82.58 +2026-09-25 19:20:31.964 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=53253 infer_calls=13314 feedback_calls=13313 infer_wait=299.152s infer_obs_pack=0.636s env_step=345.362s feedback_total=2.401s feedback_obs_pack=0.805s feedback_info_pack=0.043s effective_fps=82.24 +2026-09-25 19:21:01.971 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=56341 infer_calls=14086 feedback_calls=14085 infer_wait=308.868s infer_obs_pack=0.669s env_step=365.369s feedback_total=2.536s feedback_obs_pack=0.852s feedback_info_pack=0.046s effective_fps=83.17 +2026-09-25 19:21:31.981 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=58353 infer_calls=14589 feedback_calls=14588 infer_wait=325.475s infer_obs_pack=0.690s env_step=378.578s feedback_total=2.628s feedback_obs_pack=0.882s feedback_info_pack=0.048s effective_fps=82.49 +2026-09-25 19:22:01.990 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=61393 infer_calls=15349 feedback_calls=15348 infer_wait=335.212s infer_obs_pack=0.723s env_step=398.560s feedback_total=2.763s feedback_obs_pack=0.929s feedback_info_pack=0.050s effective_fps=83.27 +2026-09-25 19:22:31.996 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=63442 infer_calls=15861 feedback_calls=15860 infer_wait=351.910s infer_obs_pack=0.745s env_step=411.670s feedback_total=2.858s feedback_obs_pack=0.960s feedback_info_pack=0.052s effective_fps=82.69 +2026-09-25 19:23:02.630 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=65541 infer_calls=16386 feedback_calls=16385 infer_wait=368.744s infer_obs_pack=0.767s env_step=425.269s feedback_total=2.952s feedback_obs_pack=0.992s feedback_info_pack=0.054s effective_fps=82.16 +2026-09-25 19:23:32.640 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=68757 infer_calls=17190 feedback_calls=17189 infer_wait=377.877s infer_obs_pack=0.801s env_step=445.844s feedback_total=3.098s feedback_obs_pack=1.040s feedback_info_pack=0.056s effective_fps=83.08 +2026-09-25 19:24:02.654 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=70901 infer_calls=17726 feedback_calls=17725 infer_wait=394.092s infer_obs_pack=0.823s env_step=459.443s feedback_total=3.195s feedback_obs_pack=1.071s feedback_info_pack=0.058s effective_fps=82.68 +2026-09-25 19:24:37.865 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=73733 infer_calls=18434 feedback_calls=18433 infer_wait=411.406s infer_obs_pack=0.853s env_step=477.070s feedback_total=3.332s feedback_obs_pack=1.113s feedback_info_pack=0.060s effective_fps=82.60 +2026-09-25 19:25:07.878 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=77181 infer_calls=19296 feedback_calls=19295 infer_wait=419.827s infer_obs_pack=0.890s env_step=498.318s feedback_total=3.522s feedback_obs_pack=1.166s feedback_info_pack=0.063s effective_fps=83.66 +2026-09-25 19:25:37.888 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=79477 infer_calls=19870 feedback_calls=19869 infer_wait=435.418s infer_obs_pack=0.914s env_step=512.516s feedback_total=3.641s feedback_obs_pack=1.199s feedback_info_pack=0.065s effective_fps=83.44 +2026-09-25 19:26:09.097 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=81924 infer_calls=20481 feedback_calls=20481 infer_wait=441.458s infer_obs_pack=0.940s env_step=527.617s feedback_total=3.767s feedback_obs_pack=1.234s feedback_info_pack=0.067s effective_fps=84.13 +2026-09-25 19:26:09.097 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 204/81920 episodes. +Created environment with name NutAssemblySquare +Action size is 7 +Exception ignored in: +Traceback (most recent call last): + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/utils/binding_utils.py", line 199, in __del__ + self.gl_ctx.free() + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/renderers/context/egl_context.py", line 149, in free + EGL.eglMakeCurrent(EGL_DISPLAY, EGL.EGL_NO_SURFACE, EGL.EGL_NO_SURFACE, EGL.EGL_NO_CONTEXT) + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/OpenGL/error.py", line 228, in glCheckError + raise GLError( +OpenGL.error.GLError: GLError( + err = 12289, + baseOperation = eglMakeCurrent, + cArguments = ( + , + , + , + , + ), + result = 0 +) +Exception ignored in: +Traceback (most recent call last): + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/renderers/context/egl_context.py", line 155, in __del__ + self.free() + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/renderers/context/egl_context.py", line 149, in free + EGL.eglMakeCurrent(EGL_DISPLAY, EGL.EGL_NO_SURFACE, EGL.EGL_NO_SURFACE, EGL.EGL_NO_CONTEXT) + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/OpenGL/error.py", line 228, in glCheckError + raise GLError( +OpenGL.error.GLError: GLError( + err = 12289, + baseOperation = eglMakeCurrent, + cArguments = ( + , + , + , + , + ), + result = 0 +) diff --git a/experiments/e27-robomimic/results/dppo-rm/client-seed2.log b/experiments/e27-robomimic/results/dppo-rm/client-seed2.log new file mode 100644 index 0000000..8428e7f --- /dev/null +++ b/experiments/e27-robomimic/results/dppo-rm/client-seed2.log @@ -0,0 +1,84 @@ +2026-09-25 19:09:44.473 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 19:09:44.473 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 19:09:44.473 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 19:09:44.473 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +[robosuite WARNING] No private macro file found! (macros.py:53) +[robosuite WARNING] It is recommended to use a private macro file (macros.py:54) +[robosuite WARNING] To setup, run: python /home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/scripts/setup_macros.py (macros.py:55) +2026-09-25 19:09:45.204 | INFO | __main__:main:83 - Starting env client exp_name=robomimic-v1-nenv1-20260925-190945-02a7fa7c output_dir=runs/robomimic-v1-nenv1-20260925-190945-02a7fa7c recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 19:09:46.372 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9422... +2026-09-25 19:09:46.376 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'DPPOPolicy', 'action_dim': 7, 'action_horizon': 4} +2026-09-25 19:10:16.396 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=3159 infer_calls=790 feedback_calls=789 infer_wait=8.887s infer_obs_pack=0.033s env_step=20.803s feedback_total=0.158s feedback_obs_pack=0.047s feedback_info_pack=0.003s effective_fps=105.72 +2026-09-25 19:10:46.398 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=5211 infer_calls=1303 feedback_calls=1302 infer_wait=25.316s infer_obs_pack=0.055s env_step=34.174s feedback_total=0.258s feedback_obs_pack=0.078s feedback_info_pack=0.005s effective_fps=87.14 +2026-09-25 19:11:26.053 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=8197 infer_calls=2050 feedback_calls=2049 infer_wait=45.043s infer_obs_pack=0.086s env_step=53.821s feedback_total=0.390s feedback_obs_pack=0.123s feedback_info_pack=0.007s effective_fps=82.51 +2026-09-25 19:11:56.056 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=11206 infer_calls=2802 feedback_calls=2801 infer_wait=54.804s infer_obs_pack=0.119s env_step=73.779s feedback_total=0.524s feedback_obs_pack=0.168s feedback_info_pack=0.010s effective_fps=86.72 +2026-09-25 19:12:26.059 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=13196 infer_calls=3299 feedback_calls=3299 infer_wait=71.449s infer_obs_pack=0.141s env_step=86.956s feedback_total=0.612s feedback_obs_pack=0.197s feedback_info_pack=0.012s effective_fps=82.91 +2026-09-25 19:12:56.060 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=16281 infer_calls=4071 feedback_calls=4070 infer_wait=81.126s infer_obs_pack=0.173s env_step=106.992s feedback_total=0.747s feedback_obs_pack=0.243s feedback_info_pack=0.015s effective_fps=86.13 +2026-09-25 19:13:26.062 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=18329 infer_calls=4583 feedback_calls=4582 infer_wait=97.666s infer_obs_pack=0.194s env_step=120.261s feedback_total=0.841s feedback_obs_pack=0.275s feedback_info_pack=0.016s effective_fps=83.71 +2026-09-25 19:13:57.226 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=20481 infer_calls=5121 feedback_calls=5120 infer_wait=114.562s infer_obs_pack=0.217s env_step=134.326s feedback_total=0.938s feedback_obs_pack=0.307s feedback_info_pack=0.018s effective_fps=81.91 +2026-09-25 19:14:27.227 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=23580 infer_calls=5895 feedback_calls=5895 infer_wait=124.048s infer_obs_pack=0.249s env_step=154.558s feedback_total=1.074s feedback_obs_pack=0.353s feedback_info_pack=0.021s effective_fps=84.24 +2026-09-25 19:14:57.233 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=25622 infer_calls=6406 feedback_calls=6405 infer_wait=140.385s infer_obs_pack=0.271s env_step=168.033s feedback_total=1.165s feedback_obs_pack=0.383s feedback_info_pack=0.023s effective_fps=82.69 +2026-09-25 19:15:27.235 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=28676 infer_calls=7169 feedback_calls=7169 infer_wait=149.913s infer_obs_pack=0.303s env_step=188.222s feedback_total=1.302s feedback_obs_pack=0.429s feedback_info_pack=0.026s effective_fps=84.41 +2026-09-25 19:15:57.239 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=30703 infer_calls=7676 feedback_calls=7675 infer_wait=166.406s infer_obs_pack=0.395s env_step=201.476s feedback_total=1.390s feedback_obs_pack=0.459s feedback_info_pack=0.027s effective_fps=83.06 +2026-09-25 19:16:27.721 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=32773 infer_calls=8194 feedback_calls=8193 infer_wait=183.105s infer_obs_pack=0.417s env_step=215.066s feedback_total=1.482s feedback_obs_pack=0.490s feedback_info_pack=0.029s effective_fps=81.92 +2026-09-25 19:16:57.737 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=35861 infer_calls=8966 feedback_calls=8965 infer_wait=192.679s infer_obs_pack=0.450s env_step=235.220s feedback_total=1.620s feedback_obs_pack=0.537s feedback_info_pack=0.032s effective_fps=83.40 +2026-09-25 19:17:27.743 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=37869 infer_calls=9468 feedback_calls=9467 infer_wait=209.170s infer_obs_pack=0.471s env_step=248.546s feedback_total=1.711s feedback_obs_pack=0.567s feedback_info_pack=0.034s effective_fps=82.34 +2026-09-25 19:17:57.746 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=40948 infer_calls=10237 feedback_calls=10237 infer_wait=218.724s infer_obs_pack=0.503s env_step=268.710s feedback_total=1.845s feedback_obs_pack=0.613s feedback_info_pack=0.036s effective_fps=83.60 +2026-09-25 19:18:27.750 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=42959 infer_calls=10740 feedback_calls=10739 infer_wait=235.237s infer_obs_pack=0.524s env_step=282.014s feedback_total=1.934s feedback_obs_pack=0.643s feedback_info_pack=0.038s effective_fps=82.66 +2026-09-25 19:18:58.320 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=45061 infer_calls=11266 feedback_calls=11265 infer_wait=251.894s infer_obs_pack=0.546s env_step=295.729s feedback_total=2.030s feedback_obs_pack=0.675s feedback_info_pack=0.040s effective_fps=81.90 +2026-09-25 19:19:28.335 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=48149 infer_calls=12038 feedback_calls=12037 infer_wait=261.430s infer_obs_pack=0.580s env_step=315.920s feedback_total=2.166s feedback_obs_pack=0.721s feedback_info_pack=0.043s effective_fps=83.00 +2026-09-25 19:19:58.347 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=50165 infer_calls=12542 feedback_calls=12541 infer_wait=278.006s infer_obs_pack=0.602s env_step=329.162s feedback_total=2.260s feedback_obs_pack=0.753s feedback_info_pack=0.045s effective_fps=82.23 +2026-09-25 19:20:28.352 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=53250 infer_calls=13313 feedback_calls=13312 infer_wait=287.699s infer_obs_pack=0.635s env_step=349.186s feedback_total=2.397s feedback_obs_pack=0.799s feedback_info_pack=0.047s effective_fps=83.21 +2026-09-25 19:20:58.355 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=55283 infer_calls=13821 feedback_calls=13820 infer_wait=304.265s infer_obs_pack=0.656s env_step=362.436s feedback_total=2.484s feedback_obs_pack=0.829s feedback_info_pack=0.049s effective_fps=82.53 +2026-09-25 19:21:29.160 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=57349 infer_calls=14338 feedback_calls=14337 infer_wait=321.142s infer_obs_pack=0.679s env_step=376.167s feedback_total=2.579s feedback_obs_pack=0.860s feedback_info_pack=0.051s effective_fps=81.86 +2026-09-25 19:21:59.168 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=60381 infer_calls=15096 feedback_calls=15095 infer_wait=330.962s infer_obs_pack=0.711s env_step=396.071s feedback_total=2.711s feedback_obs_pack=0.906s feedback_info_pack=0.054s effective_fps=82.66 +2026-09-25 19:22:29.169 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=62385 infer_calls=15597 feedback_calls=15596 infer_wait=347.556s infer_obs_pack=0.732s env_step=409.288s feedback_total=2.802s feedback_obs_pack=0.936s feedback_info_pack=0.056s effective_fps=82.04 +2026-09-25 19:22:59.170 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=65416 infer_calls=16354 feedback_calls=16354 infer_wait=357.303s infer_obs_pack=0.765s env_step=429.247s feedback_total=2.941s feedback_obs_pack=0.982s feedback_info_pack=0.058s effective_fps=82.78 +2026-09-25 19:23:29.185 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=67553 infer_calls=16889 feedback_calls=16888 infer_wait=373.472s infer_obs_pack=0.787s env_step=442.897s feedback_total=3.036s feedback_obs_pack=1.014s feedback_info_pack=0.060s effective_fps=82.36 +2026-09-25 19:23:59.191 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=69649 infer_calls=17413 feedback_calls=17412 infer_wait=389.797s infer_obs_pack=0.809s env_step=456.379s feedback_total=3.132s feedback_obs_pack=1.044s feedback_info_pack=0.062s effective_fps=81.93 +2026-09-25 19:24:29.195 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=72981 infer_calls=18246 feedback_calls=18245 infer_wait=398.654s infer_obs_pack=0.844s env_step=477.216s feedback_total=3.289s feedback_obs_pack=1.092s feedback_info_pack=0.065s effective_fps=82.93 +2026-09-25 19:24:59.199 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=75300 infer_calls=18825 feedback_calls=18825 infer_wait=414.132s infer_obs_pack=0.868s env_step=491.520s feedback_total=3.405s feedback_obs_pack=1.125s feedback_info_pack=0.067s effective_fps=82.75 +2026-09-25 19:25:30.911 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=77825 infer_calls=19457 feedback_calls=19456 infer_wait=430.113s infer_obs_pack=0.894s env_step=507.005s feedback_total=3.537s feedback_obs_pack=1.163s feedback_info_pack=0.069s effective_fps=82.66 +2026-09-25 19:26:00.915 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=81277 infer_calls=20320 feedback_calls=20319 infer_wait=438.591s infer_obs_pack=0.930s env_step=528.218s feedback_total=3.696s feedback_obs_pack=1.211s feedback_info_pack=0.072s effective_fps=83.67 +2026-09-25 19:26:15.877 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=81920 infer_calls=20480 feedback_calls=20480 infer_wait=440.088s infer_obs_pack=0.937s env_step=532.126s feedback_total=3.730s feedback_obs_pack=1.221s feedback_info_pack=0.072s effective_fps=83.86 +2026-09-25 19:26:15.878 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 204/81920 episodes. +Created environment with name NutAssemblySquare +Action size is 7 +Exception ignored in: +Traceback (most recent call last): + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/utils/binding_utils.py", line 199, in __del__ + self.gl_ctx.free() + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/renderers/context/egl_context.py", line 149, in free + EGL.eglMakeCurrent(EGL_DISPLAY, EGL.EGL_NO_SURFACE, EGL.EGL_NO_SURFACE, EGL.EGL_NO_CONTEXT) + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/OpenGL/error.py", line 228, in glCheckError + raise GLError( +OpenGL.error.GLError: GLError( + err = 12289, + baseOperation = eglMakeCurrent, + cArguments = ( + , + , + , + , + ), + result = 0 +) +Exception ignored in: +Traceback (most recent call last): + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/renderers/context/egl_context.py", line 155, in __del__ + self.free() + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/renderers/context/egl_context.py", line 149, in free + EGL.eglMakeCurrent(EGL_DISPLAY, EGL.EGL_NO_SURFACE, EGL.EGL_NO_SURFACE, EGL.EGL_NO_CONTEXT) + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/OpenGL/error.py", line 228, in glCheckError + raise GLError( +OpenGL.error.GLError: GLError( + err = 12289, + baseOperation = eglMakeCurrent, + cArguments = ( + , + , + , + , + ), + result = 0 +) diff --git a/experiments/e27-robomimic/results/dppo-rm/server-seed0.log b/experiments/e27-robomimic/results/dppo-rm/server-seed0.log new file mode 100644 index 0000000..e504f01 --- /dev/null +++ b/experiments/e27-robomimic/results/dppo-rm/server-seed0.log @@ -0,0 +1,22 @@ +19:09:34|INFO|plugrl_server version: 0.1.0 +19:09:34|INFO|Algorithm: dppo, Config: DPPOAlgoConfig(global_steps=20480, gamma=0.99, gamma_denoising=1.0, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=None, critic_lr_scheduler=None, buffer_size=1024, gae_lambda=0.95, update_epochs=4, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.001, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.01, sampling_noise_level=0.01, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=False, batch_size=256, train_itrs=20, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +19:09:34|INFO|Policy: dppo-policy, Config: DPPOPolicyConfig(algo='dppo', device=device(type='cpu'), env_type='robomimic', env_name='square', checkpoint_path=None, critic=DPPOCriticObsConfig(mlp_dims=[256, 256, 256], activation='Mish', residual_style=True)) +19:09:34|INFO|Seeded python, numpy and torch with 0. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +19:09:34|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/e27/experiments/e27-robomimic/results/dppo-rm/dppo/dppo-policy/dppo-rm-seed0 +19:09:34|INFO|Number of network parameters: 2305212 +19:09:34|INFO|Policy created... +19:09:34|INFO|Initialized RolloutBuffer buffer_size=1024 action_shape=(1024, 20, 4, 7) value_shape=(1024,) +19:09:34|INFO|Algorithm created: + +19:09:34|INFO|Agent Server is listening on 0.0.0.0:9420 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +19:26:05|INFO|Checkpoint saved at step 20481 to /home/guangzhao/zuogou/plugrl/e27/experiments/e27-robomimic/results/dppo-rm/dppo/dppo-policy/dppo-rm-seed0/20481 +19:26:05|INFO|Stopping server as the algorithm signaled to stop. +19:26:05|INFO|Shutdown started: aborting pending infer requests. +19:26:05|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +19:26:05|INFO|Shutdown closing 1 websocket connection(s). +19:26:05|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 57206). +19:26:05|INFO|WebSocket server closed. +19:26:05|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e27-robomimic/results/dppo-rm/server-seed1.log b/experiments/e27-robomimic/results/dppo-rm/server-seed1.log new file mode 100644 index 0000000..69dc006 --- /dev/null +++ b/experiments/e27-robomimic/results/dppo-rm/server-seed1.log @@ -0,0 +1,22 @@ +19:09:39|INFO|plugrl_server version: 0.1.0 +19:09:39|INFO|Algorithm: dppo, Config: DPPOAlgoConfig(global_steps=20480, gamma=0.99, gamma_denoising=1.0, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=None, critic_lr_scheduler=None, buffer_size=1024, gae_lambda=0.95, update_epochs=4, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.001, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.01, sampling_noise_level=0.01, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=False, batch_size=256, train_itrs=20, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +19:09:39|INFO|Policy: dppo-policy, Config: DPPOPolicyConfig(algo='dppo', device=device(type='cpu'), env_type='robomimic', env_name='square', checkpoint_path=None, critic=DPPOCriticObsConfig(mlp_dims=[256, 256, 256], activation='Mish', residual_style=True)) +19:09:39|INFO|Seeded python, numpy and torch with 1. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +19:09:39|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/e27/experiments/e27-robomimic/results/dppo-rm/dppo/dppo-policy/dppo-rm-seed1 +19:09:39|INFO|Number of network parameters: 2305212 +19:09:39|INFO|Policy created... +19:09:39|INFO|Initialized RolloutBuffer buffer_size=1024 action_shape=(1024, 20, 4, 7) value_shape=(1024,) +19:09:39|INFO|Algorithm created: + +19:09:39|INFO|Agent Server is listening on 0.0.0.0:9421 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +19:26:09|INFO|Checkpoint saved at step 20481 to /home/guangzhao/zuogou/plugrl/e27/experiments/e27-robomimic/results/dppo-rm/dppo/dppo-policy/dppo-rm-seed1/20481 +19:26:09|INFO|Stopping server as the algorithm signaled to stop. +19:26:09|INFO|Shutdown started: aborting pending infer requests. +19:26:09|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +19:26:09|INFO|Shutdown closing 1 websocket connection(s). +19:26:09|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 37862). +19:26:09|INFO|WebSocket server closed. +19:26:09|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e27-robomimic/results/dppo-rm/server-seed2.log b/experiments/e27-robomimic/results/dppo-rm/server-seed2.log new file mode 100644 index 0000000..27bd835 --- /dev/null +++ b/experiments/e27-robomimic/results/dppo-rm/server-seed2.log @@ -0,0 +1,22 @@ +19:09:44|INFO|plugrl_server version: 0.1.0 +19:09:44|INFO|Algorithm: dppo, Config: DPPOAlgoConfig(global_steps=20480, gamma=0.99, gamma_denoising=1.0, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=None, critic_lr_scheduler=None, buffer_size=1024, gae_lambda=0.95, update_epochs=4, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.001, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.01, sampling_noise_level=0.01, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=False, batch_size=256, train_itrs=20, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +19:09:44|INFO|Policy: dppo-policy, Config: DPPOPolicyConfig(algo='dppo', device=device(type='cpu'), env_type='robomimic', env_name='square', checkpoint_path=None, critic=DPPOCriticObsConfig(mlp_dims=[256, 256, 256], activation='Mish', residual_style=True)) +19:09:44|INFO|Seeded python, numpy and torch with 2. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +19:09:44|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/e27/experiments/e27-robomimic/results/dppo-rm/dppo/dppo-policy/dppo-rm-seed2 +19:09:44|INFO|Number of network parameters: 2305212 +19:09:44|INFO|Policy created... +19:09:44|INFO|Initialized RolloutBuffer buffer_size=1024 action_shape=(1024, 20, 4, 7) value_shape=(1024,) +19:09:44|INFO|Algorithm created: + +19:09:44|INFO|Agent Server is listening on 0.0.0.0:9422 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +19:26:15|INFO|Checkpoint saved at step 20480 to /home/guangzhao/zuogou/plugrl/e27/experiments/e27-robomimic/results/dppo-rm/dppo/dppo-policy/dppo-rm-seed2/20480 +19:26:15|INFO|Stopping server as the algorithm signaled to stop. +19:26:15|INFO|Shutdown started: aborting pending infer requests. +19:26:15|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +19:26:15|INFO|Shutdown closing 1 websocket connection(s). +19:26:15|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 40762). +19:26:15|INFO|WebSocket server closed. +19:26:15|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e27-robomimic/results/fpo-rm.out b/experiments/e27-robomimic/results/fpo-rm.out new file mode 100644 index 0000000..6feab7f --- /dev/null +++ b/experiments/e27-robomimic/results/fpo-rm.out @@ -0,0 +1,12 @@ +cell: fpo-rm = fpo-policy/default x fpo/default x robomimic square +iters: 20 x 4096 batch: variant default replan: 1 seeds: 0 1 2 +server: /home/guangzhao/zuogou/plugrl/e27/../plugrl-server/.venv/bin/python (src /home/guangzhao/zuogou/plugrl/e27/src) +client: /home/guangzhao/zuogou/plugrl/e27/../plugrl-env-client/.venv-robomimic/bin/python +out: /home/guangzhao/zuogou/plugrl/e27/experiments/e27-robomimic/results/fpo-rm +start 2026-09-25 19:08:52 +seed 0 finished rc=0 at 19:22:56 +seed 1 finished rc=0 at 19:22:56 +seed 2 finished rc=0 at 19:23:00 +end 2026-09-25 19:23:00 +failed seeds: 0 +CELL_DONE fpo-rm diff --git a/experiments/e27-robomimic/results/fpo-rm/client-seed0.log b/experiments/e27-robomimic/results/fpo-rm/client-seed0.log new file mode 100644 index 0000000..f8b8e49 --- /dev/null +++ b/experiments/e27-robomimic/results/fpo-rm/client-seed0.log @@ -0,0 +1,79 @@ +2026-09-25 19:08:54.458 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 19:08:54.458 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 19:08:54.458 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 19:08:54.458 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +[robosuite WARNING] No private macro file found! (macros.py:53) +[robosuite WARNING] It is recommended to use a private macro file (macros.py:54) +[robosuite WARNING] To setup, run: python /home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/scripts/setup_macros.py (macros.py:55) +2026-09-25 19:08:55.080 | INFO | __main__:main:83 - Starting env client exp_name=robomimic-v1-nenv1-20260925-190855-b22565c2 output_dir=runs/robomimic-v1-nenv1-20260925-190855-b22565c2 recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 19:08:56.088 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9400... +2026-09-25 19:08:56.091 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'FPOAlgorithm', 'policy': 'FPOPolicy', 'action_dim': 7, 'action_horizon': 1} +2026-09-25 19:09:26.100 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=3696 infer_calls=3696 feedback_calls=3696 infer_wait=6.565s infer_obs_pack=0.144s env_step=22.600s feedback_total=0.469s feedback_obs_pack=0.181s feedback_info_pack=0.008s effective_fps=124.12 +2026-09-25 19:09:56.110 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=6725 infer_calls=6725 feedback_calls=6725 infer_wait=15.839s infer_obs_pack=0.269s env_step=42.537s feedback_total=0.943s feedback_obs_pack=0.360s feedback_info_pack=0.017s effective_fps=112.86 +2026-09-25 19:10:26.120 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=9573 infer_calls=9573 feedback_calls=9573 infer_wait=25.346s infer_obs_pack=0.470s env_step=62.161s feedback_total=1.425s feedback_obs_pack=0.540s feedback_info_pack=0.026s effective_fps=107.08 +2026-09-25 19:10:56.127 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=12381 infer_calls=12381 feedback_calls=12381 infer_wait=34.838s infer_obs_pack=0.594s env_step=81.882s feedback_total=1.899s feedback_obs_pack=0.714s feedback_info_pack=0.037s effective_fps=103.86 +2026-09-25 19:11:26.127 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=15343 infer_calls=15343 feedback_calls=15343 infer_wait=43.102s infer_obs_pack=0.731s env_step=102.756s feedback_total=2.413s feedback_obs_pack=0.896s feedback_info_pack=0.048s effective_fps=102.97 +2026-09-25 19:11:56.137 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=18170 infer_calls=18170 feedback_calls=18170 infer_wait=52.617s infer_obs_pack=0.857s env_step=122.455s feedback_total=2.883s feedback_obs_pack=1.069s feedback_info_pack=0.057s effective_fps=101.62 +2026-09-25 19:12:26.144 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=21041 infer_calls=21041 feedback_calls=21041 infer_wait=62.239s infer_obs_pack=0.984s env_step=142.029s feedback_total=3.372s feedback_obs_pack=1.248s feedback_info_pack=0.069s effective_fps=100.86 +2026-09-25 19:12:56.144 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=24098 infer_calls=24098 feedback_calls=24098 infer_wait=70.418s infer_obs_pack=1.121s env_step=162.987s feedback_total=3.881s feedback_obs_pack=1.439s feedback_info_pack=0.079s effective_fps=101.08 +2026-09-25 19:13:26.150 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=26958 infer_calls=26958 feedback_calls=26958 infer_wait=79.849s infer_obs_pack=1.250s env_step=182.749s feedback_total=4.365s feedback_obs_pack=1.614s feedback_info_pack=0.088s effective_fps=100.51 +2026-09-25 19:13:56.150 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=29777 infer_calls=29777 feedback_calls=29777 infer_wait=89.369s infer_obs_pack=1.379s env_step=202.424s feedback_total=4.847s feedback_obs_pack=1.786s feedback_info_pack=0.098s effective_fps=99.92 +2026-09-25 19:14:27.370 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=32770 infer_calls=32770 feedback_calls=32770 infer_wait=99.059s infer_obs_pack=1.509s env_step=223.120s feedback_total=5.340s feedback_obs_pack=1.971s feedback_info_pack=0.107s effective_fps=99.60 +2026-09-25 19:14:57.372 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=35793 infer_calls=35793 feedback_calls=35793 infer_wait=107.259s infer_obs_pack=1.643s env_step=244.059s feedback_total=5.864s feedback_obs_pack=2.162s feedback_info_pack=0.117s effective_fps=99.75 +2026-09-25 19:15:27.373 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=38626 infer_calls=38626 feedback_calls=38626 infer_wait=116.672s infer_obs_pack=1.766s env_step=263.849s feedback_total=6.344s feedback_obs_pack=2.338s feedback_info_pack=0.126s effective_fps=99.39 +2026-09-25 19:15:57.379 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=41500 infer_calls=41500 feedback_calls=41500 infer_wait=126.112s infer_obs_pack=1.890s env_step=283.614s feedback_total=6.827s feedback_obs_pack=2.515s feedback_info_pack=0.136s effective_fps=99.18 +2026-09-25 19:16:27.391 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=44519 infer_calls=44519 feedback_calls=44519 infer_wait=134.444s infer_obs_pack=2.021s env_step=304.430s feedback_total=7.351s feedback_obs_pack=2.705s feedback_info_pack=0.146s effective_fps=99.32 +2026-09-25 19:16:57.392 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=47389 infer_calls=47389 feedback_calls=47389 infer_wait=144.018s infer_obs_pack=2.149s env_step=324.056s feedback_total=7.828s feedback_obs_pack=2.881s feedback_info_pack=0.158s effective_fps=99.13 +2026-09-25 19:17:27.400 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=50268 infer_calls=50268 feedback_calls=50268 infer_wait=153.636s infer_obs_pack=2.272s env_step=343.638s feedback_total=8.316s feedback_obs_pack=3.056s feedback_info_pack=0.168s effective_fps=98.98 +2026-09-25 19:17:58.624 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=53250 infer_calls=53250 feedback_calls=53250 infer_wait=163.496s infer_obs_pack=2.403s env_step=364.170s feedback_total=8.813s feedback_obs_pack=3.240s feedback_info_pack=0.177s effective_fps=98.82 +2026-09-25 19:18:28.627 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=56279 infer_calls=56279 feedback_calls=56279 infer_wait=171.771s infer_obs_pack=2.536s env_step=385.041s feedback_total=9.330s feedback_obs_pack=3.433s feedback_info_pack=0.187s effective_fps=98.96 +2026-09-25 19:18:58.629 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=59106 infer_calls=59106 feedback_calls=59106 infer_wait=181.366s infer_obs_pack=2.666s env_step=404.635s feedback_total=9.818s feedback_obs_pack=3.610s feedback_info_pack=0.197s effective_fps=98.76 +2026-09-25 19:19:28.631 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=61961 infer_calls=61961 feedback_calls=61961 infer_wait=190.772s infer_obs_pack=2.794s env_step=424.427s feedback_total=10.294s feedback_obs_pack=3.791s feedback_info_pack=0.206s effective_fps=98.62 +2026-09-25 19:19:58.635 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=64991 infer_calls=64991 feedback_calls=64991 infer_wait=199.040s infer_obs_pack=2.928s env_step=445.301s feedback_total=10.812s feedback_obs_pack=3.981s feedback_info_pack=0.215s effective_fps=98.76 +2026-09-25 19:20:28.639 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=67852 infer_calls=67852 feedback_calls=67852 infer_wait=208.579s infer_obs_pack=3.051s env_step=464.960s feedback_total=11.295s feedback_obs_pack=4.159s feedback_info_pack=0.225s effective_fps=98.64 +2026-09-25 19:20:58.646 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=70727 infer_calls=70727 feedback_calls=70727 infer_wait=218.162s infer_obs_pack=3.177s env_step=484.584s feedback_total=11.773s feedback_obs_pack=4.335s feedback_info_pack=0.234s effective_fps=98.55 +2026-09-25 19:21:28.657 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=73725 infer_calls=73725 feedback_calls=73725 infer_wait=226.410s infer_obs_pack=3.314s env_step=505.469s feedback_total=12.303s feedback_obs_pack=4.525s feedback_info_pack=0.244s effective_fps=98.63 +2026-09-25 19:21:58.658 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=76526 infer_calls=76526 feedback_calls=76526 infer_wait=235.952s infer_obs_pack=3.441s env_step=525.122s feedback_total=12.780s feedback_obs_pack=4.700s feedback_info_pack=0.254s effective_fps=98.45 +2026-09-25 19:22:28.666 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=79335 infer_calls=79335 feedback_calls=79335 infer_wait=245.675s infer_obs_pack=3.564s env_step=544.599s feedback_total=13.269s feedback_obs_pack=4.880s feedback_info_pack=0.263s effective_fps=98.30 +2026-09-25 19:22:55.885 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=81921 infer_calls=81921 feedback_calls=81921 infer_wait=252.610s infer_obs_pack=3.678s env_step=562.347s feedback_total=13.699s feedback_obs_pack=5.037s feedback_info_pack=0.271s effective_fps=98.42 +2026-09-25 19:22:55.885 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 204/327680 episodes. +Created environment with name NutAssemblySquare +Action size is 7 +Exception ignored in: +Traceback (most recent call last): + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/utils/binding_utils.py", line 199, in __del__ + self.gl_ctx.free() + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/renderers/context/egl_context.py", line 149, in free + EGL.eglMakeCurrent(EGL_DISPLAY, EGL.EGL_NO_SURFACE, EGL.EGL_NO_SURFACE, EGL.EGL_NO_CONTEXT) + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/OpenGL/error.py", line 228, in glCheckError + raise GLError( +OpenGL.error.GLError: GLError( + err = 12289, + baseOperation = eglMakeCurrent, + cArguments = ( + , + , + , + , + ), + result = 0 +) +Exception ignored in: +Traceback (most recent call last): + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/renderers/context/egl_context.py", line 155, in __del__ + self.free() + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/renderers/context/egl_context.py", line 149, in free + EGL.eglMakeCurrent(EGL_DISPLAY, EGL.EGL_NO_SURFACE, EGL.EGL_NO_SURFACE, EGL.EGL_NO_CONTEXT) + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/OpenGL/error.py", line 228, in glCheckError + raise GLError( +OpenGL.error.GLError: GLError( + err = 12289, + baseOperation = eglMakeCurrent, + cArguments = ( + , + , + , + , + ), + result = 0 +) diff --git a/experiments/e27-robomimic/results/fpo-rm/client-seed1.log b/experiments/e27-robomimic/results/fpo-rm/client-seed1.log new file mode 100644 index 0000000..a21f496 --- /dev/null +++ b/experiments/e27-robomimic/results/fpo-rm/client-seed1.log @@ -0,0 +1,79 @@ +2026-09-25 19:08:59.452 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 19:08:59.452 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 19:08:59.452 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 19:08:59.452 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +[robosuite WARNING] No private macro file found! (macros.py:53) +[robosuite WARNING] It is recommended to use a private macro file (macros.py:54) +[robosuite WARNING] To setup, run: python /home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/scripts/setup_macros.py (macros.py:55) +2026-09-25 19:09:00.086 | INFO | __main__:main:83 - Starting env client exp_name=robomimic-v1-nenv1-20260925-190900-efb46da1 output_dir=runs/robomimic-v1-nenv1-20260925-190900-efb46da1 recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 19:09:01.041 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9401... +2026-09-25 19:09:01.043 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'FPOAlgorithm', 'policy': 'FPOPolicy', 'action_dim': 7, 'action_horizon': 1} +2026-09-25 19:09:31.060 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=3577 infer_calls=3577 feedback_calls=3577 infer_wait=6.756s infer_obs_pack=0.143s env_step=22.437s feedback_total=0.461s feedback_obs_pack=0.179s feedback_info_pack=0.008s effective_fps=120.05 +2026-09-25 19:10:01.068 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=6565 infer_calls=6565 feedback_calls=6565 infer_wait=16.189s infer_obs_pack=0.267s env_step=42.214s feedback_total=0.936s feedback_obs_pack=0.354s feedback_info_pack=0.018s effective_fps=110.14 +2026-09-25 19:10:31.069 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=9386 infer_calls=9386 feedback_calls=9386 infer_wait=25.556s infer_obs_pack=0.469s env_step=61.970s feedback_total=1.418s feedback_obs_pack=0.528s feedback_info_pack=0.027s effective_fps=104.97 +2026-09-25 19:11:01.077 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=12293 infer_calls=12293 feedback_calls=12293 infer_wait=35.029s infer_obs_pack=0.595s env_step=81.698s feedback_total=1.900s feedback_obs_pack=0.702s feedback_info_pack=0.037s effective_fps=103.11 +2026-09-25 19:11:31.080 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=15275 infer_calls=15275 feedback_calls=15275 infer_wait=43.249s infer_obs_pack=0.726s env_step=102.616s feedback_total=2.426s feedback_obs_pack=0.891s feedback_info_pack=0.047s effective_fps=102.50 +2026-09-25 19:12:01.090 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=18129 infer_calls=18129 feedback_calls=18129 infer_wait=52.656s infer_obs_pack=0.851s env_step=122.411s feedback_total=2.912s feedback_obs_pack=1.068s feedback_info_pack=0.056s effective_fps=101.38 +2026-09-25 19:12:31.092 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=20989 infer_calls=20989 feedback_calls=20989 infer_wait=62.102s infer_obs_pack=0.974s env_step=142.154s feedback_total=3.408s feedback_obs_pack=1.244s feedback_info_pack=0.066s effective_fps=100.60 +2026-09-25 19:13:01.093 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=24022 infer_calls=24022 feedback_calls=24022 infer_wait=70.345s infer_obs_pack=1.106s env_step=163.065s feedback_total=3.916s feedback_obs_pack=1.427s feedback_info_pack=0.076s effective_fps=100.75 +2026-09-25 19:13:31.093 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=26882 infer_calls=26882 feedback_calls=26882 infer_wait=79.740s infer_obs_pack=1.230s env_step=182.858s feedback_total=4.406s feedback_obs_pack=1.602s feedback_info_pack=0.085s effective_fps=100.22 +2026-09-25 19:14:01.094 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=29728 infer_calls=29728 feedback_calls=29728 infer_wait=89.328s infer_obs_pack=1.355s env_step=202.458s feedback_total=4.899s feedback_obs_pack=1.781s feedback_info_pack=0.094s effective_fps=99.74 +2026-09-25 19:14:31.097 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=32765 infer_calls=32765 feedback_calls=32765 infer_wait=97.479s infer_obs_pack=1.487s env_step=223.466s feedback_total=5.403s feedback_obs_pack=1.966s feedback_info_pack=0.104s effective_fps=99.94 +2026-09-25 19:15:01.106 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=35648 infer_calls=35648 feedback_calls=35648 infer_wait=107.004s infer_obs_pack=1.619s env_step=243.124s feedback_total=5.894s feedback_obs_pack=2.145s feedback_info_pack=0.113s effective_fps=99.68 +2026-09-25 19:15:31.115 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=38535 infer_calls=38535 feedback_calls=38535 infer_wait=116.444s infer_obs_pack=1.743s env_step=262.883s feedback_total=6.384s feedback_obs_pack=2.324s feedback_info_pack=0.123s effective_fps=99.46 +2026-09-25 19:16:01.122 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=41433 infer_calls=41433 feedback_calls=41433 infer_wait=125.936s infer_obs_pack=1.867s env_step=282.575s feedback_total=6.877s feedback_obs_pack=2.503s feedback_info_pack=0.132s effective_fps=99.30 +2026-09-25 19:16:31.126 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=44468 infer_calls=44468 feedback_calls=44468 infer_wait=134.161s infer_obs_pack=2.004s env_step=303.489s feedback_total=7.399s feedback_obs_pack=2.694s feedback_info_pack=0.142s effective_fps=99.47 +2026-09-25 19:17:01.135 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=47357 infer_calls=47357 feedback_calls=47357 infer_wait=143.628s infer_obs_pack=2.129s env_step=323.224s feedback_total=7.883s feedback_obs_pack=2.872s feedback_info_pack=0.152s effective_fps=99.31 +2026-09-25 19:17:31.144 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=50227 infer_calls=50227 feedback_calls=50227 infer_wait=153.136s infer_obs_pack=2.252s env_step=342.914s feedback_total=8.375s feedback_obs_pack=3.046s feedback_info_pack=0.161s effective_fps=99.13 +2026-09-25 19:18:02.229 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=53250 infer_calls=53250 feedback_calls=53250 infer_wait=163.119s infer_obs_pack=2.381s env_step=363.172s feedback_total=8.884s feedback_obs_pack=3.229s feedback_info_pack=0.171s effective_fps=99.06 +2026-09-25 19:18:32.229 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=56328 infer_calls=56328 feedback_calls=56328 infer_wait=171.331s infer_obs_pack=2.513s env_step=384.102s feedback_total=9.400s feedback_obs_pack=3.416s feedback_info_pack=0.182s effective_fps=99.28 +2026-09-25 19:19:02.234 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=59194 infer_calls=59194 feedback_calls=59194 infer_wait=180.897s infer_obs_pack=2.643s env_step=403.728s feedback_total=9.883s feedback_obs_pack=3.589s feedback_info_pack=0.191s effective_fps=99.13 +2026-09-25 19:19:32.235 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=62105 infer_calls=62105 feedback_calls=62105 infer_wait=190.429s infer_obs_pack=2.769s env_step=423.383s feedback_total=10.370s feedback_obs_pack=3.764s feedback_info_pack=0.201s effective_fps=99.06 +2026-09-25 19:20:02.238 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=65171 infer_calls=65171 feedback_calls=65171 infer_wait=198.676s infer_obs_pack=2.904s env_step=444.271s feedback_total=10.896s feedback_obs_pack=3.954s feedback_info_pack=0.212s effective_fps=99.23 +2026-09-25 19:20:32.240 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=68064 infer_calls=68064 feedback_calls=68064 infer_wait=208.296s infer_obs_pack=3.031s env_step=463.828s feedback_total=11.397s feedback_obs_pack=4.132s feedback_info_pack=0.221s effective_fps=99.14 +2026-09-25 19:21:02.252 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=70955 infer_calls=70955 feedback_calls=70955 infer_wait=217.768s infer_obs_pack=3.158s env_step=483.559s feedback_total=11.879s feedback_obs_pack=4.307s feedback_info_pack=0.231s effective_fps=99.05 +2026-09-25 19:21:32.254 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=73804 infer_calls=73804 feedback_calls=73804 infer_wait=227.405s infer_obs_pack=3.284s env_step=503.096s feedback_total=12.386s feedback_obs_pack=4.487s feedback_info_pack=0.242s effective_fps=98.91 +2026-09-25 19:22:02.255 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=76820 infer_calls=76820 feedback_calls=76820 infer_wait=235.629s infer_obs_pack=3.433s env_step=523.987s feedback_total=12.907s feedback_obs_pack=4.679s feedback_info_pack=0.252s effective_fps=99.00 +2026-09-25 19:22:32.261 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=79649 infer_calls=79649 feedback_calls=79649 infer_wait=245.203s infer_obs_pack=3.562s env_step=543.594s feedback_total=13.400s feedback_obs_pack=4.858s feedback_info_pack=0.261s effective_fps=98.85 +2026-09-25 19:22:56.357 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=81921 infer_calls=81921 feedback_calls=81921 infer_wait=251.317s infer_obs_pack=3.660s env_step=559.163s feedback_total=13.779s feedback_obs_pack=4.996s feedback_info_pack=0.269s effective_fps=98.95 +2026-09-25 19:22:56.357 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 204/327680 episodes. +Created environment with name NutAssemblySquare +Action size is 7 +Exception ignored in: +Traceback (most recent call last): + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/utils/binding_utils.py", line 199, in __del__ + self.gl_ctx.free() + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/renderers/context/egl_context.py", line 149, in free + EGL.eglMakeCurrent(EGL_DISPLAY, EGL.EGL_NO_SURFACE, EGL.EGL_NO_SURFACE, EGL.EGL_NO_CONTEXT) + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/OpenGL/error.py", line 228, in glCheckError + raise GLError( +OpenGL.error.GLError: GLError( + err = 12289, + baseOperation = eglMakeCurrent, + cArguments = ( + , + , + , + , + ), + result = 0 +) +Exception ignored in: +Traceback (most recent call last): + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/renderers/context/egl_context.py", line 155, in __del__ + self.free() + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/renderers/context/egl_context.py", line 149, in free + EGL.eglMakeCurrent(EGL_DISPLAY, EGL.EGL_NO_SURFACE, EGL.EGL_NO_SURFACE, EGL.EGL_NO_CONTEXT) + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/OpenGL/error.py", line 228, in glCheckError + raise GLError( +OpenGL.error.GLError: GLError( + err = 12289, + baseOperation = eglMakeCurrent, + cArguments = ( + , + , + , + , + ), + result = 0 +) diff --git a/experiments/e27-robomimic/results/fpo-rm/client-seed2.log b/experiments/e27-robomimic/results/fpo-rm/client-seed2.log new file mode 100644 index 0000000..3afc30c --- /dev/null +++ b/experiments/e27-robomimic/results/fpo-rm/client-seed2.log @@ -0,0 +1,79 @@ +2026-09-25 19:09:04.461 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 19:09:04.461 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 19:09:04.461 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 19:09:04.461 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +[robosuite WARNING] No private macro file found! (macros.py:53) +[robosuite WARNING] It is recommended to use a private macro file (macros.py:54) +[robosuite WARNING] To setup, run: python /home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/scripts/setup_macros.py (macros.py:55) +2026-09-25 19:09:05.108 | INFO | __main__:main:83 - Starting env client exp_name=robomimic-v1-nenv1-20260925-190905-61b9a3ea output_dir=runs/robomimic-v1-nenv1-20260925-190905-61b9a3ea recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 19:09:06.093 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9402... +2026-09-25 19:09:06.096 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'FPOAlgorithm', 'policy': 'FPOPolicy', 'action_dim': 7, 'action_horizon': 1} +2026-09-25 19:09:36.108 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=3578 infer_calls=3578 feedback_calls=3578 infer_wait=7.020s infer_obs_pack=0.137s env_step=22.166s feedback_total=0.466s feedback_obs_pack=0.182s feedback_info_pack=0.009s effective_fps=120.11 +2026-09-25 19:10:06.114 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=6548 infer_calls=6548 feedback_calls=6548 infer_wait=16.514s infer_obs_pack=0.260s env_step=41.870s feedback_total=0.944s feedback_obs_pack=0.356s feedback_info_pack=0.018s effective_fps=109.89 +2026-09-25 19:10:36.114 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=9433 infer_calls=9433 feedback_calls=9433 infer_wait=26.084s infer_obs_pack=0.462s env_step=61.410s feedback_total=1.432s feedback_obs_pack=0.528s feedback_info_pack=0.028s effective_fps=105.53 +2026-09-25 19:11:06.115 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=12329 infer_calls=12329 feedback_calls=12329 infer_wait=35.656s infer_obs_pack=0.585s env_step=81.037s feedback_total=1.913s feedback_obs_pack=0.705s feedback_info_pack=0.037s effective_fps=103.44 +2026-09-25 19:11:36.116 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=15338 infer_calls=15338 feedback_calls=15338 infer_wait=43.873s infer_obs_pack=0.714s env_step=101.956s feedback_total=2.437s feedback_obs_pack=0.895s feedback_info_pack=0.049s effective_fps=102.95 +2026-09-25 19:12:06.124 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=18209 infer_calls=18209 feedback_calls=18209 infer_wait=53.502s infer_obs_pack=0.836s env_step=121.530s feedback_total=2.924s feedback_obs_pack=1.072s feedback_info_pack=0.059s effective_fps=101.84 +2026-09-25 19:12:36.128 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=21069 infer_calls=21069 feedback_calls=21069 infer_wait=63.058s infer_obs_pack=0.958s env_step=141.169s feedback_total=3.413s feedback_obs_pack=1.249s feedback_info_pack=0.069s effective_fps=101.00 +2026-09-25 19:13:06.136 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=24136 infer_calls=24136 feedback_calls=24136 infer_wait=71.355s infer_obs_pack=1.090s env_step=162.017s feedback_total=3.935s feedback_obs_pack=1.436s feedback_info_pack=0.080s effective_fps=101.24 +2026-09-25 19:13:36.136 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=27021 infer_calls=27021 feedback_calls=27021 infer_wait=80.796s infer_obs_pack=1.212s env_step=181.781s feedback_total=4.411s feedback_obs_pack=1.607s feedback_info_pack=0.090s effective_fps=100.75 +2026-09-25 19:14:06.139 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=29890 infer_calls=29890 feedback_calls=29890 infer_wait=90.305s infer_obs_pack=1.336s env_step=201.454s feedback_total=4.911s feedback_obs_pack=1.786s feedback_info_pack=0.099s effective_fps=100.30 +2026-09-25 19:14:36.147 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=32799 infer_calls=32799 feedback_calls=32799 infer_wait=99.774s infer_obs_pack=1.461s env_step=221.185s feedback_total=5.394s feedback_obs_pack=1.961s feedback_info_pack=0.109s effective_fps=100.05 +2026-09-25 19:15:06.150 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=35889 infer_calls=35889 feedback_calls=35889 infer_wait=107.960s infer_obs_pack=1.591s env_step=242.147s feedback_total=5.904s feedback_obs_pack=2.147s feedback_info_pack=0.119s effective_fps=100.36 +2026-09-25 19:15:36.154 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=38772 infer_calls=38772 feedback_calls=38772 infer_wait=117.493s infer_obs_pack=1.714s env_step=261.800s feedback_total=6.396s feedback_obs_pack=2.326s feedback_info_pack=0.129s effective_fps=100.08 +2026-09-25 19:16:06.163 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=41689 infer_calls=41689 feedback_calls=41689 infer_wait=126.942s infer_obs_pack=1.839s env_step=281.545s feedback_total=6.881s feedback_obs_pack=2.502s feedback_info_pack=0.139s effective_fps=99.92 +2026-09-25 19:16:36.166 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=44752 infer_calls=44752 feedback_calls=44752 infer_wait=135.249s infer_obs_pack=1.969s env_step=302.378s feedback_total=7.408s feedback_obs_pack=2.689s feedback_info_pack=0.153s effective_fps=100.12 +2026-09-25 19:17:06.168 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=47628 infer_calls=47628 feedback_calls=47628 infer_wait=144.738s infer_obs_pack=2.094s env_step=322.083s feedback_total=7.889s feedback_obs_pack=2.868s feedback_info_pack=0.162s effective_fps=99.89 +2026-09-25 19:17:36.172 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=50475 infer_calls=50475 feedback_calls=50475 infer_wait=154.315s infer_obs_pack=2.221s env_step=341.709s feedback_total=8.370s feedback_obs_pack=3.044s feedback_info_pack=0.172s effective_fps=99.63 +2026-09-25 19:18:06.175 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=53345 infer_calls=53345 feedback_calls=53345 infer_wait=163.799s infer_obs_pack=2.344s env_step=361.402s feedback_total=8.873s feedback_obs_pack=3.228s feedback_info_pack=0.181s effective_fps=99.45 +2026-09-25 19:18:36.181 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=56420 infer_calls=56420 feedback_calls=56420 infer_wait=172.016s infer_obs_pack=2.473s env_step=382.339s feedback_total=9.380s feedback_obs_pack=3.417s feedback_info_pack=0.192s effective_fps=99.65 +2026-09-25 19:19:06.188 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=59286 infer_calls=59286 feedback_calls=59286 infer_wait=181.519s infer_obs_pack=2.599s env_step=402.030s feedback_total=9.873s feedback_obs_pack=3.597s feedback_info_pack=0.201s effective_fps=99.47 +2026-09-25 19:19:36.189 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=62185 infer_calls=62185 feedback_calls=62185 infer_wait=191.082s infer_obs_pack=2.720s env_step=421.669s feedback_total=10.351s feedback_obs_pack=3.769s feedback_info_pack=0.211s effective_fps=99.37 +2026-09-25 19:20:06.190 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=65253 infer_calls=65253 feedback_calls=65253 infer_wait=199.372s infer_obs_pack=2.851s env_step=442.517s feedback_total=10.873s feedback_obs_pack=3.956s feedback_info_pack=0.223s effective_fps=99.53 +2026-09-25 19:20:36.200 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=68118 infer_calls=68118 feedback_calls=68118 infer_wait=208.928s infer_obs_pack=2.974s env_step=462.162s feedback_total=11.364s feedback_obs_pack=4.138s feedback_info_pack=0.233s effective_fps=99.38 +2026-09-25 19:21:06.206 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=71041 infer_calls=71041 feedback_calls=71041 infer_wait=218.451s infer_obs_pack=3.100s env_step=481.836s feedback_total=11.851s feedback_obs_pack=4.314s feedback_info_pack=0.243s effective_fps=99.33 +2026-09-25 19:21:36.208 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=73841 infer_calls=73841 feedback_calls=73841 infer_wait=227.957s infer_obs_pack=3.229s env_step=501.506s feedback_total=12.351s feedback_obs_pack=4.497s feedback_info_pack=0.252s effective_fps=99.11 +2026-09-25 19:22:06.208 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=76859 infer_calls=76859 feedback_calls=76859 infer_wait=236.202s infer_obs_pack=3.360s env_step=522.403s feedback_total=12.864s feedback_obs_pack=4.681s feedback_info_pack=0.264s effective_fps=99.19 +2026-09-25 19:22:36.214 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=79687 infer_calls=79687 feedback_calls=79687 infer_wait=245.728s infer_obs_pack=3.483s env_step=542.068s feedback_total=13.361s feedback_obs_pack=4.865s feedback_info_pack=0.273s effective_fps=99.03 +2026-09-25 19:23:00.061 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=81921 infer_calls=81921 feedback_calls=81921 infer_wait=251.819s infer_obs_pack=3.578s env_step=557.462s feedback_total=13.750s feedback_obs_pack=5.004s feedback_info_pack=0.281s effective_fps=99.10 +2026-09-25 19:23:00.061 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 204/327680 episodes. +Created environment with name NutAssemblySquare +Action size is 7 +Exception ignored in: +Traceback (most recent call last): + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/utils/binding_utils.py", line 199, in __del__ + self.gl_ctx.free() + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/renderers/context/egl_context.py", line 149, in free + EGL.eglMakeCurrent(EGL_DISPLAY, EGL.EGL_NO_SURFACE, EGL.EGL_NO_SURFACE, EGL.EGL_NO_CONTEXT) + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/OpenGL/error.py", line 228, in glCheckError + raise GLError( +OpenGL.error.GLError: GLError( + err = 12289, + baseOperation = eglMakeCurrent, + cArguments = ( + , + , + , + , + ), + result = 0 +) +Exception ignored in: +Traceback (most recent call last): + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/renderers/context/egl_context.py", line 155, in __del__ + self.free() + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/renderers/context/egl_context.py", line 149, in free + EGL.eglMakeCurrent(EGL_DISPLAY, EGL.EGL_NO_SURFACE, EGL.EGL_NO_SURFACE, EGL.EGL_NO_CONTEXT) + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/OpenGL/error.py", line 228, in glCheckError + raise GLError( +OpenGL.error.GLError: GLError( + err = 12289, + baseOperation = eglMakeCurrent, + cArguments = ( + , + , + , + , + ), + result = 0 +) diff --git a/experiments/e27-robomimic/results/fpo-rm/server-seed0.log b/experiments/e27-robomimic/results/fpo-rm/server-seed0.log new file mode 100644 index 0000000..325f3b6 --- /dev/null +++ b/experiments/e27-robomimic/results/fpo-rm/server-seed0.log @@ -0,0 +1,19 @@ +19:08:53|INFO|plugrl_server version: 0.1.0 +19:08:53|INFO|Algorithm: fpo, Config: FPOAlgoConfig(global_steps=81920, buffer_size=4096, output_mode='u_but_supervise_as_eps', fpo_playground_trick=True, treat_truncated_as_done=False, discounting=0.995, reward_scaling=10.0, gae_lambda=0.95, batch_size=1024, num_updates_per_batch=16, learning_rate=0.0003, value_loss_coeff=0.25, clipping_epsilon=0.05, normalize_advantage=True, n_critic_warmup_itrs=0, max_policy_drift=0.0, n_samples_per_action=8, discretize_t_for_training=True, average_losses_before_exp=True, save_interval=20, policy_checkpoint_path=None, restore='all', master_weights_device=None) +19:08:53|INFO|Policy: fpo-policy, Config: FPOPolicyConfig(algo='fpo', device=device(type='cpu'), obs_dim=23, action_dim=7, flow_steps=10, timestep_embed_dim=8, action_horizon=1, feather_std=0.0, policy_mlp_output_scale=0.25, normalize_observations=True, hidden_dims=(32, 32, 32, 32), value_hidden_dims=(256, 256, 256, 256, 256), state_keys=('robot0_eef_pos', 'robot0_eef_quat', 'robot0_gripper_qpos', 'object')) +19:08:53|INFO|Seeded python, numpy and torch with 0. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +19:08:53|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/e27/experiments/e27-robomimic/results/fpo-rm/fpo/fpo-policy/fpo-rm-seed0 +19:08:53|INFO|Policy created... +19:08:53|INFO|Initialized RolloutBuffer buffer_size=4096 action_shape=(4096, 1, 7) value_shape=(4096, 1) +19:08:53|INFO|Algorithm created: + +19:08:53|INFO|Agent Server is listening on 0.0.0.0:9400 +19:22:55|INFO|Checkpoint saved at step 81921 to /home/guangzhao/zuogou/plugrl/e27/experiments/e27-robomimic/results/fpo-rm/fpo/fpo-policy/fpo-rm-seed0/81921 +19:22:55|INFO|Stopping server as the algorithm signaled to stop. +19:22:55|INFO|Shutdown started: aborting pending infer requests. +19:22:55|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +19:22:55|INFO|Shutdown closing 1 websocket connection(s). +19:22:55|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 39556). +19:22:55|INFO|WebSocket server closed. +19:22:55|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e27-robomimic/results/fpo-rm/server-seed1.log b/experiments/e27-robomimic/results/fpo-rm/server-seed1.log new file mode 100644 index 0000000..c03af58 --- /dev/null +++ b/experiments/e27-robomimic/results/fpo-rm/server-seed1.log @@ -0,0 +1,19 @@ +19:08:58|INFO|plugrl_server version: 0.1.0 +19:08:58|INFO|Algorithm: fpo, Config: FPOAlgoConfig(global_steps=81920, buffer_size=4096, output_mode='u_but_supervise_as_eps', fpo_playground_trick=True, treat_truncated_as_done=False, discounting=0.995, reward_scaling=10.0, gae_lambda=0.95, batch_size=1024, num_updates_per_batch=16, learning_rate=0.0003, value_loss_coeff=0.25, clipping_epsilon=0.05, normalize_advantage=True, n_critic_warmup_itrs=0, max_policy_drift=0.0, n_samples_per_action=8, discretize_t_for_training=True, average_losses_before_exp=True, save_interval=20, policy_checkpoint_path=None, restore='all', master_weights_device=None) +19:08:58|INFO|Policy: fpo-policy, Config: FPOPolicyConfig(algo='fpo', device=device(type='cpu'), obs_dim=23, action_dim=7, flow_steps=10, timestep_embed_dim=8, action_horizon=1, feather_std=0.0, policy_mlp_output_scale=0.25, normalize_observations=True, hidden_dims=(32, 32, 32, 32), value_hidden_dims=(256, 256, 256, 256, 256), state_keys=('robot0_eef_pos', 'robot0_eef_quat', 'robot0_gripper_qpos', 'object')) +19:08:58|INFO|Seeded python, numpy and torch with 1. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +19:08:58|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/e27/experiments/e27-robomimic/results/fpo-rm/fpo/fpo-policy/fpo-rm-seed1 +19:08:59|INFO|Policy created... +19:08:59|INFO|Initialized RolloutBuffer buffer_size=4096 action_shape=(4096, 1, 7) value_shape=(4096, 1) +19:08:59|INFO|Algorithm created: + +19:08:59|INFO|Agent Server is listening on 0.0.0.0:9401 +19:22:56|INFO|Checkpoint saved at step 81921 to /home/guangzhao/zuogou/plugrl/e27/experiments/e27-robomimic/results/fpo-rm/fpo/fpo-policy/fpo-rm-seed1/81921 +19:22:56|INFO|Stopping server as the algorithm signaled to stop. +19:22:56|INFO|Shutdown started: aborting pending infer requests. +19:22:56|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +19:22:56|INFO|Shutdown closing 1 websocket connection(s). +19:22:56|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 45358). +19:22:56|INFO|WebSocket server closed. +19:22:56|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e27-robomimic/results/fpo-rm/server-seed2.log b/experiments/e27-robomimic/results/fpo-rm/server-seed2.log new file mode 100644 index 0000000..cbfd722 --- /dev/null +++ b/experiments/e27-robomimic/results/fpo-rm/server-seed2.log @@ -0,0 +1,19 @@ +19:09:04|INFO|plugrl_server version: 0.1.0 +19:09:04|INFO|Algorithm: fpo, Config: FPOAlgoConfig(global_steps=81920, buffer_size=4096, output_mode='u_but_supervise_as_eps', fpo_playground_trick=True, treat_truncated_as_done=False, discounting=0.995, reward_scaling=10.0, gae_lambda=0.95, batch_size=1024, num_updates_per_batch=16, learning_rate=0.0003, value_loss_coeff=0.25, clipping_epsilon=0.05, normalize_advantage=True, n_critic_warmup_itrs=0, max_policy_drift=0.0, n_samples_per_action=8, discretize_t_for_training=True, average_losses_before_exp=True, save_interval=20, policy_checkpoint_path=None, restore='all', master_weights_device=None) +19:09:04|INFO|Policy: fpo-policy, Config: FPOPolicyConfig(algo='fpo', device=device(type='cpu'), obs_dim=23, action_dim=7, flow_steps=10, timestep_embed_dim=8, action_horizon=1, feather_std=0.0, policy_mlp_output_scale=0.25, normalize_observations=True, hidden_dims=(32, 32, 32, 32), value_hidden_dims=(256, 256, 256, 256, 256), state_keys=('robot0_eef_pos', 'robot0_eef_quat', 'robot0_gripper_qpos', 'object')) +19:09:04|INFO|Seeded python, numpy and torch with 2. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +19:09:04|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/e27/experiments/e27-robomimic/results/fpo-rm/fpo/fpo-policy/fpo-rm-seed2 +19:09:04|INFO|Policy created... +19:09:04|INFO|Initialized RolloutBuffer buffer_size=4096 action_shape=(4096, 1, 7) value_shape=(4096, 1) +19:09:04|INFO|Algorithm created: + +19:09:04|INFO|Agent Server is listening on 0.0.0.0:9402 +19:23:00|INFO|Checkpoint saved at step 81921 to /home/guangzhao/zuogou/plugrl/e27/experiments/e27-robomimic/results/fpo-rm/fpo/fpo-policy/fpo-rm-seed2/81921 +19:23:00|INFO|Stopping server as the algorithm signaled to stop. +19:23:00|INFO|Shutdown started: aborting pending infer requests. +19:23:00|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +19:23:00|INFO|Shutdown closing 1 websocket connection(s). +19:23:00|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 51134). +19:23:00|INFO|WebSocket server closed. +19:23:00|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e27-robomimic/results/fpodppo-rm.out b/experiments/e27-robomimic/results/fpodppo-rm.out new file mode 100644 index 0000000..9c49e1c --- /dev/null +++ b/experiments/e27-robomimic/results/fpodppo-rm.out @@ -0,0 +1,12 @@ +cell: fpodppo-rm = fpo-policy/default x dppo/default x robomimic square +iters: 20 x 4096 batch: 256 replan: 1 seeds: 0 1 2 +server: /home/guangzhao/zuogou/plugrl/e27/../plugrl-server/.venv/bin/python (src /home/guangzhao/zuogou/plugrl/e27/src) +client: /home/guangzhao/zuogou/plugrl/e27/../plugrl-env-client/.venv-robomimic/bin/python +out: /home/guangzhao/zuogou/plugrl/e27/experiments/e27-robomimic/results/fpodppo-rm +start 2026-09-25 19:09:12 +seed 0 finished rc=0 at 19:24:14 +seed 1 finished rc=0 at 19:24:19 +seed 2 finished rc=0 at 19:24:19 +end 2026-09-25 19:24:19 +failed seeds: 0 +CELL_DONE fpodppo-rm diff --git a/experiments/e27-robomimic/results/fpodppo-rm/client-seed0.log b/experiments/e27-robomimic/results/fpodppo-rm/client-seed0.log new file mode 100644 index 0000000..d40c629 --- /dev/null +++ b/experiments/e27-robomimic/results/fpodppo-rm/client-seed0.log @@ -0,0 +1,81 @@ +2026-09-25 19:09:14.454 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 19:09:14.454 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 19:09:14.454 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 19:09:14.455 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +[robosuite WARNING] No private macro file found! (macros.py:53) +[robosuite WARNING] It is recommended to use a private macro file (macros.py:54) +[robosuite WARNING] To setup, run: python /home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/scripts/setup_macros.py (macros.py:55) +2026-09-25 19:09:15.131 | INFO | __main__:main:83 - Starting env client exp_name=robomimic-v1-nenv1-20260925-190915-40841d76 output_dir=runs/robomimic-v1-nenv1-20260925-190915-40841d76 recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 19:09:16.169 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9410... +2026-09-25 19:09:16.171 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'FPOPolicy', 'action_dim': 7, 'action_horizon': 1} +2026-09-25 19:09:46.186 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=3126 infer_calls=3126 feedback_calls=3126 infer_wait=9.258s infer_obs_pack=0.126s env_step=19.967s feedback_total=0.453s feedback_obs_pack=0.170s feedback_info_pack=0.010s effective_fps=104.89 +2026-09-25 19:10:16.195 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=5870 infer_calls=5870 feedback_calls=5870 infer_wait=19.655s infer_obs_pack=0.246s env_step=38.817s feedback_total=0.906s feedback_obs_pack=0.335s feedback_info_pack=0.019s effective_fps=98.45 +2026-09-25 19:10:46.202 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=8554 infer_calls=8554 feedback_calls=8554 infer_wait=30.249s infer_obs_pack=0.440s env_step=57.378s feedback_total=1.380s feedback_obs_pack=0.501s feedback_info_pack=0.028s effective_fps=95.63 +2026-09-25 19:11:16.204 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=11229 infer_calls=11229 feedback_calls=11229 infer_wait=40.392s infer_obs_pack=0.561s env_step=76.450s feedback_total=1.853s feedback_obs_pack=0.672s feedback_info_pack=0.038s effective_fps=94.16 +2026-09-25 19:11:46.205 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=13884 infer_calls=13884 feedback_calls=13884 infer_wait=51.059s infer_obs_pack=0.678s env_step=95.016s feedback_total=2.319s feedback_obs_pack=0.839s feedback_info_pack=0.047s effective_fps=93.14 +2026-09-25 19:12:16.212 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=16577 infer_calls=16577 feedback_calls=16577 infer_wait=61.418s infer_obs_pack=0.797s env_step=113.886s feedback_total=2.787s feedback_obs_pack=1.005s feedback_info_pack=0.057s effective_fps=92.67 +2026-09-25 19:12:46.223 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=19296 infer_calls=19296 feedback_calls=19296 infer_wait=71.428s infer_obs_pack=0.916s env_step=133.114s feedback_total=3.255s feedback_obs_pack=1.174s feedback_info_pack=0.066s effective_fps=92.45 +2026-09-25 19:13:16.230 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=21982 infer_calls=21982 feedback_calls=21982 infer_wait=81.912s infer_obs_pack=1.034s env_step=151.865s feedback_total=3.726s feedback_obs_pack=1.342s feedback_info_pack=0.075s effective_fps=92.15 +2026-09-25 19:13:46.233 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=24695 infer_calls=24695 feedback_calls=24695 infer_wait=92.334s infer_obs_pack=1.156s env_step=170.669s feedback_total=4.194s feedback_obs_pack=1.515s feedback_info_pack=0.084s effective_fps=92.02 +2026-09-25 19:14:16.241 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=27472 infer_calls=27472 feedback_calls=27472 infer_wait=102.409s infer_obs_pack=1.276s env_step=189.803s feedback_total=4.680s feedback_obs_pack=1.691s feedback_info_pack=0.094s effective_fps=92.14 +2026-09-25 19:14:46.247 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=30143 infer_calls=30143 feedback_calls=30143 infer_wait=112.785s infer_obs_pack=1.395s env_step=208.652s feedback_total=5.152s feedback_obs_pack=1.860s feedback_info_pack=0.103s effective_fps=91.90 +2026-09-25 19:15:16.252 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=32892 infer_calls=32892 feedback_calls=32892 infer_wait=123.163s infer_obs_pack=1.514s env_step=227.504s feedback_total=5.618s feedback_obs_pack=2.035s feedback_info_pack=0.113s effective_fps=91.93 +2026-09-25 19:15:46.266 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=35587 infer_calls=35587 feedback_calls=35587 infer_wait=133.425s infer_obs_pack=1.631s env_step=246.480s feedback_total=6.088s feedback_obs_pack=2.207s feedback_info_pack=0.122s effective_fps=91.81 +2026-09-25 19:16:16.270 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=38300 infer_calls=38300 feedback_calls=38300 infer_wait=143.960s infer_obs_pack=1.753s env_step=265.172s feedback_total=6.551s feedback_obs_pack=2.376s feedback_info_pack=0.131s effective_fps=91.75 +2026-09-25 19:16:46.277 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=40974 infer_calls=40974 feedback_calls=40974 infer_wait=154.538s infer_obs_pack=1.869s env_step=283.838s feedback_total=7.013s feedback_obs_pack=2.542s feedback_info_pack=0.140s effective_fps=91.61 +2026-09-25 19:17:16.279 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=43710 infer_calls=43710 feedback_calls=43710 infer_wait=164.682s infer_obs_pack=1.988s env_step=302.905s feedback_total=7.491s feedback_obs_pack=2.713s feedback_info_pack=0.150s effective_fps=91.62 +2026-09-25 19:17:46.284 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=46396 infer_calls=46396 feedback_calls=46396 infer_wait=175.100s infer_obs_pack=2.107s env_step=321.720s feedback_total=7.953s feedback_obs_pack=2.883s feedback_info_pack=0.159s effective_fps=91.53 +2026-09-25 19:18:16.290 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=49100 infer_calls=49100 feedback_calls=49100 infer_wait=185.419s infer_obs_pack=2.226s env_step=340.624s feedback_total=8.429s feedback_obs_pack=3.057s feedback_info_pack=0.169s effective_fps=91.49 +2026-09-25 19:18:46.298 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=51813 infer_calls=51813 feedback_calls=51813 infer_wait=195.990s infer_obs_pack=2.344s env_step=359.294s feedback_total=8.890s feedback_obs_pack=3.224s feedback_info_pack=0.178s effective_fps=91.46 +2026-09-25 19:19:16.299 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=54518 infer_calls=54518 feedback_calls=54518 infer_wait=206.465s infer_obs_pack=2.464s env_step=378.050s feedback_total=9.348s feedback_obs_pack=3.392s feedback_info_pack=0.187s effective_fps=91.42 +2026-09-25 19:19:46.309 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=57249 infer_calls=57249 feedback_calls=57249 infer_wait=216.716s infer_obs_pack=2.587s env_step=397.019s feedback_total=9.827s feedback_obs_pack=3.566s feedback_info_pack=0.197s effective_fps=91.43 +2026-09-25 19:20:16.311 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=59996 infer_calls=59996 feedback_calls=59996 infer_wait=227.193s infer_obs_pack=2.705s env_step=415.770s feedback_total=10.290s feedback_obs_pack=3.735s feedback_info_pack=0.206s effective_fps=91.46 +2026-09-25 19:20:46.320 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=62693 infer_calls=62693 feedback_calls=62693 infer_wait=237.773s infer_obs_pack=2.823s env_step=434.434s feedback_total=10.751s feedback_obs_pack=3.901s feedback_info_pack=0.216s effective_fps=91.42 +2026-09-25 19:21:16.324 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=65409 infer_calls=65409 feedback_calls=65409 infer_wait=248.073s infer_obs_pack=2.941s env_step=453.360s feedback_total=11.219s feedback_obs_pack=4.072s feedback_info_pack=0.225s effective_fps=91.41 +2026-09-25 19:21:46.325 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=68001 infer_calls=68001 feedback_calls=68001 infer_wait=258.571s infer_obs_pack=3.058s env_step=472.093s feedback_total=11.687s feedback_obs_pack=4.245s feedback_info_pack=0.235s effective_fps=91.23 +2026-09-25 19:22:16.327 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=70650 infer_calls=70650 feedback_calls=70650 infer_wait=269.198s infer_obs_pack=3.178s env_step=490.682s feedback_total=12.165s feedback_obs_pack=4.420s feedback_info_pack=0.244s effective_fps=91.14 +2026-09-25 19:22:46.330 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=73374 infer_calls=73374 feedback_calls=73374 infer_wait=279.416s infer_obs_pack=3.298s env_step=509.684s feedback_total=12.635s feedback_obs_pack=4.595s feedback_info_pack=0.254s effective_fps=91.14 +2026-09-25 19:23:16.336 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=76197 infer_calls=76197 feedback_calls=76197 infer_wait=289.423s infer_obs_pack=3.420s env_step=528.875s feedback_total=13.125s feedback_obs_pack=4.773s feedback_info_pack=0.264s effective_fps=91.27 +2026-09-25 19:23:46.342 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=79148 infer_calls=79148 feedback_calls=79148 infer_wait=299.079s infer_obs_pack=3.544s env_step=548.413s feedback_total=13.614s feedback_obs_pack=4.952s feedback_info_pack=0.273s effective_fps=91.54 +2026-09-25 19:24:13.701 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=81921 infer_calls=81921 feedback_calls=81921 infer_wait=307.376s infer_obs_pack=3.660s env_step=566.259s feedback_total=14.047s feedback_obs_pack=5.110s feedback_info_pack=0.282s effective_fps=91.91 +2026-09-25 19:24:13.701 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 204/327680 episodes. +Created environment with name NutAssemblySquare +Action size is 7 +Exception ignored in: +Traceback (most recent call last): + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/utils/binding_utils.py", line 199, in __del__ + self.gl_ctx.free() + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/renderers/context/egl_context.py", line 149, in free + EGL.eglMakeCurrent(EGL_DISPLAY, EGL.EGL_NO_SURFACE, EGL.EGL_NO_SURFACE, EGL.EGL_NO_CONTEXT) + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/OpenGL/error.py", line 228, in glCheckError + raise GLError( +OpenGL.error.GLError: GLError( + err = 12289, + baseOperation = eglMakeCurrent, + cArguments = ( + , + , + , + , + ), + result = 0 +) +Exception ignored in: +Traceback (most recent call last): + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/renderers/context/egl_context.py", line 155, in __del__ + self.free() + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/renderers/context/egl_context.py", line 149, in free + EGL.eglMakeCurrent(EGL_DISPLAY, EGL.EGL_NO_SURFACE, EGL.EGL_NO_SURFACE, EGL.EGL_NO_CONTEXT) + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/OpenGL/error.py", line 228, in glCheckError + raise GLError( +OpenGL.error.GLError: GLError( + err = 12289, + baseOperation = eglMakeCurrent, + cArguments = ( + , + , + , + , + ), + result = 0 +) diff --git a/experiments/e27-robomimic/results/fpodppo-rm/client-seed1.log b/experiments/e27-robomimic/results/fpodppo-rm/client-seed1.log new file mode 100644 index 0000000..850f5ee --- /dev/null +++ b/experiments/e27-robomimic/results/fpodppo-rm/client-seed1.log @@ -0,0 +1,81 @@ +2026-09-25 19:09:19.457 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 19:09:19.458 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 19:09:19.458 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 19:09:19.458 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +[robosuite WARNING] No private macro file found! (macros.py:53) +[robosuite WARNING] It is recommended to use a private macro file (macros.py:54) +[robosuite WARNING] To setup, run: python /home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/scripts/setup_macros.py (macros.py:55) +2026-09-25 19:09:20.129 | INFO | __main__:main:83 - Starting env client exp_name=robomimic-v1-nenv1-20260925-190920-606127a5 output_dir=runs/robomimic-v1-nenv1-20260925-190920-606127a5 recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 19:09:21.173 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9411... +2026-09-25 19:09:21.176 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'FPOPolicy', 'action_dim': 7, 'action_horizon': 1} +2026-09-25 19:09:51.197 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=2987 infer_calls=2987 feedback_calls=2987 infer_wait=9.553s infer_obs_pack=0.120s env_step=19.690s feedback_total=0.456s feedback_obs_pack=0.168s feedback_info_pack=0.009s effective_fps=100.17 +2026-09-25 19:10:21.205 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=5688 infer_calls=5688 feedback_calls=5688 infer_wait=19.980s infer_obs_pack=0.239s env_step=38.507s feedback_total=0.910s feedback_obs_pack=0.332s feedback_info_pack=0.018s effective_fps=95.38 +2026-09-25 19:10:51.218 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=8404 infer_calls=8404 feedback_calls=8404 infer_wait=30.359s infer_obs_pack=0.427s env_step=57.280s feedback_total=1.387s feedback_obs_pack=0.501s feedback_info_pack=0.027s effective_fps=93.95 +2026-09-25 19:11:21.223 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=11073 infer_calls=11073 feedback_calls=11073 infer_wait=40.546s infer_obs_pack=0.553s env_step=76.309s feedback_total=1.861s feedback_obs_pack=0.671s feedback_info_pack=0.036s effective_fps=92.84 +2026-09-25 19:11:51.225 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=13728 infer_calls=13728 feedback_calls=13728 infer_wait=51.090s infer_obs_pack=0.669s env_step=94.996s feedback_total=2.327s feedback_obs_pack=0.839s feedback_info_pack=0.047s effective_fps=92.08 +2026-09-25 19:12:21.228 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=16436 infer_calls=16436 feedback_calls=16436 infer_wait=61.569s infer_obs_pack=0.785s env_step=113.744s feedback_total=2.799s feedback_obs_pack=1.004s feedback_info_pack=0.056s effective_fps=91.87 +2026-09-25 19:12:51.232 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=19202 infer_calls=19202 feedback_calls=19202 infer_wait=71.639s infer_obs_pack=0.906s env_step=132.890s feedback_total=3.271s feedback_obs_pack=1.172s feedback_info_pack=0.065s effective_fps=92.00 +2026-09-25 19:13:21.241 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=21904 infer_calls=21904 feedback_calls=21904 infer_wait=82.087s infer_obs_pack=1.024s env_step=151.676s feedback_total=3.741s feedback_obs_pack=1.340s feedback_info_pack=0.074s effective_fps=91.83 +2026-09-25 19:13:51.253 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=24589 infer_calls=24589 feedback_calls=24589 infer_wait=92.500s infer_obs_pack=1.141s env_step=170.503s feedback_total=4.204s feedback_obs_pack=1.503s feedback_info_pack=0.083s effective_fps=91.63 +2026-09-25 19:14:21.253 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=27358 infer_calls=27358 feedback_calls=27358 infer_wait=102.647s infer_obs_pack=1.261s env_step=189.568s feedback_total=4.673s feedback_obs_pack=1.674s feedback_info_pack=0.094s effective_fps=91.76 +2026-09-25 19:14:51.259 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=30038 infer_calls=30038 feedback_calls=30038 infer_wait=113.083s infer_obs_pack=1.378s env_step=208.358s feedback_total=5.148s feedback_obs_pack=1.853s feedback_info_pack=0.103s effective_fps=91.59 +2026-09-25 19:15:21.346 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=32770 infer_calls=32770 feedback_calls=32770 infer_wait=123.501s infer_obs_pack=1.495s env_step=227.253s feedback_total=5.615s feedback_obs_pack=2.022s feedback_info_pack=0.112s effective_fps=91.57 +2026-09-25 19:15:51.347 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=35520 infer_calls=35520 feedback_calls=35520 infer_wait=133.615s infer_obs_pack=1.613s env_step=246.358s feedback_total=6.086s feedback_obs_pack=2.191s feedback_info_pack=0.121s effective_fps=91.62 +2026-09-25 19:16:21.355 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=38214 infer_calls=38214 feedback_calls=38214 infer_wait=144.088s infer_obs_pack=1.729s env_step=265.112s feedback_total=6.559s feedback_obs_pack=2.358s feedback_info_pack=0.130s effective_fps=91.53 +2026-09-25 19:16:51.670 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=40962 infer_calls=40962 feedback_calls=40962 infer_wait=154.619s infer_obs_pack=1.848s env_step=284.110s feedback_total=7.030s feedback_obs_pack=2.532s feedback_info_pack=0.139s effective_fps=91.51 +2026-09-25 19:17:21.678 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=43689 infer_calls=43689 feedback_calls=43689 infer_wait=164.660s infer_obs_pack=1.968s env_step=303.291s feedback_total=7.508s feedback_obs_pack=2.705s feedback_info_pack=0.148s effective_fps=91.51 +2026-09-25 19:17:51.686 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=46391 infer_calls=46391 feedback_calls=46391 infer_wait=174.927s infer_obs_pack=2.086s env_step=322.254s feedback_total=7.980s feedback_obs_pack=2.877s feedback_info_pack=0.158s effective_fps=91.46 +2026-09-25 19:18:21.694 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=49091 infer_calls=49091 feedback_calls=49091 infer_wait=185.171s infer_obs_pack=2.206s env_step=341.219s feedback_total=8.465s feedback_obs_pack=3.050s feedback_info_pack=0.167s effective_fps=91.41 +2026-09-25 19:18:51.694 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=51796 infer_calls=51796 feedback_calls=51796 infer_wait=195.583s infer_obs_pack=2.323s env_step=360.030s feedback_total=8.935s feedback_obs_pack=3.220s feedback_info_pack=0.177s effective_fps=91.37 +2026-09-25 19:19:21.699 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=54517 infer_calls=54517 feedback_calls=54517 infer_wait=206.038s infer_obs_pack=2.442s env_step=378.810s feedback_total=9.396s feedback_obs_pack=3.385s feedback_info_pack=0.186s effective_fps=91.37 +2026-09-25 19:19:51.703 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=57223 infer_calls=57223 feedback_calls=57223 infer_wait=216.151s infer_obs_pack=2.564s env_step=397.912s feedback_total=9.874s feedback_obs_pack=3.560s feedback_info_pack=0.197s effective_fps=91.34 +2026-09-25 19:20:21.712 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=59940 infer_calls=59940 feedback_calls=59940 infer_wait=226.539s infer_obs_pack=2.686s env_step=416.752s feedback_total=10.344s feedback_obs_pack=3.728s feedback_info_pack=0.206s effective_fps=91.33 +2026-09-25 19:20:51.715 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=62653 infer_calls=62653 feedback_calls=62653 infer_wait=236.991s infer_obs_pack=2.803s env_step=435.513s feedback_total=10.828s feedback_obs_pack=3.898s feedback_info_pack=0.215s effective_fps=91.31 +2026-09-25 19:21:21.716 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=65381 infer_calls=65381 feedback_calls=65381 infer_wait=247.147s infer_obs_pack=2.924s env_step=454.572s feedback_total=11.298s feedback_obs_pack=4.066s feedback_info_pack=0.224s effective_fps=91.32 +2026-09-25 19:21:51.721 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=68014 infer_calls=68014 feedback_calls=68014 infer_wait=257.522s infer_obs_pack=3.042s env_step=473.428s feedback_total=11.762s feedback_obs_pack=4.231s feedback_info_pack=0.233s effective_fps=91.20 +2026-09-25 19:22:21.726 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=70603 infer_calls=70603 feedback_calls=70603 infer_wait=268.044s infer_obs_pack=3.156s env_step=492.122s feedback_total=12.245s feedback_obs_pack=4.409s feedback_info_pack=0.242s effective_fps=91.03 +2026-09-25 19:22:51.734 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=73336 infer_calls=73336 feedback_calls=73336 infer_wait=278.107s infer_obs_pack=3.278s env_step=511.284s feedback_total=12.716s feedback_obs_pack=4.584s feedback_info_pack=0.251s effective_fps=91.06 +2026-09-25 19:23:21.740 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=76214 infer_calls=76214 feedback_calls=76214 infer_wait=287.839s infer_obs_pack=3.399s env_step=530.749s feedback_total=13.203s feedback_obs_pack=4.761s feedback_info_pack=0.261s effective_fps=91.25 +2026-09-25 19:23:51.743 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=79199 infer_calls=79199 feedback_calls=79199 infer_wait=297.335s infer_obs_pack=3.525s env_step=550.439s feedback_total=13.692s feedback_obs_pack=4.935s feedback_info_pack=0.270s effective_fps=91.56 +2026-09-25 19:24:18.517 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=81921 infer_calls=81921 feedback_calls=81921 infer_wait=305.284s infer_obs_pack=3.639s env_step=568.087s feedback_total=14.122s feedback_obs_pack=5.091s feedback_info_pack=0.279s effective_fps=91.93 +2026-09-25 19:24:18.517 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 204/327680 episodes. +Created environment with name NutAssemblySquare +Action size is 7 +Exception ignored in: +Traceback (most recent call last): + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/utils/binding_utils.py", line 199, in __del__ + self.gl_ctx.free() + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/renderers/context/egl_context.py", line 149, in free + EGL.eglMakeCurrent(EGL_DISPLAY, EGL.EGL_NO_SURFACE, EGL.EGL_NO_SURFACE, EGL.EGL_NO_CONTEXT) + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/OpenGL/error.py", line 228, in glCheckError + raise GLError( +OpenGL.error.GLError: GLError( + err = 12289, + baseOperation = eglMakeCurrent, + cArguments = ( + , + , + , + , + ), + result = 0 +) +Exception ignored in: +Traceback (most recent call last): + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/renderers/context/egl_context.py", line 155, in __del__ + self.free() + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/renderers/context/egl_context.py", line 149, in free + EGL.eglMakeCurrent(EGL_DISPLAY, EGL.EGL_NO_SURFACE, EGL.EGL_NO_SURFACE, EGL.EGL_NO_CONTEXT) + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/OpenGL/error.py", line 228, in glCheckError + raise GLError( +OpenGL.error.GLError: GLError( + err = 12289, + baseOperation = eglMakeCurrent, + cArguments = ( + , + , + , + , + ), + result = 0 +) diff --git a/experiments/e27-robomimic/results/fpodppo-rm/client-seed2.log b/experiments/e27-robomimic/results/fpodppo-rm/client-seed2.log new file mode 100644 index 0000000..0e28813 --- /dev/null +++ b/experiments/e27-robomimic/results/fpodppo-rm/client-seed2.log @@ -0,0 +1,81 @@ +2026-09-25 19:09:24.457 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 19:09:24.457 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 19:09:24.457 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 19:09:24.457 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +[robosuite WARNING] No private macro file found! (macros.py:53) +[robosuite WARNING] It is recommended to use a private macro file (macros.py:54) +[robosuite WARNING] To setup, run: python /home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/scripts/setup_macros.py (macros.py:55) +2026-09-25 19:09:25.140 | INFO | __main__:main:83 - Starting env client exp_name=robomimic-v1-nenv1-20260925-190925-48d02422 output_dir=runs/robomimic-v1-nenv1-20260925-190925-48d02422 recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 19:09:26.171 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9412... +2026-09-25 19:09:26.174 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'FPOPolicy', 'action_dim': 7, 'action_horizon': 1} +2026-09-25 19:09:56.190 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=2972 infer_calls=2972 feedback_calls=2972 infer_wait=9.740s infer_obs_pack=0.128s env_step=19.467s feedback_total=0.470s feedback_obs_pack=0.171s feedback_info_pack=0.009s effective_fps=99.71 +2026-09-25 19:10:26.192 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=5688 infer_calls=5688 feedback_calls=5688 infer_wait=20.223s infer_obs_pack=0.245s env_step=38.199s feedback_total=0.944s feedback_obs_pack=0.342s feedback_info_pack=0.017s effective_fps=95.42 +2026-09-25 19:10:56.199 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=8371 infer_calls=8371 feedback_calls=8371 infer_wait=30.707s infer_obs_pack=0.436s env_step=56.880s feedback_total=1.407s feedback_obs_pack=0.511s feedback_info_pack=0.026s effective_fps=93.60 +2026-09-25 19:11:26.207 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=11099 infer_calls=11099 feedback_calls=11099 infer_wait=40.906s infer_obs_pack=0.556s env_step=75.890s feedback_total=1.894s feedback_obs_pack=0.681s feedback_info_pack=0.035s effective_fps=93.08 +2026-09-25 19:11:56.208 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=13800 infer_calls=13800 feedback_calls=13800 infer_wait=51.476s infer_obs_pack=0.670s env_step=94.548s feedback_total=2.361s feedback_obs_pack=0.849s feedback_info_pack=0.045s effective_fps=92.58 +2026-09-25 19:12:26.215 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=16502 infer_calls=16502 feedback_calls=16502 infer_wait=61.948s infer_obs_pack=0.790s env_step=113.290s feedback_total=2.837s feedback_obs_pack=1.023s feedback_info_pack=0.054s effective_fps=92.26 +2026-09-25 19:12:56.223 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=19260 infer_calls=19260 feedback_calls=19260 infer_wait=72.045s infer_obs_pack=0.914s env_step=132.408s feedback_total=3.313s feedback_obs_pack=1.195s feedback_info_pack=0.063s effective_fps=92.29 +2026-09-25 19:13:26.225 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=21963 infer_calls=21963 feedback_calls=21963 infer_wait=82.477s infer_obs_pack=1.032s env_step=151.199s feedback_total=3.787s feedback_obs_pack=1.369s feedback_info_pack=0.072s effective_fps=92.09 +2026-09-25 19:13:56.233 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=24666 infer_calls=24666 feedback_calls=24666 infer_wait=92.977s infer_obs_pack=1.149s env_step=169.926s feedback_total=4.258s feedback_obs_pack=1.539s feedback_info_pack=0.081s effective_fps=91.93 +2026-09-25 19:14:26.235 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=27466 infer_calls=27466 feedback_calls=27466 infer_wait=102.991s infer_obs_pack=1.270s env_step=189.133s feedback_total=4.722s feedback_obs_pack=1.707s feedback_info_pack=0.090s effective_fps=92.13 +2026-09-25 19:14:56.241 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=30151 infer_calls=30151 feedback_calls=30151 infer_wait=113.428s infer_obs_pack=1.387s env_step=207.927s feedback_total=5.193s feedback_obs_pack=1.874s feedback_info_pack=0.099s effective_fps=91.94 +2026-09-25 19:15:26.259 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=32877 infer_calls=32877 feedback_calls=32877 infer_wait=123.729s infer_obs_pack=1.507s env_step=226.853s feedback_total=5.670s feedback_obs_pack=2.041s feedback_info_pack=0.108s effective_fps=91.90 +2026-09-25 19:15:56.265 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=35637 infer_calls=35637 feedback_calls=35637 infer_wait=133.850s infer_obs_pack=1.626s env_step=245.957s feedback_total=6.141s feedback_obs_pack=2.210s feedback_info_pack=0.117s effective_fps=91.95 +2026-09-25 19:16:26.268 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=38308 infer_calls=38308 feedback_calls=38308 infer_wait=144.367s infer_obs_pack=1.742s env_step=264.662s feedback_total=6.612s feedback_obs_pack=2.379s feedback_info_pack=0.127s effective_fps=91.78 +2026-09-25 19:16:56.271 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=41017 infer_calls=41017 feedback_calls=41017 infer_wait=154.750s infer_obs_pack=1.859s env_step=283.500s feedback_total=7.085s feedback_obs_pack=2.547s feedback_info_pack=0.137s effective_fps=91.72 +2026-09-25 19:17:26.274 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=43744 infer_calls=43744 feedback_calls=43744 infer_wait=165.067s infer_obs_pack=1.978s env_step=302.397s feedback_total=7.567s feedback_obs_pack=2.720s feedback_info_pack=0.146s effective_fps=91.70 +2026-09-25 19:17:56.283 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=46480 infer_calls=46480 feedback_calls=46480 infer_wait=175.520s infer_obs_pack=2.097s env_step=321.171s feedback_total=8.042s feedback_obs_pack=2.893s feedback_info_pack=0.155s effective_fps=91.71 +2026-09-25 19:18:26.288 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=49160 infer_calls=49160 feedback_calls=49160 infer_wait=185.930s infer_obs_pack=2.218s env_step=339.979s feedback_total=8.522s feedback_obs_pack=3.073s feedback_info_pack=0.164s effective_fps=91.61 +2026-09-25 19:18:56.289 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=51923 infer_calls=51923 feedback_calls=51923 infer_wait=196.034s infer_obs_pack=2.342s env_step=359.072s feedback_total=9.009s feedback_obs_pack=3.246s feedback_info_pack=0.174s effective_fps=91.66 +2026-09-25 19:19:26.296 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=54693 infer_calls=54693 feedback_calls=54693 infer_wait=206.622s infer_obs_pack=2.461s env_step=377.702s feedback_total=9.483s feedback_obs_pack=3.420s feedback_info_pack=0.183s effective_fps=91.73 +2026-09-25 19:19:56.297 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=57422 infer_calls=57422 feedback_calls=57422 infer_wait=217.116s infer_obs_pack=2.579s env_step=396.421s feedback_total=9.965s feedback_obs_pack=3.592s feedback_info_pack=0.194s effective_fps=91.72 +2026-09-25 19:20:26.302 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=60157 infer_calls=60157 feedback_calls=60157 infer_wait=227.402s infer_obs_pack=2.696s env_step=415.350s feedback_total=10.447s feedback_obs_pack=3.763s feedback_info_pack=0.204s effective_fps=91.72 +2026-09-25 19:20:56.305 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=62890 infer_calls=62890 feedback_calls=62890 infer_wait=237.870s infer_obs_pack=2.816s env_step=434.102s feedback_total=10.916s feedback_obs_pack=3.934s feedback_info_pack=0.213s effective_fps=91.72 +2026-09-25 19:21:26.315 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=65547 infer_calls=65547 feedback_calls=65547 infer_wait=248.503s infer_obs_pack=2.932s env_step=452.713s feedback_total=11.380s feedback_obs_pack=4.100s feedback_info_pack=0.222s effective_fps=91.61 +2026-09-25 19:21:56.316 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=68245 infer_calls=68245 feedback_calls=68245 infer_wait=258.645s infer_obs_pack=3.052s env_step=471.788s feedback_total=11.850s feedback_obs_pack=4.269s feedback_info_pack=0.231s effective_fps=91.56 +2026-09-25 19:22:26.320 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=70910 infer_calls=70910 feedback_calls=70910 infer_wait=269.191s infer_obs_pack=3.171s env_step=490.448s feedback_total=12.338s feedback_obs_pack=4.444s feedback_info_pack=0.241s effective_fps=91.48 +2026-09-25 19:22:56.322 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=73625 infer_calls=73625 feedback_calls=73625 infer_wait=279.231s infer_obs_pack=3.294s env_step=509.614s feedback_total=12.820s feedback_obs_pack=4.618s feedback_info_pack=0.250s effective_fps=91.46 +2026-09-25 19:23:26.325 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=76602 infer_calls=76602 feedback_calls=76602 infer_wait=288.934s infer_obs_pack=3.421s env_step=529.093s feedback_total=13.309s feedback_obs_pack=4.797s feedback_info_pack=0.259s effective_fps=91.77 +2026-09-25 19:23:56.330 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=79624 infer_calls=79624 feedback_calls=79624 infer_wait=298.482s infer_obs_pack=3.548s env_step=548.722s feedback_total=13.800s feedback_obs_pack=4.975s feedback_info_pack=0.269s effective_fps=92.10 +2026-09-25 19:24:18.882 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=81921 infer_calls=81921 feedback_calls=81921 infer_wait=305.199s infer_obs_pack=3.642s env_step=563.479s feedback_total=14.145s feedback_obs_pack=5.101s feedback_info_pack=0.275s effective_fps=92.41 +2026-09-25 19:24:18.882 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 204/327680 episodes. +Created environment with name NutAssemblySquare +Action size is 7 +Exception ignored in: +Traceback (most recent call last): + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/utils/binding_utils.py", line 199, in __del__ + self.gl_ctx.free() + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/renderers/context/egl_context.py", line 149, in free + EGL.eglMakeCurrent(EGL_DISPLAY, EGL.EGL_NO_SURFACE, EGL.EGL_NO_SURFACE, EGL.EGL_NO_CONTEXT) + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/OpenGL/error.py", line 228, in glCheckError + raise GLError( +OpenGL.error.GLError: GLError( + err = 12289, + baseOperation = eglMakeCurrent, + cArguments = ( + , + , + , + , + ), + result = 0 +) +Exception ignored in: +Traceback (most recent call last): + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/renderers/context/egl_context.py", line 155, in __del__ + self.free() + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/robosuite/renderers/context/egl_context.py", line 149, in free + EGL.eglMakeCurrent(EGL_DISPLAY, EGL.EGL_NO_SURFACE, EGL.EGL_NO_SURFACE, EGL.EGL_NO_CONTEXT) + File "/home/guangzhao/zuogou/plugrl/plugrl-env-client/.venv-robomimic/lib/python3.11/site-packages/OpenGL/error.py", line 228, in glCheckError + raise GLError( +OpenGL.error.GLError: GLError( + err = 12289, + baseOperation = eglMakeCurrent, + cArguments = ( + , + , + , + , + ), + result = 0 +) diff --git a/experiments/e27-robomimic/results/fpodppo-rm/server-seed0.log b/experiments/e27-robomimic/results/fpodppo-rm/server-seed0.log new file mode 100644 index 0000000..53a513b --- /dev/null +++ b/experiments/e27-robomimic/results/fpodppo-rm/server-seed0.log @@ -0,0 +1,21 @@ +19:09:14|INFO|plugrl_server version: 0.1.0 +19:09:14|INFO|Algorithm: dppo, Config: DPPOAlgoConfig(global_steps=81920, gamma=0.99, gamma_denoising=1.0, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=None, critic_lr_scheduler=None, buffer_size=4096, gae_lambda=0.95, update_epochs=4, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.001, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.01, sampling_noise_level=0.01, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=False, batch_size=256, train_itrs=20, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +19:09:14|INFO|Policy: fpo-policy, Config: FPOPolicyConfig(algo='dppo', device=device(type='cpu'), obs_dim=23, action_dim=7, flow_steps=10, timestep_embed_dim=8, action_horizon=1, feather_std=0.0, policy_mlp_output_scale=0.25, normalize_observations=True, hidden_dims=(32, 32, 32, 32), value_hidden_dims=(256, 256, 256, 256, 256), state_keys=('robot0_eef_pos', 'robot0_eef_quat', 'robot0_gripper_qpos', 'object')) +19:09:14|INFO|Seeded python, numpy and torch with 0. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +19:09:14|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/e27/experiments/e27-robomimic/results/fpodppo-rm/dppo/fpo-policy/fpodppo-rm-seed0 +19:09:14|INFO|Policy created... +19:09:14|INFO|Initialized RolloutBuffer buffer_size=4096 action_shape=(4096, 10, 1, 7) value_shape=(4096,) +19:09:14|INFO|Algorithm created: + +19:09:14|INFO|Agent Server is listening on 0.0.0.0:9410 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +19:24:13|INFO|Checkpoint saved at step 81921 to /home/guangzhao/zuogou/plugrl/e27/experiments/e27-robomimic/results/fpodppo-rm/dppo/fpo-policy/fpodppo-rm-seed0/81921 +19:24:13|INFO|Stopping server as the algorithm signaled to stop. +19:24:13|INFO|Shutdown started: aborting pending infer requests. +19:24:13|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +19:24:13|INFO|Shutdown closing 1 websocket connection(s). +19:24:13|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 52184). +19:24:13|INFO|WebSocket server closed. +19:24:13|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e27-robomimic/results/fpodppo-rm/server-seed1.log b/experiments/e27-robomimic/results/fpodppo-rm/server-seed1.log new file mode 100644 index 0000000..f0cf757 --- /dev/null +++ b/experiments/e27-robomimic/results/fpodppo-rm/server-seed1.log @@ -0,0 +1,21 @@ +19:09:19|INFO|plugrl_server version: 0.1.0 +19:09:19|INFO|Algorithm: dppo, Config: DPPOAlgoConfig(global_steps=81920, gamma=0.99, gamma_denoising=1.0, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=None, critic_lr_scheduler=None, buffer_size=4096, gae_lambda=0.95, update_epochs=4, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.001, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.01, sampling_noise_level=0.01, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=False, batch_size=256, train_itrs=20, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +19:09:19|INFO|Policy: fpo-policy, Config: FPOPolicyConfig(algo='dppo', device=device(type='cpu'), obs_dim=23, action_dim=7, flow_steps=10, timestep_embed_dim=8, action_horizon=1, feather_std=0.0, policy_mlp_output_scale=0.25, normalize_observations=True, hidden_dims=(32, 32, 32, 32), value_hidden_dims=(256, 256, 256, 256, 256), state_keys=('robot0_eef_pos', 'robot0_eef_quat', 'robot0_gripper_qpos', 'object')) +19:09:19|INFO|Seeded python, numpy and torch with 1. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +19:09:19|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/e27/experiments/e27-robomimic/results/fpodppo-rm/dppo/fpo-policy/fpodppo-rm-seed1 +19:09:19|INFO|Policy created... +19:09:19|INFO|Initialized RolloutBuffer buffer_size=4096 action_shape=(4096, 10, 1, 7) value_shape=(4096,) +19:09:19|INFO|Algorithm created: + +19:09:19|INFO|Agent Server is listening on 0.0.0.0:9411 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +19:24:18|INFO|Checkpoint saved at step 81921 to /home/guangzhao/zuogou/plugrl/e27/experiments/e27-robomimic/results/fpodppo-rm/dppo/fpo-policy/fpodppo-rm-seed1/81921 +19:24:18|INFO|Stopping server as the algorithm signaled to stop. +19:24:18|INFO|Shutdown started: aborting pending infer requests. +19:24:18|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +19:24:18|INFO|Shutdown closing 1 websocket connection(s). +19:24:18|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 42372). +19:24:18|INFO|WebSocket server closed. +19:24:18|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e27-robomimic/results/fpodppo-rm/server-seed2.log b/experiments/e27-robomimic/results/fpodppo-rm/server-seed2.log new file mode 100644 index 0000000..1e6b267 --- /dev/null +++ b/experiments/e27-robomimic/results/fpodppo-rm/server-seed2.log @@ -0,0 +1,21 @@ +19:09:24|INFO|plugrl_server version: 0.1.0 +19:09:24|INFO|Algorithm: dppo, Config: DPPOAlgoConfig(global_steps=81920, gamma=0.99, gamma_denoising=1.0, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=None, critic_lr_scheduler=None, buffer_size=4096, gae_lambda=0.95, update_epochs=4, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.001, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.01, sampling_noise_level=0.01, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=False, batch_size=256, train_itrs=20, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +19:09:24|INFO|Policy: fpo-policy, Config: FPOPolicyConfig(algo='dppo', device=device(type='cpu'), obs_dim=23, action_dim=7, flow_steps=10, timestep_embed_dim=8, action_horizon=1, feather_std=0.0, policy_mlp_output_scale=0.25, normalize_observations=True, hidden_dims=(32, 32, 32, 32), value_hidden_dims=(256, 256, 256, 256, 256), state_keys=('robot0_eef_pos', 'robot0_eef_quat', 'robot0_gripper_qpos', 'object')) +19:09:24|INFO|Seeded python, numpy and torch with 2. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +19:09:24|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/e27/experiments/e27-robomimic/results/fpodppo-rm/dppo/fpo-policy/fpodppo-rm-seed2 +19:09:24|INFO|Policy created... +19:09:24|INFO|Initialized RolloutBuffer buffer_size=4096 action_shape=(4096, 10, 1, 7) value_shape=(4096,) +19:09:24|INFO|Algorithm created: + +19:09:24|INFO|Agent Server is listening on 0.0.0.0:9412 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +19:24:18|INFO|Checkpoint saved at step 81921 to /home/guangzhao/zuogou/plugrl/e27/experiments/e27-robomimic/results/fpodppo-rm/dppo/fpo-policy/fpodppo-rm-seed2/81921 +19:24:18|INFO|Stopping server as the algorithm signaled to stop. +19:24:18|INFO|Shutdown started: aborting pending infer requests. +19:24:18|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +19:24:18|INFO|Shutdown closing 1 websocket connection(s). +19:24:18|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 46706). +19:24:18|INFO|WebSocket server closed. +19:24:18|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e27-robomimic/results/pilot.txt b/experiments/e27-robomimic/results/pilot.txt new file mode 100644 index 0000000..c24769c --- /dev/null +++ b/experiments/e27-robomimic/results/pilot.txt @@ -0,0 +1,32 @@ +# E27 pilots, 2026-09-25, guangzhao, before PROTOCOL.md was committed. +# Each: run_cell.sh with SEEDS=0 and one iteration, the three cells at once. + +## Pilot 1, 19:00 - client at plugrl-env-client main (cbed9c4) + +fpo-rm seed 0 finished rc=0, env_steps 4097 +fpodppo-rm seed 0 finished rc=0, env_steps 4097 +dppo-rm seed 0 finished rc=0, env_steps 4100 + +Every client log: "Server signalled the end of the run; collection stopped +after 0/... episodes" - no episode ended in 4,096 steps. square-img.json sets +ignore_done: true and horizon: null; the client took `terminated` from +robomimic's done, which is never true, and hard-coded `truncated` to False. +Fixed in plugrl-env-client#9 (931ab56). + +Every client log also ends with, after the run: + + Exception ignored in: + File ".../robosuite/renderers/context/egl_context.py", line 149, in free + EGL.eglMakeCurrent(EGL_DISPLAY, EGL.EGL_NO_SURFACE, ...) + OpenGL.error.GLError: GLError(err = 12289, baseOperation = eglMakeCurrent, ...) + +robosuite freeing its EGL context at interpreter shutdown; nothing the run +did depends on it. + +## Pilot 2, 19:06 - client at 931ab56 + +fpo-rm seed 0 finished rc=0, "collection stopped after 10/16384 episodes" +fpodppo-rm seed 0 finished rc=0, "collection stopped after 10/16384 episodes" +dppo-rm seed 0 finished rc=0, "collection stopped after 10/4096 episodes" + +Ten episodes of 400 steps in about 4,096: the horizon now ends them. diff --git a/experiments/e27-robomimic/results/run.out b/experiments/e27-robomimic/results/run.out new file mode 100644 index 0000000..c85b2fa --- /dev/null +++ b/experiments/e27-robomimic/results/run.out @@ -0,0 +1,6 @@ +start 2026-09-25 19:08:52 +end 2026-09-25 19:26:16 +fpo-rm failed seeds: 0 +fpodppo-rm failed seeds: 0 +dppo-rm failed seeds: 0 +E27_DONE diff --git a/experiments/e27-robomimic/results/verdicts.txt b/experiments/e27-robomimic/results/verdicts.txt new file mode 100644 index 0000000..1d9eb49 --- /dev/null +++ b/experiments/e27-robomimic/results/verdicts.txt @@ -0,0 +1,28 @@ +P1 every cell runs end to end + fpo-rm fpo-policy fpo robomimic-square HOLDS (14 min) + fpodppo-rm fpo-policy dppo robomimic-square HOLDS (15 min) + dppo-rm dppo-policy dppo robomimic-square HOLDS (17 min) + +reported: success rate, first iteration and max over all twenty + fpo-rm seed 0 success 0.000, best 0.000 + fpo-rm seed 1 success 0.000, best 0.000 + fpo-rm seed 2 success 0.000, best 0.000 + fpodppo-rm seed 0 success 0.000, best 0.000 + fpodppo-rm seed 1 success 0.000, best 0.000 + fpodppo-rm seed 2 success 0.000, best 0.000 + dppo-rm seed 0 success 0.000, best 0.000 + dppo-rm seed 1 success 0.000, best 0.000 + dppo-rm seed 2 success 0.000, best 0.000 + +reported: first iteration and mean of the last ten, return and episode length + fpo-rm seed 0 return 0.0 -> 0.0 length 400.0 -> 400.0 + fpo-rm seed 1 return 0.0 -> 0.0 length 400.0 -> 400.0 + fpo-rm seed 2 return 0.0 -> 0.0 length 400.0 -> 400.0 + fpodppo-rm seed 0 return 0.0 -> 0.0 length 400.0 -> 400.0 + fpodppo-rm seed 1 return 0.0 -> 0.0 length 400.0 -> 400.0 + fpodppo-rm seed 2 return 0.0 -> 0.0 length 400.0 -> 400.0 + dppo-rm seed 0 return 0.0 -> 0.0 length 400.0 -> 400.0 + dppo-rm seed 1 return 0.0 -> 0.0 length 400.0 -> 400.0 + dppo-rm seed 2 return 0.0 -> 0.0 length 400.0 -> 400.0 + +wrote /home/guangzhao/zuogou/plugrl/e27/experiments/e27-robomimic/summary.tsv diff --git a/experiments/e27-robomimic/run.sh b/experiments/e27-robomimic/run.sh new file mode 100644 index 0000000..9d98379 --- /dev/null +++ b/experiments/e27-robomimic/run.sh @@ -0,0 +1,26 @@ +#!/usr/bin/env bash +# E27: the three robomimic cells, concurrently. See PROTOCOL.md. +# +# OMP_NUM_THREADS=1 bash run.sh +set -uo pipefail + +HERE="$(cd "$(dirname "$0")" && pwd)" +R="$HERE/results" +mkdir -p "$R" +CELL="$HERE/run_cell.sh" + +date '+start %F %T' +PORT_BASE=9400 bash "$CELL" fpo-rm fpo-policy default fpo default 20 > "$R/fpo-rm.out" 2>&1 & +A=$! +sleep 20 +PORT_BASE=9410 BATCH=256 bash "$CELL" fpodppo-rm fpo-policy default dppo default 20 > "$R/fpodppo-rm.out" 2>&1 & +B=$! +sleep 20 +PORT_BASE=9420 BUFFER=1024 BATCH=256 bash "$CELL" dppo-rm dppo-policy default dppo default 20 > "$R/dppo-rm.out" 2>&1 & +C=$! +wait "$A" "$B" "$C" +date '+end %F %T' +for c in fpo-rm fpodppo-rm dppo-rm; do + printf '%-12s %s\n' "$c" "$(grep -E 'failed seeds' "$R/$c.out" 2>/dev/null)" +done +echo E27_DONE diff --git a/experiments/e27-robomimic/run_cell.sh b/experiments/e27-robomimic/run_cell.sh new file mode 100644 index 0000000..095605f --- /dev/null +++ b/experiments/e27-robomimic/run_cell.sh @@ -0,0 +1,107 @@ +#!/usr/bin/env bash +# One cell of E27: a policy and an algorithm on robomimic's square task (low +# dimensional state, NutAssemblySquare), SEEDS concurrently. E24's run_cell.sh, +# with the environment changed: +# +# bash run_cell.sh CELL POLICY POLICY_VARIANT ALGO ALGO_VARIANT ITERS [OUT_DIR] +# +# * The client is robomimic-v1 on `square-img`, from its own venv +# (.venv-robomimic: robomimic v0.4.0, robosuite 1.4.1, mujoco 2.3.7), with +# EGL rendering. Its agentview image is 84x84: no policy here reads images. +# * The client is not seeded. robomimic-v1 refuses a seed - it cannot reach +# the randomness of the robosuite simulation underneath - so SEED varies +# the server, which initialises the policy, and not the environment. +# * fpo-policy reads the four low-dimensional keys dppo-policy's square +# config reads - 23 values - through `--policy.state-keys` (#61); +# dppo-policy reads them through that config. +# +# BUFFER (default 4096) and BATCH (the algorithm variant's default when unset) +# come from the environment. +set -uo pipefail + +CELL="$1" +POLICY="$2" +POLICY_VARIANT="$3" +ALGO="$4" +ALGO_VARIANT="$5" +ITERS="$6" +HERE="$(cd "$(dirname "$0")" && pwd)" +OUT="${7:-$HERE/results/$CELL}" +case "$OUT" in /*) ;; *) OUT="$(pwd)/$OUT" ;; esac + +SEEDS="${SEEDS:-0 1 2}" +PORT_BASE="${PORT_BASE:-9400}" +BUFFER="${BUFFER:-4096}" +SERVER_DIR="$(cd "$HERE/../.." && pwd)" +CLIENT_DIR="${PLUGRL_ENV_CLIENT:-$SERVER_DIR/../plugrl-env-client}" +SERVER_PY="${PLUGRL_SERVER_PYTHON:-$SERVER_DIR/../plugrl-server/.venv/bin/python}" +CLIENT_PY="${PLUGRL_CLIENT_PYTHON:-$CLIENT_DIR/.venv-robomimic/bin/python}" +KEYS=(robot0_eef_pos robot0_eef_quat robot0_gripper_qpos object) +mkdir -p "$OUT" + +POLICY_ARGS=() +REPLAN=1 +if [ "$POLICY" = "fpo-policy" ]; then + POLICY_ARGS=(--policy.obs-dim 23 --policy.action-dim 7 --policy.state-keys "${KEYS[@]}") +elif [ "$POLICY" = "dppo-policy" ]; then + POLICY_ARGS=(--policy.env-type robomimic --policy.env-name square) + REPLAN=4 +fi + +ALGO_ARGS=(--algo.buffer-size "$BUFFER") +[ -n "${BATCH:-}" ] && ALGO_ARGS+=(--algo.batch-size "$BATCH") +if [ "$ALGO" = "fpo" ]; then + ALGO_ARGS+=(--algo.global-steps $((BUFFER * ITERS))) +else + ALGO_ARGS+=(--algo.train-itrs "$ITERS") +fi + +echo "cell: $CELL = $POLICY/$POLICY_VARIANT x $ALGO/$ALGO_VARIANT x robomimic square" +echo "iters: $ITERS x $BUFFER batch: ${BATCH:-variant default} replan: $REPLAN seeds: $SEEDS" +echo "server: $SERVER_PY (src $SERVER_DIR/src)" +echo "client: $CLIENT_PY" +echo "out: $OUT" +date '+start %F %T' + +run_seed() { + local seed="$1" port=$((PORT_BASE + $1)) + (cd "$SERVER_DIR" && PYTHONPATH="$SERVER_DIR/src" "$SERVER_PY" -m plugrl_server.cli \ + "$POLICY" "$POLICY_VARIANT" "$ALGO" "$ALGO_VARIANT" \ + --port "$port" --seed "$seed" --policy.device cpu \ + "${POLICY_ARGS[@]}" "${ALGO_ARGS[@]}" \ + --algo.save-interval 20 \ + --no-show-progress-bar --no-show-metric-table \ + --checkpoint-base-dir "$OUT" --exp-name "$CELL-seed$seed" --overwrite \ + > "$OUT/server-seed$seed.log" 2>&1 &) + + local waited=0 + until grep -q "is listening on" "$OUT/server-seed$seed.log" 2>/dev/null; do + sleep 1 + waited=$((waited + 1)) + if [ "$waited" -ge 300 ]; then + echo "seed $seed: server never listened; see $OUT/server-seed$seed.log" >&2 + return 1 + fi + done + + (cd "$CLIENT_DIR" && MUJOCO_GL=egl PYOPENGL_PLATFORM=egl "$CLIENT_PY" -m plugrl_env_client.cli robomimic-v1 \ + --server-port "$port" --server-host 127.0.0.1 \ + --num-envs 1 --num-episodes $((BUFFER * ITERS * 4)) \ + --env.name square-img --env.agentview-image-size 84 84 \ + --runner.replan-steps "$REPLAN" \ + > "$OUT/client-seed$seed.log" 2>&1) + echo "seed $seed finished rc=$? at $(date '+%T')" +} + +PIDS=() +for seed in $SEEDS; do + run_seed "$seed" & + PIDS+=($!) + sleep 5 +done +FAILED=0 +for pid in "${PIDS[@]}"; do wait "$pid" || FAILED=$((FAILED + 1)); done + +date '+end %F %T' +echo "failed seeds: $FAILED" +echo "CELL_DONE $CELL" diff --git a/experiments/e27-robomimic/summarise.py b/experiments/e27-robomimic/summarise.py new file mode 100644 index 0000000..0535717 --- /dev/null +++ b/experiments/e27-robomimic/summarise.py @@ -0,0 +1,167 @@ +"""Read E27's verdicts from each cell's logs and tensorboards. + + python summarise.py + +P1 cell by cell, then the descriptive figures - PROTOCOL.md's reading +order. Writes `summary.tsv`, one row per cell and seed, in E24's columns, so +the coverage matrix reads both the same way. E24's summarise.py with E27's +cells, no P2, and success rates reported. +""" + +import datetime +import pathlib +import re +import sys + +from tensorboard.backend.event_processing.event_accumulator import EventAccumulator + +HERE = pathlib.Path(__file__).resolve().parent +RESULTS = HERE / "results" +SEEDS = (0, 1, 2) + +# cell: (policy, algorithm, task, iterations, claim) +CELLS = { + "fpo-rm": ("fpo-policy", "fpo", "robomimic-square", 20, "runs"), + "fpodppo-rm": ("fpo-policy", "dppo", "robomimic-square", 20, "runs"), + "dppo-rm": ("dppo-policy", "dppo", "robomimic-square", 20, "runs"), +} + + +def run_dir(cell: str, seed: int) -> pathlib.Path | None: + hits = sorted((RESULTS / cell).glob(f"*/*/{cell}-seed{seed}")) + return hits[0] if hits else None + + +def curve(run: pathlib.Path, tag: str) -> list[float]: + events = sorted((run / "tensorboard").glob("events.*")) + if not events: + return [] + acc = EventAccumulator(str(events[0]), size_guidance={"scalars": 0}) + acc.Reload() + if tag not in acc.Tags()["scalars"]: + return [] + return [e.value for e in acc.Scalars(tag)] + + +def wall_minutes(out: str) -> float | None: + stamps = dict(re.findall(r"^(start|end)\s+(\S+ \S+)$", out, flags=re.M)) + if "start" not in stamps or "end" not in stamps: + return None + fmt = "%Y-%m-%d %H:%M:%S" + delta = datetime.datetime.strptime(stamps["end"], fmt) - datetime.datetime.strptime( + stamps["start"], fmt + ) + return delta.total_seconds() / 60 + + +def main() -> int: + rows = [] + print("P1 every cell runs end to end") + runs_ok = {} + for cell, (policy, algo, task, iters, claim) in CELLS.items(): + out_path = RESULTS / f"{cell}.out" + out = ( + out_path.read_text(encoding="utf-8", errors="replace") + if out_path.exists() + else "" + ) + exits = { + int(s): int(rc) for s, rc in re.findall(r"seed (\d) finished rc=(\d+)", out) + } + ok_cell = True + for seed in SEEDS: + run = run_dir(cell, seed) + reward = curve(run, "rollout/reward") if run else [] + length = curve(run, "rollout/length") if run else [] + saves = [p for p in run.iterdir() if p.name.isdigit()] if run else [] + log_path = RESULTS / cell / f"server-seed{seed}.log" + log = ( + log_path.read_text(encoding="utf-8", errors="replace") + if log_path.exists() + else "" + ) + traceback = "Traceback" in log + ok = ( + len(reward) >= iters + and len(saves) >= iters // 20 + and not traceback + and exits.get(seed) == 0 + ) + ok_cell &= ok + last10 = ( + sum(reward[iters - 10 : iters]) / 10 + if len(reward) >= iters + else float("nan") + ) + last10_len = ( + sum(length[iters - 10 : iters]) / 10 + if len(length) >= iters + else float("nan") + ) + rows.append( + ( + cell, + policy, + algo, + task, + claim, + seed, + len(reward), + ok, + reward[0] if reward else float("nan"), + last10, + length[0] if length else float("nan"), + last10_len, + ) + ) + runs_ok[cell] = ok_cell + minutes = wall_minutes(out) + print( + f" {cell:15s} {policy:12s} {algo:5s} {task:15s} " + f"{'HOLDS' if ok_cell else 'FAILS'}" + + (f" ({minutes:.0f} min)" if minutes is not None else "") + ) + if not ok_cell: + for r in rows: + if r[0] == cell and not r[7]: + print(f" seed {r[5]}: {r[6]} of {iters} iterations logged") + + print("\nreported: success rate, first iteration and max over all twenty") + for cell in CELLS: + for seed in SEEDS: + run = run_dir(cell, seed) + success = curve(run, "rollout/success") if run else [] + if success: + print( + f" {cell:15s} seed {seed} success {success[0]:.3f}, " + f"best {max(success):.3f}" + ) + + print( + "\nreported: first iteration and mean of the last ten, return and episode length" + ) + for r in rows: + print( + f" {r[0]:15s} seed {r[5]} return {r[8]:8.1f} -> {r[9]:8.1f} " + f"length {r[10]:6.1f} -> {r[11]:6.1f}" + ) + + header = "cell\tpolicy\talgorithm\ttask\tclaim\tseed\titerations\truns\tfirst_return\tlast10_return\tfirst_length\tlast10_length" + lines = [header] + [ + "\t".join( + f"{v:.1f}" + if isinstance(v, float) + else str(v).lower() + if isinstance(v, bool) + else str(v) + for v in r + ) + for r in rows + ] + (HERE / "summary.tsv").write_text("\n".join(lines) + "\n", encoding="utf-8") + print(f"\nwrote {HERE / 'summary.tsv'}") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/experiments/e27-robomimic/summary.tsv b/experiments/e27-robomimic/summary.tsv new file mode 100644 index 0000000..19b5577 --- /dev/null +++ b/experiments/e27-robomimic/summary.tsv @@ -0,0 +1,10 @@ +cell policy algorithm task claim seed iterations runs first_return last10_return first_length last10_length +fpo-rm fpo-policy fpo robomimic-square runs 0 20 true 0.0 0.0 400.0 400.0 +fpo-rm fpo-policy fpo robomimic-square runs 1 20 true 0.0 0.0 400.0 400.0 +fpo-rm fpo-policy fpo robomimic-square runs 2 20 true 0.0 0.0 400.0 400.0 +fpodppo-rm fpo-policy dppo robomimic-square runs 0 20 true 0.0 0.0 400.0 400.0 +fpodppo-rm fpo-policy dppo robomimic-square runs 1 20 true 0.0 0.0 400.0 400.0 +fpodppo-rm fpo-policy dppo robomimic-square runs 2 20 true 0.0 0.0 400.0 400.0 +dppo-rm dppo-policy dppo robomimic-square runs 0 20 true 0.0 0.0 400.0 400.0 +dppo-rm dppo-policy dppo robomimic-square runs 1 20 true 0.0 0.0 400.0 400.0 +dppo-rm dppo-policy dppo robomimic-square runs 2 20 true 0.0 0.0 400.0 400.0 diff --git a/src/plugrl_server/policy/fpo/fpo_policy.py b/src/plugrl_server/policy/fpo/fpo_policy.py index b9c1306..509ea19 100644 --- a/src/plugrl_server/policy/fpo/fpo_policy.py +++ b/src/plugrl_server/policy/fpo/fpo_policy.py @@ -88,6 +88,11 @@ class FPOPolicyConfig(BasePolicyGradientFlowPolicyConfig): normalize_observations: bool = True hidden_dims: tuple[int, ...] = (32, 32, 32, 32) value_hidden_dims: tuple[int, ...] = (256, 256, 256, 256, 256) + # The observation's state keys, concatenated in this order into the + # obs_dim-wide vector the networks read. The MuJoCo client sends one key, + # "obs"; the robomimic client sends one per quantity (robot0_eef_pos, + # object, ...). + state_keys: tuple[str, ...] = ("obs",) @register_policy(UID) @@ -173,11 +178,25 @@ def _initialize_x(self, batch_size: int) -> torch.Tensor: ) def extract_model_obs_tensor(self, _obs: dict[str, Any]) -> TorchTree: - return torch.as_tensor( - _obs["states"]["obs"], - dtype=torch.float32, - device=self.device, - ) + states = _obs["states"] + keys = self.config.state_keys + missing = [k for k in keys if k not in states] + if missing: + raise KeyError( + f"state keys {missing} are not in the observation, which has " + f"{sorted(states)}; set FPOPolicyConfig.state_keys" + ) + parts = [ + torch.as_tensor(states[k], dtype=torch.float32, device=self.device) + for k in keys + ] + x = parts[0] if len(parts) == 1 else torch.cat(parts, dim=-1) + if x.shape[-1] != self.config.obs_dim: + raise ValueError( + f"state keys {list(keys)} give {x.shape[-1]} values per " + f"observation, but obs_dim is {self.config.obs_dim}" + ) + return x def build_obs_cache(self, obs: TorchTree) -> Any: assert isinstance(obs, torch.Tensor) diff --git a/tests/test_fpo_state_keys.py b/tests/test_fpo_state_keys.py new file mode 100644 index 0000000..516410d --- /dev/null +++ b/tests/test_fpo_state_keys.py @@ -0,0 +1,67 @@ +"""fpo-policy reads its observation from named state keys, in order. + +It read `states["obs"]` and nothing else, which is what the MuJoCo client +sends. The robomimic client sends one state per quantity - `robot0_eef_pos`, +`object` and the rest - so fpo-policy could not run on robomimic at all. +`state_keys` names the keys to concatenate; the default, `("obs",)`, is the +old behaviour exactly. + +A key the observation lacks, or keys whose widths do not add up to +`obs_dim`, raise with what was there: the alternative is a shape error deep +in the network, or - worse - a policy silently fed the wrong slice. +""" + +from __future__ import annotations + +import numpy as np +import pytest +import torch + +from plugrl_server.policy.fpo.fpo_policy import FPOPolicy, FPOPolicyConfig + + +def _policy(**kw) -> FPOPolicy: + return FPOPolicy(FPOPolicyConfig(device="cpu", **kw)) + + +def test_the_default_reads_obs_as_before(): + policy = _policy(obs_dim=3, action_dim=2) + x = np.arange(6, dtype=np.float32).reshape(2, 3) + + got = policy.extract_model_obs_tensor({"states": {"obs": x, "other": x * 10}}) + + assert torch.equal(got, torch.as_tensor(x)) + + +def test_named_keys_are_concatenated_in_the_order_given(): + policy = _policy(obs_dim=5, action_dim=2, state_keys=("b", "a")) + a = np.ones((2, 2), dtype=np.float32) + b = np.full((2, 3), 2.0, dtype=np.float32) + + got = policy.extract_model_obs_tensor({"states": {"a": a, "b": b, "c": a}}) + + assert got.shape == (2, 5) + assert torch.equal(got, torch.as_tensor(np.concatenate([b, a], axis=-1))) + + +def test_a_missing_key_names_what_was_there(): + policy = _policy(obs_dim=3, action_dim=2, state_keys=("object",)) + + with pytest.raises(KeyError, match="robot0_eef_pos"): + policy.extract_model_obs_tensor( + {"states": {"robot0_eef_pos": np.zeros((1, 3), dtype=np.float32)}} + ) + + +def test_widths_that_do_not_add_up_to_obs_dim_raise(): + policy = _policy(obs_dim=4, action_dim=2, state_keys=("a", "b")) + + with pytest.raises(ValueError, match="obs_dim"): + policy.extract_model_obs_tensor( + { + "states": { + "a": np.zeros((1, 2), dtype=np.float32), + "b": np.zeros((1, 3), dtype=np.float32), + } + } + )