From d2edfc95f03795d58f9a1d8e83d6c2a1aa32ff4a Mon Sep 17 00:00:00 2001 From: tactino <18781106300@163.com> Date: Fri, 25 Sep 2026 17:41:12 -0400 Subject: [PATCH 1/5] exp: pre-register E24 - which policy x algorithm x MuJoCo combinations run Fills the MuJoCo block of the platform's coverage: fpo-policy with FPO on Walker2d (a learning claim, threshold 500 against a pilot null of -0.28 to 0.71), fpo-policy with DPPO on Hopper and Walker2d, and dppo-policy - never run here before - with DPPO on HalfCheetah, Hopper and Walker2d (run-end-to-end claims). dppo-policy with FPO does not exist: FPO needs a flow policy. --- experiments/e24-coverage/.gitignore | 5 + experiments/e24-coverage/PROTOCOL.md | 131 ++++++++++++++++ experiments/e24-coverage/results/pilot.txt | 39 +++++ experiments/e24-coverage/run.sh | 45 ++++++ experiments/e24-coverage/run_cell.sh | 143 ++++++++++++++++++ experiments/e24-coverage/summarise.py | 168 +++++++++++++++++++++ 6 files changed, 531 insertions(+) create mode 100644 experiments/e24-coverage/.gitignore create mode 100644 experiments/e24-coverage/PROTOCOL.md create mode 100644 experiments/e24-coverage/results/pilot.txt create mode 100644 experiments/e24-coverage/run.sh create mode 100644 experiments/e24-coverage/run_cell.sh create mode 100644 experiments/e24-coverage/summarise.py diff --git a/experiments/e24-coverage/.gitignore b/experiments/e24-coverage/.gitignore new file mode 100644 index 0000000..13f6377 --- /dev/null +++ b/experiments/e24-coverage/.gitignore @@ -0,0 +1,5 @@ +# Checkpoints and tensorboard event files: binary, and large. What they hold +# that is the result is read out by summarise.py into summary.tsv. The logs +# are committed whole. +results/*/fpo/ +results/*/dppo/ diff --git a/experiments/e24-coverage/PROTOCOL.md b/experiments/e24-coverage/PROTOCOL.md new file mode 100644 index 0000000..c825611 --- /dev/null +++ b/experiments/e24-coverage/PROTOCOL.md @@ -0,0 +1,131 @@ +# E24 measurement protocol (pre-registered) + +**Written 2026-09-25, after the pilots in `results/pilot.txt` and before any +registered run.** + +This file must not be edited after the first registered data point. Anything +learned afterwards goes in `AMENDMENT.md`, dated. + +--- + +## The question + +PlugRL separates the policy, the algorithm that trains it and the +environment it trains in. How many of those combinations actually run on it, +end to end? On a CPU, before today, the answer was two: `fpo-policy` with FPO +on HalfCheetah (E6, E16) and on Hopper (E23), and `fpo-policy` with DPPO on +HalfCheetah (E17-E19). DPPO's own policy class, `dppo-policy`, had never run +here at all. + +E24 fills the MuJoCo block of that matrix: every combination of the two +MLP policies and the two algorithms that the code allows, on HalfCheetah, +Hopper and Walker2d. **Which of them run end to end, and does FPO also learn +Walker2d?** + +### What each cell claims, and what it does not + +A cell claims one of two things, never more than its evidence: + +* **runs end to end** - every seed completes its iterations, writes its + checkpoint, has no traceback and a client that exits 0. Nothing about + learning. DPPO from a random initialisation did not learn HalfCheetah in + this configuration (E17, E18), so no DPPO cell here claims to learn. +* **learns** - only `fpo-policy · FPO · Walker2d`, against a threshold set + from the pilot's untrained policy. + +### What the code rules out + +FPO requires a flow policy (`BasePolicyGradientFlowPolicy`), and +`dppo-policy` is a diffusion policy, so **`dppo-policy · FPO` does not exist** +and is not a cell. DPPO accepts both. That leaves three policy-algorithm +pairs, and with three tasks, nine cells, three of which are already filled. + +--- + +## Declared in advance: what was already known + +1. **Filled before E24**: `fpo-policy · FPO` on HalfCheetah (learns, E6, E16) + and Hopper (learns, E23); `fpo-policy · DPPO` on HalfCheetah (runs, E17, + E18; barely changes a trained policy, E19). +2. **The pilots** (`results/pilot.txt`). `fpo-policy · FPO · Walker2d`, one + iteration on three seeds: first-iteration return **0.71, −0.28, 0.64**, + episode length about 19, all three ending cleanly. The five DPPO cells, + one seed each: the first attempt at one iteration was refused at startup + by DPPO's scheduler check, correctly - its warmup is 10 iterations - and a + second at 11 iterations was stopped once every cell had completed its + first learning updates without an error. +3. **`dppo-policy` in this repository.** Each variant reads its network, its + action chunk (4 actions) and its D4RL min-max normalisation from files + bundled under `meta/dppo`, for `halfcheetah-medium-v2`, `hopper-medium-v2` + and `walker2d-medium-v2`. The `dppo` package it builds on comes from the + pinned fork in `pyproject.toml`, installed from a local clone of that + commit; installing it added packages and changed none. + +--- + +## Design + +| cell | policy | algorithm | task | iterations | buffer / batch | seeds | claims | +| --- | --- | --- | --- | --- | --- | --- | --- | +| `fpo-walker` | `fpo-policy` | `fpo` | Walker2d-v5 | 100 | 4,096 / FPO's default | 0, 1, 2 | learns | +| `fpodppo-hopper` | `fpo-policy` | `dppo hopper` | Hopper-v5 | 20 | 4,096 / 2,048 | 0, 1, 2 | runs | +| `fpodppo-walker` | `fpo-policy` | `dppo walker` | Walker2d-v5 | 20 | 4,096 / 2,048 | 0, 1, 2 | runs | +| `dppo-cheetah` | `dppo-policy cheetah` | `dppo cheetah` | HalfCheetah-v5 | 20 | 1,024 / 512 | 0, 1, 2 | runs | +| `dppo-hopper` | `dppo-policy hopper` | `dppo hopper` | Hopper-v5 | 20 | 1,024 / 512 | 0, 1, 2 | runs | +| `dppo-walker` | `dppo-policy walker` | `dppo walker` | Walker2d-v5 | 20 | 1,024 / 512 | 0, 1, 2 | runs | + +`run_cell.sh` runs one cell; `run.sh` runs all six. Everything not in the +table is E17-E23's: CPU, one env per client, server and client seeds matched, +checkpoints every 20 iterations. `fpo-policy` executes every action it +returns (`replan-steps 1`, a chunk of 1); `dppo-policy` executes its whole +chunk of 4 before the next inference, as DPPO does, so one buffer entry is 4 +environment steps. Its buffer is therefore 1,024 entries, so that an +iteration is still 4,096 environment steps, and its minibatch 512, so that +an epoch is still two minibatches, as with 4,096 and 2,048. 20 iterations +stay past the scheduler's 10-iteration warmup. + +Scheduling: `fpo-walker` runs alongside `fpodppo-hopper` and +`fpodppo-walker` - nine server-client pairs - and the three `dppo-policy` +cells follow. No wall-clock prediction is registered, so contention has +nothing to invalidate. + +--- + +## Predictions, and what falsifies each + +**P1 - every cell runs end to end.** For each of the six cells, on every +seed: all its iterations logged, a checkpoint at its last save, no traceback +in the server log, and a client that exits 0. + +> Reported cell by cell. A cell that fails is reported as not running, with +> its error, and the matrix shows it that way - not rerun until it passes. +> Falsified, for that cell, by any seed failing any part. + +**P2 - FPO learns Walker2d.** In `fpo-walker`, the mean of iterations 91-100 +is at least **500** on at least **2 of 3** seeds. + +> The untrained policy scores −0.28 to 0.71 (known item 2), so 500 is far +> above anything a policy that learned nothing reaches. E23 used the same +> threshold on Hopper and every seed cleared it by more than 1,000. Falsified +> by two or more seeds below 500, and then, by the project's rule, it is a +> platform defect to locate before anything else. + +**Reported, not predicted:** each cell's first and last-ten returns and +episode lengths, and each run's wall clock. + +--- + +## Declared deviations allowed in advance + +1. One restart of any run that dies for a reason outside the experiment - + including a restart of the machine - recorded in `AMENDMENT.md`. +2. If a cell fails because of how it was launched rather than what it runs - + a wrong flag, a port in use - the launch is fixed and the cell rerun once, + recorded in `AMENDMENT.md`. A failure inside the code under test is a + result and is not rerun. + +--- + +## Reading order + +P1 cell by cell, then P2, then the descriptive figures. diff --git a/experiments/e24-coverage/results/pilot.txt b/experiments/e24-coverage/results/pilot.txt new file mode 100644 index 0000000..99540b5 --- /dev/null +++ b/experiments/e24-coverage/results/pilot.txt @@ -0,0 +1,39 @@ +# E24 pilots, 2026-09-25, before PROTOCOL.md was written. +# Output directories were in the job's scratch space; the figures are copied here. + +## Pilot 1, 17:29: one iteration of every new cell + +fpo-walker (fpo-policy x fpo x Walker2d-v5), seeds 0 1 2: all finished rc=0 + first iteration, from each run's tensorboard (collected before any update): + seed rollout/reward rollout/length + 0 0.71 19.22 + 1 -0.28 18.49 + 2 0.64 19.99 + +The five DPPO cells, seed 0 each: every server refused to start with + + ValueError: warmup_steps (10) must be fewer than the iterations being run (1). + Either train for longer or lower the scheduler's warmup. + +That is DPPO's scheduler check doing its job: one iteration is shorter than +the 10-iteration warmup. Not a defect; the pilot asked for too little. + +## Pilot 2, 17:34: the five DPPO cells at 11 iterations, seed 0 + +Stopped once every cell had completed its first learning updates with no +traceback in any server log. By then: + + fpodppo-hopper 3 iterations logged, return 14.9, 17.6, 14.4 + fpodppo-walker 3 iterations logged, return 1.6, 2.1, 1.6 + dppo-cheetah 1 iteration logged, return -387.6, approx_kl 8.6e-06 + dppo-hopper 1 iteration logged, return 11.6, approx_kl 1.3e-05 + dppo-walker 1 iteration logged, return -2.1, approx_kl 1.7e-05 + + (stopped at 17:41 by ending its processes; not a clean shutdown, and not + meant to be one - the Walker2d pilot and every earlier run cover that) + +It also showed that with a buffer of 4,096 entries, a dppo-policy iteration +is 16,384 environment steps - each entry is a chunk of 4 - about 5 minutes +per iteration here. The protocol sets dppo-policy's buffer to 1,024 and its +minibatch to 512 so an iteration is 4,096 steps and an epoch two +minibatches, as elsewhere. diff --git a/experiments/e24-coverage/run.sh b/experiments/e24-coverage/run.sh new file mode 100644 index 0000000..8486712 --- /dev/null +++ b/experiments/e24-coverage/run.sh @@ -0,0 +1,45 @@ +#!/usr/bin/env bash +# E24: the six cells, in two waves. See PROTOCOL.md. +# +# bash run.sh +# +# Wave 1 runs the long FPO cell beside the two short fpo-policy DPPO cells - +# nine server-client pairs. Wave 2 runs the three dppo-policy cells. Each cell +# writes to results//, and its own output to results/.out. +set -uo pipefail + +HERE="$(cd "$(dirname "$0")" && pwd)" +R="$HERE/results" +mkdir -p "$R" +CELL="$HERE/run_cell.sh" + +date '+start %F %T' + +echo "--- wave 1" +PORT_BASE=9300 bash "$CELL" fpo-walker fpo-policy default fpo default Walker2d-v5 100 > "$R/fpo-walker.out" 2>&1 & +W1=$! +sleep 20 +PORT_BASE=9310 bash "$CELL" fpodppo-hopper fpo-policy default dppo hopper Hopper-v5 20 > "$R/fpodppo-hopper.out" 2>&1 & +W2=$! +sleep 20 +PORT_BASE=9320 bash "$CELL" fpodppo-walker fpo-policy default dppo walker Walker2d-v5 20 > "$R/fpodppo-walker.out" 2>&1 & +W3=$! +wait "$W2" "$W3" +echo "short cells of wave 1 done at $(date '+%T')" + +echo "--- wave 2" +BUFFER=1024 BATCH=512 PORT_BASE=9330 bash "$CELL" dppo-cheetah dppo-policy cheetah dppo cheetah HalfCheetah-v5 20 > "$R/dppo-cheetah.out" 2>&1 & +D1=$! +sleep 20 +BUFFER=1024 BATCH=512 PORT_BASE=9340 bash "$CELL" dppo-hopper dppo-policy hopper dppo hopper Hopper-v5 20 > "$R/dppo-hopper.out" 2>&1 & +D2=$! +sleep 20 +BUFFER=1024 BATCH=512 PORT_BASE=9350 bash "$CELL" dppo-walker dppo-policy walker dppo walker Walker2d-v5 20 > "$R/dppo-walker.out" 2>&1 & +D3=$! +wait "$W1" "$D1" "$D2" "$D3" + +date '+end %F %T' +for c in fpo-walker fpodppo-hopper fpodppo-walker dppo-cheetah dppo-hopper dppo-walker; do + printf '%-15s %s\n' "$c" "$(grep -E 'failed seeds' "$R/$c.out" 2>/dev/null)" +done +echo E24_DONE diff --git a/experiments/e24-coverage/run_cell.sh b/experiments/e24-coverage/run_cell.sh new file mode 100644 index 0000000..2afd539 --- /dev/null +++ b/experiments/e24-coverage/run_cell.sh @@ -0,0 +1,143 @@ +#!/usr/bin/env bash +# One cell of E24's coverage matrix: a policy, an algorithm and a MuJoCo task, +# run on SEEDS concurrently, each seed a server and an env client. +# +# bash run_cell.sh CELL POLICY POLICY_VARIANT ALGO ALGO_VARIANT ENV ITERS [OUT_DIR] +# +# CELL a name for the directory and the checkpoints +# POLICY fpo-policy | dppo-policy +# ALGO fpo | dppo +# ENV a gymnasium MuJoCo task, e.g. Walker2d-v5 +# ITERS iterations of BUFFER entries +# +# Settings are E17-E23's wherever they apply: buffer 4,096, save every 20 +# iterations, server and client seeds matched. What differs by combination: +# +# * fpo-policy takes the task's dimensions on the command line; dppo-policy +# reads them, and its D4RL normalisation, from the config its variant names. +# * FPO counts --algo.global-steps; DPPO derives it from --algo.train-itrs. +# * dppo-policy returns a chunk of 4 actions, and DPPO's own setting executes +# the whole chunk before the next inference, so its client replans every 4 +# steps. fpo-policy's chunk is 1. +# * The client's episode budget is an upper bound large enough never to bind: +# the server ends every run. +# +# BUFFER (default 4096) and BATCH (the algorithm variant's own default when +# unset) come from the environment, so run.sh can set them per cell. +set -uo pipefail + +CELL="$1" +POLICY="$2" +POLICY_VARIANT="$3" +ALGO="$4" +ALGO_VARIANT="$5" +ENV_NAME="$6" +ITERS="$7" +HERE="$(cd "$(dirname "$0")" && pwd)" +OUT="${8:-$HERE/results/$CELL}" +case "$OUT" in /*) ;; *) OUT="$(pwd)/$OUT" ;; esac + +SEEDS="${SEEDS:-0 1 2}" +PORT_BASE="${PORT_BASE:-9300}" +BUFFER="${BUFFER:-4096}" +SERVER_DIR="$(cd "$HERE/../.." && pwd)" +mkdir -p "$OUT" + +case "$ENV_NAME" in + HalfCheetah-v5) DIMS=(17 6) ;; + Hopper-v5) DIMS=(11 3) ;; + Walker2d-v5) DIMS=(17 6) ;; + *) echo "error: no dimensions recorded for $ENV_NAME" >&2; exit 1 ;; +esac + +POLICY_ARGS=() +REPLAN=1 +if [ "$POLICY" = "fpo-policy" ]; then + POLICY_ARGS=(--policy.obs-dim "${DIMS[0]}" --policy.action-dim "${DIMS[1]}") +elif [ "$POLICY" = "dppo-policy" ]; then + REPLAN=4 +fi + +ALGO_ARGS=(--algo.buffer-size "$BUFFER") +if [ -n "${BATCH:-}" ]; then + ALGO_ARGS+=(--algo.batch-size "$BATCH") +fi +if [ "$ALGO" = "fpo" ]; then + ALGO_ARGS+=(--algo.global-steps $((BUFFER * ITERS))) +else + ALGO_ARGS+=(--algo.train-itrs "$ITERS") +fi + +find_client_dir() { + local candidate + for candidate in \ + "${PLUGRL_ENV_CLIENT:-}" \ + "$SERVER_DIR/../plugrl-env-client" \ + "$SERVER_DIR/../../../../plugrl-env-client"; do + [ -n "$candidate" ] && [ -d "$candidate" ] && (cd "$candidate" && pwd) && return + done + return 1 +} +CLIENT_DIR="$(find_client_dir)" || { + echo "error: no plugrl-env-client found. Set PLUGRL_ENV_CLIENT." >&2 + exit 1 +} + +pick_python() { + local repo="$1" override="${2:-}" candidate + if [ -n "$override" ]; then echo "$override"; return; fi + for candidate in "$repo/.venv/Scripts/python.exe" "$repo/.venv/bin/python"; do + [ -x "$candidate" ] && { echo "$candidate"; return; } + done + command -v python3 || command -v python +} +SERVER_PY="$(pick_python "$SERVER_DIR" "${PLUGRL_SERVER_PYTHON:-}")" +CLIENT_PY="$(pick_python "$CLIENT_DIR" "${PLUGRL_CLIENT_PYTHON:-}")" + +echo "cell: $CELL = $POLICY/$POLICY_VARIANT x $ALGO/$ALGO_VARIANT x $ENV_NAME" +echo "iters: $ITERS x $BUFFER batch: ${BATCH:-variant default} replan: $REPLAN seeds: $SEEDS" +echo "out: $OUT" +date '+start %F %T' + +run_seed() { + local seed="$1" port=$((PORT_BASE + $1)) + (cd "$SERVER_DIR" && "$SERVER_PY" -m plugrl_server.cli \ + "$POLICY" "$POLICY_VARIANT" "$ALGO" "$ALGO_VARIANT" \ + --port "$port" --seed "$seed" --policy.device cpu \ + "${POLICY_ARGS[@]}" "${ALGO_ARGS[@]}" \ + --algo.save-interval 20 \ + --no-show-progress-bar --no-show-metric-table \ + --checkpoint-base-dir "$OUT" --exp-name "$CELL-seed$seed" --overwrite \ + > "$OUT/server-seed$seed.log" 2>&1 &) + + local waited=0 + until grep -q "is listening on" "$OUT/server-seed$seed.log" 2>/dev/null; do + sleep 1 + waited=$((waited + 1)) + if [ "$waited" -ge 300 ]; then + echo "seed $seed: server never listened; see $OUT/server-seed$seed.log" >&2 + return 1 + fi + done + + (cd "$CLIENT_DIR" && "$CLIENT_PY" -m plugrl_env_client.cli mujoco-v1 \ + --server-port "$port" --server-host 127.0.0.1 \ + --num-envs 1 --num-episodes $((BUFFER * ITERS * 4)) \ + --env.name "$ENV_NAME" \ + --runner.replan-steps "$REPLAN" --runner.seed "$seed" \ + > "$OUT/client-seed$seed.log" 2>&1) + echo "seed $seed finished rc=$? at $(date '+%T')" +} + +PIDS=() +for seed in $SEEDS; do + run_seed "$seed" & + PIDS+=($!) + sleep 5 +done +FAILED=0 +for pid in "${PIDS[@]}"; do wait "$pid" || FAILED=$((FAILED + 1)); done + +date '+end %F %T' +echo "failed seeds: $FAILED" +echo "CELL_DONE $CELL" diff --git a/experiments/e24-coverage/summarise.py b/experiments/e24-coverage/summarise.py new file mode 100644 index 0000000..0a68310 --- /dev/null +++ b/experiments/e24-coverage/summarise.py @@ -0,0 +1,168 @@ +"""Read E24's verdicts from each cell's logs and tensorboards. + + python summarise.py + +P1 cell by cell, then P2, then the descriptive figures - PROTOCOL.md's +reading order. Writes `summary.tsv`, one row per cell and seed, which is what +the coverage matrix is drawn from. +""" + +import datetime +import pathlib +import re +import sys + +from tensorboard.backend.event_processing.event_accumulator import EventAccumulator + +HERE = pathlib.Path(__file__).resolve().parent +RESULTS = HERE / "results" +SEEDS = (0, 1, 2) + +# cell: (policy, algorithm, task, iterations, claim) +CELLS = { + "fpo-walker": ("fpo-policy", "fpo", "Walker2d-v5", 100, "learns"), + "fpodppo-hopper": ("fpo-policy", "dppo", "Hopper-v5", 20, "runs"), + "fpodppo-walker": ("fpo-policy", "dppo", "Walker2d-v5", 20, "runs"), + "dppo-cheetah": ("dppo-policy", "dppo", "HalfCheetah-v5", 20, "runs"), + "dppo-hopper": ("dppo-policy", "dppo", "Hopper-v5", 20, "runs"), + "dppo-walker": ("dppo-policy", "dppo", "Walker2d-v5", 20, "runs"), +} + + +def run_dir(cell: str, seed: int) -> pathlib.Path | None: + hits = sorted((RESULTS / cell).glob(f"*/*/{cell}-seed{seed}")) + return hits[0] if hits else None + + +def curve(run: pathlib.Path, tag: str) -> list[float]: + events = sorted((run / "tensorboard").glob("events.*")) + if not events: + return [] + acc = EventAccumulator(str(events[0]), size_guidance={"scalars": 0}) + acc.Reload() + if tag not in acc.Tags()["scalars"]: + return [] + return [e.value for e in acc.Scalars(tag)] + + +def wall_minutes(out: str) -> float | None: + stamps = dict(re.findall(r"^(start|end)\s+(\S+ \S+)$", out, flags=re.M)) + if "start" not in stamps or "end" not in stamps: + return None + fmt = "%Y-%m-%d %H:%M:%S" + delta = datetime.datetime.strptime(stamps["end"], fmt) - datetime.datetime.strptime( + stamps["start"], fmt + ) + return delta.total_seconds() / 60 + + +def main() -> int: + rows = [] + print("P1 every cell runs end to end") + runs_ok = {} + for cell, (policy, algo, task, iters, claim) in CELLS.items(): + out_path = RESULTS / f"{cell}.out" + out = ( + out_path.read_text(encoding="utf-8", errors="replace") + if out_path.exists() + else "" + ) + exits = { + int(s): int(rc) for s, rc in re.findall(r"seed (\d) finished rc=(\d+)", out) + } + ok_cell = True + for seed in SEEDS: + run = run_dir(cell, seed) + reward = curve(run, "rollout/reward") if run else [] + length = curve(run, "rollout/length") if run else [] + saves = [p for p in run.iterdir() if p.name.isdigit()] if run else [] + log_path = RESULTS / cell / f"server-seed{seed}.log" + log = ( + log_path.read_text(encoding="utf-8", errors="replace") + if log_path.exists() + else "" + ) + traceback = "Traceback" in log + ok = ( + len(reward) >= iters + and len(saves) >= iters // 20 + and not traceback + and exits.get(seed) == 0 + ) + ok_cell &= ok + last10 = ( + sum(reward[iters - 10 : iters]) / 10 + if len(reward) >= iters + else float("nan") + ) + last10_len = ( + sum(length[iters - 10 : iters]) / 10 + if len(length) >= iters + else float("nan") + ) + rows.append( + ( + cell, + policy, + algo, + task, + claim, + seed, + len(reward), + ok, + reward[0] if reward else float("nan"), + last10, + length[0] if length else float("nan"), + last10_len, + ) + ) + runs_ok[cell] = ok_cell + minutes = wall_minutes(out) + print( + f" {cell:15s} {policy:12s} {algo:5s} {task:15s} " + f"{'HOLDS' if ok_cell else 'FAILS'}" + + (f" ({minutes:.0f} min)" if minutes is not None else "") + ) + if not ok_cell: + for r in rows: + if r[0] == cell and not r[7]: + print(f" seed {r[5]}: {r[6]} of {iters} iterations logged") + + print("\nP2 FPO learns Walker2d: mean of iterations 91-100 >= 500 on 2 of 3") + if not runs_ok["fpo-walker"]: + print(" not read - fpo-walker did not run end to end") + else: + ends = [r[9] for r in rows if r[0] == "fpo-walker"] + for seed, end in zip(SEEDS, ends): + print(f" seed {seed}: {end:8.1f}") + passed = sum(e >= 500 for e in ends) + print(f" {'HOLDS' if passed >= 2 else 'FALSIFIED'} ({passed} of 3)") + + print( + "\nreported: first iteration and mean of the last ten, return and episode length" + ) + for r in rows: + print( + f" {r[0]:15s} seed {r[5]} return {r[8]:8.1f} -> {r[9]:8.1f} " + f"length {r[10]:6.1f} -> {r[11]:6.1f}" + ) + + header = "cell\tpolicy\talgorithm\ttask\tclaim\tseed\titerations\truns\tfirst_return\tlast10_return\tfirst_length\tlast10_length" + lines = [header] + [ + "\t".join( + f"{v:.1f}" + if isinstance(v, float) + else str(v).lower() + if isinstance(v, bool) + else str(v) + for v in r + ) + for r in rows + ] + (HERE / "summary.tsv").write_text("\n".join(lines) + "\n", encoding="utf-8") + print(f"\nwrote {HERE / 'summary.tsv'}") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) From 2ccaed070907994734604da3965a0d8bddebdb8d Mon Sep 17 00:00:00 2001 From: tactino <18781106300@163.com> Date: Fri, 25 Sep 2026 18:08:18 -0400 Subject: [PATCH 2/5] exp(e24): amendment 1 - moved to a Linux workstation and rerun from the start --- experiments/e24-coverage/.gitignore | 2 + experiments/e24-coverage/AMENDMENT.md | 41 ++++++++++++++ .../results/attempt-1/fpo-walker.out | 4 ++ .../attempt-1/fpo-walker/client-seed0.log | 45 ++++++++++++++++ .../attempt-1/fpo-walker/client-seed1.log | 45 ++++++++++++++++ .../attempt-1/fpo-walker/client-seed2.log | 44 +++++++++++++++ .../attempt-1/fpo-walker/server-seed0.log | 11 ++++ .../attempt-1/fpo-walker/server-seed1.log | 11 ++++ .../attempt-1/fpo-walker/server-seed2.log | 11 ++++ .../results/attempt-1/fpodppo-hopper.out | 4 ++ .../attempt-1/fpodppo-hopper/client-seed0.log | 54 +++++++++++++++++++ .../attempt-1/fpodppo-hopper/client-seed1.log | 52 ++++++++++++++++++ .../attempt-1/fpodppo-hopper/client-seed2.log | 53 ++++++++++++++++++ .../attempt-1/fpodppo-hopper/server-seed0.log | 13 +++++ .../attempt-1/fpodppo-hopper/server-seed1.log | 13 +++++ .../attempt-1/fpodppo-hopper/server-seed2.log | 13 +++++ .../results/attempt-1/fpodppo-walker.out | 4 ++ .../attempt-1/fpodppo-walker/client-seed0.log | 52 ++++++++++++++++++ .../attempt-1/fpodppo-walker/client-seed1.log | 52 ++++++++++++++++++ .../attempt-1/fpodppo-walker/client-seed2.log | 52 ++++++++++++++++++ .../attempt-1/fpodppo-walker/server-seed0.log | 13 +++++ .../attempt-1/fpodppo-walker/server-seed1.log | 13 +++++ .../attempt-1/fpodppo-walker/server-seed2.log | 13 +++++ .../e24-coverage/results/attempt-1/run.out | 2 + 24 files changed, 617 insertions(+) create mode 100644 experiments/e24-coverage/AMENDMENT.md create mode 100644 experiments/e24-coverage/results/attempt-1/fpo-walker.out create mode 100644 experiments/e24-coverage/results/attempt-1/fpo-walker/client-seed0.log create mode 100644 experiments/e24-coverage/results/attempt-1/fpo-walker/client-seed1.log create mode 100644 experiments/e24-coverage/results/attempt-1/fpo-walker/client-seed2.log create mode 100644 experiments/e24-coverage/results/attempt-1/fpo-walker/server-seed0.log create mode 100644 experiments/e24-coverage/results/attempt-1/fpo-walker/server-seed1.log create mode 100644 experiments/e24-coverage/results/attempt-1/fpo-walker/server-seed2.log create mode 100644 experiments/e24-coverage/results/attempt-1/fpodppo-hopper.out create mode 100644 experiments/e24-coverage/results/attempt-1/fpodppo-hopper/client-seed0.log create mode 100644 experiments/e24-coverage/results/attempt-1/fpodppo-hopper/client-seed1.log create mode 100644 experiments/e24-coverage/results/attempt-1/fpodppo-hopper/client-seed2.log create mode 100644 experiments/e24-coverage/results/attempt-1/fpodppo-hopper/server-seed0.log create mode 100644 experiments/e24-coverage/results/attempt-1/fpodppo-hopper/server-seed1.log create mode 100644 experiments/e24-coverage/results/attempt-1/fpodppo-hopper/server-seed2.log create mode 100644 experiments/e24-coverage/results/attempt-1/fpodppo-walker.out create mode 100644 experiments/e24-coverage/results/attempt-1/fpodppo-walker/client-seed0.log create mode 100644 experiments/e24-coverage/results/attempt-1/fpodppo-walker/client-seed1.log create mode 100644 experiments/e24-coverage/results/attempt-1/fpodppo-walker/client-seed2.log create mode 100644 experiments/e24-coverage/results/attempt-1/fpodppo-walker/server-seed0.log create mode 100644 experiments/e24-coverage/results/attempt-1/fpodppo-walker/server-seed1.log create mode 100644 experiments/e24-coverage/results/attempt-1/fpodppo-walker/server-seed2.log create mode 100644 experiments/e24-coverage/results/attempt-1/run.out diff --git a/experiments/e24-coverage/.gitignore b/experiments/e24-coverage/.gitignore index 13f6377..467e341 100644 --- a/experiments/e24-coverage/.gitignore +++ b/experiments/e24-coverage/.gitignore @@ -3,3 +3,5 @@ # are committed whole. results/*/fpo/ results/*/dppo/ +results/attempt-1/*/fpo/ +results/attempt-1/*/dppo/ diff --git a/experiments/e24-coverage/AMENDMENT.md b/experiments/e24-coverage/AMENDMENT.md new file mode 100644 index 0000000..f9cd035 --- /dev/null +++ b/experiments/e24-coverage/AMENDMENT.md @@ -0,0 +1,41 @@ +# E24 amendments + +Dated changes and declared deviations. `PROTOCOL.md` is not edited after the +first registered data point; anything learned afterwards goes here. + +--- + +## 1. Moved off the Windows machine, and rerun from the start + +**2026-09-25 18:15, before any cell had finished.** Declared deviation 1: a +run that dies for a reason outside the experiment is restarted once. + +The first attempt started at 17:41 on the Windows machine every earlier CPU +experiment used. At 18:06 it was stopped, on the user's request, to free that +machine, which was short of memory with everything else running on it. Wave 1 +had been running for 25 minutes; no seed of any cell had finished. Its logs +and checkpoints are kept, untouched and unread, in `results/attempt-1/`. + +All six cells rerun from the start on a Linux workstation instead, +`guangzhao` - 24 cores, 125 GB of memory, otherwise idle - under +`~/zuogou/plugrl`, with the same `run.sh`, `run_cell.sh`, seeds, code +(`main` at 8812b54 plus this branch) and settings. What differs, and is +recorded rather than matched: + +| | Windows machine | guangzhao | +| --- | --- | --- | +| OS | Windows 11 | Linux | +| torch, numpy | 2.7.1+cpu, 2.3.2 | 2.7.1+cpu, 2.3.2 | +| gymnasium, mujoco (client) | 1.2.3, 3.6.0 | 1.3.0, 3.14.0 | +| dppo | the pinned fork, 89ac416 | the pinned fork, 89ac416 | +| threads per process | torch's default | `OMP_NUM_THREADS=1` | + +The thread cap is there because the machine is shared: nine server-client +pairs at torch's default would try to use every core. It changes how long an +iteration takes, not what it computes. + +Nothing registered depends on the machine. P1 asks whether each combination +runs end to end, and P2's threshold, 500 against an untrained policy near 0, +is far from anything a change of simulator version could move. The pilot's +null was measured on the Windows machine; the registered runs' own first +iterations are reported beside it. diff --git a/experiments/e24-coverage/results/attempt-1/fpo-walker.out b/experiments/e24-coverage/results/attempt-1/fpo-walker.out new file mode 100644 index 0000000..2e43f3d --- /dev/null +++ b/experiments/e24-coverage/results/attempt-1/fpo-walker.out @@ -0,0 +1,4 @@ +cell: fpo-walker = fpo-policy/default x fpo/default x Walker2d-v5 +iters: 100 x 4096 batch: variant default replan: 1 seeds: 0 1 2 +out: /d/75128/Desktop/plugrl-work/plugrl-server/.claude/worktrees/fix-pi0-image-mask-batching/experiments/e24-coverage/results/fpo-walker +start 2026-09-25 17:41:23 diff --git a/experiments/e24-coverage/results/attempt-1/fpo-walker/client-seed0.log b/experiments/e24-coverage/results/attempt-1/fpo-walker/client-seed0.log new file mode 100644 index 0000000..1b93fca --- /dev/null +++ b/experiments/e24-coverage/results/attempt-1/fpo-walker/client-seed0.log @@ -0,0 +1,45 @@ +2026-09-25 17:41:39.141 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 17:41:39.142 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 17:41:39.145 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 17:41:39.148 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 17:41:39.155 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 17:41:39.281 | INFO  | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-174139-e9e00c1c output_dir=runs\mujoco-v1-nenv1-20260925-174139-e9e00c1c recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 17:41:39.414 | INFO  | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9300... +2026-09-25 17:41:39.422 | INFO  | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'FPOAlgorithm', 'policy': 'FPOPolicy', 'action_dim': 6, 'action_horizon': 1} +2026-09-25 17:42:09.430 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=3850 infer_calls=3850 feedback_calls=3850 infer_wait=21.502s infer_obs_pack=0.473s env_step=4.237s feedback_total=2.268s feedback_obs_pack=0.531s feedback_info_pack=0.099s effective_fps=135.18 +2026-09-25 17:43:23.064 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=4097 infer_calls=4097 feedback_calls=4097 infer_wait=94.324s infer_obs_pack=0.522s env_step=4.643s feedback_total=2.489s feedback_obs_pack=0.580s feedback_info_pack=0.108s effective_fps=40.18 +2026-09-25 17:43:53.153 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=4656 infer_calls=4656 feedback_calls=4656 infer_wait=123.130s infer_obs_pack=0.601s env_step=5.266s feedback_total=2.845s feedback_obs_pack=0.667s feedback_info_pack=0.125s effective_fps=35.32 +2026-09-25 17:44:23.159 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=5584 infer_calls=5584 feedback_calls=5584 infer_wait=151.034s infer_obs_pack=0.722s env_step=6.299s feedback_total=3.425s feedback_obs_pack=0.805s feedback_info_pack=0.151s effective_fps=34.58 +2026-09-25 17:44:53.186 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=7642 infer_calls=7642 feedback_calls=7642 infer_wait=176.380s infer_obs_pack=0.991s env_step=8.621s feedback_total=4.722s feedback_obs_pack=1.115s feedback_info_pack=0.213s effective_fps=40.07 +2026-09-25 17:46:04.577 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=8193 infer_calls=8193 feedback_calls=8193 infer_wait=246.422s infer_obs_pack=1.073s env_step=9.285s feedback_total=5.098s feedback_obs_pack=1.206s feedback_info_pack=0.229s effective_fps=31.29 +2026-09-25 17:46:34.577 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=9391 infer_calls=9391 feedback_calls=9391 infer_wait=273.744s infer_obs_pack=1.221s env_step=10.636s feedback_total=5.809s feedback_obs_pack=1.377s feedback_info_pack=0.261s effective_fps=32.23 +2026-09-25 17:47:04.578 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=11169 infer_calls=11169 feedback_calls=11169 infer_wait=299.939s infer_obs_pack=1.439s env_step=12.552s feedback_total=6.844s feedback_obs_pack=1.630s feedback_info_pack=0.307s effective_fps=34.82 +2026-09-25 17:48:39.217 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=12289 infer_calls=12289 feedback_calls=12289 infer_wait=392.100s infer_obs_pack=1.590s env_step=13.804s feedback_total=7.514s feedback_obs_pack=1.786s feedback_info_pack=0.337s effective_fps=29.61 +2026-09-25 17:49:09.226 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=14430 infer_calls=14430 feedback_calls=14430 infer_wait=417.558s infer_obs_pack=1.920s env_step=16.095s feedback_total=8.727s feedback_obs_pack=2.080s feedback_info_pack=0.392s effective_fps=32.48 +2026-09-25 17:49:39.231 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=16292 infer_calls=16292 feedback_calls=16292 infer_wait=443.953s infer_obs_pack=2.129s env_step=17.926s feedback_total=9.714s feedback_obs_pack=2.316s feedback_info_pack=0.437s effective_fps=34.39 +2026-09-25 17:50:13.736 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=16385 infer_calls=16385 feedback_calls=16385 infer_wait=478.238s infer_obs_pack=2.142s env_step=18.040s feedback_total=9.772s feedback_obs_pack=2.330s feedback_info_pack=0.440s effective_fps=32.24 +2026-09-25 17:50:43.753 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=18518 infer_calls=18518 feedback_calls=18518 infer_wait=503.537s infer_obs_pack=2.431s env_step=20.467s feedback_total=11.029s feedback_obs_pack=2.640s feedback_info_pack=0.497s effective_fps=34.45 +2026-09-25 17:51:13.804 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=20249 infer_calls=20249 feedback_calls=20249 infer_wait=529.739s infer_obs_pack=2.660s env_step=22.451s feedback_total=12.055s feedback_obs_pack=2.893s feedback_info_pack=0.546s effective_fps=35.72 +2026-09-25 17:51:54.314 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=20481 infer_calls=20481 feedback_calls=20481 infer_wait=569.748s infer_obs_pack=2.689s env_step=22.709s feedback_total=12.190s feedback_obs_pack=2.925s feedback_info_pack=0.552s effective_fps=33.72 +2026-09-25 17:52:24.323 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=21901 infer_calls=21901 feedback_calls=21901 infer_wait=596.714s infer_obs_pack=2.860s env_step=24.273s feedback_total=12.977s feedback_obs_pack=3.115s feedback_info_pack=0.588s effective_fps=34.39 +2026-09-25 17:52:54.336 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=23749 infer_calls=23749 feedback_calls=23749 infer_wait=622.926s infer_obs_pack=3.083s env_step=26.215s feedback_total=14.013s feedback_obs_pack=3.362s feedback_info_pack=0.636s effective_fps=35.65 +2026-09-25 17:53:58.529 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=24577 infer_calls=24577 feedback_calls=24577 infer_wait=685.418s infer_obs_pack=3.183s env_step=27.087s feedback_total=14.474s feedback_obs_pack=3.473s feedback_info_pack=0.656s effective_fps=33.66 +2026-09-25 17:54:28.549 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=25891 infer_calls=25891 feedback_calls=25891 infer_wait=712.825s infer_obs_pack=3.336s env_step=28.430s feedback_total=15.170s feedback_obs_pack=3.641s feedback_info_pack=0.686s effective_fps=34.08 +2026-09-25 17:54:58.557 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=27409 infer_calls=27409 feedback_calls=27409 infer_wait=739.480s infer_obs_pack=3.527s env_step=30.189s feedback_total=16.062s feedback_obs_pack=3.862s feedback_info_pack=0.727s effective_fps=34.73 +2026-09-25 17:55:28.590 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=28670 infer_calls=28670 feedback_calls=28670 infer_wait=766.664s infer_obs_pack=3.694s env_step=31.645s feedback_total=16.837s feedback_obs_pack=4.046s feedback_info_pack=0.764s effective_fps=35.01 +2026-09-25 17:56:13.088 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=28674 infer_calls=28674 feedback_calls=28674 infer_wait=811.156s infer_obs_pack=3.695s env_step=31.647s feedback_total=16.838s feedback_obs_pack=4.046s feedback_info_pack=0.764s effective_fps=33.21 +2026-09-25 17:56:43.104 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=30014 infer_calls=30014 feedback_calls=30014 infer_wait=837.953s infer_obs_pack=3.888s env_step=33.296s feedback_total=17.705s feedback_obs_pack=4.254s feedback_info_pack=0.802s effective_fps=33.62 +2026-09-25 17:57:13.124 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=31225 infer_calls=31225 feedback_calls=31225 infer_wait=864.979s infer_obs_pack=4.062s env_step=34.824s feedback_total=18.524s feedback_obs_pack=4.452s feedback_info_pack=0.838s effective_fps=33.85 +2026-09-25 17:58:27.206 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=32769 infer_calls=32769 feedback_calls=32769 infer_wait=935.407s infer_obs_pack=4.287s env_step=36.684s feedback_total=19.512s feedback_obs_pack=4.685s feedback_info_pack=0.885s effective_fps=32.90 +2026-09-25 17:58:57.213 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=34237 infer_calls=34237 feedback_calls=34237 infer_wait=961.577s infer_obs_pack=4.511s env_step=38.687s feedback_total=20.533s feedback_obs_pack=4.933s feedback_info_pack=0.930s effective_fps=33.39 +2026-09-25 17:59:27.217 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=35400 infer_calls=35400 feedback_calls=35400 infer_wait=988.796s infer_obs_pack=4.667s env_step=40.136s feedback_total=21.280s feedback_obs_pack=5.111s feedback_info_pack=0.963s effective_fps=33.56 +2026-09-25 17:59:57.224 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=36766 infer_calls=36766 feedback_calls=36766 infer_wait=1015.456s infer_obs_pack=4.858s env_step=41.868s feedback_total=22.183s feedback_obs_pack=5.335s feedback_info_pack=1.003s effective_fps=33.91 +2026-09-25 18:00:54.991 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=36865 infer_calls=36865 feedback_calls=36865 infer_wait=1073.006s infer_obs_pack=4.871s env_step=41.980s feedback_total=22.240s feedback_obs_pack=5.348s feedback_info_pack=1.006s effective_fps=32.28 +2026-09-25 18:01:25.005 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=38190 infer_calls=38190 feedback_calls=38190 infer_wait=1099.993s infer_obs_pack=5.057s env_step=43.518s feedback_total=23.069s feedback_obs_pack=5.546s feedback_info_pack=1.045s effective_fps=32.60 +2026-09-25 18:01:55.021 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=39521 infer_calls=39521 feedback_calls=39521 infer_wait=1126.766s infer_obs_pack=5.250s env_step=45.187s feedback_total=23.945s feedback_obs_pack=5.757s feedback_info_pack=1.085s effective_fps=32.90 +2026-09-25 18:02:25.032 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=40938 infer_calls=40938 feedback_calls=40938 infer_wait=1153.299s infer_obs_pack=5.456s env_step=46.987s feedback_total=24.881s feedback_obs_pack=5.984s feedback_info_pack=1.127s effective_fps=33.27 +2026-09-25 18:03:17.224 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=40961 infer_calls=40961 feedback_calls=40961 infer_wait=1205.447s infer_obs_pack=5.458s env_step=47.010s feedback_total=24.892s feedback_obs_pack=5.987s feedback_info_pack=1.127s effective_fps=31.93 +2026-09-25 18:03:47.235 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=41959 infer_calls=41959 feedback_calls=41959 infer_wait=1233.202s infer_obs_pack=5.588s env_step=48.171s feedback_total=25.500s feedback_obs_pack=6.134s feedback_info_pack=1.153s effective_fps=31.97 +2026-09-25 18:04:17.246 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=43327 infer_calls=43327 feedback_calls=43327 infer_wait=1260.071s infer_obs_pack=5.771s env_step=49.808s feedback_total=26.335s feedback_obs_pack=6.341s feedback_info_pack=1.191s effective_fps=32.29 +2026-09-25 18:04:47.277 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=44213 infer_calls=44213 feedback_calls=44213 infer_wait=1287.972s infer_obs_pack=5.894s env_step=50.927s feedback_total=26.902s feedback_obs_pack=6.475s feedback_info_pack=1.215s effective_fps=32.23 +2026-09-25 18:05:17.295 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=44878 infer_calls=44878 feedback_calls=44878 infer_wait=1316.682s infer_obs_pack=5.967s env_step=51.610s feedback_total=27.247s feedback_obs_pack=6.556s feedback_info_pack=1.230s effective_fps=32.02 diff --git a/experiments/e24-coverage/results/attempt-1/fpo-walker/client-seed1.log b/experiments/e24-coverage/results/attempt-1/fpo-walker/client-seed1.log new file mode 100644 index 0000000..93c5827 --- /dev/null +++ b/experiments/e24-coverage/results/attempt-1/fpo-walker/client-seed1.log @@ -0,0 +1,45 @@ +2026-09-25 17:41:44.483 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 17:41:44.488 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 17:41:44.490 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 17:41:44.494 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 17:41:44.502 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 17:41:44.697 | INFO  | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-174144-a7bead4e output_dir=runs\mujoco-v1-nenv1-20260925-174144-a7bead4e recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 17:41:44.837 | INFO  | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9301... +2026-09-25 17:41:44.843 | INFO  | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'FPOAlgorithm', 'policy': 'FPOPolicy', 'action_dim': 6, 'action_horizon': 1} +2026-09-25 17:42:14.866 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=3690 infer_calls=3690 feedback_calls=3690 infer_wait=21.216s infer_obs_pack=0.536s env_step=4.335s feedback_total=2.412s feedback_obs_pack=0.565s feedback_info_pack=0.107s effective_fps=129.48 +2026-09-25 17:43:32.365 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=4097 infer_calls=4097 feedback_calls=4097 infer_wait=97.530s infer_obs_pack=0.599s env_step=4.933s feedback_total=2.733s feedback_obs_pack=0.645s feedback_info_pack=0.121s effective_fps=38.73 +2026-09-25 17:44:02.383 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=4520 infer_calls=4520 feedback_calls=4520 infer_wait=126.642s infer_obs_pack=0.653s env_step=5.375s feedback_total=2.981s feedback_obs_pack=0.704s feedback_info_pack=0.132s effective_fps=33.32 +2026-09-25 17:44:32.420 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=6035 infer_calls=6035 feedback_calls=6035 infer_wait=153.199s infer_obs_pack=0.857s env_step=7.079s feedback_total=3.939s feedback_obs_pack=0.925s feedback_info_pack=0.174s effective_fps=36.56 +2026-09-25 17:45:02.436 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=7953 infer_calls=7953 feedback_calls=7953 infer_wait=178.721s infer_obs_pack=1.120s env_step=9.294s feedback_total=5.174s feedback_obs_pack=1.219s feedback_info_pack=0.233s effective_fps=40.93 +2026-09-25 17:46:11.545 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=8194 infer_calls=8194 feedback_calls=8194 infer_wait=247.262s infer_obs_pack=1.154s env_step=9.574s feedback_total=5.331s feedback_obs_pack=1.256s feedback_info_pack=0.239s effective_fps=31.12 +2026-09-25 17:46:41.547 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=9608 infer_calls=9608 feedback_calls=9608 infer_wait=274.188s infer_obs_pack=1.331s env_step=11.097s feedback_total=6.193s feedback_obs_pack=1.462s feedback_info_pack=0.278s effective_fps=32.81 +2026-09-25 17:47:11.560 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=11462 infer_calls=11462 feedback_calls=11462 infer_wait=300.291s infer_obs_pack=1.559s env_step=13.061s feedback_total=7.256s feedback_obs_pack=1.714s feedback_info_pack=0.326s effective_fps=35.58 +2026-09-25 17:48:42.386 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=12289 infer_calls=12289 feedback_calls=12289 infer_wait=389.141s infer_obs_pack=1.671s env_step=14.066s feedback_total=7.785s feedback_obs_pack=1.845s feedback_info_pack=0.351s effective_fps=29.78 +2026-09-25 17:49:12.414 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=14440 infer_calls=14440 feedback_calls=14440 infer_wait=414.747s infer_obs_pack=2.007s env_step=16.266s feedback_total=8.972s feedback_obs_pack=2.123s feedback_info_pack=0.404s effective_fps=32.67 +2026-09-25 17:49:42.432 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=16271 infer_calls=16271 feedback_calls=16271 infer_wait=441.158s infer_obs_pack=2.214s env_step=18.097s feedback_total=9.961s feedback_obs_pack=2.358s feedback_info_pack=0.450s effective_fps=34.51 +2026-09-25 17:50:15.494 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=16386 infer_calls=16386 feedback_calls=16386 infer_wait=474.017s infer_obs_pack=2.225s env_step=18.201s feedback_total=10.014s feedback_obs_pack=2.371s feedback_info_pack=0.452s effective_fps=32.48 +2026-09-25 17:50:45.639 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=18492 infer_calls=18492 feedback_calls=18492 infer_wait=499.443s infer_obs_pack=2.506s env_step=20.620s feedback_total=11.283s feedback_obs_pack=2.687s feedback_info_pack=0.510s effective_fps=34.64 +2026-09-25 17:51:15.648 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=20219 infer_calls=20219 feedback_calls=20219 infer_wait=525.519s infer_obs_pack=2.737s env_step=22.646s feedback_total=12.335s feedback_obs_pack=2.940s feedback_info_pack=0.557s effective_fps=35.90 +2026-09-25 17:51:57.508 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=20481 infer_calls=20481 feedback_calls=20481 infer_wait=566.896s infer_obs_pack=2.765s env_step=22.891s feedback_total=12.466s feedback_obs_pack=2.971s feedback_info_pack=0.563s effective_fps=33.85 +2026-09-25 17:52:27.514 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=22046 infer_calls=22046 feedback_calls=22046 infer_wait=593.692s infer_obs_pack=2.948s env_step=24.548s feedback_total=13.328s feedback_obs_pack=3.179s feedback_info_pack=0.602s effective_fps=34.74 +2026-09-25 17:52:57.515 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=23840 infer_calls=23840 feedback_calls=23840 infer_wait=619.894s infer_obs_pack=3.170s env_step=26.510s feedback_total=14.343s feedback_obs_pack=3.422s feedback_info_pack=0.652s effective_fps=35.91 +2026-09-25 17:53:59.536 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=24578 infer_calls=24578 feedback_calls=24578 infer_wait=680.328s infer_obs_pack=3.259s env_step=27.325s feedback_total=14.768s feedback_obs_pack=3.523s feedback_info_pack=0.671s effective_fps=33.87 +2026-09-25 17:54:29.542 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=25874 infer_calls=25874 feedback_calls=25874 infer_wait=707.615s infer_obs_pack=3.413s env_step=28.739s feedback_total=15.491s feedback_obs_pack=3.695s feedback_info_pack=0.702s effective_fps=34.26 +2026-09-25 17:54:59.608 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=27376 infer_calls=27376 feedback_calls=27376 infer_wait=734.384s infer_obs_pack=3.604s env_step=30.437s feedback_total=16.384s feedback_obs_pack=3.908s feedback_info_pack=0.743s effective_fps=34.88 +2026-09-25 17:55:29.798 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=28673 infer_calls=28673 feedback_calls=28673 infer_wait=761.663s infer_obs_pack=3.777s env_step=31.924s feedback_total=17.175s feedback_obs_pack=4.103s feedback_info_pack=0.779s effective_fps=35.20 +2026-09-25 17:56:14.494 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=28674 infer_calls=28674 feedback_calls=28674 infer_wait=806.355s infer_obs_pack=3.778s env_step=31.925s feedback_total=17.175s feedback_obs_pack=4.103s feedback_info_pack=0.779s effective_fps=33.37 +2026-09-25 17:56:44.505 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=30013 infer_calls=30013 feedback_calls=30013 infer_wait=833.136s infer_obs_pack=3.961s env_step=33.610s feedback_total=18.025s feedback_obs_pack=4.308s feedback_info_pack=0.822s effective_fps=33.77 +2026-09-25 17:57:14.506 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=31235 infer_calls=31235 feedback_calls=31235 infer_wait=860.209s infer_obs_pack=4.134s env_step=35.113s feedback_total=18.818s feedback_obs_pack=4.498s feedback_info_pack=0.857s effective_fps=34.01 +2026-09-25 17:58:29.831 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=32769 infer_calls=32769 feedback_calls=32769 infer_wait=931.902s infer_obs_pack=4.348s env_step=36.990s feedback_total=19.786s feedback_obs_pack=4.733s feedback_info_pack=0.902s effective_fps=33.00 +2026-09-25 17:58:59.847 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=34234 infer_calls=34234 feedback_calls=34234 infer_wait=958.233s infer_obs_pack=4.563s env_step=38.889s feedback_total=20.782s feedback_obs_pack=4.973s feedback_info_pack=0.946s effective_fps=33.48 +2026-09-25 17:59:29.848 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=35413 infer_calls=35413 feedback_calls=35413 infer_wait=985.455s infer_obs_pack=4.717s env_step=40.327s feedback_total=21.534s feedback_obs_pack=5.153s feedback_info_pack=0.980s effective_fps=33.66 +2026-09-25 17:59:59.851 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=36773 infer_calls=36773 feedback_calls=36773 infer_wait=1012.107s infer_obs_pack=4.910s env_step=42.062s feedback_total=22.429s feedback_obs_pack=5.368s feedback_info_pack=1.022s effective_fps=34.00 +2026-09-25 18:00:58.269 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=36866 infer_calls=36866 feedback_calls=36866 infer_wait=1070.339s infer_obs_pack=4.920s env_step=42.156s feedback_total=22.480s feedback_obs_pack=5.380s feedback_info_pack=1.024s effective_fps=32.34 +2026-09-25 18:01:28.286 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=38181 infer_calls=38181 feedback_calls=38181 infer_wait=1097.232s infer_obs_pack=5.102s env_step=43.764s feedback_total=23.328s feedback_obs_pack=5.585s feedback_info_pack=1.062s effective_fps=32.65 +2026-09-25 18:01:58.300 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=39506 infer_calls=39506 feedback_calls=39506 infer_wait=1124.040s infer_obs_pack=5.303s env_step=45.402s feedback_total=24.192s feedback_obs_pack=5.793s feedback_info_pack=1.100s effective_fps=32.95 +2026-09-25 18:02:28.317 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=40939 infer_calls=40939 feedback_calls=40939 infer_wait=1150.584s infer_obs_pack=5.504s env_step=47.197s feedback_total=25.125s feedback_obs_pack=6.022s feedback_info_pack=1.143s effective_fps=33.33 +2026-09-25 18:03:19.670 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=40961 infer_calls=40961 feedback_calls=40961 infer_wait=1201.873s infer_obs_pack=5.508s env_step=47.230s feedback_total=25.142s feedback_obs_pack=6.027s feedback_info_pack=1.143s effective_fps=32.01 +2026-09-25 18:03:49.677 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=41988 infer_calls=41988 feedback_calls=41988 infer_wait=1229.523s infer_obs_pack=5.645s env_step=48.444s feedback_total=25.776s feedback_obs_pack=6.178s feedback_info_pack=1.172s effective_fps=32.07 +2026-09-25 18:04:19.697 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=43358 infer_calls=43358 feedback_calls=43358 infer_wait=1256.219s infer_obs_pack=5.842s env_step=50.145s feedback_total=26.687s feedback_obs_pack=6.393s feedback_info_pack=1.213s effective_fps=32.38 +2026-09-25 18:04:49.706 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=44203 infer_calls=44203 feedback_calls=44203 infer_wait=1284.201s infer_obs_pack=5.956s env_step=51.201s feedback_total=27.228s feedback_obs_pack=6.524s feedback_info_pack=1.237s effective_fps=32.30 +2026-09-25 18:05:19.723 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=44866 infer_calls=44866 feedback_calls=44866 infer_wait=1312.856s infer_obs_pack=6.032s env_step=51.910s feedback_total=27.588s feedback_obs_pack=6.610s feedback_info_pack=1.254s effective_fps=32.08 diff --git a/experiments/e24-coverage/results/attempt-1/fpo-walker/client-seed2.log b/experiments/e24-coverage/results/attempt-1/fpo-walker/client-seed2.log new file mode 100644 index 0000000..d6a8b63 --- /dev/null +++ b/experiments/e24-coverage/results/attempt-1/fpo-walker/client-seed2.log @@ -0,0 +1,44 @@ +2026-09-25 17:41:50.598 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 17:41:50.603 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 17:41:50.607 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 17:41:50.611 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 17:41:50.619 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 17:41:50.819 | INFO  | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-174150-64dff1e3 output_dir=runs\mujoco-v1-nenv1-20260925-174150-64dff1e3 recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 17:41:50.954 | INFO  | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9302... +2026-09-25 17:41:50.960 | INFO  | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'FPOAlgorithm', 'policy': 'FPOPolicy', 'action_dim': 6, 'action_horizon': 1} +2026-09-25 17:42:20.971 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=3506 infer_calls=3506 feedback_calls=3506 infer_wait=21.239s infer_obs_pack=0.535s env_step=4.404s feedback_total=2.295s feedback_obs_pack=0.562s feedback_info_pack=0.099s effective_fps=123.13 +2026-09-25 17:43:41.050 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=4097 infer_calls=4097 feedback_calls=4097 infer_wait=100.010s infer_obs_pack=0.606s env_step=5.053s feedback_total=2.656s feedback_obs_pack=0.650s feedback_info_pack=0.118s effective_fps=37.82 +2026-09-25 17:44:11.070 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=4475 infer_calls=4475 feedback_calls=4475 infer_wait=129.160s infer_obs_pack=0.654s env_step=5.486s feedback_total=2.902s feedback_obs_pack=0.709s feedback_info_pack=0.130s effective_fps=32.38 +2026-09-25 17:44:41.084 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=6417 infer_calls=6417 feedback_calls=6417 infer_wait=154.705s infer_obs_pack=0.920s env_step=7.698s feedback_total=4.126s feedback_obs_pack=1.004s feedback_info_pack=0.185s effective_fps=38.32 +2026-09-25 17:45:11.086 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=8069 infer_calls=8069 feedback_calls=8069 infer_wait=180.934s infer_obs_pack=1.147s env_step=9.575s feedback_total=5.149s feedback_obs_pack=1.250s feedback_info_pack=0.230s effective_fps=41.00 +2026-09-25 17:46:19.083 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=8193 infer_calls=8193 feedback_calls=8193 infer_wait=248.682s infer_obs_pack=1.161s env_step=9.698s feedback_total=5.217s feedback_obs_pack=1.266s feedback_info_pack=0.233s effective_fps=30.95 +2026-09-25 17:46:49.101 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=9766 infer_calls=9766 feedback_calls=9766 infer_wait=275.497s infer_obs_pack=1.354s env_step=11.302s feedback_total=6.094s feedback_obs_pack=1.475s feedback_info_pack=0.272s effective_fps=33.19 +2026-09-25 17:47:19.102 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=11720 infer_calls=11720 feedback_calls=11720 infer_wait=301.103s infer_obs_pack=1.614s env_step=13.522s feedback_total=7.259s feedback_obs_pack=1.756s feedback_info_pack=0.326s effective_fps=36.23 +2026-09-25 17:48:45.615 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=12289 infer_calls=12289 feedback_calls=12289 infer_wait=386.251s infer_obs_pack=1.698s env_step=14.201s feedback_total=7.631s feedback_obs_pack=1.848s feedback_info_pack=0.342s effective_fps=29.99 +2026-09-25 17:49:15.622 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=14355 infer_calls=14355 feedback_calls=14355 infer_wait=412.053s infer_obs_pack=2.009s env_step=16.277s feedback_total=8.740s feedback_obs_pack=2.116s feedback_info_pack=0.391s effective_fps=32.69 +2026-09-25 17:49:45.631 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=16194 infer_calls=16194 feedback_calls=16194 infer_wait=438.426s infer_obs_pack=2.222s env_step=18.135s feedback_total=9.713s feedback_obs_pack=2.347s feedback_info_pack=0.435s effective_fps=34.57 +2026-09-25 17:50:18.184 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=16385 infer_calls=16385 feedback_calls=16385 infer_wait=470.625s infer_obs_pack=2.244s env_step=18.313s feedback_total=9.810s feedback_obs_pack=2.370s feedback_info_pack=0.439s effective_fps=32.71 +2026-09-25 17:50:48.213 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=18503 infer_calls=18503 feedback_calls=18503 infer_wait=495.751s infer_obs_pack=2.544s env_step=20.830s feedback_total=11.123s feedback_obs_pack=2.691s feedback_info_pack=0.499s effective_fps=34.90 +2026-09-25 17:51:18.220 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=20170 infer_calls=20170 feedback_calls=20170 infer_wait=522.089s infer_obs_pack=2.770s env_step=22.689s feedback_total=12.112s feedback_obs_pack=2.933s feedback_info_pack=0.544s effective_fps=36.04 +2026-09-25 17:52:00.539 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=20481 infer_calls=20481 feedback_calls=20481 infer_wait=563.812s infer_obs_pack=2.807s env_step=22.990s feedback_total=12.274s feedback_obs_pack=2.971s feedback_info_pack=0.551s effective_fps=34.03 +2026-09-25 17:52:30.540 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=22110 infer_calls=22110 feedback_calls=22110 infer_wait=590.364s infer_obs_pack=3.010s env_step=24.735s feedback_total=13.243s feedback_obs_pack=3.196s feedback_info_pack=0.592s effective_fps=35.02 +2026-09-25 17:53:00.573 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=23824 infer_calls=23824 feedback_calls=23824 infer_wait=616.817s infer_obs_pack=3.215s env_step=26.569s feedback_total=14.203s feedback_obs_pack=3.431s feedback_info_pack=0.634s effective_fps=36.05 +2026-09-25 17:54:02.614 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=24577 infer_calls=24577 feedback_calls=24577 infer_wait=677.325s infer_obs_pack=3.299s env_step=27.359s feedback_total=14.612s feedback_obs_pack=3.529s feedback_info_pack=0.652s effective_fps=34.01 +2026-09-25 17:54:32.619 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=25962 infer_calls=25962 feedback_calls=25962 infer_wait=704.422s infer_obs_pack=3.465s env_step=28.866s feedback_total=15.385s feedback_obs_pack=3.715s feedback_info_pack=0.685s effective_fps=34.52 +2026-09-25 17:55:02.625 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=27403 infer_calls=27403 feedback_calls=27403 infer_wait=731.389s infer_obs_pack=3.639s env_step=30.439s feedback_total=16.195s feedback_obs_pack=3.917s feedback_info_pack=0.720s effective_fps=35.06 +2026-09-25 17:56:16.050 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=28673 infer_calls=28673 feedback_calls=28673 infer_wait=802.010s infer_obs_pack=3.807s env_step=31.869s feedback_total=16.950s feedback_obs_pack=4.099s feedback_info_pack=0.753s effective_fps=33.55 +2026-09-25 17:56:46.058 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=29986 infer_calls=29986 feedback_calls=29986 infer_wait=828.856s infer_obs_pack=3.991s env_step=33.525s feedback_total=17.783s feedback_obs_pack=4.308s feedback_info_pack=0.790s effective_fps=33.91 +2026-09-25 17:57:16.058 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=31246 infer_calls=31246 feedback_calls=31246 infer_wait=855.684s infer_obs_pack=4.179s env_step=35.156s feedback_total=18.639s feedback_obs_pack=4.516s feedback_info_pack=0.827s effective_fps=34.20 +2026-09-25 17:58:31.294 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=32769 infer_calls=32769 feedback_calls=32769 infer_wait=927.375s infer_obs_pack=4.382s env_step=36.993s feedback_total=19.589s feedback_obs_pack=4.744s feedback_info_pack=0.868s effective_fps=33.16 +2026-09-25 17:59:01.313 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=34250 infer_calls=34250 feedback_calls=34250 infer_wait=953.664s infer_obs_pack=4.609s env_step=38.911s feedback_total=20.589s feedback_obs_pack=4.988s feedback_info_pack=0.913s effective_fps=33.65 +2026-09-25 17:59:31.330 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=35447 infer_calls=35447 feedback_calls=35447 infer_wait=980.855s infer_obs_pack=4.768s env_step=40.365s feedback_total=21.350s feedback_obs_pack=5.170s feedback_info_pack=0.945s effective_fps=33.84 +2026-09-25 18:00:01.350 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=36776 infer_calls=36776 feedback_calls=36776 infer_wait=1007.625s infer_obs_pack=4.964s env_step=42.027s feedback_total=22.231s feedback_obs_pack=5.383s feedback_info_pack=0.984s effective_fps=34.15 +2026-09-25 18:00:59.403 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=36865 infer_calls=36865 feedback_calls=36865 infer_wait=1065.486s infer_obs_pack=4.974s env_step=42.128s feedback_total=22.282s feedback_obs_pack=5.396s feedback_info_pack=0.987s effective_fps=32.48 +2026-09-25 18:01:29.432 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=38212 infer_calls=38212 feedback_calls=38212 infer_wait=1092.527s infer_obs_pack=5.154s env_step=43.654s feedback_total=23.089s feedback_obs_pack=5.591s feedback_info_pack=1.021s effective_fps=32.82 +2026-09-25 18:01:59.475 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=39547 infer_calls=39547 feedback_calls=39547 infer_wait=1119.233s infer_obs_pack=5.344s env_step=45.392s feedback_total=23.982s feedback_obs_pack=5.811s feedback_info_pack=1.062s effective_fps=33.12 +2026-09-25 18:02:29.501 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=40955 infer_calls=40955 feedback_calls=40955 infer_wait=1145.819s infer_obs_pack=5.551s env_step=47.169s feedback_total=24.904s feedback_obs_pack=6.036s feedback_info_pack=1.102s effective_fps=33.48 +2026-09-25 18:03:20.696 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=40961 infer_calls=40961 feedback_calls=40961 infer_wait=1197.003s infer_obs_pack=5.552s env_step=47.174s feedback_total=24.907s feedback_obs_pack=6.037s feedback_info_pack=1.102s effective_fps=32.14 +2026-09-25 18:03:50.713 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=42010 infer_calls=42010 feedback_calls=42010 infer_wait=1224.646s infer_obs_pack=5.691s env_step=48.404s feedback_total=25.538s feedback_obs_pack=6.189s feedback_info_pack=1.129s effective_fps=32.21 +2026-09-25 18:04:20.754 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=43371 infer_calls=43371 feedback_calls=43371 infer_wait=1251.565s infer_obs_pack=5.876s env_step=50.016s feedback_total=26.379s feedback_obs_pack=6.398s feedback_info_pack=1.165s effective_fps=32.52 +2026-09-25 18:04:50.775 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=44209 infer_calls=44209 feedback_calls=44209 infer_wait=1279.588s infer_obs_pack=5.995s env_step=51.041s feedback_total=26.922s feedback_obs_pack=6.529s feedback_info_pack=1.189s effective_fps=32.42 +2026-09-25 18:05:20.783 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=44866 infer_calls=44866 feedback_calls=44866 infer_wait=1308.262s infer_obs_pack=6.071s env_step=51.734s feedback_total=27.281s feedback_obs_pack=6.616s feedback_info_pack=1.205s effective_fps=32.20 diff --git a/experiments/e24-coverage/results/attempt-1/fpo-walker/server-seed0.log b/experiments/e24-coverage/results/attempt-1/fpo-walker/server-seed0.log new file mode 100644 index 0000000..bebdfce --- /dev/null +++ b/experiments/e24-coverage/results/attempt-1/fpo-walker/server-seed0.log @@ -0,0 +1,11 @@ +17:41:37|INFO|plugrl_server version: 0.1.0 +17:41:37|INFO|Algorithm: fpo, Config: FPOAlgoConfig(global_steps=409600, buffer_size=4096, output_mode='u_but_supervise_as_eps', fpo_playground_trick=True, treat_truncated_as_done=False, discounting=0.995, reward_scaling=10.0, gae_lambda=0.95, batch_size=1024, num_updates_per_batch=16, learning_rate=0.0003, value_loss_coeff=0.25, clipping_epsilon=0.05, normalize_advantage=True, n_critic_warmup_itrs=0, max_policy_drift=0.0, n_samples_per_action=8, discretize_t_for_training=True, average_losses_before_exp=True, save_interval=20, policy_checkpoint_path=None, restore='all', master_weights_device=None) +17:41:37|INFO|Policy: fpo-policy, Config: FPOPolicyConfig(algo='fpo', device=device(type='cpu'), obs_dim=17, action_dim=6, flow_steps=10, timestep_embed_dim=8, action_horizon=1, feather_std=0.0, policy_mlp_output_scale=0.25, normalize_observations=True, hidden_dims=(32, 32, 32, 32), value_hidden_dims=(256, 256, 256, 256, 256)) +17:41:37|INFO|Seeded python, numpy and torch with 0. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +17:41:37|INFO|Checkpoint Manager created: + at D:\75128\Desktop\plugrl-work\plugrl-server\.claude\worktrees\fix-pi0-image-mask-batching\experiments\e24-coverage\results\fpo-walker\fpo\fpo-policy\fpo-walker-seed0 +17:41:38|INFO|Policy created... +17:41:38|INFO|Initialized RolloutBuffer buffer_size=4096 action_shape=(4096, 1, 6) value_shape=(4096, 1) +17:41:38|INFO|Algorithm created: + +17:41:38|INFO|Agent Server is listening on 0.0.0.0:9300 diff --git a/experiments/e24-coverage/results/attempt-1/fpo-walker/server-seed1.log b/experiments/e24-coverage/results/attempt-1/fpo-walker/server-seed1.log new file mode 100644 index 0000000..f14e97e --- /dev/null +++ b/experiments/e24-coverage/results/attempt-1/fpo-walker/server-seed1.log @@ -0,0 +1,11 @@ +17:41:41|INFO|plugrl_server version: 0.1.0 +17:41:41|INFO|Algorithm: fpo, Config: FPOAlgoConfig(global_steps=409600, buffer_size=4096, output_mode='u_but_supervise_as_eps', fpo_playground_trick=True, treat_truncated_as_done=False, discounting=0.995, reward_scaling=10.0, gae_lambda=0.95, batch_size=1024, num_updates_per_batch=16, learning_rate=0.0003, value_loss_coeff=0.25, clipping_epsilon=0.05, normalize_advantage=True, n_critic_warmup_itrs=0, max_policy_drift=0.0, n_samples_per_action=8, discretize_t_for_training=True, average_losses_before_exp=True, save_interval=20, policy_checkpoint_path=None, restore='all', master_weights_device=None) +17:41:41|INFO|Policy: fpo-policy, Config: FPOPolicyConfig(algo='fpo', device=device(type='cpu'), obs_dim=17, action_dim=6, flow_steps=10, timestep_embed_dim=8, action_horizon=1, feather_std=0.0, policy_mlp_output_scale=0.25, normalize_observations=True, hidden_dims=(32, 32, 32, 32), value_hidden_dims=(256, 256, 256, 256, 256)) +17:41:41|INFO|Seeded python, numpy and torch with 1. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +17:41:41|INFO|Checkpoint Manager created: + at D:\75128\Desktop\plugrl-work\plugrl-server\.claude\worktrees\fix-pi0-image-mask-batching\experiments\e24-coverage\results\fpo-walker\fpo\fpo-policy\fpo-walker-seed1 +17:41:42|INFO|Policy created... +17:41:42|INFO|Initialized RolloutBuffer buffer_size=4096 action_shape=(4096, 1, 6) value_shape=(4096, 1) +17:41:42|INFO|Algorithm created: + +17:41:42|INFO|Agent Server is listening on 0.0.0.0:9301 diff --git a/experiments/e24-coverage/results/attempt-1/fpo-walker/server-seed2.log b/experiments/e24-coverage/results/attempt-1/fpo-walker/server-seed2.log new file mode 100644 index 0000000..e6ceecd --- /dev/null +++ b/experiments/e24-coverage/results/attempt-1/fpo-walker/server-seed2.log @@ -0,0 +1,11 @@ +17:41:47|INFO|plugrl_server version: 0.1.0 +17:41:47|INFO|Algorithm: fpo, Config: FPOAlgoConfig(global_steps=409600, buffer_size=4096, output_mode='u_but_supervise_as_eps', fpo_playground_trick=True, treat_truncated_as_done=False, discounting=0.995, reward_scaling=10.0, gae_lambda=0.95, batch_size=1024, num_updates_per_batch=16, learning_rate=0.0003, value_loss_coeff=0.25, clipping_epsilon=0.05, normalize_advantage=True, n_critic_warmup_itrs=0, max_policy_drift=0.0, n_samples_per_action=8, discretize_t_for_training=True, average_losses_before_exp=True, save_interval=20, policy_checkpoint_path=None, restore='all', master_weights_device=None) +17:41:47|INFO|Policy: fpo-policy, Config: FPOPolicyConfig(algo='fpo', device=device(type='cpu'), obs_dim=17, action_dim=6, flow_steps=10, timestep_embed_dim=8, action_horizon=1, feather_std=0.0, policy_mlp_output_scale=0.25, normalize_observations=True, hidden_dims=(32, 32, 32, 32), value_hidden_dims=(256, 256, 256, 256, 256)) +17:41:47|INFO|Seeded python, numpy and torch with 2. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +17:41:47|INFO|Checkpoint Manager created: + at D:\75128\Desktop\plugrl-work\plugrl-server\.claude\worktrees\fix-pi0-image-mask-batching\experiments\e24-coverage\results\fpo-walker\fpo\fpo-policy\fpo-walker-seed2 +17:41:49|INFO|Policy created... +17:41:49|INFO|Initialized RolloutBuffer buffer_size=4096 action_shape=(4096, 1, 6) value_shape=(4096, 1) +17:41:49|INFO|Algorithm created: + +17:41:49|INFO|Agent Server is listening on 0.0.0.0:9302 diff --git a/experiments/e24-coverage/results/attempt-1/fpodppo-hopper.out b/experiments/e24-coverage/results/attempt-1/fpodppo-hopper.out new file mode 100644 index 0000000..620a898 --- /dev/null +++ b/experiments/e24-coverage/results/attempt-1/fpodppo-hopper.out @@ -0,0 +1,4 @@ +cell: fpodppo-hopper = fpo-policy/default x dppo/hopper x Hopper-v5 +iters: 20 x 4096 batch: variant default replan: 1 seeds: 0 1 2 +out: /d/75128/Desktop/plugrl-work/plugrl-server/.claude/worktrees/fix-pi0-image-mask-batching/experiments/e24-coverage/results/fpodppo-hopper +start 2026-09-25 17:41:43 diff --git a/experiments/e24-coverage/results/attempt-1/fpodppo-hopper/client-seed0.log b/experiments/e24-coverage/results/attempt-1/fpodppo-hopper/client-seed0.log new file mode 100644 index 0000000..e34dfa1 --- /dev/null +++ b/experiments/e24-coverage/results/attempt-1/fpodppo-hopper/client-seed0.log @@ -0,0 +1,54 @@ +2026-09-25 17:42:15.614 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 17:42:15.617 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 17:42:15.620 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 17:42:15.627 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 17:42:15.636 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 17:42:15.824 | INFO  | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-174215-167b25c1 output_dir=runs\mujoco-v1-nenv1-20260925-174215-167b25c1 recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 17:42:15.984 | INFO  | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9310... +2026-09-25 17:42:15.990 | INFO  | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'FPOPolicy', 'action_dim': 3, 'action_horizon': 1} +2026-09-25 17:42:45.996 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=2611 infer_calls=2611 feedback_calls=2611 infer_wait=23.603s infer_obs_pack=0.369s env_step=3.149s feedback_total=1.770s feedback_obs_pack=0.395s feedback_info_pack=0.071s effective_fps=90.37 +2026-09-25 17:43:26.325 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=4097 infer_calls=4097 feedback_calls=4097 infer_wait=59.577s infer_obs_pack=0.641s env_step=5.318s feedback_total=2.923s feedback_obs_pack=0.667s feedback_info_pack=0.122s effective_fps=59.85 +2026-09-25 17:43:56.350 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=4551 infer_calls=4551 feedback_calls=4551 infer_wait=88.491s infer_obs_pack=0.707s env_step=5.882s feedback_total=3.221s feedback_obs_pack=0.738s feedback_info_pack=0.137s effective_fps=46.30 +2026-09-25 17:44:26.372 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=5465 infer_calls=5465 feedback_calls=5465 infer_wait=116.397s infer_obs_pack=0.830s env_step=6.921s feedback_total=3.790s feedback_obs_pack=0.872s feedback_info_pack=0.163s effective_fps=42.72 +2026-09-25 17:44:56.381 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=7008 infer_calls=7008 feedback_calls=7008 infer_wait=142.870s infer_obs_pack=1.045s env_step=8.687s feedback_total=4.758s feedback_obs_pack=1.106s feedback_info_pack=0.208s effective_fps=44.53 +2026-09-25 17:45:26.441 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=8091 infer_calls=8091 feedback_calls=8091 infer_wait=170.468s infer_obs_pack=1.177s env_step=9.934s feedback_total=5.422s feedback_obs_pack=1.262s feedback_info_pack=0.239s effective_fps=43.27 +2026-09-25 17:45:56.759 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=8452 infer_calls=8452 feedback_calls=8452 infer_wait=199.998s infer_obs_pack=1.220s env_step=10.331s feedback_total=5.640s feedback_obs_pack=1.314s feedback_info_pack=0.249s effective_fps=38.92 +2026-09-25 17:46:26.774 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=9399 infer_calls=9399 feedback_calls=9399 infer_wait=227.927s infer_obs_pack=1.341s env_step=11.356s feedback_total=6.210s feedback_obs_pack=1.448s feedback_info_pack=0.277s effective_fps=38.08 +2026-09-25 17:46:56.776 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=10654 infer_calls=10654 feedback_calls=10654 infer_wait=255.221s infer_obs_pack=1.491s env_step=12.728s feedback_total=6.927s feedback_obs_pack=1.619s feedback_info_pack=0.312s effective_fps=38.55 +2026-09-25 17:47:26.790 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=12208 infer_calls=12208 feedback_calls=12208 infer_wait=281.624s infer_obs_pack=1.700s env_step=14.531s feedback_total=7.906s feedback_obs_pack=1.855s feedback_info_pack=0.359s effective_fps=39.93 +2026-09-25 17:47:56.821 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=12291 infer_calls=12291 feedback_calls=12291 infer_wait=311.432s infer_obs_pack=1.712s env_step=14.646s feedback_total=7.963s feedback_obs_pack=1.869s feedback_info_pack=0.362s effective_fps=36.61 +2026-09-25 17:48:26.824 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=12796 infer_calls=12796 feedback_calls=12796 infer_wait=340.222s infer_obs_pack=1.859s env_step=15.209s feedback_total=8.270s feedback_obs_pack=1.940s feedback_info_pack=0.376s effective_fps=35.00 +2026-09-25 17:48:56.830 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=14242 infer_calls=14242 feedback_calls=14242 infer_wait=367.117s infer_obs_pack=2.042s env_step=16.748s feedback_total=9.121s feedback_obs_pack=2.146s feedback_info_pack=0.416s effective_fps=36.05 +2026-09-25 17:49:26.849 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=15640 infer_calls=15640 feedback_calls=15640 infer_wait=394.322s infer_obs_pack=2.206s env_step=18.149s feedback_total=9.876s feedback_obs_pack=2.330s feedback_info_pack=0.452s effective_fps=36.84 +2026-09-25 17:49:56.852 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=16853 infer_calls=16853 feedback_calls=16853 infer_wait=421.983s infer_obs_pack=2.340s env_step=19.315s feedback_total=10.510s feedback_obs_pack=2.484s feedback_info_pack=0.481s effective_fps=37.11 +2026-09-25 17:50:26.854 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=18305 infer_calls=18305 feedback_calls=18305 infer_wait=448.837s infer_obs_pack=2.528s env_step=20.870s feedback_total=11.373s feedback_obs_pack=2.692s feedback_info_pack=0.522s effective_fps=37.85 +2026-09-25 17:50:56.854 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=19855 infer_calls=19855 feedback_calls=19855 infer_wait=475.460s infer_obs_pack=2.729s env_step=22.591s feedback_total=12.261s feedback_obs_pack=2.908s feedback_info_pack=0.565s effective_fps=38.70 +2026-09-25 17:51:26.864 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=20944 infer_calls=20944 feedback_calls=20944 infer_wait=503.104s infer_obs_pack=2.866s env_step=23.780s feedback_total=12.898s feedback_obs_pack=3.063s feedback_info_pack=0.595s effective_fps=38.60 +2026-09-25 17:51:56.895 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=22147 infer_calls=22147 feedback_calls=22147 infer_wait=530.714s infer_obs_pack=3.002s env_step=24.984s feedback_total=13.553s feedback_obs_pack=3.218s feedback_info_pack=0.626s effective_fps=38.70 +2026-09-25 17:52:26.909 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=23375 infer_calls=23375 feedback_calls=23375 infer_wait=558.100s infer_obs_pack=3.154s env_step=26.310s feedback_total=14.254s feedback_obs_pack=3.388s feedback_info_pack=0.660s effective_fps=38.84 +2026-09-25 17:52:57.294 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=24577 infer_calls=24577 feedback_calls=24577 infer_wait=585.930s infer_obs_pack=3.301s env_step=27.583s feedback_total=14.958s feedback_obs_pack=3.560s feedback_info_pack=0.694s effective_fps=38.90 +2026-09-25 17:53:27.324 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=25468 infer_calls=25468 feedback_calls=25468 infer_wait=614.137s infer_obs_pack=3.404s env_step=28.483s feedback_total=15.450s feedback_obs_pack=3.678s feedback_info_pack=0.717s effective_fps=38.50 +2026-09-25 17:53:57.337 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=26530 infer_calls=26530 feedback_calls=26530 infer_wait=642.225s infer_obs_pack=3.512s env_step=29.426s feedback_total=15.984s feedback_obs_pack=3.801s feedback_info_pack=0.741s effective_fps=38.39 +2026-09-25 17:54:27.370 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=27574 infer_calls=27574 feedback_calls=27574 infer_wait=670.181s infer_obs_pack=3.632s env_step=30.462s feedback_total=16.549s feedback_obs_pack=3.934s feedback_info_pack=0.767s effective_fps=38.25 +2026-09-25 17:55:01.237 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=28673 infer_calls=28673 feedback_calls=28673 infer_wait=701.722s infer_obs_pack=3.771s env_step=31.615s feedback_total=17.181s feedback_obs_pack=4.083s feedback_info_pack=0.796s effective_fps=38.01 +2026-09-25 17:55:31.242 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=29799 infer_calls=29799 feedback_calls=29799 infer_wait=729.182s infer_obs_pack=3.919s env_step=32.875s feedback_total=17.871s feedback_obs_pack=4.252s feedback_info_pack=0.828s effective_fps=38.02 +2026-09-25 17:56:01.275 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=30823 infer_calls=30823 feedback_calls=30823 infer_wait=756.965s infer_obs_pack=4.046s env_step=33.995s feedback_total=18.482s feedback_obs_pack=4.397s feedback_info_pack=0.857s effective_fps=37.89 +2026-09-25 17:56:31.281 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=31899 infer_calls=31899 feedback_calls=31899 infer_wait=784.570s infer_obs_pack=4.183s env_step=35.194s feedback_total=19.134s feedback_obs_pack=4.553s feedback_info_pack=0.888s effective_fps=37.84 +2026-09-25 17:57:04.118 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=32769 infer_calls=32769 feedback_calls=32769 infer_wait=815.200s infer_obs_pack=4.317s env_step=36.297s feedback_total=19.733s feedback_obs_pack=4.694s feedback_info_pack=0.917s effective_fps=37.43 +2026-09-25 17:57:34.150 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=33848 infer_calls=33848 feedback_calls=33848 infer_wait=842.550s infer_obs_pack=4.478s env_step=37.636s feedback_total=20.455s feedback_obs_pack=4.868s feedback_info_pack=0.952s effective_fps=37.40 +2026-09-25 17:58:04.192 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=35071 infer_calls=35071 feedback_calls=35071 infer_wait=869.813s infer_obs_pack=4.643s env_step=39.003s feedback_total=21.220s feedback_obs_pack=5.055s feedback_info_pack=0.988s effective_fps=37.52 +2026-09-25 17:58:34.210 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=36133 infer_calls=36133 feedback_calls=36133 infer_wait=897.237s infer_obs_pack=4.798s env_step=40.281s feedback_total=21.938s feedback_obs_pack=5.223s feedback_info_pack=1.022s effective_fps=37.47 +2026-09-25 17:59:04.232 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=37014 infer_calls=37014 feedback_calls=37014 infer_wait=925.073s infer_obs_pack=4.925s env_step=41.378s feedback_total=22.527s feedback_obs_pack=5.367s feedback_info_pack=1.050s effective_fps=37.24 +2026-09-25 17:59:34.244 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=38045 infer_calls=38045 feedback_calls=38045 infer_wait=952.572s infer_obs_pack=5.067s env_step=42.653s feedback_total=23.193s feedback_obs_pack=5.525s feedback_info_pack=1.082s effective_fps=37.17 +2026-09-25 18:00:04.292 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=39183 infer_calls=39183 feedback_calls=39183 infer_wait=979.929s infer_obs_pack=5.225s env_step=43.986s feedback_total=23.936s feedback_obs_pack=5.703s feedback_info_pack=1.120s effective_fps=37.21 +2026-09-25 18:00:34.297 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=40001 infer_calls=40001 feedback_calls=40001 infer_wait=1008.039s infer_obs_pack=5.334s env_step=44.930s feedback_total=24.454s feedback_obs_pack=5.826s feedback_info_pack=1.143s effective_fps=36.94 +2026-09-25 18:01:09.539 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=40961 infer_calls=40961 feedback_calls=40961 infer_wait=1041.090s infer_obs_pack=5.461s env_step=46.026s feedback_total=25.043s feedback_obs_pack=5.965s feedback_info_pack=1.171s effective_fps=36.65 +2026-09-25 18:01:39.541 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=42030 infer_calls=42030 feedback_calls=42030 infer_wait=1068.641s infer_obs_pack=5.602s env_step=47.243s feedback_total=25.705s feedback_obs_pack=6.124s feedback_info_pack=1.202s effective_fps=36.64 +2026-09-25 18:02:09.546 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=43171 infer_calls=43171 feedback_calls=43171 infer_wait=1095.791s infer_obs_pack=5.771s env_step=48.674s feedback_total=26.478s feedback_obs_pack=6.310s feedback_info_pack=1.239s effective_fps=36.69 +2026-09-25 18:02:39.559 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=44410 infer_calls=44410 feedback_calls=44410 infer_wait=1122.958s infer_obs_pack=5.936s env_step=50.113s feedback_total=27.239s feedback_obs_pack=6.499s feedback_info_pack=1.275s effective_fps=36.82 +2026-09-25 18:03:13.264 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=45058 infer_calls=45058 feedback_calls=45058 infer_wait=1155.255s infer_obs_pack=6.011s env_step=50.823s feedback_total=27.620s feedback_obs_pack=6.590s feedback_info_pack=1.292s effective_fps=36.35 +2026-09-25 18:03:43.295 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=45999 infer_calls=45999 feedback_calls=45999 infer_wait=1183.228s infer_obs_pack=6.131s env_step=51.836s feedback_total=28.188s feedback_obs_pack=6.724s feedback_info_pack=1.319s effective_fps=36.24 +2026-09-25 18:04:13.309 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=47049 infer_calls=47049 feedback_calls=47049 infer_wait=1210.861s infer_obs_pack=6.273s env_step=53.002s feedback_total=28.857s feedback_obs_pack=6.885s feedback_info_pack=1.351s effective_fps=36.22 +2026-09-25 18:04:43.336 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=47935 infer_calls=47935 feedback_calls=47935 infer_wait=1238.770s infer_obs_pack=6.399s env_step=54.054s feedback_total=29.431s feedback_obs_pack=7.024s feedback_info_pack=1.378s effective_fps=36.08 +2026-09-25 18:05:13.370 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=48496 infer_calls=48496 feedback_calls=48496 infer_wait=1267.646s infer_obs_pack=6.467s env_step=54.627s feedback_total=29.743s feedback_obs_pack=7.097s feedback_info_pack=1.393s effective_fps=35.70 +2026-09-25 18:05:56.961 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=49153 infer_calls=49153 feedback_calls=49153 infer_wait=1309.900s infer_obs_pack=6.546s env_step=55.288s feedback_total=30.106s feedback_obs_pack=7.184s feedback_info_pack=1.410s effective_fps=35.06 diff --git a/experiments/e24-coverage/results/attempt-1/fpodppo-hopper/client-seed1.log b/experiments/e24-coverage/results/attempt-1/fpodppo-hopper/client-seed1.log new file mode 100644 index 0000000..10036bf --- /dev/null +++ b/experiments/e24-coverage/results/attempt-1/fpodppo-hopper/client-seed1.log @@ -0,0 +1,52 @@ +2026-09-25 17:42:28.224 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 17:42:28.226 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 17:42:28.228 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 17:42:28.232 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 17:42:28.241 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 17:42:28.430 | INFO  | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-174228-8189717b output_dir=runs\mujoco-v1-nenv1-20260925-174228-8189717b recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 17:42:28.698 | INFO  | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9311... +2026-09-25 17:42:28.713 | INFO  | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'FPOPolicy', 'action_dim': 3, 'action_horizon': 1} +2026-09-25 17:42:58.725 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=2491 infer_calls=2491 feedback_calls=2491 infer_wait=23.369s infer_obs_pack=0.388s env_step=3.310s feedback_total=1.782s feedback_obs_pack=0.393s feedback_info_pack=0.077s effective_fps=86.35 +2026-09-25 17:43:35.137 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=4097 infer_calls=4097 feedback_calls=4097 infer_wait=54.797s infer_obs_pack=0.655s env_step=5.724s feedback_total=3.229s feedback_obs_pack=0.731s feedback_info_pack=0.139s effective_fps=63.61 +2026-09-25 17:44:05.484 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=4412 infer_calls=4412 feedback_calls=4412 infer_wait=84.430s infer_obs_pack=0.696s env_step=6.073s feedback_total=3.430s feedback_obs_pack=0.778s feedback_info_pack=0.149s effective_fps=46.62 +2026-09-25 17:44:35.521 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=5812 infer_calls=5812 feedback_calls=5812 infer_wait=111.248s infer_obs_pack=0.883s env_step=7.677s feedback_total=4.310s feedback_obs_pack=0.983s feedback_info_pack=0.189s effective_fps=46.83 +2026-09-25 17:45:05.521 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=7198 infer_calls=7198 feedback_calls=7198 infer_wait=138.076s infer_obs_pack=1.070s env_step=9.244s feedback_total=5.187s feedback_obs_pack=1.192s feedback_info_pack=0.228s effective_fps=46.87 +2026-09-25 17:45:48.592 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=8193 infer_calls=8193 feedback_calls=8193 infer_wait=178.899s infer_obs_pack=1.198s env_step=10.349s feedback_total=5.816s feedback_obs_pack=1.336s feedback_info_pack=0.258s effective_fps=41.75 +2026-09-25 17:46:18.617 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=8874 infer_calls=8874 feedback_calls=8874 infer_wait=207.443s infer_obs_pack=1.279s env_step=11.077s feedback_total=6.236s feedback_obs_pack=1.433s feedback_info_pack=0.276s effective_fps=39.26 +2026-09-25 17:46:48.623 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=10099 infer_calls=10099 feedback_calls=10099 infer_wait=234.917s infer_obs_pack=1.430s env_step=12.320s feedback_total=6.931s feedback_obs_pack=1.595s feedback_info_pack=0.306s effective_fps=39.51 +2026-09-25 17:47:18.665 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=11638 infer_calls=11638 feedback_calls=11638 infer_wait=261.537s infer_obs_pack=1.633s env_step=14.018s feedback_total=7.853s feedback_obs_pack=1.814s feedback_info_pack=0.348s effective_fps=40.83 +2026-09-25 17:48:17.291 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=12290 infer_calls=12290 feedback_calls=12290 infer_wait=318.606s infer_obs_pack=1.721s env_step=14.796s feedback_total=8.275s feedback_obs_pack=1.916s feedback_info_pack=0.368s effective_fps=35.79 +2026-09-25 17:48:47.292 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=13458 infer_calls=13458 feedback_calls=13458 infer_wait=345.855s infer_obs_pack=1.947s env_step=16.110s feedback_total=9.023s feedback_obs_pack=2.094s feedback_info_pack=0.400s effective_fps=36.09 +2026-09-25 17:49:17.300 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=14967 infer_calls=14967 feedback_calls=14967 infer_wait=372.854s infer_obs_pack=2.117s env_step=17.610s feedback_total=9.843s feedback_obs_pack=2.290s feedback_info_pack=0.437s effective_fps=37.19 +2026-09-25 17:49:51.734 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=16385 infer_calls=16385 feedback_calls=16385 infer_wait=404.410s infer_obs_pack=2.289s env_step=19.038s feedback_total=10.630s feedback_obs_pack=2.478s feedback_info_pack=0.473s effective_fps=37.55 +2026-09-25 17:50:21.754 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=17851 infer_calls=17851 feedback_calls=17851 infer_wait=431.277s infer_obs_pack=2.474s env_step=20.608s feedback_total=11.489s feedback_obs_pack=2.683s feedback_info_pack=0.513s effective_fps=38.32 +2026-09-25 17:50:51.757 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=19410 infer_calls=19410 feedback_calls=19410 infer_wait=457.796s infer_obs_pack=2.684s env_step=22.350s feedback_total=12.435s feedback_obs_pack=2.915s feedback_info_pack=0.555s effective_fps=39.19 +2026-09-25 17:51:21.765 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=20502 infer_calls=20502 feedback_calls=20502 infer_wait=485.306s infer_obs_pack=2.833s env_step=23.605s feedback_total=13.112s feedback_obs_pack=3.080s feedback_info_pack=0.586s effective_fps=39.06 +2026-09-25 17:51:51.772 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=21777 infer_calls=21777 feedback_calls=21777 infer_wait=512.808s infer_obs_pack=2.971s env_step=24.855s feedback_total=13.801s feedback_obs_pack=3.238s feedback_info_pack=0.616s effective_fps=39.28 +2026-09-25 17:52:21.780 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=22914 infer_calls=22914 feedback_calls=22914 infer_wait=540.321s infer_obs_pack=3.111s env_step=26.126s feedback_total=14.461s feedback_obs_pack=3.400s feedback_info_pack=0.646s effective_fps=39.23 +2026-09-25 17:52:51.841 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=24353 infer_calls=24353 feedback_calls=24353 infer_wait=567.479s infer_obs_pack=3.279s env_step=27.574s feedback_total=15.256s feedback_obs_pack=3.588s feedback_info_pack=0.683s effective_fps=39.69 +2026-09-25 17:53:21.896 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=25096 infer_calls=25096 feedback_calls=25096 infer_wait=595.898s infer_obs_pack=3.368s env_step=28.401s feedback_total=15.694s feedback_obs_pack=3.693s feedback_info_pack=0.703s effective_fps=39.01 +2026-09-25 17:53:51.906 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=26051 infer_calls=26051 feedback_calls=26051 infer_wait=624.220s infer_obs_pack=3.459s env_step=29.237s feedback_total=16.163s feedback_obs_pack=3.798s feedback_info_pack=0.723s effective_fps=38.70 +2026-09-25 17:54:21.934 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=27040 infer_calls=27040 feedback_calls=27040 infer_wait=652.305s infer_obs_pack=3.565s env_step=30.213s feedback_total=16.694s feedback_obs_pack=3.924s feedback_info_pack=0.746s effective_fps=38.48 +2026-09-25 17:54:51.961 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=28398 infer_calls=28398 feedback_calls=28398 infer_wait=679.481s infer_obs_pack=3.733s env_step=31.622s feedback_total=17.476s feedback_obs_pack=4.114s feedback_info_pack=0.784s effective_fps=38.78 +2026-09-25 17:55:22.000 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=29175 infer_calls=29175 feedback_calls=29175 infer_wait=707.779s infer_obs_pack=3.837s env_step=32.493s feedback_total=17.945s feedback_obs_pack=4.226s feedback_info_pack=0.805s effective_fps=38.28 +2026-09-25 17:55:52.016 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=30238 infer_calls=30238 feedback_calls=30238 infer_wait=735.410s infer_obs_pack=3.975s env_step=33.665s feedback_total=18.597s feedback_obs_pack=4.379s feedback_info_pack=0.834s effective_fps=38.20 +2026-09-25 17:56:22.027 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=31352 infer_calls=31352 feedback_calls=31352 infer_wait=762.956s infer_obs_pack=4.116s env_step=34.904s feedback_total=19.265s feedback_obs_pack=4.541s feedback_info_pack=0.864s effective_fps=38.18 +2026-09-25 17:56:52.035 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=32445 infer_calls=32445 feedback_calls=32445 infer_wait=790.309s infer_obs_pack=4.273s env_step=36.229s feedback_total=19.978s feedback_obs_pack=4.711s feedback_info_pack=0.898s effective_fps=38.14 +2026-09-25 17:57:22.057 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=33162 infer_calls=33162 feedback_calls=33162 infer_wait=818.639s infer_obs_pack=4.374s env_step=37.074s feedback_total=20.437s feedback_obs_pack=4.823s feedback_info_pack=0.919s effective_fps=37.66 +2026-09-25 17:57:52.071 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=34475 infer_calls=34475 feedback_calls=34475 infer_wait=845.666s infer_obs_pack=4.547s env_step=38.555s feedback_total=21.256s feedback_obs_pack=5.023s feedback_info_pack=0.956s effective_fps=37.88 +2026-09-25 17:58:22.082 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=35509 infer_calls=35509 feedback_calls=35509 infer_wait=873.290s infer_obs_pack=4.681s env_step=39.737s feedback_total=21.919s feedback_obs_pack=5.179s feedback_info_pack=0.986s effective_fps=37.79 +2026-09-25 17:58:52.097 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=36707 infer_calls=36707 feedback_calls=36707 infer_wait=900.336s infer_obs_pack=4.851s env_step=41.236s feedback_total=22.715s feedback_obs_pack=5.368s feedback_info_pack=1.023s effective_fps=37.88 +2026-09-25 17:59:22.122 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=37477 infer_calls=37477 feedback_calls=37477 infer_wait=928.471s infer_obs_pack=4.958s env_step=42.180s feedback_total=23.229s feedback_obs_pack=5.488s feedback_info_pack=1.047s effective_fps=37.52 +2026-09-25 17:59:52.131 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=38571 infer_calls=38571 feedback_calls=38571 infer_wait=955.822s infer_obs_pack=5.119s env_step=43.497s feedback_total=23.956s feedback_obs_pack=5.661s feedback_info_pack=1.081s effective_fps=37.51 +2026-09-25 18:00:22.149 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=39452 infer_calls=39452 feedback_calls=39452 infer_wait=983.662s infer_obs_pack=5.249s env_step=44.577s feedback_total=24.555s feedback_obs_pack=5.805s feedback_info_pack=1.108s effective_fps=37.29 +2026-09-25 18:00:52.151 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=40420 infer_calls=40420 feedback_calls=40420 infer_wait=1011.434s infer_obs_pack=5.371s env_step=45.687s feedback_total=25.168s feedback_obs_pack=5.949s feedback_info_pack=1.136s effective_fps=37.16 +2026-09-25 18:01:22.187 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=41255 infer_calls=41255 feedback_calls=41255 infer_wait=1039.527s infer_obs_pack=5.487s env_step=46.631s feedback_total=25.705s feedback_obs_pack=6.083s feedback_info_pack=1.164s effective_fps=36.92 +2026-09-25 18:01:52.235 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=42395 infer_calls=42395 feedback_calls=42395 infer_wait=1066.816s infer_obs_pack=5.655s env_step=47.988s feedback_total=26.459s feedback_obs_pack=6.263s feedback_info_pack=1.198s effective_fps=36.96 +2026-09-25 18:02:22.245 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=43584 infer_calls=43584 feedback_calls=43584 infer_wait=1093.883s infer_obs_pack=5.829s env_step=49.452s feedback_total=27.260s feedback_obs_pack=6.457s feedback_info_pack=1.236s effective_fps=37.05 +2026-09-25 18:02:52.246 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=44734 infer_calls=44734 feedback_calls=44734 infer_wait=1121.571s infer_obs_pack=5.959s env_step=50.600s feedback_total=27.893s feedback_obs_pack=6.606s feedback_info_pack=1.265s effective_fps=37.09 +2026-09-25 18:03:22.255 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=45269 infer_calls=45269 feedback_calls=45269 infer_wait=1150.386s infer_obs_pack=6.027s env_step=51.188s feedback_total=28.227s feedback_obs_pack=6.684s feedback_info_pack=1.280s effective_fps=36.63 +2026-09-25 18:03:52.258 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=46195 infer_calls=46195 feedback_calls=46195 infer_wait=1178.307s infer_obs_pack=6.150s env_step=52.224s feedback_total=28.795s feedback_obs_pack=6.818s feedback_info_pack=1.306s effective_fps=36.50 +2026-09-25 18:04:22.275 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=47293 infer_calls=47293 feedback_calls=47293 infer_wait=1205.727s infer_obs_pack=6.303s env_step=53.508s feedback_total=29.507s feedback_obs_pack=6.992s feedback_info_pack=1.339s effective_fps=36.52 +2026-09-25 18:04:52.297 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=48008 infer_calls=48008 feedback_calls=48008 infer_wait=1234.097s infer_obs_pack=6.399s env_step=54.320s feedback_total=29.962s feedback_obs_pack=7.101s feedback_info_pack=1.360s effective_fps=36.24 +2026-09-25 18:05:22.305 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=48605 infer_calls=48605 feedback_calls=48605 infer_wait=1262.904s infer_obs_pack=6.471s env_step=54.913s feedback_total=30.292s feedback_obs_pack=7.179s feedback_info_pack=1.375s effective_fps=35.88 diff --git a/experiments/e24-coverage/results/attempt-1/fpodppo-hopper/client-seed2.log b/experiments/e24-coverage/results/attempt-1/fpodppo-hopper/client-seed2.log new file mode 100644 index 0000000..68b7c6d --- /dev/null +++ b/experiments/e24-coverage/results/attempt-1/fpodppo-hopper/client-seed2.log @@ -0,0 +1,53 @@ +2026-09-25 17:42:34.943 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 17:42:34.948 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 17:42:34.952 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 17:42:34.957 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 17:42:34.964 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 17:42:35.154 | INFO  | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-174235-233730fb output_dir=runs\mujoco-v1-nenv1-20260925-174235-233730fb recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 17:42:35.368 | INFO  | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9312... +2026-09-25 17:42:35.385 | INFO  | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'FPOPolicy', 'action_dim': 3, 'action_horizon': 1} +2026-09-25 17:43:05.428 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=2324 infer_calls=2324 feedback_calls=2324 infer_wait=23.650s infer_obs_pack=0.392s env_step=3.140s feedback_total=1.720s feedback_obs_pack=0.413s feedback_info_pack=0.077s effective_fps=80.41 +2026-09-25 17:43:45.175 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=4097 infer_calls=4097 feedback_calls=4097 infer_wait=58.161s infer_obs_pack=0.686s env_step=5.760s feedback_total=3.070s feedback_obs_pack=0.738s feedback_info_pack=0.139s effective_fps=60.54 +2026-09-25 17:44:15.208 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=4557 infer_calls=4557 feedback_calls=4557 infer_wait=87.087s infer_obs_pack=0.748s env_step=6.318s feedback_total=3.369s feedback_obs_pack=0.811s feedback_info_pack=0.153s effective_fps=46.73 +2026-09-25 17:44:45.211 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=6076 infer_calls=6076 feedback_calls=6076 infer_wait=113.676s infer_obs_pack=0.947s env_step=8.024s feedback_total=4.293s feedback_obs_pack=1.032s feedback_info_pack=0.195s effective_fps=47.87 +2026-09-25 17:45:15.252 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=7383 infer_calls=7383 feedback_calls=7383 infer_wait=140.661s infer_obs_pack=1.128s env_step=9.527s feedback_total=5.137s feedback_obs_pack=1.233s feedback_info_pack=0.234s effective_fps=47.19 +2026-09-25 17:46:02.499 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=8193 infer_calls=8193 feedback_calls=8193 infer_wait=186.119s infer_obs_pack=1.229s env_step=10.414s feedback_total=5.629s feedback_obs_pack=1.350s feedback_info_pack=0.255s effective_fps=40.28 +2026-09-25 17:46:32.508 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=9226 infer_calls=9226 feedback_calls=9226 infer_wait=213.899s infer_obs_pack=1.363s env_step=11.509s feedback_total=6.242s feedback_obs_pack=1.497s feedback_info_pack=0.282s effective_fps=39.59 +2026-09-25 17:47:02.517 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=10556 infer_calls=10556 feedback_calls=10556 infer_wait=241.097s infer_obs_pack=1.529s env_step=12.903s feedback_total=7.008s feedback_obs_pack=1.682s feedback_info_pack=0.318s effective_fps=40.21 +2026-09-25 17:47:32.519 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=12044 infer_calls=12044 feedback_calls=12044 infer_wait=267.665s infer_obs_pack=1.730s env_step=14.644s feedback_total=7.924s feedback_obs_pack=1.903s feedback_info_pack=0.361s effective_fps=41.25 +2026-09-25 17:48:02.596 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=12286 infer_calls=12286 feedback_calls=12286 infer_wait=297.260s infer_obs_pack=1.757s env_step=14.878s feedback_total=8.061s feedback_obs_pack=1.935s feedback_info_pack=0.367s effective_fps=38.16 +2026-09-25 17:48:32.631 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=12631 infer_calls=12631 feedback_calls=12631 infer_wait=326.495s infer_obs_pack=1.801s env_step=15.268s feedback_total=8.286s feedback_obs_pack=1.987s feedback_info_pack=0.378s effective_fps=35.90 +2026-09-25 17:49:02.633 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=14228 infer_calls=14228 feedback_calls=14228 infer_wait=353.096s infer_obs_pack=2.045s env_step=16.942s feedback_total=9.194s feedback_obs_pack=2.208s feedback_info_pack=0.419s effective_fps=37.32 +2026-09-25 17:49:32.647 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=15634 infer_calls=15634 feedback_calls=15634 infer_wait=380.231s infer_obs_pack=2.211s env_step=18.392s feedback_total=9.968s feedback_obs_pack=2.392s feedback_info_pack=0.456s effective_fps=38.06 +2026-09-25 17:50:02.666 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=16724 infer_calls=16724 feedback_calls=16724 infer_wait=408.081s infer_obs_pack=2.337s env_step=19.475s feedback_total=10.555s feedback_obs_pack=2.534s feedback_info_pack=0.482s effective_fps=37.97 +2026-09-25 17:50:32.694 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=18270 infer_calls=18270 feedback_calls=18270 infer_wait=434.709s infer_obs_pack=2.538s env_step=21.178s feedback_total=11.473s feedback_obs_pack=2.755s feedback_info_pack=0.525s effective_fps=38.88 +2026-09-25 17:51:02.695 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=19827 infer_calls=19827 feedback_calls=19827 infer_wait=461.325s infer_obs_pack=2.741s env_step=22.873s feedback_total=12.383s feedback_obs_pack=2.979s feedback_info_pack=0.568s effective_fps=39.71 +2026-09-25 17:51:32.708 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=20838 infer_calls=20838 feedback_calls=20838 infer_wait=489.300s infer_obs_pack=2.855s env_step=23.897s feedback_total=12.922s feedback_obs_pack=3.107s feedback_info_pack=0.593s effective_fps=39.39 +2026-09-25 17:52:02.762 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=22090 infer_calls=22090 feedback_calls=22090 infer_wait=516.650s infer_obs_pack=3.007s env_step=25.269s feedback_total=13.640s feedback_obs_pack=3.281s feedback_info_pack=0.625s effective_fps=39.55 +2026-09-25 17:52:32.770 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=23362 infer_calls=23362 feedback_calls=23362 infer_wait=544.154s infer_obs_pack=3.153s env_step=26.521s feedback_total=14.321s feedback_obs_pack=3.448s feedback_info_pack=0.656s effective_fps=39.72 +2026-09-25 17:53:07.737 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=24577 infer_calls=24577 feedback_calls=24577 infer_wait=576.537s infer_obs_pack=3.296s env_step=27.833s feedback_total=15.014s feedback_obs_pack=3.621s feedback_info_pack=0.687s effective_fps=39.47 +2026-09-25 17:53:37.740 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=25415 infer_calls=25415 feedback_calls=25415 infer_wait=604.930s infer_obs_pack=3.382s env_step=28.654s feedback_total=15.438s feedback_obs_pack=3.723s feedback_info_pack=0.705s effective_fps=38.96 +2026-09-25 17:54:07.791 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=26467 infer_calls=26467 feedback_calls=26467 infer_wait=633.053s infer_obs_pack=3.486s env_step=29.630s feedback_total=15.960s feedback_obs_pack=3.846s feedback_info_pack=0.728s effective_fps=38.80 +2026-09-25 17:54:37.800 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=27616 infer_calls=27616 feedback_calls=27616 infer_wait=660.612s infer_obs_pack=3.627s env_step=30.872s feedback_total=16.615s feedback_obs_pack=4.003s feedback_info_pack=0.758s effective_fps=38.80 +2026-09-25 17:55:13.186 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=28673 infer_calls=28673 feedback_calls=28673 infer_wait=693.668s infer_obs_pack=3.773s env_step=32.023s feedback_total=17.254s feedback_obs_pack=4.161s feedback_info_pack=0.788s effective_fps=38.40 +2026-09-25 17:55:43.195 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=29778 infer_calls=29778 feedback_calls=29778 infer_wait=721.248s infer_obs_pack=3.916s env_step=33.236s feedback_total=17.916s feedback_obs_pack=4.316s feedback_info_pack=0.817s effective_fps=38.36 +2026-09-25 17:56:13.197 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=30825 infer_calls=30825 feedback_calls=30825 infer_wait=748.967s infer_obs_pack=4.042s env_step=34.375s feedback_total=18.536s feedback_obs_pack=4.465s feedback_info_pack=0.845s effective_fps=38.25 +2026-09-25 17:56:43.198 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=31951 infer_calls=31951 feedback_calls=31951 infer_wait=776.226s infer_obs_pack=4.209s env_step=35.739s feedback_total=19.280s feedback_obs_pack=4.642s feedback_info_pack=0.879s effective_fps=38.24 +2026-09-25 17:57:16.218 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=32769 infer_calls=32769 feedback_calls=32769 infer_wait=807.199s infer_obs_pack=4.334s env_step=36.770s feedback_total=19.822s feedback_obs_pack=4.773s feedback_info_pack=0.903s effective_fps=37.75 +2026-09-25 17:57:46.264 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=34026 infer_calls=34026 feedback_calls=34026 infer_wait=834.380s infer_obs_pack=4.499s env_step=38.207s feedback_total=20.589s feedback_obs_pack=4.958s feedback_info_pack=0.938s effective_fps=37.90 +2026-09-25 17:58:16.275 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=35097 infer_calls=35097 feedback_calls=35097 infer_wait=861.930s infer_obs_pack=4.640s env_step=39.429s feedback_total=21.267s feedback_obs_pack=5.119s feedback_info_pack=0.970s effective_fps=37.85 +2026-09-25 17:58:46.306 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=36220 infer_calls=36220 feedback_calls=36220 infer_wait=889.197s infer_obs_pack=4.800s env_step=40.817s feedback_total=22.020s feedback_obs_pack=5.306s feedback_info_pack=1.003s effective_fps=37.85 +2026-09-25 17:59:16.310 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=37068 infer_calls=37068 feedback_calls=37068 infer_wait=917.155s infer_obs_pack=4.922s env_step=41.850s feedback_total=22.570s feedback_obs_pack=5.438s feedback_info_pack=1.028s effective_fps=37.58 +2026-09-25 17:59:46.330 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=38138 infer_calls=38138 feedback_calls=38138 infer_wait=944.583s infer_obs_pack=5.072s env_step=43.156s feedback_total=23.267s feedback_obs_pack=5.605s feedback_info_pack=1.059s effective_fps=37.53 +2026-09-25 18:00:16.333 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=39082 infer_calls=39082 feedback_calls=39082 infer_wait=972.274s infer_obs_pack=5.213s env_step=44.315s feedback_total=23.890s feedback_obs_pack=5.753s feedback_info_pack=1.087s effective_fps=37.37 +2026-09-25 18:00:46.381 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=39999 infer_calls=39999 feedback_calls=39999 infer_wait=1000.159s infer_obs_pack=5.340s env_step=45.396s feedback_total=24.479s feedback_obs_pack=5.892s feedback_info_pack=1.113s effective_fps=37.20 +2026-09-25 18:01:20.204 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=40961 infer_calls=40961 feedback_calls=40961 infer_wait=1031.756s infer_obs_pack=5.467s env_step=46.529s feedback_total=25.068s feedback_obs_pack=6.034s feedback_info_pack=1.139s effective_fps=36.94 +2026-09-25 18:01:50.215 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=42087 infer_calls=42087 feedback_calls=42087 infer_wait=1059.059s infer_obs_pack=5.624s env_step=47.902s feedback_total=25.787s feedback_obs_pack=6.207s feedback_info_pack=1.172s effective_fps=36.97 +2026-09-25 18:02:20.245 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=43252 infer_calls=43252 feedback_calls=43252 infer_wait=1086.213s infer_obs_pack=5.792s env_step=49.347s feedback_total=26.561s feedback_obs_pack=6.395s feedback_info_pack=1.209s effective_fps=37.03 +2026-09-25 18:02:50.249 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=44406 infer_calls=44406 feedback_calls=44406 infer_wait=1113.824s infer_obs_pack=5.930s env_step=50.549s feedback_total=27.205s feedback_obs_pack=6.548s feedback_info_pack=1.239s effective_fps=37.08 +2026-09-25 18:03:23.905 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=45057 infer_calls=45057 feedback_calls=45057 infer_wait=1146.099s infer_obs_pack=6.008s env_step=51.233s feedback_total=27.585s feedback_obs_pack=6.637s feedback_info_pack=1.258s effective_fps=36.60 +2026-09-25 18:03:53.916 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=46043 infer_calls=46043 feedback_calls=46043 infer_wait=1173.873s infer_obs_pack=6.139s env_step=52.353s feedback_total=28.184s feedback_obs_pack=6.778s feedback_info_pack=1.285s effective_fps=36.53 +2026-09-25 18:04:23.988 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=47151 infer_calls=47151 feedback_calls=47151 infer_wait=1201.381s infer_obs_pack=6.291s env_step=53.639s feedback_total=28.885s feedback_obs_pack=6.945s feedback_info_pack=1.316s effective_fps=36.55 +2026-09-25 18:04:53.996 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=47826 infer_calls=47826 feedback_calls=47826 infer_wait=1229.782s infer_obs_pack=6.383s env_step=54.438s feedback_total=29.329s feedback_obs_pack=7.051s feedback_info_pack=1.337s effective_fps=36.23 +2026-09-25 18:05:24.008 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=48428 infer_calls=48428 feedback_calls=48428 infer_wait=1258.604s infer_obs_pack=6.452s env_step=55.027s feedback_total=29.653s feedback_obs_pack=7.132s feedback_info_pack=1.352s effective_fps=35.88 +2026-09-25 18:05:54.042 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=49108 infer_calls=49108 feedback_calls=49108 infer_wait=1287.208s infer_obs_pack=6.534s env_step=55.758s feedback_total=30.026s feedback_obs_pack=7.221s feedback_info_pack=1.368s effective_fps=35.60 diff --git a/experiments/e24-coverage/results/attempt-1/fpodppo-hopper/server-seed0.log b/experiments/e24-coverage/results/attempt-1/fpodppo-hopper/server-seed0.log new file mode 100644 index 0000000..41c83f9 --- /dev/null +++ b/experiments/e24-coverage/results/attempt-1/fpodppo-hopper/server-seed0.log @@ -0,0 +1,13 @@ +17:42:12|INFO|plugrl_server version: 0.1.0 +17:42:12|INFO|Algorithm: dppo, Config: DPPOAlgoConfigHopper(global_steps=81920, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=4096, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=2048, train_itrs=20, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +17:42:12|INFO|Policy: fpo-policy, Config: FPOPolicyConfig(algo='dppo', device=device(type='cpu'), obs_dim=11, action_dim=3, flow_steps=10, timestep_embed_dim=8, action_horizon=1, feather_std=0.0, policy_mlp_output_scale=0.25, normalize_observations=True, hidden_dims=(32, 32, 32, 32), value_hidden_dims=(256, 256, 256, 256, 256)) +17:42:12|INFO|Seeded python, numpy and torch with 0. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +17:42:12|INFO|Checkpoint Manager created: + at D:\75128\Desktop\plugrl-work\plugrl-server\.claude\worktrees\fix-pi0-image-mask-batching\experiments\e24-coverage\results\fpodppo-hopper\dppo\fpo-policy\fpodppo-hopper-seed0 +17:42:14|INFO|Policy created... +17:42:14|INFO|Initialized RolloutBuffer buffer_size=4096 action_shape=(4096, 10, 1, 3) value_shape=(4096,) +17:42:14|INFO|Algorithm created: + +17:42:14|INFO|Agent Server is listening on 0.0.0.0:9310 +D:\75128\Desktop\plugrl-work\plugrl-server\.claude\worktrees\fix-pi0-image-mask-batching\.venv\Lib\site-packages\torch\utils\data\dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) diff --git a/experiments/e24-coverage/results/attempt-1/fpodppo-hopper/server-seed1.log b/experiments/e24-coverage/results/attempt-1/fpodppo-hopper/server-seed1.log new file mode 100644 index 0000000..5347278 --- /dev/null +++ b/experiments/e24-coverage/results/attempt-1/fpodppo-hopper/server-seed1.log @@ -0,0 +1,13 @@ +17:42:23|INFO|plugrl_server version: 0.1.0 +17:42:23|INFO|Algorithm: dppo, Config: DPPOAlgoConfigHopper(global_steps=81920, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=4096, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=2048, train_itrs=20, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +17:42:23|INFO|Policy: fpo-policy, Config: FPOPolicyConfig(algo='dppo', device=device(type='cpu'), obs_dim=11, action_dim=3, flow_steps=10, timestep_embed_dim=8, action_horizon=1, feather_std=0.0, policy_mlp_output_scale=0.25, normalize_observations=True, hidden_dims=(32, 32, 32, 32), value_hidden_dims=(256, 256, 256, 256, 256)) +17:42:23|INFO|Seeded python, numpy and torch with 1. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +17:42:23|INFO|Checkpoint Manager created: + at D:\75128\Desktop\plugrl-work\plugrl-server\.claude\worktrees\fix-pi0-image-mask-batching\experiments\e24-coverage\results\fpodppo-hopper\dppo\fpo-policy\fpodppo-hopper-seed1 +17:42:26|INFO|Policy created... +17:42:26|INFO|Initialized RolloutBuffer buffer_size=4096 action_shape=(4096, 10, 1, 3) value_shape=(4096,) +17:42:26|INFO|Algorithm created: + +17:42:26|INFO|Agent Server is listening on 0.0.0.0:9311 +D:\75128\Desktop\plugrl-work\plugrl-server\.claude\worktrees\fix-pi0-image-mask-batching\.venv\Lib\site-packages\torch\utils\data\dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) diff --git a/experiments/e24-coverage/results/attempt-1/fpodppo-hopper/server-seed2.log b/experiments/e24-coverage/results/attempt-1/fpodppo-hopper/server-seed2.log new file mode 100644 index 0000000..9328707 --- /dev/null +++ b/experiments/e24-coverage/results/attempt-1/fpodppo-hopper/server-seed2.log @@ -0,0 +1,13 @@ +17:42:31|INFO|plugrl_server version: 0.1.0 +17:42:31|INFO|Algorithm: dppo, Config: DPPOAlgoConfigHopper(global_steps=81920, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=4096, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=2048, train_itrs=20, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +17:42:31|INFO|Policy: fpo-policy, Config: FPOPolicyConfig(algo='dppo', device=device(type='cpu'), obs_dim=11, action_dim=3, flow_steps=10, timestep_embed_dim=8, action_horizon=1, feather_std=0.0, policy_mlp_output_scale=0.25, normalize_observations=True, hidden_dims=(32, 32, 32, 32), value_hidden_dims=(256, 256, 256, 256, 256)) +17:42:31|INFO|Seeded python, numpy and torch with 2. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +17:42:31|INFO|Checkpoint Manager created: + at D:\75128\Desktop\plugrl-work\plugrl-server\.claude\worktrees\fix-pi0-image-mask-batching\experiments\e24-coverage\results\fpodppo-hopper\dppo\fpo-policy\fpodppo-hopper-seed2 +17:42:33|INFO|Policy created... +17:42:33|INFO|Initialized RolloutBuffer buffer_size=4096 action_shape=(4096, 10, 1, 3) value_shape=(4096,) +17:42:33|INFO|Algorithm created: + +17:42:33|INFO|Agent Server is listening on 0.0.0.0:9312 +D:\75128\Desktop\plugrl-work\plugrl-server\.claude\worktrees\fix-pi0-image-mask-batching\.venv\Lib\site-packages\torch\utils\data\dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) diff --git a/experiments/e24-coverage/results/attempt-1/fpodppo-walker.out b/experiments/e24-coverage/results/attempt-1/fpodppo-walker.out new file mode 100644 index 0000000..1480f2c --- /dev/null +++ b/experiments/e24-coverage/results/attempt-1/fpodppo-walker.out @@ -0,0 +1,4 @@ +cell: fpodppo-walker = fpo-policy/default x dppo/walker x Walker2d-v5 +iters: 20 x 4096 batch: variant default replan: 1 seeds: 0 1 2 +out: /d/75128/Desktop/plugrl-work/plugrl-server/.claude/worktrees/fix-pi0-image-mask-batching/experiments/e24-coverage/results/fpodppo-walker +start 2026-09-25 17:42:03 diff --git a/experiments/e24-coverage/results/attempt-1/fpodppo-walker/client-seed0.log b/experiments/e24-coverage/results/attempt-1/fpodppo-walker/client-seed0.log new file mode 100644 index 0000000..2dd5840 --- /dev/null +++ b/experiments/e24-coverage/results/attempt-1/fpodppo-walker/client-seed0.log @@ -0,0 +1,52 @@ +2026-09-25 17:42:48.467 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 17:42:48.470 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 17:42:48.473 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 17:42:48.477 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 17:42:48.486 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 17:42:48.707 | INFO  | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-174248-c7bfb14c output_dir=runs\mujoco-v1-nenv1-20260925-174248-c7bfb14c recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 17:42:48.924 | INFO  | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9320... +2026-09-25 17:42:48.959 | INFO  | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'FPOPolicy', 'action_dim': 6, 'action_horizon': 1} +2026-09-25 17:43:18.967 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=2198 infer_calls=2198 feedback_calls=2198 infer_wait=23.955s infer_obs_pack=0.360s env_step=3.096s feedback_total=1.552s feedback_obs_pack=0.366s feedback_info_pack=0.072s effective_fps=75.89 +2026-09-25 17:44:13.236 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=4097 infer_calls=4097 feedback_calls=4097 infer_wait=72.717s infer_obs_pack=0.762s env_step=5.777s feedback_total=3.019s feedback_obs_pack=0.722s feedback_info_pack=0.134s effective_fps=49.80 +2026-09-25 17:44:43.256 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=5535 infer_calls=5535 feedback_calls=5535 infer_wait=99.537s infer_obs_pack=0.949s env_step=7.381s feedback_total=3.886s feedback_obs_pack=0.925s feedback_info_pack=0.174s effective_fps=49.53 +2026-09-25 17:45:13.271 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=6838 infer_calls=6838 feedback_calls=6838 infer_wait=126.581s infer_obs_pack=1.124s env_step=8.842s feedback_total=4.713s feedback_obs_pack=1.118s feedback_info_pack=0.211s effective_fps=48.41 +2026-09-25 17:45:43.293 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=7819 infer_calls=7819 feedback_calls=7819 infer_wait=154.527s infer_obs_pack=1.253s env_step=9.852s feedback_total=5.293s feedback_obs_pack=1.254s feedback_info_pack=0.236s effective_fps=45.75 +2026-09-25 17:46:18.334 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=8193 infer_calls=8193 feedback_calls=8193 infer_wait=188.828s infer_obs_pack=1.295s env_step=10.210s feedback_total=5.497s feedback_obs_pack=1.303s feedback_info_pack=0.245s effective_fps=39.80 +2026-09-25 17:46:48.342 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=9438 infer_calls=9438 feedback_calls=9438 infer_wait=216.281s infer_obs_pack=1.442s env_step=11.464s feedback_total=6.213s feedback_obs_pack=1.490s feedback_info_pack=0.277s effective_fps=40.09 +2026-09-25 17:47:18.353 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=10960 infer_calls=10960 feedback_calls=10960 infer_wait=242.879s infer_obs_pack=1.641s env_step=13.173s feedback_total=7.126s feedback_obs_pack=1.713s feedback_info_pack=0.319s effective_fps=41.39 +2026-09-25 17:47:48.394 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=11706 infer_calls=11706 feedback_calls=11706 infer_wait=271.184s infer_obs_pack=1.742s env_step=14.023s feedback_total=7.612s feedback_obs_pack=1.826s feedback_info_pack=0.341s effective_fps=39.74 +2026-09-25 17:48:18.415 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=11990 infer_calls=11990 feedback_calls=11990 infer_wait=300.663s infer_obs_pack=1.773s env_step=14.288s feedback_total=7.762s feedback_obs_pack=1.859s feedback_info_pack=0.348s effective_fps=36.95 +2026-09-25 17:48:48.422 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=12827 infer_calls=12827 feedback_calls=12827 infer_wait=328.774s infer_obs_pack=1.943s env_step=15.198s feedback_total=8.262s feedback_obs_pack=1.979s feedback_info_pack=0.371s effective_fps=36.22 +2026-09-25 17:49:18.431 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=14355 infer_calls=14355 feedback_calls=14355 infer_wait=355.802s infer_obs_pack=2.113s env_step=16.687s feedback_total=9.070s feedback_obs_pack=2.175s feedback_info_pack=0.407s effective_fps=37.41 +2026-09-25 17:49:48.432 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=15843 infer_calls=15843 feedback_calls=15843 infer_wait=382.794s infer_obs_pack=2.291s env_step=18.180s feedback_total=9.886s feedback_obs_pack=2.372s feedback_info_pack=0.444s effective_fps=38.35 +2026-09-25 17:50:18.434 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=16981 infer_calls=16981 feedback_calls=16981 infer_wait=410.457s infer_obs_pack=2.427s env_step=19.338s feedback_total=10.526s feedback_obs_pack=2.527s feedback_info_pack=0.474s effective_fps=38.35 +2026-09-25 17:50:48.475 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=18629 infer_calls=18629 feedback_calls=18629 infer_wait=436.762s infer_obs_pack=2.648s env_step=21.220s feedback_total=11.520s feedback_obs_pack=2.771s feedback_info_pack=0.521s effective_fps=39.46 +2026-09-25 17:51:18.495 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=19998 infer_calls=19998 feedback_calls=19998 infer_wait=463.809s infer_obs_pack=2.825s env_step=22.709s feedback_total=12.327s feedback_obs_pack=2.965s feedback_info_pack=0.557s effective_fps=39.86 +2026-09-25 17:51:48.497 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=21028 infer_calls=21028 feedback_calls=21028 infer_wait=491.721s infer_obs_pack=2.949s env_step=23.737s feedback_total=12.898s feedback_obs_pack=3.099s feedback_info_pack=0.584s effective_fps=39.58 +2026-09-25 17:52:18.584 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=22159 infer_calls=22159 feedback_calls=22159 infer_wait=519.326s infer_obs_pack=3.096s env_step=24.987s feedback_total=13.555s feedback_obs_pack=3.260s feedback_info_pack=0.613s effective_fps=39.50 +2026-09-25 17:52:48.592 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=23567 infer_calls=23567 feedback_calls=23567 infer_wait=546.444s infer_obs_pack=3.261s env_step=26.442s feedback_total=14.323s feedback_obs_pack=3.443s feedback_info_pack=0.652s effective_fps=39.91 +2026-09-25 17:53:28.656 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=24577 infer_calls=24577 feedback_calls=24577 infer_wait=584.413s infer_obs_pack=3.385s env_step=27.483s feedback_total=14.898s feedback_obs_pack=3.579s feedback_info_pack=0.678s effective_fps=39.00 +2026-09-25 17:53:58.658 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=25662 infer_calls=25662 feedback_calls=25662 infer_wait=612.502s infer_obs_pack=3.493s env_step=28.419s feedback_total=15.433s feedback_obs_pack=3.700s feedback_info_pack=0.700s effective_fps=38.89 +2026-09-25 17:54:28.706 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=26737 infer_calls=26737 feedback_calls=26737 infer_wait=640.393s infer_obs_pack=3.615s env_step=29.488s feedback_total=16.026s feedback_obs_pack=3.836s feedback_info_pack=0.727s effective_fps=38.78 +2026-09-25 17:54:58.740 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=28001 infer_calls=28001 feedback_calls=28001 infer_wait=667.730s infer_obs_pack=3.785s env_step=30.809s feedback_total=16.770s feedback_obs_pack=4.016s feedback_info_pack=0.764s effective_fps=38.94 +2026-09-25 17:55:28.765 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=28763 infer_calls=28763 feedback_calls=28763 infer_wait=696.039s infer_obs_pack=3.881s env_step=31.670s feedback_total=17.222s feedback_obs_pack=4.125s feedback_info_pack=0.786s effective_fps=38.41 +2026-09-25 17:55:58.765 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=29833 infer_calls=29833 feedback_calls=29833 infer_wait=723.664s infer_obs_pack=4.019s env_step=32.851s feedback_total=17.874s feedback_obs_pack=4.281s feedback_info_pack=0.816s effective_fps=38.33 +2026-09-25 17:56:28.799 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=30950 infer_calls=30950 feedback_calls=30950 infer_wait=751.261s infer_obs_pack=4.162s env_step=34.065s feedback_total=18.526s feedback_obs_pack=4.438s feedback_info_pack=0.845s effective_fps=38.30 +2026-09-25 17:56:58.805 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=32069 infer_calls=32069 feedback_calls=32069 infer_wait=778.584s infer_obs_pack=4.329s env_step=35.373s feedback_total=19.254s feedback_obs_pack=4.610s feedback_info_pack=0.879s effective_fps=38.29 +2026-09-25 17:57:29.375 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=32769 infer_calls=32769 feedback_calls=32769 infer_wait=807.337s infer_obs_pack=4.443s env_step=36.261s feedback_total=19.757s feedback_obs_pack=4.730s feedback_info_pack=0.902s effective_fps=37.76 +2026-09-25 17:57:59.394 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=34066 infer_calls=34066 feedback_calls=34066 infer_wait=834.479s infer_obs_pack=4.617s env_step=37.674s feedback_total=20.544s feedback_obs_pack=4.915s feedback_info_pack=0.937s effective_fps=37.96 +2026-09-25 17:58:29.398 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=35086 infer_calls=35086 feedback_calls=35086 infer_wait=862.108s infer_obs_pack=4.750s env_step=38.867s feedback_total=21.186s feedback_obs_pack=5.070s feedback_info_pack=0.967s effective_fps=37.85 +2026-09-25 17:58:59.417 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=36329 infer_calls=36329 feedback_calls=36329 infer_wait=889.047s infer_obs_pack=4.941s env_step=40.393s feedback_total=22.026s feedback_obs_pack=5.271s feedback_info_pack=1.005s effective_fps=37.98 +2026-09-25 17:59:29.429 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=37015 infer_calls=37015 feedback_calls=37015 infer_wait=917.395s infer_obs_pack=5.034s env_step=41.225s feedback_total=22.483s feedback_obs_pack=5.380s feedback_info_pack=1.026s effective_fps=37.54 +2026-09-25 17:59:59.444 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=38159 infer_calls=38159 feedback_calls=38159 infer_wait=944.648s infer_obs_pack=5.198s env_step=42.589s feedback_total=23.243s feedback_obs_pack=5.562s feedback_info_pack=1.061s effective_fps=37.57 +2026-09-25 18:00:29.498 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=38991 infer_calls=38991 feedback_calls=38991 infer_wait=972.738s infer_obs_pack=5.315s env_step=43.550s feedback_total=23.785s feedback_obs_pack=5.691s feedback_info_pack=1.085s effective_fps=37.30 +2026-09-25 18:00:59.509 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=39982 infer_calls=39982 feedback_calls=39982 infer_wait=1000.563s infer_obs_pack=5.438s env_step=44.637s feedback_total=24.384s feedback_obs_pack=5.831s feedback_info_pack=1.113s effective_fps=37.19 +2026-09-25 18:01:35.664 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=40962 infer_calls=40962 feedback_calls=40962 infer_wait=1034.411s infer_obs_pack=5.571s env_step=45.787s feedback_total=25.012s feedback_obs_pack=5.983s feedback_info_pack=1.142s effective_fps=36.88 +2026-09-25 18:02:05.686 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=42130 infer_calls=42130 feedback_calls=42130 infer_wait=1061.590s infer_obs_pack=5.741s env_step=47.209s feedback_total=25.773s feedback_obs_pack=6.169s feedback_info_pack=1.177s effective_fps=36.95 +2026-09-25 18:02:35.702 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=43400 infer_calls=43400 feedback_calls=43400 infer_wait=1088.663s infer_obs_pack=5.913s env_step=48.666s feedback_total=26.585s feedback_obs_pack=6.367s feedback_info_pack=1.214s effective_fps=37.10 +2026-09-25 18:03:05.703 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=44278 infer_calls=44278 feedback_calls=44278 infer_wait=1116.836s infer_obs_pack=6.016s env_step=49.578s feedback_total=27.082s feedback_obs_pack=6.483s feedback_info_pack=1.236s effective_fps=36.91 +2026-09-25 18:03:42.695 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=45057 infer_calls=45057 feedback_calls=45057 infer_wait=1152.245s infer_obs_pack=6.107s env_step=50.359s feedback_total=27.521s feedback_obs_pack=6.587s feedback_info_pack=1.255s effective_fps=36.45 +2026-09-25 18:04:12.719 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=46123 infer_calls=46123 feedback_calls=46123 infer_wait=1179.879s infer_obs_pack=6.250s env_step=51.537s feedback_total=28.180s feedback_obs_pack=6.745s feedback_info_pack=1.286s effective_fps=36.44 +2026-09-25 18:04:42.749 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=47021 infer_calls=47021 feedback_calls=47021 infer_wait=1207.778s infer_obs_pack=6.379s env_step=52.582s feedback_total=28.762s feedback_obs_pack=6.885s feedback_info_pack=1.313s effective_fps=36.30 +2026-09-25 18:05:12.799 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=47580 infer_calls=47580 feedback_calls=47580 infer_wait=1236.575s infer_obs_pack=6.453s env_step=53.204s feedback_total=29.105s feedback_obs_pack=6.968s feedback_info_pack=1.329s effective_fps=35.90 +2026-09-25 18:05:42.817 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=48348 infer_calls=48348 feedback_calls=48348 infer_wait=1265.072s infer_obs_pack=6.543s env_step=53.958s feedback_total=29.516s feedback_obs_pack=7.068s feedback_info_pack=1.348s effective_fps=35.68 diff --git a/experiments/e24-coverage/results/attempt-1/fpodppo-walker/client-seed1.log b/experiments/e24-coverage/results/attempt-1/fpodppo-walker/client-seed1.log new file mode 100644 index 0000000..9f1cd78 --- /dev/null +++ b/experiments/e24-coverage/results/attempt-1/fpodppo-walker/client-seed1.log @@ -0,0 +1,52 @@ +2026-09-25 17:42:52.225 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 17:42:52.228 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 17:42:52.234 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 17:42:52.239 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 17:42:52.247 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 17:42:52.479 | INFO  | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-174252-18cd6120 output_dir=runs\mujoco-v1-nenv1-20260925-174252-18cd6120 recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 17:42:52.736 | INFO  | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9321... +2026-09-25 17:42:52.747 | INFO  | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'FPOPolicy', 'action_dim': 6, 'action_horizon': 1} +2026-09-25 17:43:22.756 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=2182 infer_calls=2182 feedback_calls=2182 infer_wait=23.501s infer_obs_pack=0.386s env_step=3.308s feedback_total=1.716s feedback_obs_pack=0.399s feedback_info_pack=0.076s effective_fps=75.47 +2026-09-25 17:44:13.264 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=4098 infer_calls=4098 feedback_calls=4098 infer_wait=68.580s infer_obs_pack=0.695s env_step=5.994s feedback_total=3.214s feedback_obs_pack=0.780s feedback_info_pack=0.162s effective_fps=52.22 +2026-09-25 17:44:43.272 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=5565 infer_calls=5565 feedback_calls=5565 infer_wait=95.255s infer_obs_pack=0.891s env_step=7.646s feedback_total=4.118s feedback_obs_pack=0.993s feedback_info_pack=0.203s effective_fps=51.57 +2026-09-25 17:45:13.289 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=6880 infer_calls=6880 feedback_calls=6880 infer_wait=122.292s infer_obs_pack=1.066s env_step=9.116s feedback_total=4.938s feedback_obs_pack=1.191s feedback_info_pack=0.239s effective_fps=50.07 +2026-09-25 17:45:43.309 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=7854 infer_calls=7854 feedback_calls=7854 infer_wait=150.171s infer_obs_pack=1.187s env_step=10.169s feedback_total=5.524s feedback_obs_pack=1.329s feedback_info_pack=0.265s effective_fps=47.02 +2026-09-25 17:46:17.207 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=8193 infer_calls=8193 feedback_calls=8193 infer_wait=183.400s infer_obs_pack=1.226s env_step=10.495s feedback_total=5.712s feedback_obs_pack=1.374s feedback_info_pack=0.273s effective_fps=40.80 +2026-09-25 17:46:47.215 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=9427 infer_calls=9427 feedback_calls=9427 infer_wait=210.814s infer_obs_pack=1.375s env_step=11.784s feedback_total=6.417s feedback_obs_pack=1.546s feedback_info_pack=0.305s effective_fps=40.92 +2026-09-25 17:47:17.246 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=10982 infer_calls=10982 feedback_calls=10982 infer_wait=237.370s infer_obs_pack=1.581s env_step=13.488s feedback_total=7.378s feedback_obs_pack=1.772s feedback_info_pack=0.349s effective_fps=42.27 +2026-09-25 17:47:47.284 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=11774 infer_calls=11774 feedback_calls=11774 infer_wait=265.596s infer_obs_pack=1.686s env_step=14.388s feedback_total=7.876s feedback_obs_pack=1.891s feedback_info_pack=0.371s effective_fps=40.66 +2026-09-25 17:48:17.297 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=12030 infer_calls=12030 feedback_calls=12030 infer_wait=295.118s infer_obs_pack=1.712s env_step=14.626s feedback_total=8.014s feedback_obs_pack=1.923s feedback_info_pack=0.377s effective_fps=37.66 +2026-09-25 17:48:47.320 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=12855 infer_calls=12855 feedback_calls=12855 infer_wait=323.160s infer_obs_pack=1.884s env_step=15.571s feedback_total=8.540s feedback_obs_pack=2.050s feedback_info_pack=0.401s effective_fps=36.82 +2026-09-25 17:49:17.321 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=14382 infer_calls=14382 feedback_calls=14382 infer_wait=350.144s infer_obs_pack=2.058s env_step=17.067s feedback_total=9.360s feedback_obs_pack=2.246s feedback_info_pack=0.438s effective_fps=37.98 +2026-09-25 17:49:47.329 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=15822 infer_calls=15822 feedback_calls=15822 infer_wait=377.271s infer_obs_pack=2.225s env_step=18.508s feedback_total=10.133s feedback_obs_pack=2.429s feedback_info_pack=0.474s effective_fps=38.77 +2026-09-25 17:50:17.339 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=16972 infer_calls=16972 feedback_calls=16972 infer_wait=404.957s infer_obs_pack=2.355s env_step=19.664s feedback_total=10.766s feedback_obs_pack=2.582s feedback_info_pack=0.502s effective_fps=38.77 +2026-09-25 17:50:47.350 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=18655 infer_calls=18655 feedback_calls=18655 infer_wait=431.202s infer_obs_pack=2.579s env_step=21.550s feedback_total=11.767s feedback_obs_pack=2.828s feedback_info_pack=0.549s effective_fps=39.94 +2026-09-25 17:51:17.353 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=20019 infer_calls=20019 feedback_calls=20019 infer_wait=458.202s infer_obs_pack=2.761s env_step=23.024s feedback_total=12.592s feedback_obs_pack=3.028s feedback_info_pack=0.586s effective_fps=40.31 +2026-09-25 17:51:47.375 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=21065 infer_calls=21065 feedback_calls=21065 infer_wait=486.168s infer_obs_pack=2.877s env_step=24.036s feedback_total=13.153s feedback_obs_pack=3.158s feedback_info_pack=0.610s effective_fps=40.03 +2026-09-25 17:52:17.375 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=22196 infer_calls=22196 feedback_calls=22196 infer_wait=513.724s infer_obs_pack=3.013s env_step=25.265s feedback_total=13.806s feedback_obs_pack=3.315s feedback_info_pack=0.640s effective_fps=39.93 +2026-09-25 17:52:47.395 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=23621 infer_calls=23621 feedback_calls=23621 infer_wait=540.892s infer_obs_pack=3.183s env_step=26.666s feedback_total=14.588s feedback_obs_pack=3.500s feedback_info_pack=0.675s effective_fps=40.36 +2026-09-25 17:53:24.298 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=24578 infer_calls=24578 feedback_calls=24578 infer_wait=575.756s infer_obs_pack=3.300s env_step=27.681s feedback_total=15.136s feedback_obs_pack=3.632s feedback_info_pack=0.700s effective_fps=39.52 +2026-09-25 17:53:54.326 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=25587 infer_calls=25587 feedback_calls=25587 infer_wait=604.045s infer_obs_pack=3.397s env_step=28.526s feedback_total=15.622s feedback_obs_pack=3.743s feedback_info_pack=0.721s effective_fps=39.27 +2026-09-25 17:54:24.365 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=26621 infer_calls=26621 feedback_calls=26621 infer_wait=632.018s infer_obs_pack=3.514s env_step=29.552s feedback_total=16.186s feedback_obs_pack=3.876s feedback_info_pack=0.746s effective_fps=39.08 +2026-09-25 17:54:54.372 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=27912 infer_calls=27912 feedback_calls=27912 infer_wait=659.308s infer_obs_pack=3.675s env_step=30.908s feedback_total=16.918s feedback_obs_pack=4.054s feedback_info_pack=0.778s effective_fps=39.27 +2026-09-25 17:55:25.227 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=28674 infer_calls=28674 feedback_calls=28674 infer_wait=688.462s infer_obs_pack=3.773s env_step=31.751s feedback_total=17.378s feedback_obs_pack=4.166s feedback_info_pack=0.799s effective_fps=38.68 +2026-09-25 17:55:55.244 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=29759 infer_calls=29759 feedback_calls=29759 infer_wait=716.077s infer_obs_pack=3.912s env_step=32.923s feedback_total=18.037s feedback_obs_pack=4.319s feedback_info_pack=0.829s effective_fps=38.60 +2026-09-25 17:56:25.262 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=30900 infer_calls=30900 feedback_calls=30900 infer_wait=743.672s infer_obs_pack=4.053s env_step=34.116s feedback_total=18.701s feedback_obs_pack=4.479s feedback_info_pack=0.861s effective_fps=38.60 +2026-09-25 17:56:55.277 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=32020 infer_calls=32020 feedback_calls=32020 infer_wait=770.948s infer_obs_pack=4.225s env_step=35.461s feedback_total=19.449s feedback_obs_pack=4.658s feedback_info_pack=0.895s effective_fps=38.57 +2026-09-25 17:57:27.348 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=32769 infer_calls=32769 feedback_calls=32769 infer_wait=801.249s infer_obs_pack=4.327s env_step=36.339s feedback_total=19.927s feedback_obs_pack=4.770s feedback_info_pack=0.916s effective_fps=38.02 +2026-09-25 17:57:57.367 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=34061 infer_calls=34061 feedback_calls=34061 infer_wait=828.395s infer_obs_pack=4.494s env_step=37.753s feedback_total=20.714s feedback_obs_pack=4.959s feedback_info_pack=0.952s effective_fps=38.21 +2026-09-25 17:58:27.367 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=35097 infer_calls=35097 feedback_calls=35097 infer_wait=855.982s infer_obs_pack=4.632s env_step=38.952s feedback_total=21.371s feedback_obs_pack=5.115s feedback_info_pack=0.982s effective_fps=38.11 +2026-09-25 17:58:57.381 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=36308 infer_calls=36308 feedback_calls=36308 infer_wait=882.959s infer_obs_pack=4.812s env_step=40.460s feedback_total=22.182s feedback_obs_pack=5.314s feedback_info_pack=1.019s effective_fps=38.20 +2026-09-25 17:59:27.382 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=36992 infer_calls=36992 feedback_calls=36992 infer_wait=911.296s infer_obs_pack=4.908s env_step=41.269s feedback_total=22.638s feedback_obs_pack=5.420s feedback_info_pack=1.039s effective_fps=37.74 +2026-09-25 17:59:57.398 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=38130 infer_calls=38130 feedback_calls=38130 infer_wait=938.517s infer_obs_pack=5.068s env_step=42.664s feedback_total=23.395s feedback_obs_pack=5.602s feedback_info_pack=1.073s effective_fps=37.77 +2026-09-25 18:00:27.436 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=38986 infer_calls=38986 feedback_calls=38986 infer_wait=966.618s infer_obs_pack=5.180s env_step=43.616s feedback_total=23.930s feedback_obs_pack=5.729s feedback_info_pack=1.097s effective_fps=37.51 +2026-09-25 18:00:57.460 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=39980 infer_calls=39980 feedback_calls=39980 infer_wait=994.367s infer_obs_pack=5.312s env_step=44.727s feedback_total=24.564s feedback_obs_pack=5.879s feedback_info_pack=1.125s effective_fps=37.40 +2026-09-25 18:01:33.690 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=40961 infer_calls=40961 feedback_calls=40961 infer_wait=1028.289s infer_obs_pack=5.445s env_step=45.872s feedback_total=25.200s feedback_obs_pack=6.031s feedback_info_pack=1.154s effective_fps=37.08 +2026-09-25 18:02:03.703 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=42114 infer_calls=42114 feedback_calls=42114 infer_wait=1055.433s infer_obs_pack=5.617s env_step=47.281s feedback_total=25.986s feedback_obs_pack=6.219s feedback_info_pack=1.190s effective_fps=37.13 +2026-09-25 18:02:33.706 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=43355 infer_calls=43355 feedback_calls=43355 infer_wait=1082.581s infer_obs_pack=5.789s env_step=48.690s feedback_total=26.762s feedback_obs_pack=6.405s feedback_info_pack=1.226s effective_fps=37.25 +2026-09-25 18:03:03.761 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=44294 infer_calls=44294 feedback_calls=44294 infer_wait=1110.685s infer_obs_pack=5.899s env_step=49.659s feedback_total=27.286s feedback_obs_pack=6.528s feedback_info_pack=1.248s effective_fps=37.11 +2026-09-25 18:03:40.985 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=45057 infer_calls=45057 feedback_calls=45057 infer_wait=1146.383s infer_obs_pack=5.982s env_step=50.410s feedback_total=27.706s feedback_obs_pack=6.625s feedback_info_pack=1.267s effective_fps=36.62 +2026-09-25 18:04:10.997 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=46074 infer_calls=46074 feedback_calls=46074 infer_wait=1174.136s infer_obs_pack=6.115s env_step=51.535s feedback_total=28.312s feedback_obs_pack=6.771s feedback_info_pack=1.293s effective_fps=36.56 +2026-09-25 18:04:41.084 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=47008 infer_calls=47008 feedback_calls=47008 infer_wait=1201.990s infer_obs_pack=6.248s env_step=52.651s feedback_total=28.906s feedback_obs_pack=6.916s feedback_info_pack=1.321s effective_fps=36.45 +2026-09-25 18:05:11.086 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=47593 infer_calls=47593 feedback_calls=47593 infer_wait=1230.789s infer_obs_pack=6.319s env_step=53.243s feedback_total=29.236s feedback_obs_pack=6.996s feedback_info_pack=1.335s effective_fps=36.07 +2026-09-25 18:05:41.099 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=48347 infer_calls=48347 feedback_calls=48347 infer_wait=1259.258s infer_obs_pack=6.409s env_step=54.007s feedback_total=29.660s feedback_obs_pack=7.098s feedback_info_pack=1.354s effective_fps=35.83 diff --git a/experiments/e24-coverage/results/attempt-1/fpodppo-walker/client-seed2.log b/experiments/e24-coverage/results/attempt-1/fpodppo-walker/client-seed2.log new file mode 100644 index 0000000..adb73dc --- /dev/null +++ b/experiments/e24-coverage/results/attempt-1/fpodppo-walker/client-seed2.log @@ -0,0 +1,52 @@ +2026-09-25 17:43:01.321 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 17:43:01.331 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 17:43:01.335 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 17:43:01.340 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 17:43:01.346 | WARNING  | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 17:43:01.570 | INFO  | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-174301-3d5d99b2 output_dir=runs\mujoco-v1-nenv1-20260925-174301-3d5d99b2 recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 17:43:01.790 | INFO  | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9322... +2026-09-25 17:43:01.807 | INFO  | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'FPOPolicy', 'action_dim': 6, 'action_horizon': 1} +2026-09-25 17:43:31.829 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=2189 infer_calls=2189 feedback_calls=2189 infer_wait=23.684s infer_obs_pack=0.392s env_step=3.131s feedback_total=1.736s feedback_obs_pack=0.402s feedback_info_pack=0.071s effective_fps=75.63 +2026-09-25 17:44:13.293 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=4097 infer_calls=4097 feedback_calls=4097 infer_wait=59.392s infer_obs_pack=0.675s env_step=5.930s feedback_total=3.370s feedback_obs_pack=0.764s feedback_info_pack=0.131s effective_fps=59.06 +2026-09-25 17:44:43.304 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=5567 infer_calls=5567 feedback_calls=5567 infer_wait=86.092s infer_obs_pack=0.871s env_step=7.569s feedback_total=4.281s feedback_obs_pack=0.979s feedback_info_pack=0.173s effective_fps=56.34 +2026-09-25 17:45:13.327 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=6895 infer_calls=6895 feedback_calls=6895 infer_wait=113.029s infer_obs_pack=1.057s env_step=9.078s feedback_total=5.142s feedback_obs_pack=1.186s feedback_info_pack=0.213s effective_fps=53.74 +2026-09-25 17:45:43.329 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=7853 infer_calls=7853 feedback_calls=7853 infer_wait=140.940s infer_obs_pack=1.171s env_step=10.110s feedback_total=5.721s feedback_obs_pack=1.322s feedback_info_pack=0.238s effective_fps=49.72 +2026-09-25 17:46:17.174 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=8193 infer_calls=8193 feedback_calls=8193 infer_wait=174.118s infer_obs_pack=1.208s env_step=10.439s feedback_total=5.902s feedback_obs_pack=1.364s feedback_info_pack=0.247s effective_fps=42.75 +2026-09-25 17:46:47.187 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=9418 infer_calls=9418 feedback_calls=9418 infer_wait=201.480s infer_obs_pack=1.365s env_step=11.757s feedback_total=6.625s feedback_obs_pack=1.533s feedback_info_pack=0.282s effective_fps=42.57 +2026-09-25 17:47:17.240 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=10986 infer_calls=10986 feedback_calls=10986 infer_wait=228.071s infer_obs_pack=1.567s env_step=13.467s feedback_total=7.577s feedback_obs_pack=1.764s feedback_info_pack=0.327s effective_fps=43.82 +2026-09-25 17:47:47.253 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=11777 infer_calls=11777 feedback_calls=11777 infer_wait=256.245s infer_obs_pack=1.671s env_step=14.385s feedback_total=8.079s feedback_obs_pack=1.882s feedback_info_pack=0.349s effective_fps=42.00 +2026-09-25 17:48:17.260 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=12039 infer_calls=12039 feedback_calls=12039 infer_wait=285.739s infer_obs_pack=1.699s env_step=14.639s feedback_total=8.223s feedback_obs_pack=1.915s feedback_info_pack=0.355s effective_fps=38.80 +2026-09-25 17:48:47.262 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=12863 infer_calls=12863 feedback_calls=12863 infer_wait=313.784s infer_obs_pack=1.879s env_step=15.576s feedback_total=8.738s feedback_obs_pack=2.037s feedback_info_pack=0.379s effective_fps=37.83 +2026-09-25 17:49:17.285 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=14410 infer_calls=14410 feedback_calls=14410 infer_wait=340.700s infer_obs_pack=2.059s env_step=17.125s feedback_total=9.588s feedback_obs_pack=2.240s feedback_info_pack=0.419s effective_fps=39.00 +2026-09-25 17:49:47.300 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=15901 infer_calls=15901 feedback_calls=15901 infer_wait=367.775s infer_obs_pack=2.231s env_step=18.576s feedback_total=10.400s feedback_obs_pack=2.432s feedback_info_pack=0.458s effective_fps=39.85 +2026-09-25 17:50:17.303 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=17053 infer_calls=17053 feedback_calls=17053 infer_wait=395.404s infer_obs_pack=2.365s env_step=19.769s feedback_total=11.042s feedback_obs_pack=2.586s feedback_info_pack=0.487s effective_fps=39.79 +2026-09-25 17:50:47.315 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=18749 infer_calls=18749 feedback_calls=18749 infer_wait=421.604s infer_obs_pack=2.587s env_step=21.670s feedback_total=12.065s feedback_obs_pack=2.836s feedback_info_pack=0.535s effective_fps=40.94 +2026-09-25 17:51:17.331 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=20104 infer_calls=20104 feedback_calls=20104 infer_wait=448.679s infer_obs_pack=2.766s env_step=23.143s feedback_total=12.852s feedback_obs_pack=3.028s feedback_info_pack=0.573s effective_fps=41.24 +2026-09-25 17:51:47.333 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=21136 infer_calls=21136 feedback_calls=21136 infer_wait=476.686s infer_obs_pack=2.882s env_step=24.119s feedback_total=13.411s feedback_obs_pack=3.161s feedback_info_pack=0.598s effective_fps=40.87 +2026-09-25 17:52:17.341 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=22275 infer_calls=22275 feedback_calls=22275 infer_wait=504.220s infer_obs_pack=3.027s env_step=25.326s feedback_total=14.095s feedback_obs_pack=3.325s feedback_info_pack=0.630s effective_fps=40.75 +2026-09-25 17:52:47.392 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=23689 infer_calls=23689 feedback_calls=23689 infer_wait=531.414s infer_obs_pack=3.190s env_step=26.736s feedback_total=14.887s feedback_obs_pack=3.515s feedback_info_pack=0.668s effective_fps=41.11 +2026-09-25 17:53:19.583 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=24577 infer_calls=24577 feedback_calls=24577 infer_wait=561.661s infer_obs_pack=3.306s env_step=27.688s feedback_total=15.426s feedback_obs_pack=3.643s feedback_info_pack=0.693s effective_fps=40.42 +2026-09-25 17:53:49.593 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=25475 infer_calls=25475 feedback_calls=25475 infer_wait=590.032s infer_obs_pack=3.409s env_step=28.481s feedback_total=15.884s feedback_obs_pack=3.745s feedback_info_pack=0.713s effective_fps=39.94 +2026-09-25 17:54:19.656 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=26498 infer_calls=26498 feedback_calls=26498 infer_wait=618.183s infer_obs_pack=3.518s env_step=29.429s feedback_total=16.410s feedback_obs_pack=3.868s feedback_info_pack=0.737s effective_fps=39.69 +2026-09-25 17:54:49.667 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=27816 infer_calls=27816 feedback_calls=27816 infer_wait=645.369s infer_obs_pack=3.687s env_step=30.847s feedback_total=17.169s feedback_obs_pack=4.052s feedback_info_pack=0.773s effective_fps=39.90 +2026-09-25 17:55:22.830 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=28673 infer_calls=28673 feedback_calls=28673 infer_wait=676.654s infer_obs_pack=3.798s env_step=31.775s feedback_total=17.675s feedback_obs_pack=4.173s feedback_info_pack=0.796s effective_fps=39.28 +2026-09-25 17:55:52.851 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=29735 infer_calls=29735 feedback_calls=29735 infer_wait=704.365s infer_obs_pack=3.931s env_step=32.927s feedback_total=18.304s feedback_obs_pack=4.322s feedback_info_pack=0.825s effective_fps=39.15 +2026-09-25 17:56:22.874 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=30871 infer_calls=30871 feedback_calls=30871 infer_wait=731.972s infer_obs_pack=4.070s env_step=34.119s feedback_total=18.971s feedback_obs_pack=4.482s feedback_info_pack=0.856s effective_fps=39.12 +2026-09-25 17:56:52.876 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=31970 infer_calls=31970 feedback_calls=31970 infer_wait=759.317s infer_obs_pack=4.225s env_step=35.442s feedback_total=19.698s feedback_obs_pack=4.655s feedback_info_pack=0.891s effective_fps=39.05 +2026-09-25 17:57:25.488 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=32769 infer_calls=32769 feedback_calls=32769 infer_wait=789.969s infer_obs_pack=4.339s env_step=36.404s feedback_total=20.239s feedback_obs_pack=4.786s feedback_info_pack=0.916s effective_fps=38.51 +2026-09-25 17:57:55.491 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=34095 infer_calls=34095 feedback_calls=34095 infer_wait=816.931s infer_obs_pack=4.518s env_step=37.912s feedback_total=21.072s feedback_obs_pack=4.986s feedback_info_pack=0.955s effective_fps=38.73 +2026-09-25 17:58:25.492 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=35113 infer_calls=35113 feedback_calls=35113 infer_wait=844.584s infer_obs_pack=4.654s env_step=39.073s feedback_total=21.713s feedback_obs_pack=5.137s feedback_info_pack=0.986s effective_fps=38.58 +2026-09-25 17:58:55.499 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=36334 infer_calls=36334 feedback_calls=36334 infer_wait=871.509s infer_obs_pack=4.838s env_step=40.589s feedback_total=22.560s feedback_obs_pack=5.337s feedback_info_pack=1.026s effective_fps=38.67 +2026-09-25 17:59:25.501 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=37054 infer_calls=37054 feedback_calls=37054 infer_wait=899.804s infer_obs_pack=4.936s env_step=41.446s feedback_total=23.022s feedback_obs_pack=5.446s feedback_info_pack=1.047s effective_fps=38.23 +2026-09-25 17:59:55.517 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=38179 infer_calls=38179 feedback_calls=38179 infer_wait=927.073s infer_obs_pack=5.098s env_step=42.800s feedback_total=23.788s feedback_obs_pack=5.631s feedback_info_pack=1.084s effective_fps=38.23 +2026-09-25 18:00:25.530 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=39049 infer_calls=39049 feedback_calls=39049 infer_wait=955.036s infer_obs_pack=5.218s env_step=43.816s feedback_total=24.344s feedback_obs_pack=5.765s feedback_info_pack=1.110s effective_fps=37.97 +2026-09-25 18:00:55.561 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=40000 infer_calls=40000 feedback_calls=40000 infer_wait=982.893s infer_obs_pack=5.345s env_step=44.886s feedback_total=24.943s feedback_obs_pack=5.907s feedback_info_pack=1.137s effective_fps=37.80 +2026-09-25 18:01:29.617 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=40961 infer_calls=40961 feedback_calls=40961 infer_wait=1014.764s infer_obs_pack=5.476s env_step=45.971s feedback_total=25.541s feedback_obs_pack=6.053s feedback_info_pack=1.165s effective_fps=37.52 +2026-09-25 18:01:59.624 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=42090 infer_calls=42090 feedback_calls=42090 infer_wait=1041.957s infer_obs_pack=5.640s env_step=47.369s feedback_total=26.315s feedback_obs_pack=6.244s feedback_info_pack=1.202s effective_fps=37.54 +2026-09-25 18:02:29.632 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=43294 infer_calls=43294 feedback_calls=43294 infer_wait=1069.000s infer_obs_pack=5.815s env_step=48.846s feedback_total=27.128s feedback_obs_pack=6.439s feedback_info_pack=1.242s effective_fps=37.62 +2026-09-25 18:02:59.650 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=44334 infer_calls=44334 feedback_calls=44334 infer_wait=1096.875s infer_obs_pack=5.933s env_step=49.915s feedback_total=27.717s feedback_obs_pack=6.580s feedback_info_pack=1.269s effective_fps=37.56 +2026-09-25 18:03:38.074 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=45057 infer_calls=45057 feedback_calls=45057 infer_wait=1133.889s infer_obs_pack=6.010s env_step=50.606s feedback_total=28.116s feedback_obs_pack=6.673s feedback_info_pack=1.287s effective_fps=36.97 +2026-09-25 18:04:08.099 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=46121 infer_calls=46121 feedback_calls=46121 infer_wait=1161.584s infer_obs_pack=6.150s env_step=51.752s feedback_total=28.762s feedback_obs_pack=6.827s feedback_info_pack=1.316s effective_fps=36.95 +2026-09-25 18:04:38.174 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=47063 infer_calls=47063 feedback_calls=47063 infer_wait=1189.393s infer_obs_pack=6.279s env_step=52.878s feedback_total=29.386s feedback_obs_pack=6.980s feedback_info_pack=1.345s effective_fps=36.83 +2026-09-25 18:05:08.251 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=47659 infer_calls=47659 feedback_calls=47659 infer_wait=1218.163s infer_obs_pack=6.356s env_step=53.522s feedback_total=29.747s feedback_obs_pack=7.067s feedback_info_pack=1.361s effective_fps=36.44 +2026-09-25 18:05:38.288 | INFO  | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=48406 infer_calls=48406 feedback_calls=48406 infer_wait=1246.663s infer_obs_pack=6.446s env_step=54.280s feedback_total=30.165s feedback_obs_pack=7.167s feedback_info_pack=1.381s effective_fps=36.19 diff --git a/experiments/e24-coverage/results/attempt-1/fpodppo-walker/server-seed0.log b/experiments/e24-coverage/results/attempt-1/fpodppo-walker/server-seed0.log new file mode 100644 index 0000000..186a8f2 --- /dev/null +++ b/experiments/e24-coverage/results/attempt-1/fpodppo-walker/server-seed0.log @@ -0,0 +1,13 @@ +17:42:41|INFO|plugrl_server version: 0.1.0 +17:42:41|INFO|Algorithm: dppo, Config: DPPOAlgoConfigWalker(global_steps=81920, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=4096, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=2048, train_itrs=20, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +17:42:41|INFO|Policy: fpo-policy, Config: FPOPolicyConfig(algo='dppo', device=device(type='cpu'), obs_dim=17, action_dim=6, flow_steps=10, timestep_embed_dim=8, action_horizon=1, feather_std=0.0, policy_mlp_output_scale=0.25, normalize_observations=True, hidden_dims=(32, 32, 32, 32), value_hidden_dims=(256, 256, 256, 256, 256)) +17:42:41|INFO|Seeded python, numpy and torch with 0. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +17:42:41|INFO|Checkpoint Manager created: + at D:\75128\Desktop\plugrl-work\plugrl-server\.claude\worktrees\fix-pi0-image-mask-batching\experiments\e24-coverage\results\fpodppo-walker\dppo\fpo-policy\fpodppo-walker-seed0 +17:42:47|INFO|Policy created... +17:42:47|INFO|Initialized RolloutBuffer buffer_size=4096 action_shape=(4096, 10, 1, 6) value_shape=(4096,) +17:42:47|INFO|Algorithm created: + +17:42:47|INFO|Agent Server is listening on 0.0.0.0:9320 +D:\75128\Desktop\plugrl-work\plugrl-server\.claude\worktrees\fix-pi0-image-mask-batching\.venv\Lib\site-packages\torch\utils\data\dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) diff --git a/experiments/e24-coverage/results/attempt-1/fpodppo-walker/server-seed1.log b/experiments/e24-coverage/results/attempt-1/fpodppo-walker/server-seed1.log new file mode 100644 index 0000000..0576ee3 --- /dev/null +++ b/experiments/e24-coverage/results/attempt-1/fpodppo-walker/server-seed1.log @@ -0,0 +1,13 @@ +17:42:46|INFO|plugrl_server version: 0.1.0 +17:42:46|INFO|Algorithm: dppo, Config: DPPOAlgoConfigWalker(global_steps=81920, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=4096, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=2048, train_itrs=20, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +17:42:46|INFO|Policy: fpo-policy, Config: FPOPolicyConfig(algo='dppo', device=device(type='cpu'), obs_dim=17, action_dim=6, flow_steps=10, timestep_embed_dim=8, action_horizon=1, feather_std=0.0, policy_mlp_output_scale=0.25, normalize_observations=True, hidden_dims=(32, 32, 32, 32), value_hidden_dims=(256, 256, 256, 256, 256)) +17:42:46|INFO|Seeded python, numpy and torch with 1. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +17:42:46|INFO|Checkpoint Manager created: + at D:\75128\Desktop\plugrl-work\plugrl-server\.claude\worktrees\fix-pi0-image-mask-batching\experiments\e24-coverage\results\fpodppo-walker\dppo\fpo-policy\fpodppo-walker-seed1 +17:42:50|INFO|Policy created... +17:42:50|INFO|Initialized RolloutBuffer buffer_size=4096 action_shape=(4096, 10, 1, 6) value_shape=(4096,) +17:42:50|INFO|Algorithm created: + +17:42:50|INFO|Agent Server is listening on 0.0.0.0:9321 +D:\75128\Desktop\plugrl-work\plugrl-server\.claude\worktrees\fix-pi0-image-mask-batching\.venv\Lib\site-packages\torch\utils\data\dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) diff --git a/experiments/e24-coverage/results/attempt-1/fpodppo-walker/server-seed2.log b/experiments/e24-coverage/results/attempt-1/fpodppo-walker/server-seed2.log new file mode 100644 index 0000000..7a2f529 --- /dev/null +++ b/experiments/e24-coverage/results/attempt-1/fpodppo-walker/server-seed2.log @@ -0,0 +1,13 @@ +17:42:54|INFO|plugrl_server version: 0.1.0 +17:42:54|INFO|Algorithm: dppo, Config: DPPOAlgoConfigWalker(global_steps=81920, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=4096, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=2048, train_itrs=20, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +17:42:54|INFO|Policy: fpo-policy, Config: FPOPolicyConfig(algo='dppo', device=device(type='cpu'), obs_dim=17, action_dim=6, flow_steps=10, timestep_embed_dim=8, action_horizon=1, feather_std=0.0, policy_mlp_output_scale=0.25, normalize_observations=True, hidden_dims=(32, 32, 32, 32), value_hidden_dims=(256, 256, 256, 256, 256)) +17:42:54|INFO|Seeded python, numpy and torch with 2. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +17:42:54|INFO|Checkpoint Manager created: + at D:\75128\Desktop\plugrl-work\plugrl-server\.claude\worktrees\fix-pi0-image-mask-batching\experiments\e24-coverage\results\fpodppo-walker\dppo\fpo-policy\fpodppo-walker-seed2 +17:42:59|INFO|Policy created... +17:42:59|INFO|Initialized RolloutBuffer buffer_size=4096 action_shape=(4096, 10, 1, 6) value_shape=(4096,) +17:42:59|INFO|Algorithm created: + +17:42:59|INFO|Agent Server is listening on 0.0.0.0:9322 +D:\75128\Desktop\plugrl-work\plugrl-server\.claude\worktrees\fix-pi0-image-mask-batching\.venv\Lib\site-packages\torch\utils\data\dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) diff --git a/experiments/e24-coverage/results/attempt-1/run.out b/experiments/e24-coverage/results/attempt-1/run.out new file mode 100644 index 0000000..1bbe99f --- /dev/null +++ b/experiments/e24-coverage/results/attempt-1/run.out @@ -0,0 +1,2 @@ +start 2026-09-25 17:41:23 +--- wave 1 From 7af7568434b5e0a6b9f7fa6de4eb0d057f14be03 Mon Sep 17 00:00:00 2001 From: tactino <18781106300@163.com> Date: Fri, 25 Sep 2026 18:30:21 -0400 Subject: [PATCH 3/5] exp: E24 findings - every MuJoCo combination the code allows runs P1 holds 6 of 6: fpo-policy under FPO and DPPO, and dppo-policy - run here for the first time - under DPPO, on HalfCheetah, Hopper and Walker2d, eighteen seeds, no error. P2 holds 3 of 3: FPO learns Walker2d, 964-1101 from about 0. Unregistered: in twenty iterations dppo-policy under DPPO starts to learn Hopper and Walker2d where fpo-policy under DPPO does not move. --- experiments/e24-coverage/FINDINGS.md | 107 ++++++++++++++++++ .../e24-coverage/results/dppo-cheetah.out | 10 ++ .../results/dppo-cheetah/client-seed0.log | 16 +++ .../results/dppo-cheetah/client-seed1.log | 16 +++ .../results/dppo-cheetah/client-seed2.log | 16 +++ .../results/dppo-cheetah/server-seed0.log | 22 ++++ .../results/dppo-cheetah/server-seed1.log | 22 ++++ .../results/dppo-cheetah/server-seed2.log | 22 ++++ .../e24-coverage/results/dppo-hopper.out | 10 ++ .../results/dppo-hopper/client-seed0.log | 16 +++ .../results/dppo-hopper/client-seed1.log | 16 +++ .../results/dppo-hopper/client-seed2.log | 16 +++ .../results/dppo-hopper/server-seed0.log | 22 ++++ .../results/dppo-hopper/server-seed1.log | 22 ++++ .../results/dppo-hopper/server-seed2.log | 22 ++++ .../e24-coverage/results/dppo-walker.out | 10 ++ .../results/dppo-walker/client-seed0.log | 16 +++ .../results/dppo-walker/client-seed1.log | 16 +++ .../results/dppo-walker/client-seed2.log | 16 +++ .../results/dppo-walker/server-seed0.log | 22 ++++ .../results/dppo-walker/server-seed1.log | 22 ++++ .../results/dppo-walker/server-seed2.log | 22 ++++ .../e24-coverage/results/fpo-walker.out | 10 ++ .../results/fpo-walker/client-seed0.log | 51 +++++++++ .../results/fpo-walker/client-seed1.log | 50 ++++++++ .../results/fpo-walker/client-seed2.log | 50 ++++++++ .../results/fpo-walker/server-seed0.log | 23 ++++ .../results/fpo-walker/server-seed1.log | 23 ++++ .../results/fpo-walker/server-seed2.log | 23 ++++ .../e24-coverage/results/fpodppo-hopper.out | 10 ++ .../results/fpodppo-hopper/client-seed0.log | 17 +++ .../results/fpodppo-hopper/client-seed1.log | 17 +++ .../results/fpodppo-hopper/client-seed2.log | 17 +++ .../results/fpodppo-hopper/server-seed0.log | 21 ++++ .../results/fpodppo-hopper/server-seed1.log | 21 ++++ .../results/fpodppo-hopper/server-seed2.log | 21 ++++ .../e24-coverage/results/fpodppo-walker.out | 10 ++ .../results/fpodppo-walker/client-seed0.log | 17 +++ .../results/fpodppo-walker/client-seed1.log | 17 +++ .../results/fpodppo-walker/client-seed2.log | 17 +++ .../results/fpodppo-walker/server-seed0.log | 21 ++++ .../results/fpodppo-walker/server-seed1.log | 21 ++++ .../results/fpodppo-walker/server-seed2.log | 21 ++++ experiments/e24-coverage/results/run.out | 12 ++ experiments/e24-coverage/results/verdicts.txt | 34 ++++++ experiments/e24-coverage/summary.tsv | 19 ++++ 46 files changed, 1022 insertions(+) create mode 100644 experiments/e24-coverage/FINDINGS.md create mode 100644 experiments/e24-coverage/results/dppo-cheetah.out create mode 100644 experiments/e24-coverage/results/dppo-cheetah/client-seed0.log create mode 100644 experiments/e24-coverage/results/dppo-cheetah/client-seed1.log create mode 100644 experiments/e24-coverage/results/dppo-cheetah/client-seed2.log create mode 100644 experiments/e24-coverage/results/dppo-cheetah/server-seed0.log create mode 100644 experiments/e24-coverage/results/dppo-cheetah/server-seed1.log create mode 100644 experiments/e24-coverage/results/dppo-cheetah/server-seed2.log create mode 100644 experiments/e24-coverage/results/dppo-hopper.out create mode 100644 experiments/e24-coverage/results/dppo-hopper/client-seed0.log create mode 100644 experiments/e24-coverage/results/dppo-hopper/client-seed1.log create mode 100644 experiments/e24-coverage/results/dppo-hopper/client-seed2.log create mode 100644 experiments/e24-coverage/results/dppo-hopper/server-seed0.log create mode 100644 experiments/e24-coverage/results/dppo-hopper/server-seed1.log create mode 100644 experiments/e24-coverage/results/dppo-hopper/server-seed2.log create mode 100644 experiments/e24-coverage/results/dppo-walker.out create mode 100644 experiments/e24-coverage/results/dppo-walker/client-seed0.log create mode 100644 experiments/e24-coverage/results/dppo-walker/client-seed1.log create mode 100644 experiments/e24-coverage/results/dppo-walker/client-seed2.log create mode 100644 experiments/e24-coverage/results/dppo-walker/server-seed0.log create mode 100644 experiments/e24-coverage/results/dppo-walker/server-seed1.log create mode 100644 experiments/e24-coverage/results/dppo-walker/server-seed2.log create mode 100644 experiments/e24-coverage/results/fpo-walker.out create mode 100644 experiments/e24-coverage/results/fpo-walker/client-seed0.log create mode 100644 experiments/e24-coverage/results/fpo-walker/client-seed1.log create mode 100644 experiments/e24-coverage/results/fpo-walker/client-seed2.log create mode 100644 experiments/e24-coverage/results/fpo-walker/server-seed0.log create mode 100644 experiments/e24-coverage/results/fpo-walker/server-seed1.log create mode 100644 experiments/e24-coverage/results/fpo-walker/server-seed2.log create mode 100644 experiments/e24-coverage/results/fpodppo-hopper.out create mode 100644 experiments/e24-coverage/results/fpodppo-hopper/client-seed0.log create mode 100644 experiments/e24-coverage/results/fpodppo-hopper/client-seed1.log create mode 100644 experiments/e24-coverage/results/fpodppo-hopper/client-seed2.log create mode 100644 experiments/e24-coverage/results/fpodppo-hopper/server-seed0.log create mode 100644 experiments/e24-coverage/results/fpodppo-hopper/server-seed1.log create mode 100644 experiments/e24-coverage/results/fpodppo-hopper/server-seed2.log create mode 100644 experiments/e24-coverage/results/fpodppo-walker.out create mode 100644 experiments/e24-coverage/results/fpodppo-walker/client-seed0.log create mode 100644 experiments/e24-coverage/results/fpodppo-walker/client-seed1.log create mode 100644 experiments/e24-coverage/results/fpodppo-walker/client-seed2.log create mode 100644 experiments/e24-coverage/results/fpodppo-walker/server-seed0.log create mode 100644 experiments/e24-coverage/results/fpodppo-walker/server-seed1.log create mode 100644 experiments/e24-coverage/results/fpodppo-walker/server-seed2.log create mode 100644 experiments/e24-coverage/results/run.out create mode 100644 experiments/e24-coverage/results/verdicts.txt create mode 100644 experiments/e24-coverage/summary.tsv diff --git a/experiments/e24-coverage/FINDINGS.md b/experiments/e24-coverage/FINDINGS.md new file mode 100644 index 0000000..210f269 --- /dev/null +++ b/experiments/e24-coverage/FINDINGS.md @@ -0,0 +1,107 @@ +# E24: every MuJoCo combination the code allows runs, and FPO learns Walker2d + +2026-09-25 · Linux workstation (`guangzhao`), CPU only · two policies, two +algorithms, three MuJoCo tasks, three seeds per cell · protocol: +[`PROTOCOL.md`](PROTOCOL.md) · amendment: [`AMENDMENT.md`](AMENDMENT.md) - +moved off the Windows machine before any cell finished, and rerun from the start + +--- + +## The result + +PlugRL keeps the policy, the algorithm that trains it and the environment +apart. Before E24, three combinations of the two MLP policies and two +algorithms had run on MuJoCo, and DPPO's own policy class, `dppo-policy`, had +never run on this platform at all. E24 filled the rest of that block. + +**All six cells run end to end, and FPO learns Walker2d on every seed.** + +| cell | policy · algorithm · task | claim | result | +| --- | --- | --- | --- | +| `fpo-walker` | `fpo-policy` · FPO · Walker2d-v5 | learns | **964, 1027, 1101** from −0.3 to 0.7 | +| `fpodppo-hopper` | `fpo-policy` · DPPO · Hopper-v5 | runs | runs, 3 of 3 | +| `fpodppo-walker` | `fpo-policy` · DPPO · Walker2d-v5 | runs | runs, 3 of 3 | +| `dppo-cheetah` | `dppo-policy` · DPPO · HalfCheetah-v5 | runs | runs, 3 of 3 | +| `dppo-hopper` | `dppo-policy` · DPPO · Hopper-v5 | runs | runs, 3 of 3 | +| `dppo-walker` | `dppo-policy` · DPPO · Walker2d-v5 | runs | runs, 3 of 3 | + +**P1 holds, 6 of 6**: on all eighteen seeds every iteration is logged, the +checkpoint is written, no server log holds a traceback and every client +exits 0. **P2 holds, 3 of 3**: the mean of iterations 91-100 of FPO on +Walker2d is 964.0, 1026.5 and 1101.0, against the threshold of 500 and an +untrained first iteration of −0.3 to 0.7; episodes grew from about 19 steps to +about 415. + +With E6, E16 and E23 before it, the MuJoCo block now reads: + +| | HalfCheetah | Hopper | Walker2d | +| --- | --- | --- | --- | +| `fpo-policy` · FPO | learns (E6, E16) | learns (E23) | **learns (E24)** | +| `fpo-policy` · DPPO | runs (E17, E18) | **runs (E24)** | **runs (E24)** | +| `dppo-policy` · DPPO | **runs (E24)** | **runs (E24)** | **runs (E24)** | +| `dppo-policy` · FPO | does not exist: FPO needs a flow policy | | | + +--- + +## One thing the run-only cells showed, unregistered + +No DPPO cell claimed to learn: E17 and E18 found DPPO did not, from a random +initialisation, in `fpo-policy`. Read as description, the twenty iterations +here say that finding belongs to `fpo-policy`, not to DPPO: + +| cell | return, first → mean of last ten | episode length | +| --- | --- | --- | +| `fpo-policy` · DPPO · Hopper | 14.9, 19.7, 17.0 → 15.7, 17.6, 18.1 | ~21 → ~22 | +| `fpo-policy` · DPPO · Walker2d | 1.6, 0.0, 0.5 → 1.4, 0.1, 1.1 | ~20 → ~20 | +| `dppo-policy` · DPPO · Hopper | 11.2, 12.4, 15.8 → **75.7, 77.2, 81.5** | ~19 → **~50** | +| `dppo-policy` · DPPO · Walker2d | −2.2, −1.6, −2.6 → **220.0, 191.9, 156.5** | ~17 → **~194** | +| `dppo-policy` · DPPO · HalfCheetah | −386.3, −445.0, −382.1 → −384.0, −365.7, −418.2 | 1,000 throughout | + +Under the same algorithm, the same buffer of 4,096 environment steps and the +same twenty iterations, DPPO's own policy class starts to learn Hopper and +Walker2d on every seed, and `fpo-policy` does not move. That fits E18's +untested candidate: `fpo-policy` denoises all ten steps under one small +sampling noise, while DPPO's policy clamps each step's noise from below and +predicts a chunk of four actions. It does not test it. Twenty iterations is +also far too short to call anything learned, and HalfCheetah shows nothing +either way. + +--- + +## What this does and does not support + +**Supported:** + +* On this platform, both MLP policies run under DPPO on HalfCheetah, Hopper + and Walker2d, and `fpo-policy` runs under FPO on all three - every + combination of these policies, algorithms and tasks the code allows. +* FPO learns Walker2d-v5 from a random initialisation, to 964-1,101 in + 409,600 steps on three seeds, with defaults tuned for nothing in + particular. +* `dppo-policy`, DPPO's own policy class with its bundled D4RL configurations + and normalisation, runs end to end here for the first time. + +**Not supported:** + +* That DPPO learns any of these tasks. No DPPO cell was registered to learn, + and twenty iterations cannot show it. The early rise in two + `dppo-policy` cells is a reason to run them longer, not a result. +* Anything about wall-clock cost: the machine changed mid-experiment, and + every process was capped at one thread (amendment 1). +* Anything outside the MuJoCo block. robomimic, which `dppo-policy` also has a + configuration for, does not start on the GPU cluster's environments + (`import mujoco_py` fails in both); that is recorded as a gap, not tested + here. + +--- + +## Reproducing + +```bash +OMP_NUM_THREADS=1 bash run.sh # six cells in two waves; about 21 minutes on 24 cores +python summarise.py # P1 cell by cell, P2, the figures above; writes summary.tsv +``` + +`summary.tsv` has one row per cell and seed and is what a coverage figure +should be drawn from. The first attempt, stopped on the Windows machine +before any cell finished, is in `results/attempt-1/` and is not read. diff --git a/experiments/e24-coverage/results/dppo-cheetah.out b/experiments/e24-coverage/results/dppo-cheetah.out new file mode 100644 index 0000000..93871c8 --- /dev/null +++ b/experiments/e24-coverage/results/dppo-cheetah.out @@ -0,0 +1,10 @@ +cell: dppo-cheetah = dppo-policy/cheetah x dppo/cheetah x HalfCheetah-v5 +iters: 20 x 1024 batch: 512 replan: 4 seeds: 0 1 2 +out: /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/dppo-cheetah +start 2026-09-25 18:12:33 +seed 0 finished rc=0 at 18:15:42 +seed 1 finished rc=0 at 18:15:48 +seed 2 finished rc=0 at 18:15:58 +end 2026-09-25 18:15:58 +failed seeds: 0 +CELL_DONE dppo-cheetah diff --git a/experiments/e24-coverage/results/dppo-cheetah/client-seed0.log b/experiments/e24-coverage/results/dppo-cheetah/client-seed0.log new file mode 100644 index 0000000..19020b4 --- /dev/null +++ b/experiments/e24-coverage/results/dppo-cheetah/client-seed0.log @@ -0,0 +1,16 @@ +2026-09-25 18:12:35.317 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 18:12:35.317 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 18:12:35.318 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 18:12:35.318 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 18:12:35.318 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 18:12:35.358 | INFO | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-181235-33ab04bd output_dir=runs/mujoco-v1-nenv1-20260925-181235-33ab04bd recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 18:12:35.394 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9330... +2026-09-25 18:12:35.396 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'DPPOPolicy', 'action_dim': 6, 'action_horizon': 4} +2026-09-25 18:13:05.399 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=16137 infer_calls=4035 feedback_calls=4034 infer_wait=28.209s infer_obs_pack=0.060s env_step=1.024s feedback_total=0.351s feedback_obs_pack=0.088s feedback_info_pack=0.017s effective_fps=544.35 +2026-09-25 18:13:36.187 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=28673 infer_calls=7169 feedback_calls=7168 infer_wait=57.294s infer_obs_pack=0.113s env_step=1.981s feedback_total=0.722s feedback_obs_pack=0.173s feedback_info_pack=0.032s effective_fps=477.01 +2026-09-25 18:14:06.192 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=41789 infer_calls=10448 feedback_calls=10447 infer_wait=85.372s infer_obs_pack=0.173s env_step=3.032s feedback_total=1.177s feedback_obs_pack=0.275s feedback_info_pack=0.049s effective_fps=465.60 +2026-09-25 18:14:36.193 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=54417 infer_calls=13605 feedback_calls=13604 infer_wait=113.497s infer_obs_pack=0.255s env_step=4.056s feedback_total=1.590s feedback_obs_pack=0.369s feedback_info_pack=0.067s effective_fps=455.76 +2026-09-25 18:15:06.198 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=67233 infer_calls=16809 feedback_calls=16808 infer_wait=141.596s infer_obs_pack=0.314s env_step=5.120s feedback_total=2.012s feedback_obs_pack=0.464s feedback_info_pack=0.084s effective_fps=451.10 +2026-09-25 18:15:36.199 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=80385 infer_calls=20097 feedback_calls=20096 infer_wait=169.679s infer_obs_pack=0.372s env_step=6.180s feedback_total=2.448s feedback_obs_pack=0.560s feedback_info_pack=0.103s effective_fps=449.89 +2026-09-25 18:15:41.977 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=81924 infer_calls=20481 feedback_calls=20481 infer_wait=172.071s infer_obs_pack=0.380s env_step=6.314s feedback_total=2.500s feedback_obs_pack=0.572s feedback_info_pack=0.105s effective_fps=451.96 +2026-09-25 18:15:41.977 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 81/81920 episodes. diff --git a/experiments/e24-coverage/results/dppo-cheetah/client-seed1.log b/experiments/e24-coverage/results/dppo-cheetah/client-seed1.log new file mode 100644 index 0000000..30a59e3 --- /dev/null +++ b/experiments/e24-coverage/results/dppo-cheetah/client-seed1.log @@ -0,0 +1,16 @@ +2026-09-25 18:12:40.317 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 18:12:40.317 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 18:12:40.318 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 18:12:40.318 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 18:12:40.318 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 18:12:40.359 | INFO | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-181240-b345e1df output_dir=runs/mujoco-v1-nenv1-20260925-181240-b345e1df recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 18:12:40.394 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9331... +2026-09-25 18:12:40.397 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'DPPOPolicy', 'action_dim': 6, 'action_horizon': 4} +2026-09-25 18:13:10.403 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=16101 infer_calls=4026 feedback_calls=4025 infer_wait=28.153s infer_obs_pack=0.060s env_step=1.059s feedback_total=0.365s feedback_obs_pack=0.088s feedback_info_pack=0.017s effective_fps=543.28 +2026-09-25 18:13:40.945 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=28677 infer_calls=7170 feedback_calls=7169 infer_wait=56.958s infer_obs_pack=0.113s env_step=2.010s feedback_total=0.766s feedback_obs_pack=0.177s feedback_info_pack=0.033s effective_fps=479.17 +2026-09-25 18:14:10.948 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=41197 infer_calls=10300 feedback_calls=10299 infer_wait=85.110s infer_obs_pack=0.172s env_step=3.060s feedback_total=1.154s feedback_obs_pack=0.275s feedback_info_pack=0.050s effective_fps=460.33 +2026-09-25 18:14:40.949 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=53937 infer_calls=13485 feedback_calls=13484 infer_wait=113.188s infer_obs_pack=0.252s env_step=4.128s feedback_total=1.568s feedback_obs_pack=0.376s feedback_info_pack=0.067s effective_fps=452.73 +2026-09-25 18:15:10.956 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=65821 infer_calls=16456 feedback_calls=16455 infer_wait=141.420s infer_obs_pack=0.307s env_step=5.132s feedback_total=1.943s feedback_obs_pack=0.464s feedback_info_pack=0.083s effective_fps=442.34 +2026-09-25 18:15:40.958 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=79205 infer_calls=19802 feedback_calls=19801 infer_wait=169.553s infer_obs_pack=0.365s env_step=6.175s feedback_total=2.354s feedback_obs_pack=0.557s feedback_info_pack=0.100s effective_fps=443.86 +2026-09-25 18:15:48.662 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=81924 infer_calls=20481 feedback_calls=20481 infer_wait=173.700s infer_obs_pack=0.380s env_step=6.406s feedback_total=2.437s feedback_obs_pack=0.576s feedback_info_pack=0.103s effective_fps=447.86 +2026-09-25 18:15:48.662 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 81/81920 episodes. diff --git a/experiments/e24-coverage/results/dppo-cheetah/client-seed2.log b/experiments/e24-coverage/results/dppo-cheetah/client-seed2.log new file mode 100644 index 0000000..e5b58ba --- /dev/null +++ b/experiments/e24-coverage/results/dppo-cheetah/client-seed2.log @@ -0,0 +1,16 @@ +2026-09-25 18:12:45.329 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 18:12:45.329 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 18:12:45.330 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 18:12:45.330 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 18:12:45.331 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 18:12:45.375 | INFO | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-181245-e752fa46 output_dir=runs/mujoco-v1-nenv1-20260925-181245-e752fa46 recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 18:12:45.413 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9332... +2026-09-25 18:12:45.416 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'DPPOPolicy', 'action_dim': 6, 'action_horizon': 4} +2026-09-25 18:13:15.421 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=15717 infer_calls=3930 feedback_calls=3929 infer_wait=28.141s infer_obs_pack=0.062s env_step=1.050s feedback_total=0.388s feedback_obs_pack=0.091s feedback_info_pack=0.018s effective_fps=530.25 +2026-09-25 18:13:47.343 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=28677 infer_calls=7170 feedback_calls=7169 infer_wait=58.263s infer_obs_pack=0.117s env_step=2.051s feedback_total=0.791s feedback_obs_pack=0.179s feedback_info_pack=0.036s effective_fps=468.40 +2026-09-25 18:14:18.853 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=40965 infer_calls=10242 feedback_calls=10241 infer_wait=87.849s infer_obs_pack=0.182s env_step=3.150s feedback_total=1.181s feedback_obs_pack=0.280s feedback_info_pack=0.053s effective_fps=443.52 +2026-09-25 18:14:50.221 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=53253 infer_calls=13314 feedback_calls=13313 infer_wait=117.372s infer_obs_pack=0.264s env_step=4.182s feedback_total=1.565s feedback_obs_pack=0.370s feedback_info_pack=0.072s effective_fps=431.61 +2026-09-25 18:15:20.592 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=65541 infer_calls=16386 feedback_calls=16385 infer_wait=145.904s infer_obs_pack=0.322s env_step=5.217s feedback_total=1.964s feedback_obs_pack=0.467s feedback_info_pack=0.092s effective_fps=427.24 +2026-09-25 18:15:50.593 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=78685 infer_calls=19672 feedback_calls=19671 infer_wait=174.059s infer_obs_pack=0.379s env_step=6.252s feedback_total=2.371s feedback_obs_pack=0.563s feedback_info_pack=0.110s effective_fps=429.83 +2026-09-25 18:15:58.212 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=81924 infer_calls=20481 feedback_calls=20481 infer_wait=178.130s infer_obs_pack=0.392s env_step=6.484s feedback_total=2.462s feedback_obs_pack=0.583s feedback_info_pack=0.114s effective_fps=437.00 +2026-09-25 18:15:58.212 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 81/81920 episodes. diff --git a/experiments/e24-coverage/results/dppo-cheetah/server-seed0.log b/experiments/e24-coverage/results/dppo-cheetah/server-seed0.log new file mode 100644 index 0000000..a1c9c5c --- /dev/null +++ b/experiments/e24-coverage/results/dppo-cheetah/server-seed0.log @@ -0,0 +1,22 @@ +18:12:34|INFO|plugrl_server version: 0.1.0 +18:12:34|INFO|Algorithm: dppo, Config: DPPOAlgoConfigCheetah(global_steps=20480, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=1024, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=512, train_itrs=20, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +18:12:34|INFO|Policy: dppo-policy, Config: DPPOPolicyConfigCheetah(algo='dppo', device=device(type='cpu'), env_type='gym', env_name='halfcheetah-medium-v2', checkpoint_path=None, critic=DPPOCriticObsConfig(mlp_dims=[256, 256, 256], activation='Mish', residual_style=True)) +18:12:34|INFO|Seeded python, numpy and torch with 0. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +18:12:34|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/dppo-cheetah/dppo/dppo-policy/dppo-cheetah-seed0 +18:12:35|INFO|Number of network parameters: 568392 +18:12:35|INFO|Policy created... +18:12:35|INFO|Initialized RolloutBuffer buffer_size=1024 action_shape=(1024, 20, 4, 6) value_shape=(1024,) +18:12:35|INFO|Algorithm created: + +18:12:35|INFO|Agent Server is listening on 0.0.0.0:9330 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +18:15:41|INFO|Checkpoint saved at step 20481 to /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/dppo-cheetah/dppo/dppo-policy/dppo-cheetah-seed0/20481 +18:15:41|INFO|Stopping server as the algorithm signaled to stop. +18:15:41|INFO|Shutdown started: aborting pending infer requests. +18:15:41|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +18:15:41|INFO|Shutdown closing 1 websocket connection(s). +18:15:41|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 55466). +18:15:41|INFO|WebSocket server closed. +18:15:41|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e24-coverage/results/dppo-cheetah/server-seed1.log b/experiments/e24-coverage/results/dppo-cheetah/server-seed1.log new file mode 100644 index 0000000..2e3b18f --- /dev/null +++ b/experiments/e24-coverage/results/dppo-cheetah/server-seed1.log @@ -0,0 +1,22 @@ +18:12:40|INFO|plugrl_server version: 0.1.0 +18:12:40|INFO|Algorithm: dppo, Config: DPPOAlgoConfigCheetah(global_steps=20480, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=1024, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=512, train_itrs=20, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +18:12:40|INFO|Policy: dppo-policy, Config: DPPOPolicyConfigCheetah(algo='dppo', device=device(type='cpu'), env_type='gym', env_name='halfcheetah-medium-v2', checkpoint_path=None, critic=DPPOCriticObsConfig(mlp_dims=[256, 256, 256], activation='Mish', residual_style=True)) +18:12:40|INFO|Seeded python, numpy and torch with 1. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +18:12:40|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/dppo-cheetah/dppo/dppo-policy/dppo-cheetah-seed1 +18:12:40|INFO|Number of network parameters: 568392 +18:12:40|INFO|Policy created... +18:12:40|INFO|Initialized RolloutBuffer buffer_size=1024 action_shape=(1024, 20, 4, 6) value_shape=(1024,) +18:12:40|INFO|Algorithm created: + +18:12:40|INFO|Agent Server is listening on 0.0.0.0:9331 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +18:15:48|INFO|Checkpoint saved at step 20481 to /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/dppo-cheetah/dppo/dppo-policy/dppo-cheetah-seed1/20481 +18:15:48|INFO|Stopping server as the algorithm signaled to stop. +18:15:48|INFO|Shutdown started: aborting pending infer requests. +18:15:48|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +18:15:48|INFO|Shutdown closing 1 websocket connection(s). +18:15:48|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 56160). +18:15:48|INFO|WebSocket server closed. +18:15:48|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e24-coverage/results/dppo-cheetah/server-seed2.log b/experiments/e24-coverage/results/dppo-cheetah/server-seed2.log new file mode 100644 index 0000000..f68e6fd --- /dev/null +++ b/experiments/e24-coverage/results/dppo-cheetah/server-seed2.log @@ -0,0 +1,22 @@ +18:12:45|INFO|plugrl_server version: 0.1.0 +18:12:45|INFO|Algorithm: dppo, Config: DPPOAlgoConfigCheetah(global_steps=20480, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=1024, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=512, train_itrs=20, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +18:12:45|INFO|Policy: dppo-policy, Config: DPPOPolicyConfigCheetah(algo='dppo', device=device(type='cpu'), env_type='gym', env_name='halfcheetah-medium-v2', checkpoint_path=None, critic=DPPOCriticObsConfig(mlp_dims=[256, 256, 256], activation='Mish', residual_style=True)) +18:12:45|INFO|Seeded python, numpy and torch with 2. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +18:12:45|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/dppo-cheetah/dppo/dppo-policy/dppo-cheetah-seed2 +18:12:45|INFO|Number of network parameters: 568392 +18:12:45|INFO|Policy created... +18:12:45|INFO|Initialized RolloutBuffer buffer_size=1024 action_shape=(1024, 20, 4, 6) value_shape=(1024,) +18:12:45|INFO|Algorithm created: + +18:12:45|INFO|Agent Server is listening on 0.0.0.0:9332 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +18:15:58|INFO|Checkpoint saved at step 20481 to /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/dppo-cheetah/dppo/dppo-policy/dppo-cheetah-seed2/20481 +18:15:58|INFO|Stopping server as the algorithm signaled to stop. +18:15:58|INFO|Shutdown started: aborting pending infer requests. +18:15:58|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +18:15:58|INFO|Shutdown closing 1 websocket connection(s). +18:15:58|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 52766). +18:15:58|INFO|WebSocket server closed. +18:15:58|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e24-coverage/results/dppo-hopper.out b/experiments/e24-coverage/results/dppo-hopper.out new file mode 100644 index 0000000..2ae4f98 --- /dev/null +++ b/experiments/e24-coverage/results/dppo-hopper.out @@ -0,0 +1,10 @@ +cell: dppo-hopper = dppo-policy/hopper x dppo/hopper x Hopper-v5 +iters: 20 x 1024 batch: 512 replan: 4 seeds: 0 1 2 +out: /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/dppo-hopper +start 2026-09-25 18:12:53 +seed 0 finished rc=0 at 18:16:10 +seed 1 finished rc=0 at 18:16:16 +seed 2 finished rc=0 at 18:16:23 +end 2026-09-25 18:16:23 +failed seeds: 0 +CELL_DONE dppo-hopper diff --git a/experiments/e24-coverage/results/dppo-hopper/client-seed0.log b/experiments/e24-coverage/results/dppo-hopper/client-seed0.log new file mode 100644 index 0000000..142f0c2 --- /dev/null +++ b/experiments/e24-coverage/results/dppo-hopper/client-seed0.log @@ -0,0 +1,16 @@ +2026-09-25 18:12:55.322 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 18:12:55.323 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 18:12:55.323 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 18:12:55.323 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 18:12:55.324 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 18:12:55.366 | INFO | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-181255-f978ccd1 output_dir=runs/mujoco-v1-nenv1-20260925-181255-f978ccd1 recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 18:12:55.407 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9340... +2026-09-25 18:12:55.410 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'DPPOPolicy', 'action_dim': 3, 'action_horizon': 4} +2026-09-25 18:13:25.415 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=13757 infer_calls=3705 feedback_calls=3704 infer_wait=27.480s infer_obs_pack=0.057s env_step=1.637s feedback_total=0.389s feedback_obs_pack=0.098s feedback_info_pack=0.021s effective_fps=465.35 +2026-09-25 18:13:55.415 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=25540 infer_calls=6799 feedback_calls=6799 infer_wait=55.007s infer_obs_pack=0.112s env_step=3.268s feedback_total=0.782s feedback_obs_pack=0.190s feedback_info_pack=0.040s effective_fps=431.64 +2026-09-25 18:14:25.417 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=37624 infer_calls=9923 feedback_calls=9922 infer_wait=82.502s infer_obs_pack=0.167s env_step=4.930s feedback_total=1.179s feedback_obs_pack=0.287s feedback_info_pack=0.061s effective_fps=423.80 +2026-09-25 18:14:55.419 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=48799 infer_calls=12798 feedback_calls=12797 infer_wait=110.107s infer_obs_pack=0.221s env_step=6.543s feedback_total=1.545s feedback_obs_pack=0.377s feedback_info_pack=0.080s effective_fps=412.10 +2026-09-25 18:15:25.419 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=60513 infer_calls=15830 feedback_calls=15829 infer_wait=137.635s infer_obs_pack=0.302s env_step=8.173s feedback_total=1.928s feedback_obs_pack=0.467s feedback_info_pack=0.100s effective_fps=408.76 +2026-09-25 18:15:55.421 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=72411 infer_calls=18899 feedback_calls=18898 infer_wait=165.176s infer_obs_pack=0.360s env_step=9.811s feedback_total=2.302s feedback_obs_pack=0.562s feedback_info_pack=0.120s effective_fps=407.61 +2026-09-25 18:16:10.728 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=78544 infer_calls=20481 feedback_calls=20481 infer_wait=176.455s infer_obs_pack=0.385s env_step=10.553s feedback_total=2.476s feedback_obs_pack=0.603s feedback_info_pack=0.129s effective_fps=413.67 +2026-09-25 18:16:10.728 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 2216/81920 episodes. diff --git a/experiments/e24-coverage/results/dppo-hopper/client-seed1.log b/experiments/e24-coverage/results/dppo-hopper/client-seed1.log new file mode 100644 index 0000000..0954e44 --- /dev/null +++ b/experiments/e24-coverage/results/dppo-hopper/client-seed1.log @@ -0,0 +1,16 @@ +2026-09-25 18:13:01.324 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 18:13:01.325 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 18:13:01.325 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 18:13:01.325 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 18:13:01.326 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 18:13:01.371 | INFO | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-181301-6941068b output_dir=runs/mujoco-v1-nenv1-20260925-181301-6941068b recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 18:13:01.412 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9341... +2026-09-25 18:13:01.415 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'DPPOPolicy', 'action_dim': 3, 'action_horizon': 4} +2026-09-25 18:13:31.421 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=12661 infer_calls=3431 feedback_calls=3430 infer_wait=27.486s infer_obs_pack=0.058s env_step=1.633s feedback_total=0.391s feedback_obs_pack=0.091s feedback_info_pack=0.022s effective_fps=428.21 +2026-09-25 18:14:01.424 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=24479 infer_calls=6546 feedback_calls=6545 infer_wait=54.996s infer_obs_pack=0.117s env_step=3.265s feedback_total=0.789s feedback_obs_pack=0.188s feedback_info_pack=0.046s effective_fps=413.73 +2026-09-25 18:14:31.427 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=36050 infer_calls=9538 feedback_calls=9537 infer_wait=82.561s infer_obs_pack=0.170s env_step=4.891s feedback_total=1.159s feedback_obs_pack=0.278s feedback_info_pack=0.066s effective_fps=406.06 +2026-09-25 18:15:01.429 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=47525 infer_calls=12496 feedback_calls=12495 infer_wait=110.134s infer_obs_pack=0.227s env_step=6.516s feedback_total=1.537s feedback_obs_pack=0.366s feedback_info_pack=0.085s effective_fps=401.34 +2026-09-25 18:15:31.432 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=59658 infer_calls=15632 feedback_calls=15631 infer_wait=137.645s infer_obs_pack=0.300s env_step=8.174s feedback_total=1.912s feedback_obs_pack=0.454s feedback_info_pack=0.105s effective_fps=403.01 +2026-09-25 18:16:01.432 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=72402 infer_calls=18916 feedback_calls=18915 infer_wait=165.199s infer_obs_pack=0.355s env_step=9.808s feedback_total=2.287s feedback_obs_pack=0.542s feedback_info_pack=0.124s effective_fps=407.56 +2026-09-25 18:16:16.279 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=78502 infer_calls=20481 feedback_calls=20481 infer_wait=176.055s infer_obs_pack=0.380s env_step=10.514s feedback_total=2.445s feedback_obs_pack=0.580s feedback_info_pack=0.133s effective_fps=414.49 +2026-09-25 18:16:16.279 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 2292/81920 episodes. diff --git a/experiments/e24-coverage/results/dppo-hopper/client-seed2.log b/experiments/e24-coverage/results/dppo-hopper/client-seed2.log new file mode 100644 index 0000000..b1747d9 --- /dev/null +++ b/experiments/e24-coverage/results/dppo-hopper/client-seed2.log @@ -0,0 +1,16 @@ +2026-09-25 18:13:06.337 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 18:13:06.337 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 18:13:06.337 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 18:13:06.337 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 18:13:06.338 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 18:13:06.384 | INFO | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-181306-c86933fa output_dir=runs/mujoco-v1-nenv1-20260925-181306-c86933fa recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 18:13:06.431 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9342... +2026-09-25 18:13:06.435 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'DPPOPolicy', 'action_dim': 3, 'action_horizon': 4} +2026-09-25 18:13:36.440 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=12810 infer_calls=3404 feedback_calls=3403 infer_wait=27.481s infer_obs_pack=0.059s env_step=1.647s feedback_total=0.395s feedback_obs_pack=0.095s feedback_info_pack=0.020s effective_fps=433.03 +2026-09-25 18:14:06.447 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=24151 infer_calls=6369 feedback_calls=6368 infer_wait=55.060s infer_obs_pack=0.112s env_step=3.263s feedback_total=0.760s feedback_obs_pack=0.189s feedback_info_pack=0.039s effective_fps=407.99 +2026-09-25 18:14:36.448 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=35787 infer_calls=9382 feedback_calls=9381 infer_wait=82.583s infer_obs_pack=0.167s env_step=4.917s feedback_total=1.143s feedback_obs_pack=0.282s feedback_info_pack=0.058s effective_fps=402.96 +2026-09-25 18:15:07.521 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=47082 infer_calls=12290 feedback_calls=12289 infer_wait=111.164s infer_obs_pack=0.224s env_step=6.599s feedback_total=1.518s feedback_obs_pack=0.374s feedback_info_pack=0.078s effective_fps=393.97 +2026-09-25 18:15:37.640 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=59038 infer_calls=15362 feedback_calls=15361 infer_wait=138.762s infer_obs_pack=0.294s env_step=8.298s feedback_total=1.888s feedback_obs_pack=0.467s feedback_info_pack=0.099s effective_fps=395.59 +2026-09-25 18:16:07.641 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=72000 infer_calls=18688 feedback_calls=18687 infer_wait=166.276s infer_obs_pack=0.351s env_step=9.962s feedback_total=2.276s feedback_obs_pack=0.566s feedback_info_pack=0.118s effective_fps=402.54 +2026-09-25 18:16:23.477 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=79007 infer_calls=20481 feedback_calls=20481 infer_wait=178.054s infer_obs_pack=0.378s env_step=10.769s feedback_total=2.452s feedback_obs_pack=0.614s feedback_info_pack=0.127s effective_fps=412.24 +2026-09-25 18:16:23.478 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 1924/81920 episodes. diff --git a/experiments/e24-coverage/results/dppo-hopper/server-seed0.log b/experiments/e24-coverage/results/dppo-hopper/server-seed0.log new file mode 100644 index 0000000..5cd8229 --- /dev/null +++ b/experiments/e24-coverage/results/dppo-hopper/server-seed0.log @@ -0,0 +1,22 @@ +18:12:55|INFO|plugrl_server version: 0.1.0 +18:12:55|INFO|Algorithm: dppo, Config: DPPOAlgoConfigHopper(global_steps=20480, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=1024, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=512, train_itrs=20, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +18:12:55|INFO|Policy: dppo-policy, Config: DPPOPolicyConfigHopper(algo='dppo', device=device(type='cpu'), env_type='gym', env_name='hopper-medium-v2', checkpoint_path=None, critic=DPPOCriticObsConfig(mlp_dims=[256, 256, 256], activation='Mish', residual_style=True)) +18:12:55|INFO|Seeded python, numpy and torch with 0. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +18:12:55|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed0 +18:12:55|INFO|Number of network parameters: 553020 +18:12:55|INFO|Policy created... +18:12:55|INFO|Initialized RolloutBuffer buffer_size=1024 action_shape=(1024, 20, 4, 3) value_shape=(1024,) +18:12:55|INFO|Algorithm created: + +18:12:55|INFO|Agent Server is listening on 0.0.0.0:9340 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +18:16:10|INFO|Checkpoint saved at step 20481 to /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed0/20481 +18:16:10|INFO|Stopping server as the algorithm signaled to stop. +18:16:10|INFO|Shutdown started: aborting pending infer requests. +18:16:10|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +18:16:10|INFO|Shutdown closing 1 websocket connection(s). +18:16:10|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 45478). +18:16:10|INFO|WebSocket server closed. +18:16:10|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e24-coverage/results/dppo-hopper/server-seed1.log b/experiments/e24-coverage/results/dppo-hopper/server-seed1.log new file mode 100644 index 0000000..18cbb1f --- /dev/null +++ b/experiments/e24-coverage/results/dppo-hopper/server-seed1.log @@ -0,0 +1,22 @@ +18:13:00|INFO|plugrl_server version: 0.1.0 +18:13:00|INFO|Algorithm: dppo, Config: DPPOAlgoConfigHopper(global_steps=20480, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=1024, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=512, train_itrs=20, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +18:13:00|INFO|Policy: dppo-policy, Config: DPPOPolicyConfigHopper(algo='dppo', device=device(type='cpu'), env_type='gym', env_name='hopper-medium-v2', checkpoint_path=None, critic=DPPOCriticObsConfig(mlp_dims=[256, 256, 256], activation='Mish', residual_style=True)) +18:13:00|INFO|Seeded python, numpy and torch with 1. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +18:13:00|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed1 +18:13:00|INFO|Number of network parameters: 553020 +18:13:00|INFO|Policy created... +18:13:00|INFO|Initialized RolloutBuffer buffer_size=1024 action_shape=(1024, 20, 4, 3) value_shape=(1024,) +18:13:00|INFO|Algorithm created: + +18:13:00|INFO|Agent Server is listening on 0.0.0.0:9341 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +18:16:16|INFO|Checkpoint saved at step 20481 to /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed1/20481 +18:16:16|INFO|Stopping server as the algorithm signaled to stop. +18:16:16|INFO|Shutdown started: aborting pending infer requests. +18:16:16|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +18:16:16|INFO|Shutdown closing 1 websocket connection(s). +18:16:16|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 34002). +18:16:16|INFO|WebSocket server closed. +18:16:16|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e24-coverage/results/dppo-hopper/server-seed2.log b/experiments/e24-coverage/results/dppo-hopper/server-seed2.log new file mode 100644 index 0000000..5032957 --- /dev/null +++ b/experiments/e24-coverage/results/dppo-hopper/server-seed2.log @@ -0,0 +1,22 @@ +18:13:05|INFO|plugrl_server version: 0.1.0 +18:13:05|INFO|Algorithm: dppo, Config: DPPOAlgoConfigHopper(global_steps=20480, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=1024, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=512, train_itrs=20, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +18:13:05|INFO|Policy: dppo-policy, Config: DPPOPolicyConfigHopper(algo='dppo', device=device(type='cpu'), env_type='gym', env_name='hopper-medium-v2', checkpoint_path=None, critic=DPPOCriticObsConfig(mlp_dims=[256, 256, 256], activation='Mish', residual_style=True)) +18:13:05|INFO|Seeded python, numpy and torch with 2. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +18:13:05|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed2 +18:13:05|INFO|Number of network parameters: 553020 +18:13:05|INFO|Policy created... +18:13:05|INFO|Initialized RolloutBuffer buffer_size=1024 action_shape=(1024, 20, 4, 3) value_shape=(1024,) +18:13:05|INFO|Algorithm created: + +18:13:05|INFO|Agent Server is listening on 0.0.0.0:9342 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +18:16:23|INFO|Checkpoint saved at step 20481 to /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed2/20481 +18:16:23|INFO|Stopping server as the algorithm signaled to stop. +18:16:23|INFO|Shutdown started: aborting pending infer requests. +18:16:23|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +18:16:23|INFO|Shutdown closing 1 websocket connection(s). +18:16:23|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 38224). +18:16:23|INFO|WebSocket server closed. +18:16:23|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e24-coverage/results/dppo-walker.out b/experiments/e24-coverage/results/dppo-walker.out new file mode 100644 index 0000000..8423162 --- /dev/null +++ b/experiments/e24-coverage/results/dppo-walker.out @@ -0,0 +1,10 @@ +cell: dppo-walker = dppo-policy/walker x dppo/walker x Walker2d-v5 +iters: 20 x 1024 batch: 512 replan: 4 seeds: 0 1 2 +out: /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/dppo-walker +start 2026-09-25 18:13:13 +seed 0 finished rc=0 at 18:16:34 +seed 1 finished rc=0 at 18:16:41 +seed 2 finished rc=0 at 18:16:44 +end 2026-09-25 18:16:44 +failed seeds: 0 +CELL_DONE dppo-walker diff --git a/experiments/e24-coverage/results/dppo-walker/client-seed0.log b/experiments/e24-coverage/results/dppo-walker/client-seed0.log new file mode 100644 index 0000000..20f8b26 --- /dev/null +++ b/experiments/e24-coverage/results/dppo-walker/client-seed0.log @@ -0,0 +1,16 @@ +2026-09-25 18:13:16.346 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 18:13:16.346 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 18:13:16.346 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 18:13:16.346 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 18:13:16.347 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 18:13:16.397 | INFO | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-181316-1a5949c1 output_dir=runs/mujoco-v1-nenv1-20260925-181316-1a5949c1 recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 18:13:16.443 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9350... +2026-09-25 18:13:16.446 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'DPPOPolicy', 'action_dim': 6, 'action_horizon': 4} +2026-09-25 18:13:46.685 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=11441 infer_calls=3074 feedback_calls=3073 infer_wait=27.703s infer_obs_pack=0.056s env_step=1.674s feedback_total=0.374s feedback_obs_pack=0.095s feedback_info_pack=0.019s effective_fps=383.83 +2026-09-25 18:14:16.689 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=23417 infer_calls=6191 feedback_calls=6190 infer_wait=55.129s infer_obs_pack=0.115s env_step=3.410s feedback_total=0.764s feedback_obs_pack=0.192s feedback_info_pack=0.040s effective_fps=394.11 +2026-09-25 18:14:47.878 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=35332 infer_calls=9218 feedback_calls=9217 infer_wait=83.697s infer_obs_pack=0.171s env_step=5.239s feedback_total=1.135s feedback_obs_pack=0.283s feedback_info_pack=0.059s effective_fps=391.52 +2026-09-25 18:15:18.856 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=47496 infer_calls=12289 feedback_calls=12288 infer_wait=112.079s infer_obs_pack=0.224s env_step=7.037s feedback_total=1.519s feedback_obs_pack=0.373s feedback_info_pack=0.081s effective_fps=392.99 +2026-09-25 18:15:49.302 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=59679 infer_calls=15362 feedback_calls=15361 infer_wait=139.908s infer_obs_pack=0.293s env_step=8.856s feedback_total=1.898s feedback_obs_pack=0.464s feedback_info_pack=0.099s effective_fps=395.34 +2026-09-25 18:16:19.302 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=73430 infer_calls=18822 feedback_calls=18822 infer_wait=167.328s infer_obs_pack=0.347s env_step=10.680s feedback_total=2.248s feedback_obs_pack=0.555s feedback_info_pack=0.117s effective_fps=406.58 +2026-09-25 18:16:34.374 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=80025 infer_calls=20481 feedback_calls=20481 infer_wait=178.359s infer_obs_pack=0.370s env_step=11.450s feedback_total=2.389s feedback_obs_pack=0.591s feedback_info_pack=0.125s effective_fps=415.57 +2026-09-25 18:16:34.374 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 1232/81920 episodes. diff --git a/experiments/e24-coverage/results/dppo-walker/client-seed1.log b/experiments/e24-coverage/results/dppo-walker/client-seed1.log new file mode 100644 index 0000000..689dcb8 --- /dev/null +++ b/experiments/e24-coverage/results/dppo-walker/client-seed1.log @@ -0,0 +1,16 @@ +2026-09-25 18:13:21.339 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 18:13:21.339 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 18:13:21.339 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 18:13:21.339 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 18:13:21.340 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 18:13:21.389 | INFO | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-181321-39618e80 output_dir=runs/mujoco-v1-nenv1-20260925-181321-39618e80 recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 18:13:21.432 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9351... +2026-09-25 18:13:21.438 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'DPPOPolicy', 'action_dim': 6, 'action_horizon': 4} +2026-09-25 18:13:52.116 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=11474 infer_calls=3074 feedback_calls=3073 infer_wait=28.094s infer_obs_pack=0.054s env_step=1.713s feedback_total=0.387s feedback_obs_pack=0.091s feedback_info_pack=0.024s effective_fps=379.33 +2026-09-25 18:14:23.274 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=23357 infer_calls=6146 feedback_calls=6145 infer_wait=56.683s infer_obs_pack=0.110s env_step=3.456s feedback_total=0.772s feedback_obs_pack=0.184s feedback_info_pack=0.043s effective_fps=382.77 +2026-09-25 18:14:55.855 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=35450 infer_calls=9217 feedback_calls=9216 infer_wait=86.597s infer_obs_pack=0.168s env_step=5.300s feedback_total=1.165s feedback_obs_pack=0.279s feedback_info_pack=0.064s effective_fps=380.24 +2026-09-25 18:15:26.525 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=47605 infer_calls=12290 feedback_calls=12289 infer_wait=114.656s infer_obs_pack=0.225s env_step=7.097s feedback_total=1.557s feedback_obs_pack=0.371s feedback_info_pack=0.087s effective_fps=385.35 +2026-09-25 18:15:56.772 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=59794 infer_calls=15362 feedback_calls=15361 infer_wait=142.294s infer_obs_pack=0.300s env_step=8.893s feedback_total=1.935s feedback_obs_pack=0.461s feedback_info_pack=0.107s effective_fps=389.74 +2026-09-25 18:16:26.774 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=73863 infer_calls=18902 feedback_calls=18901 infer_wait=169.744s infer_obs_pack=0.355s env_step=10.694s feedback_total=2.283s feedback_obs_pack=0.547s feedback_info_pack=0.125s effective_fps=403.46 +2026-09-25 18:16:41.393 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=80135 infer_calls=20481 feedback_calls=20481 infer_wait=180.403s infer_obs_pack=0.377s env_step=11.435s feedback_total=2.411s feedback_obs_pack=0.581s feedback_info_pack=0.131s effective_fps=411.74 +2026-09-25 18:16:41.393 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 1177/81920 episodes. diff --git a/experiments/e24-coverage/results/dppo-walker/client-seed2.log b/experiments/e24-coverage/results/dppo-walker/client-seed2.log new file mode 100644 index 0000000..02689fd --- /dev/null +++ b/experiments/e24-coverage/results/dppo-walker/client-seed2.log @@ -0,0 +1,16 @@ +2026-09-25 18:13:26.348 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 18:13:26.349 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 18:13:26.349 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 18:13:26.349 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 18:13:26.350 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 18:13:26.399 | INFO | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-181326-ca20b577 output_dir=runs/mujoco-v1-nenv1-20260925-181326-ca20b577 recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 18:13:26.439 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9352... +2026-09-25 18:13:26.444 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'DPPOPolicy', 'action_dim': 6, 'action_horizon': 4} +2026-09-25 18:13:57.135 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=11245 infer_calls=3074 feedback_calls=3073 infer_wait=28.208s infer_obs_pack=0.056s env_step=1.605s feedback_total=0.386s feedback_obs_pack=0.092s feedback_info_pack=0.021s effective_fps=371.67 +2026-09-25 18:14:28.858 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=22705 infer_calls=6145 feedback_calls=6144 infer_wait=57.366s infer_obs_pack=0.113s env_step=3.293s feedback_total=0.778s feedback_obs_pack=0.199s feedback_info_pack=0.041s effective_fps=368.89 +2026-09-25 18:15:01.444 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=34656 infer_calls=9218 feedback_calls=9217 infer_wait=87.315s infer_obs_pack=0.170s env_step=5.107s feedback_total=1.158s feedback_obs_pack=0.297s feedback_info_pack=0.060s effective_fps=369.66 +2026-09-25 18:15:31.447 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=46859 infer_calls=12305 feedback_calls=12304 infer_wait=114.746s infer_obs_pack=0.225s env_step=6.896s feedback_total=1.528s feedback_obs_pack=0.390s feedback_info_pack=0.078s effective_fps=379.75 +2026-09-25 18:16:01.452 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=59208 infer_calls=15418 feedback_calls=15417 infer_wait=142.145s infer_obs_pack=0.298s env_step=8.691s feedback_total=1.911s feedback_obs_pack=0.484s feedback_info_pack=0.097s effective_fps=386.87 +2026-09-25 18:16:31.457 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=73939 infer_calls=19128 feedback_calls=19127 infer_wait=169.548s infer_obs_pack=0.354s env_step=10.526s feedback_total=2.262s feedback_obs_pack=0.577s feedback_info_pack=0.114s effective_fps=404.73 +2026-09-25 18:16:44.348 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=79321 infer_calls=20481 feedback_calls=20481 infer_wait=178.774s infer_obs_pack=0.372s env_step=11.117s feedback_total=2.366s feedback_obs_pack=0.604s feedback_info_pack=0.120s effective_fps=411.78 +2026-09-25 18:16:44.348 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 1761/81920 episodes. diff --git a/experiments/e24-coverage/results/dppo-walker/server-seed0.log b/experiments/e24-coverage/results/dppo-walker/server-seed0.log new file mode 100644 index 0000000..2b8aea4 --- /dev/null +++ b/experiments/e24-coverage/results/dppo-walker/server-seed0.log @@ -0,0 +1,22 @@ +18:13:15|INFO|plugrl_server version: 0.1.0 +18:13:15|INFO|Algorithm: dppo, Config: DPPOAlgoConfigWalker(global_steps=20480, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=1024, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=512, train_itrs=20, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +18:13:15|INFO|Policy: dppo-policy, Config: DPPOPolicyConfigWalker(algo='dppo', device=device(type='cpu'), env_type='gym', env_name='walker2d-medium-v2', checkpoint_path=None, critic=DPPOCriticObsConfig(mlp_dims=[256, 256, 256], activation='Mish', residual_style=True)) +18:13:15|INFO|Seeded python, numpy and torch with 0. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +18:13:15|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed0 +18:13:15|INFO|Number of network parameters: 568392 +18:13:15|INFO|Policy created... +18:13:15|INFO|Initialized RolloutBuffer buffer_size=1024 action_shape=(1024, 20, 4, 6) value_shape=(1024,) +18:13:15|INFO|Algorithm created: + +18:13:15|INFO|Agent Server is listening on 0.0.0.0:9350 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +18:16:34|INFO|Checkpoint saved at step 20481 to /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed0/20481 +18:16:34|INFO|Stopping server as the algorithm signaled to stop. +18:16:34|INFO|Shutdown started: aborting pending infer requests. +18:16:34|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +18:16:34|INFO|Shutdown closing 1 websocket connection(s). +18:16:34|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 47800). +18:16:34|INFO|WebSocket server closed. +18:16:34|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e24-coverage/results/dppo-walker/server-seed1.log b/experiments/e24-coverage/results/dppo-walker/server-seed1.log new file mode 100644 index 0000000..b99e097 --- /dev/null +++ b/experiments/e24-coverage/results/dppo-walker/server-seed1.log @@ -0,0 +1,22 @@ +18:13:20|INFO|plugrl_server version: 0.1.0 +18:13:20|INFO|Algorithm: dppo, Config: DPPOAlgoConfigWalker(global_steps=20480, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=1024, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=512, train_itrs=20, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +18:13:20|INFO|Policy: dppo-policy, Config: DPPOPolicyConfigWalker(algo='dppo', device=device(type='cpu'), env_type='gym', env_name='walker2d-medium-v2', checkpoint_path=None, critic=DPPOCriticObsConfig(mlp_dims=[256, 256, 256], activation='Mish', residual_style=True)) +18:13:20|INFO|Seeded python, numpy and torch with 1. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +18:13:20|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed1 +18:13:20|INFO|Number of network parameters: 568392 +18:13:20|INFO|Policy created... +18:13:20|INFO|Initialized RolloutBuffer buffer_size=1024 action_shape=(1024, 20, 4, 6) value_shape=(1024,) +18:13:20|INFO|Algorithm created: + +18:13:20|INFO|Agent Server is listening on 0.0.0.0:9351 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +18:16:41|INFO|Checkpoint saved at step 20481 to /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed1/20481 +18:16:41|INFO|Stopping server as the algorithm signaled to stop. +18:16:41|INFO|Shutdown started: aborting pending infer requests. +18:16:41|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +18:16:41|INFO|Shutdown closing 1 websocket connection(s). +18:16:41|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 56672). +18:16:41|INFO|WebSocket server closed. +18:16:41|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e24-coverage/results/dppo-walker/server-seed2.log b/experiments/e24-coverage/results/dppo-walker/server-seed2.log new file mode 100644 index 0000000..f8b3b13 --- /dev/null +++ b/experiments/e24-coverage/results/dppo-walker/server-seed2.log @@ -0,0 +1,22 @@ +18:13:25|INFO|plugrl_server version: 0.1.0 +18:13:25|INFO|Algorithm: dppo, Config: DPPOAlgoConfigWalker(global_steps=20480, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=1024, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=512, train_itrs=20, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +18:13:25|INFO|Policy: dppo-policy, Config: DPPOPolicyConfigWalker(algo='dppo', device=device(type='cpu'), env_type='gym', env_name='walker2d-medium-v2', checkpoint_path=None, critic=DPPOCriticObsConfig(mlp_dims=[256, 256, 256], activation='Mish', residual_style=True)) +18:13:25|INFO|Seeded python, numpy and torch with 2. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +18:13:25|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed2 +18:13:25|INFO|Number of network parameters: 568392 +18:13:25|INFO|Policy created... +18:13:25|INFO|Initialized RolloutBuffer buffer_size=1024 action_shape=(1024, 20, 4, 6) value_shape=(1024,) +18:13:25|INFO|Algorithm created: + +18:13:25|INFO|Agent Server is listening on 0.0.0.0:9352 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +18:16:44|INFO|Checkpoint saved at step 20481 to /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed2/20481 +18:16:44|INFO|Stopping server as the algorithm signaled to stop. +18:16:44|INFO|Shutdown started: aborting pending infer requests. +18:16:44|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +18:16:44|INFO|Shutdown closing 1 websocket connection(s). +18:16:44|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 37540). +18:16:44|INFO|WebSocket server closed. +18:16:44|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e24-coverage/results/fpo-walker.out b/experiments/e24-coverage/results/fpo-walker.out new file mode 100644 index 0000000..53d148e --- /dev/null +++ b/experiments/e24-coverage/results/fpo-walker.out @@ -0,0 +1,10 @@ +cell: fpo-walker = fpo-policy/default x fpo/default x Walker2d-v5 +iters: 100 x 4096 batch: variant default replan: 1 seeds: 0 1 2 +out: /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/fpo-walker +start 2026-09-25 18:07:44 +seed 1 finished rc=0 at 18:28:04 +seed 2 finished rc=0 at 18:28:11 +seed 0 finished rc=0 at 18:28:31 +end 2026-09-25 18:28:31 +failed seeds: 0 +CELL_DONE fpo-walker diff --git a/experiments/e24-coverage/results/fpo-walker/client-seed0.log b/experiments/e24-coverage/results/fpo-walker/client-seed0.log new file mode 100644 index 0000000..37069df --- /dev/null +++ b/experiments/e24-coverage/results/fpo-walker/client-seed0.log @@ -0,0 +1,51 @@ +2026-09-25 18:07:46.297 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 18:07:46.297 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 18:07:46.298 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 18:07:46.299 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 18:07:46.300 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 18:07:46.356 | INFO | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-180746-80f0b894 output_dir=runs/mujoco-v1-nenv1-20260925-180746-80f0b894 recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 18:07:46.390 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9300... +2026-09-25 18:07:46.392 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'FPOAlgorithm', 'policy': 'FPOPolicy', 'action_dim': 6, 'action_horizon': 1} +2026-09-25 18:08:16.395 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=10386 infer_calls=10386 feedback_calls=10386 infer_wait=27.137s infer_obs_pack=0.149s env_step=1.465s feedback_total=0.720s feedback_obs_pack=0.190s feedback_info_pack=0.036s effective_fps=352.42 +2026-09-25 18:08:48.159 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=20482 infer_calls=20482 feedback_calls=20482 infer_wait=55.981s infer_obs_pack=0.319s env_step=3.008s feedback_total=1.440s feedback_obs_pack=0.386s feedback_info_pack=0.075s effective_fps=337.16 +2026-09-25 18:09:18.161 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=30594 infer_calls=30594 feedback_calls=30594 infer_wait=82.846s infer_obs_pack=0.477s env_step=4.695s feedback_total=2.229s feedback_obs_pack=0.605s feedback_info_pack=0.118s effective_fps=339.00 +2026-09-25 18:09:48.162 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=40702 infer_calls=40702 feedback_calls=40702 infer_wait=109.730s infer_obs_pack=0.639s env_step=6.377s feedback_total=2.998s feedback_obs_pack=0.817s feedback_info_pack=0.158s effective_fps=339.91 +2026-09-25 18:10:18.163 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=50038 infer_calls=50038 feedback_calls=50038 infer_wait=136.818s infer_obs_pack=0.789s env_step=7.964s feedback_total=3.712s feedback_obs_pack=1.017s feedback_info_pack=0.199s effective_fps=335.19 +2026-09-25 18:10:48.166 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=60195 infer_calls=60195 feedback_calls=60195 infer_wait=163.606s infer_obs_pack=0.956s env_step=9.708s feedback_total=4.501s feedback_obs_pack=1.233s feedback_info_pack=0.240s effective_fps=336.72 +2026-09-25 18:11:18.281 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=69634 infer_calls=69634 feedback_calls=69634 infer_wait=190.679s infer_obs_pack=1.111s env_step=11.367s feedback_total=5.247s feedback_obs_pack=1.436s feedback_info_pack=0.280s effective_fps=334.13 +2026-09-25 18:11:48.282 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=79698 infer_calls=79698 feedback_calls=79698 infer_wait=217.530s infer_obs_pack=1.271s env_step=13.092s feedback_total=6.014s feedback_obs_pack=1.643s feedback_info_pack=0.319s effective_fps=335.00 +2026-09-25 18:12:18.284 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=89731 infer_calls=89731 feedback_calls=89731 infer_wait=244.429s infer_obs_pack=1.432s env_step=14.781s feedback_total=6.771s feedback_obs_pack=1.851s feedback_info_pack=0.360s effective_fps=335.55 +2026-09-25 18:12:48.286 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=99099 infer_calls=99099 feedback_calls=99099 infer_wait=271.712s infer_obs_pack=1.568s env_step=16.269s feedback_total=7.416s feedback_obs_pack=2.026s feedback_info_pack=0.394s effective_fps=333.71 +2026-09-25 18:13:18.288 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=109605 infer_calls=109605 feedback_calls=109605 infer_wait=298.361s infer_obs_pack=1.736s env_step=18.102s feedback_total=8.238s feedback_obs_pack=2.243s feedback_info_pack=0.438s effective_fps=335.76 +2026-09-25 18:13:48.529 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=118786 infer_calls=118786 feedback_calls=118786 infer_wait=324.955s infer_obs_pack=1.909s env_step=20.080s feedback_total=9.187s feedback_obs_pack=2.484s feedback_info_pack=0.486s effective_fps=333.55 +2026-09-25 18:14:18.529 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=128323 infer_calls=128323 feedback_calls=128323 infer_wait=351.036s infer_obs_pack=2.093s env_step=22.186s feedback_total=10.220s feedback_obs_pack=2.743s feedback_info_pack=0.537s effective_fps=332.84 +2026-09-25 18:14:48.530 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=137713 infer_calls=137713 feedback_calls=137713 infer_wait=377.088s infer_obs_pack=2.274s env_step=24.289s feedback_total=11.288s feedback_obs_pack=3.013s feedback_info_pack=0.589s effective_fps=331.89 +2026-09-25 18:15:18.530 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=147369 infer_calls=147369 feedback_calls=147369 infer_wait=403.108s infer_obs_pack=2.466s env_step=26.412s feedback_total=12.348s feedback_obs_pack=3.277s feedback_info_pack=0.643s effective_fps=331.66 +2026-09-25 18:15:48.532 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=156152 infer_calls=156152 feedback_calls=156152 infer_wait=429.540s infer_obs_pack=2.638s env_step=28.328s feedback_total=13.288s feedback_obs_pack=3.513s feedback_info_pack=0.690s effective_fps=329.58 +2026-09-25 18:16:18.532 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=166441 infer_calls=166441 feedback_calls=166441 infer_wait=456.042s infer_obs_pack=2.810s env_step=30.246s feedback_total=14.149s feedback_obs_pack=3.742s feedback_info_pack=0.734s effective_fps=330.73 +2026-09-25 18:16:48.850 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=176130 infer_calls=176130 feedback_calls=176130 infer_wait=483.584s infer_obs_pack=2.948s env_step=31.748s feedback_total=14.820s feedback_obs_pack=3.925s feedback_info_pack=0.769s effective_fps=330.39 +2026-09-25 18:17:18.852 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=186543 infer_calls=186543 feedback_calls=186543 infer_wait=510.567s infer_obs_pack=3.102s env_step=33.368s feedback_total=15.564s feedback_obs_pack=4.118s feedback_info_pack=0.806s effective_fps=331.57 +2026-09-25 18:17:50.162 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=196610 infer_calls=196610 feedback_calls=196610 infer_wait=538.991s infer_obs_pack=3.249s env_step=34.922s feedback_total=16.266s feedback_obs_pack=4.302s feedback_info_pack=0.841s effective_fps=331.31 +2026-09-25 18:18:20.164 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=206839 infer_calls=206839 feedback_calls=206839 infer_wait=566.006s infer_obs_pack=3.403s env_step=36.525s feedback_total=17.004s feedback_obs_pack=4.495s feedback_info_pack=0.877s effective_fps=332.04 +2026-09-25 18:18:50.164 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=217008 infer_calls=217008 feedback_calls=217008 infer_wait=593.106s infer_obs_pack=3.550s env_step=38.084s feedback_total=17.705s feedback_obs_pack=4.682s feedback_info_pack=0.913s effective_fps=332.61 +2026-09-25 18:19:20.164 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=226454 infer_calls=226454 feedback_calls=226454 infer_wait=620.392s infer_obs_pack=3.690s env_step=39.536s feedback_total=18.371s feedback_obs_pack=4.858s feedback_info_pack=0.946s effective_fps=332.05 +2026-09-25 18:19:50.166 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=236819 infer_calls=236819 feedback_calls=236819 infer_wait=647.439s infer_obs_pack=3.840s env_step=41.119s feedback_total=19.086s feedback_obs_pack=5.050s feedback_info_pack=0.983s effective_fps=332.85 +2026-09-25 18:20:20.166 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=246472 infer_calls=246472 feedback_calls=246472 infer_wait=674.644s infer_obs_pack=3.985s env_step=42.603s feedback_total=19.783s feedback_obs_pack=5.231s feedback_info_pack=1.017s effective_fps=332.61 +2026-09-25 18:20:50.168 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=256854 infer_calls=256854 feedback_calls=256854 infer_wait=701.700s infer_obs_pack=4.136s env_step=44.174s feedback_total=20.517s feedback_obs_pack=5.420s feedback_info_pack=1.053s effective_fps=333.35 +2026-09-25 18:21:20.169 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=266390 infer_calls=266390 feedback_calls=266390 infer_wait=728.946s infer_obs_pack=4.277s env_step=45.655s feedback_total=21.185s feedback_obs_pack=5.600s feedback_info_pack=1.086s effective_fps=332.96 +2026-09-25 18:21:50.170 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=276682 infer_calls=276682 feedback_calls=276682 infer_wait=755.960s infer_obs_pack=4.430s env_step=47.255s feedback_total=21.911s feedback_obs_pack=5.793s feedback_info_pack=1.123s effective_fps=333.53 +2026-09-25 18:22:21.511 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=286722 infer_calls=286722 feedback_calls=286722 infer_wait=784.441s infer_obs_pack=4.578s env_step=48.788s feedback_total=22.609s feedback_obs_pack=5.979s feedback_info_pack=1.158s effective_fps=333.24 +2026-09-25 18:22:51.513 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=296887 infer_calls=296887 feedback_calls=296887 infer_wait=811.530s infer_obs_pack=4.728s env_step=50.353s feedback_total=23.318s feedback_obs_pack=6.167s feedback_info_pack=1.195s effective_fps=333.61 +2026-09-25 18:23:21.516 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=306985 infer_calls=306985 feedback_calls=306985 infer_wait=838.594s infer_obs_pack=4.879s env_step=51.937s feedback_total=24.028s feedback_obs_pack=6.358s feedback_info_pack=1.231s effective_fps=333.88 +2026-09-25 18:23:51.517 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=316467 infer_calls=316467 feedback_calls=316467 infer_wait=865.849s infer_obs_pack=5.019s env_step=53.409s feedback_total=24.695s feedback_obs_pack=6.535s feedback_info_pack=1.265s effective_fps=333.48 +2026-09-25 18:24:21.517 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=326721 infer_calls=326721 feedback_calls=326721 infer_wait=892.912s infer_obs_pack=5.173s env_step=54.979s feedback_total=25.414s feedback_obs_pack=6.726s feedback_info_pack=1.301s effective_fps=333.91 +2026-09-25 18:24:51.518 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=336181 infer_calls=336181 feedback_calls=336181 infer_wait=920.169s infer_obs_pack=5.313s env_step=56.442s feedback_total=26.091s feedback_obs_pack=6.905s feedback_info_pack=1.336s effective_fps=333.51 +2026-09-25 18:25:21.520 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=346496 infer_calls=346496 feedback_calls=346496 infer_wait=947.216s infer_obs_pack=5.464s env_step=58.019s feedback_total=26.821s feedback_obs_pack=7.093s feedback_info_pack=1.372s effective_fps=333.97 +2026-09-25 18:25:52.425 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=356354 infer_calls=356354 feedback_calls=356354 infer_wait=975.305s infer_obs_pack=5.606s env_step=59.530s feedback_total=27.507s feedback_obs_pack=7.274s feedback_info_pack=1.407s effective_fps=333.68 +2026-09-25 18:26:22.425 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=366754 infer_calls=366754 feedback_calls=366754 infer_wait=1002.319s infer_obs_pack=5.761s env_step=61.123s feedback_total=28.247s feedback_obs_pack=7.467s feedback_info_pack=1.444s effective_fps=334.19 +2026-09-25 18:26:53.837 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=376834 infer_calls=376834 feedback_calls=376834 infer_wait=1030.856s infer_obs_pack=5.910s env_step=62.662s feedback_total=28.957s feedback_obs_pack=7.652s feedback_info_pack=1.480s effective_fps=333.96 +2026-09-25 18:27:23.838 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=387211 infer_calls=387211 feedback_calls=387211 infer_wait=1057.875s infer_obs_pack=6.065s env_step=64.258s feedback_total=29.687s feedback_obs_pack=7.844s feedback_info_pack=1.516s effective_fps=334.41 +2026-09-25 18:27:54.922 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=397314 infer_calls=397314 feedback_calls=397314 infer_wait=1086.097s infer_obs_pack=6.213s env_step=65.781s feedback_total=30.390s feedback_obs_pack=8.027s feedback_info_pack=1.551s effective_fps=334.30 +2026-09-25 18:28:24.922 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=407968 infer_calls=407968 feedback_calls=407968 infer_wait=1113.191s infer_obs_pack=6.359s env_step=67.304s feedback_total=31.119s feedback_obs_pack=8.212s feedback_info_pack=1.586s effective_fps=334.96 +2026-09-25 18:28:31.061 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=409601 infer_calls=409601 feedback_calls=409601 infer_wait=1116.990s infer_obs_pack=6.379s env_step=67.528s feedback_total=31.217s feedback_obs_pack=8.239s feedback_info_pack=1.591s effective_fps=335.16 +2026-09-25 18:28:31.061 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 1950/1638400 episodes. diff --git a/experiments/e24-coverage/results/fpo-walker/client-seed1.log b/experiments/e24-coverage/results/fpo-walker/client-seed1.log new file mode 100644 index 0000000..f7e04ae --- /dev/null +++ b/experiments/e24-coverage/results/fpo-walker/client-seed1.log @@ -0,0 +1,50 @@ +2026-09-25 18:07:51.230 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 18:07:51.230 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 18:07:51.230 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 18:07:51.230 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 18:07:51.231 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 18:07:51.270 | INFO | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-180751-83345c87 output_dir=runs/mujoco-v1-nenv1-20260925-180751-83345c87 recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 18:07:51.308 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9301... +2026-09-25 18:07:51.310 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'FPOAlgorithm', 'policy': 'FPOPolicy', 'action_dim': 6, 'action_horizon': 1} +2026-09-25 18:08:21.313 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=10638 infer_calls=10638 feedback_calls=10638 infer_wait=27.101s infer_obs_pack=0.145s env_step=1.493s feedback_total=0.722s feedback_obs_pack=0.192s feedback_info_pack=0.040s effective_fps=361.07 +2026-09-25 18:08:51.653 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=20482 infer_calls=20482 feedback_calls=20482 infer_wait=54.586s infer_obs_pack=0.310s env_step=3.009s feedback_total=1.429s feedback_obs_pack=0.384s feedback_info_pack=0.079s effective_fps=345.20 +2026-09-25 18:09:21.654 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=30659 infer_calls=30659 feedback_calls=30659 infer_wait=81.441s infer_obs_pack=0.469s env_step=4.714s feedback_total=2.202s feedback_obs_pack=0.595s feedback_info_pack=0.121s effective_fps=345.16 +2026-09-25 18:09:51.654 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=40891 infer_calls=40891 feedback_calls=40891 infer_wait=108.227s infer_obs_pack=0.635s env_step=6.457s feedback_total=2.987s feedback_obs_pack=0.814s feedback_info_pack=0.163s effective_fps=345.64 +2026-09-25 18:10:21.656 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=50444 infer_calls=50444 feedback_calls=50444 infer_wait=135.287s infer_obs_pack=0.783s env_step=8.051s feedback_total=3.717s feedback_obs_pack=1.014s feedback_info_pack=0.202s effective_fps=341.21 +2026-09-25 18:10:51.657 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=60763 infer_calls=60763 feedback_calls=60763 infer_wait=162.046s infer_obs_pack=0.948s env_step=9.810s feedback_total=4.516s feedback_obs_pack=1.231s feedback_info_pack=0.243s effective_fps=342.68 +2026-09-25 18:11:21.658 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=70292 infer_calls=70292 feedback_calls=70292 infer_wait=189.119s infer_obs_pack=1.097s env_step=11.400s feedback_total=5.233s feedback_obs_pack=1.427s feedback_info_pack=0.280s effective_fps=339.82 +2026-09-25 18:11:51.658 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=80596 infer_calls=80596 feedback_calls=80596 infer_wait=215.899s infer_obs_pack=1.264s env_step=13.156s feedback_total=6.021s feedback_obs_pack=1.643s feedback_info_pack=0.327s effective_fps=341.02 +2026-09-25 18:12:21.661 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=90264 infer_calls=90264 feedback_calls=90264 infer_wait=242.970s infer_obs_pack=1.417s env_step=14.747s feedback_total=6.738s feedback_obs_pack=1.837s feedback_info_pack=0.366s effective_fps=339.50 +2026-09-25 18:12:51.662 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=100783 infer_calls=100783 feedback_calls=100783 infer_wait=270.012s infer_obs_pack=1.566s env_step=16.348s feedback_total=7.458s feedback_obs_pack=2.028s feedback_info_pack=0.403s effective_fps=341.19 +2026-09-25 18:13:21.663 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=110608 infer_calls=110608 feedback_calls=110608 infer_wait=296.807s infer_obs_pack=1.723s env_step=18.095s feedback_total=8.257s feedback_obs_pack=2.239s feedback_info_pack=0.444s effective_fps=340.46 +2026-09-25 18:13:51.666 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=120689 infer_calls=120689 feedback_calls=120689 infer_wait=322.793s infer_obs_pack=1.911s env_step=20.258s feedback_total=9.302s feedback_obs_pack=2.506s feedback_info_pack=0.497s effective_fps=340.68 +2026-09-25 18:14:21.668 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=130660 infer_calls=130660 feedback_calls=130660 infer_wait=348.715s infer_obs_pack=2.097s env_step=22.431s feedback_total=10.398s feedback_obs_pack=2.780s feedback_info_pack=0.556s effective_fps=340.58 +2026-09-25 18:14:51.671 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=139667 infer_calls=139667 feedback_calls=139667 infer_wait=374.927s infer_obs_pack=2.273s env_step=24.455s feedback_total=11.415s feedback_obs_pack=3.034s feedback_info_pack=0.606s effective_fps=338.12 +2026-09-25 18:15:21.673 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=149330 infer_calls=149330 feedback_calls=149330 infer_wait=400.968s infer_obs_pack=2.451s env_step=26.591s feedback_total=12.456s feedback_obs_pack=3.293s feedback_info_pack=0.662s effective_fps=337.49 +2026-09-25 18:15:51.673 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=159629 infer_calls=159629 feedback_calls=159629 infer_wait=427.041s infer_obs_pack=2.628s env_step=28.712s feedback_total=13.481s feedback_obs_pack=3.557s feedback_info_pack=0.715s effective_fps=338.30 +2026-09-25 18:16:21.674 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=169465 infer_calls=169465 feedback_calls=169465 infer_wait=453.822s infer_obs_pack=2.785s env_step=30.462s feedback_total=14.287s feedback_obs_pack=3.772s feedback_info_pack=0.759s effective_fps=338.01 +2026-09-25 18:16:51.676 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=180160 infer_calls=180160 feedback_calls=180160 infer_wait=480.760s infer_obs_pack=2.939s env_step=32.100s feedback_total=15.056s feedback_obs_pack=3.968s feedback_info_pack=0.798s effective_fps=339.38 +2026-09-25 18:17:21.677 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=190066 infer_calls=190066 feedback_calls=190066 infer_wait=507.899s infer_obs_pack=3.082s env_step=33.643s feedback_total=15.756s feedback_obs_pack=4.149s feedback_info_pack=0.833s effective_fps=339.17 +2026-09-25 18:17:51.678 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=200492 infer_calls=200492 feedback_calls=200492 infer_wait=534.872s infer_obs_pack=3.232s env_step=35.286s feedback_total=16.492s feedback_obs_pack=4.346s feedback_info_pack=0.871s effective_fps=339.88 +2026-09-25 18:18:21.678 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=210381 infer_calls=210381 feedback_calls=210381 infer_wait=562.012s infer_obs_pack=3.374s env_step=36.823s feedback_total=17.195s feedback_obs_pack=4.526s feedback_info_pack=0.907s effective_fps=339.65 +2026-09-25 18:18:51.679 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=220958 infer_calls=220958 feedback_calls=220958 infer_wait=588.975s infer_obs_pack=3.526s env_step=38.458s feedback_total=17.946s feedback_obs_pack=4.722s feedback_info_pack=0.944s effective_fps=340.51 +2026-09-25 18:19:21.679 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=230840 infer_calls=230840 feedback_calls=230840 infer_wait=616.107s infer_obs_pack=3.669s env_step=40.011s feedback_total=18.635s feedback_obs_pack=4.906s feedback_info_pack=0.980s effective_fps=340.26 +2026-09-25 18:19:51.681 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=241420 infer_calls=241420 feedback_calls=241420 infer_wait=643.064s infer_obs_pack=3.820s env_step=41.661s feedback_total=19.370s feedback_obs_pack=5.101s feedback_info_pack=1.017s effective_fps=341.03 +2026-09-25 18:20:21.683 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=251239 infer_calls=251239 feedback_calls=251239 infer_wait=670.200s infer_obs_pack=3.963s env_step=43.213s feedback_total=20.064s feedback_obs_pack=5.285s feedback_info_pack=1.052s effective_fps=340.69 +2026-09-25 18:20:51.684 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=261640 infer_calls=261640 feedback_calls=261640 infer_wait=697.182s infer_obs_pack=4.115s env_step=44.859s feedback_total=20.784s feedback_obs_pack=5.481s feedback_info_pack=1.090s effective_fps=341.15 +2026-09-25 18:21:21.684 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=271525 infer_calls=271525 feedback_calls=271525 infer_wait=724.310s infer_obs_pack=4.259s env_step=46.413s feedback_total=21.478s feedback_obs_pack=5.666s feedback_info_pack=1.125s effective_fps=340.91 +2026-09-25 18:21:51.686 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=282062 infer_calls=282062 feedback_calls=282062 infer_wait=751.253s infer_obs_pack=4.410s env_step=48.059s feedback_total=22.226s feedback_obs_pack=5.863s feedback_info_pack=1.162s effective_fps=341.50 +2026-09-25 18:22:21.686 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=291945 infer_calls=291945 feedback_calls=291945 infer_wait=778.411s infer_obs_pack=4.554s env_step=49.585s feedback_total=22.925s feedback_obs_pack=6.047s feedback_info_pack=1.202s effective_fps=341.27 +2026-09-25 18:22:51.687 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=302657 infer_calls=302657 feedback_calls=302657 infer_wait=805.329s infer_obs_pack=4.711s env_step=51.235s feedback_total=23.690s feedback_obs_pack=6.246s feedback_info_pack=1.240s effective_fps=342.00 +2026-09-25 18:23:21.688 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=312658 infer_calls=312658 feedback_calls=312658 infer_wait=832.468s infer_obs_pack=4.854s env_step=52.773s feedback_total=24.384s feedback_obs_pack=6.433s feedback_info_pack=1.276s effective_fps=341.90 +2026-09-25 18:23:51.689 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=323446 infer_calls=323446 feedback_calls=323446 infer_wait=859.432s infer_obs_pack=5.009s env_step=54.405s feedback_total=25.125s feedback_obs_pack=6.631s feedback_info_pack=1.314s effective_fps=342.64 +2026-09-25 18:24:21.689 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=333434 infer_calls=333434 feedback_calls=333434 infer_wait=886.595s infer_obs_pack=5.151s env_step=55.926s feedback_total=25.815s feedback_obs_pack=6.815s feedback_info_pack=1.349s effective_fps=342.51 +2026-09-25 18:24:53.153 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=344066 infer_calls=344066 feedback_calls=344066 infer_wait=915.017s infer_obs_pack=5.306s env_step=57.547s feedback_total=26.563s feedback_obs_pack=7.013s feedback_info_pack=1.386s effective_fps=342.55 +2026-09-25 18:25:23.154 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=354699 infer_calls=354699 feedback_calls=354699 infer_wait=941.962s infer_obs_pack=5.462s env_step=59.184s feedback_total=27.311s feedback_obs_pack=7.212s feedback_info_pack=1.424s effective_fps=343.06 +2026-09-25 18:25:53.156 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=364620 infer_calls=364620 feedback_calls=364620 infer_wait=969.131s infer_obs_pack=5.605s env_step=60.704s feedback_total=28.007s feedback_obs_pack=7.397s feedback_info_pack=1.460s effective_fps=342.87 +2026-09-25 18:26:23.158 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=375235 infer_calls=375235 feedback_calls=375235 infer_wait=996.061s infer_obs_pack=5.762s env_step=62.349s feedback_total=28.757s feedback_obs_pack=7.597s feedback_info_pack=1.498s effective_fps=343.33 +2026-09-25 18:26:53.159 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=385181 infer_calls=385181 feedback_calls=385181 infer_wait=1023.229s infer_obs_pack=5.905s env_step=63.860s feedback_total=29.464s feedback_obs_pack=7.780s feedback_info_pack=1.532s effective_fps=343.16 +2026-09-25 18:27:23.159 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=395854 infer_calls=395854 feedback_calls=395854 infer_wait=1050.180s infer_obs_pack=6.060s env_step=65.491s feedback_total=30.210s feedback_obs_pack=7.982s feedback_info_pack=1.571s effective_fps=343.64 +2026-09-25 18:27:53.161 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=405888 infer_calls=405888 feedback_calls=405888 infer_wait=1077.278s infer_obs_pack=6.207s env_step=67.044s feedback_total=30.922s feedback_obs_pack=8.172s feedback_info_pack=1.608s effective_fps=343.55 +2026-09-25 18:28:04.197 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=409601 infer_calls=409601 feedback_calls=409601 infer_wait=1085.628s infer_obs_pack=6.260s env_step=67.615s feedback_total=31.182s feedback_obs_pack=8.241s feedback_info_pack=1.621s effective_fps=344.00 +2026-09-25 18:28:04.198 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 2184/1638400 episodes. diff --git a/experiments/e24-coverage/results/fpo-walker/client-seed2.log b/experiments/e24-coverage/results/fpo-walker/client-seed2.log new file mode 100644 index 0000000..32dcf66 --- /dev/null +++ b/experiments/e24-coverage/results/fpo-walker/client-seed2.log @@ -0,0 +1,50 @@ +2026-09-25 18:07:56.231 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 18:07:56.232 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 18:07:56.232 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 18:07:56.232 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 18:07:56.233 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 18:07:56.271 | INFO | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-180756-366ec78e output_dir=runs/mujoco-v1-nenv1-20260925-180756-366ec78e recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 18:07:56.310 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9302... +2026-09-25 18:07:56.313 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'FPOAlgorithm', 'policy': 'FPOPolicy', 'action_dim': 6, 'action_horizon': 1} +2026-09-25 18:08:26.315 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=10585 infer_calls=10585 feedback_calls=10585 infer_wait=27.048s infer_obs_pack=0.152s env_step=1.532s feedback_total=0.737s feedback_obs_pack=0.198s feedback_info_pack=0.037s effective_fps=359.20 +2026-09-25 18:08:57.013 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=20482 infer_calls=20482 feedback_calls=20482 infer_wait=54.762s infer_obs_pack=0.325s env_step=3.109s feedback_total=1.477s feedback_obs_pack=0.403s feedback_info_pack=0.075s effective_fps=343.23 +2026-09-25 18:09:27.015 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=30491 infer_calls=30491 feedback_calls=30491 infer_wait=81.642s infer_obs_pack=0.485s env_step=4.792s feedback_total=2.253s feedback_obs_pack=0.624s feedback_info_pack=0.115s effective_fps=341.94 +2026-09-25 18:09:57.017 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=40751 infer_calls=40751 feedback_calls=40751 infer_wait=108.371s infer_obs_pack=0.647s env_step=6.566s feedback_total=3.063s feedback_obs_pack=0.850s feedback_info_pack=0.158s effective_fps=343.46 +2026-09-25 18:10:27.017 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=50185 infer_calls=50185 feedback_calls=50185 infer_wait=135.420s infer_obs_pack=0.794s env_step=8.175s feedback_total=3.793s feedback_obs_pack=1.052s feedback_info_pack=0.195s effective_fps=338.67 +2026-09-25 18:10:57.020 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=60471 infer_calls=60471 feedback_calls=60471 infer_wait=162.151s infer_obs_pack=0.962s env_step=9.963s feedback_total=4.593s feedback_obs_pack=1.288s feedback_info_pack=0.239s effective_fps=340.36 +2026-09-25 18:11:27.022 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=69954 infer_calls=69954 feedback_calls=69954 infer_wait=189.201s infer_obs_pack=1.118s env_step=11.561s feedback_total=5.320s feedback_obs_pack=1.499s feedback_info_pack=0.276s effective_fps=337.61 +2026-09-25 18:11:57.023 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=80201 infer_calls=80201 feedback_calls=80201 infer_wait=215.914s infer_obs_pack=1.286s env_step=13.363s feedback_total=6.119s feedback_obs_pack=1.725s feedback_info_pack=0.318s effective_fps=338.86 +2026-09-25 18:12:27.495 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=90114 infer_calls=90114 feedback_calls=90114 infer_wait=243.353s infer_obs_pack=1.446s env_step=15.008s feedback_total=6.853s feedback_obs_pack=1.934s feedback_info_pack=0.356s effective_fps=337.93 +2026-09-25 18:12:57.495 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=100870 infer_calls=100870 feedback_calls=100870 infer_wait=270.219s infer_obs_pack=1.608s env_step=16.707s feedback_total=7.607s feedback_obs_pack=2.139s feedback_info_pack=0.395s effective_fps=340.61 +2026-09-25 18:13:27.497 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=110725 infer_calls=110725 feedback_calls=110725 infer_wait=296.776s infer_obs_pack=1.772s env_step=18.583s feedback_total=8.474s feedback_obs_pack=2.373s feedback_info_pack=0.442s effective_fps=340.06 +2026-09-25 18:13:57.498 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=120715 infer_calls=120715 feedback_calls=120715 infer_wait=322.680s infer_obs_pack=1.963s env_step=20.792s feedback_total=9.550s feedback_obs_pack=2.652s feedback_info_pack=0.498s effective_fps=340.06 +2026-09-25 18:14:27.499 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=130665 infer_calls=130665 feedback_calls=130665 infer_wait=348.633s infer_obs_pack=2.151s env_step=22.956s feedback_total=10.625s feedback_obs_pack=2.928s feedback_info_pack=0.550s effective_fps=339.95 +2026-09-25 18:14:57.501 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=139825 infer_calls=139825 feedback_calls=139825 infer_wait=374.756s infer_obs_pack=2.332s env_step=25.015s feedback_total=11.687s feedback_obs_pack=3.191s feedback_info_pack=0.600s effective_fps=337.91 +2026-09-25 18:15:27.502 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=149689 infer_calls=149689 feedback_calls=149689 infer_wait=400.801s infer_obs_pack=2.519s env_step=27.148s feedback_total=12.720s feedback_obs_pack=3.468s feedback_info_pack=0.653s effective_fps=337.76 +2026-09-25 18:15:59.126 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=159746 infer_calls=159746 feedback_calls=159746 infer_wait=428.583s infer_obs_pack=2.702s env_step=29.241s feedback_total=13.696s feedback_obs_pack=3.729s feedback_info_pack=0.705s effective_fps=336.86 +2026-09-25 18:16:29.128 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=170368 infer_calls=170368 feedback_calls=170368 infer_wait=455.225s infer_obs_pack=2.866s env_step=31.057s feedback_total=14.545s feedback_obs_pack=3.955s feedback_info_pack=0.749s effective_fps=338.24 +2026-09-25 18:16:59.584 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=180226 infer_calls=180226 feedback_calls=180226 infer_wait=482.889s infer_obs_pack=3.006s env_step=32.570s feedback_total=15.216s feedback_obs_pack=4.137s feedback_info_pack=0.783s effective_fps=337.70 +2026-09-25 18:17:29.585 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=190630 infer_calls=190630 feedback_calls=190630 infer_wait=509.884s infer_obs_pack=3.160s env_step=34.187s feedback_total=15.953s feedback_obs_pack=4.336s feedback_info_pack=0.820s effective_fps=338.49 +2026-09-25 18:18:00.661 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=200706 infer_calls=200706 feedback_calls=200706 infer_wait=537.998s infer_obs_pack=3.311s env_step=35.794s feedback_total=16.660s feedback_obs_pack=4.529s feedback_info_pack=0.855s effective_fps=338.02 +2026-09-25 18:18:30.662 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=211155 infer_calls=211155 feedback_calls=211155 infer_wait=564.971s infer_obs_pack=3.463s env_step=37.423s feedback_total=17.407s feedback_obs_pack=4.728s feedback_info_pack=0.891s effective_fps=338.79 +2026-09-25 18:19:00.993 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=221186 infer_calls=221186 feedback_calls=221186 infer_wait=592.417s infer_obs_pack=3.611s env_step=38.969s feedback_total=18.112s feedback_obs_pack=4.919s feedback_info_pack=0.925s effective_fps=338.67 +2026-09-25 18:19:30.994 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=232029 infer_calls=232029 feedback_calls=232029 infer_wait=619.305s infer_obs_pack=3.769s env_step=40.631s feedback_total=18.877s feedback_obs_pack=5.122s feedback_info_pack=0.962s effective_fps=339.93 +2026-09-25 18:20:00.996 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=241866 infer_calls=241866 feedback_calls=241866 infer_wait=646.428s infer_obs_pack=3.915s env_step=42.190s feedback_total=19.572s feedback_obs_pack=5.312s feedback_info_pack=0.996s effective_fps=339.65 +2026-09-25 18:20:30.997 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=252493 infer_calls=252493 feedback_calls=252493 infer_wait=673.319s infer_obs_pack=4.070s env_step=43.870s feedback_total=20.325s feedback_obs_pack=5.514s feedback_info_pack=1.033s effective_fps=340.48 +2026-09-25 18:21:00.999 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=262329 infer_calls=262329 feedback_calls=262329 infer_wait=700.451s infer_obs_pack=4.217s env_step=45.415s feedback_total=21.015s feedback_obs_pack=5.702s feedback_info_pack=1.067s effective_fps=340.20 +2026-09-25 18:21:31.000 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=273104 infer_calls=273104 feedback_calls=273104 infer_wait=727.355s infer_obs_pack=4.373s env_step=47.080s feedback_total=21.776s feedback_obs_pack=5.902s feedback_info_pack=1.103s effective_fps=341.13 +2026-09-25 18:22:01.002 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=282922 infer_calls=282922 feedback_calls=282922 infer_wait=754.544s infer_obs_pack=4.515s env_step=48.591s feedback_total=22.466s feedback_obs_pack=6.085s feedback_info_pack=1.137s effective_fps=340.82 +2026-09-25 18:22:31.002 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=293482 infer_calls=293482 feedback_calls=293482 infer_wait=781.474s infer_obs_pack=4.673s env_step=50.230s feedback_total=23.228s feedback_obs_pack=6.286s feedback_info_pack=1.174s effective_fps=341.42 +2026-09-25 18:23:01.005 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=303400 infer_calls=303400 feedback_calls=303400 infer_wait=808.596s infer_obs_pack=4.821s env_step=51.769s feedback_total=23.943s feedback_obs_pack=6.475s feedback_info_pack=1.208s effective_fps=341.23 +2026-09-25 18:23:31.007 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=313909 infer_calls=313909 feedback_calls=313909 infer_wait=835.566s infer_obs_pack=4.975s env_step=53.395s feedback_total=24.692s feedback_obs_pack=6.674s feedback_info_pack=1.244s effective_fps=341.71 +2026-09-25 18:24:01.009 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=323740 infer_calls=323740 feedback_calls=323740 infer_wait=862.730s infer_obs_pack=5.119s env_step=54.921s feedback_total=25.388s feedback_obs_pack=6.861s feedback_info_pack=1.278s effective_fps=341.44 +2026-09-25 18:24:31.012 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=334205 infer_calls=334205 feedback_calls=334205 infer_wait=889.694s infer_obs_pack=5.275s env_step=56.549s feedback_total=26.135s feedback_obs_pack=7.062s feedback_info_pack=1.315s effective_fps=341.84 +2026-09-25 18:25:01.050 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=344066 infer_calls=344066 feedback_calls=344066 infer_wait=916.880s infer_obs_pack=5.423s env_step=58.066s feedback_total=26.839s feedback_obs_pack=7.250s feedback_info_pack=1.349s effective_fps=341.60 +2026-09-25 18:25:31.052 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=354427 infer_calls=354427 feedback_calls=354427 infer_wait=943.900s infer_obs_pack=5.573s env_step=59.672s feedback_total=27.566s feedback_obs_pack=7.445s feedback_info_pack=1.385s effective_fps=341.88 +2026-09-25 18:26:01.217 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=364546 infer_calls=364546 feedback_calls=364546 infer_wait=971.137s infer_obs_pack=5.722s env_step=61.227s feedback_total=28.289s feedback_obs_pack=7.638s feedback_info_pack=1.421s effective_fps=341.86 +2026-09-25 18:26:31.218 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=375385 infer_calls=375385 feedback_calls=375385 infer_wait=998.051s infer_obs_pack=5.881s env_step=62.873s feedback_total=29.043s feedback_obs_pack=7.841s feedback_info_pack=1.458s effective_fps=342.55 +2026-09-25 18:27:01.218 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=385549 infer_calls=385549 feedback_calls=385549 infer_wait=1025.125s infer_obs_pack=6.032s env_step=64.417s feedback_total=29.784s feedback_obs_pack=8.031s feedback_info_pack=1.494s effective_fps=342.60 +2026-09-25 18:27:31.219 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=396222 infer_calls=396222 feedback_calls=396222 infer_wait=1052.042s infer_obs_pack=6.189s env_step=66.069s feedback_total=30.548s feedback_obs_pack=8.235s feedback_info_pack=1.531s effective_fps=343.09 +2026-09-25 18:28:01.220 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=406060 infer_calls=406060 feedback_calls=406060 infer_wait=1079.228s infer_obs_pack=6.336s env_step=67.578s feedback_total=31.236s feedback_obs_pack=8.419s feedback_info_pack=1.565s effective_fps=342.85 +2026-09-25 18:28:11.473 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=409601 infer_calls=409601 feedback_calls=409601 infer_wait=1086.830s infer_obs_pack=6.386s env_step=68.085s feedback_total=31.481s feedback_obs_pack=8.482s feedback_info_pack=1.577s effective_fps=343.40 +2026-09-25 18:28:11.474 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 1953/1638400 episodes. diff --git a/experiments/e24-coverage/results/fpo-walker/server-seed0.log b/experiments/e24-coverage/results/fpo-walker/server-seed0.log new file mode 100644 index 0000000..1590a10 --- /dev/null +++ b/experiments/e24-coverage/results/fpo-walker/server-seed0.log @@ -0,0 +1,23 @@ +18:07:45|INFO|plugrl_server version: 0.1.0 +18:07:45|INFO|Algorithm: fpo, Config: FPOAlgoConfig(global_steps=409600, buffer_size=4096, output_mode='u_but_supervise_as_eps', fpo_playground_trick=True, treat_truncated_as_done=False, discounting=0.995, reward_scaling=10.0, gae_lambda=0.95, batch_size=1024, num_updates_per_batch=16, learning_rate=0.0003, value_loss_coeff=0.25, clipping_epsilon=0.05, normalize_advantage=True, n_critic_warmup_itrs=0, max_policy_drift=0.0, n_samples_per_action=8, discretize_t_for_training=True, average_losses_before_exp=True, save_interval=20, policy_checkpoint_path=None, restore='all', master_weights_device=None) +18:07:45|INFO|Policy: fpo-policy, Config: FPOPolicyConfig(algo='fpo', device=device(type='cpu'), obs_dim=17, action_dim=6, flow_steps=10, timestep_embed_dim=8, action_horizon=1, feather_std=0.0, policy_mlp_output_scale=0.25, normalize_observations=True, hidden_dims=(32, 32, 32, 32), value_hidden_dims=(256, 256, 256, 256, 256)) +18:07:45|INFO|Seeded python, numpy and torch with 0. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +18:07:45|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/fpo-walker/fpo/fpo-policy/fpo-walker-seed0 +18:07:46|INFO|Policy created... +18:07:46|INFO|Initialized RolloutBuffer buffer_size=4096 action_shape=(4096, 1, 6) value_shape=(4096, 1) +18:07:46|INFO|Algorithm created: + +18:07:46|INFO|Agent Server is listening on 0.0.0.0:9300 +18:11:56|INFO|Checkpoint saved at step 81921 to /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/fpo-walker/fpo/fpo-policy/fpo-walker-seed0/81921 +18:16:12|INFO|Checkpoint saved at step 163841 to /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/fpo-walker/fpo/fpo-policy/fpo-walker-seed0/163841 +18:20:18|INFO|Checkpoint saved at step 245761 to /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/fpo-walker/fpo/fpo-policy/fpo-walker-seed0/245761 +18:24:25|INFO|Checkpoint saved at step 327681 to /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/fpo-walker/fpo/fpo-policy/fpo-walker-seed0/327681 +18:28:31|INFO|Checkpoint saved at step 409601 to /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/fpo-walker/fpo/fpo-policy/fpo-walker-seed0/409601 +18:28:31|INFO|Stopping server as the algorithm signaled to stop. +18:28:31|INFO|Shutdown started: aborting pending infer requests. +18:28:31|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +18:28:31|INFO|Shutdown closing 1 websocket connection(s). +18:28:31|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 44090). +18:28:31|INFO|WebSocket server closed. +18:28:31|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e24-coverage/results/fpo-walker/server-seed1.log b/experiments/e24-coverage/results/fpo-walker/server-seed1.log new file mode 100644 index 0000000..7d9e9e9 --- /dev/null +++ b/experiments/e24-coverage/results/fpo-walker/server-seed1.log @@ -0,0 +1,23 @@ +18:07:50|INFO|plugrl_server version: 0.1.0 +18:07:50|INFO|Algorithm: fpo, Config: FPOAlgoConfig(global_steps=409600, buffer_size=4096, output_mode='u_but_supervise_as_eps', fpo_playground_trick=True, treat_truncated_as_done=False, discounting=0.995, reward_scaling=10.0, gae_lambda=0.95, batch_size=1024, num_updates_per_batch=16, learning_rate=0.0003, value_loss_coeff=0.25, clipping_epsilon=0.05, normalize_advantage=True, n_critic_warmup_itrs=0, max_policy_drift=0.0, n_samples_per_action=8, discretize_t_for_training=True, average_losses_before_exp=True, save_interval=20, policy_checkpoint_path=None, restore='all', master_weights_device=None) +18:07:50|INFO|Policy: fpo-policy, Config: FPOPolicyConfig(algo='fpo', device=device(type='cpu'), obs_dim=17, action_dim=6, flow_steps=10, timestep_embed_dim=8, action_horizon=1, feather_std=0.0, policy_mlp_output_scale=0.25, normalize_observations=True, hidden_dims=(32, 32, 32, 32), value_hidden_dims=(256, 256, 256, 256, 256)) +18:07:50|INFO|Seeded python, numpy and torch with 1. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +18:07:50|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/fpo-walker/fpo/fpo-policy/fpo-walker-seed1 +18:07:50|INFO|Policy created... +18:07:50|INFO|Initialized RolloutBuffer buffer_size=4096 action_shape=(4096, 1, 6) value_shape=(4096, 1) +18:07:50|INFO|Algorithm created: + +18:07:50|INFO|Agent Server is listening on 0.0.0.0:9301 +18:11:56|INFO|Checkpoint saved at step 81921 to /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/fpo-walker/fpo/fpo-policy/fpo-walker-seed1/81921 +18:16:05|INFO|Checkpoint saved at step 163841 to /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/fpo-walker/fpo/fpo-policy/fpo-walker-seed1/163841 +18:20:06|INFO|Checkpoint saved at step 245761 to /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/fpo-walker/fpo/fpo-policy/fpo-walker-seed1/245761 +18:24:05|INFO|Checkpoint saved at step 327681 to /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/fpo-walker/fpo/fpo-policy/fpo-walker-seed1/327681 +18:28:04|INFO|Checkpoint saved at step 409601 to /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/fpo-walker/fpo/fpo-policy/fpo-walker-seed1/409601 +18:28:04|INFO|Stopping server as the algorithm signaled to stop. +18:28:04|INFO|Shutdown started: aborting pending infer requests. +18:28:04|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +18:28:04|INFO|Shutdown closing 1 websocket connection(s). +18:28:04|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 45950). +18:28:04|INFO|WebSocket server closed. +18:28:04|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e24-coverage/results/fpo-walker/server-seed2.log b/experiments/e24-coverage/results/fpo-walker/server-seed2.log new file mode 100644 index 0000000..51c75b9 --- /dev/null +++ b/experiments/e24-coverage/results/fpo-walker/server-seed2.log @@ -0,0 +1,23 @@ +18:07:55|INFO|plugrl_server version: 0.1.0 +18:07:55|INFO|Algorithm: fpo, Config: FPOAlgoConfig(global_steps=409600, buffer_size=4096, output_mode='u_but_supervise_as_eps', fpo_playground_trick=True, treat_truncated_as_done=False, discounting=0.995, reward_scaling=10.0, gae_lambda=0.95, batch_size=1024, num_updates_per_batch=16, learning_rate=0.0003, value_loss_coeff=0.25, clipping_epsilon=0.05, normalize_advantage=True, n_critic_warmup_itrs=0, max_policy_drift=0.0, n_samples_per_action=8, discretize_t_for_training=True, average_losses_before_exp=True, save_interval=20, policy_checkpoint_path=None, restore='all', master_weights_device=None) +18:07:55|INFO|Policy: fpo-policy, Config: FPOPolicyConfig(algo='fpo', device=device(type='cpu'), obs_dim=17, action_dim=6, flow_steps=10, timestep_embed_dim=8, action_horizon=1, feather_std=0.0, policy_mlp_output_scale=0.25, normalize_observations=True, hidden_dims=(32, 32, 32, 32), value_hidden_dims=(256, 256, 256, 256, 256)) +18:07:55|INFO|Seeded python, numpy and torch with 2. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +18:07:55|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/fpo-walker/fpo/fpo-policy/fpo-walker-seed2 +18:07:55|INFO|Policy created... +18:07:55|INFO|Initialized RolloutBuffer buffer_size=4096 action_shape=(4096, 1, 6) value_shape=(4096, 1) +18:07:55|INFO|Algorithm created: + +18:07:55|INFO|Agent Server is listening on 0.0.0.0:9302 +18:12:03|INFO|Checkpoint saved at step 81921 to /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/fpo-walker/fpo/fpo-policy/fpo-walker-seed2/81921 +18:16:11|INFO|Checkpoint saved at step 163841 to /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/fpo-walker/fpo/fpo-policy/fpo-walker-seed2/163841 +18:20:12|INFO|Checkpoint saved at step 245761 to /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/fpo-walker/fpo/fpo-policy/fpo-walker-seed2/245761 +18:24:12|INFO|Checkpoint saved at step 327681 to /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/fpo-walker/fpo/fpo-policy/fpo-walker-seed2/327681 +18:28:11|INFO|Checkpoint saved at step 409601 to /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/fpo-walker/fpo/fpo-policy/fpo-walker-seed2/409601 +18:28:11|INFO|Stopping server as the algorithm signaled to stop. +18:28:11|INFO|Shutdown started: aborting pending infer requests. +18:28:11|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +18:28:11|INFO|Shutdown closing 1 websocket connection(s). +18:28:11|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 55832). +18:28:11|INFO|WebSocket server closed. +18:28:11|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e24-coverage/results/fpodppo-hopper.out b/experiments/e24-coverage/results/fpodppo-hopper.out new file mode 100644 index 0000000..c1be6e7 --- /dev/null +++ b/experiments/e24-coverage/results/fpodppo-hopper.out @@ -0,0 +1,10 @@ +cell: fpodppo-hopper = fpo-policy/default x dppo/hopper x Hopper-v5 +iters: 20 x 4096 batch: variant default replan: 1 seeds: 0 1 2 +out: /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/fpodppo-hopper +start 2026-09-25 18:08:04 +seed 0 finished rc=0 at 18:12:06 +seed 1 finished rc=0 at 18:12:11 +seed 2 finished rc=0 at 18:12:14 +end 2026-09-25 18:12:14 +failed seeds: 0 +CELL_DONE fpodppo-hopper diff --git a/experiments/e24-coverage/results/fpodppo-hopper/client-seed0.log b/experiments/e24-coverage/results/fpodppo-hopper/client-seed0.log new file mode 100644 index 0000000..773e240 --- /dev/null +++ b/experiments/e24-coverage/results/fpodppo-hopper/client-seed0.log @@ -0,0 +1,17 @@ +2026-09-25 18:08:06.256 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 18:08:06.256 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 18:08:06.257 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 18:08:06.257 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 18:08:06.257 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 18:08:06.297 | INFO | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-180806-bdb5a715 output_dir=runs/mujoco-v1-nenv1-20260925-180806-bdb5a715 recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 18:08:06.332 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9310... +2026-09-25 18:08:06.335 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'FPOPolicy', 'action_dim': 3, 'action_horizon': 1} +2026-09-25 18:08:36.337 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=10094 infer_calls=10094 feedback_calls=10094 infer_wait=27.221s infer_obs_pack=0.141s env_step=1.409s feedback_total=0.706s feedback_obs_pack=0.189s feedback_info_pack=0.038s effective_fps=342.42 +2026-09-25 18:09:06.338 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=20373 infer_calls=20373 feedback_calls=20373 infer_wait=54.131s infer_obs_pack=0.319s env_step=2.978s feedback_total=1.498s feedback_obs_pack=0.403s feedback_info_pack=0.082s effective_fps=345.74 +2026-09-25 18:09:36.339 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=30598 infer_calls=30598 feedback_calls=30598 infer_wait=81.019s infer_obs_pack=0.490s env_step=4.558s feedback_total=2.300s feedback_obs_pack=0.619s feedback_info_pack=0.129s effective_fps=346.26 +2026-09-25 18:10:06.340 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=40914 infer_calls=40914 feedback_calls=40914 infer_wait=107.935s infer_obs_pack=0.648s env_step=6.135s feedback_total=3.098s feedback_obs_pack=0.832s feedback_info_pack=0.174s effective_fps=347.27 +2026-09-25 18:10:36.342 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=51056 infer_calls=51056 feedback_calls=51056 infer_wait=134.805s infer_obs_pack=0.808s env_step=7.733s feedback_total=3.902s feedback_obs_pack=1.052s feedback_info_pack=0.222s effective_fps=346.73 +2026-09-25 18:11:06.574 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=61442 infer_calls=61442 feedback_calls=61442 infer_wait=161.882s infer_obs_pack=0.970s env_step=9.350s feedback_total=4.707s feedback_obs_pack=1.272s feedback_info_pack=0.265s effective_fps=347.31 +2026-09-25 18:11:36.575 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=71752 infer_calls=71752 feedback_calls=71752 infer_wait=188.738s infer_obs_pack=1.139s env_step=10.943s feedback_total=5.522s feedback_obs_pack=1.493s feedback_info_pack=0.309s effective_fps=347.73 +2026-09-25 18:12:06.446 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=81921 infer_calls=81921 feedback_calls=81921 infer_wait=214.960s infer_obs_pack=1.304s env_step=12.548s feedback_total=6.318s feedback_obs_pack=1.710s feedback_info_pack=0.353s effective_fps=348.41 +2026-09-25 18:12:06.446 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 3888/327680 episodes. diff --git a/experiments/e24-coverage/results/fpodppo-hopper/client-seed1.log b/experiments/e24-coverage/results/fpodppo-hopper/client-seed1.log new file mode 100644 index 0000000..c3eb5cb --- /dev/null +++ b/experiments/e24-coverage/results/fpodppo-hopper/client-seed1.log @@ -0,0 +1,17 @@ +2026-09-25 18:08:11.237 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 18:08:11.237 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 18:08:11.237 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 18:08:11.237 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 18:08:11.238 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 18:08:11.278 | INFO | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-180811-384fe955 output_dir=runs/mujoco-v1-nenv1-20260925-180811-384fe955 recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 18:08:11.322 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9311... +2026-09-25 18:08:11.324 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'FPOPolicy', 'action_dim': 3, 'action_horizon': 1} +2026-09-25 18:08:41.326 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=10082 infer_calls=10082 feedback_calls=10082 infer_wait=27.156s infer_obs_pack=0.146s env_step=1.449s feedback_total=0.723s feedback_obs_pack=0.195s feedback_info_pack=0.039s effective_fps=342.07 +2026-09-25 18:09:11.327 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=20294 infer_calls=20294 feedback_calls=20294 infer_wait=54.040s infer_obs_pack=0.325s env_step=3.029s feedback_total=1.524s feedback_obs_pack=0.411s feedback_info_pack=0.084s effective_fps=344.45 +2026-09-25 18:09:41.328 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=30455 infer_calls=30455 feedback_calls=30455 infer_wait=80.941s infer_obs_pack=0.477s env_step=4.624s feedback_total=2.321s feedback_obs_pack=0.622s feedback_info_pack=0.128s effective_fps=344.65 +2026-09-25 18:10:11.329 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=40738 infer_calls=40738 feedback_calls=40738 infer_wait=107.818s infer_obs_pack=0.635s env_step=6.224s feedback_total=3.126s feedback_obs_pack=0.834s feedback_info_pack=0.174s effective_fps=345.82 +2026-09-25 18:10:41.330 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=51069 infer_calls=51069 feedback_calls=51069 infer_wait=134.732s infer_obs_pack=0.792s env_step=7.802s feedback_total=3.921s feedback_obs_pack=1.047s feedback_info_pack=0.217s effective_fps=346.83 +2026-09-25 18:11:11.684 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=61442 infer_calls=61442 feedback_calls=61442 infer_wait=161.938s infer_obs_pack=0.953s env_step=9.403s feedback_total=4.733s feedback_obs_pack=1.260s feedback_info_pack=0.260s effective_fps=347.08 +2026-09-25 18:11:41.686 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=71812 infer_calls=71812 feedback_calls=71812 infer_wait=188.822s infer_obs_pack=1.114s env_step=10.989s feedback_total=5.540s feedback_obs_pack=1.476s feedback_info_pack=0.303s effective_fps=347.82 +2026-09-25 18:12:11.317 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=81921 infer_calls=81921 feedback_calls=81921 infer_wait=214.867s infer_obs_pack=1.273s env_step=12.564s feedback_total=6.342s feedback_obs_pack=1.691s feedback_info_pack=0.347s effective_fps=348.53 +2026-09-25 18:12:11.317 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 3811/327680 episodes. diff --git a/experiments/e24-coverage/results/fpodppo-hopper/client-seed2.log b/experiments/e24-coverage/results/fpodppo-hopper/client-seed2.log new file mode 100644 index 0000000..238998e --- /dev/null +++ b/experiments/e24-coverage/results/fpodppo-hopper/client-seed2.log @@ -0,0 +1,17 @@ +2026-09-25 18:08:16.238 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 18:08:16.238 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 18:08:16.238 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 18:08:16.239 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 18:08:16.239 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 18:08:16.280 | INFO | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-180816-8526835d output_dir=runs/mujoco-v1-nenv1-20260925-180816-8526835d recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 18:08:16.324 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9312... +2026-09-25 18:08:16.328 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'FPOPolicy', 'action_dim': 3, 'action_horizon': 1} +2026-09-25 18:08:46.333 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=10241 infer_calls=10241 feedback_calls=10241 infer_wait=27.094s infer_obs_pack=0.147s env_step=1.484s feedback_total=0.745s feedback_obs_pack=0.194s feedback_info_pack=0.041s effective_fps=347.51 +2026-09-25 18:09:16.334 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=20483 infer_calls=20483 feedback_calls=20483 infer_wait=53.998s infer_obs_pack=0.325s env_step=3.060s feedback_total=1.531s feedback_obs_pack=0.399s feedback_info_pack=0.087s effective_fps=347.68 +2026-09-25 18:09:46.334 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=30842 infer_calls=30842 feedback_calls=30842 infer_wait=80.863s infer_obs_pack=0.490s env_step=4.670s feedback_total=2.320s feedback_obs_pack=0.613s feedback_info_pack=0.134s effective_fps=349.12 +2026-09-25 18:10:16.336 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=41083 infer_calls=41083 feedback_calls=41083 infer_wait=107.775s infer_obs_pack=0.651s env_step=6.261s feedback_total=3.108s feedback_obs_pack=0.824s feedback_info_pack=0.179s effective_fps=348.77 +2026-09-25 18:10:46.336 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=51635 infer_calls=51635 feedback_calls=51635 infer_wait=134.543s infer_obs_pack=0.822s env_step=7.926s feedback_total=3.931s feedback_obs_pack=1.041s feedback_info_pack=0.225s effective_fps=350.73 +2026-09-25 18:11:16.337 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=61946 infer_calls=61946 feedback_calls=61946 infer_wait=161.387s infer_obs_pack=0.988s env_step=9.526s feedback_total=4.757s feedback_obs_pack=1.255s feedback_info_pack=0.272s effective_fps=350.65 +2026-09-25 18:11:46.339 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=72463 infer_calls=72463 feedback_calls=72463 infer_wait=188.210s infer_obs_pack=1.147s env_step=11.158s feedback_total=5.575s feedback_obs_pack=1.473s feedback_info_pack=0.323s effective_fps=351.61 +2026-09-25 18:12:13.948 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=81921 infer_calls=81921 feedback_calls=81921 infer_wait=212.490s infer_obs_pack=1.296s env_step=12.613s feedback_total=6.296s feedback_obs_pack=1.664s feedback_info_pack=0.363s effective_fps=352.05 +2026-09-25 18:12:13.948 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 3693/327680 episodes. diff --git a/experiments/e24-coverage/results/fpodppo-hopper/server-seed0.log b/experiments/e24-coverage/results/fpodppo-hopper/server-seed0.log new file mode 100644 index 0000000..c6b94d2 --- /dev/null +++ b/experiments/e24-coverage/results/fpodppo-hopper/server-seed0.log @@ -0,0 +1,21 @@ +18:08:05|INFO|plugrl_server version: 0.1.0 +18:08:05|INFO|Algorithm: dppo, Config: DPPOAlgoConfigHopper(global_steps=81920, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=4096, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=2048, train_itrs=20, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +18:08:05|INFO|Policy: fpo-policy, Config: FPOPolicyConfig(algo='dppo', device=device(type='cpu'), obs_dim=11, action_dim=3, flow_steps=10, timestep_embed_dim=8, action_horizon=1, feather_std=0.0, policy_mlp_output_scale=0.25, normalize_observations=True, hidden_dims=(32, 32, 32, 32), value_hidden_dims=(256, 256, 256, 256, 256)) +18:08:05|INFO|Seeded python, numpy and torch with 0. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +18:08:05|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/fpodppo-hopper/dppo/fpo-policy/fpodppo-hopper-seed0 +18:08:05|INFO|Policy created... +18:08:05|INFO|Initialized RolloutBuffer buffer_size=4096 action_shape=(4096, 10, 1, 3) value_shape=(4096,) +18:08:05|INFO|Algorithm created: + +18:08:05|INFO|Agent Server is listening on 0.0.0.0:9310 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +18:12:06|INFO|Checkpoint saved at step 81921 to /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/fpodppo-hopper/dppo/fpo-policy/fpodppo-hopper-seed0/81921 +18:12:06|INFO|Stopping server as the algorithm signaled to stop. +18:12:06|INFO|Shutdown started: aborting pending infer requests. +18:12:06|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +18:12:06|INFO|Shutdown closing 1 websocket connection(s). +18:12:06|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 36238). +18:12:06|INFO|WebSocket server closed. +18:12:06|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e24-coverage/results/fpodppo-hopper/server-seed1.log b/experiments/e24-coverage/results/fpodppo-hopper/server-seed1.log new file mode 100644 index 0000000..d63326c --- /dev/null +++ b/experiments/e24-coverage/results/fpodppo-hopper/server-seed1.log @@ -0,0 +1,21 @@ +18:08:10|INFO|plugrl_server version: 0.1.0 +18:08:10|INFO|Algorithm: dppo, Config: DPPOAlgoConfigHopper(global_steps=81920, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=4096, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=2048, train_itrs=20, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +18:08:10|INFO|Policy: fpo-policy, Config: FPOPolicyConfig(algo='dppo', device=device(type='cpu'), obs_dim=11, action_dim=3, flow_steps=10, timestep_embed_dim=8, action_horizon=1, feather_std=0.0, policy_mlp_output_scale=0.25, normalize_observations=True, hidden_dims=(32, 32, 32, 32), value_hidden_dims=(256, 256, 256, 256, 256)) +18:08:10|INFO|Seeded python, numpy and torch with 1. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +18:08:10|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/fpodppo-hopper/dppo/fpo-policy/fpodppo-hopper-seed1 +18:08:10|INFO|Policy created... +18:08:10|INFO|Initialized RolloutBuffer buffer_size=4096 action_shape=(4096, 10, 1, 3) value_shape=(4096,) +18:08:10|INFO|Algorithm created: + +18:08:10|INFO|Agent Server is listening on 0.0.0.0:9311 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +18:12:11|INFO|Checkpoint saved at step 81921 to /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/fpodppo-hopper/dppo/fpo-policy/fpodppo-hopper-seed1/81921 +18:12:11|INFO|Stopping server as the algorithm signaled to stop. +18:12:11|INFO|Shutdown started: aborting pending infer requests. +18:12:11|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +18:12:11|INFO|Shutdown closing 1 websocket connection(s). +18:12:11|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 52128). +18:12:11|INFO|WebSocket server closed. +18:12:11|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e24-coverage/results/fpodppo-hopper/server-seed2.log b/experiments/e24-coverage/results/fpodppo-hopper/server-seed2.log new file mode 100644 index 0000000..6af2727 --- /dev/null +++ b/experiments/e24-coverage/results/fpodppo-hopper/server-seed2.log @@ -0,0 +1,21 @@ +18:08:15|INFO|plugrl_server version: 0.1.0 +18:08:15|INFO|Algorithm: dppo, Config: DPPOAlgoConfigHopper(global_steps=81920, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=4096, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=2048, train_itrs=20, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +18:08:15|INFO|Policy: fpo-policy, Config: FPOPolicyConfig(algo='dppo', device=device(type='cpu'), obs_dim=11, action_dim=3, flow_steps=10, timestep_embed_dim=8, action_horizon=1, feather_std=0.0, policy_mlp_output_scale=0.25, normalize_observations=True, hidden_dims=(32, 32, 32, 32), value_hidden_dims=(256, 256, 256, 256, 256)) +18:08:15|INFO|Seeded python, numpy and torch with 2. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +18:08:15|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/fpodppo-hopper/dppo/fpo-policy/fpodppo-hopper-seed2 +18:08:16|INFO|Policy created... +18:08:16|INFO|Initialized RolloutBuffer buffer_size=4096 action_shape=(4096, 10, 1, 3) value_shape=(4096,) +18:08:16|INFO|Algorithm created: + +18:08:16|INFO|Agent Server is listening on 0.0.0.0:9312 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +18:12:13|INFO|Checkpoint saved at step 81921 to /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/fpodppo-hopper/dppo/fpo-policy/fpodppo-hopper-seed2/81921 +18:12:13|INFO|Stopping server as the algorithm signaled to stop. +18:12:13|INFO|Shutdown started: aborting pending infer requests. +18:12:13|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +18:12:13|INFO|Shutdown closing 1 websocket connection(s). +18:12:13|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 57474). +18:12:13|INFO|WebSocket server closed. +18:12:13|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e24-coverage/results/fpodppo-walker.out b/experiments/e24-coverage/results/fpodppo-walker.out new file mode 100644 index 0000000..686524c --- /dev/null +++ b/experiments/e24-coverage/results/fpodppo-walker.out @@ -0,0 +1,10 @@ +cell: fpodppo-walker = fpo-policy/default x dppo/walker x Walker2d-v5 +iters: 20 x 4096 batch: variant default replan: 1 seeds: 0 1 2 +out: /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/fpodppo-walker +start 2026-09-25 18:08:24 +seed 0 finished rc=0 at 18:12:22 +seed 1 finished rc=0 at 18:12:28 +seed 2 finished rc=0 at 18:12:33 +end 2026-09-25 18:12:33 +failed seeds: 0 +CELL_DONE fpodppo-walker diff --git a/experiments/e24-coverage/results/fpodppo-walker/client-seed0.log b/experiments/e24-coverage/results/fpodppo-walker/client-seed0.log new file mode 100644 index 0000000..fbc2b66 --- /dev/null +++ b/experiments/e24-coverage/results/fpodppo-walker/client-seed0.log @@ -0,0 +1,17 @@ +2026-09-25 18:08:26.246 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 18:08:26.247 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 18:08:26.247 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 18:08:26.247 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 18:08:26.248 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 18:08:26.294 | INFO | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-180826-1b515fde output_dir=runs/mujoco-v1-nenv1-20260925-180826-1b515fde recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 18:08:26.332 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9320... +2026-09-25 18:08:26.339 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'FPOPolicy', 'action_dim': 6, 'action_horizon': 1} +2026-09-25 18:08:56.342 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=10300 infer_calls=10300 feedback_calls=10300 infer_wait=26.939s infer_obs_pack=0.162s env_step=1.596s feedback_total=0.760s feedback_obs_pack=0.206s feedback_info_pack=0.041s effective_fps=349.67 +2026-09-25 18:09:26.344 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=20666 infer_calls=20666 feedback_calls=20666 infer_wait=53.701s infer_obs_pack=0.343s env_step=3.265s feedback_total=1.572s feedback_obs_pack=0.426s feedback_info_pack=0.086s effective_fps=350.98 +2026-09-25 18:09:56.346 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=31183 infer_calls=31183 feedback_calls=31183 infer_wait=80.432s infer_obs_pack=0.516s env_step=4.965s feedback_total=2.394s feedback_obs_pack=0.649s feedback_info_pack=0.130s effective_fps=353.12 +2026-09-25 18:10:26.349 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=41462 infer_calls=41462 feedback_calls=41462 infer_wait=107.263s infer_obs_pack=0.682s env_step=6.615s feedback_total=3.186s feedback_obs_pack=0.870s feedback_info_pack=0.175s effective_fps=352.13 +2026-09-25 18:10:56.353 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=52000 infer_calls=52000 feedback_calls=52000 infer_wait=133.946s infer_obs_pack=0.849s env_step=8.337s feedback_total=4.014s feedback_obs_pack=1.095s feedback_info_pack=0.220s effective_fps=353.39 +2026-09-25 18:11:26.353 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=62414 infer_calls=62414 feedback_calls=62414 infer_wait=160.753s infer_obs_pack=1.016s env_step=9.981s feedback_total=4.838s feedback_obs_pack=1.311s feedback_info_pack=0.263s effective_fps=353.44 +2026-09-25 18:11:56.354 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=73036 infer_calls=73036 feedback_calls=73036 infer_wait=187.456s infer_obs_pack=1.190s env_step=11.686s feedback_total=5.672s feedback_obs_pack=1.538s feedback_info_pack=0.307s effective_fps=354.54 +2026-09-25 18:12:22.628 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=81921 infer_calls=81921 feedback_calls=81921 infer_wait=210.528s infer_obs_pack=1.331s env_step=13.070s feedback_total=6.340s feedback_obs_pack=1.719s feedback_info_pack=0.345s effective_fps=354.22 +2026-09-25 18:12:22.628 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 3951/327680 episodes. diff --git a/experiments/e24-coverage/results/fpodppo-walker/client-seed1.log b/experiments/e24-coverage/results/fpodppo-walker/client-seed1.log new file mode 100644 index 0000000..635e6ad --- /dev/null +++ b/experiments/e24-coverage/results/fpodppo-walker/client-seed1.log @@ -0,0 +1,17 @@ +2026-09-25 18:08:31.246 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 18:08:31.246 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 18:08:31.246 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 18:08:31.246 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 18:08:31.247 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 18:08:31.289 | INFO | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-180831-a43312fa output_dir=runs/mujoco-v1-nenv1-20260925-180831-a43312fa recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 18:08:31.333 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9321... +2026-09-25 18:08:31.337 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'FPOPolicy', 'action_dim': 6, 'action_horizon': 1} +2026-09-25 18:09:01.340 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=10321 infer_calls=10321 feedback_calls=10321 infer_wait=26.909s infer_obs_pack=0.154s env_step=1.595s feedback_total=0.782s feedback_obs_pack=0.218s feedback_info_pack=0.042s effective_fps=350.57 +2026-09-25 18:09:31.343 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=20641 infer_calls=20641 feedback_calls=20641 infer_wait=53.747s infer_obs_pack=0.325s env_step=3.235s feedback_total=1.570s feedback_obs_pack=0.433s feedback_info_pack=0.084s effective_fps=350.58 +2026-09-25 18:10:01.346 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=31085 infer_calls=31085 feedback_calls=31085 infer_wait=80.542s infer_obs_pack=0.485s env_step=4.896s feedback_total=2.373s feedback_obs_pack=0.651s feedback_info_pack=0.126s effective_fps=352.06 +2026-09-25 18:10:31.349 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=41293 infer_calls=41293 feedback_calls=41293 infer_wait=107.403s infer_obs_pack=0.648s env_step=6.509s feedback_total=3.171s feedback_obs_pack=0.870s feedback_info_pack=0.169s effective_fps=350.74 +2026-09-25 18:11:01.350 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=51902 infer_calls=51902 feedback_calls=51902 infer_wait=134.117s infer_obs_pack=0.821s env_step=8.208s feedback_total=3.989s feedback_obs_pack=1.095s feedback_info_pack=0.213s effective_fps=352.75 +2026-09-25 18:11:31.351 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=62105 infer_calls=62105 feedback_calls=62105 infer_wait=160.934s infer_obs_pack=0.981s env_step=9.865s feedback_total=4.775s feedback_obs_pack=1.314s feedback_info_pack=0.257s effective_fps=351.76 +2026-09-25 18:12:01.351 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=72655 infer_calls=72655 feedback_calls=72655 infer_wait=187.664s infer_obs_pack=1.146s env_step=11.555s feedback_total=5.598s feedback_obs_pack=1.540s feedback_info_pack=0.304s effective_fps=352.76 +2026-09-25 18:12:28.728 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=81921 infer_calls=81921 feedback_calls=81921 infer_wait=211.810s infer_obs_pack=1.284s env_step=12.948s feedback_total=6.273s feedback_obs_pack=1.725s feedback_info_pack=0.341s effective_fps=352.63 +2026-09-25 18:12:28.728 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 4275/327680 episodes. diff --git a/experiments/e24-coverage/results/fpodppo-walker/client-seed2.log b/experiments/e24-coverage/results/fpodppo-walker/client-seed2.log new file mode 100644 index 0000000..1c55c01 --- /dev/null +++ b/experiments/e24-coverage/results/fpodppo-walker/client-seed2.log @@ -0,0 +1,17 @@ +2026-09-25 18:08:36.244 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 18:08:36.244 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 18:08:36.244 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 18:08:36.244 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 18:08:36.245 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 18:08:36.288 | INFO | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-180836-1dbcb674 output_dir=runs/mujoco-v1-nenv1-20260925-180836-1dbcb674 recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 18:08:36.330 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9322... +2026-09-25 18:08:36.337 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'FPOPolicy', 'action_dim': 6, 'action_horizon': 1} +2026-09-25 18:09:06.340 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=10384 infer_calls=10384 feedback_calls=10384 infer_wait=26.805s infer_obs_pack=0.165s env_step=1.649s feedback_total=0.827s feedback_obs_pack=0.232s feedback_info_pack=0.043s effective_fps=352.65 +2026-09-25 18:09:36.342 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=20674 infer_calls=20674 feedback_calls=20674 infer_wait=53.582s infer_obs_pack=0.341s env_step=3.306s feedback_total=1.643s feedback_obs_pack=0.459s feedback_info_pack=0.088s effective_fps=351.17 +2026-09-25 18:10:06.341 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=31145 infer_calls=31145 feedback_calls=31145 infer_wait=80.310s infer_obs_pack=0.513s env_step=4.986s feedback_total=2.485s feedback_obs_pack=0.695s feedback_info_pack=0.132s effective_fps=352.74 +2026-09-25 18:10:36.343 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=41446 infer_calls=41446 feedback_calls=41446 infer_wait=107.050s infer_obs_pack=0.690s env_step=6.668s feedback_total=3.317s feedback_obs_pack=0.927s feedback_info_pack=0.176s effective_fps=352.06 +2026-09-25 18:11:06.344 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=52066 infer_calls=52066 feedback_calls=52066 infer_wait=133.749s infer_obs_pack=0.861s env_step=8.368s feedback_total=4.169s feedback_obs_pack=1.161s feedback_info_pack=0.219s effective_fps=353.84 +2026-09-25 18:11:36.346 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=62371 infer_calls=62371 feedback_calls=62371 infer_wait=160.495s infer_obs_pack=1.033s env_step=10.040s feedback_total=5.005s feedback_obs_pack=1.394s feedback_info_pack=0.262s effective_fps=353.23 +2026-09-25 18:12:06.346 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=72940 infer_calls=72940 feedback_calls=72940 infer_wait=187.174s infer_obs_pack=1.208s env_step=11.760s feedback_total=5.854s feedback_obs_pack=1.629s feedback_info_pack=0.308s effective_fps=354.08 +2026-09-25 18:12:33.123 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=81921 infer_calls=81921 feedback_calls=81921 infer_wait=210.840s infer_obs_pack=1.340s env_step=13.101s feedback_total=6.501s feedback_obs_pack=1.809s feedback_info_pack=0.343s effective_fps=353.44 +2026-09-25 18:12:33.124 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 3911/327680 episodes. diff --git a/experiments/e24-coverage/results/fpodppo-walker/server-seed0.log b/experiments/e24-coverage/results/fpodppo-walker/server-seed0.log new file mode 100644 index 0000000..e0f13e2 --- /dev/null +++ b/experiments/e24-coverage/results/fpodppo-walker/server-seed0.log @@ -0,0 +1,21 @@ +18:08:26|INFO|plugrl_server version: 0.1.0 +18:08:26|INFO|Algorithm: dppo, Config: DPPOAlgoConfigWalker(global_steps=81920, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=4096, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=2048, train_itrs=20, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +18:08:26|INFO|Policy: fpo-policy, Config: FPOPolicyConfig(algo='dppo', device=device(type='cpu'), obs_dim=17, action_dim=6, flow_steps=10, timestep_embed_dim=8, action_horizon=1, feather_std=0.0, policy_mlp_output_scale=0.25, normalize_observations=True, hidden_dims=(32, 32, 32, 32), value_hidden_dims=(256, 256, 256, 256, 256)) +18:08:26|INFO|Seeded python, numpy and torch with 0. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +18:08:26|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/fpodppo-walker/dppo/fpo-policy/fpodppo-walker-seed0 +18:08:26|INFO|Policy created... +18:08:26|INFO|Initialized RolloutBuffer buffer_size=4096 action_shape=(4096, 10, 1, 6) value_shape=(4096,) +18:08:26|INFO|Algorithm created: + +18:08:26|INFO|Agent Server is listening on 0.0.0.0:9320 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +18:12:22|INFO|Checkpoint saved at step 81921 to /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/fpodppo-walker/dppo/fpo-policy/fpodppo-walker-seed0/81921 +18:12:22|INFO|Stopping server as the algorithm signaled to stop. +18:12:22|INFO|Shutdown started: aborting pending infer requests. +18:12:22|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +18:12:22|INFO|Shutdown closing 1 websocket connection(s). +18:12:22|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 59236). +18:12:22|INFO|WebSocket server closed. +18:12:22|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e24-coverage/results/fpodppo-walker/server-seed1.log b/experiments/e24-coverage/results/fpodppo-walker/server-seed1.log new file mode 100644 index 0000000..9e0dff1 --- /dev/null +++ b/experiments/e24-coverage/results/fpodppo-walker/server-seed1.log @@ -0,0 +1,21 @@ +18:08:31|INFO|plugrl_server version: 0.1.0 +18:08:31|INFO|Algorithm: dppo, Config: DPPOAlgoConfigWalker(global_steps=81920, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=4096, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=2048, train_itrs=20, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +18:08:31|INFO|Policy: fpo-policy, Config: FPOPolicyConfig(algo='dppo', device=device(type='cpu'), obs_dim=17, action_dim=6, flow_steps=10, timestep_embed_dim=8, action_horizon=1, feather_std=0.0, policy_mlp_output_scale=0.25, normalize_observations=True, hidden_dims=(32, 32, 32, 32), value_hidden_dims=(256, 256, 256, 256, 256)) +18:08:31|INFO|Seeded python, numpy and torch with 1. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +18:08:31|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/fpodppo-walker/dppo/fpo-policy/fpodppo-walker-seed1 +18:08:31|INFO|Policy created... +18:08:31|INFO|Initialized RolloutBuffer buffer_size=4096 action_shape=(4096, 10, 1, 6) value_shape=(4096,) +18:08:31|INFO|Algorithm created: + +18:08:31|INFO|Agent Server is listening on 0.0.0.0:9321 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +18:12:28|INFO|Checkpoint saved at step 81921 to /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/fpodppo-walker/dppo/fpo-policy/fpodppo-walker-seed1/81921 +18:12:28|INFO|Stopping server as the algorithm signaled to stop. +18:12:28|INFO|Shutdown started: aborting pending infer requests. +18:12:28|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +18:12:28|INFO|Shutdown closing 1 websocket connection(s). +18:12:28|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 60228). +18:12:28|INFO|WebSocket server closed. +18:12:28|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e24-coverage/results/fpodppo-walker/server-seed2.log b/experiments/e24-coverage/results/fpodppo-walker/server-seed2.log new file mode 100644 index 0000000..ac57a16 --- /dev/null +++ b/experiments/e24-coverage/results/fpodppo-walker/server-seed2.log @@ -0,0 +1,21 @@ +18:08:36|INFO|plugrl_server version: 0.1.0 +18:08:36|INFO|Algorithm: dppo, Config: DPPOAlgoConfigWalker(global_steps=81920, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=4096, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=2048, train_itrs=20, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +18:08:36|INFO|Policy: fpo-policy, Config: FPOPolicyConfig(algo='dppo', device=device(type='cpu'), obs_dim=17, action_dim=6, flow_steps=10, timestep_embed_dim=8, action_horizon=1, feather_std=0.0, policy_mlp_output_scale=0.25, normalize_observations=True, hidden_dims=(32, 32, 32, 32), value_hidden_dims=(256, 256, 256, 256, 256)) +18:08:36|INFO|Seeded python, numpy and torch with 2. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +18:08:36|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/fpodppo-walker/dppo/fpo-policy/fpodppo-walker-seed2 +18:08:36|INFO|Policy created... +18:08:36|INFO|Initialized RolloutBuffer buffer_size=4096 action_shape=(4096, 10, 1, 6) value_shape=(4096,) +18:08:36|INFO|Algorithm created: + +18:08:36|INFO|Agent Server is listening on 0.0.0.0:9322 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +18:12:33|INFO|Checkpoint saved at step 81921 to /home/guangzhao/zuogou/plugrl/plugrl-server/experiments/e24-coverage/results/fpodppo-walker/dppo/fpo-policy/fpodppo-walker-seed2/81921 +18:12:33|INFO|Stopping server as the algorithm signaled to stop. +18:12:33|INFO|Shutdown started: aborting pending infer requests. +18:12:33|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +18:12:33|INFO|Shutdown closing 1 websocket connection(s). +18:12:33|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 34946). +18:12:33|INFO|WebSocket server closed. +18:12:33|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e24-coverage/results/run.out b/experiments/e24-coverage/results/run.out new file mode 100644 index 0000000..4a9c204 --- /dev/null +++ b/experiments/e24-coverage/results/run.out @@ -0,0 +1,12 @@ +start 2026-09-25 18:07:44 +--- wave 1 +short cells of wave 1 done at 18:12:33 +--- wave 2 +end 2026-09-25 18:28:31 +fpo-walker failed seeds: 0 +fpodppo-hopper failed seeds: 0 +fpodppo-walker failed seeds: 0 +dppo-cheetah failed seeds: 0 +dppo-hopper failed seeds: 0 +dppo-walker failed seeds: 0 +E24_DONE diff --git a/experiments/e24-coverage/results/verdicts.txt b/experiments/e24-coverage/results/verdicts.txt new file mode 100644 index 0000000..4912881 --- /dev/null +++ b/experiments/e24-coverage/results/verdicts.txt @@ -0,0 +1,34 @@ +P1 every cell runs end to end + fpo-walker fpo-policy fpo Walker2d-v5 HOLDS (21 min) + fpodppo-hopper fpo-policy dppo Hopper-v5 HOLDS (4 min) + fpodppo-walker fpo-policy dppo Walker2d-v5 HOLDS (4 min) + dppo-cheetah dppo-policy dppo HalfCheetah-v5 HOLDS (3 min) + dppo-hopper dppo-policy dppo Hopper-v5 HOLDS (4 min) + dppo-walker dppo-policy dppo Walker2d-v5 HOLDS (4 min) + +P2 FPO learns Walker2d: mean of iterations 91-100 >= 500 on 2 of 3 + seed 0: 964.0 + seed 1: 1026.5 + seed 2: 1101.0 + HOLDS (3 of 3) + +reported: first iteration and mean of the last ten, return and episode length + fpo-walker seed 0 return 0.7 -> 964.0 length 19.2 -> 413.6 + fpo-walker seed 1 return -0.3 -> 1026.5 length 18.5 -> 409.3 + fpo-walker seed 2 return 0.6 -> 1101.0 length 20.0 -> 424.0 + fpodppo-hopper seed 0 return 14.9 -> 15.7 length 20.2 -> 21.1 + fpodppo-hopper seed 1 return 19.7 -> 17.6 length 22.8 -> 21.3 + fpodppo-hopper seed 2 return 17.0 -> 18.1 length 21.9 -> 22.4 + fpodppo-walker seed 0 return 1.6 -> 1.4 length 20.6 -> 20.6 + fpodppo-walker seed 1 return -0.0 -> 0.1 length 18.9 -> 19.2 + fpodppo-walker seed 2 return 0.5 -> 1.1 length 20.4 -> 21.3 + dppo-cheetah seed 0 return -386.3 -> -384.0 length 1000.0 -> 1000.0 + dppo-cheetah seed 1 return -445.0 -> -365.7 length 1000.0 -> 1000.0 + dppo-cheetah seed 2 return -382.1 -> -418.2 length 1000.0 -> 1000.0 + dppo-hopper seed 0 return 11.2 -> 75.7 length 18.7 -> 46.5 + dppo-hopper seed 1 return 12.4 -> 77.2 length 16.8 -> 47.7 + dppo-hopper seed 2 return 15.8 -> 81.5 length 21.0 -> 55.9 + dppo-walker seed 0 return -2.2 -> 220.0 length 17.6 -> 202.6 + dppo-walker seed 1 return -1.6 -> 191.9 length 17.6 -> 197.2 + dppo-walker seed 2 return -2.6 -> 156.5 length 15.1 -> 181.3 + diff --git a/experiments/e24-coverage/summary.tsv b/experiments/e24-coverage/summary.tsv new file mode 100644 index 0000000..eef20d7 --- /dev/null +++ b/experiments/e24-coverage/summary.tsv @@ -0,0 +1,19 @@ +cell policy algorithm task claim seed iterations runs first_return last10_return first_length last10_length +fpo-walker fpo-policy fpo Walker2d-v5 learns 0 100 true 0.7 964.0 19.2 413.6 +fpo-walker fpo-policy fpo Walker2d-v5 learns 1 100 true -0.3 1026.5 18.5 409.3 +fpo-walker fpo-policy fpo Walker2d-v5 learns 2 100 true 0.6 1101.0 20.0 424.0 +fpodppo-hopper fpo-policy dppo Hopper-v5 runs 0 20 true 14.9 15.7 20.2 21.1 +fpodppo-hopper fpo-policy dppo Hopper-v5 runs 1 20 true 19.7 17.6 22.8 21.3 +fpodppo-hopper fpo-policy dppo Hopper-v5 runs 2 20 true 17.0 18.1 21.9 22.4 +fpodppo-walker fpo-policy dppo Walker2d-v5 runs 0 20 true 1.6 1.4 20.6 20.6 +fpodppo-walker fpo-policy dppo Walker2d-v5 runs 1 20 true -0.0 0.1 18.9 19.2 +fpodppo-walker fpo-policy dppo Walker2d-v5 runs 2 20 true 0.5 1.1 20.4 21.3 +dppo-cheetah dppo-policy dppo HalfCheetah-v5 runs 0 20 true -386.3 -384.0 1000.0 1000.0 +dppo-cheetah dppo-policy dppo HalfCheetah-v5 runs 1 20 true -445.0 -365.7 1000.0 1000.0 +dppo-cheetah dppo-policy dppo HalfCheetah-v5 runs 2 20 true -382.1 -418.2 1000.0 1000.0 +dppo-hopper dppo-policy dppo Hopper-v5 runs 0 20 true 11.2 75.7 18.7 46.5 +dppo-hopper dppo-policy dppo Hopper-v5 runs 1 20 true 12.4 77.2 16.8 47.7 +dppo-hopper dppo-policy dppo Hopper-v5 runs 2 20 true 15.8 81.5 21.0 55.9 +dppo-walker dppo-policy dppo Walker2d-v5 runs 0 20 true -2.2 220.0 17.6 202.6 +dppo-walker dppo-policy dppo Walker2d-v5 runs 1 20 true -1.6 191.9 17.6 197.2 +dppo-walker dppo-policy dppo Walker2d-v5 runs 2 20 true -2.6 156.5 15.1 181.3 From d1f47830fb1db971bcff78774eccb1667458a5af Mon Sep 17 00:00:00 2001 From: tactino <18781106300@163.com> Date: Fri, 25 Sep 2026 19:34:37 -0400 Subject: [PATCH 4/5] exp: pre-register E28 - which DPPO cells learn MuJoCo in 409,600 steps E24's five DPPO cells, unchanged but run to 100 iterations. A status rule for every cell (Hopper, Walker2d: iterations 91-100 >= 500; HalfCheetah: +200 over the first, on 2 of 3 seeds); P2 dppo-policy learns Walker2d, P3 Hopper, P4 fpo-policy under DPPO learns neither. summarise.py smoke-tested on E24's twenty-iteration results, whose figures it reproduces. --- experiments/e28-dppo-learns/.gitignore | 3 + experiments/e28-dppo-learns/PROTOCOL.md | 157 ++++++++++++++++ experiments/e28-dppo-learns/run.sh | 55 ++++++ experiments/e28-dppo-learns/summarise.py | 230 +++++++++++++++++++++++ 4 files changed, 445 insertions(+) create mode 100644 experiments/e28-dppo-learns/.gitignore create mode 100644 experiments/e28-dppo-learns/PROTOCOL.md create mode 100644 experiments/e28-dppo-learns/run.sh create mode 100644 experiments/e28-dppo-learns/summarise.py diff --git a/experiments/e28-dppo-learns/.gitignore b/experiments/e28-dppo-learns/.gitignore new file mode 100644 index 0000000..99eef5b --- /dev/null +++ b/experiments/e28-dppo-learns/.gitignore @@ -0,0 +1,3 @@ +# Checkpoints and tensorboard event files; summarise.py reads out what they hold. +results/*/fpo/ +results/*/dppo/ diff --git a/experiments/e28-dppo-learns/PROTOCOL.md b/experiments/e28-dppo-learns/PROTOCOL.md new file mode 100644 index 0000000..152005f --- /dev/null +++ b/experiments/e28-dppo-learns/PROTOCOL.md @@ -0,0 +1,157 @@ +# E28 measurement protocol (pre-registered) + +**Written 2026-09-25, before any E28 run.** + +This file must not be edited after the first registered data point. Anything +learned afterwards goes in `AMENDMENT.md`, dated. + +--- + +## The question + +E24 showed that every DPPO combination on MuJoCo runs, and registered no +learning claim for any of them: DPPO from a random initialisation had not +learned HalfCheetah with `fpo-policy` (E17, E18). But in E24's twenty +iterations, unregistered, `dppo-policy` under DPPO rose on every seed of +Hopper and Walker2d, while `fpo-policy` under the same algorithm did not +move. Twenty iterations cannot call that learning. + +**Run long enough to decide it: which DPPO cells learn their task, and is +it the policy class that decides?** Same five cells, same settings, five +times as long - the length at which E24 read FPO on Walker2d. + +### What this cannot settle + +* *Why* one policy class learns and the other does not, if that is what + happens. `dppo-policy` and `fpo-policy` differ in the network, the action + chunk (4 against 1), the number of denoising steps and how each step's + noise is set. E28 can locate the difference in the policy class, not + inside it. +* Anything about DPPO as published: DPPO fine-tunes policies pretrained on + demonstrations, and every cell here starts from a random initialisation. + +--- + +## Declared in advance: what was already known + +1. **E24, the same five cells at twenty iterations**, on the same machine, + code and settings. Return at the first iteration and the mean of the + last ten, seeds 0, 1, 2: + + | cell | first | mean of iterations 11-20 | episode length | + | --- | --- | --- | --- | + | `dppo-hopper` | 11.2, 12.4, 15.8 | 75.7, 77.2, 81.5 | ~19 → ~50 | + | `dppo-walker` | −2.2, −1.6, −2.6 | 220.0, 191.9, 156.5 | ~17 → ~194 | + | `dppo-cheetah` | −386.3, −445.0, −382.1 | −384.0, −365.7, −418.2 | 1,000 | + | `fpodppo-hopper` | 14.9, 19.7, 17.0 | 15.7, 17.6, 18.1 | ~21 → ~22 | + | `fpodppo-walker` | 1.6, −0.0, 0.5 | 1.4, 0.1, 1.1 | ~20 → ~20 | + +2. **`fpo-policy · DPPO · HalfCheetah` at this length** (E17, E18): 100 + iterations of 4,096 steps, three seeds, no learning - E18, with the + observation statistics fixed, gained +5, +20, +2 against a threshold of + +200. +3. **FPO at this length** learns Hopper (E23: 1,587, 2,006, 1,776) and + Walker2d (E24: 964, 1,027, 1,101), from untrained policies near 0 to 20. +4. **Learning is not reproducible bit for bit** once the first update has + run (E20), so E28's first twenty iterations are not expected to repeat + E24's exactly; the collection before the first update is. + +--- + +## Design + +| cell | policy | algorithm | task | iterations | buffer / batch | seeds | +| --- | --- | --- | --- | --- | --- | --- | +| `dppo-hopper` | `dppo-policy hopper` | `dppo hopper` | Hopper-v5 | 100 | 1,024 / 512 | 0, 1, 2 | +| `dppo-walker` | `dppo-policy walker` | `dppo walker` | Walker2d-v5 | 100 | 1,024 / 512 | 0, 1, 2 | +| `dppo-cheetah` | `dppo-policy cheetah` | `dppo cheetah` | HalfCheetah-v5 | 100 | 1,024 / 512 | 0, 1, 2 | +| `fpodppo-hopper` | `fpo-policy` | `dppo hopper` | Hopper-v5 | 100 | 4,096 / 2,048 | 0, 1, 2 | +| `fpodppo-walker` | `fpo-policy` | `dppo walker` | Walker2d-v5 | 100 | 4,096 / 2,048 | 0, 1, 2 | + +Every cell is run by E24's own `run_cell.sh`, unchanged; only the number of +iterations differs from E24. Every iteration is 4,096 environment steps +(`dppo-policy` executes chunks of 4, so its buffer holds 1,024 entries), so +each seed sees 409,600 steps - E23's and E24's length for FPO. The DPPO +MuJoCo variants warm the learning rate up over 10 iterations and then hold +it (their scheduler's minimum equals its maximum), so the length changes +nothing about the schedule. Checkpoints every 20 iterations. + +Machine: `guangzhao`, CPU, every process capped at one thread, as E24 and +E27. Code: E24's branch, whose code is `main` at 8812b54, run from its own +worktree through `PYTHONPATH`. Client: plugrl-env-client at 931ab56 (#9, +which changes only robomimic). The three `dppo-policy` cells run first, the +two `fpo-policy` cells after them. + +--- + +## The status rule - what "learns" means, for every cell + +A cell **learns** if, on at least **2 of 3** seeds: + +* **Hopper, Walker2d**: the mean return of iterations 91-100 is at least + **500** - E23's and E24's threshold for FPO on the same tasks, from + untrained policies scoring −3 to 20 (known items 1, 3). +* **HalfCheetah**: the mean return of iterations 91-100 exceeds the first + iteration's by at least **+200** - E18's threshold for the same question on + the same task. An untrained policy scores −445 to −382 and runs every + episode to 1,000 steps, so an absolute threshold would read the starting + point, not the change; in E24's twenty iterations the change was −36 to + +79. + +Otherwise the cell **did not learn in 409,600 steps** - the matrix says +that, not "does not learn". + +--- + +## Predictions, and what falsifies each + +**P1 - every cell runs end to end.** For each cell, on every seed: all 100 +iterations logged, the checkpoint at iteration 100, no traceback in the +server log, and a client that exits 0. + +> Reported cell by cell, as in E24. Falsified, for that cell, by any seed +> failing any part. + +**P2 - `dppo-policy` learns Walker2d.** `dppo-walker` meets the status rule. + +> Grounds: +159 to +222 in twenty iterations on every seed, with episodes +> ten times longer. Falsified if fewer than two seeds reach 500: its early +> rise stalls. + +**P3 - `dppo-policy` learns Hopper.** `dppo-hopper` meets the status rule. + +> Weaker grounds: +64 to +66 in twenty iterations. Falsified if fewer than +> two seeds reach 500. + +**P4 - `fpo-policy` under DPPO does not learn Hopper or Walker2d.** Neither +`fpodppo-hopper` nor `fpodppo-walker` meets the status rule. + +> Grounds: no movement in twenty iterations (known item 1), and none in 100 +> on HalfCheetah (known item 2). Falsified if either cell meets it: then +> `fpo-policy` learns under DPPO too, only more slowly, and E24's contrast was +> one of speed. + +**Reading P2-P4 together.** If P2 or P3 holds and P4 holds, then under the +same algorithm, variant settings and data, whether DPPO learns from a random +start depends on the policy class, and E17 and E18's failure to learn belongs +to `fpo-policy`, not to DPPO. + +**Reported, not predicted:** `dppo-cheetah`'s status; every cell's first and +last-ten returns and episode lengths; wall clock. + +--- + +## Declared deviations allowed in advance + +1. One restart of any run that dies for a reason outside the experiment, + recorded in `AMENDMENT.md`. +2. A failure caused by how a cell was launched rather than what it runs is + fixed and the cell rerun once, recorded in `AMENDMENT.md`. A failure + inside the code under test is a result and is not rerun. + +--- + +## Reading order + +P1 cell by cell, then P2, P3, P4, then `dppo-cheetah`'s status and the +descriptive figures. diff --git a/experiments/e28-dppo-learns/run.sh b/experiments/e28-dppo-learns/run.sh new file mode 100644 index 0000000..721f5d4 --- /dev/null +++ b/experiments/e28-dppo-learns/run.sh @@ -0,0 +1,55 @@ +#!/usr/bin/env bash +# E28: E24's five DPPO cells, run to 100 iterations. See PROTOCOL.md. +# +# bash run.sh +# +# Every cell is E24's run_cell.sh, unchanged, writing into this experiment's +# results/. The server runs this checkout's code through PYTHONPATH; the +# Pythons are the venvs beside it on guangzhao, overridable from the +# environment. The three dppo-policy cells run first, then the two +# fpo-policy cells. +set -uo pipefail + +HERE="$(cd "$(dirname "$0")" && pwd)" +ROOT="$(cd "$HERE/../.." && pwd)" +R="$HERE/results" +CELL="$ROOT/experiments/e24-coverage/run_cell.sh" +mkdir -p "$R" + +export OMP_NUM_THREADS=1 +export PYTHONPATH="$ROOT/src" +export PLUGRL_ENV_CLIENT="${PLUGRL_ENV_CLIENT:-$ROOT/../plugrl-env-client}" +export PLUGRL_SERVER_PYTHON="${PLUGRL_SERVER_PYTHON:-$ROOT/../plugrl-server/.venv/bin/python}" +export PLUGRL_CLIENT_PYTHON="${PLUGRL_CLIENT_PYTHON:-$PLUGRL_ENV_CLIENT/.venv/bin/python}" +for py in "$PLUGRL_SERVER_PYTHON" "$PLUGRL_CLIENT_PYTHON"; do + [ -x "$py" ] || { echo "error: no Python at $py" >&2; exit 1; } +done + +date '+start %F %T' +echo "code: $(git -C "$ROOT" rev-parse --short HEAD) (src $PYTHONPATH)" +echo "client: $(git -C "$PLUGRL_ENV_CLIENT" rev-parse --short HEAD) ($PLUGRL_CLIENT_PYTHON)" + +echo "--- wave 1: dppo-policy" +BUFFER=1024 BATCH=512 PORT_BASE=9500 bash "$CELL" dppo-hopper dppo-policy hopper dppo hopper Hopper-v5 100 "$R/dppo-hopper" > "$R/dppo-hopper.out" 2>&1 & +A=$! +sleep 20 +BUFFER=1024 BATCH=512 PORT_BASE=9510 bash "$CELL" dppo-walker dppo-policy walker dppo walker Walker2d-v5 100 "$R/dppo-walker" > "$R/dppo-walker.out" 2>&1 & +B=$! +sleep 20 +BUFFER=1024 BATCH=512 PORT_BASE=9520 bash "$CELL" dppo-cheetah dppo-policy cheetah dppo cheetah HalfCheetah-v5 100 "$R/dppo-cheetah" > "$R/dppo-cheetah.out" 2>&1 & +C=$! +wait "$A" "$B" "$C" + +echo "--- wave 2: fpo-policy" +PORT_BASE=9530 bash "$CELL" fpodppo-hopper fpo-policy default dppo hopper Hopper-v5 100 "$R/fpodppo-hopper" > "$R/fpodppo-hopper.out" 2>&1 & +D=$! +sleep 20 +PORT_BASE=9540 bash "$CELL" fpodppo-walker fpo-policy default dppo walker Walker2d-v5 100 "$R/fpodppo-walker" > "$R/fpodppo-walker.out" 2>&1 & +E=$! +wait "$D" "$E" + +date '+end %F %T' +for c in dppo-hopper dppo-walker dppo-cheetah fpodppo-hopper fpodppo-walker; do + printf '%-15s %s\n' "$c" "$(grep -E 'failed seeds' "$R/$c.out" 2>/dev/null)" +done +echo E28_DONE diff --git a/experiments/e28-dppo-learns/summarise.py b/experiments/e28-dppo-learns/summarise.py new file mode 100644 index 0000000..1eb7ced --- /dev/null +++ b/experiments/e28-dppo-learns/summarise.py @@ -0,0 +1,230 @@ +"""Read E28's verdicts from each cell's logs and tensorboards. + + python summarise.py + +P1 cell by cell, then the status rule applied to every cell, then P2, P3, +P4 - PROTOCOL.md's reading order. Writes `summary.tsv`, one row per cell and +seed, in E24's columns with the cell's status as its claim, so the coverage +matrix reads E24, E27 and E28 the same way. +""" + +import datetime +import pathlib +import re +import sys + +from tensorboard.backend.event_processing.event_accumulator import EventAccumulator + +HERE = pathlib.Path(__file__).resolve().parent +RESULTS = HERE / "results" +SEEDS = (0, 1, 2) +ITERS = 100 + +# cell: (policy, algorithm, task) +CELLS = { + "dppo-hopper": ("dppo-policy", "dppo", "Hopper-v5"), + "dppo-walker": ("dppo-policy", "dppo", "Walker2d-v5"), + "dppo-cheetah": ("dppo-policy", "dppo", "HalfCheetah-v5"), + "fpodppo-hopper": ("fpo-policy", "dppo", "Hopper-v5"), + "fpodppo-walker": ("fpo-policy", "dppo", "Walker2d-v5"), +} +ABSOLUTE = 500.0 # Hopper, Walker2d: mean of iterations 91-100 +GAIN = 200.0 # HalfCheetah: mean of iterations 91-100 minus the first + + +def run_dir(cell: str, seed: int) -> pathlib.Path | None: + hits = sorted((RESULTS / cell).glob(f"*/*/{cell}-seed{seed}")) + return hits[0] if hits else None + + +def curve(run: pathlib.Path, tag: str) -> list[float]: + events = sorted((run / "tensorboard").glob("events.*")) + if not events: + return [] + acc = EventAccumulator(str(events[0]), size_guidance={"scalars": 0}) + acc.Reload() + if tag not in acc.Tags()["scalars"]: + return [] + return [e.value for e in acc.Scalars(tag)] + + +def wall_minutes(out: str) -> float | None: + stamps = dict(re.findall(r"^(start|end)\s+(\S+ \S+)$", out, flags=re.M)) + if "start" not in stamps or "end" not in stamps: + return None + fmt = "%Y-%m-%d %H:%M:%S" + delta = datetime.datetime.strptime(stamps["end"], fmt) - datetime.datetime.strptime( + stamps["start"], fmt + ) + return delta.total_seconds() / 60 + + +def mean(xs: list[float]) -> float: + return sum(xs) / len(xs) if xs else float("nan") + + +def seed_passes(task: str, reward: list[float]) -> bool: + if len(reward) < ITERS: + return False + last10 = mean(reward[ITERS - 10 : ITERS]) + if task == "HalfCheetah-v5": + return last10 - reward[0] >= GAIN + return last10 >= ABSOLUTE + + +def main() -> int: + rows = {} + print("P1 every cell runs end to end") + runs_ok = {} + for cell, (policy, algo, task) in CELLS.items(): + out_path = RESULTS / f"{cell}.out" + out = ( + out_path.read_text(encoding="utf-8", errors="replace") + if out_path.exists() + else "" + ) + exits = { + int(s): int(rc) for s, rc in re.findall(r"seed (\d) finished rc=(\d+)", out) + } + ok_cell = True + for seed in SEEDS: + run = run_dir(cell, seed) + reward = curve(run, "rollout/reward") if run else [] + length = curve(run, "rollout/length") if run else [] + saves = ( + [int(p.name) for p in run.iterdir() if p.name.isdigit()] if run else [] + ) + log_path = RESULTS / cell / f"server-seed{seed}.log" + log = ( + log_path.read_text(encoding="utf-8", errors="replace") + if log_path.exists() + else "" + ) + # The checkpoint directory's step can be off by one (E16), so + # the last save is found by count, not by name. + ok = ( + len(reward) >= ITERS + and len(saves) >= ITERS // 20 + and bool(log) + and "Traceback" not in log + and exits.get(seed) == 0 + ) + ok_cell &= ok + rows[cell, seed] = dict( + ok=ok, reward=reward, length=length, passes=seed_passes(task, reward) + ) + runs_ok[cell] = ok_cell + minutes = wall_minutes(out) + print( + f" {cell:15s} {policy:12s} {task:15s} " + f"{'HOLDS' if ok_cell else 'FAILS'}" + + (f" ({minutes:.0f} min)" if minutes is not None else "") + ) + for seed in SEEDS: + r = rows[cell, seed] + if not r["ok"]: + print( + f" seed {seed}: {len(r['reward'])} of {ITERS} iterations logged" + ) + + print( + f"\nstatus learns: mean of iterations 91-100 >= {ABSOLUTE:.0f} " + f"(HalfCheetah: >= first + {GAIN:.0f}) on 2 of 3" + ) + status = {} + for cell, (policy, algo, task) in CELLS.items(): + passed = sum(rows[cell, s]["passes"] for s in SEEDS) + if not runs_ok[cell]: + status[cell] = "did not run" + elif passed >= 2: + status[cell] = "learns" + else: + status[cell] = "did not learn in 409,600 steps" + figures = [] + for s in SEEDS: + reward = rows[cell, s]["reward"] + last10 = ( + mean(reward[ITERS - 10 : ITERS]) + if len(reward) >= ITERS + else float("nan") + ) + if task == "HalfCheetah-v5" and reward: + figures.append(f"{last10 - reward[0]:+.1f}") + else: + figures.append(f"{last10:.1f}") + print( + f" {cell:15s} {', '.join(figures):28s} {passed} of 3 -> {status[cell]}" + ) + + def verdict(name: str, text: str, holds: bool | None) -> None: + mark = ( + "not read - a cell did not run" + if holds is None + else ("HOLDS" if holds else "FALSIFIED") + ) + print(f"\n{name} {text}\n {mark}") + + def learns(cell: str) -> bool | None: + return None if status[cell] == "did not run" else status[cell] == "learns" + + verdict("P2", "dppo-policy learns Walker2d", learns("dppo-walker")) + verdict("P3", "dppo-policy learns Hopper", learns("dppo-hopper")) + h, w = learns("fpodppo-hopper"), learns("fpodppo-walker") + verdict( + "P4", + "fpo-policy under DPPO learns neither Hopper nor Walker2d", + None if h is None or w is None else not h and not w, + ) + + print( + "\nreported: first iteration and mean of the last ten, return and episode length" + ) + lines = [ + "cell\tpolicy\talgorithm\ttask\tclaim\tseed\titerations\truns\tfirst_return\tlast10_return\tfirst_length\tlast10_length" + ] + for cell, (policy, algo, task) in CELLS.items(): + for s in SEEDS: + r = rows[cell, s] + reward, length = r["reward"], r["length"] + first = reward[0] if reward else float("nan") + last10 = ( + mean(reward[ITERS - 10 : ITERS]) + if len(reward) >= ITERS + else float("nan") + ) + first_len = length[0] if length else float("nan") + last10_len = ( + mean(length[ITERS - 10 : ITERS]) + if len(length) >= ITERS + else float("nan") + ) + print( + f" {cell:15s} seed {s} return {first:8.1f} -> {last10:8.1f} " + f"length {first_len:6.1f} -> {last10_len:6.1f}" + ) + claim = "learns" if status[cell] == "learns" else "runs" + lines.append( + "\t".join( + [ + cell, + policy, + algo, + task, + claim, + str(s), + str(len(reward)), + str(r["ok"]).lower(), + f"{first:.1f}", + f"{last10:.1f}", + f"{first_len:.1f}", + f"{last10_len:.1f}", + ] + ) + ) + (HERE / "summary.tsv").write_text("\n".join(lines) + "\n", encoding="utf-8") + print(f"\nwrote {HERE / 'summary.tsv'}") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) From 4e847625f0f611d3fd96e1f36cf670811efacebd Mon Sep 17 00:00:00 2001 From: tactino <18781106300@163.com> Date: Fri, 25 Sep 2026 21:17:19 -0400 Subject: [PATCH 5/5] exp(e28): results - dppo-policy learns HalfCheetah under DPPO; fpo-policy does not move P1 holds 5 of 5. P2 and P3 falsified: dppo-policy reaches 260-292 on Walker2d and 276-430 on Hopper, below 500. P4 holds: fpo-policy under DPPO ends at 25-27 and 1-3. dppo-cheetah learns by the rule (+481, +84, +544). The registered joint reading does not apply; the contrast is described, with the per-step noise schedule named as an untested candidate. --- experiments/e28-dppo-learns/FINDINGS.md | 127 ++++++++++++++++++ .../e28-dppo-learns/results/dppo-cheetah.out | 10 ++ .../results/dppo-cheetah/client-seed0.log | 39 ++++++ .../results/dppo-cheetah/client-seed1.log | 39 ++++++ .../results/dppo-cheetah/client-seed2.log | 39 ++++++ .../results/dppo-cheetah/server-seed0.log | 26 ++++ .../results/dppo-cheetah/server-seed1.log | 26 ++++ .../results/dppo-cheetah/server-seed2.log | 26 ++++ .../e28-dppo-learns/results/dppo-hopper.out | 10 ++ .../results/dppo-hopper/client-seed0.log | 40 ++++++ .../results/dppo-hopper/client-seed1.log | 40 ++++++ .../results/dppo-hopper/client-seed2.log | 40 ++++++ .../results/dppo-hopper/server-seed0.log | 26 ++++ .../results/dppo-hopper/server-seed1.log | 26 ++++ .../results/dppo-hopper/server-seed2.log | 26 ++++ .../e28-dppo-learns/results/dppo-walker.out | 10 ++ .../results/dppo-walker/client-seed0.log | 40 ++++++ .../results/dppo-walker/client-seed1.log | 41 ++++++ .../results/dppo-walker/client-seed2.log | 40 ++++++ .../results/dppo-walker/server-seed0.log | 26 ++++ .../results/dppo-walker/server-seed1.log | 26 ++++ .../results/dppo-walker/server-seed2.log | 26 ++++ .../results/fpodppo-hopper.out | 10 ++ .../results/fpodppo-hopper/client-seed0.log | 49 +++++++ .../results/fpodppo-hopper/client-seed1.log | 50 +++++++ .../results/fpodppo-hopper/client-seed2.log | 49 +++++++ .../results/fpodppo-hopper/server-seed0.log | 25 ++++ .../results/fpodppo-hopper/server-seed1.log | 25 ++++ .../results/fpodppo-hopper/server-seed2.log | 25 ++++ .../results/fpodppo-walker.out | 10 ++ .../results/fpodppo-walker/client-seed0.log | 50 +++++++ .../results/fpodppo-walker/client-seed1.log | 49 +++++++ .../results/fpodppo-walker/client-seed2.log | 50 +++++++ .../results/fpodppo-walker/server-seed0.log | 25 ++++ .../results/fpodppo-walker/server-seed1.log | 25 ++++ .../results/fpodppo-walker/server-seed2.log | 25 ++++ experiments/e28-dppo-learns/results/run.out | 12 ++ .../e28-dppo-learns/results/verdicts.txt | 41 ++++++ experiments/e28-dppo-learns/summary.tsv | 16 +++ 39 files changed, 1285 insertions(+) create mode 100644 experiments/e28-dppo-learns/FINDINGS.md create mode 100644 experiments/e28-dppo-learns/results/dppo-cheetah.out create mode 100644 experiments/e28-dppo-learns/results/dppo-cheetah/client-seed0.log create mode 100644 experiments/e28-dppo-learns/results/dppo-cheetah/client-seed1.log create mode 100644 experiments/e28-dppo-learns/results/dppo-cheetah/client-seed2.log create mode 100644 experiments/e28-dppo-learns/results/dppo-cheetah/server-seed0.log create mode 100644 experiments/e28-dppo-learns/results/dppo-cheetah/server-seed1.log create mode 100644 experiments/e28-dppo-learns/results/dppo-cheetah/server-seed2.log create mode 100644 experiments/e28-dppo-learns/results/dppo-hopper.out create mode 100644 experiments/e28-dppo-learns/results/dppo-hopper/client-seed0.log create mode 100644 experiments/e28-dppo-learns/results/dppo-hopper/client-seed1.log create mode 100644 experiments/e28-dppo-learns/results/dppo-hopper/client-seed2.log create mode 100644 experiments/e28-dppo-learns/results/dppo-hopper/server-seed0.log create mode 100644 experiments/e28-dppo-learns/results/dppo-hopper/server-seed1.log create mode 100644 experiments/e28-dppo-learns/results/dppo-hopper/server-seed2.log create mode 100644 experiments/e28-dppo-learns/results/dppo-walker.out create mode 100644 experiments/e28-dppo-learns/results/dppo-walker/client-seed0.log create mode 100644 experiments/e28-dppo-learns/results/dppo-walker/client-seed1.log create mode 100644 experiments/e28-dppo-learns/results/dppo-walker/client-seed2.log create mode 100644 experiments/e28-dppo-learns/results/dppo-walker/server-seed0.log create mode 100644 experiments/e28-dppo-learns/results/dppo-walker/server-seed1.log create mode 100644 experiments/e28-dppo-learns/results/dppo-walker/server-seed2.log create mode 100644 experiments/e28-dppo-learns/results/fpodppo-hopper.out create mode 100644 experiments/e28-dppo-learns/results/fpodppo-hopper/client-seed0.log create mode 100644 experiments/e28-dppo-learns/results/fpodppo-hopper/client-seed1.log create mode 100644 experiments/e28-dppo-learns/results/fpodppo-hopper/client-seed2.log create mode 100644 experiments/e28-dppo-learns/results/fpodppo-hopper/server-seed0.log create mode 100644 experiments/e28-dppo-learns/results/fpodppo-hopper/server-seed1.log create mode 100644 experiments/e28-dppo-learns/results/fpodppo-hopper/server-seed2.log create mode 100644 experiments/e28-dppo-learns/results/fpodppo-walker.out create mode 100644 experiments/e28-dppo-learns/results/fpodppo-walker/client-seed0.log create mode 100644 experiments/e28-dppo-learns/results/fpodppo-walker/client-seed1.log create mode 100644 experiments/e28-dppo-learns/results/fpodppo-walker/client-seed2.log create mode 100644 experiments/e28-dppo-learns/results/fpodppo-walker/server-seed0.log create mode 100644 experiments/e28-dppo-learns/results/fpodppo-walker/server-seed1.log create mode 100644 experiments/e28-dppo-learns/results/fpodppo-walker/server-seed2.log create mode 100644 experiments/e28-dppo-learns/results/run.out create mode 100644 experiments/e28-dppo-learns/results/verdicts.txt create mode 100644 experiments/e28-dppo-learns/summary.tsv diff --git a/experiments/e28-dppo-learns/FINDINGS.md b/experiments/e28-dppo-learns/FINDINGS.md new file mode 100644 index 0000000..404e99c --- /dev/null +++ b/experiments/e28-dppo-learns/FINDINGS.md @@ -0,0 +1,127 @@ +# E28: under DPPO, `dppo-policy` learns HalfCheetah and climbs Hopper and Walker2d; `fpo-policy` does not move + +2026-09-25 · Linux workstation (`guangzhao`), CPU only · five DPPO cells, +three seeds each, 100 iterations of 4,096 steps · protocol: +[`PROTOCOL.md`](PROTOCOL.md) (`d1f4783`, before any run) + +--- + +## The result + +E24 ran every DPPO combination on MuJoCo for twenty iterations and claimed +only that each runs. E28 ran the same five cells, unchanged, five times as +long, with a rule for what "learns" means fixed in advance. + +| cell | mean of iterations 91-100, seeds 0 / 1 / 2 | status | +| --- | --- | --- | +| `dppo-policy` · DPPO · HalfCheetah | **+481 / +84 / +544** over the first iteration | **learns** (2 of 3 past +200) | +| `dppo-policy` · DPPO · Hopper | 339 / 430 / 276, from 11-16 | did not learn in 409,600 steps (0 of 3 past 500) | +| `dppo-policy` · DPPO · Walker2d | 260 / 292 / 274, from −3 to −2 | did not learn in 409,600 steps (0 of 3) | +| `fpo-policy` · DPPO · Hopper | 25 / 25 / 27, from 15-20 | did not learn in 409,600 steps (0 of 3) | +| `fpo-policy` · DPPO · Walker2d | 3 / 1 / 2, from −0 to 2 | did not learn in 409,600 steps (0 of 3) | + +* **P1 holds, 5 of 5**: all fifteen seeds logged 100 iterations, wrote the + checkpoint at 100, have no traceback and a client that exited 0; 15 to 20 + minutes per cell. +* **P2 falsified**: `dppo-policy` does not reach 500 on Walker2d on any seed. +* **P3 falsified**: nor on Hopper. +* **P4 holds**: `fpo-policy` under DPPO clears neither threshold, by far. +* **Reported**: `dppo-policy` learns HalfCheetah by the rule, on seeds 0 and + 2. + +With E24 and E27, the matrix of the two MLP policies now reads: + +| | HalfCheetah | Hopper | Walker2d | robomimic square | +| --- | --- | --- | --- | --- | +| `fpo-policy` · FPO | learns (E6, E16) | learns (E23) | learns (E24) | runs (E27) | +| `fpo-policy` · DPPO | runs, no learning in 409,600 steps (E17, E18) | **runs, no learning (E28)** | **runs, no learning (E28)** | runs (E27) | +| `dppo-policy` · DPPO | **learns (E28)** | **runs, rising: 276-430 (E28)** | **runs, rising: 260-292 (E28)** | runs (E27) | + +--- + +## Where my predictions went wrong + +P2 and P3 extrapolated E24's twenty-iteration slope, and neither curve kept +it. Mean return over iterations 11-20, 41-50 and 91-100, seeds 0 / 1 / 2: + +| cell | 11-20 | 41-50 | 91-100 | +| --- | --- | --- | --- | +| `dppo-policy` · Walker2d | 186 / 172 / 155 | 251 / 224 / 267 | 260 / 292 / 274 | +| `dppo-policy` · Hopper | 77 / 74 / 80 | 195 / 214 / 201 | 339 / 430 / 276 | +| `dppo-policy` · HalfCheetah | −382 / −394 / −403 | −332 / −379 / −331 | 95 / −361 / 162 | + +Walker2d flattened after about fifty iterations. Hopper was still climbing +at the end - seed 1's best iteration reached 493. HalfCheetah, which showed +nothing in twenty iterations and was registered without a prediction, is the +one that crossed its threshold. + +Every seed's first iteration is identical to E24's, to the decimal: the +collection before the first update is deterministic, as E20 found, and +what follows is not (known item 4). + +--- + +## What the contrast shows, and what it does not + +The protocol's joint reading - "the policy class decides whether DPPO learns +from a random start" - required P2 or P3 to hold, and neither did, so it is +not claimed. Read as description, with every setting held equal - the same +algorithm and variant, 4,096 environment steps per iteration, two minibatches +per epoch and so five optimiser steps per iteration (the accumulation window +of 8 never fills, and each epoch's end forces the step) - the two policy +classes do not behave alike at all: + +| | HalfCheetah | Hopper | Walker2d | +| --- | --- | --- | --- | +| `dppo-policy`, last ten minus first | +84 to +544 | +260 to +417 | +262 to +294 | +| `fpo-policy`, the same | +2 to +20 (E18; last ten minus iterations 1-10) | +5 to +11 | +1 to +2 | + +`fpo-policy` learns all three tasks under FPO (E6, E16, E23, E24) and none +under DPPO. By the project's rule, that is a defect of ours to locate, not a +finding about DPPO: the same policy is trainable, and the same algorithm +trains a different policy. + +**One candidate, from reading the code, not tested.** The two policies set +each denoising step's noise differently. `dppo-policy` clamps each step's +standard deviation from below at the variant's `sampling_noise_level`, 0.1 +here, so every step is at least 0.1. `fpo-policy`, a flow policy, uses +σ_t = level · √(t / (1 − t)) and a step deviation of σ_t · √dt, which over its +ten steps is 0.1 at the first, about 0.03 midway and about 0.01 at the last. +E18 found `approx_kl` near 10⁻⁸ and `clipfrac` at 0 for `fpo-policy` under +DPPO - the ratio barely leaves 1. Whether the noise schedule is why is the +next controlled comparison, not something E28 can say. + +--- + +## What this does and does not support + +**Supported:** + +* `dppo-policy`, trained by DPPO from a random initialisation on this + platform, learns HalfCheetah-v5 within 409,600 steps on two of three seeds, + and improves Hopper-v5 and Walker2d-v5 on every seed. +* `fpo-policy` under DPPO does not learn Hopper or Walker2d in 409,600 steps, + as it did not learn HalfCheetah (E17, E18). + +**Not supported:** + +* That DPPO learns Hopper or Walker2d on this platform: neither reached the + bar set in advance. +* Any reason for the contrast. The two policy classes differ in network, + action chunk, denoising steps and noise; E28 holds the algorithm fixed and + varies all of those at once. +* Anything about DPPO as published, which fine-tunes pretrained policies. + +--- + +## Reproducing + +```bash +bash run.sh # five cells in two waves; about 37 minutes on 24 cores +python summarise.py # P1, the status rule, P2-P4, the figures; writes summary.tsv +``` + +`run.sh` calls E24's `run_cell.sh` unchanged. `results/verdicts.txt` is +`summarise.py`'s output; `summary.tsv` has one row per cell and seed in E24's +columns, with `claim` set from each cell's status. Checkpoints and tensorboard +files stay on `guangzhao` (`.gitignore`). diff --git a/experiments/e28-dppo-learns/results/dppo-cheetah.out b/experiments/e28-dppo-learns/results/dppo-cheetah.out new file mode 100644 index 0000000..bb4653e --- /dev/null +++ b/experiments/e28-dppo-learns/results/dppo-cheetah.out @@ -0,0 +1,10 @@ +cell: dppo-cheetah = dppo-policy/cheetah x dppo/cheetah x HalfCheetah-v5 +iters: 100 x 1024 batch: 512 replan: 4 seeds: 0 1 2 +out: /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-cheetah +start 2026-09-25 19:34:50 +seed 0 finished rc=0 at 19:49:50 +seed 2 finished rc=0 at 19:49:56 +seed 1 finished rc=0 at 19:49:59 +end 2026-09-25 19:49:59 +failed seeds: 0 +CELL_DONE dppo-cheetah diff --git a/experiments/e28-dppo-learns/results/dppo-cheetah/client-seed0.log b/experiments/e28-dppo-learns/results/dppo-cheetah/client-seed0.log new file mode 100644 index 0000000..ac99231 --- /dev/null +++ b/experiments/e28-dppo-learns/results/dppo-cheetah/client-seed0.log @@ -0,0 +1,39 @@ +2026-09-25 19:34:52.602 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 19:34:52.602 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 19:34:52.602 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 19:34:52.602 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 19:34:52.603 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 19:34:52.647 | INFO | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-193452-1e20a56e output_dir=runs/mujoco-v1-nenv1-20260925-193452-1e20a56e recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 19:34:52.684 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9520... +2026-09-25 19:34:52.687 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'DPPOPolicy', 'action_dim': 6, 'action_horizon': 4} +2026-09-25 19:35:22.692 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=15253 infer_calls=3814 feedback_calls=3813 infer_wait=28.028s infer_obs_pack=0.062s env_step=1.084s feedback_total=0.452s feedback_obs_pack=0.096s feedback_info_pack=0.019s effective_fps=514.87 +2026-09-25 19:35:53.411 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=28677 infer_calls=7170 feedback_calls=7169 infer_wait=56.991s infer_obs_pack=0.118s env_step=2.057s feedback_total=0.840s feedback_obs_pack=0.184s feedback_info_pack=0.036s effective_fps=477.90 +2026-09-25 19:36:23.415 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=43189 infer_calls=10798 feedback_calls=10797 infer_wait=85.038s infer_obs_pack=0.181s env_step=3.147s feedback_total=1.266s feedback_obs_pack=0.281s feedback_info_pack=0.055s effective_fps=481.84 +2026-09-25 19:36:55.837 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=57349 infer_calls=14338 feedback_calls=14337 infer_wait=115.558s infer_obs_pack=0.254s env_step=4.198s feedback_total=1.683s feedback_obs_pack=0.374s feedback_info_pack=0.072s effective_fps=471.26 +2026-09-25 19:37:25.843 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=71845 infer_calls=17962 feedback_calls=17961 infer_wait=143.585s infer_obs_pack=0.316s env_step=5.285s feedback_total=2.135s feedback_obs_pack=0.473s feedback_info_pack=0.092s effective_fps=474.79 +2026-09-25 19:37:58.767 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=86021 infer_calls=21506 feedback_calls=21505 infer_wait=174.593s infer_obs_pack=0.381s env_step=6.339s feedback_total=2.565s feedback_obs_pack=0.567s feedback_info_pack=0.110s effective_fps=467.81 +2026-09-25 19:38:28.768 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=100461 infer_calls=25116 feedback_calls=25115 infer_wait=202.613s infer_obs_pack=0.448s env_step=7.441s feedback_total=2.995s feedback_obs_pack=0.665s feedback_info_pack=0.131s effective_fps=470.55 +2026-09-25 19:38:58.771 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=114333 infer_calls=28584 feedback_calls=28583 infer_wait=230.700s infer_obs_pack=0.508s env_step=8.513s feedback_total=3.409s feedback_obs_pack=0.760s feedback_info_pack=0.151s effective_fps=470.26 +2026-09-25 19:39:30.270 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=126981 infer_calls=31746 feedback_calls=31745 infer_wait=260.397s infer_obs_pack=0.566s env_step=9.526s feedback_total=3.789s feedback_obs_pack=0.851s feedback_info_pack=0.170s effective_fps=462.97 +2026-09-25 19:40:00.272 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=141017 infer_calls=35255 feedback_calls=35254 infer_wait=288.396s infer_obs_pack=0.631s env_step=10.646s feedback_total=4.224s feedback_obs_pack=0.950s feedback_info_pack=0.191s effective_fps=464.03 +2026-09-25 19:40:30.278 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=154989 infer_calls=38748 feedback_calls=38747 infer_wait=316.462s infer_obs_pack=0.693s env_step=11.723s feedback_total=4.652s feedback_obs_pack=1.052s feedback_info_pack=0.212s effective_fps=464.69 +2026-09-25 19:41:01.904 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=167941 infer_calls=41986 feedback_calls=41985 infer_wait=346.285s infer_obs_pack=0.751s env_step=12.739s feedback_total=5.037s feedback_obs_pack=1.143s feedback_info_pack=0.232s effective_fps=460.35 +2026-09-25 19:41:31.909 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=181737 infer_calls=45435 feedback_calls=45434 infer_wait=374.333s infer_obs_pack=0.813s env_step=13.833s feedback_total=5.460s feedback_obs_pack=1.241s feedback_info_pack=0.250s effective_fps=460.75 +2026-09-25 19:42:01.911 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=195837 infer_calls=48960 feedback_calls=48959 infer_wait=402.386s infer_obs_pack=0.876s env_step=14.920s feedback_total=5.887s feedback_obs_pack=1.338s feedback_info_pack=0.268s effective_fps=461.80 +2026-09-25 19:42:32.864 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=208901 infer_calls=52226 feedback_calls=52225 infer_wait=431.556s infer_obs_pack=0.935s env_step=15.904s feedback_total=6.285s feedback_obs_pack=1.425s feedback_info_pack=0.285s effective_fps=459.45 +2026-09-25 19:43:02.864 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=223665 infer_calls=55917 feedback_calls=55916 infer_wait=459.606s infer_obs_pack=0.998s env_step=16.972s feedback_total=6.735s feedback_obs_pack=1.523s feedback_info_pack=0.305s effective_fps=461.82 +2026-09-25 19:43:34.686 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=237573 infer_calls=59394 feedback_calls=59393 infer_wait=489.572s infer_obs_pack=1.057s env_step=17.987s feedback_total=7.162s feedback_obs_pack=1.615s feedback_info_pack=0.323s effective_fps=460.61 +2026-09-25 19:44:04.690 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=252757 infer_calls=63190 feedback_calls=63189 infer_wait=517.520s infer_obs_pack=1.123s env_step=19.101s feedback_total=7.648s feedback_obs_pack=1.714s feedback_info_pack=0.342s effective_fps=463.44 +2026-09-25 19:44:36.099 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=266245 infer_calls=66562 feedback_calls=66561 infer_wait=547.103s infer_obs_pack=1.181s env_step=20.105s feedback_total=8.063s feedback_obs_pack=1.804s feedback_info_pack=0.359s effective_fps=461.87 +2026-09-25 19:45:06.099 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=280089 infer_calls=70023 feedback_calls=70022 infer_wait=575.151s infer_obs_pack=1.244s env_step=21.194s feedback_total=8.485s feedback_obs_pack=1.903s feedback_info_pack=0.378s effective_fps=462.14 +2026-09-25 19:45:36.103 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=294169 infer_calls=73543 feedback_calls=73542 infer_wait=603.200s infer_obs_pack=1.307s env_step=22.289s feedback_total=8.903s feedback_obs_pack=2.000s feedback_info_pack=0.398s effective_fps=462.75 +2026-09-25 19:46:06.640 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=307205 infer_calls=76802 feedback_calls=76801 infer_wait=631.973s infer_obs_pack=1.363s env_step=23.257s feedback_total=9.305s feedback_obs_pack=2.087s feedback_info_pack=0.416s effective_fps=461.34 +2026-09-25 19:46:36.643 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=321325 infer_calls=80332 feedback_calls=80331 infer_wait=660.007s infer_obs_pack=1.427s env_step=24.348s feedback_total=9.741s feedback_obs_pack=2.185s feedback_info_pack=0.433s effective_fps=461.99 +2026-09-25 19:47:06.648 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=335285 infer_calls=83822 feedback_calls=83821 infer_wait=688.056s infer_obs_pack=1.490s env_step=25.433s feedback_total=10.171s feedback_obs_pack=2.281s feedback_info_pack=0.451s effective_fps=462.37 +2026-09-25 19:47:37.741 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=348165 infer_calls=87042 feedback_calls=87041 infer_wait=717.392s infer_obs_pack=1.548s env_step=26.399s feedback_total=10.571s feedback_obs_pack=2.369s feedback_info_pack=0.467s effective_fps=460.59 +2026-09-25 19:48:07.743 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=362365 infer_calls=90592 feedback_calls=90591 infer_wait=745.495s infer_obs_pack=1.608s env_step=27.448s feedback_total=10.995s feedback_obs_pack=2.464s feedback_info_pack=0.483s effective_fps=461.29 +2026-09-25 19:48:40.495 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=376833 infer_calls=94209 feedback_calls=94208 infer_wait=776.320s infer_obs_pack=1.670s env_step=28.510s feedback_total=11.426s feedback_obs_pack=2.559s feedback_info_pack=0.500s effective_fps=460.72 +2026-09-25 19:49:10.499 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=391597 infer_calls=97900 feedback_calls=97899 infer_wait=804.347s infer_obs_pack=1.734s env_step=29.602s feedback_total=11.869s feedback_obs_pack=2.657s feedback_info_pack=0.518s effective_fps=462.03 +2026-09-25 19:49:42.119 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=405509 infer_calls=101378 feedback_calls=101377 infer_wait=834.130s infer_obs_pack=1.793s env_step=30.607s feedback_total=12.288s feedback_obs_pack=2.748s feedback_info_pack=0.534s effective_fps=461.42 +2026-09-25 19:49:50.599 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=409604 infer_calls=102401 feedback_calls=102401 infer_wait=839.060s infer_obs_pack=1.809s env_step=30.876s feedback_total=12.394s feedback_obs_pack=2.772s feedback_info_pack=0.538s effective_fps=463.28 +2026-09-25 19:49:50.599 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 409/409600 episodes. diff --git a/experiments/e28-dppo-learns/results/dppo-cheetah/client-seed1.log b/experiments/e28-dppo-learns/results/dppo-cheetah/client-seed1.log new file mode 100644 index 0000000..6ff6852 --- /dev/null +++ b/experiments/e28-dppo-learns/results/dppo-cheetah/client-seed1.log @@ -0,0 +1,39 @@ +2026-09-25 19:34:58.592 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 19:34:58.592 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 19:34:58.592 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 19:34:58.593 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 19:34:58.593 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 19:34:58.637 | INFO | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-193458-052a5174 output_dir=runs/mujoco-v1-nenv1-20260925-193458-052a5174 recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 19:34:58.683 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9521... +2026-09-25 19:34:58.686 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'DPPOPolicy', 'action_dim': 6, 'action_horizon': 4} +2026-09-25 19:35:28.688 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=14277 infer_calls=3570 feedback_calls=3569 infer_wait=28.024s infer_obs_pack=0.063s env_step=1.107s feedback_total=0.419s feedback_obs_pack=0.100s feedback_info_pack=0.021s effective_fps=482.13 +2026-09-25 19:35:58.692 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=28589 infer_calls=7148 feedback_calls=7147 infer_wait=56.050s infer_obs_pack=0.125s env_step=2.204s feedback_total=0.855s feedback_obs_pack=0.202s feedback_info_pack=0.042s effective_fps=482.65 +2026-09-25 19:36:28.692 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=41009 infer_calls=10253 feedback_calls=10252 infer_wait=84.344s infer_obs_pack=0.178s env_step=3.146s feedback_total=1.234s feedback_obs_pack=0.288s feedback_info_pack=0.061s effective_fps=461.28 +2026-09-25 19:36:58.697 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=55481 infer_calls=13871 feedback_calls=13870 infer_wait=112.329s infer_obs_pack=0.254s env_step=4.251s feedback_total=1.682s feedback_obs_pack=0.391s feedback_info_pack=0.082s effective_fps=468.13 +2026-09-25 19:37:31.048 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=69637 infer_calls=17410 feedback_calls=17409 infer_wait=142.740s infer_obs_pack=0.316s env_step=5.316s feedback_total=2.125s feedback_obs_pack=0.491s feedback_info_pack=0.106s effective_fps=462.71 +2026-09-25 19:38:01.058 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=83921 infer_calls=20981 feedback_calls=20980 infer_wait=170.763s infer_obs_pack=0.380s env_step=6.411s feedback_total=2.569s feedback_obs_pack=0.591s feedback_info_pack=0.127s effective_fps=465.91 +2026-09-25 19:38:33.971 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=98309 infer_calls=24578 feedback_calls=24577 infer_wait=201.678s infer_obs_pack=0.444s env_step=7.512s feedback_total=3.022s feedback_obs_pack=0.693s feedback_info_pack=0.149s effective_fps=462.29 +2026-09-25 19:39:03.975 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=112641 infer_calls=28161 feedback_calls=28160 infer_wait=229.642s infer_obs_pack=0.508s env_step=8.630s feedback_total=3.493s feedback_obs_pack=0.796s feedback_info_pack=0.170s effective_fps=464.94 +2026-09-25 19:39:33.979 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=126605 infer_calls=31652 feedback_calls=31651 infer_wait=257.660s infer_obs_pack=0.571s env_step=9.736s feedback_total=3.927s feedback_obs_pack=0.896s feedback_info_pack=0.191s effective_fps=465.64 +2026-09-25 19:40:05.022 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=139265 infer_calls=34817 feedback_calls=34816 infer_wait=286.881s infer_obs_pack=0.629s env_step=10.758s feedback_total=4.319s feedback_obs_pack=0.987s feedback_info_pack=0.211s effective_fps=460.25 +2026-09-25 19:40:35.022 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=153133 infer_calls=38284 feedback_calls=38283 infer_wait=314.870s infer_obs_pack=0.694s env_step=11.894s feedback_total=4.743s feedback_obs_pack=1.090s feedback_info_pack=0.232s effective_fps=460.97 +2026-09-25 19:41:05.024 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=167185 infer_calls=41797 feedback_calls=41796 infer_wait=342.879s infer_obs_pack=0.757s env_step=12.999s feedback_total=5.188s feedback_obs_pack=1.192s feedback_info_pack=0.254s effective_fps=462.06 +2026-09-25 19:41:36.446 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=180229 infer_calls=45058 feedback_calls=45057 infer_wait=372.478s infer_obs_pack=0.815s env_step=14.008s feedback_total=5.595s feedback_obs_pack=1.284s feedback_info_pack=0.273s effective_fps=458.72 +2026-09-25 19:42:06.446 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=194389 infer_calls=48598 feedback_calls=48597 infer_wait=400.498s infer_obs_pack=0.878s env_step=15.107s feedback_total=6.027s feedback_obs_pack=1.386s feedback_info_pack=0.294s effective_fps=460.08 +2026-09-25 19:42:39.421 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=208901 infer_calls=52226 feedback_calls=52225 infer_wait=431.461s infer_obs_pack=0.943s env_step=16.221s feedback_total=6.474s feedback_obs_pack=1.489s feedback_info_pack=0.315s effective_fps=459.02 +2026-09-25 19:43:09.423 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=223489 infer_calls=55873 feedback_calls=55872 infer_wait=459.443s infer_obs_pack=1.008s env_step=17.328s feedback_total=6.933s feedback_obs_pack=1.591s feedback_info_pack=0.337s effective_fps=461.08 +2026-09-25 19:43:41.971 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=237573 infer_calls=59394 feedback_calls=59393 infer_wait=490.058s infer_obs_pack=1.070s env_step=18.398s feedback_total=7.362s feedback_obs_pack=1.690s feedback_info_pack=0.358s effective_fps=459.62 +2026-09-25 19:44:11.972 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=252105 infer_calls=63027 feedback_calls=63026 infer_wait=518.034s infer_obs_pack=1.135s env_step=19.499s feedback_total=7.834s feedback_obs_pack=1.792s feedback_info_pack=0.379s effective_fps=461.31 +2026-09-25 19:44:44.289 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=266245 infer_calls=66562 feedback_calls=66561 infer_wait=548.420s infer_obs_pack=1.197s env_step=20.559s feedback_total=8.273s feedback_obs_pack=1.890s feedback_info_pack=0.400s effective_fps=460.27 +2026-09-25 19:45:14.294 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=280777 infer_calls=70195 feedback_calls=70194 infer_wait=576.404s infer_obs_pack=1.262s env_step=21.669s feedback_total=8.735s feedback_obs_pack=1.992s feedback_info_pack=0.421s effective_fps=461.75 +2026-09-25 19:45:46.882 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=294917 infer_calls=73730 feedback_calls=73729 infer_wait=607.034s infer_obs_pack=1.324s env_step=22.748s feedback_total=9.173s feedback_obs_pack=2.092s feedback_info_pack=0.442s effective_fps=460.61 +2026-09-25 19:46:16.887 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=309113 infer_calls=77279 feedback_calls=77278 infer_wait=635.076s infer_obs_pack=1.388s env_step=23.833s feedback_total=9.608s feedback_obs_pack=2.192s feedback_info_pack=0.463s effective_fps=461.43 +2026-09-25 19:46:46.888 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=323209 infer_calls=80803 feedback_calls=80802 infer_wait=663.149s infer_obs_pack=1.452s env_step=24.908s feedback_total=10.020s feedback_obs_pack=2.292s feedback_info_pack=0.483s effective_fps=462.04 +2026-09-25 19:47:18.070 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=335877 infer_calls=83970 feedback_calls=83969 infer_wait=692.594s infer_obs_pack=1.509s env_step=25.865s feedback_total=10.408s feedback_obs_pack=2.381s feedback_info_pack=0.501s effective_fps=459.87 +2026-09-25 19:47:48.071 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=349953 infer_calls=87489 feedback_calls=87488 infer_wait=720.650s infer_obs_pack=1.571s env_step=26.932s feedback_total=10.850s feedback_obs_pack=2.481s feedback_info_pack=0.522s effective_fps=460.46 +2026-09-25 19:48:18.074 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=363925 infer_calls=90982 feedback_calls=90981 infer_wait=748.770s infer_obs_pack=1.632s env_step=27.978s feedback_total=11.261s feedback_obs_pack=2.579s feedback_info_pack=0.542s effective_fps=460.87 +2026-09-25 19:48:48.891 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=376837 infer_calls=94210 feedback_calls=94209 infer_wait=777.826s infer_obs_pack=1.689s env_step=28.949s feedback_total=11.654s feedback_obs_pack=2.669s feedback_info_pack=0.561s effective_fps=459.49 +2026-09-25 19:49:18.891 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=391013 infer_calls=97754 feedback_calls=97753 infer_wait=805.866s infer_obs_pack=1.753s env_step=30.032s feedback_total=12.087s feedback_obs_pack=2.769s feedback_info_pack=0.582s effective_fps=460.16 +2026-09-25 19:49:51.060 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=405509 infer_calls=101378 feedback_calls=101377 infer_wait=836.155s infer_obs_pack=1.813s env_step=31.070s feedback_total=12.501s feedback_obs_pack=2.865s feedback_info_pack=0.602s effective_fps=460.00 +2026-09-25 19:49:59.059 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=409604 infer_calls=102401 feedback_calls=102401 infer_wait=840.791s infer_obs_pack=1.828s env_step=31.324s feedback_total=12.591s feedback_obs_pack=2.887s feedback_info_pack=0.606s effective_fps=462.03 +2026-09-25 19:49:59.059 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 409/409600 episodes. diff --git a/experiments/e28-dppo-learns/results/dppo-cheetah/client-seed2.log b/experiments/e28-dppo-learns/results/dppo-cheetah/client-seed2.log new file mode 100644 index 0000000..61cf274 --- /dev/null +++ b/experiments/e28-dppo-learns/results/dppo-cheetah/client-seed2.log @@ -0,0 +1,39 @@ +2026-09-25 19:35:03.602 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 19:35:03.603 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 19:35:03.603 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 19:35:03.603 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 19:35:03.604 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 19:35:03.650 | INFO | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-193503-617d03e7 output_dir=runs/mujoco-v1-nenv1-20260925-193503-617d03e7 recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 19:35:03.697 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9522... +2026-09-25 19:35:03.700 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'DPPOPolicy', 'action_dim': 6, 'action_horizon': 4} +2026-09-25 19:35:33.703 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=14236 infer_calls=3559 feedback_calls=3559 infer_wait=28.033s infer_obs_pack=0.062s env_step=1.091s feedback_total=0.439s feedback_obs_pack=0.096s feedback_info_pack=0.020s effective_fps=480.53 +2026-09-25 19:36:03.703 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=28565 infer_calls=7142 feedback_calls=7141 infer_wait=56.022s infer_obs_pack=0.126s env_step=2.208s feedback_total=0.883s feedback_obs_pack=0.195s feedback_info_pack=0.041s effective_fps=482.20 +2026-09-25 19:36:33.705 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=41021 infer_calls=10256 feedback_calls=10255 infer_wait=84.250s infer_obs_pack=0.181s env_step=3.186s feedback_total=1.286s feedback_obs_pack=0.281s feedback_info_pack=0.059s effective_fps=461.41 +2026-09-25 19:37:03.714 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=55537 infer_calls=13885 feedback_calls=13884 infer_wait=112.231s infer_obs_pack=0.261s env_step=4.298s feedback_total=1.740s feedback_obs_pack=0.380s feedback_info_pack=0.079s effective_fps=468.55 +2026-09-25 19:37:35.866 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=69637 infer_calls=17410 feedback_calls=17409 infer_wait=142.444s infer_obs_pack=0.322s env_step=5.382s feedback_total=2.164s feedback_obs_pack=0.475s feedback_info_pack=0.099s effective_fps=463.28 +2026-09-25 19:38:05.875 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=84237 infer_calls=21060 feedback_calls=21059 infer_wait=170.468s infer_obs_pack=0.385s env_step=6.486s feedback_total=2.602s feedback_obs_pack=0.573s feedback_info_pack=0.119s effective_fps=468.14 +2026-09-25 19:38:38.690 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=98309 infer_calls=24578 feedback_calls=24577 infer_wait=201.253s infer_obs_pack=0.449s env_step=7.620s feedback_total=3.047s feedback_obs_pack=0.672s feedback_info_pack=0.140s effective_fps=462.91 +2026-09-25 19:39:08.693 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=112213 infer_calls=28054 feedback_calls=28053 infer_wait=229.286s infer_obs_pack=0.512s env_step=8.714s feedback_total=3.485s feedback_obs_pack=0.769s feedback_info_pack=0.161s effective_fps=463.70 +2026-09-25 19:39:38.698 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=126357 infer_calls=31590 feedback_calls=31589 infer_wait=257.260s infer_obs_pack=0.580s env_step=9.849s feedback_total=3.918s feedback_obs_pack=0.868s feedback_info_pack=0.182s effective_fps=465.22 +2026-09-25 19:40:09.366 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=139269 infer_calls=34818 feedback_calls=34817 infer_wait=286.140s infer_obs_pack=0.637s env_step=10.839s feedback_total=4.317s feedback_obs_pack=0.959s feedback_info_pack=0.201s effective_fps=461.26 +2026-09-25 19:40:39.371 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=153657 infer_calls=38415 feedback_calls=38414 infer_wait=314.131s infer_obs_pack=0.703s env_step=11.955s feedback_total=4.766s feedback_obs_pack=1.061s feedback_info_pack=0.221s effective_fps=463.44 +2026-09-25 19:41:09.375 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=167753 infer_calls=41939 feedback_calls=41938 infer_wait=342.143s infer_obs_pack=0.766s env_step=13.075s feedback_total=5.193s feedback_obs_pack=1.160s feedback_info_pack=0.242s effective_fps=464.46 +2026-09-25 19:41:39.481 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=180229 infer_calls=45058 feedback_calls=45057 infer_wait=370.471s infer_obs_pack=0.823s env_step=14.057s feedback_total=5.597s feedback_obs_pack=1.247s feedback_info_pack=0.260s effective_fps=461.01 +2026-09-25 19:42:09.481 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=194417 infer_calls=48605 feedback_calls=48604 infer_wait=398.467s infer_obs_pack=0.890s env_step=15.167s feedback_total=6.042s feedback_obs_pack=1.349s feedback_info_pack=0.281s effective_fps=462.27 +2026-09-25 19:42:39.483 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=208721 infer_calls=52181 feedback_calls=52180 infer_wait=426.438s infer_obs_pack=0.957s env_step=16.305s feedback_total=6.473s feedback_obs_pack=1.450s feedback_info_pack=0.301s effective_fps=463.65 +2026-09-25 19:43:09.981 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=221189 infer_calls=55298 feedback_calls=55297 infer_wait=455.194s infer_obs_pack=1.014s env_step=17.281s feedback_total=6.845s feedback_obs_pack=1.536s feedback_info_pack=0.319s effective_fps=460.49 +2026-09-25 19:43:39.985 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=236133 infer_calls=59034 feedback_calls=59033 infer_wait=483.155s infer_obs_pack=1.080s env_step=18.415s feedback_total=7.295s feedback_obs_pack=1.636s feedback_info_pack=0.339s effective_fps=463.06 +2026-09-25 19:44:11.610 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=249861 infer_calls=62466 feedback_calls=62465 infer_wait=512.885s infer_obs_pack=1.140s env_step=19.467s feedback_total=7.717s feedback_obs_pack=1.729s feedback_info_pack=0.359s effective_fps=461.67 +2026-09-25 19:44:41.610 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=264669 infer_calls=66168 feedback_calls=66167 infer_wait=540.862s infer_obs_pack=1.205s env_step=20.584s feedback_total=8.173s feedback_obs_pack=1.829s feedback_info_pack=0.379s effective_fps=463.66 +2026-09-25 19:45:13.670 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=278533 infer_calls=69634 feedback_calls=69633 infer_wait=571.031s infer_obs_pack=1.265s env_step=21.634s feedback_total=8.587s feedback_obs_pack=1.922s feedback_info_pack=0.399s effective_fps=462.28 +2026-09-25 19:45:43.674 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=293249 infer_calls=73313 feedback_calls=73312 infer_wait=599.050s infer_obs_pack=1.330s env_step=22.734s feedback_total=9.026s feedback_obs_pack=2.021s feedback_info_pack=0.419s effective_fps=463.90 +2026-09-25 19:46:15.570 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=307205 infer_calls=76802 feedback_calls=76801 infer_wait=629.032s infer_obs_pack=1.392s env_step=23.788s feedback_total=9.461s feedback_obs_pack=2.116s feedback_info_pack=0.439s effective_fps=462.89 +2026-09-25 19:46:45.574 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=321833 infer_calls=80459 feedback_calls=80458 infer_wait=657.050s infer_obs_pack=1.456s env_step=24.879s feedback_total=9.916s feedback_obs_pack=2.213s feedback_info_pack=0.459s effective_fps=464.20 +2026-09-25 19:47:17.299 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=335873 infer_calls=83969 feedback_calls=83968 infer_wait=686.944s infer_obs_pack=1.515s env_step=25.881s feedback_total=10.341s feedback_obs_pack=2.303s feedback_info_pack=0.478s effective_fps=463.48 +2026-09-25 19:47:47.302 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=350033 infer_calls=87509 feedback_calls=87508 infer_wait=715.001s infer_obs_pack=1.578s env_step=26.966s feedback_total=10.763s feedback_obs_pack=2.401s feedback_info_pack=0.498s effective_fps=464.05 +2026-09-25 19:48:17.307 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=364273 infer_calls=91069 feedback_calls=91068 infer_wait=743.013s infer_obs_pack=1.643s env_step=28.073s feedback_total=11.197s feedback_obs_pack=2.500s feedback_info_pack=0.518s effective_fps=464.68 +2026-09-25 19:48:47.364 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=376837 infer_calls=94210 feedback_calls=94209 infer_wait=771.312s infer_obs_pack=1.699s env_step=29.041s feedback_total=11.597s feedback_obs_pack=2.586s feedback_info_pack=0.536s effective_fps=463.14 +2026-09-25 19:49:17.369 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=391553 infer_calls=97889 feedback_calls=97888 infer_wait=799.289s infer_obs_pack=1.765s env_step=30.162s feedback_total=12.054s feedback_obs_pack=2.685s feedback_info_pack=0.557s effective_fps=464.33 +2026-09-25 19:49:48.730 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=405509 infer_calls=101378 feedback_calls=101377 infer_wait=828.837s infer_obs_pack=1.824s env_step=31.175s feedback_total=12.444s feedback_obs_pack=2.775s feedback_info_pack=0.575s effective_fps=463.82 +2026-09-25 19:49:56.686 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=409604 infer_calls=102401 feedback_calls=102401 infer_wait=833.419s infer_obs_pack=1.840s env_step=31.442s feedback_total=12.542s feedback_obs_pack=2.798s feedback_info_pack=0.580s effective_fps=465.86 +2026-09-25 19:49:56.686 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 409/409600 episodes. diff --git a/experiments/e28-dppo-learns/results/dppo-cheetah/server-seed0.log b/experiments/e28-dppo-learns/results/dppo-cheetah/server-seed0.log new file mode 100644 index 0000000..5ab45ff --- /dev/null +++ b/experiments/e28-dppo-learns/results/dppo-cheetah/server-seed0.log @@ -0,0 +1,26 @@ +19:34:52|INFO|plugrl_server version: 0.1.0 +19:34:52|INFO|Algorithm: dppo, Config: DPPOAlgoConfigCheetah(global_steps=102400, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=1024, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=512, train_itrs=100, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +19:34:52|INFO|Policy: dppo-policy, Config: DPPOPolicyConfigCheetah(algo='dppo', device=device(type='cpu'), env_type='gym', env_name='halfcheetah-medium-v2', checkpoint_path=None, critic=DPPOCriticObsConfig(mlp_dims=[256, 256, 256], activation='Mish', residual_style=True)) +19:34:52|INFO|Seeded python, numpy and torch with 0. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +19:34:52|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-cheetah/dppo/dppo-policy/dppo-cheetah-seed0 +19:34:52|INFO|Number of network parameters: 568392 +19:34:52|INFO|Policy created... +19:34:52|INFO|Initialized RolloutBuffer buffer_size=1024 action_shape=(1024, 20, 4, 6) value_shape=(1024,) +19:34:52|INFO|Algorithm created: + +19:34:52|INFO|Agent Server is listening on 0.0.0.0:9520 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +19:37:49|INFO|Checkpoint saved at step 20481 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-cheetah/dppo/dppo-policy/dppo-cheetah-seed0/20481 +19:40:52|INFO|Checkpoint saved at step 40961 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-cheetah/dppo/dppo-policy/dppo-cheetah-seed0/40961 +19:43:52|INFO|Checkpoint saved at step 61441 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-cheetah/dppo/dppo-policy/dppo-cheetah-seed0/61441 +19:46:52|INFO|Checkpoint saved at step 81921 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-cheetah/dppo/dppo-policy/dppo-cheetah-seed0/81921 +19:49:50|INFO|Checkpoint saved at step 102401 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-cheetah/dppo/dppo-policy/dppo-cheetah-seed0/102401 +19:49:50|INFO|Stopping server as the algorithm signaled to stop. +19:49:50|INFO|Shutdown started: aborting pending infer requests. +19:49:50|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +19:49:50|INFO|Shutdown closing 1 websocket connection(s). +19:49:50|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 53196). +19:49:50|INFO|WebSocket server closed. +19:49:50|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e28-dppo-learns/results/dppo-cheetah/server-seed1.log b/experiments/e28-dppo-learns/results/dppo-cheetah/server-seed1.log new file mode 100644 index 0000000..90480c7 --- /dev/null +++ b/experiments/e28-dppo-learns/results/dppo-cheetah/server-seed1.log @@ -0,0 +1,26 @@ +19:34:57|INFO|plugrl_server version: 0.1.0 +19:34:57|INFO|Algorithm: dppo, Config: DPPOAlgoConfigCheetah(global_steps=102400, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=1024, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=512, train_itrs=100, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +19:34:57|INFO|Policy: dppo-policy, Config: DPPOPolicyConfigCheetah(algo='dppo', device=device(type='cpu'), env_type='gym', env_name='halfcheetah-medium-v2', checkpoint_path=None, critic=DPPOCriticObsConfig(mlp_dims=[256, 256, 256], activation='Mish', residual_style=True)) +19:34:57|INFO|Seeded python, numpy and torch with 1. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +19:34:57|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-cheetah/dppo/dppo-policy/dppo-cheetah-seed1 +19:34:57|INFO|Number of network parameters: 568392 +19:34:57|INFO|Policy created... +19:34:57|INFO|Initialized RolloutBuffer buffer_size=1024 action_shape=(1024, 20, 4, 6) value_shape=(1024,) +19:34:57|INFO|Algorithm created: + +19:34:57|INFO|Agent Server is listening on 0.0.0.0:9521 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +19:37:58|INFO|Checkpoint saved at step 20481 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-cheetah/dppo/dppo-policy/dppo-cheetah-seed1/20481 +19:40:59|INFO|Checkpoint saved at step 40961 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-cheetah/dppo/dppo-policy/dppo-cheetah-seed1/40961 +19:43:59|INFO|Checkpoint saved at step 61441 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-cheetah/dppo/dppo-policy/dppo-cheetah-seed1/61441 +19:46:59|INFO|Checkpoint saved at step 81921 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-cheetah/dppo/dppo-policy/dppo-cheetah-seed1/81921 +19:49:59|INFO|Checkpoint saved at step 102401 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-cheetah/dppo/dppo-policy/dppo-cheetah-seed1/102401 +19:49:59|INFO|Stopping server as the algorithm signaled to stop. +19:49:59|INFO|Shutdown started: aborting pending infer requests. +19:49:59|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +19:49:59|INFO|Shutdown closing 1 websocket connection(s). +19:49:59|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 53350). +19:49:59|INFO|WebSocket server closed. +19:49:59|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e28-dppo-learns/results/dppo-cheetah/server-seed2.log b/experiments/e28-dppo-learns/results/dppo-cheetah/server-seed2.log new file mode 100644 index 0000000..f4aae20 --- /dev/null +++ b/experiments/e28-dppo-learns/results/dppo-cheetah/server-seed2.log @@ -0,0 +1,26 @@ +19:35:02|INFO|plugrl_server version: 0.1.0 +19:35:02|INFO|Algorithm: dppo, Config: DPPOAlgoConfigCheetah(global_steps=102400, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=1024, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=512, train_itrs=100, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +19:35:02|INFO|Policy: dppo-policy, Config: DPPOPolicyConfigCheetah(algo='dppo', device=device(type='cpu'), env_type='gym', env_name='halfcheetah-medium-v2', checkpoint_path=None, critic=DPPOCriticObsConfig(mlp_dims=[256, 256, 256], activation='Mish', residual_style=True)) +19:35:02|INFO|Seeded python, numpy and torch with 2. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +19:35:02|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-cheetah/dppo/dppo-policy/dppo-cheetah-seed2 +19:35:02|INFO|Number of network parameters: 568392 +19:35:02|INFO|Policy created... +19:35:02|INFO|Initialized RolloutBuffer buffer_size=1024 action_shape=(1024, 20, 4, 6) value_shape=(1024,) +19:35:02|INFO|Algorithm created: + +19:35:02|INFO|Agent Server is listening on 0.0.0.0:9522 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +19:38:02|INFO|Checkpoint saved at step 20481 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-cheetah/dppo/dppo-policy/dppo-cheetah-seed2/20481 +19:41:03|INFO|Checkpoint saved at step 40961 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-cheetah/dppo/dppo-policy/dppo-cheetah-seed2/40961 +19:44:02|INFO|Checkpoint saved at step 61441 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-cheetah/dppo/dppo-policy/dppo-cheetah-seed2/61441 +19:46:59|INFO|Checkpoint saved at step 81921 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-cheetah/dppo/dppo-policy/dppo-cheetah-seed2/81921 +19:49:56|INFO|Checkpoint saved at step 102401 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-cheetah/dppo/dppo-policy/dppo-cheetah-seed2/102401 +19:49:56|INFO|Stopping server as the algorithm signaled to stop. +19:49:56|INFO|Shutdown started: aborting pending infer requests. +19:49:56|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +19:49:56|INFO|Shutdown closing 1 websocket connection(s). +19:49:56|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 37810). +19:49:56|INFO|WebSocket server closed. +19:49:56|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e28-dppo-learns/results/dppo-hopper.out b/experiments/e28-dppo-learns/results/dppo-hopper.out new file mode 100644 index 0000000..9309585 --- /dev/null +++ b/experiments/e28-dppo-learns/results/dppo-hopper.out @@ -0,0 +1,10 @@ +cell: dppo-hopper = dppo-policy/hopper x dppo/hopper x Hopper-v5 +iters: 100 x 1024 batch: 512 replan: 4 seeds: 0 1 2 +out: /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-hopper +start 2026-09-25 19:34:10 +seed 0 finished rc=0 at 19:49:35 +seed 1 finished rc=0 at 19:49:37 +seed 2 finished rc=0 at 19:49:44 +end 2026-09-25 19:49:44 +failed seeds: 0 +CELL_DONE dppo-hopper diff --git a/experiments/e28-dppo-learns/results/dppo-hopper/client-seed0.log b/experiments/e28-dppo-learns/results/dppo-hopper/client-seed0.log new file mode 100644 index 0000000..460b511 --- /dev/null +++ b/experiments/e28-dppo-learns/results/dppo-hopper/client-seed0.log @@ -0,0 +1,40 @@ +2026-09-25 19:34:12.575 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 19:34:12.575 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 19:34:12.576 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 19:34:12.576 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 19:34:12.577 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 19:34:12.614 | INFO | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-193412-09150fe0 output_dir=runs/mujoco-v1-nenv1-20260925-193412-09150fe0 recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 19:34:12.647 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9500... +2026-09-25 19:34:12.650 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'DPPOPolicy', 'action_dim': 3, 'action_horizon': 4} +2026-09-25 19:34:45.362 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=15191 infer_calls=4098 feedback_calls=4097 infer_wait=30.454s infer_obs_pack=0.055s env_step=1.490s feedback_total=0.307s feedback_obs_pack=0.078s feedback_info_pack=0.019s effective_fps=470.24 +2026-09-25 19:35:15.363 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=29475 infer_calls=7821 feedback_calls=7820 infer_wait=57.859s infer_obs_pack=0.114s env_step=3.212s feedback_total=0.703s feedback_obs_pack=0.174s feedback_info_pack=0.043s effective_fps=476.26 +2026-09-25 19:35:48.002 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=42847 infer_calls=11265 feedback_calls=11264 infer_wait=87.920s infer_obs_pack=0.172s env_step=4.926s feedback_total=1.114s feedback_obs_pack=0.269s feedback_info_pack=0.067s effective_fps=455.18 +2026-09-25 19:36:18.003 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=56555 infer_calls=14789 feedback_calls=14788 infer_wait=115.264s infer_obs_pack=0.244s env_step=6.691s feedback_total=1.528s feedback_obs_pack=0.369s feedback_info_pack=0.092s effective_fps=457.09 +2026-09-25 19:36:48.007 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=69911 infer_calls=18229 feedback_calls=18229 infer_wait=142.654s infer_obs_pack=0.304s env_step=8.432s feedback_total=1.938s feedback_obs_pack=0.470s feedback_info_pack=0.116s effective_fps=455.96 +2026-09-25 19:37:20.185 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=82663 infer_calls=21506 feedback_calls=21505 infer_wait=172.331s infer_obs_pack=0.363s env_step=10.105s feedback_total=2.324s feedback_obs_pack=0.562s feedback_info_pack=0.139s effective_fps=446.53 +2026-09-25 19:37:50.190 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=95981 infer_calls=24922 feedback_calls=24921 infer_wait=199.765s infer_obs_pack=0.422s env_step=11.824s feedback_total=2.721s feedback_obs_pack=0.657s feedback_info_pack=0.163s effective_fps=446.98 +2026-09-25 19:38:20.192 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=109032 infer_calls=28266 feedback_calls=28265 infer_wait=227.199s infer_obs_pack=0.483s env_step=13.549s feedback_total=3.109s feedback_obs_pack=0.751s feedback_info_pack=0.186s effective_fps=446.23 +2026-09-25 19:38:50.194 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=121920 infer_calls=31565 feedback_calls=31564 infer_wait=254.693s infer_obs_pack=0.539s env_step=15.234s feedback_total=3.498s feedback_obs_pack=0.846s feedback_info_pack=0.209s effective_fps=445.02 +2026-09-25 19:39:22.416 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=134667 infer_calls=34818 feedback_calls=34817 infer_wait=284.461s infer_obs_pack=0.596s env_step=16.886s feedback_total=3.878s feedback_obs_pack=0.940s feedback_info_pack=0.232s effective_fps=440.35 +2026-09-25 19:39:52.421 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=147878 infer_calls=38184 feedback_calls=38183 infer_wait=311.952s infer_obs_pack=0.653s env_step=18.583s feedback_total=4.262s feedback_obs_pack=1.034s feedback_info_pack=0.256s effective_fps=440.83 +2026-09-25 19:40:22.421 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=161372 infer_calls=41611 feedback_calls=41611 infer_wait=339.348s infer_obs_pack=0.713s env_step=20.338s feedback_total=4.670s feedback_obs_pack=1.131s feedback_info_pack=0.279s effective_fps=442.03 +2026-09-25 19:40:52.422 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=174679 infer_calls=44989 feedback_calls=44989 infer_wait=366.774s infer_obs_pack=0.771s env_step=22.080s feedback_total=5.064s feedback_obs_pack=1.230s feedback_info_pack=0.303s effective_fps=442.57 +2026-09-25 19:41:23.950 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=187066 infer_calls=48130 feedback_calls=48129 infer_wait=395.882s infer_obs_pack=0.826s env_step=23.704s feedback_total=5.448s feedback_obs_pack=1.321s feedback_info_pack=0.327s effective_fps=439.27 +2026-09-25 19:41:53.951 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=199956 infer_calls=51394 feedback_calls=51393 infer_wait=423.382s infer_obs_pack=0.883s env_step=25.399s feedback_total=5.827s feedback_obs_pack=1.417s feedback_info_pack=0.350s effective_fps=438.99 +2026-09-25 19:42:23.951 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=213738 infer_calls=54889 feedback_calls=54888 infer_wait=450.832s infer_obs_pack=0.941s env_step=27.101s feedback_total=6.239s feedback_obs_pack=1.512s feedback_info_pack=0.374s effective_fps=440.59 +2026-09-25 19:42:53.955 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=227283 infer_calls=58318 feedback_calls=58317 infer_wait=478.284s infer_obs_pack=1.001s env_step=28.813s feedback_total=6.643s feedback_obs_pack=1.607s feedback_info_pack=0.397s effective_fps=441.55 +2026-09-25 19:43:24.743 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=239612 infer_calls=61442 feedback_calls=61441 infer_wait=506.739s infer_obs_pack=1.055s env_step=30.374s feedback_total=7.015s feedback_obs_pack=1.693s feedback_info_pack=0.418s effective_fps=439.51 +2026-09-25 19:43:54.744 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=252966 infer_calls=64823 feedback_calls=64822 infer_wait=534.155s infer_obs_pack=1.118s env_step=32.121s feedback_total=7.408s feedback_obs_pack=1.789s feedback_info_pack=0.442s effective_fps=440.09 +2026-09-25 19:44:24.746 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=266815 infer_calls=68325 feedback_calls=68324 infer_wait=561.552s infer_obs_pack=1.178s env_step=33.866s feedback_total=7.825s feedback_obs_pack=1.886s feedback_info_pack=0.466s effective_fps=441.44 +2026-09-25 19:44:56.868 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=280110 infer_calls=71682 feedback_calls=71681 infer_wait=591.194s infer_obs_pack=1.234s env_step=35.535s feedback_total=8.215s feedback_obs_pack=1.977s feedback_info_pack=0.488s effective_fps=440.30 +2026-09-25 19:45:26.869 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=293943 infer_calls=75176 feedback_calls=75175 infer_wait=618.612s infer_obs_pack=1.295s env_step=37.272s feedback_total=8.623s feedback_obs_pack=2.071s feedback_info_pack=0.511s effective_fps=441.49 +2026-09-25 19:45:56.873 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=307741 infer_calls=78658 feedback_calls=78657 infer_wait=646.010s infer_obs_pack=1.353s env_step=39.037s feedback_total=9.027s feedback_obs_pack=2.167s feedback_info_pack=0.535s effective_fps=442.52 +2026-09-25 19:46:28.937 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=320650 infer_calls=81922 feedback_calls=81921 infer_wait=675.591s infer_obs_pack=1.410s env_step=40.711s feedback_total=9.415s feedback_obs_pack=2.261s feedback_info_pack=0.557s effective_fps=440.98 +2026-09-25 19:46:58.939 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=333708 infer_calls=85219 feedback_calls=85218 infer_wait=703.032s infer_obs_pack=1.469s env_step=42.445s feedback_total=9.811s feedback_obs_pack=2.355s feedback_info_pack=0.580s effective_fps=440.97 +2026-09-25 19:47:28.939 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=347096 infer_calls=88600 feedback_calls=88599 infer_wait=730.420s infer_obs_pack=1.530s env_step=44.217s feedback_total=10.210s feedback_obs_pack=2.452s feedback_info_pack=0.605s effective_fps=441.39 +2026-09-25 19:47:58.943 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=360484 infer_calls=91977 feedback_calls=91976 infer_wait=757.872s infer_obs_pack=1.592s env_step=45.946s feedback_total=10.601s feedback_obs_pack=2.547s feedback_info_pack=0.628s effective_fps=441.76 +2026-09-25 19:48:30.620 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=373399 infer_calls=95234 feedback_calls=95233 infer_wait=787.141s infer_obs_pack=1.647s env_step=47.569s feedback_total=10.985s feedback_obs_pack=2.635s feedback_info_pack=0.651s effective_fps=440.67 +2026-09-25 19:49:00.629 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=387276 infer_calls=98733 feedback_calls=98732 infer_wait=814.544s infer_obs_pack=1.708s env_step=49.339s feedback_total=11.381s feedback_obs_pack=2.731s feedback_info_pack=0.675s effective_fps=441.61 +2026-09-25 19:49:30.630 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=400852 infer_calls=102151 feedback_calls=102150 infer_wait=841.958s infer_obs_pack=1.773s env_step=51.097s feedback_total=11.773s feedback_obs_pack=2.825s feedback_info_pack=0.699s effective_fps=442.15 +2026-09-25 19:49:35.072 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=401850 infer_calls=102401 feedback_calls=102401 infer_wait=843.312s infer_obs_pack=1.777s env_step=51.229s feedback_total=11.804s feedback_obs_pack=2.832s feedback_info_pack=0.701s effective_fps=442.51 +2026-09-25 19:49:35.073 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 5088/409600 episodes. diff --git a/experiments/e28-dppo-learns/results/dppo-hopper/client-seed1.log b/experiments/e28-dppo-learns/results/dppo-hopper/client-seed1.log new file mode 100644 index 0000000..d02a089 --- /dev/null +++ b/experiments/e28-dppo-learns/results/dppo-hopper/client-seed1.log @@ -0,0 +1,40 @@ +2026-09-25 19:34:17.571 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 19:34:17.571 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 19:34:17.572 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 19:34:17.572 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 19:34:17.572 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 19:34:17.610 | INFO | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-193417-728799c1 output_dir=runs/mujoco-v1-nenv1-20260925-193417-728799c1 recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 19:34:17.646 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9501... +2026-09-25 19:34:17.648 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'DPPOPolicy', 'action_dim': 3, 'action_horizon': 4} +2026-09-25 19:34:50.272 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=15172 infer_calls=4098 feedback_calls=4097 infer_wait=30.290s infer_obs_pack=0.056s env_step=1.524s feedback_total=0.324s feedback_obs_pack=0.079s feedback_info_pack=0.019s effective_fps=471.26 +2026-09-25 19:35:20.272 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=29012 infer_calls=7712 feedback_calls=7711 infer_wait=57.746s infer_obs_pack=0.115s env_step=3.213s feedback_total=0.716s feedback_obs_pack=0.172s feedback_info_pack=0.040s effective_fps=469.53 +2026-09-25 19:35:50.273 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=42529 infer_calls=11203 feedback_calls=11202 infer_wait=85.157s infer_obs_pack=0.174s env_step=4.931s feedback_total=1.128s feedback_obs_pack=0.268s feedback_info_pack=0.061s effective_fps=465.36 +2026-09-25 19:36:20.966 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=54675 infer_calls=14338 feedback_calls=14337 infer_wait=113.488s infer_obs_pack=0.246s env_step=6.487s feedback_total=1.499s feedback_obs_pack=0.354s feedback_info_pack=0.079s effective_fps=449.19 +2026-09-25 19:36:50.966 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=68024 infer_calls=17772 feedback_calls=17771 infer_wait=140.932s infer_obs_pack=0.304s env_step=8.204s feedback_total=1.890s feedback_obs_pack=0.449s feedback_info_pack=0.100s effective_fps=449.51 +2026-09-25 19:37:20.973 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=81542 infer_calls=21243 feedback_calls=21242 infer_wait=168.338s infer_obs_pack=0.363s env_step=9.942s feedback_total=2.296s feedback_obs_pack=0.546s feedback_info_pack=0.121s effective_fps=450.66 +2026-09-25 19:37:53.487 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=94536 infer_calls=24578 feedback_calls=24577 infer_wait=198.316s infer_obs_pack=0.420s env_step=11.646s feedback_total=2.689s feedback_obs_pack=0.639s feedback_info_pack=0.142s effective_fps=443.68 +2026-09-25 19:38:23.494 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=107753 infer_calls=27956 feedback_calls=27955 infer_wait=225.732s infer_obs_pack=0.482s env_step=13.388s feedback_total=3.089s feedback_obs_pack=0.733s feedback_info_pack=0.162s effective_fps=443.99 +2026-09-25 19:38:53.494 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=120960 infer_calls=31328 feedback_calls=31327 infer_wait=253.179s infer_obs_pack=0.539s env_step=15.102s feedback_total=3.484s feedback_obs_pack=0.828s feedback_info_pack=0.183s effective_fps=444.21 +2026-09-25 19:39:23.495 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=134284 infer_calls=34718 feedback_calls=34717 infer_wait=280.578s infer_obs_pack=0.598s env_step=16.854s feedback_total=3.886s feedback_obs_pack=0.924s feedback_info_pack=0.204s effective_fps=444.77 +2026-09-25 19:39:55.134 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=146737 infer_calls=37890 feedback_calls=37889 infer_wait=309.788s infer_obs_pack=0.654s env_step=18.485s feedback_total=4.266s feedback_obs_pack=1.013s feedback_info_pack=0.223s effective_fps=440.40 +2026-09-25 19:40:25.134 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=160093 infer_calls=41283 feedback_calls=41282 infer_wait=337.175s infer_obs_pack=0.714s env_step=20.250s feedback_total=4.673s feedback_obs_pack=1.108s feedback_info_pack=0.244s effective_fps=441.26 +2026-09-25 19:40:55.138 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=173503 infer_calls=44684 feedback_calls=44683 infer_wait=364.533s infer_obs_pack=0.779s env_step=22.036s feedback_total=5.081s feedback_obs_pack=1.206s feedback_info_pack=0.266s effective_fps=442.13 +2026-09-25 19:41:25.140 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=186691 infer_calls=48025 feedback_calls=48024 infer_wait=392.018s infer_obs_pack=0.838s env_step=23.741s feedback_total=5.468s feedback_obs_pack=1.298s feedback_info_pack=0.285s effective_fps=442.33 +2026-09-25 19:41:56.950 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=199252 infer_calls=51202 feedback_calls=51201 infer_wait=421.414s infer_obs_pack=0.892s env_step=25.362s feedback_total=5.849s feedback_obs_pack=1.389s feedback_info_pack=0.305s effective_fps=439.35 +2026-09-25 19:42:26.955 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=212981 infer_calls=54673 feedback_calls=54672 infer_wait=448.845s infer_obs_pack=0.953s env_step=27.104s feedback_total=6.251s feedback_obs_pack=1.481s feedback_info_pack=0.325s effective_fps=440.82 +2026-09-25 19:42:56.956 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=227074 infer_calls=58240 feedback_calls=58239 infer_wait=476.255s infer_obs_pack=1.012s env_step=28.845s feedback_total=6.662s feedback_obs_pack=1.575s feedback_info_pack=0.346s effective_fps=442.83 +2026-09-25 19:43:28.360 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=239736 infer_calls=61442 feedback_calls=61441 infer_wait=505.263s infer_obs_pack=1.069s env_step=30.465s feedback_total=7.031s feedback_obs_pack=1.663s feedback_info_pack=0.364s effective_fps=440.83 +2026-09-25 19:43:58.364 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=253233 infer_calls=64847 feedback_calls=64846 infer_wait=532.699s infer_obs_pack=1.128s env_step=32.205s feedback_total=7.428s feedback_obs_pack=1.755s feedback_info_pack=0.384s effective_fps=441.59 +2026-09-25 19:44:28.370 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=266905 infer_calls=68297 feedback_calls=68296 infer_wait=560.079s infer_obs_pack=1.189s env_step=33.975s feedback_total=7.835s feedback_obs_pack=1.851s feedback_info_pack=0.405s effective_fps=442.57 +2026-09-25 19:45:00.844 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=280328 infer_calls=71682 feedback_calls=71681 infer_wait=589.984s infer_obs_pack=1.248s env_step=35.708s feedback_total=8.239s feedback_obs_pack=1.946s feedback_info_pack=0.426s effective_fps=441.34 +2026-09-25 19:45:30.845 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=294139 infer_calls=75165 feedback_calls=75164 infer_wait=617.341s infer_obs_pack=1.311s env_step=37.493s feedback_total=8.654s feedback_obs_pack=2.042s feedback_info_pack=0.447s effective_fps=442.45 +2026-09-25 19:46:00.849 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=307771 infer_calls=78601 feedback_calls=78600 infer_wait=644.784s infer_obs_pack=1.371s env_step=39.236s feedback_total=9.041s feedback_obs_pack=2.136s feedback_info_pack=0.468s effective_fps=443.20 +2026-09-25 19:46:32.692 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=320974 infer_calls=81922 feedback_calls=81921 infer_wait=674.186s infer_obs_pack=1.428s env_step=40.881s feedback_total=9.431s feedback_obs_pack=2.224s feedback_info_pack=0.487s effective_fps=442.16 +2026-09-25 19:47:02.698 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=334184 infer_calls=85247 feedback_calls=85246 infer_wait=701.704s infer_obs_pack=1.487s env_step=42.581s feedback_total=9.803s feedback_obs_pack=2.313s feedback_info_pack=0.508s effective_fps=442.29 +2026-09-25 19:47:32.699 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=348086 infer_calls=88743 feedback_calls=88742 infer_wait=729.129s infer_obs_pack=1.546s env_step=44.323s feedback_total=10.199s feedback_obs_pack=2.407s feedback_info_pack=0.528s effective_fps=443.31 +2026-09-25 19:48:02.702 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=361585 infer_calls=92138 feedback_calls=92137 infer_wait=756.592s infer_obs_pack=1.604s env_step=46.039s feedback_total=10.593s feedback_obs_pack=2.500s feedback_info_pack=0.548s effective_fps=443.76 +2026-09-25 19:48:33.462 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=373904 infer_calls=95234 feedback_calls=95233 infer_wait=785.050s infer_obs_pack=1.658s env_step=47.599s feedback_total=10.951s feedback_obs_pack=2.586s feedback_info_pack=0.566s effective_fps=442.36 +2026-09-25 19:49:03.462 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=387675 infer_calls=98694 feedback_calls=98693 infer_wait=812.453s infer_obs_pack=1.717s env_step=49.365s feedback_total=11.350s feedback_obs_pack=2.684s feedback_info_pack=0.587s effective_fps=443.12 +2026-09-25 19:49:33.462 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=401613 infer_calls=102199 feedback_calls=102198 infer_wait=839.788s infer_obs_pack=1.779s env_step=51.162s feedback_total=11.772s feedback_obs_pack=2.786s feedback_info_pack=0.607s effective_fps=444.02 +2026-09-25 19:49:37.660 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=402418 infer_calls=102401 feedback_calls=102401 infer_wait=840.935s infer_obs_pack=1.782s env_step=51.269s feedback_total=11.794s feedback_obs_pack=2.792s feedback_info_pack=0.609s effective_fps=444.28 +2026-09-25 19:49:37.660 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 4802/409600 episodes. diff --git a/experiments/e28-dppo-learns/results/dppo-hopper/client-seed2.log b/experiments/e28-dppo-learns/results/dppo-hopper/client-seed2.log new file mode 100644 index 0000000..d357672 --- /dev/null +++ b/experiments/e28-dppo-learns/results/dppo-hopper/client-seed2.log @@ -0,0 +1,40 @@ +2026-09-25 19:34:22.577 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 19:34:22.577 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 19:34:22.578 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 19:34:22.578 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 19:34:22.579 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 19:34:22.619 | INFO | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-193422-857724f6 output_dir=runs/mujoco-v1-nenv1-20260925-193422-857724f6 recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 19:34:22.660 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9502... +2026-09-25 19:34:22.662 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'DPPOPolicy', 'action_dim': 3, 'action_horizon': 4} +2026-09-25 19:34:52.664 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=15309 infer_calls=4063 feedback_calls=4062 infer_wait=27.625s infer_obs_pack=0.056s env_step=1.573s feedback_total=0.333s feedback_obs_pack=0.082s feedback_info_pack=0.018s effective_fps=517.41 +2026-09-25 19:35:22.787 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=27229 infer_calls=7170 feedback_calls=7169 infer_wait=55.461s infer_obs_pack=0.110s env_step=3.089s feedback_total=0.693s feedback_obs_pack=0.165s feedback_info_pack=0.035s effective_fps=458.76 +2026-09-25 19:35:52.791 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=40410 infer_calls=10575 feedback_calls=10574 infer_wait=82.895s infer_obs_pack=0.168s env_step=4.813s feedback_total=1.090s feedback_obs_pack=0.258s feedback_info_pack=0.055s effective_fps=454.22 +2026-09-25 19:36:22.802 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=53976 infer_calls=14064 feedback_calls=14063 infer_wait=110.262s infer_obs_pack=0.240s env_step=6.575s feedback_total=1.506s feedback_obs_pack=0.354s feedback_info_pack=0.075s effective_fps=455.18 +2026-09-25 19:36:54.801 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=67039 infer_calls=17410 feedback_calls=17409 infer_wait=139.736s infer_obs_pack=0.296s env_step=8.263s feedback_total=1.906s feedback_obs_pack=0.445s feedback_info_pack=0.095s effective_fps=446.33 +2026-09-25 19:37:24.805 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=80449 infer_calls=20856 feedback_calls=20855 infer_wait=167.120s infer_obs_pack=0.354s env_step=10.014s feedback_total=2.323s feedback_obs_pack=0.541s feedback_info_pack=0.116s effective_fps=447.41 +2026-09-25 19:37:54.809 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=93509 infer_calls=24199 feedback_calls=24198 infer_wait=194.567s infer_obs_pack=0.413s env_step=11.736s feedback_total=2.717s feedback_obs_pack=0.636s feedback_info_pack=0.135s effective_fps=446.48 +2026-09-25 19:38:27.523 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=106993 infer_calls=27650 feedback_calls=27649 infer_wait=224.678s infer_obs_pack=0.473s env_step=13.479s feedback_total=3.130s feedback_obs_pack=0.731s feedback_info_pack=0.157s effective_fps=442.56 +2026-09-25 19:38:57.526 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=120278 infer_calls=31038 feedback_calls=31037 infer_wait=252.114s infer_obs_pack=0.532s env_step=15.202s feedback_total=3.534s feedback_obs_pack=0.825s feedback_info_pack=0.177s effective_fps=443.21 +2026-09-25 19:39:27.527 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=133380 infer_calls=34372 feedback_calls=34371 infer_wait=279.547s infer_obs_pack=0.589s env_step=16.943s feedback_total=3.922s feedback_obs_pack=0.920s feedback_info_pack=0.197s effective_fps=443.12 +2026-09-25 19:39:57.529 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=146353 infer_calls=37675 feedback_calls=37674 infer_wait=306.994s infer_obs_pack=0.647s env_step=18.670s feedback_total=4.314s feedback_obs_pack=1.013s feedback_info_pack=0.218s effective_fps=442.65 +2026-09-25 19:40:29.894 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=159310 infer_calls=40962 feedback_calls=40961 infer_wait=336.838s infer_obs_pack=0.703s env_step=20.369s feedback_total=4.714s feedback_obs_pack=1.106s feedback_info_pack=0.237s effective_fps=439.33 +2026-09-25 19:40:59.895 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=172411 infer_calls=44284 feedback_calls=44283 infer_wait=364.264s infer_obs_pack=0.760s env_step=22.104s feedback_total=5.117s feedback_obs_pack=1.205s feedback_info_pack=0.256s effective_fps=439.55 +2026-09-25 19:41:29.895 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=185553 infer_calls=47618 feedback_calls=47617 infer_wait=391.729s infer_obs_pack=0.818s env_step=23.822s feedback_total=5.509s feedback_obs_pack=1.296s feedback_info_pack=0.275s effective_fps=439.83 +2026-09-25 19:41:59.900 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=198907 infer_calls=51001 feedback_calls=51000 infer_wait=419.176s infer_obs_pack=0.877s env_step=25.541s feedback_total=5.913s feedback_obs_pack=1.390s feedback_info_pack=0.294s effective_fps=440.54 +2026-09-25 19:42:31.852 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=211805 infer_calls=54274 feedback_calls=54273 infer_wait=448.713s infer_obs_pack=0.934s env_step=27.183s feedback_total=6.275s feedback_obs_pack=1.478s feedback_info_pack=0.312s effective_fps=438.42 +2026-09-25 19:43:01.856 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=225476 infer_calls=57741 feedback_calls=57740 infer_wait=476.152s infer_obs_pack=0.993s env_step=28.910s feedback_total=6.678s feedback_obs_pack=1.571s feedback_info_pack=0.331s effective_fps=439.75 +2026-09-25 19:43:31.859 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=239204 infer_calls=61226 feedback_calls=61225 infer_wait=503.547s infer_obs_pack=1.054s env_step=30.659s feedback_total=7.089s feedback_obs_pack=1.666s feedback_info_pack=0.352s effective_fps=441.05 +2026-09-25 19:44:03.668 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=252156 infer_calls=64514 feedback_calls=64513 infer_wait=532.869s infer_obs_pack=1.114s env_step=32.326s feedback_total=7.482s feedback_obs_pack=1.755s feedback_info_pack=0.370s effective_fps=439.46 +2026-09-25 19:44:33.672 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=265880 infer_calls=67989 feedback_calls=67988 infer_wait=560.251s infer_obs_pack=1.174s env_step=34.081s feedback_total=7.906s feedback_obs_pack=1.854s feedback_info_pack=0.391s effective_fps=440.63 +2026-09-25 19:45:03.673 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=279859 infer_calls=71532 feedback_calls=71531 infer_wait=587.629s infer_obs_pack=1.233s env_step=35.835s feedback_total=8.336s feedback_obs_pack=1.953s feedback_info_pack=0.412s effective_fps=442.09 +2026-09-25 19:45:34.627 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=292571 infer_calls=74754 feedback_calls=74753 infer_wait=616.235s infer_obs_pack=1.286s env_step=37.415s feedback_total=8.709s feedback_obs_pack=2.039s feedback_info_pack=0.430s effective_fps=440.85 +2026-09-25 19:46:04.631 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=306122 infer_calls=78187 feedback_calls=78186 infer_wait=643.685s infer_obs_pack=1.344s env_step=39.137s feedback_total=9.105s feedback_obs_pack=2.134s feedback_info_pack=0.449s effective_fps=441.56 +2026-09-25 19:46:34.634 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=319350 infer_calls=81543 feedback_calls=81542 infer_wait=671.119s infer_obs_pack=1.402s env_step=40.872s feedback_total=9.506s feedback_obs_pack=2.227s feedback_info_pack=0.468s effective_fps=441.76 +2026-09-25 19:47:04.643 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=332666 infer_calls=84914 feedback_calls=84913 infer_wait=698.588s infer_obs_pack=1.461s env_step=42.587s feedback_total=9.900s feedback_obs_pack=2.320s feedback_info_pack=0.487s effective_fps=442.06 +2026-09-25 19:47:36.054 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=345153 infer_calls=88066 feedback_calls=88065 infer_wait=727.608s infer_obs_pack=1.517s env_step=44.210s feedback_total=10.268s feedback_obs_pack=2.408s feedback_info_pack=0.505s effective_fps=440.47 +2026-09-25 19:48:06.058 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=358392 infer_calls=91411 feedback_calls=91410 infer_wait=755.077s infer_obs_pack=1.576s env_step=45.920s feedback_total=10.665s feedback_obs_pack=2.502s feedback_info_pack=0.524s effective_fps=440.70 +2026-09-25 19:48:36.058 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=371996 infer_calls=94847 feedback_calls=94846 infer_wait=782.521s infer_obs_pack=1.634s env_step=47.653s feedback_total=11.060s feedback_obs_pack=2.595s feedback_info_pack=0.543s effective_fps=441.34 +2026-09-25 19:49:08.670 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=385694 infer_calls=98305 feedback_calls=98304 infer_wait=812.503s infer_obs_pack=1.695s env_step=49.432s feedback_total=11.473s feedback_obs_pack=2.690s feedback_info_pack=0.566s effective_fps=440.74 +2026-09-25 19:49:38.673 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=399458 infer_calls=101781 feedback_calls=101780 infer_wait=839.880s infer_obs_pack=1.756s env_step=51.217s feedback_total=11.876s feedback_obs_pack=2.785s feedback_info_pack=0.586s effective_fps=441.52 +2026-09-25 19:49:44.925 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=401917 infer_calls=102400 feedback_calls=102400 infer_wait=842.875s infer_obs_pack=1.767s env_step=51.489s feedback_total=11.939s feedback_obs_pack=2.800s feedback_info_pack=0.589s effective_fps=442.61 +2026-09-25 19:49:44.925 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 5084/409600 episodes. diff --git a/experiments/e28-dppo-learns/results/dppo-hopper/server-seed0.log b/experiments/e28-dppo-learns/results/dppo-hopper/server-seed0.log new file mode 100644 index 0000000..fbd0620 --- /dev/null +++ b/experiments/e28-dppo-learns/results/dppo-hopper/server-seed0.log @@ -0,0 +1,26 @@ +19:34:12|INFO|plugrl_server version: 0.1.0 +19:34:12|INFO|Algorithm: dppo, Config: DPPOAlgoConfigHopper(global_steps=102400, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=1024, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=512, train_itrs=100, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +19:34:12|INFO|Policy: dppo-policy, Config: DPPOPolicyConfigHopper(algo='dppo', device=device(type='cpu'), env_type='gym', env_name='hopper-medium-v2', checkpoint_path=None, critic=DPPOCriticObsConfig(mlp_dims=[256, 256, 256], activation='Mish', residual_style=True)) +19:34:12|INFO|Seeded python, numpy and torch with 0. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +19:34:12|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed0 +19:34:12|INFO|Number of network parameters: 553020 +19:34:12|INFO|Policy created... +19:34:12|INFO|Initialized RolloutBuffer buffer_size=1024 action_shape=(1024, 20, 4, 3) value_shape=(1024,) +19:34:12|INFO|Algorithm created: + +19:34:12|INFO|Agent Server is listening on 0.0.0.0:9500 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +19:37:10|INFO|Checkpoint saved at step 20481 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed0/20481 +19:40:18|INFO|Checkpoint saved at step 40961 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed0/40961 +19:43:24|INFO|Checkpoint saved at step 61441 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed0/61441 +19:46:28|INFO|Checkpoint saved at step 81921 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed0/81921 +19:49:35|INFO|Checkpoint saved at step 102401 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed0/102401 +19:49:35|INFO|Stopping server as the algorithm signaled to stop. +19:49:35|INFO|Shutdown started: aborting pending infer requests. +19:49:35|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +19:49:35|INFO|Shutdown closing 1 websocket connection(s). +19:49:35|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 53368). +19:49:35|INFO|WebSocket server closed. +19:49:35|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e28-dppo-learns/results/dppo-hopper/server-seed1.log b/experiments/e28-dppo-learns/results/dppo-hopper/server-seed1.log new file mode 100644 index 0000000..8039d3e --- /dev/null +++ b/experiments/e28-dppo-learns/results/dppo-hopper/server-seed1.log @@ -0,0 +1,26 @@ +19:34:17|INFO|plugrl_server version: 0.1.0 +19:34:17|INFO|Algorithm: dppo, Config: DPPOAlgoConfigHopper(global_steps=102400, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=1024, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=512, train_itrs=100, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +19:34:17|INFO|Policy: dppo-policy, Config: DPPOPolicyConfigHopper(algo='dppo', device=device(type='cpu'), env_type='gym', env_name='hopper-medium-v2', checkpoint_path=None, critic=DPPOCriticObsConfig(mlp_dims=[256, 256, 256], activation='Mish', residual_style=True)) +19:34:17|INFO|Seeded python, numpy and torch with 1. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +19:34:17|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed1 +19:34:17|INFO|Number of network parameters: 553020 +19:34:17|INFO|Policy created... +19:34:17|INFO|Initialized RolloutBuffer buffer_size=1024 action_shape=(1024, 20, 4, 3) value_shape=(1024,) +19:34:17|INFO|Algorithm created: + +19:34:17|INFO|Agent Server is listening on 0.0.0.0:9501 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +19:37:16|INFO|Checkpoint saved at step 20481 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed1/20481 +19:40:23|INFO|Checkpoint saved at step 40961 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed1/40961 +19:43:28|INFO|Checkpoint saved at step 61441 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed1/61441 +19:46:32|INFO|Checkpoint saved at step 81921 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed1/81921 +19:49:37|INFO|Checkpoint saved at step 102401 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed1/102401 +19:49:37|INFO|Stopping server as the algorithm signaled to stop. +19:49:37|INFO|Shutdown started: aborting pending infer requests. +19:49:37|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +19:49:37|INFO|Shutdown closing 1 websocket connection(s). +19:49:37|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 47300). +19:49:37|INFO|WebSocket server closed. +19:49:37|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e28-dppo-learns/results/dppo-hopper/server-seed2.log b/experiments/e28-dppo-learns/results/dppo-hopper/server-seed2.log new file mode 100644 index 0000000..ef7cc2d --- /dev/null +++ b/experiments/e28-dppo-learns/results/dppo-hopper/server-seed2.log @@ -0,0 +1,26 @@ +19:34:22|INFO|plugrl_server version: 0.1.0 +19:34:22|INFO|Algorithm: dppo, Config: DPPOAlgoConfigHopper(global_steps=102400, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=1024, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=512, train_itrs=100, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +19:34:22|INFO|Policy: dppo-policy, Config: DPPOPolicyConfigHopper(algo='dppo', device=device(type='cpu'), env_type='gym', env_name='hopper-medium-v2', checkpoint_path=None, critic=DPPOCriticObsConfig(mlp_dims=[256, 256, 256], activation='Mish', residual_style=True)) +19:34:22|INFO|Seeded python, numpy and torch with 2. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +19:34:22|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed2 +19:34:22|INFO|Number of network parameters: 553020 +19:34:22|INFO|Policy created... +19:34:22|INFO|Initialized RolloutBuffer buffer_size=1024 action_shape=(1024, 20, 4, 3) value_shape=(1024,) +19:34:22|INFO|Algorithm created: + +19:34:22|INFO|Agent Server is listening on 0.0.0.0:9502 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +19:37:22|INFO|Checkpoint saved at step 20481 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed2/20481 +19:40:29|INFO|Checkpoint saved at step 40961 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed2/40961 +19:43:36|INFO|Checkpoint saved at step 61441 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed2/61441 +19:46:39|INFO|Checkpoint saved at step 81921 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed2/81921 +19:49:44|INFO|Checkpoint saved at step 102400 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed2/102400 +19:49:44|INFO|Stopping server as the algorithm signaled to stop. +19:49:44|INFO|Shutdown started: aborting pending infer requests. +19:49:44|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +19:49:44|INFO|Shutdown closing 1 websocket connection(s). +19:49:44|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 36754). +19:49:44|INFO|WebSocket server closed. +19:49:44|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e28-dppo-learns/results/dppo-walker.out b/experiments/e28-dppo-learns/results/dppo-walker.out new file mode 100644 index 0000000..24e1a64 --- /dev/null +++ b/experiments/e28-dppo-learns/results/dppo-walker.out @@ -0,0 +1,10 @@ +cell: dppo-walker = dppo-policy/walker x dppo/walker x Walker2d-v5 +iters: 100 x 1024 batch: 512 replan: 4 seeds: 0 1 2 +out: /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-walker +start 2026-09-25 19:34:30 +seed 0 finished rc=0 at 19:50:14 +seed 2 finished rc=0 at 19:50:23 +seed 1 finished rc=0 at 19:50:25 +end 2026-09-25 19:50:25 +failed seeds: 0 +CELL_DONE dppo-walker diff --git a/experiments/e28-dppo-learns/results/dppo-walker/client-seed0.log b/experiments/e28-dppo-learns/results/dppo-walker/client-seed0.log new file mode 100644 index 0000000..d1a32fa --- /dev/null +++ b/experiments/e28-dppo-learns/results/dppo-walker/client-seed0.log @@ -0,0 +1,40 @@ +2026-09-25 19:34:32.581 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 19:34:32.581 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 19:34:32.581 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 19:34:32.581 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 19:34:32.582 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 19:34:32.622 | INFO | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-193432-0f40908e output_dir=runs/mujoco-v1-nenv1-20260925-193432-0f40908e recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 19:34:32.660 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9510... +2026-09-25 19:34:32.663 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'DPPOPolicy', 'action_dim': 6, 'action_horizon': 4} +2026-09-25 19:35:02.664 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=14655 infer_calls=3919 feedback_calls=3919 infer_wait=27.506s infer_obs_pack=0.056s env_step=1.656s feedback_total=0.356s feedback_obs_pack=0.090s feedback_info_pack=0.019s effective_fps=495.54 +2026-09-25 19:35:34.764 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=27256 infer_calls=7170 feedback_calls=7169 infer_wait=57.141s infer_obs_pack=0.109s env_step=3.332s feedback_total=0.721s feedback_obs_pack=0.179s feedback_info_pack=0.036s effective_fps=444.61 +2026-09-25 19:36:04.764 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=40667 infer_calls=10565 feedback_calls=10564 infer_wait=84.487s infer_obs_pack=0.166s env_step=5.161s feedback_total=1.117s feedback_obs_pack=0.273s feedback_info_pack=0.055s effective_fps=447.22 +2026-09-25 19:36:34.768 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=54063 infer_calls=13944 feedback_calls=13943 infer_wait=111.815s infer_obs_pack=0.235s env_step=7.012s feedback_total=1.503s feedback_obs_pack=0.368s feedback_info_pack=0.073s effective_fps=448.41 +2026-09-25 19:37:04.768 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=67438 infer_calls=17309 feedback_calls=17308 infer_wait=139.080s infer_obs_pack=0.292s env_step=8.917s feedback_total=1.906s feedback_obs_pack=0.460s feedback_info_pack=0.093s effective_fps=449.00 +2026-09-25 19:37:36.367 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=80004 infer_calls=20482 feedback_calls=20481 infer_wait=168.168s infer_obs_pack=0.345s env_step=10.647s feedback_total=2.285s feedback_obs_pack=0.553s feedback_info_pack=0.110s effective_fps=440.92 +2026-09-25 19:38:06.371 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=93274 infer_calls=23822 feedback_calls=23821 infer_wait=195.482s infer_obs_pack=0.402s env_step=12.523s feedback_total=2.677s feedback_obs_pack=0.644s feedback_info_pack=0.128s effective_fps=441.88 +2026-09-25 19:38:36.372 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=106417 infer_calls=27133 feedback_calls=27132 infer_wait=222.850s infer_obs_pack=0.458s env_step=14.355s feedback_total=3.063s feedback_obs_pack=0.734s feedback_info_pack=0.146s effective_fps=442.07 +2026-09-25 19:39:06.378 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=119369 infer_calls=30393 feedback_calls=30392 infer_wait=250.202s infer_obs_pack=0.517s env_step=16.211s feedback_total=3.443s feedback_obs_pack=0.826s feedback_info_pack=0.167s effective_fps=441.50 +2026-09-25 19:39:36.382 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=132465 infer_calls=33694 feedback_calls=33693 infer_wait=277.474s infer_obs_pack=0.576s env_step=18.121s feedback_total=3.833s feedback_obs_pack=0.922s feedback_info_pack=0.184s effective_fps=441.54 +2026-09-25 19:40:09.069 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=145067 infer_calls=36866 feedback_calls=36865 infer_wait=307.536s infer_obs_pack=0.631s env_step=19.963s feedback_total=4.204s feedback_obs_pack=1.014s feedback_info_pack=0.205s effective_fps=436.51 +2026-09-25 19:40:39.070 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=157977 infer_calls=40118 feedback_calls=40117 infer_wait=334.809s infer_obs_pack=0.689s env_step=21.881s feedback_total=4.590s feedback_obs_pack=1.103s feedback_info_pack=0.224s effective_fps=436.44 +2026-09-25 19:41:09.075 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=170829 infer_calls=43351 feedback_calls=43351 infer_wait=362.160s infer_obs_pack=0.749s env_step=23.743s feedback_total=4.966s feedback_obs_pack=1.194s feedback_info_pack=0.243s effective_fps=436.21 +2026-09-25 19:41:39.077 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=183697 infer_calls=46592 feedback_calls=46591 infer_wait=389.470s infer_obs_pack=0.806s env_step=25.623s feedback_total=5.360s feedback_obs_pack=1.289s feedback_info_pack=0.261s effective_fps=436.07 +2026-09-25 19:42:09.081 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=196990 infer_calls=49944 feedback_calls=49943 infer_wait=416.761s infer_obs_pack=0.863s env_step=27.529s feedback_total=5.742s feedback_obs_pack=1.382s feedback_info_pack=0.279s effective_fps=436.89 +2026-09-25 19:42:41.398 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=210116 infer_calls=53249 feedback_calls=53248 infer_wait=446.433s infer_obs_pack=0.918s env_step=29.379s feedback_total=6.127s feedback_obs_pack=1.471s feedback_info_pack=0.296s effective_fps=435.15 +2026-09-25 19:43:11.398 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=223490 infer_calls=56614 feedback_calls=56613 infer_wait=473.719s infer_obs_pack=0.975s env_step=31.288s feedback_total=6.515s feedback_obs_pack=1.564s feedback_info_pack=0.314s effective_fps=436.08 +2026-09-25 19:43:41.399 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=236666 infer_calls=59928 feedback_calls=59928 infer_wait=500.993s infer_obs_pack=1.031s env_step=33.214s feedback_total=6.893s feedback_obs_pack=1.655s feedback_info_pack=0.332s effective_fps=436.55 +2026-09-25 19:44:11.401 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=249797 infer_calls=63238 feedback_calls=63237 infer_wait=528.255s infer_obs_pack=1.091s env_step=35.135s feedback_total=7.288s feedback_obs_pack=1.750s feedback_info_pack=0.350s effective_fps=436.88 +2026-09-25 19:44:44.295 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=262993 infer_calls=66562 feedback_calls=66561 infer_wait=558.385s infer_obs_pack=1.151s env_step=37.074s feedback_total=7.687s feedback_obs_pack=1.844s feedback_info_pack=0.368s effective_fps=435.21 +2026-09-25 19:45:14.295 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=276334 infer_calls=69924 feedback_calls=69923 infer_wait=585.641s infer_obs_pack=1.209s env_step=38.998s feedback_total=8.084s feedback_obs_pack=1.935s feedback_info_pack=0.386s effective_fps=435.91 +2026-09-25 19:45:44.305 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=289526 infer_calls=73241 feedback_calls=73240 infer_wait=612.917s infer_obs_pack=1.267s env_step=40.927s feedback_total=8.466s feedback_obs_pack=2.029s feedback_info_pack=0.404s effective_fps=436.31 +2026-09-25 19:46:14.305 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=302824 infer_calls=76588 feedback_calls=76587 infer_wait=640.130s infer_obs_pack=1.326s env_step=42.886s feedback_total=8.869s feedback_obs_pack=2.123s feedback_info_pack=0.422s effective_fps=436.84 +2026-09-25 19:46:44.311 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=315519 infer_calls=79784 feedback_calls=79783 infer_wait=667.458s infer_obs_pack=1.385s env_step=44.766s feedback_total=9.246s feedback_obs_pack=2.213s feedback_info_pack=0.440s effective_fps=436.49 +2026-09-25 19:47:17.414 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=328072 infer_calls=82946 feedback_calls=82945 infer_wait=697.923s infer_obs_pack=1.440s env_step=46.631s feedback_total=9.611s feedback_obs_pack=2.302s feedback_info_pack=0.457s effective_fps=434.18 +2026-09-25 19:47:47.418 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=341142 infer_calls=86237 feedback_calls=86236 infer_wait=725.246s infer_obs_pack=1.497s env_step=48.526s feedback_total=9.986s feedback_obs_pack=2.392s feedback_info_pack=0.475s effective_fps=434.43 +2026-09-25 19:48:17.421 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=354244 infer_calls=89530 feedback_calls=89529 infer_wait=752.588s infer_obs_pack=1.552s env_step=50.420s feedback_total=10.347s feedback_obs_pack=2.481s feedback_info_pack=0.492s effective_fps=434.70 +2026-09-25 19:48:47.424 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=367240 infer_calls=92804 feedback_calls=92803 infer_wait=779.862s infer_obs_pack=1.609s env_step=52.357s feedback_total=10.721s feedback_obs_pack=2.573s feedback_info_pack=0.511s effective_fps=434.84 +2026-09-25 19:49:17.427 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=380535 infer_calls=96146 feedback_calls=96145 infer_wait=807.131s infer_obs_pack=1.666s env_step=54.286s feedback_total=11.104s feedback_obs_pack=2.667s feedback_info_pack=0.529s effective_fps=435.30 +2026-09-25 19:49:48.681 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=393191 infer_calls=99330 feedback_calls=99329 infer_wait=835.867s infer_obs_pack=1.720s env_step=56.066s feedback_total=11.454s feedback_obs_pack=2.750s feedback_info_pack=0.548s effective_fps=434.41 +2026-09-25 19:50:14.052 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=405402 infer_calls=102401 feedback_calls=102401 infer_wait=856.336s infer_obs_pack=1.763s env_step=57.529s feedback_total=11.696s feedback_obs_pack=2.815s feedback_info_pack=0.560s effective_fps=437.17 +2026-09-25 19:50:14.052 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 2775/409600 episodes. diff --git a/experiments/e28-dppo-learns/results/dppo-walker/client-seed1.log b/experiments/e28-dppo-learns/results/dppo-walker/client-seed1.log new file mode 100644 index 0000000..fff1037 --- /dev/null +++ b/experiments/e28-dppo-learns/results/dppo-walker/client-seed1.log @@ -0,0 +1,41 @@ +2026-09-25 19:34:37.582 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 19:34:37.582 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 19:34:37.582 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 19:34:37.582 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 19:34:37.583 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 19:34:37.625 | INFO | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-193437-c4ee2d98 output_dir=runs/mujoco-v1-nenv1-20260925-193437-c4ee2d98 recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 19:34:37.666 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9511... +2026-09-25 19:34:37.669 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'DPPOPolicy', 'action_dim': 6, 'action_horizon': 4} +2026-09-25 19:35:07.672 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=14043 infer_calls=3747 feedback_calls=3746 infer_wait=27.455s infer_obs_pack=0.055s env_step=1.696s feedback_total=0.373s feedback_obs_pack=0.089s feedback_info_pack=0.019s effective_fps=474.75 +2026-09-25 19:35:37.677 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=27135 infer_calls=7105 feedback_calls=7104 infer_wait=54.894s infer_obs_pack=0.111s env_step=3.452s feedback_total=0.756s feedback_obs_pack=0.182s feedback_info_pack=0.037s effective_fps=458.26 +2026-09-25 19:36:09.629 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=39515 infer_calls=10242 feedback_calls=10241 infer_wait=84.347s infer_obs_pack=0.164s env_step=5.184s feedback_total=1.118s feedback_obs_pack=0.269s feedback_info_pack=0.054s effective_fps=435.12 +2026-09-25 19:36:39.634 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=52640 infer_calls=13558 feedback_calls=13557 infer_wait=111.751s infer_obs_pack=0.230s env_step=6.984s feedback_total=1.498s feedback_obs_pack=0.366s feedback_info_pack=0.071s effective_fps=436.98 +2026-09-25 19:37:09.643 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=65935 infer_calls=16912 feedback_calls=16911 infer_wait=139.115s infer_obs_pack=0.287s env_step=8.819s feedback_total=1.893s feedback_obs_pack=0.468s feedback_info_pack=0.089s effective_fps=439.23 +2026-09-25 19:37:39.643 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=78951 infer_calls=20189 feedback_calls=20188 infer_wait=166.457s infer_obs_pack=0.343s env_step=10.676s feedback_total=2.283s feedback_obs_pack=0.561s feedback_info_pack=0.107s effective_fps=439.21 +2026-09-25 19:38:09.644 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=91981 infer_calls=23469 feedback_calls=23468 infer_wait=193.832s infer_obs_pack=0.399s env_step=12.520s feedback_total=2.654s feedback_obs_pack=0.650s feedback_info_pack=0.125s effective_fps=439.25 +2026-09-25 19:38:42.316 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=104512 infer_calls=26626 feedback_calls=26625 infer_wait=223.912s infer_obs_pack=0.454s env_step=14.341s feedback_total=3.022s feedback_obs_pack=0.739s feedback_info_pack=0.142s effective_fps=432.35 +2026-09-25 19:39:12.321 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=117323 infer_calls=29854 feedback_calls=29853 infer_wait=251.287s infer_obs_pack=0.514s env_step=16.184s feedback_total=3.401s feedback_obs_pack=0.829s feedback_info_pack=0.163s effective_fps=432.31 +2026-09-25 19:39:42.322 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=130080 infer_calls=33061 feedback_calls=33060 infer_wait=278.642s infer_obs_pack=0.569s env_step=18.052s feedback_total=3.769s feedback_obs_pack=0.922s feedback_info_pack=0.181s effective_fps=432.11 +2026-09-25 19:40:12.323 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=142802 infer_calls=36265 feedback_calls=36264 infer_wait=306.026s infer_obs_pack=0.625s env_step=19.887s feedback_total=4.141s feedback_obs_pack=1.012s feedback_info_pack=0.198s effective_fps=431.85 +2026-09-25 19:40:42.327 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=155450 infer_calls=39445 feedback_calls=39444 infer_wait=333.424s infer_obs_pack=0.678s env_step=21.708s feedback_total=4.523s feedback_obs_pack=1.107s feedback_info_pack=0.216s effective_fps=431.41 +2026-09-25 19:41:12.328 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=168266 infer_calls=42672 feedback_calls=42671 infer_wait=360.755s infer_obs_pack=0.732s env_step=23.581s feedback_total=4.901s feedback_obs_pack=1.198s feedback_info_pack=0.234s effective_fps=431.49 +2026-09-25 19:41:42.332 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=180991 infer_calls=45874 feedback_calls=45873 infer_wait=388.181s infer_obs_pack=0.787s env_step=25.387s feedback_total=5.273s feedback_obs_pack=1.287s feedback_info_pack=0.251s effective_fps=431.31 +2026-09-25 19:42:12.337 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=193927 infer_calls=49131 feedback_calls=49130 infer_wait=415.566s infer_obs_pack=0.842s env_step=27.236s feedback_total=5.639s feedback_obs_pack=1.379s feedback_info_pack=0.269s effective_fps=431.64 +2026-09-25 19:42:44.091 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=206228 infer_calls=52226 feedback_calls=52225 infer_wait=444.834s infer_obs_pack=0.893s env_step=28.994s feedback_total=5.992s feedback_obs_pack=1.464s feedback_info_pack=0.289s effective_fps=429.00 +2026-09-25 19:43:14.092 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=219256 infer_calls=55506 feedback_calls=55505 infer_wait=472.169s infer_obs_pack=0.949s env_step=30.870s feedback_total=6.371s feedback_obs_pack=1.555s feedback_info_pack=0.307s effective_fps=429.61 +2026-09-25 19:43:44.097 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=232504 infer_calls=58844 feedback_calls=58843 infer_wait=499.498s infer_obs_pack=1.006s env_step=32.739s feedback_total=6.760s feedback_obs_pack=1.652s feedback_info_pack=0.325s effective_fps=430.56 +2026-09-25 19:44:14.098 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=245667 infer_calls=62162 feedback_calls=62161 infer_wait=526.880s infer_obs_pack=1.061s env_step=34.569s feedback_total=7.144s feedback_obs_pack=1.743s feedback_info_pack=0.343s effective_fps=431.26 +2026-09-25 19:44:44.100 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=258726 infer_calls=65448 feedback_calls=65447 infer_wait=554.223s infer_obs_pack=1.118s env_step=36.450s feedback_total=7.515s feedback_obs_pack=1.833s feedback_info_pack=0.360s effective_fps=431.71 +2026-09-25 19:45:16.177 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=271277 infer_calls=68610 feedback_calls=68609 infer_wait=583.757s infer_obs_pack=1.171s env_step=38.232s feedback_total=7.882s feedback_obs_pack=1.923s feedback_info_pack=0.377s effective_fps=429.89 +2026-09-25 19:45:46.180 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=284787 infer_calls=72010 feedback_calls=72009 infer_wait=611.042s infer_obs_pack=1.228s env_step=40.146s feedback_total=8.269s feedback_obs_pack=2.017s feedback_info_pack=0.397s effective_fps=431.05 +2026-09-25 19:46:16.184 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=297984 infer_calls=75331 feedback_calls=75330 infer_wait=638.367s infer_obs_pack=1.283s env_step=42.033s feedback_total=8.648s feedback_obs_pack=2.108s feedback_info_pack=0.415s effective_fps=431.65 +2026-09-25 19:46:46.185 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=310989 infer_calls=78605 feedback_calls=78604 infer_wait=665.709s infer_obs_pack=1.338s env_step=43.927s feedback_total=9.010s feedback_obs_pack=2.197s feedback_info_pack=0.433s effective_fps=431.94 +2026-09-25 19:47:16.191 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=323881 infer_calls=81853 feedback_calls=81852 infer_wait=693.106s infer_obs_pack=1.393s env_step=45.780s feedback_total=9.372s feedback_obs_pack=2.285s feedback_info_pack=0.450s effective_fps=432.04 +2026-09-25 19:47:48.665 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=336367 infer_calls=84994 feedback_calls=84993 infer_wait=722.958s infer_obs_pack=1.446s env_step=47.641s feedback_total=9.737s feedback_obs_pack=2.373s feedback_info_pack=0.467s effective_fps=430.26 +2026-09-25 19:48:18.668 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=349266 infer_calls=88239 feedback_calls=88238 infer_wait=750.236s infer_obs_pack=1.504s env_step=49.570s feedback_total=10.118s feedback_obs_pack=2.464s feedback_info_pack=0.485s effective_fps=430.43 +2026-09-25 19:48:48.673 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=362463 infer_calls=91562 feedback_calls=91561 infer_wait=777.485s infer_obs_pack=1.560s env_step=51.522s feedback_total=10.505s feedback_obs_pack=2.560s feedback_info_pack=0.504s effective_fps=430.95 +2026-09-25 19:49:18.678 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=375663 infer_calls=94885 feedback_calls=94884 infer_wait=804.771s infer_obs_pack=1.616s env_step=53.444s feedback_total=10.888s feedback_obs_pack=2.655s feedback_info_pack=0.521s effective_fps=431.44 +2026-09-25 19:49:51.466 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=389255 infer_calls=98306 feedback_calls=98305 infer_wait=834.936s infer_obs_pack=1.670s env_step=55.292s feedback_total=11.258s feedback_obs_pack=2.749s feedback_info_pack=0.539s effective_fps=430.99 +2026-09-25 19:50:21.470 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=404886 infer_calls=102235 feedback_calls=102234 infer_wait=862.435s infer_obs_pack=1.727s env_step=57.104s feedback_total=11.556s feedback_obs_pack=2.825s feedback_info_pack=0.553s effective_fps=434.04 +2026-09-25 19:50:25.158 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=405548 infer_calls=102401 feedback_calls=102401 infer_wait=863.216s infer_obs_pack=1.730s env_step=57.175s feedback_total=11.568s feedback_obs_pack=2.828s feedback_info_pack=0.553s effective_fps=434.35 +2026-09-25 19:50:25.159 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 2681/409600 episodes. diff --git a/experiments/e28-dppo-learns/results/dppo-walker/client-seed2.log b/experiments/e28-dppo-learns/results/dppo-walker/client-seed2.log new file mode 100644 index 0000000..43fff95 --- /dev/null +++ b/experiments/e28-dppo-learns/results/dppo-walker/client-seed2.log @@ -0,0 +1,40 @@ +2026-09-25 19:34:42.587 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 19:34:42.587 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 19:34:42.587 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 19:34:42.587 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 19:34:42.588 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 19:34:42.631 | INFO | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-193442-412c79d1 output_dir=runs/mujoco-v1-nenv1-20260925-193442-412c79d1 recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 19:34:42.667 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9512... +2026-09-25 19:34:42.670 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'DPPOPolicy', 'action_dim': 6, 'action_horizon': 4} +2026-09-25 19:35:12.673 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=13451 infer_calls=3673 feedback_calls=3672 infer_wait=27.485s infer_obs_pack=0.055s env_step=1.638s feedback_total=0.386s feedback_obs_pack=0.095s feedback_info_pack=0.020s effective_fps=454.97 +2026-09-25 19:35:42.673 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=26227 infer_calls=7058 feedback_calls=7057 infer_wait=54.895s infer_obs_pack=0.112s env_step=3.362s feedback_total=0.779s feedback_obs_pack=0.187s feedback_info_pack=0.041s effective_fps=443.41 +2026-09-25 19:36:14.841 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=38742 infer_calls=10241 feedback_calls=10240 infer_wait=84.544s infer_obs_pack=0.165s env_step=5.093s feedback_total=1.152s feedback_obs_pack=0.276s feedback_info_pack=0.061s effective_fps=425.95 +2026-09-25 19:36:44.843 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=52225 infer_calls=13640 feedback_calls=13639 infer_wait=111.850s infer_obs_pack=0.238s env_step=6.944s feedback_total=1.556s feedback_obs_pack=0.372s feedback_info_pack=0.079s effective_fps=433.09 +2026-09-25 19:37:14.845 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=65527 infer_calls=16998 feedback_calls=16997 infer_wait=139.183s infer_obs_pack=0.295s env_step=8.801s feedback_total=1.943s feedback_obs_pack=0.468s feedback_info_pack=0.097s effective_fps=436.20 +2026-09-25 19:37:44.849 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=78734 infer_calls=20331 feedback_calls=20330 infer_wait=166.573s infer_obs_pack=0.351s env_step=10.609s feedback_total=2.336s feedback_obs_pack=0.561s feedback_info_pack=0.115s effective_fps=437.73 +2026-09-25 19:38:17.067 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=91536 infer_calls=23554 feedback_calls=23553 infer_wait=196.261s infer_obs_pack=0.405s env_step=12.370s feedback_total=2.702s feedback_obs_pack=0.649s feedback_info_pack=0.132s effective_fps=432.30 +2026-09-25 19:38:47.071 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=104497 infer_calls=26816 feedback_calls=26815 infer_wait=223.642s infer_obs_pack=0.462s env_step=14.204s feedback_total=3.079s feedback_obs_pack=0.739s feedback_info_pack=0.150s effective_fps=432.90 +2026-09-25 19:39:17.073 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=117589 infer_calls=30112 feedback_calls=30111 infer_wait=250.990s infer_obs_pack=0.518s env_step=16.062s feedback_total=3.457s feedback_obs_pack=0.832s feedback_info_pack=0.168s effective_fps=433.86 +2026-09-25 19:39:47.075 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=130468 infer_calls=33351 feedback_calls=33350 infer_wait=278.275s infer_obs_pack=0.581s env_step=17.963s feedback_total=3.842s feedback_obs_pack=0.929s feedback_info_pack=0.187s effective_fps=433.94 +2026-09-25 19:40:17.081 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=143138 infer_calls=36542 feedback_calls=36541 infer_wait=305.591s infer_obs_pack=0.637s env_step=19.862s feedback_total=4.215s feedback_obs_pack=1.022s feedback_info_pack=0.205s effective_fps=433.35 +2026-09-25 19:40:47.082 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=155940 infer_calls=39759 feedback_calls=39758 infer_wait=332.935s infer_obs_pack=0.691s env_step=21.743s feedback_total=4.581s feedback_obs_pack=1.113s feedback_info_pack=0.223s effective_fps=433.23 +2026-09-25 19:41:17.083 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=168569 infer_calls=42937 feedback_calls=42936 infer_wait=360.281s infer_obs_pack=0.746s env_step=23.616s feedback_total=4.951s feedback_obs_pack=1.203s feedback_info_pack=0.243s effective_fps=432.68 +2026-09-25 19:41:49.823 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=181074 infer_calls=46082 feedback_calls=46081 infer_wait=390.455s infer_obs_pack=0.799s env_step=25.422s feedback_total=5.320s feedback_obs_pack=1.291s feedback_info_pack=0.260s effective_fps=429.09 +2026-09-25 19:42:19.823 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=194204 infer_calls=49385 feedback_calls=49385 infer_wait=417.757s infer_obs_pack=0.856s env_step=27.316s feedback_total=5.707s feedback_obs_pack=1.386s feedback_info_pack=0.278s effective_fps=430.00 +2026-09-25 19:42:49.826 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=207519 infer_calls=52737 feedback_calls=52736 infer_wait=445.012s infer_obs_pack=0.913s env_step=29.238s feedback_total=6.106s feedback_obs_pack=1.480s feedback_info_pack=0.297s effective_fps=431.19 +2026-09-25 19:43:19.827 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=221018 infer_calls=56133 feedback_calls=56132 infer_wait=472.304s infer_obs_pack=0.970s env_step=31.141s feedback_total=6.491s feedback_obs_pack=1.575s feedback_info_pack=0.319s effective_fps=432.60 +2026-09-25 19:43:52.151 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=233978 infer_calls=59394 feedback_calls=59393 infer_wait=502.061s infer_obs_pack=1.024s env_step=32.939s feedback_total=6.865s feedback_obs_pack=1.663s feedback_info_pack=0.336s effective_fps=430.99 +2026-09-25 19:44:22.151 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=247431 infer_calls=62780 feedback_calls=62779 infer_wait=529.453s infer_obs_pack=1.079s env_step=34.761s feedback_total=7.244s feedback_obs_pack=1.755s feedback_info_pack=0.354s effective_fps=432.17 +2026-09-25 19:44:52.158 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=260899 infer_calls=66168 feedback_calls=66167 infer_wait=556.760s infer_obs_pack=1.136s env_step=36.655s feedback_total=7.637s feedback_obs_pack=1.848s feedback_info_pack=0.372s effective_fps=433.25 +2026-09-25 19:45:22.159 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=274258 infer_calls=69528 feedback_calls=69527 infer_wait=584.063s infer_obs_pack=1.194s env_step=38.550s feedback_total=8.023s feedback_obs_pack=1.940s feedback_info_pack=0.390s effective_fps=434.07 +2026-09-25 19:45:54.186 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=286895 infer_calls=72706 feedback_calls=72705 infer_wait=613.504s infer_obs_pack=1.249s env_step=40.367s feedback_total=8.398s feedback_obs_pack=2.032s feedback_info_pack=0.407s effective_fps=432.38 +2026-09-25 19:46:24.189 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=300381 infer_calls=76103 feedback_calls=76102 infer_wait=640.821s infer_obs_pack=1.305s env_step=42.242s feedback_total=8.789s feedback_obs_pack=2.124s feedback_info_pack=0.426s effective_fps=433.35 +2026-09-25 19:46:54.191 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=313236 infer_calls=79339 feedback_calls=79338 infer_wait=668.248s infer_obs_pack=1.362s env_step=44.057s feedback_total=9.146s feedback_obs_pack=2.211s feedback_info_pack=0.443s effective_fps=433.36 +2026-09-25 19:47:24.193 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=326486 infer_calls=82674 feedback_calls=82673 infer_wait=695.590s infer_obs_pack=1.418s env_step=45.923s feedback_total=9.529s feedback_obs_pack=2.302s feedback_info_pack=0.460s effective_fps=433.89 +2026-09-25 19:47:54.196 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=339661 infer_calls=85988 feedback_calls=85987 infer_wait=722.883s infer_obs_pack=1.478s env_step=47.827s feedback_total=9.913s feedback_obs_pack=2.393s feedback_info_pack=0.478s effective_fps=434.29 +2026-09-25 19:48:26.001 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=351976 infer_calls=89090 feedback_calls=89089 infer_wait=752.111s infer_obs_pack=1.531s env_step=49.641s feedback_total=10.283s feedback_obs_pack=2.483s feedback_info_pack=0.496s effective_fps=432.63 +2026-09-25 19:48:56.001 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=365314 infer_calls=92444 feedback_calls=92443 infer_wait=779.365s infer_obs_pack=1.591s env_step=51.576s feedback_total=10.676s feedback_obs_pack=2.575s feedback_info_pack=0.515s effective_fps=433.24 +2026-09-25 19:49:26.003 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=378528 infer_calls=95769 feedback_calls=95768 infer_wait=806.666s infer_obs_pack=1.647s env_step=53.482s feedback_total=11.062s feedback_obs_pack=2.666s feedback_info_pack=0.533s effective_fps=433.67 +2026-09-25 19:49:58.010 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=392674 infer_calls=99330 feedback_calls=99329 infer_wait=836.104s infer_obs_pack=1.702s env_step=55.293s feedback_total=11.419s feedback_obs_pack=2.752s feedback_info_pack=0.549s effective_fps=434.13 +2026-09-25 19:50:23.478 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=404875 infer_calls=102401 feedback_calls=102401 infer_wait=856.747s infer_obs_pack=1.745s env_step=56.770s feedback_total=11.658s feedback_obs_pack=2.809s feedback_info_pack=0.560s effective_fps=436.80 +2026-09-25 19:50:23.478 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 3229/409600 episodes. diff --git a/experiments/e28-dppo-learns/results/dppo-walker/server-seed0.log b/experiments/e28-dppo-learns/results/dppo-walker/server-seed0.log new file mode 100644 index 0000000..9b7a6d7 --- /dev/null +++ b/experiments/e28-dppo-learns/results/dppo-walker/server-seed0.log @@ -0,0 +1,26 @@ +19:34:32|INFO|plugrl_server version: 0.1.0 +19:34:32|INFO|Algorithm: dppo, Config: DPPOAlgoConfigWalker(global_steps=102400, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=1024, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=512, train_itrs=100, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +19:34:32|INFO|Policy: dppo-policy, Config: DPPOPolicyConfigWalker(algo='dppo', device=device(type='cpu'), env_type='gym', env_name='walker2d-medium-v2', checkpoint_path=None, critic=DPPOCriticObsConfig(mlp_dims=[256, 256, 256], activation='Mish', residual_style=True)) +19:34:32|INFO|Seeded python, numpy and torch with 0. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +19:34:32|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed0 +19:34:32|INFO|Number of network parameters: 568392 +19:34:32|INFO|Policy created... +19:34:32|INFO|Initialized RolloutBuffer buffer_size=1024 action_shape=(1024, 20, 4, 6) value_shape=(1024,) +19:34:32|INFO|Algorithm created: + +19:34:32|INFO|Agent Server is listening on 0.0.0.0:9510 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +19:37:36|INFO|Checkpoint saved at step 20481 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed0/20481 +19:40:47|INFO|Checkpoint saved at step 40961 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed0/40961 +19:43:56|INFO|Checkpoint saved at step 61441 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed0/61441 +19:47:07|INFO|Checkpoint saved at step 81921 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed0/81921 +19:50:14|INFO|Checkpoint saved at step 102401 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed0/102401 +19:50:14|INFO|Stopping server as the algorithm signaled to stop. +19:50:14|INFO|Shutdown started: aborting pending infer requests. +19:50:14|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +19:50:14|INFO|Shutdown closing 1 websocket connection(s). +19:50:14|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 39400). +19:50:14|INFO|WebSocket server closed. +19:50:14|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e28-dppo-learns/results/dppo-walker/server-seed1.log b/experiments/e28-dppo-learns/results/dppo-walker/server-seed1.log new file mode 100644 index 0000000..b45efc4 --- /dev/null +++ b/experiments/e28-dppo-learns/results/dppo-walker/server-seed1.log @@ -0,0 +1,26 @@ +19:34:37|INFO|plugrl_server version: 0.1.0 +19:34:37|INFO|Algorithm: dppo, Config: DPPOAlgoConfigWalker(global_steps=102400, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=1024, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=512, train_itrs=100, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +19:34:37|INFO|Policy: dppo-policy, Config: DPPOPolicyConfigWalker(algo='dppo', device=device(type='cpu'), env_type='gym', env_name='walker2d-medium-v2', checkpoint_path=None, critic=DPPOCriticObsConfig(mlp_dims=[256, 256, 256], activation='Mish', residual_style=True)) +19:34:37|INFO|Seeded python, numpy and torch with 1. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +19:34:37|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed1 +19:34:37|INFO|Number of network parameters: 568392 +19:34:37|INFO|Policy created... +19:34:37|INFO|Initialized RolloutBuffer buffer_size=1024 action_shape=(1024, 20, 4, 6) value_shape=(1024,) +19:34:37|INFO|Algorithm created: + +19:34:37|INFO|Agent Server is listening on 0.0.0.0:9511 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +19:37:44|INFO|Checkpoint saved at step 20481 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed1/20481 +19:40:58|INFO|Checkpoint saved at step 40961 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed1/40961 +19:44:09|INFO|Checkpoint saved at step 61441 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed1/61441 +19:47:19|INFO|Checkpoint saved at step 81921 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed1/81921 +19:50:25|INFO|Checkpoint saved at step 102401 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed1/102401 +19:50:25|INFO|Stopping server as the algorithm signaled to stop. +19:50:25|INFO|Shutdown started: aborting pending infer requests. +19:50:25|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +19:50:25|INFO|Shutdown closing 1 websocket connection(s). +19:50:25|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 48646). +19:50:25|INFO|WebSocket server closed. +19:50:25|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e28-dppo-learns/results/dppo-walker/server-seed2.log b/experiments/e28-dppo-learns/results/dppo-walker/server-seed2.log new file mode 100644 index 0000000..6bc6c56 --- /dev/null +++ b/experiments/e28-dppo-learns/results/dppo-walker/server-seed2.log @@ -0,0 +1,26 @@ +19:34:42|INFO|plugrl_server version: 0.1.0 +19:34:42|INFO|Algorithm: dppo, Config: DPPOAlgoConfigWalker(global_steps=102400, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=1024, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=512, train_itrs=100, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +19:34:42|INFO|Policy: dppo-policy, Config: DPPOPolicyConfigWalker(algo='dppo', device=device(type='cpu'), env_type='gym', env_name='walker2d-medium-v2', checkpoint_path=None, critic=DPPOCriticObsConfig(mlp_dims=[256, 256, 256], activation='Mish', residual_style=True)) +19:34:42|INFO|Seeded python, numpy and torch with 2. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +19:34:42|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed2 +19:34:42|INFO|Number of network parameters: 568392 +19:34:42|INFO|Policy created... +19:34:42|INFO|Initialized RolloutBuffer buffer_size=1024 action_shape=(1024, 20, 4, 6) value_shape=(1024,) +19:34:42|INFO|Algorithm created: + +19:34:42|INFO|Agent Server is listening on 0.0.0.0:9512 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +19:37:48|INFO|Checkpoint saved at step 20481 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed2/20481 +19:41:01|INFO|Checkpoint saved at step 40961 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed2/40961 +19:44:10|INFO|Checkpoint saved at step 61441 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed2/61441 +19:47:19|INFO|Checkpoint saved at step 81921 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed2/81921 +19:50:23|INFO|Checkpoint saved at step 102401 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed2/102401 +19:50:23|INFO|Stopping server as the algorithm signaled to stop. +19:50:23|INFO|Shutdown started: aborting pending infer requests. +19:50:23|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +19:50:23|INFO|Shutdown closing 1 websocket connection(s). +19:50:23|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 40658). +19:50:23|INFO|WebSocket server closed. +19:50:23|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e28-dppo-learns/results/fpodppo-hopper.out b/experiments/e28-dppo-learns/results/fpodppo-hopper.out new file mode 100644 index 0000000..b78b220 --- /dev/null +++ b/experiments/e28-dppo-learns/results/fpodppo-hopper.out @@ -0,0 +1,10 @@ +cell: fpodppo-hopper = fpo-policy/default x dppo/hopper x Hopper-v5 +iters: 100 x 4096 batch: variant default replan: 1 seeds: 0 1 2 +out: /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/fpodppo-hopper +start 2026-09-25 19:50:25 +seed 0 finished rc=0 at 20:10:26 +seed 2 finished rc=0 at 20:10:31 +seed 1 finished rc=0 at 20:10:51 +end 2026-09-25 20:10:51 +failed seeds: 0 +CELL_DONE fpodppo-hopper diff --git a/experiments/e28-dppo-learns/results/fpodppo-hopper/client-seed0.log b/experiments/e28-dppo-learns/results/fpodppo-hopper/client-seed0.log new file mode 100644 index 0000000..a3c2d8c --- /dev/null +++ b/experiments/e28-dppo-learns/results/fpodppo-hopper/client-seed0.log @@ -0,0 +1,49 @@ +2026-09-25 19:50:27.320 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 19:50:27.320 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 19:50:27.321 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 19:50:27.321 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 19:50:27.321 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 19:50:27.357 | INFO | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-195027-7b99849e output_dir=runs/mujoco-v1-nenv1-20260925-195027-7b99849e recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 19:50:27.389 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9530... +2026-09-25 19:50:27.391 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'FPOPolicy', 'action_dim': 3, 'action_horizon': 1} +2026-09-25 19:50:57.394 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=10067 infer_calls=10067 feedback_calls=10067 infer_wait=27.233s infer_obs_pack=0.141s env_step=1.391s feedback_total=0.716s feedback_obs_pack=0.183s feedback_info_pack=0.035s effective_fps=341.47 +2026-09-25 19:51:27.394 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=20296 infer_calls=20296 feedback_calls=20296 infer_wait=54.280s infer_obs_pack=0.304s env_step=2.881s feedback_total=1.455s feedback_obs_pack=0.382s feedback_info_pack=0.074s effective_fps=344.46 +2026-09-25 19:51:57.395 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=30322 infer_calls=30322 feedback_calls=30322 infer_wait=81.437s infer_obs_pack=0.453s env_step=4.322s feedback_total=2.170s feedback_obs_pack=0.573s feedback_info_pack=0.111s effective_fps=343.08 +2026-09-25 19:52:27.395 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=40545 infer_calls=40545 feedback_calls=40545 infer_wait=108.463s infer_obs_pack=0.608s env_step=5.835s feedback_total=2.918s feedback_obs_pack=0.773s feedback_info_pack=0.150s effective_fps=344.11 +2026-09-25 19:52:57.397 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=50874 infer_calls=50874 feedback_calls=50874 infer_wait=135.502s infer_obs_pack=0.764s env_step=7.326s feedback_total=3.669s feedback_obs_pack=0.972s feedback_info_pack=0.189s effective_fps=345.47 +2026-09-25 19:53:27.400 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=61325 infer_calls=61325 feedback_calls=61325 infer_wait=162.477s infer_obs_pack=0.920s env_step=8.849s feedback_total=4.450s feedback_obs_pack=1.174s feedback_info_pack=0.228s effective_fps=347.07 +2026-09-25 19:53:57.402 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=71522 infer_calls=71522 feedback_calls=71522 infer_wait=189.578s infer_obs_pack=1.069s env_step=10.316s feedback_total=5.193s feedback_obs_pack=1.372s feedback_info_pack=0.268s effective_fps=346.93 +2026-09-25 19:54:27.403 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=81795 infer_calls=81795 feedback_calls=81795 infer_wait=216.652s infer_obs_pack=1.221s env_step=11.793s feedback_total=5.941s feedback_obs_pack=1.567s feedback_info_pack=0.306s effective_fps=347.17 +2026-09-25 19:54:57.404 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=92046 infer_calls=92046 feedback_calls=92046 infer_wait=243.650s infer_obs_pack=1.376s env_step=13.315s feedback_total=6.697s feedback_obs_pack=1.765s feedback_info_pack=0.345s effective_fps=347.29 +2026-09-25 19:55:27.404 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=102372 infer_calls=102372 feedback_calls=102372 infer_wait=270.517s infer_obs_pack=1.544s env_step=14.917s feedback_total=7.477s feedback_obs_pack=1.977s feedback_info_pack=0.387s effective_fps=347.67 +2026-09-25 19:55:57.406 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=112519 infer_calls=112519 feedback_calls=112519 infer_wait=297.508s infer_obs_pack=1.699s env_step=16.453s feedback_total=8.230s feedback_obs_pack=2.177s feedback_info_pack=0.426s effective_fps=347.40 +2026-09-25 19:56:27.803 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=122882 infer_calls=122882 feedback_calls=122882 infer_wait=324.892s infer_obs_pack=1.858s env_step=17.985s feedback_total=8.985s feedback_obs_pack=2.379s feedback_info_pack=0.467s effective_fps=347.40 +2026-09-25 19:56:57.804 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=133266 infer_calls=133266 feedback_calls=133266 infer_wait=351.874s infer_obs_pack=2.018s env_step=19.517s feedback_total=9.746s feedback_obs_pack=2.580s feedback_info_pack=0.509s effective_fps=347.81 +2026-09-25 19:57:27.804 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=143420 infer_calls=143420 feedback_calls=143420 infer_wait=378.945s infer_obs_pack=2.174s env_step=21.005s feedback_total=10.482s feedback_obs_pack=2.776s feedback_info_pack=0.546s effective_fps=347.60 +2026-09-25 19:57:57.805 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=153646 infer_calls=153646 feedback_calls=153646 infer_wait=405.992s infer_obs_pack=2.324s env_step=22.521s feedback_total=11.217s feedback_obs_pack=2.975s feedback_info_pack=0.585s effective_fps=347.57 +2026-09-25 19:58:27.904 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=163842 infer_calls=163842 feedback_calls=163842 infer_wait=433.200s infer_obs_pack=2.473s env_step=23.995s feedback_total=11.939s feedback_obs_pack=3.168s feedback_info_pack=0.623s effective_fps=347.41 +2026-09-25 19:58:57.905 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=174161 infer_calls=174161 feedback_calls=174161 infer_wait=460.195s infer_obs_pack=2.632s env_step=25.525s feedback_total=12.688s feedback_obs_pack=3.370s feedback_info_pack=0.662s effective_fps=347.60 +2026-09-25 19:59:28.045 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=184322 infer_calls=184322 feedback_calls=184322 infer_wait=487.414s infer_obs_pack=2.787s env_step=27.006s feedback_total=13.431s feedback_obs_pack=3.564s feedback_info_pack=0.702s effective_fps=347.36 +2026-09-25 19:59:58.047 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=194578 infer_calls=194578 feedback_calls=194578 infer_wait=514.438s infer_obs_pack=2.940s env_step=28.525s feedback_total=14.173s feedback_obs_pack=3.762s feedback_info_pack=0.741s effective_fps=347.41 +2026-09-25 20:00:28.154 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=204802 infer_calls=204802 feedback_calls=204802 infer_wait=541.593s infer_obs_pack=3.101s env_step=30.026s feedback_total=14.912s feedback_obs_pack=3.958s feedback_info_pack=0.780s effective_fps=347.34 +2026-09-25 20:00:58.155 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=215129 infer_calls=215129 feedback_calls=215129 infer_wait=568.567s infer_obs_pack=3.262s env_step=31.565s feedback_total=15.658s feedback_obs_pack=4.158s feedback_info_pack=0.820s effective_fps=347.51 +2026-09-25 20:01:28.224 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=225282 infer_calls=225282 feedback_calls=225282 infer_wait=595.711s infer_obs_pack=3.414s env_step=33.048s feedback_total=16.399s feedback_obs_pack=4.354s feedback_info_pack=0.862s effective_fps=347.35 +2026-09-25 20:01:58.227 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=235641 infer_calls=235641 feedback_calls=235641 infer_wait=622.678s infer_obs_pack=3.572s env_step=34.597s feedback_total=17.157s feedback_obs_pack=4.555s feedback_info_pack=0.902s effective_fps=347.55 +2026-09-25 20:02:28.228 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=245838 infer_calls=245838 feedback_calls=245838 infer_wait=649.652s infer_obs_pack=3.728s env_step=36.129s feedback_total=17.934s feedback_obs_pack=4.760s feedback_info_pack=0.942s effective_fps=347.50 +2026-09-25 20:02:58.229 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=256214 infer_calls=256214 feedback_calls=256214 infer_wait=676.560s infer_obs_pack=3.896s env_step=37.696s feedback_total=18.715s feedback_obs_pack=4.964s feedback_info_pack=0.983s effective_fps=347.71 +2026-09-25 20:03:28.229 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=266310 infer_calls=266310 feedback_calls=266310 infer_wait=703.525s infer_obs_pack=4.053s env_step=39.246s feedback_total=19.471s feedback_obs_pack=5.166s feedback_info_pack=1.022s effective_fps=347.53 +2026-09-25 20:03:58.230 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=276607 infer_calls=276607 feedback_calls=276607 infer_wait=730.433s infer_obs_pack=4.213s env_step=40.833s feedback_total=20.242s feedback_obs_pack=5.373s feedback_info_pack=1.062s effective_fps=347.62 +2026-09-25 20:04:28.328 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=286722 infer_calls=286722 feedback_calls=286722 infer_wait=757.550s infer_obs_pack=4.372s env_step=42.343s feedback_total=21.002s feedback_obs_pack=5.572s feedback_info_pack=1.101s effective_fps=347.43 +2026-09-25 20:04:58.329 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=297112 infer_calls=297112 feedback_calls=297112 infer_wait=784.458s infer_obs_pack=4.535s env_step=43.930s feedback_total=21.769s feedback_obs_pack=5.783s feedback_info_pack=1.142s effective_fps=347.62 +2026-09-25 20:05:28.330 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=307299 infer_calls=307299 feedback_calls=307299 infer_wait=811.339s infer_obs_pack=4.695s env_step=45.544s feedback_total=22.531s feedback_obs_pack=5.988s feedback_info_pack=1.183s effective_fps=347.58 +2026-09-25 20:05:58.331 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=317716 infer_calls=317716 feedback_calls=317716 infer_wait=838.129s infer_obs_pack=4.867s env_step=47.184s feedback_total=23.329s feedback_obs_pack=6.201s feedback_info_pack=1.226s effective_fps=347.80 +2026-09-25 20:06:28.332 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=328011 infer_calls=328011 feedback_calls=328011 infer_wait=865.017s infer_obs_pack=5.027s env_step=48.777s feedback_total=24.110s feedback_obs_pack=6.409s feedback_info_pack=1.266s effective_fps=347.86 +2026-09-25 20:06:58.334 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=338455 infer_calls=338455 feedback_calls=338455 infer_wait=891.859s infer_obs_pack=5.190s env_step=50.387s feedback_total=24.907s feedback_obs_pack=6.619s feedback_info_pack=1.309s effective_fps=348.08 +2026-09-25 20:07:28.343 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=348672 infer_calls=348672 feedback_calls=348672 infer_wait=918.777s infer_obs_pack=5.352s env_step=51.975s feedback_total=25.677s feedback_obs_pack=6.822s feedback_info_pack=1.349s effective_fps=348.05 +2026-09-25 20:07:58.345 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=358988 infer_calls=358988 feedback_calls=358988 infer_wait=945.658s infer_obs_pack=5.513s env_step=53.576s feedback_total=26.449s feedback_obs_pack=7.035s feedback_info_pack=1.391s effective_fps=348.13 +2026-09-25 20:08:28.348 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=369151 infer_calls=369151 feedback_calls=369151 infer_wait=972.586s infer_obs_pack=5.673s env_step=55.161s feedback_total=27.214s feedback_obs_pack=7.238s feedback_info_pack=1.431s effective_fps=348.05 +2026-09-25 20:08:58.349 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=379565 infer_calls=379565 feedback_calls=379565 infer_wait=999.473s infer_obs_pack=5.837s env_step=56.751s feedback_total=27.997s feedback_obs_pack=7.449s feedback_info_pack=1.472s effective_fps=348.21 +2026-09-25 20:09:28.349 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=389719 infer_calls=389719 feedback_calls=389719 infer_wait=1026.446s infer_obs_pack=5.995s env_step=58.309s feedback_total=28.756s feedback_obs_pack=7.652s feedback_info_pack=1.511s effective_fps=348.12 +2026-09-25 20:09:58.352 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=399976 infer_calls=399976 feedback_calls=399976 infer_wait=1053.377s infer_obs_pack=6.154s env_step=59.890s feedback_total=29.517s feedback_obs_pack=7.863s feedback_info_pack=1.551s effective_fps=348.13 +2026-09-25 20:10:26.658 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=409601 infer_calls=409601 feedback_calls=409601 infer_wait=1078.292s infer_obs_pack=6.305s env_step=61.365s feedback_total=30.239s feedback_obs_pack=8.058s feedback_info_pack=1.589s effective_fps=348.24 +2026-09-25 20:10:26.659 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 17616/1638400 episodes. diff --git a/experiments/e28-dppo-learns/results/fpodppo-hopper/client-seed1.log b/experiments/e28-dppo-learns/results/fpodppo-hopper/client-seed1.log new file mode 100644 index 0000000..6b0503d --- /dev/null +++ b/experiments/e28-dppo-learns/results/fpodppo-hopper/client-seed1.log @@ -0,0 +1,50 @@ +2026-09-25 19:50:32.320 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 19:50:32.321 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 19:50:32.321 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 19:50:32.321 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 19:50:32.322 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 19:50:32.359 | INFO | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-195032-faba891b output_dir=runs/mujoco-v1-nenv1-20260925-195032-faba891b recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 19:50:32.391 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9531... +2026-09-25 19:50:32.394 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'FPOPolicy', 'action_dim': 3, 'action_horizon': 1} +2026-09-25 19:51:02.397 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=9766 infer_calls=9766 feedback_calls=9766 infer_wait=27.318s infer_obs_pack=0.140s env_step=1.363s feedback_total=0.679s feedback_obs_pack=0.185s feedback_info_pack=0.035s effective_fps=331.05 +2026-09-25 19:51:32.402 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=19781 infer_calls=19781 feedback_calls=19781 infer_wait=54.455s infer_obs_pack=0.302s env_step=2.808s feedback_total=1.398s feedback_obs_pack=0.379s feedback_info_pack=0.073s effective_fps=335.48 +2026-09-25 19:52:02.404 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=29739 infer_calls=29739 feedback_calls=29739 infer_wait=81.542s infer_obs_pack=0.457s env_step=4.287s feedback_total=2.134s feedback_obs_pack=0.578s feedback_info_pack=0.111s effective_fps=336.34 +2026-09-25 19:52:32.405 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=39809 infer_calls=39809 feedback_calls=39809 infer_wait=108.641s infer_obs_pack=0.610s env_step=5.754s feedback_total=2.869s feedback_obs_pack=0.778s feedback_info_pack=0.149s effective_fps=337.72 +2026-09-25 19:53:02.407 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=49848 infer_calls=49848 feedback_calls=49848 infer_wait=135.705s infer_obs_pack=0.767s env_step=7.232s feedback_total=3.623s feedback_obs_pack=0.979s feedback_info_pack=0.189s effective_fps=338.35 +2026-09-25 19:53:32.407 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=60092 infer_calls=60092 feedback_calls=60092 infer_wait=162.712s infer_obs_pack=0.928s env_step=8.751s feedback_total=4.381s feedback_obs_pack=1.179s feedback_info_pack=0.229s effective_fps=339.94 +2026-09-25 19:54:02.409 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=70178 infer_calls=70178 feedback_calls=70178 infer_wait=189.826s infer_obs_pack=1.079s env_step=10.209s feedback_total=5.110s feedback_obs_pack=1.379s feedback_info_pack=0.267s effective_fps=340.30 +2026-09-25 19:54:32.410 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=80235 infer_calls=80235 feedback_calls=80235 infer_wait=216.919s infer_obs_pack=1.229s env_step=11.692s feedback_total=5.830s feedback_obs_pack=1.578s feedback_info_pack=0.306s effective_fps=340.45 +2026-09-25 19:55:02.411 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=90310 infer_calls=90310 feedback_calls=90310 infer_wait=243.929s infer_obs_pack=1.387s env_step=13.212s feedback_total=6.581s feedback_obs_pack=1.783s feedback_info_pack=0.345s effective_fps=340.65 +2026-09-25 19:55:32.414 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=100637 infer_calls=100637 feedback_calls=100637 infer_wait=270.842s infer_obs_pack=1.545s env_step=14.779s feedback_total=7.358s feedback_obs_pack=1.996s feedback_info_pack=0.385s effective_fps=341.69 +2026-09-25 19:56:02.414 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=110708 infer_calls=110708 feedback_calls=110708 infer_wait=297.864s infer_obs_pack=1.702s env_step=16.301s feedback_total=8.098s feedback_obs_pack=2.203s feedback_info_pack=0.424s effective_fps=341.73 +2026-09-25 19:56:32.416 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=120939 infer_calls=120939 feedback_calls=120939 infer_wait=324.909s infer_obs_pack=1.861s env_step=17.800s feedback_total=8.842s feedback_obs_pack=2.404s feedback_info_pack=0.461s effective_fps=342.20 +2026-09-25 19:57:02.528 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=131074 infer_calls=131074 feedback_calls=131074 infer_wait=352.122s infer_obs_pack=2.013s env_step=19.265s feedback_total=9.582s feedback_obs_pack=2.605s feedback_info_pack=0.498s effective_fps=342.25 +2026-09-25 19:57:32.530 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=141340 infer_calls=141340 feedback_calls=141340 infer_wait=379.146s infer_obs_pack=2.175s env_step=20.776s feedback_total=10.327s feedback_obs_pack=2.806s feedback_info_pack=0.538s effective_fps=342.70 +2026-09-25 19:58:02.533 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=151455 infer_calls=151455 feedback_calls=151455 infer_wait=406.239s infer_obs_pack=2.333s env_step=22.253s feedback_total=11.047s feedback_obs_pack=3.005s feedback_info_pack=0.578s effective_fps=342.76 +2026-09-25 19:58:32.536 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=161485 infer_calls=161485 feedback_calls=161485 infer_wait=433.365s infer_obs_pack=2.487s env_step=23.725s feedback_total=11.755s feedback_obs_pack=3.200s feedback_info_pack=0.615s effective_fps=342.61 +2026-09-25 19:59:02.536 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=171656 infer_calls=171656 feedback_calls=171656 infer_wait=460.411s infer_obs_pack=2.641s env_step=25.222s feedback_total=12.499s feedback_obs_pack=3.405s feedback_info_pack=0.653s effective_fps=342.78 +2026-09-25 19:59:32.538 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=181643 infer_calls=181643 feedback_calls=181643 infer_wait=487.559s infer_obs_pack=2.791s env_step=26.664s feedback_total=13.221s feedback_obs_pack=3.600s feedback_info_pack=0.693s effective_fps=342.57 +2026-09-25 20:00:02.540 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=191824 infer_calls=191824 feedback_calls=191824 infer_wait=514.663s infer_obs_pack=2.946s env_step=28.126s feedback_total=13.957s feedback_obs_pack=3.801s feedback_info_pack=0.730s effective_fps=342.73 +2026-09-25 20:00:32.541 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=201898 infer_calls=201898 feedback_calls=201898 infer_wait=541.759s infer_obs_pack=3.100s env_step=29.589s feedback_total=14.710s feedback_obs_pack=3.997s feedback_info_pack=0.767s effective_fps=342.69 +2026-09-25 20:01:02.541 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=212151 infer_calls=212151 feedback_calls=212151 infer_wait=568.778s infer_obs_pack=3.262s env_step=31.087s feedback_total=15.476s feedback_obs_pack=4.197s feedback_info_pack=0.807s effective_fps=342.95 +2026-09-25 20:01:32.543 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=222078 infer_calls=222078 feedback_calls=222078 infer_wait=595.955s infer_obs_pack=3.413s env_step=32.513s feedback_total=16.194s feedback_obs_pack=4.390s feedback_info_pack=0.844s effective_fps=342.67 +2026-09-25 20:02:02.546 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=232294 infer_calls=232294 feedback_calls=232294 infer_wait=622.987s infer_obs_pack=3.567s env_step=34.025s feedback_total=16.940s feedback_obs_pack=4.593s feedback_info_pack=0.882s effective_fps=342.86 +2026-09-25 20:02:32.548 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=242240 infer_calls=242240 feedback_calls=242240 infer_wait=650.010s infer_obs_pack=3.725s env_step=35.531s feedback_total=17.689s feedback_obs_pack=4.799s feedback_info_pack=0.922s effective_fps=342.65 +2026-09-25 20:03:02.550 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=252308 infer_calls=252308 feedback_calls=252308 infer_wait=677.016s infer_obs_pack=3.880s env_step=37.054s feedback_total=18.436s feedback_obs_pack=5.004s feedback_info_pack=0.962s effective_fps=342.63 +2026-09-25 20:03:32.553 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=262196 infer_calls=262196 feedback_calls=262196 infer_wait=704.041s infer_obs_pack=4.033s env_step=38.568s feedback_total=19.176s feedback_obs_pack=5.210s feedback_info_pack=1.003s effective_fps=342.37 +2026-09-25 20:04:02.556 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=272296 infer_calls=272296 feedback_calls=272296 infer_wait=731.020s infer_obs_pack=4.190s env_step=40.118s feedback_total=19.929s feedback_obs_pack=5.422s feedback_info_pack=1.043s effective_fps=342.40 +2026-09-25 20:04:32.558 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=282305 infer_calls=282305 feedback_calls=282305 infer_wait=758.112s infer_obs_pack=4.343s env_step=41.608s feedback_total=20.651s feedback_obs_pack=5.619s feedback_info_pack=1.081s effective_fps=342.31 +2026-09-25 20:05:02.560 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=292340 infer_calls=292340 feedback_calls=292340 infer_wait=785.107s infer_obs_pack=4.500s env_step=43.134s feedback_total=21.413s feedback_obs_pack=5.822s feedback_info_pack=1.123s effective_fps=342.26 +2026-09-25 20:05:32.562 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=302538 infer_calls=302538 feedback_calls=302538 infer_wait=812.025s infer_obs_pack=4.663s env_step=44.702s feedback_total=22.190s feedback_obs_pack=6.035s feedback_info_pack=1.163s effective_fps=342.40 +2026-09-25 20:06:02.563 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=312603 infer_calls=312603 feedback_calls=312603 infer_wait=839.035s infer_obs_pack=4.818s env_step=46.219s feedback_total=22.949s feedback_obs_pack=6.238s feedback_info_pack=1.204s effective_fps=342.38 +2026-09-25 20:06:32.566 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=322860 infer_calls=322860 feedback_calls=322860 infer_wait=865.965s infer_obs_pack=4.980s env_step=47.773s feedback_total=23.725s feedback_obs_pack=6.445s feedback_info_pack=1.243s effective_fps=342.58 +2026-09-25 20:07:02.570 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=332798 infer_calls=332798 feedback_calls=332798 infer_wait=892.934s infer_obs_pack=5.138s env_step=49.321s feedback_total=24.485s feedback_obs_pack=6.654s feedback_info_pack=1.286s effective_fps=342.43 +2026-09-25 20:07:32.571 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=343041 infer_calls=343041 feedback_calls=343041 infer_wait=919.856s infer_obs_pack=5.296s env_step=50.903s feedback_total=25.251s feedback_obs_pack=6.865s feedback_info_pack=1.326s effective_fps=342.59 +2026-09-25 20:08:02.574 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=353034 infer_calls=353034 feedback_calls=353034 infer_wait=946.805s infer_obs_pack=5.454s env_step=52.457s feedback_total=26.029s feedback_obs_pack=7.078s feedback_info_pack=1.365s effective_fps=342.50 +2026-09-25 20:08:32.575 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=363092 infer_calls=363092 feedback_calls=363092 infer_wait=973.759s infer_obs_pack=5.615s env_step=54.020s feedback_total=26.786s feedback_obs_pack=7.289s feedback_info_pack=1.405s effective_fps=342.48 +2026-09-25 20:09:02.578 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=373134 infer_calls=373134 feedback_calls=373134 infer_wait=1000.783s infer_obs_pack=5.769s env_step=55.532s feedback_total=27.547s feedback_obs_pack=7.495s feedback_info_pack=1.445s effective_fps=342.44 +2026-09-25 20:09:32.579 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=383215 infer_calls=383215 feedback_calls=383215 infer_wait=1027.813s infer_obs_pack=5.926s env_step=57.055s feedback_total=28.284s feedback_obs_pack=7.699s feedback_info_pack=1.483s effective_fps=342.44 +2026-09-25 20:10:02.757 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=393218 infer_calls=393218 feedback_calls=393218 infer_wait=1054.978s infer_obs_pack=6.088s env_step=58.590s feedback_total=29.039s feedback_obs_pack=7.907s feedback_info_pack=1.524s effective_fps=342.32 +2026-09-25 20:10:32.761 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=403398 infer_calls=403398 feedback_calls=403398 infer_wait=1081.947s infer_obs_pack=6.248s env_step=60.152s feedback_total=29.787s feedback_obs_pack=8.114s feedback_info_pack=1.564s effective_fps=342.40 +2026-09-25 20:10:51.757 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=409601 infer_calls=409601 feedback_calls=409601 infer_wait=1098.541s infer_obs_pack=6.346s env_step=61.117s feedback_total=30.254s feedback_obs_pack=8.245s feedback_info_pack=1.590s effective_fps=342.40 +2026-09-25 20:10:51.758 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 17932/1638400 episodes. diff --git a/experiments/e28-dppo-learns/results/fpodppo-hopper/client-seed2.log b/experiments/e28-dppo-learns/results/fpodppo-hopper/client-seed2.log new file mode 100644 index 0000000..44c32b4 --- /dev/null +++ b/experiments/e28-dppo-learns/results/fpodppo-hopper/client-seed2.log @@ -0,0 +1,49 @@ +2026-09-25 19:50:37.321 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 19:50:37.321 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 19:50:37.322 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 19:50:37.322 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 19:50:37.322 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 19:50:37.361 | INFO | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-195037-1940e96e output_dir=runs/mujoco-v1-nenv1-20260925-195037-1940e96e recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 19:50:37.395 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9532... +2026-09-25 19:50:37.398 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'FPOPolicy', 'action_dim': 3, 'action_horizon': 1} +2026-09-25 19:51:07.404 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=9879 infer_calls=9879 feedback_calls=9879 infer_wait=27.209s infer_obs_pack=0.141s env_step=1.433s feedback_total=0.691s feedback_obs_pack=0.187s feedback_info_pack=0.041s effective_fps=335.18 +2026-09-25 19:51:37.404 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=20168 infer_calls=20168 feedback_calls=20168 infer_wait=54.278s infer_obs_pack=0.300s env_step=2.912s feedback_total=1.441s feedback_obs_pack=0.382s feedback_info_pack=0.083s effective_fps=342.23 +2026-09-25 19:52:07.406 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=30262 infer_calls=30262 feedback_calls=30262 infer_wait=81.345s infer_obs_pack=0.451s env_step=4.407s feedback_total=2.185s feedback_obs_pack=0.578s feedback_info_pack=0.123s effective_fps=342.38 +2026-09-25 19:52:37.408 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=40556 infer_calls=40556 feedback_calls=40556 infer_wait=108.323s infer_obs_pack=0.612s env_step=5.947s feedback_total=2.941s feedback_obs_pack=0.779s feedback_info_pack=0.166s effective_fps=344.21 +2026-09-25 19:53:07.410 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=50911 infer_calls=50911 feedback_calls=50911 infer_wait=135.325s infer_obs_pack=0.767s env_step=7.463s feedback_total=3.714s feedback_obs_pack=0.986s feedback_info_pack=0.208s effective_fps=345.70 +2026-09-25 19:53:37.412 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=61397 infer_calls=61397 feedback_calls=61397 infer_wait=162.309s infer_obs_pack=0.924s env_step=8.999s feedback_total=4.475s feedback_obs_pack=1.189s feedback_info_pack=0.249s effective_fps=347.45 +2026-09-25 19:54:07.414 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=71648 infer_calls=71648 feedback_calls=71648 infer_wait=189.348s infer_obs_pack=1.074s env_step=10.507s feedback_total=5.217s feedback_obs_pack=1.390s feedback_info_pack=0.289s effective_fps=347.56 +2026-09-25 19:54:37.589 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=81922 infer_calls=81922 feedback_calls=81922 infer_wait=216.560s infer_obs_pack=1.226s env_step=12.014s feedback_total=5.967s feedback_obs_pack=1.587s feedback_info_pack=0.329s effective_fps=347.47 +2026-09-25 19:55:07.591 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=92484 infer_calls=92484 feedback_calls=92484 infer_wait=243.421s infer_obs_pack=1.388s env_step=13.612s feedback_total=6.757s feedback_obs_pack=1.799s feedback_info_pack=0.372s effective_fps=348.76 +2026-09-25 19:55:37.593 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=102770 infer_calls=102770 feedback_calls=102770 infer_wait=270.281s infer_obs_pack=1.549s env_step=15.204s feedback_total=7.557s feedback_obs_pack=2.014s feedback_info_pack=0.415s effective_fps=348.86 +2026-09-25 19:56:07.594 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=113224 infer_calls=113224 feedback_calls=113224 infer_wait=297.169s infer_obs_pack=1.709s env_step=16.793s feedback_total=8.328s feedback_obs_pack=2.227s feedback_info_pack=0.458s effective_fps=349.46 +2026-09-25 19:56:37.594 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=123519 infer_calls=123519 feedback_calls=123519 infer_wait=324.203s infer_obs_pack=1.867s env_step=18.294s feedback_total=9.077s feedback_obs_pack=2.428s feedback_info_pack=0.498s effective_fps=349.47 +2026-09-25 19:57:07.594 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=133882 infer_calls=133882 feedback_calls=133882 infer_wait=351.219s infer_obs_pack=2.019s env_step=19.813s feedback_total=9.834s feedback_obs_pack=2.633s feedback_info_pack=0.539s effective_fps=349.67 +2026-09-25 19:57:37.595 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=144146 infer_calls=144146 feedback_calls=144146 infer_wait=378.237s infer_obs_pack=2.176s env_step=21.326s feedback_total=10.584s feedback_obs_pack=2.832s feedback_info_pack=0.579s effective_fps=349.60 +2026-09-25 19:58:07.598 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=154357 infer_calls=154357 feedback_calls=154357 infer_wait=405.244s infer_obs_pack=2.330s env_step=22.870s feedback_total=11.321s feedback_obs_pack=3.038s feedback_info_pack=0.621s effective_fps=349.41 +2026-09-25 19:58:37.600 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=164652 infer_calls=164652 feedback_calls=164652 infer_wait=432.273s infer_obs_pack=2.483s env_step=24.371s feedback_total=12.080s feedback_obs_pack=3.237s feedback_info_pack=0.662s effective_fps=349.43 +2026-09-25 19:59:07.602 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=175142 infer_calls=175142 feedback_calls=175142 infer_wait=459.264s infer_obs_pack=2.639s env_step=25.900s feedback_total=12.848s feedback_obs_pack=3.441s feedback_info_pack=0.703s effective_fps=349.83 +2026-09-25 19:59:37.603 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=185234 infer_calls=185234 feedback_calls=185234 infer_wait=486.384s infer_obs_pack=2.788s env_step=27.369s feedback_total=13.565s feedback_obs_pack=3.633s feedback_info_pack=0.742s effective_fps=349.43 +2026-09-25 20:00:07.606 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=195602 infer_calls=195602 feedback_calls=195602 infer_wait=513.401s infer_obs_pack=2.942s env_step=28.887s feedback_total=14.315s feedback_obs_pack=3.830s feedback_info_pack=0.784s effective_fps=349.57 +2026-09-25 20:00:37.608 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=205770 infer_calls=205770 feedback_calls=205770 infer_wait=540.499s infer_obs_pack=3.094s env_step=30.363s feedback_total=15.050s feedback_obs_pack=4.024s feedback_info_pack=0.823s effective_fps=349.35 +2026-09-25 20:01:07.609 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=216054 infer_calls=216054 feedback_calls=216054 infer_wait=567.474s infer_obs_pack=3.250s env_step=31.916s feedback_total=15.801s feedback_obs_pack=4.231s feedback_info_pack=0.864s effective_fps=349.35 +2026-09-25 20:01:37.610 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=226081 infer_calls=226081 feedback_calls=226081 infer_wait=594.575s infer_obs_pack=3.402s env_step=33.394s feedback_total=16.528s feedback_obs_pack=4.427s feedback_info_pack=0.903s effective_fps=348.94 +2026-09-25 20:02:07.614 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=236506 infer_calls=236506 feedback_calls=236506 infer_wait=621.533s infer_obs_pack=3.560s env_step=34.946s feedback_total=17.290s feedback_obs_pack=4.632s feedback_info_pack=0.944s effective_fps=349.17 +2026-09-25 20:02:37.614 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=246710 infer_calls=246710 feedback_calls=246710 infer_wait=648.489s infer_obs_pack=3.718s env_step=36.503s feedback_total=18.046s feedback_obs_pack=4.836s feedback_info_pack=0.985s effective_fps=349.07 +2026-09-25 20:03:07.617 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=257055 infer_calls=257055 feedback_calls=257055 infer_wait=675.416s infer_obs_pack=3.874s env_step=38.077s feedback_total=18.812s feedback_obs_pack=5.041s feedback_info_pack=1.027s effective_fps=349.17 +2026-09-25 20:03:37.618 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=267242 infer_calls=267242 feedback_calls=267242 infer_wait=702.369s infer_obs_pack=4.033s env_step=39.624s feedback_total=19.584s feedback_obs_pack=5.245s feedback_info_pack=1.070s effective_fps=349.06 +2026-09-25 20:04:07.620 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=277546 infer_calls=277546 feedback_calls=277546 infer_wait=729.297s infer_obs_pack=4.191s env_step=41.187s feedback_total=20.356s feedback_obs_pack=5.454s feedback_info_pack=1.112s effective_fps=349.10 +2026-09-25 20:04:37.622 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=287668 infer_calls=287668 feedback_calls=287668 infer_wait=756.321s infer_obs_pack=4.347s env_step=42.695s feedback_total=21.112s feedback_obs_pack=5.654s feedback_info_pack=1.152s effective_fps=348.91 +2026-09-25 20:05:07.624 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=298145 infer_calls=298145 feedback_calls=298145 infer_wait=783.173s infer_obs_pack=4.512s env_step=44.297s feedback_total=21.903s feedback_obs_pack=5.864s feedback_info_pack=1.196s effective_fps=349.16 +2026-09-25 20:05:37.626 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=308295 infer_calls=308295 feedback_calls=308295 infer_wait=810.056s infer_obs_pack=4.673s env_step=45.894s feedback_total=22.684s feedback_obs_pack=6.070s feedback_info_pack=1.239s effective_fps=349.02 +2026-09-25 20:06:07.627 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=318756 infer_calls=318756 feedback_calls=318756 infer_wait=836.872s infer_obs_pack=4.840s env_step=47.518s feedback_total=23.479s feedback_obs_pack=6.286s feedback_info_pack=1.283s effective_fps=349.24 +2026-09-25 20:06:37.630 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=329058 infer_calls=329058 feedback_calls=329058 infer_wait=863.804s infer_obs_pack=5.001s env_step=49.085s feedback_total=24.251s feedback_obs_pack=6.489s feedback_info_pack=1.325s effective_fps=349.27 +2026-09-25 20:07:07.632 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=339524 infer_calls=339524 feedback_calls=339524 infer_wait=890.651s infer_obs_pack=5.161s env_step=50.711s feedback_total=25.029s feedback_obs_pack=6.699s feedback_info_pack=1.368s effective_fps=349.47 +2026-09-25 20:07:37.632 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=349828 infer_calls=349828 feedback_calls=349828 infer_wait=917.521s infer_obs_pack=5.323s env_step=52.319s feedback_total=25.810s feedback_obs_pack=6.909s feedback_info_pack=1.412s effective_fps=349.49 +2026-09-25 20:08:07.632 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=360229 infer_calls=360229 feedback_calls=360229 infer_wait=944.387s infer_obs_pack=5.483s env_step=53.917s feedback_total=26.608s feedback_obs_pack=7.125s feedback_info_pack=1.455s effective_fps=349.60 +2026-09-25 20:08:37.634 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=370510 infer_calls=370510 feedback_calls=370510 infer_wait=971.362s infer_obs_pack=5.636s env_step=55.469s feedback_total=27.368s feedback_obs_pack=7.328s feedback_info_pack=1.496s effective_fps=349.59 +2026-09-25 20:09:07.636 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=380868 infer_calls=380868 feedback_calls=380868 infer_wait=998.270s infer_obs_pack=5.794s env_step=57.062s feedback_total=28.135s feedback_obs_pack=7.541s feedback_info_pack=1.539s effective_fps=349.66 +2026-09-25 20:09:37.636 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=391093 infer_calls=391093 feedback_calls=391093 infer_wait=1025.200s infer_obs_pack=5.958s env_step=58.647s feedback_total=28.895s feedback_obs_pack=7.744s feedback_info_pack=1.580s effective_fps=349.60 +2026-09-25 20:10:07.744 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=401410 infer_calls=401410 feedback_calls=401410 infer_wait=1052.251s infer_obs_pack=6.115s env_step=60.213s feedback_total=29.674s feedback_obs_pack=7.950s feedback_info_pack=1.621s effective_fps=349.58 +2026-09-25 20:10:31.490 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=409601 infer_calls=409601 feedback_calls=409601 infer_wait=1072.900s infer_obs_pack=6.248s env_step=61.494s feedback_total=30.291s feedback_obs_pack=8.120s feedback_info_pack=1.656s effective_fps=349.81 +2026-09-25 20:10:31.491 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 17579/1638400 episodes. diff --git a/experiments/e28-dppo-learns/results/fpodppo-hopper/server-seed0.log b/experiments/e28-dppo-learns/results/fpodppo-hopper/server-seed0.log new file mode 100644 index 0000000..4d24757 --- /dev/null +++ b/experiments/e28-dppo-learns/results/fpodppo-hopper/server-seed0.log @@ -0,0 +1,25 @@ +19:50:26|INFO|plugrl_server version: 0.1.0 +19:50:26|INFO|Algorithm: dppo, Config: DPPOAlgoConfigHopper(global_steps=409600, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=4096, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=2048, train_itrs=100, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +19:50:26|INFO|Policy: fpo-policy, Config: FPOPolicyConfig(algo='dppo', device=device(type='cpu'), obs_dim=11, action_dim=3, flow_steps=10, timestep_embed_dim=8, action_horizon=1, feather_std=0.0, policy_mlp_output_scale=0.25, normalize_observations=True, hidden_dims=(32, 32, 32, 32), value_hidden_dims=(256, 256, 256, 256, 256)) +19:50:26|INFO|Seeded python, numpy and torch with 0. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +19:50:26|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/fpodppo-hopper/dppo/fpo-policy/fpodppo-hopper-seed0 +19:50:26|INFO|Policy created... +19:50:26|INFO|Initialized RolloutBuffer buffer_size=4096 action_shape=(4096, 10, 1, 3) value_shape=(4096,) +19:50:26|INFO|Algorithm created: + +19:50:26|INFO|Agent Server is listening on 0.0.0.0:9530 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +19:54:28|INFO|Checkpoint saved at step 81921 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/fpodppo-hopper/dppo/fpo-policy/fpodppo-hopper-seed0/81921 +19:58:27|INFO|Checkpoint saved at step 163841 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/fpodppo-hopper/dppo/fpo-policy/fpodppo-hopper-seed0/163841 +20:02:28|INFO|Checkpoint saved at step 245761 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/fpodppo-hopper/dppo/fpo-policy/fpodppo-hopper-seed0/245761 +20:06:27|INFO|Checkpoint saved at step 327681 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/fpodppo-hopper/dppo/fpo-policy/fpodppo-hopper-seed0/327681 +20:10:26|INFO|Checkpoint saved at step 409601 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/fpodppo-hopper/dppo/fpo-policy/fpodppo-hopper-seed0/409601 +20:10:26|INFO|Stopping server as the algorithm signaled to stop. +20:10:26|INFO|Shutdown started: aborting pending infer requests. +20:10:26|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +20:10:26|INFO|Shutdown closing 1 websocket connection(s). +20:10:26|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 39742). +20:10:26|INFO|WebSocket server closed. +20:10:26|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e28-dppo-learns/results/fpodppo-hopper/server-seed1.log b/experiments/e28-dppo-learns/results/fpodppo-hopper/server-seed1.log new file mode 100644 index 0000000..f402c9d --- /dev/null +++ b/experiments/e28-dppo-learns/results/fpodppo-hopper/server-seed1.log @@ -0,0 +1,25 @@ +19:50:31|INFO|plugrl_server version: 0.1.0 +19:50:31|INFO|Algorithm: dppo, Config: DPPOAlgoConfigHopper(global_steps=409600, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=4096, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=2048, train_itrs=100, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +19:50:31|INFO|Policy: fpo-policy, Config: FPOPolicyConfig(algo='dppo', device=device(type='cpu'), obs_dim=11, action_dim=3, flow_steps=10, timestep_embed_dim=8, action_horizon=1, feather_std=0.0, policy_mlp_output_scale=0.25, normalize_observations=True, hidden_dims=(32, 32, 32, 32), value_hidden_dims=(256, 256, 256, 256, 256)) +19:50:31|INFO|Seeded python, numpy and torch with 1. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +19:50:31|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/fpodppo-hopper/dppo/fpo-policy/fpodppo-hopper-seed1 +19:50:31|INFO|Policy created... +19:50:31|INFO|Initialized RolloutBuffer buffer_size=4096 action_shape=(4096, 10, 1, 3) value_shape=(4096,) +19:50:31|INFO|Algorithm created: + +19:50:31|INFO|Agent Server is listening on 0.0.0.0:9531 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +19:54:37|INFO|Checkpoint saved at step 81921 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/fpodppo-hopper/dppo/fpo-policy/fpodppo-hopper-seed1/81921 +19:58:39|INFO|Checkpoint saved at step 163841 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/fpodppo-hopper/dppo/fpo-policy/fpodppo-hopper-seed1/163841 +20:02:43|INFO|Checkpoint saved at step 245761 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/fpodppo-hopper/dppo/fpo-policy/fpodppo-hopper-seed1/245761 +20:06:47|INFO|Checkpoint saved at step 327681 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/fpodppo-hopper/dppo/fpo-policy/fpodppo-hopper-seed1/327681 +20:10:51|INFO|Checkpoint saved at step 409601 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/fpodppo-hopper/dppo/fpo-policy/fpodppo-hopper-seed1/409601 +20:10:51|INFO|Stopping server as the algorithm signaled to stop. +20:10:51|INFO|Shutdown started: aborting pending infer requests. +20:10:51|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +20:10:51|INFO|Shutdown closing 1 websocket connection(s). +20:10:51|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 35836). +20:10:51|INFO|WebSocket server closed. +20:10:51|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e28-dppo-learns/results/fpodppo-hopper/server-seed2.log b/experiments/e28-dppo-learns/results/fpodppo-hopper/server-seed2.log new file mode 100644 index 0000000..11900c9 --- /dev/null +++ b/experiments/e28-dppo-learns/results/fpodppo-hopper/server-seed2.log @@ -0,0 +1,25 @@ +19:50:36|INFO|plugrl_server version: 0.1.0 +19:50:36|INFO|Algorithm: dppo, Config: DPPOAlgoConfigHopper(global_steps=409600, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=4096, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=2048, train_itrs=100, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +19:50:36|INFO|Policy: fpo-policy, Config: FPOPolicyConfig(algo='dppo', device=device(type='cpu'), obs_dim=11, action_dim=3, flow_steps=10, timestep_embed_dim=8, action_horizon=1, feather_std=0.0, policy_mlp_output_scale=0.25, normalize_observations=True, hidden_dims=(32, 32, 32, 32), value_hidden_dims=(256, 256, 256, 256, 256)) +19:50:36|INFO|Seeded python, numpy and torch with 2. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +19:50:36|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/fpodppo-hopper/dppo/fpo-policy/fpodppo-hopper-seed2 +19:50:36|INFO|Policy created... +19:50:36|INFO|Initialized RolloutBuffer buffer_size=4096 action_shape=(4096, 10, 1, 3) value_shape=(4096,) +19:50:36|INFO|Algorithm created: + +19:50:36|INFO|Agent Server is listening on 0.0.0.0:9532 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +19:54:37|INFO|Checkpoint saved at step 81921 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/fpodppo-hopper/dppo/fpo-policy/fpodppo-hopper-seed2/81921 +19:58:35|INFO|Checkpoint saved at step 163841 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/fpodppo-hopper/dppo/fpo-policy/fpodppo-hopper-seed2/163841 +20:02:35|INFO|Checkpoint saved at step 245761 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/fpodppo-hopper/dppo/fpo-policy/fpodppo-hopper-seed2/245761 +20:06:33|INFO|Checkpoint saved at step 327681 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/fpodppo-hopper/dppo/fpo-policy/fpodppo-hopper-seed2/327681 +20:10:31|INFO|Checkpoint saved at step 409601 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/fpodppo-hopper/dppo/fpo-policy/fpodppo-hopper-seed2/409601 +20:10:31|INFO|Stopping server as the algorithm signaled to stop. +20:10:31|INFO|Shutdown started: aborting pending infer requests. +20:10:31|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +20:10:31|INFO|Shutdown closing 1 websocket connection(s). +20:10:31|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 60916). +20:10:31|INFO|WebSocket server closed. +20:10:31|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e28-dppo-learns/results/fpodppo-walker.out b/experiments/e28-dppo-learns/results/fpodppo-walker.out new file mode 100644 index 0000000..72eea89 --- /dev/null +++ b/experiments/e28-dppo-learns/results/fpodppo-walker.out @@ -0,0 +1,10 @@ +cell: fpodppo-walker = fpo-policy/default x dppo/walker x Walker2d-v5 +iters: 100 x 4096 batch: variant default replan: 1 seeds: 0 1 2 +out: /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/fpodppo-walker +start 2026-09-25 19:50:45 +seed 1 finished rc=0 at 20:10:46 +seed 0 finished rc=0 at 20:11:02 +seed 2 finished rc=0 at 20:11:12 +end 2026-09-25 20:11:12 +failed seeds: 0 +CELL_DONE fpodppo-walker diff --git a/experiments/e28-dppo-learns/results/fpodppo-walker/client-seed0.log b/experiments/e28-dppo-learns/results/fpodppo-walker/client-seed0.log new file mode 100644 index 0000000..e38503a --- /dev/null +++ b/experiments/e28-dppo-learns/results/fpodppo-walker/client-seed0.log @@ -0,0 +1,50 @@ +2026-09-25 19:50:47.317 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 19:50:47.318 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 19:50:47.318 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 19:50:47.318 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 19:50:47.319 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 19:50:47.358 | INFO | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-195047-c43dc031 output_dir=runs/mujoco-v1-nenv1-20260925-195047-c43dc031 recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 19:50:47.392 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9540... +2026-09-25 19:50:47.395 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'FPOPolicy', 'action_dim': 6, 'action_horizon': 1} +2026-09-25 19:51:17.397 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=9938 infer_calls=9938 feedback_calls=9938 infer_wait=27.117s infer_obs_pack=0.142s env_step=1.505s feedback_total=0.707s feedback_obs_pack=0.190s feedback_info_pack=0.041s effective_fps=337.21 +2026-09-25 19:51:47.399 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=19966 infer_calls=19966 feedback_calls=19966 infer_wait=54.168s infer_obs_pack=0.301s env_step=3.019s feedback_total=1.438s feedback_obs_pack=0.385s feedback_info_pack=0.082s effective_fps=338.83 +2026-09-25 19:52:17.401 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=29933 infer_calls=29933 feedback_calls=29933 infer_wait=81.202s infer_obs_pack=0.452s env_step=4.545s feedback_total=2.184s feedback_obs_pack=0.585s feedback_info_pack=0.127s effective_fps=338.67 +2026-09-25 19:52:47.402 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=40112 infer_calls=40112 feedback_calls=40112 infer_wait=108.175s infer_obs_pack=0.607s env_step=6.110s feedback_total=2.930s feedback_obs_pack=0.789s feedback_info_pack=0.171s effective_fps=340.45 +2026-09-25 19:53:17.406 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=50153 infer_calls=50153 feedback_calls=50153 infer_wait=135.131s infer_obs_pack=0.761s env_step=7.690s feedback_total=3.669s feedback_obs_pack=0.990s feedback_info_pack=0.214s effective_fps=340.60 +2026-09-25 19:53:47.407 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=60476 infer_calls=60476 feedback_calls=60476 infer_wait=162.060s infer_obs_pack=0.914s env_step=9.271s feedback_total=4.442s feedback_obs_pack=1.195s feedback_info_pack=0.259s effective_fps=342.28 +2026-09-25 19:54:17.408 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=70439 infer_calls=70439 feedback_calls=70439 infer_wait=189.076s infer_obs_pack=1.069s env_step=10.814s feedback_total=5.179s feedback_obs_pack=1.394s feedback_info_pack=0.300s effective_fps=341.71 +2026-09-25 19:54:47.408 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=80652 infer_calls=80652 feedback_calls=80652 infer_wait=215.972s infer_obs_pack=1.228s env_step=12.413s feedback_total=5.947s feedback_obs_pack=1.601s feedback_info_pack=0.344s effective_fps=342.38 +2026-09-25 19:55:17.410 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=90746 infer_calls=90746 feedback_calls=90746 infer_wait=242.824s infer_obs_pack=1.389s env_step=14.027s feedback_total=6.737s feedback_obs_pack=1.808s feedback_info_pack=0.388s effective_fps=342.47 +2026-09-25 19:55:47.411 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=101005 infer_calls=101005 feedback_calls=101005 infer_wait=269.655s infer_obs_pack=1.548s env_step=15.677s feedback_total=7.510s feedback_obs_pack=2.018s feedback_info_pack=0.434s effective_fps=343.10 +2026-09-25 19:56:17.416 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=111062 infer_calls=111062 feedback_calls=111062 infer_wait=296.642s infer_obs_pack=1.705s env_step=17.224s feedback_total=8.263s feedback_obs_pack=2.220s feedback_info_pack=0.476s effective_fps=342.96 +2026-09-25 19:56:47.416 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=121341 infer_calls=121341 feedback_calls=121341 infer_wait=323.591s infer_obs_pack=1.862s env_step=18.790s feedback_total=9.032s feedback_obs_pack=2.421s feedback_info_pack=0.518s effective_fps=343.47 +2026-09-25 19:57:17.416 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=131323 infer_calls=131323 feedback_calls=131323 infer_wait=350.604s infer_obs_pack=2.015s env_step=20.337s feedback_total=9.761s feedback_obs_pack=2.624s feedback_info_pack=0.560s effective_fps=343.13 +2026-09-25 19:57:47.418 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=141552 infer_calls=141552 feedback_calls=141552 infer_wait=377.563s infer_obs_pack=2.170s env_step=21.902s feedback_total=10.523s feedback_obs_pack=2.825s feedback_info_pack=0.602s effective_fps=343.44 +2026-09-25 19:58:17.421 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=151563 infer_calls=151563 feedback_calls=151563 infer_wait=404.584s infer_obs_pack=2.323s env_step=23.448s feedback_total=11.258s feedback_obs_pack=3.022s feedback_info_pack=0.644s effective_fps=343.20 +2026-09-25 19:58:47.421 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=161837 infer_calls=161837 feedback_calls=161837 infer_wait=431.485s infer_obs_pack=2.478s env_step=25.058s feedback_total=12.007s feedback_obs_pack=3.230s feedback_info_pack=0.687s effective_fps=343.58 +2026-09-25 19:59:17.716 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=172034 infer_calls=172034 feedback_calls=172034 infer_wait=458.776s infer_obs_pack=2.631s env_step=26.603s feedback_total=12.756s feedback_obs_pack=3.431s feedback_info_pack=0.729s effective_fps=343.54 +2026-09-25 19:59:47.717 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=182248 infer_calls=182248 feedback_calls=182248 infer_wait=485.710s infer_obs_pack=2.786s env_step=28.178s feedback_total=13.526s feedback_obs_pack=3.636s feedback_info_pack=0.772s effective_fps=343.73 +2026-09-25 20:00:17.718 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=192396 infer_calls=192396 feedback_calls=192396 infer_wait=512.670s infer_obs_pack=2.943s env_step=29.740s feedback_total=14.283s feedback_obs_pack=3.840s feedback_info_pack=0.816s effective_fps=343.79 +2026-09-25 20:00:47.719 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=202480 infer_calls=202480 feedback_calls=202480 infer_wait=539.645s infer_obs_pack=3.097s env_step=31.305s feedback_total=15.033s feedback_obs_pack=4.045s feedback_info_pack=0.858s effective_fps=343.72 +2026-09-25 20:01:17.720 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=212715 infer_calls=212715 feedback_calls=212715 infer_wait=566.580s infer_obs_pack=3.250s env_step=32.880s feedback_total=15.797s feedback_obs_pack=4.254s feedback_info_pack=0.903s effective_fps=343.92 +2026-09-25 20:01:47.722 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=222718 infer_calls=222718 feedback_calls=222718 infer_wait=593.572s infer_obs_pack=3.408s env_step=34.446s feedback_total=16.531s feedback_obs_pack=4.452s feedback_info_pack=0.945s effective_fps=343.72 +2026-09-25 20:02:17.725 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=232983 infer_calls=232983 feedback_calls=232983 infer_wait=620.415s infer_obs_pack=3.569s env_step=36.084s feedback_total=17.307s feedback_obs_pack=4.661s feedback_info_pack=0.989s effective_fps=343.95 +2026-09-25 20:02:47.725 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=242928 infer_calls=242928 feedback_calls=242928 infer_wait=647.327s infer_obs_pack=3.720s env_step=37.698s feedback_total=18.055s feedback_obs_pack=4.869s feedback_info_pack=1.036s effective_fps=343.70 +2026-09-25 20:03:17.727 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=253040 infer_calls=253040 feedback_calls=253040 infer_wait=674.192s infer_obs_pack=3.879s env_step=39.321s feedback_total=18.827s feedback_obs_pack=5.083s feedback_info_pack=1.081s effective_fps=343.70 +2026-09-25 20:03:47.727 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=263043 infer_calls=263043 feedback_calls=263043 infer_wait=701.126s infer_obs_pack=4.034s env_step=40.915s feedback_total=19.581s feedback_obs_pack=5.286s feedback_info_pack=1.124s effective_fps=343.55 +2026-09-25 20:04:17.727 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=273148 infer_calls=273148 feedback_calls=273148 infer_wait=728.055s infer_obs_pack=4.191s env_step=42.514s feedback_total=20.333s feedback_obs_pack=5.488s feedback_info_pack=1.167s effective_fps=343.54 +2026-09-25 20:04:47.730 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=283079 infer_calls=283079 feedback_calls=283079 infer_wait=755.083s infer_obs_pack=4.342s env_step=44.063s feedback_total=21.060s feedback_obs_pack=5.687s feedback_info_pack=1.209s effective_fps=343.31 +2026-09-25 20:05:17.731 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=293314 infer_calls=293314 feedback_calls=293314 infer_wait=781.906s infer_obs_pack=4.503s env_step=45.717s feedback_total=21.833s feedback_obs_pack=5.901s feedback_info_pack=1.253s effective_fps=343.48 +2026-09-25 20:05:47.733 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=303375 infer_calls=303375 feedback_calls=303375 infer_wait=808.739s infer_obs_pack=4.663s env_step=47.360s feedback_total=22.619s feedback_obs_pack=6.113s feedback_info_pack=1.298s effective_fps=343.42 +2026-09-25 20:06:17.734 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=313701 infer_calls=313701 feedback_calls=313701 infer_wait=835.558s infer_obs_pack=4.823s env_step=49.002s feedback_total=23.410s feedback_obs_pack=6.323s feedback_info_pack=1.342s effective_fps=343.67 +2026-09-25 20:06:47.737 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=323741 infer_calls=323741 feedback_calls=323741 infer_wait=862.422s infer_obs_pack=4.978s env_step=50.632s feedback_total=24.184s feedback_obs_pack=6.531s feedback_info_pack=1.386s effective_fps=343.60 +2026-09-25 20:07:17.739 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=334071 infer_calls=334071 feedback_calls=334071 infer_wait=889.230s infer_obs_pack=5.142s env_step=52.280s feedback_total=24.970s feedback_obs_pack=6.748s feedback_info_pack=1.432s effective_fps=343.83 +2026-09-25 20:07:47.745 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=344066 infer_calls=344066 feedback_calls=344066 infer_wait=916.060s infer_obs_pack=5.305s env_step=53.924s feedback_total=25.758s feedback_obs_pack=6.961s feedback_info_pack=1.476s effective_fps=343.71 +2026-09-25 20:08:17.746 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=354135 infer_calls=354135 feedback_calls=354135 infer_wait=942.926s infer_obs_pack=5.460s env_step=55.554s feedback_total=26.531s feedback_obs_pack=7.168s feedback_info_pack=1.522s effective_fps=343.66 +2026-09-25 20:08:47.747 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=364358 infer_calls=364358 feedback_calls=364358 infer_wait=969.820s infer_obs_pack=5.622s env_step=57.151s feedback_total=27.310s feedback_obs_pack=7.379s feedback_info_pack=1.565s effective_fps=343.77 +2026-09-25 20:09:17.747 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=374389 infer_calls=374389 feedback_calls=374389 infer_wait=996.714s infer_obs_pack=5.781s env_step=58.745s feedback_total=28.078s feedback_obs_pack=7.591s feedback_info_pack=1.611s effective_fps=343.69 +2026-09-25 20:09:47.749 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=384500 infer_calls=384500 feedback_calls=384500 infer_wait=1023.574s infer_obs_pack=5.938s env_step=60.375s feedback_total=28.854s feedback_obs_pack=7.808s feedback_info_pack=1.662s effective_fps=343.69 +2026-09-25 20:10:17.750 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=394784 infer_calls=394784 feedback_calls=394784 infer_wait=1050.368s infer_obs_pack=6.102s env_step=62.036s feedback_total=29.650s feedback_obs_pack=8.023s feedback_info_pack=1.707s effective_fps=343.84 +2026-09-25 20:10:47.752 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=404909 infer_calls=404909 feedback_calls=404909 infer_wait=1077.243s infer_obs_pack=6.256s env_step=63.667s feedback_total=30.408s feedback_obs_pack=8.233s feedback_info_pack=1.750s effective_fps=343.85 +2026-09-25 20:11:02.244 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=409601 infer_calls=409601 feedback_calls=409601 infer_wait=1089.793s infer_obs_pack=6.328s env_step=64.407s feedback_total=30.755s feedback_obs_pack=8.328s feedback_info_pack=1.770s effective_fps=343.83 +2026-09-25 20:11:02.246 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 19222/1638400 episodes. diff --git a/experiments/e28-dppo-learns/results/fpodppo-walker/client-seed1.log b/experiments/e28-dppo-learns/results/fpodppo-walker/client-seed1.log new file mode 100644 index 0000000..c631b5a --- /dev/null +++ b/experiments/e28-dppo-learns/results/fpodppo-walker/client-seed1.log @@ -0,0 +1,49 @@ +2026-09-25 19:50:52.329 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 19:50:52.329 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 19:50:52.329 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 19:50:52.329 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 19:50:52.330 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 19:50:52.371 | INFO | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-195052-22bb3113 output_dir=runs/mujoco-v1-nenv1-20260925-195052-22bb3113 recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 19:50:52.405 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9541... +2026-09-25 19:50:52.408 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'FPOPolicy', 'action_dim': 6, 'action_horizon': 1} +2026-09-25 19:51:22.410 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=10207 infer_calls=10207 feedback_calls=10207 infer_wait=26.999s infer_obs_pack=0.155s env_step=1.544s feedback_total=0.741s feedback_obs_pack=0.197s feedback_info_pack=0.039s effective_fps=346.71 +2026-09-25 19:51:52.826 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=20482 infer_calls=20482 feedback_calls=20482 infer_wait=54.369s infer_obs_pack=0.318s env_step=3.095s feedback_total=1.502s feedback_obs_pack=0.397s feedback_info_pack=0.079s effective_fps=345.49 +2026-09-25 19:52:22.827 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=30766 infer_calls=30766 feedback_calls=30766 infer_wait=81.296s infer_obs_pack=0.473s env_step=4.675s feedback_total=2.265s feedback_obs_pack=0.598s feedback_info_pack=0.120s effective_fps=346.82 +2026-09-25 19:52:52.853 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=40962 infer_calls=40962 feedback_calls=40962 infer_wait=108.294s infer_obs_pack=0.630s env_step=6.225s feedback_total=3.015s feedback_obs_pack=0.797s feedback_info_pack=0.159s effective_fps=346.65 +2026-09-25 19:53:22.853 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=51417 infer_calls=51417 feedback_calls=51417 infer_wait=135.151s infer_obs_pack=0.794s env_step=7.838s feedback_total=3.800s feedback_obs_pack=1.002s feedback_info_pack=0.199s effective_fps=348.39 +2026-09-25 19:53:52.856 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=61690 infer_calls=61690 feedback_calls=61690 infer_wait=162.133s infer_obs_pack=0.947s env_step=9.395s feedback_total=4.543s feedback_obs_pack=1.201s feedback_info_pack=0.238s effective_fps=348.49 +2026-09-25 19:54:22.857 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=72121 infer_calls=72121 feedback_calls=72121 infer_wait=189.085s infer_obs_pack=1.102s env_step=10.959s feedback_total=5.300s feedback_obs_pack=1.401s feedback_info_pack=0.280s effective_fps=349.35 +2026-09-25 19:54:52.860 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=82395 infer_calls=82395 feedback_calls=82395 infer_wait=216.029s infer_obs_pack=1.255s env_step=12.545s feedback_total=6.051s feedback_obs_pack=1.603s feedback_info_pack=0.319s effective_fps=349.31 +2026-09-25 19:55:22.861 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=92786 infer_calls=92786 feedback_calls=92786 infer_wait=242.796s infer_obs_pack=1.416s env_step=14.212s feedback_total=6.835s feedback_obs_pack=1.818s feedback_info_pack=0.362s effective_fps=349.79 +2026-09-25 19:55:52.861 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=103029 infer_calls=103029 feedback_calls=103029 infer_wait=269.647s infer_obs_pack=1.576s env_step=15.832s feedback_total=7.618s feedback_obs_pack=2.023s feedback_info_pack=0.406s effective_fps=349.64 +2026-09-25 19:56:22.864 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=113413 infer_calls=113413 feedback_calls=113413 infer_wait=296.510s infer_obs_pack=1.733s env_step=17.462s feedback_total=8.379s feedback_obs_pack=2.232s feedback_info_pack=0.446s effective_fps=349.95 +2026-09-25 19:56:52.866 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=123663 infer_calls=123663 feedback_calls=123663 infer_wait=323.527s infer_obs_pack=1.885s env_step=18.992s feedback_total=9.132s feedback_obs_pack=2.432s feedback_info_pack=0.484s effective_fps=349.79 +2026-09-25 19:57:22.866 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=134029 infer_calls=134029 feedback_calls=134029 infer_wait=350.443s infer_obs_pack=2.043s env_step=20.579s feedback_total=9.891s feedback_obs_pack=2.636s feedback_info_pack=0.525s effective_fps=349.99 +2026-09-25 19:57:52.867 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=144124 infer_calls=144124 feedback_calls=144124 infer_wait=377.431s infer_obs_pack=2.202s env_step=22.135s feedback_total=10.629s feedback_obs_pack=2.834s feedback_info_pack=0.567s effective_fps=349.48 +2026-09-25 19:58:22.868 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=154444 infer_calls=154444 feedback_calls=154444 infer_wait=404.409s infer_obs_pack=2.357s env_step=23.696s feedback_total=11.366s feedback_obs_pack=3.032s feedback_info_pack=0.607s effective_fps=349.56 +2026-09-25 19:58:52.871 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=164738 infer_calls=164738 feedback_calls=164738 infer_wait=431.347s infer_obs_pack=2.517s env_step=25.271s feedback_total=12.131s feedback_obs_pack=3.234s feedback_info_pack=0.646s effective_fps=349.56 +2026-09-25 19:59:22.873 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=175123 infer_calls=175123 feedback_calls=175123 infer_wait=458.244s infer_obs_pack=2.673s env_step=26.861s feedback_total=12.910s feedback_obs_pack=3.440s feedback_info_pack=0.686s effective_fps=349.76 +2026-09-25 19:59:52.875 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=185401 infer_calls=185401 feedback_calls=185401 infer_wait=485.238s infer_obs_pack=2.830s env_step=28.420s feedback_total=13.640s feedback_obs_pack=3.639s feedback_info_pack=0.725s effective_fps=349.73 +2026-09-25 20:00:22.878 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=195765 infer_calls=195765 feedback_calls=195765 infer_wait=512.121s infer_obs_pack=2.987s env_step=30.027s feedback_total=14.412s feedback_obs_pack=3.846s feedback_info_pack=0.765s effective_fps=349.86 +2026-09-25 20:00:52.880 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=206009 infer_calls=206009 feedback_calls=206009 infer_wait=539.058s infer_obs_pack=3.142s env_step=31.600s feedback_total=15.170s feedback_obs_pack=4.047s feedback_info_pack=0.805s effective_fps=349.78 +2026-09-25 20:01:22.881 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=216411 infer_calls=216411 feedback_calls=216411 infer_wait=565.971s infer_obs_pack=3.300s env_step=33.190s feedback_total=15.929s feedback_obs_pack=4.254s feedback_info_pack=0.845s effective_fps=349.96 +2026-09-25 20:01:52.884 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=226690 infer_calls=226690 feedback_calls=226690 infer_wait=592.903s infer_obs_pack=3.456s env_step=34.770s feedback_total=16.684s feedback_obs_pack=4.456s feedback_info_pack=0.884s effective_fps=349.93 +2026-09-25 20:02:22.885 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=237136 infer_calls=237136 feedback_calls=237136 infer_wait=619.734s infer_obs_pack=3.615s env_step=36.405s feedback_total=17.473s feedback_obs_pack=4.666s feedback_info_pack=0.925s effective_fps=350.16 +2026-09-25 20:02:52.888 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=247318 infer_calls=247318 feedback_calls=247318 infer_wait=646.627s infer_obs_pack=3.770s env_step=38.014s feedback_total=18.244s feedback_obs_pack=4.868s feedback_info_pack=0.965s effective_fps=349.98 +2026-09-25 20:03:22.888 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=257654 infer_calls=257654 feedback_calls=257654 infer_wait=673.495s infer_obs_pack=3.929s env_step=39.634s feedback_total=19.010s feedback_obs_pack=5.076s feedback_info_pack=1.006s effective_fps=350.04 +2026-09-25 20:03:52.891 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=267858 infer_calls=267858 feedback_calls=267858 infer_wait=700.411s infer_obs_pack=4.087s env_step=41.217s feedback_total=19.779s feedback_obs_pack=5.280s feedback_info_pack=1.045s effective_fps=349.92 +2026-09-25 20:04:22.892 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=278139 infer_calls=278139 feedback_calls=278139 infer_wait=727.320s infer_obs_pack=4.244s env_step=42.812s feedback_total=20.543s feedback_obs_pack=5.483s feedback_info_pack=1.086s effective_fps=349.90 +2026-09-25 20:04:52.895 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=288300 infer_calls=288300 feedback_calls=288300 infer_wait=754.247s infer_obs_pack=4.397s env_step=44.400s feedback_total=21.301s feedback_obs_pack=5.685s feedback_info_pack=1.127s effective_fps=349.73 +2026-09-25 20:05:22.897 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=298710 infer_calls=298710 feedback_calls=298710 infer_wait=781.057s infer_obs_pack=4.554s env_step=46.051s feedback_total=22.094s feedback_obs_pack=5.897s feedback_info_pack=1.172s effective_fps=349.88 +2026-09-25 20:05:52.898 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=308873 infer_calls=308873 feedback_calls=308873 infer_wait=807.817s infer_obs_pack=4.715s env_step=47.729s feedback_total=22.881s feedback_obs_pack=6.113s feedback_info_pack=1.215s effective_fps=349.74 +2026-09-25 20:06:22.900 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=319338 infer_calls=319338 feedback_calls=319338 infer_wait=834.588s infer_obs_pack=4.877s env_step=49.401s feedback_total=23.675s feedback_obs_pack=6.327s feedback_info_pack=1.257s effective_fps=349.94 +2026-09-25 20:06:52.900 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=329569 infer_calls=329569 feedback_calls=329569 infer_wait=861.452s infer_obs_pack=5.031s env_step=51.019s feedback_total=24.452s feedback_obs_pack=6.536s feedback_info_pack=1.298s effective_fps=349.88 +2026-09-25 20:07:22.986 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=339970 infer_calls=339970 feedback_calls=339970 infer_wait=888.316s infer_obs_pack=5.195s env_step=52.682s feedback_total=25.245s feedback_obs_pack=6.749s feedback_info_pack=1.340s effective_fps=349.97 +2026-09-25 20:07:52.987 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=350367 infer_calls=350367 feedback_calls=350367 infer_wait=915.049s infer_obs_pack=5.364s env_step=54.363s feedback_total=26.053s feedback_obs_pack=6.964s feedback_info_pack=1.382s effective_fps=350.08 +2026-09-25 20:08:23.058 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=360450 infer_calls=360450 feedback_calls=360450 infer_wait=942.009s infer_obs_pack=5.524s env_step=55.972s feedback_total=26.816s feedback_obs_pack=7.168s feedback_info_pack=1.422s effective_fps=349.84 +2026-09-25 20:08:53.060 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=370839 infer_calls=370839 feedback_calls=370839 infer_wait=968.863s infer_obs_pack=5.683s env_step=57.593s feedback_total=27.600s feedback_obs_pack=7.373s feedback_info_pack=1.463s effective_fps=349.93 +2026-09-25 20:09:23.061 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=380993 infer_calls=380993 feedback_calls=380993 infer_wait=995.785s infer_obs_pack=5.839s env_step=59.183s feedback_total=28.361s feedback_obs_pack=7.575s feedback_info_pack=1.503s effective_fps=349.80 +2026-09-25 20:09:53.063 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=391331 infer_calls=391331 feedback_calls=391331 infer_wait=1022.631s infer_obs_pack=5.998s env_step=60.809s feedback_total=29.139s feedback_obs_pack=7.783s feedback_info_pack=1.543s effective_fps=349.85 +2026-09-25 20:10:23.064 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=401662 infer_calls=401662 feedback_calls=401662 infer_wait=1049.434s infer_obs_pack=6.158s env_step=62.465s feedback_total=29.918s feedback_obs_pack=7.990s feedback_info_pack=1.584s effective_fps=349.89 +2026-09-25 20:10:46.535 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=409601 infer_calls=409601 feedback_calls=409601 infer_wait=1069.942s infer_obs_pack=6.281s env_step=63.726s feedback_total=30.513s feedback_obs_pack=8.150s feedback_info_pack=1.615s effective_fps=349.95 +2026-09-25 20:10:46.535 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 20717/1638400 episodes. diff --git a/experiments/e28-dppo-learns/results/fpodppo-walker/client-seed2.log b/experiments/e28-dppo-learns/results/fpodppo-walker/client-seed2.log new file mode 100644 index 0000000..46a11a5 --- /dev/null +++ b/experiments/e28-dppo-learns/results/fpodppo-walker/client-seed2.log @@ -0,0 +1,50 @@ +2026-09-25 19:50:57.323 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-25 19:50:57.323 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-25 19:50:57.323 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-25 19:50:57.324 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-25 19:50:57.324 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-25 19:50:57.366 | INFO | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260925-195057-7729d3cf output_dir=runs/mujoco-v1-nenv1-20260925-195057-7729d3cf recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-25 19:50:57.402 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9542... +2026-09-25 19:50:57.404 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'FPOPolicy', 'action_dim': 6, 'action_horizon': 1} +2026-09-25 19:51:27.408 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=10006 infer_calls=10006 feedback_calls=10006 infer_wait=27.027s infer_obs_pack=0.151s env_step=1.533s feedback_total=0.743s feedback_obs_pack=0.198s feedback_info_pack=0.041s effective_fps=339.72 +2026-09-25 19:51:57.411 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=20090 infer_calls=20090 feedback_calls=20090 infer_wait=54.008s infer_obs_pack=0.326s env_step=3.089s feedback_total=1.481s feedback_obs_pack=0.399s feedback_info_pack=0.083s effective_fps=341.06 +2026-09-25 19:52:27.414 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=30023 infer_calls=30023 feedback_calls=30023 infer_wait=81.030s infer_obs_pack=0.479s env_step=4.615s feedback_total=2.228s feedback_obs_pack=0.601s feedback_info_pack=0.121s effective_fps=339.81 +2026-09-25 19:52:57.415 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=40169 infer_calls=40169 feedback_calls=40169 infer_wait=107.987s infer_obs_pack=0.633s env_step=6.187s feedback_total=2.985s feedback_obs_pack=0.804s feedback_info_pack=0.161s effective_fps=341.02 +2026-09-25 19:53:27.417 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=50205 infer_calls=50205 feedback_calls=50205 infer_wait=134.958s infer_obs_pack=0.788s env_step=7.744s feedback_total=3.753s feedback_obs_pack=1.016s feedback_info_pack=0.201s effective_fps=340.97 +2026-09-25 19:53:57.418 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=60576 infer_calls=60576 feedback_calls=60576 infer_wait=161.865s infer_obs_pack=0.946s env_step=9.317s feedback_total=4.550s feedback_obs_pack=1.223s feedback_info_pack=0.244s effective_fps=342.86 +2026-09-25 19:54:27.419 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=70470 infer_calls=70470 feedback_calls=70470 infer_wait=188.927s infer_obs_pack=1.094s env_step=10.845s feedback_total=5.279s feedback_obs_pack=1.424s feedback_info_pack=0.282s effective_fps=341.85 +2026-09-25 19:54:57.419 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=80758 infer_calls=80758 feedback_calls=80758 infer_wait=215.771s infer_obs_pack=1.258s env_step=12.471s feedback_total=6.065s feedback_obs_pack=1.636s feedback_info_pack=0.325s effective_fps=342.83 +2026-09-25 19:55:27.420 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=90847 infer_calls=90847 feedback_calls=90847 infer_wait=242.657s infer_obs_pack=1.415s env_step=14.069s feedback_total=6.845s feedback_obs_pack=1.846s feedback_info_pack=0.369s effective_fps=342.84 +2026-09-25 19:55:57.420 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=101215 infer_calls=101215 feedback_calls=101215 infer_wait=269.429s infer_obs_pack=1.577s env_step=15.733s feedback_total=7.648s feedback_obs_pack=2.065s feedback_info_pack=0.411s effective_fps=343.82 +2026-09-25 19:56:27.421 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=111261 infer_calls=111261 feedback_calls=111261 infer_wait=296.350s infer_obs_pack=1.733s env_step=17.315s feedback_total=8.423s feedback_obs_pack=2.271s feedback_info_pack=0.455s effective_fps=343.59 +2026-09-25 19:56:57.423 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=121522 infer_calls=121522 feedback_calls=121522 infer_wait=323.289s infer_obs_pack=1.891s env_step=18.886s feedback_total=9.198s feedback_obs_pack=2.477s feedback_info_pack=0.495s effective_fps=344.00 +2026-09-25 19:57:27.426 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=131579 infer_calls=131579 feedback_calls=131579 infer_wait=350.271s infer_obs_pack=2.042s env_step=20.442s feedback_total=9.953s feedback_obs_pack=2.679s feedback_info_pack=0.535s effective_fps=343.81 +2026-09-25 19:57:57.428 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=141769 infer_calls=141769 feedback_calls=141769 infer_wait=377.226s infer_obs_pack=2.196s env_step=22.017s feedback_total=10.711s feedback_obs_pack=2.883s feedback_info_pack=0.575s effective_fps=343.97 +2026-09-25 19:58:27.430 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=151778 infer_calls=151778 feedback_calls=151778 infer_wait=404.262s infer_obs_pack=2.347s env_step=23.544s feedback_total=11.448s feedback_obs_pack=3.080s feedback_info_pack=0.615s effective_fps=343.70 +2026-09-25 19:58:57.432 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=162059 infer_calls=162059 feedback_calls=162059 infer_wait=431.157s infer_obs_pack=2.505s env_step=25.149s feedback_total=12.217s feedback_obs_pack=3.290s feedback_info_pack=0.657s effective_fps=344.05 +2026-09-25 19:59:27.435 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=172034 infer_calls=172034 feedback_calls=172034 infer_wait=458.205s infer_obs_pack=2.655s env_step=26.666s feedback_total=12.963s feedback_obs_pack=3.490s feedback_info_pack=0.696s effective_fps=343.73 +2026-09-25 19:59:57.440 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=182108 infer_calls=182108 feedback_calls=182108 infer_wait=485.188s infer_obs_pack=2.808s env_step=28.224s feedback_total=13.711s feedback_obs_pack=3.692s feedback_info_pack=0.738s effective_fps=343.64 +2026-09-25 20:00:27.441 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=192269 infer_calls=192269 feedback_calls=192269 infer_wait=512.145s infer_obs_pack=2.965s env_step=29.797s feedback_total=14.465s feedback_obs_pack=3.896s feedback_info_pack=0.779s effective_fps=343.72 +2026-09-25 20:00:57.442 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=202214 infer_calls=202214 feedback_calls=202214 infer_wait=539.126s infer_obs_pack=3.118s env_step=31.351s feedback_total=15.215s feedback_obs_pack=4.101s feedback_info_pack=0.818s effective_fps=343.43 +2026-09-25 20:01:27.442 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=212370 infer_calls=212370 feedback_calls=212370 infer_wait=566.071s infer_obs_pack=3.272s env_step=32.926s feedback_total=15.974s feedback_obs_pack=4.307s feedback_info_pack=0.859s effective_fps=343.51 +2026-09-25 20:01:57.445 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=222426 infer_calls=222426 feedback_calls=222426 infer_wait=593.049s infer_obs_pack=3.427s env_step=34.478s feedback_total=16.735s feedback_obs_pack=4.511s feedback_info_pack=0.900s effective_fps=343.41 +2026-09-25 20:02:27.447 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=232590 infer_calls=232590 feedback_calls=232590 infer_wait=619.906s infer_obs_pack=3.587s env_step=36.112s feedback_total=17.510s feedback_obs_pack=4.724s feedback_info_pack=0.941s effective_fps=343.50 +2026-09-25 20:02:57.449 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=242649 infer_calls=242649 feedback_calls=242649 infer_wait=646.774s infer_obs_pack=3.749s env_step=37.724s feedback_total=18.296s feedback_obs_pack=4.933s feedback_info_pack=0.986s effective_fps=343.43 +2026-09-25 20:03:27.449 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=252803 infer_calls=252803 feedback_calls=252803 infer_wait=673.639s infer_obs_pack=3.910s env_step=39.338s feedback_total=19.078s feedback_obs_pack=5.145s feedback_info_pack=1.028s effective_fps=343.50 +2026-09-25 20:03:57.449 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=262736 infer_calls=262736 feedback_calls=262736 infer_wait=700.542s infer_obs_pack=4.072s env_step=40.937s feedback_total=19.846s feedback_obs_pack=5.356s feedback_info_pack=1.069s effective_fps=343.27 +2026-09-25 20:04:27.450 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=272828 infer_calls=272828 feedback_calls=272828 infer_wait=727.479s infer_obs_pack=4.229s env_step=42.518s feedback_total=20.609s feedback_obs_pack=5.560s feedback_info_pack=1.111s effective_fps=343.25 +2026-09-25 20:04:57.451 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=282909 infer_calls=282909 feedback_calls=282909 infer_wait=754.385s infer_obs_pack=4.386s env_step=44.115s feedback_total=21.384s feedback_obs_pack=5.771s feedback_info_pack=1.153s effective_fps=343.22 +2026-09-25 20:05:27.452 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=293131 infer_calls=293131 feedback_calls=293131 infer_wait=781.131s infer_obs_pack=4.550s env_step=45.807s feedback_total=22.188s feedback_obs_pack=5.990s feedback_info_pack=1.197s effective_fps=343.37 +2026-09-25 20:05:57.455 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=303177 infer_calls=303177 feedback_calls=303177 infer_wait=807.942s infer_obs_pack=4.712s env_step=47.462s feedback_total=22.972s feedback_obs_pack=6.206s feedback_info_pack=1.240s effective_fps=343.31 +2026-09-25 20:06:27.456 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=313520 infer_calls=313520 feedback_calls=313520 infer_wait=834.748s infer_obs_pack=4.873s env_step=49.105s feedback_total=23.771s feedback_obs_pack=6.418s feedback_info_pack=1.286s effective_fps=343.58 +2026-09-25 20:06:57.456 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=323595 infer_calls=323595 feedback_calls=323595 infer_wait=861.574s infer_obs_pack=5.034s env_step=50.742s feedback_total=24.560s feedback_obs_pack=6.630s feedback_info_pack=1.329s effective_fps=343.55 +2026-09-25 20:07:27.456 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=333906 infer_calls=333906 feedback_calls=333906 infer_wait=888.359s infer_obs_pack=5.196s env_step=52.394s feedback_total=25.369s feedback_obs_pack=6.843s feedback_info_pack=1.373s effective_fps=343.77 +2026-09-25 20:07:57.458 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=344036 infer_calls=344036 feedback_calls=344036 infer_wait=915.138s infer_obs_pack=5.359s env_step=54.058s feedback_total=26.172s feedback_obs_pack=7.064s feedback_info_pack=1.416s effective_fps=343.79 +2026-09-25 20:08:27.460 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=354048 infer_calls=354048 feedback_calls=354048 infer_wait=942.080s infer_obs_pack=5.513s env_step=55.635s feedback_total=26.940s feedback_obs_pack=7.272s feedback_info_pack=1.457s effective_fps=343.68 +2026-09-25 20:08:57.461 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=364252 infer_calls=364252 feedback_calls=364252 infer_wait=968.956s infer_obs_pack=5.678s env_step=57.243s feedback_total=27.712s feedback_obs_pack=7.479s feedback_info_pack=1.499s effective_fps=343.77 +2026-09-25 20:09:27.463 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=374180 infer_calls=374180 feedback_calls=374180 infer_wait=995.894s infer_obs_pack=5.840s env_step=58.825s feedback_total=28.466s feedback_obs_pack=7.680s feedback_info_pack=1.539s effective_fps=343.59 +2026-09-25 20:09:57.463 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=384438 infer_calls=384438 feedback_calls=384438 infer_wait=1022.727s infer_obs_pack=6.004s env_step=60.454s feedback_total=29.253s feedback_obs_pack=7.894s feedback_info_pack=1.582s effective_fps=343.73 +2026-09-25 20:10:27.467 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=394572 infer_calls=394572 feedback_calls=394572 infer_wait=1049.584s infer_obs_pack=6.159s env_step=62.077s feedback_total=30.037s feedback_obs_pack=8.108s feedback_info_pack=1.625s effective_fps=343.75 +2026-09-25 20:10:57.470 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=404644 infer_calls=404644 feedback_calls=404644 infer_wait=1076.505s infer_obs_pack=6.318s env_step=63.664s feedback_total=30.794s feedback_obs_pack=8.310s feedback_info_pack=1.665s effective_fps=343.71 +2026-09-25 20:11:12.555 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=409601 infer_calls=409601 feedback_calls=409601 infer_wait=1089.669s infer_obs_pack=6.391s env_step=64.397s feedback_total=31.156s feedback_obs_pack=8.404s feedback_info_pack=1.684s effective_fps=343.74 +2026-09-25 20:11:12.555 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 19309/1638400 episodes. diff --git a/experiments/e28-dppo-learns/results/fpodppo-walker/server-seed0.log b/experiments/e28-dppo-learns/results/fpodppo-walker/server-seed0.log new file mode 100644 index 0000000..da06b04 --- /dev/null +++ b/experiments/e28-dppo-learns/results/fpodppo-walker/server-seed0.log @@ -0,0 +1,25 @@ +19:50:46|INFO|plugrl_server version: 0.1.0 +19:50:46|INFO|Algorithm: dppo, Config: DPPOAlgoConfigWalker(global_steps=409600, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=4096, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=2048, train_itrs=100, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +19:50:46|INFO|Policy: fpo-policy, Config: FPOPolicyConfig(algo='dppo', device=device(type='cpu'), obs_dim=17, action_dim=6, flow_steps=10, timestep_embed_dim=8, action_horizon=1, feather_std=0.0, policy_mlp_output_scale=0.25, normalize_observations=True, hidden_dims=(32, 32, 32, 32), value_hidden_dims=(256, 256, 256, 256, 256)) +19:50:46|INFO|Seeded python, numpy and torch with 0. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +19:50:46|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/fpodppo-walker/dppo/fpo-policy/fpodppo-walker-seed0 +19:50:46|INFO|Policy created... +19:50:46|INFO|Initialized RolloutBuffer buffer_size=4096 action_shape=(4096, 10, 1, 6) value_shape=(4096,) +19:50:46|INFO|Algorithm created: + +19:50:46|INFO|Agent Server is listening on 0.0.0.0:9540 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +19:54:51|INFO|Checkpoint saved at step 81921 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/fpodppo-walker/dppo/fpo-policy/fpodppo-walker-seed0/81921 +19:58:53|INFO|Checkpoint saved at step 163841 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/fpodppo-walker/dppo/fpo-policy/fpodppo-walker-seed0/163841 +20:02:56|INFO|Checkpoint saved at step 245761 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/fpodppo-walker/dppo/fpo-policy/fpodppo-walker-seed0/245761 +20:06:59|INFO|Checkpoint saved at step 327681 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/fpodppo-walker/dppo/fpo-policy/fpodppo-walker-seed0/327681 +20:11:02|INFO|Checkpoint saved at step 409601 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/fpodppo-walker/dppo/fpo-policy/fpodppo-walker-seed0/409601 +20:11:02|INFO|Stopping server as the algorithm signaled to stop. +20:11:02|INFO|Shutdown started: aborting pending infer requests. +20:11:02|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +20:11:02|INFO|Shutdown closing 1 websocket connection(s). +20:11:02|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 60602). +20:11:02|INFO|WebSocket server closed. +20:11:02|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e28-dppo-learns/results/fpodppo-walker/server-seed1.log b/experiments/e28-dppo-learns/results/fpodppo-walker/server-seed1.log new file mode 100644 index 0000000..d54c396 --- /dev/null +++ b/experiments/e28-dppo-learns/results/fpodppo-walker/server-seed1.log @@ -0,0 +1,25 @@ +19:50:52|INFO|plugrl_server version: 0.1.0 +19:50:52|INFO|Algorithm: dppo, Config: DPPOAlgoConfigWalker(global_steps=409600, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=4096, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=2048, train_itrs=100, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +19:50:52|INFO|Policy: fpo-policy, Config: FPOPolicyConfig(algo='dppo', device=device(type='cpu'), obs_dim=17, action_dim=6, flow_steps=10, timestep_embed_dim=8, action_horizon=1, feather_std=0.0, policy_mlp_output_scale=0.25, normalize_observations=True, hidden_dims=(32, 32, 32, 32), value_hidden_dims=(256, 256, 256, 256, 256)) +19:50:52|INFO|Seeded python, numpy and torch with 1. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +19:50:52|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/fpodppo-walker/dppo/fpo-policy/fpodppo-walker-seed1 +19:50:52|INFO|Policy created... +19:50:52|INFO|Initialized RolloutBuffer buffer_size=4096 action_shape=(4096, 10, 1, 6) value_shape=(4096,) +19:50:52|INFO|Algorithm created: + +19:50:52|INFO|Agent Server is listening on 0.0.0.0:9541 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +19:54:51|INFO|Checkpoint saved at step 81921 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/fpodppo-walker/dppo/fpo-policy/fpodppo-walker-seed1/81921 +19:58:50|INFO|Checkpoint saved at step 163841 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/fpodppo-walker/dppo/fpo-policy/fpodppo-walker-seed1/163841 +20:02:48|INFO|Checkpoint saved at step 245761 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/fpodppo-walker/dppo/fpo-policy/fpodppo-walker-seed1/245761 +20:06:47|INFO|Checkpoint saved at step 327681 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/fpodppo-walker/dppo/fpo-policy/fpodppo-walker-seed1/327681 +20:10:46|INFO|Checkpoint saved at step 409601 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/fpodppo-walker/dppo/fpo-policy/fpodppo-walker-seed1/409601 +20:10:46|INFO|Stopping server as the algorithm signaled to stop. +20:10:46|INFO|Shutdown started: aborting pending infer requests. +20:10:46|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +20:10:46|INFO|Shutdown closing 1 websocket connection(s). +20:10:46|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 57114). +20:10:46|INFO|WebSocket server closed. +20:10:46|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e28-dppo-learns/results/fpodppo-walker/server-seed2.log b/experiments/e28-dppo-learns/results/fpodppo-walker/server-seed2.log new file mode 100644 index 0000000..223a61a --- /dev/null +++ b/experiments/e28-dppo-learns/results/fpodppo-walker/server-seed2.log @@ -0,0 +1,25 @@ +19:50:57|INFO|plugrl_server version: 0.1.0 +19:50:57|INFO|Algorithm: dppo, Config: DPPOAlgoConfigWalker(global_steps=409600, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=4096, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=2048, train_itrs=100, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +19:50:57|INFO|Policy: fpo-policy, Config: FPOPolicyConfig(algo='dppo', device=device(type='cpu'), obs_dim=17, action_dim=6, flow_steps=10, timestep_embed_dim=8, action_horizon=1, feather_std=0.0, policy_mlp_output_scale=0.25, normalize_observations=True, hidden_dims=(32, 32, 32, 32), value_hidden_dims=(256, 256, 256, 256, 256)) +19:50:57|INFO|Seeded python, numpy and torch with 2. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +19:50:57|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/fpodppo-walker/dppo/fpo-policy/fpodppo-walker-seed2 +19:50:57|INFO|Policy created... +19:50:57|INFO|Initialized RolloutBuffer buffer_size=4096 action_shape=(4096, 10, 1, 6) value_shape=(4096,) +19:50:57|INFO|Algorithm created: + +19:50:57|INFO|Agent Server is listening on 0.0.0.0:9542 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +19:55:01|INFO|Checkpoint saved at step 81921 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/fpodppo-walker/dppo/fpo-policy/fpodppo-walker-seed2/81921 +19:59:03|INFO|Checkpoint saved at step 163841 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/fpodppo-walker/dppo/fpo-policy/fpodppo-walker-seed2/163841 +20:03:06|INFO|Checkpoint saved at step 245761 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/fpodppo-walker/dppo/fpo-policy/fpodppo-walker-seed2/245761 +20:07:09|INFO|Checkpoint saved at step 327681 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/fpodppo-walker/dppo/fpo-policy/fpodppo-walker-seed2/327681 +20:11:12|INFO|Checkpoint saved at step 409601 to /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/results/fpodppo-walker/dppo/fpo-policy/fpodppo-walker-seed2/409601 +20:11:12|INFO|Stopping server as the algorithm signaled to stop. +20:11:12|INFO|Shutdown started: aborting pending infer requests. +20:11:12|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +20:11:12|INFO|Shutdown closing 1 websocket connection(s). +20:11:12|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 58538). +20:11:12|INFO|WebSocket server closed. +20:11:12|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e28-dppo-learns/results/run.out b/experiments/e28-dppo-learns/results/run.out new file mode 100644 index 0000000..d35d125 --- /dev/null +++ b/experiments/e28-dppo-learns/results/run.out @@ -0,0 +1,12 @@ +start 2026-09-25 19:34:10 +code: d1f4783 (src /home/guangzhao/zuogou/plugrl/e28/src) +client: 931ab56 (/home/guangzhao/zuogou/plugrl/e28/../plugrl-env-client/.venv/bin/python) +--- wave 1: dppo-policy +--- wave 2: fpo-policy +end 2026-09-25 20:11:12 +dppo-hopper failed seeds: 0 +dppo-walker failed seeds: 0 +dppo-cheetah failed seeds: 0 +fpodppo-hopper failed seeds: 0 +fpodppo-walker failed seeds: 0 +E28_DONE diff --git a/experiments/e28-dppo-learns/results/verdicts.txt b/experiments/e28-dppo-learns/results/verdicts.txt new file mode 100644 index 0000000..6133634 --- /dev/null +++ b/experiments/e28-dppo-learns/results/verdicts.txt @@ -0,0 +1,41 @@ +P1 every cell runs end to end + dppo-hopper dppo-policy Hopper-v5 HOLDS (16 min) + dppo-walker dppo-policy Walker2d-v5 HOLDS (16 min) + dppo-cheetah dppo-policy HalfCheetah-v5 HOLDS (15 min) + fpodppo-hopper fpo-policy Hopper-v5 HOLDS (20 min) + fpodppo-walker fpo-policy Walker2d-v5 HOLDS (20 min) + +status learns: mean of iterations 91-100 >= 500 (HalfCheetah: >= first + 200) on 2 of 3 + dppo-hopper 339.0, 429.5, 275.7 0 of 3 -> did not learn in 409,600 steps + dppo-walker 259.8, 292.4, 273.7 0 of 3 -> did not learn in 409,600 steps + dppo-cheetah +480.8, +83.8, +543.9 2 of 3 -> learns + fpodppo-hopper 25.5, 24.6, 26.7 0 of 3 -> did not learn in 409,600 steps + fpodppo-walker 3.1, 1.4, 2.2 0 of 3 -> did not learn in 409,600 steps + +P2 dppo-policy learns Walker2d + FALSIFIED + +P3 dppo-policy learns Hopper + FALSIFIED + +P4 fpo-policy under DPPO learns neither Hopper nor Walker2d + HOLDS + +reported: first iteration and mean of the last ten, return and episode length + dppo-hopper seed 0 return 11.2 -> 339.0 length 18.7 -> 202.8 + dppo-hopper seed 1 return 12.4 -> 429.5 length 16.8 -> 279.5 + dppo-hopper seed 2 return 15.8 -> 275.7 length 21.0 -> 152.0 + dppo-walker seed 0 return -2.2 -> 259.8 length 17.6 -> 230.3 + dppo-walker seed 1 return -1.6 -> 292.4 length 17.6 -> 255.8 + dppo-walker seed 2 return -2.6 -> 273.7 length 15.1 -> 223.0 + dppo-cheetah seed 0 return -386.3 -> 94.5 length 1000.0 -> 1000.0 + dppo-cheetah seed 1 return -445.0 -> -361.2 length 1000.0 -> 1000.0 + dppo-cheetah seed 2 return -382.1 -> 161.8 length 1000.0 -> 1000.0 + fpodppo-hopper seed 0 return 14.9 -> 25.5 length 20.2 -> 27.4 + fpodppo-hopper seed 1 return 19.7 -> 24.6 length 22.8 -> 25.4 + fpodppo-hopper seed 2 return 17.0 -> 26.7 length 21.9 -> 27.4 + fpodppo-walker seed 0 return 1.6 -> 3.1 length 20.6 -> 22.5 + fpodppo-walker seed 1 return -0.0 -> 1.4 length 18.9 -> 20.8 + fpodppo-walker seed 2 return 0.5 -> 2.2 length 20.4 -> 22.0 + +wrote /home/guangzhao/zuogou/plugrl/e28/experiments/e28-dppo-learns/summary.tsv diff --git a/experiments/e28-dppo-learns/summary.tsv b/experiments/e28-dppo-learns/summary.tsv new file mode 100644 index 0000000..d748156 --- /dev/null +++ b/experiments/e28-dppo-learns/summary.tsv @@ -0,0 +1,16 @@ +cell policy algorithm task claim seed iterations runs first_return last10_return first_length last10_length +dppo-hopper dppo-policy dppo Hopper-v5 runs 0 100 true 11.2 339.0 18.7 202.8 +dppo-hopper dppo-policy dppo Hopper-v5 runs 1 100 true 12.4 429.5 16.8 279.5 +dppo-hopper dppo-policy dppo Hopper-v5 runs 2 100 true 15.8 275.7 21.0 152.0 +dppo-walker dppo-policy dppo Walker2d-v5 runs 0 100 true -2.2 259.8 17.6 230.3 +dppo-walker dppo-policy dppo Walker2d-v5 runs 1 100 true -1.6 292.4 17.6 255.8 +dppo-walker dppo-policy dppo Walker2d-v5 runs 2 100 true -2.6 273.7 15.1 223.0 +dppo-cheetah dppo-policy dppo HalfCheetah-v5 learns 0 100 true -386.3 94.5 1000.0 1000.0 +dppo-cheetah dppo-policy dppo HalfCheetah-v5 learns 1 100 true -445.0 -361.2 1000.0 1000.0 +dppo-cheetah dppo-policy dppo HalfCheetah-v5 learns 2 100 true -382.1 161.8 1000.0 1000.0 +fpodppo-hopper fpo-policy dppo Hopper-v5 runs 0 100 true 14.9 25.5 20.2 27.4 +fpodppo-hopper fpo-policy dppo Hopper-v5 runs 1 100 true 19.7 24.6 22.8 25.4 +fpodppo-hopper fpo-policy dppo Hopper-v5 runs 2 100 true 17.0 26.7 21.9 27.4 +fpodppo-walker fpo-policy dppo Walker2d-v5 runs 0 100 true 1.6 3.1 20.6 22.5 +fpodppo-walker fpo-policy dppo Walker2d-v5 runs 1 100 true -0.0 1.4 18.9 20.8 +fpodppo-walker fpo-policy dppo Walker2d-v5 runs 2 100 true 0.5 2.2 20.4 22.0