fix: sessions give no verdict only for failures that are ours for certain

deepgym/CHANGELOG.md

@@ -2,6 +2,7 @@
 
 ## Unreleased
 
+- **Sessions: no verdict only for failures that are ours for certain.** Once the agent has acted, a failure is its judged 0 (`policy_sandbox_failure`) unless it is ours for certain: a network, provider or auth failure (HTTP 401 and 403 now count as infrastructure in `is_infrastructure_failure`), a verifier that couldn't run (its upload or launch failed) or timed out before it read the agent's output, an interrupt, or E2B or Modal retiring the sandbox at the end of its one-hour lifetime (new `SandboxExpiredError`, before a `SandboxDiedError`) unless the agent's own actions took at least half the session's time. A failure the rule can't read is the agent's 0. The rule is `deepgym.session.failure_result`, which verl's `deepgym_agent` loop and Harbor's agent phase also use; before, a 401 uploading verifier code or a verl final verifier that timed out before reading anything scored 0, and a Harbor agent that replaced its work directory with a link, grew it past the archive cap, made packing outrun its deadline, killed its sandbox or spent most of the provider lifetime in its script got no verdict; in Harbor any failure once the agent's script has started is now its 0 unless it is ours for certain. Session verifiers now get the consulted mark single runs use, embedded in the verifier's own script rather than on a command line: a step or final verifier that times out after reading the agent's output (a FIFO it left, say) is a judged 0 with `solution_timeout` (for a step, its reward, and the episode goes on), and so is a verifier that is killed, or gives no verdict, after reading it (`solution_error`), unless the agent's code failed to launch. Each verifier also prints a start line first, so a timeout whose output the provider lost is a judged 0 rather than passing for one that never read anything. Listing the agent's folder, or starting a program in it or on one of its files, now counts as reading its output, and the mark is written to the stdout file itself, so capturing `sys.stdout` can't hide it, and cut from what the verifier printed exactly, so a line it fell inside reads whole. Daytona enforces command deadlines in the sandbox, so a timed-out command keeps what it printed. A FIFO the agent leaves where the harness writes is refused at once instead of hanging the write, the local backend shows output that isn't UTF-8 instead of ending the episode, and E2B, Daytona and Docker/Apple uploads to a stopped sandbox raise `SandboxDiedError`. `deepgym_solution.run_solution` prints the consulted mark before it releases the solution, so a solution that dies the moment it starts can't pass for one that never ran.
 - **QA gate:** multi-turn and computer-use envs with a `reference_actions.json` are checked: the reference replays at full marks every time, and agents that quit at once (`empty_fails`), use every action (`busy_fails`) or act at random (`random_fails`) must earn nothing. The computer-use built-ins ship their reference episodes (one per variant) and run on `--mode daytona`. Negative controls (empty, stub, idle agents) fail on any reward, not just full marks; attacks define the env's real entry point; reports list the checks that `ran`. `difficulty` is reported, never gated: it gives mean and spread of rewards and flags `too_easy`, `too_hard` and `no_variance` (partial rewards count as variance), retries 429s, timeouts and 5xx with backoff, and is inconclusive rather than aborting when the endpoint stays down. Missing verdicts never pass: incomplete black-box scans are `inconclusive` and never saved as ROBUST, an audit that judged nothing has no risk rating (`status` says why), and RL exploit discovery reports suspected exploits with their signs from every round instead of a genuineness verdict. The exploit DB keys results by benchmark, verifier and attack mode (legacy tables are rebuilt in place), reuses one only for the same verifier contents and attack setup, and summaries filter by benchmark and mode. Reports and approvals are written atomically, and attack payloads carry JSON test values as valid Python. Built-ins: `data_pipeline` and `web_path_traversal_toy` no longer pay for an empty answer, and `anagram_check`, `palindrome_check` and `valid_parentheses` score balanced recall on real booleans, so a constant answer earns nothing. Breaking: `ExploitDB.get` needs `benchmark=`, `DiscoveryReport` replaces `is_genuine_solution`/`exploit_discovered` with `suspected_exploits`/`exploit_suspected`, and `VerifierAuditReport.risk_score`/`risk_level` are None when inconclusive.
 
 - **Environment authoring:** `deepgym new <name> --kind function|script|repo-patch|multi-turn` scaffolds a working env folder (task, a verifier that runs the solution only through `deepgym_solution` and reports every case with a reason, reference, seeded `variants.py`, metadata). `deepgym check <folder>` validates the files, metadata schema and protocol output, runs the reference, empty and stub solutions, and prints warnings from an advisory lint over the judge code (never evidence of isolation, which comes from `load_solution`/`run_solution`). `deepgym qa` takes folders (multi-turn ones replay `reference_actions.json`); `--mutants` fails a verifier that passes any AST mutant of the reference and reports the mutation score, running each mutant's whole judge confined in a local bwrap sandbox (`DeepGym(confine_local_judge=True)`: writes only to the run's temp directory, no network, a PID namespace that takes down everything the run started, CPU capped) so a mutant stays contained even if the verifier runs it in-process; macOS's sandbox-exec can't reap detached processes, so there mutants need docker, modal or e2b (sandbox providers run mutants only where the whole sandbox is offline: docker, modal, e2b); `--approve --by NAME` (which needs a rerun and every requested check passed) runs QA on a private copy of a self-contained folder and writes a read-only `approval.json` (report, file hashes, version, time, lint warnings marked advisory) that `check` flags once the files change. Reports list skipped checks under `not_run`, and approvals name them. `deepgym_solution` gains `apply_patch` (refusing file creation, deletion, renames and copies) and `load_files` (a normal import of the patched tree) for repo-patch verifiers, and raises `LaunchError` when the machine can't start the solution, so verifiers end the run as an error instead of scoring 0. `empty_fails` and `stub_fails` are now `inconclusive`, not passed, when the verifier errors instead of scoring, `hack_resistant` is skipped, not passed, when the verifier rejects every attack as a malformed answer, `deterministic` is skipped after a single reference run, `reference_passes` and `deterministic` are inconclusive when a reference run gets no verdict, and so is `difficulty` when a sample gets none. Breaking: `run_qa(target, dg, QAOptions(...))` replaces the `reruns`/`client`/`samples` keywords.

deepgym/docs/environments/sessions.md

@@ -98,12 +98,22 @@ Docker sessions: Docker runs them with no-new-privileges, which blocks sudo.
 - A step that fails after its action may have run ends the episode: `step()` raises, the
   action stays on the trajectory with no step reward (so `step_rewards` is one shorter
   than `steps`), and `score()` returns the result that failure fixed instead of running
-  the final verifier. A provider or network failure, or a failed step verifier (bad
-  output, a timeout, exit 2, or an `error_type` that isn't the model's fault), has no
-  verdict; a failed step verifier reports `verifier_error`, or `invalid_score` for a NaN
-  or out-of-range score. Anything else follows from the agent's actions (its sandbox
-  died, a write failed, its files blocked the step's output) and scores 0 with
+  the final verifier. Only a failure that is ours for certain has no verdict
+  (`deepgym.session.failure_result`): a provider, network or auth failure, a step
+  verifier that couldn't run or failed (bad output, exit 2, an `error_type` that isn't the
+  model's fault, or a timeout before it read the agent's output), or the provider retiring
+  the sandbox at the end of its lifetime, unless the agent's own actions took at least half
+  the session's time. A failed step verifier reports `verifier_error`, or `invalid_score`
+  for a NaN or out-of-range score. Anything else follows from the agent's actions (its
+  sandbox died, a write failed, even with a disk or I/O error, since the agent can fill
+  the disk or plant a link, its files blocked the step's output) and scores 0 with
   `error_type='policy_sandbox_failure'`.
+- A step or final verifier that times out after reading the agent's output (it opened a
+  file, listed a folder or started a program in the work directory or on a file there, or
+  started the agent's code) is a judged 0 with `error_type='solution_timeout'`; for a step
+  verifier that is the step's reward, and the episode goes on. One that is killed, or
+  gives no verdict, after reading it is a judged 0 with `solution_error` (unless the
+  agent's code failed to launch), and so is a timeout whose output was lost.
 - A rejected action (not allowed, or the agent's own files in the way of a `write_file`)
   raises `ActionError` and the episode goes on.
 - `run_episode` returns the result a failed step fixed instead of raising.

deepgym/docs/training/reward-contract.md

@@ -53,11 +53,18 @@ agent's work directory.
   sandbox that dies on Daytona, or in an E2B or Modal call that raises, since those
   return nothing the judge printed.
 
-In agent episodes, `deepgym.exceptions.is_infrastructure_failure(exc)` draws the line: a
-connection or transport error, a provider call timing out, a provider HTTP 5xx or 429, or
-the provider evicting the sandbox is infrastructure. A `SandboxDiedError` (the sandbox
-itself stopped) never is. Sessions, verl's multi-turn agent loop and the OpenRLHF endpoint
-use this test.
+In agent episodes, `deepgym.session.failure_result(exc, sandbox_id)` draws the line once the
+agent has acted. Only a failure that is ours for certain has no verdict: what
+`deepgym.exceptions.is_infrastructure_failure(exc)` calls infrastructure (a connection or
+transport error, a provider call timing out, a provider HTTP 5xx or 429, the provider
+refusing our credentials with 401 or 403, or the provider evicting the sandbox), a
+verifier that couldn't run or timed out before it read the agent's output, or the
+provider retiring the sandbox at the end of its lifetime (`SandboxExpiredError`), unless the
+agent's own actions took at least half the session's time. Anything else, a
+`SandboxDiedError` or a failed write included, is the agent's 0, and so is a failure the
+rule can't read. Sessions, verl's
+multi-turn agent loop and Harbor's agent phase use this rule; the OpenRLHF endpoint
+grades once more what `is_infrastructure_failure` calls infrastructure.
 
 ## What each trainer receives
 

deepgym/docs/training/verl.md

@@ -99,17 +99,20 @@ tools:
 - Written against verl 0.9.1. CI runs the loop against a stand-in for verl's; it was also
   run on Daytona sandboxes through verl 0.9.1's own `ToolAgentLoop._call_tool`.
 
-Only failures that aren't the policy's doing get no verdict and are kept out of training
-the same way (`error` 1.0): a sandbox that fails to start, a provider or network failure
-during an action or the final verifier (`deepgym.exceptions.is_infrastructure_failure`),
-and a step or final verifier that fails before it runs the agent's code. Everything that
-follows from the policy's own actions is scored:
+Only failures that are ours for certain get no verdict and are kept out of training the
+same way (`error` 1.0): a sandbox that fails to start, a provider, network or auth
+failure, a provider lifetime that ran out, and a step or final verifier that couldn't
+run or timed out before it read the agent's output (`deepgym.session.failure_result`);
+an expiry only if the policy's own actions didn't take most of the time. Everything else
+is scored:
 
 - An invalid or disallowed action comes back to the model as text, and the rollout goes
   on.
 - Any other failed action (a failed write, a sandbox the policy killed) comes back as an
   error and ends the episode with a 0, `error_type` `policy_sandbox_failure`, as does a
   final verifier that can't run in a sandbox the policy killed or hung.
+- A verifier that times out after reading the agent's output scores 0 with
+  `solution_timeout`.
 - A rollout where the model never calls the tool scores 0.
 
 This loop also warns when more than 5 samples in a block of 100 are kept out.

deepgym/src/deepgym/exceptions.py

@@ -74,6 +74,12 @@ def __init__(self, message: str, output: str = '') -> None:
         self.output = output
 
 
+class SandboxExpiredError(SandboxError):
+    """Raise when the provider retired the sandbox at the end of its lifetime, whatever ran
+    in it. Not the policy's doing unless its own actions ran the clock out (see
+    :func:`deepgym.session.failure_result`)."""
+
+
 class TimeoutError(DeepGymError):
     """Raise when execution exceeds the configured timeout."""
 
@@ -112,8 +118,9 @@ def is_infrastructure_failure(exc: BaseException) -> bool:
     """Return True if *exc* is a failure of the network or the sandbox provider.
 
     That is a connection or transport error, a provider call timing out, a provider HTTP
-    5xx or 429, or the provider evicting the sandbox, anywhere along the exception's
-    ``__cause__`` chain. Such a run got no verdict, and trainers keep it out of training.
+    5xx or 429, the provider refusing our credentials (401, 403), or the provider evicting
+    the sandbox, anywhere along the exception's ``__cause__`` chain. Such a run got no
+    verdict, and trainers keep it out of training.
 
     Anything else is the policy's doing and is scored: a command or verifier that ran out
     of time, a sandbox the provider reports gone or stopped after the policy acted, a
@@ -132,7 +139,7 @@ def is_infrastructure_failure(exc: BaseException) -> bool:
         if isinstance(exc, (ConnectionError, builtins.TimeoutError, *transport)):
             return True
         status = _status_code(exc)
-        if status is not None and (status >= 500 or status == 429):
+        if status is not None and (status >= 500 or status in (401, 403, 429)):
             return True
         exc = exc.__cause__
     return False

deepgym/src/deepgym/harbor/agent.py

@@ -3,12 +3,14 @@
 from __future__ import annotations
 
 import math
+import time
 from typing import NamedTuple
 
-from deepgym.exceptions import DeepGymError, TimeoutError
+from deepgym.exceptions import DeepGymError, SandboxExpiredError, TimeoutError
 from deepgym.harbor.task import HARNESS_PATHS, STAGING_DIR, HarborTask
+from deepgym.models import RunResult
 from deepgym.sandboxes import Sandbox
-from deepgym.session import SandboxBackend, check_workdir
+from deepgym.session import SandboxBackend, check_workdir, failure_result, ran_out_the_clock
 
 DEFAULT_AGENT_TIMEOUT_S = 1800
 """Seconds the agent's script may run when task.toml sets no ``[agent].timeout_sec``."""
@@ -30,33 +32,53 @@ class AgentRun(NamedTuple):
     """Whether the script was killed at the agent timeout."""
 
 
-def run_agent(sandbox: Sandbox, task: HarborTask, script: str) -> AgentRun:
+def run_agent(sandbox: Sandbox, task: HarborTask, script: str) -> AgentRun | RunResult:
     """Run *script* with bash in *task*'s work directory, then archive that directory.
 
     The script runs as an unprivileged user with no capabilities and no network
     (:class:`~deepgym.session.SandboxBackend`), and owns the work directory. Its processes
     are killed before the archive is made. A timeout ends the script, not the run.
 
+    Returns:
+        What the agent left behind; or, once its script has started, the result of any
+        failure (see :func:`~deepgym.session.failure_result`): no verdict only if it is ours
+        for certain (or the provider's lifetime ran out before the script took half the
+        time), else the agent's judged 0 (its sandbox died, its work directory can't be
+        packed or read back, ...).
+
     Args:
         sandbox: A fresh sandbox from the task's image; deleted afterwards.
         task: The task; its workdir, else the image's WORKDIR, is the work directory.
         script: The model's output.
 
     Raises:
         DeepGymError: If the image's WORKDIR can't be the agent's (``/``, /root, /tmp ...).
-        SandboxError: If setting up the sandbox or archiving fails.
+        SandboxError: If setting up the sandbox or uploading the script fails.
     """
+    opened = time.monotonic()
     workdir = task.workdir or _image_workdir(sandbox, task.name)
     backend = SandboxBackend(sandbox, workdir)  # deletes the sandbox if setup fails
     try:
         sandbox.upload(script.encode('utf-8'), _SCRIPT)
         timeout = math.ceil(task.agent_timeout_sec or DEFAULT_AGENT_TIMEOUT_S)
+        started = time.monotonic()
+        acting_s = None
         try:
-            backend.exec(f'bash {_SCRIPT}', timeout)
-            timed_out = False
-        except TimeoutError:
-            timed_out = True
-        return AgentRun(backend.archive_workdir(), backend.workdir, timed_out)
+            try:
+                backend.exec(f'bash {_SCRIPT}', timeout)
+                timed_out = False
+            except TimeoutError:
+                timed_out = True
+            acting_s = time.monotonic() - started
+            workspace = backend.archive_workdir()
+        except SandboxExpiredError as exc:
+            acting_s = time.monotonic() - started if acting_s is None else acting_s
+            exhausted = ran_out_the_clock(exc, acting_s, time.monotonic() - opened)
+            return failure_result(exc, sandbox.id, exhausted=exhausted)
+        except Exception as exc:
+            # The script has started: only a failure that is ours for certain has no verdict.
+            return failure_result(exc, sandbox.id)
+        return AgentRun(workspace, backend.workdir, timed_out)
     finally:
         sandbox.delete()  # Never raises; a failed delete is logged, not traded for the run.
 

deepgym/src/deepgym/harbor/environment.py

@@ -97,6 +97,8 @@ def run_with_deepgym(self, dg: DeepGym, model_output: str, **kwargs: object) ->
         start = time.perf_counter()
         env = self if self.image else self.model_copy(update={'image': self._built_image(dg)})
         agent = run_agent(dg.open_sandbox(env), self.harbor_task, model_output)
+        if isinstance(agent, RunResult):  # the agent broke its own sandbox or work directory
+            return agent
         grader = dg.open_sandbox(env)
         result = grade(grader, self.harbor_task, agent.workspace, agent.workdir)
         return build_run_result(