Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -428,6 +428,9 @@ Changes are grouped by software version. Entries under Unreleased describe work

### Fixed

- Continual Harness and Harness² now declare the adapter parts they write (`needs: [llm, skills, agents]` and
`needs: [skills, agents, hooks]`), so a run whose acting agent does not take them (`react`) is refused before it
starts, instead of running a harness whose skills and sub-agents the acting agent drops.
- In a task's Harbor environment, a CLI's output kept in `/logs/agent` as it ran reached a shared verifier unscrubbed
when Harbor's agent timeout ended the phase before Hermit's own logs were copied over it. The phase's end now scrubs
each such stream there of the credentials passed to the agent's commands (their variables and descriptors) before
Expand Down
3 changes: 2 additions & 1 deletion hermit/agent/evolver/methods/continual_harness/evolver.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@ description: Continual Harness's Refiner (Karten et al. 2026, arXiv:2605.09998;
score is optimised and no validation gate is applied - every pass is applied in place, as the paper's reset-free
loop does; a pass whose result fails the template check is dropped, the others stand.
class: evolver.py:ContinualHarnessEvolver
needs: [llm] # it calls the run's model (ctx.agent, the llm tool)
needs: [llm, skills, agents] # it calls the run's model (ctx.agent, the llm tool); the skills and sub-agents its
# Refiner writes, which the acting agent runs
params:
components: [policy, subagents, skills, memory] # the passes to run, in this order
feedback: true # show the judge's feedback to the Refiner (off: trajectory-only, as the paper's setting)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ description: Harness² (Xu et al., arXiv:2610.14224) under the paper's own proto
upstream's claude_cli channel on the bound model or on a login, each in its own sandbox; executions go through
run_agent, with the run's acting agent (Codex, Claude Code or Pi). No judge score reaches any session.
class: evolver.py:Harness2UpstreamEvolver
needs: []
needs: [skills, agents, hooks] # the harness's components the editors write, which the acting agent runs
params:
backend: claude # the editor sessions run Claude Code (upstream's claude_cli channel) on the bound model;
# codex: Codex (a model only Codex reaches: login: chatgpt with model a Codex model name);
Expand Down
14 changes: 14 additions & 0 deletions tests/core/test_environment.py
Original file line number Diff line number Diff line change
Expand Up @@ -294,6 +294,20 @@ def test_a_method_that_needs_tools_is_refused_an_agent_that_takes_none(tmp_path)
assert adapter_needs_errors(sandboxed, ["tools"]) == []


@pytest.mark.parametrize("method,parts", [("ahe_adapter", "skills, agents, hooks"),
("continual_harness", "skills, agents"),
("harness2_upstream", "skills, agents, hooks")])
def test_a_method_whose_harness_has_skills_and_roles_is_refused_react(tmp_path, method, parts):
"""A method that writes skills and sub-agents (and hooks) declares them: a run whose agent drops them is refused."""
from hermit.agent.evolver.loader import find
from hermit.trainer.trainer import adapter_needs_errors
needs = find(method).needs
react = _config(tmp_path, {"name": "react", "params": {"model": {"base_url": "u", "model": "m"}}}, method)
assert f"acting agent {parts}, which agent react" in adapter_needs_errors(react, needs)[0]
codex = _config(tmp_path, {"name": "codex", "params": {"web_search": False}}, method)
assert adapter_needs_errors(codex, needs) == []


def _declared_supports(file: Path, name: str):
"""A task class's or function's `supports`, read from its source without importing it (None: not declared)."""
import ast
Expand Down
Loading