Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); ver

### Added

- Consolidated configuration reference table in docs/configuration.md detailing every environment variable, default value, and reader subsystem.
- Nine agents through the CLI and the MCP server: `allrecipes` and `flights` (browser use), `desktop` (computer use
on Windows or macOS), `ticket_router` (30 labelled tickets to five queues), `blackjack`, `game2048`, `millionaire`,
`alfworld` (games and embodied text), `injection_guard` (rail).
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,6 +140,7 @@ interface fits: [docs/architecture.md](docs/architecture.md), [docs/decision-mod
- [docs/agents.md](docs/agents.md): every agent with its flags, run command and extra.
- [docs/architecture.md](docs/architecture.md) and [docs/decision-models.md](docs/decision-models.md): the fronts, the model slot, the model interface, adding a backend.
- [docs/browser-front.md](docs/browser-front.md): the browser policy, decision by decision.
- [docs/configuration.md](docs/configuration.md): environment variables, defaults and reader subsystems in one table.
- [docs/why.md](docs/why.md): the problem, the philosophy, the precedents.
- [docs/roadmap.md](docs/roadmap.md) and [CHANGELOG.md](CHANGELOG.md).

Expand Down
41 changes: 41 additions & 0 deletions docs/configuration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Configuration reference

Environment variables configure decision models, the comparison chat model, logging paths, driver binaries, and evaluation datasets.

Variables can be exported in your shell or placed in a `.env` file at the root of the repository. Exported environment variables take precedence over values in `.env`.

## Environment variables

| Variable | Who reads it | Default | What it does |
|:---|:---|:---|:---|
| `TYPESAFE_API_KEY` | `jev` model | *(unset)* | API key for direct TypeSafe decisions endpoint (`https://api.typesafe.ai/v1/systemone`). |
| `TYPESAFE_API_URL` | `jev` model | `https://openrouter.ai/api/alpha/decisions` | Endpoint URL for decisions; defaults to OpenRouter proxy, or can be overridden to a custom proxy URL. |
| `TYPESAFE_MODEL` | `jev` model | `typesafe/jev-1.13` | Model identifier when proxying Jev decisions through OpenRouter. |
| `OPENROUTER_API_KEY` | `jev` model (proxy), chat model fallback | *(unset)* | OpenRouter API key, used for proxying Jev decisions or as a fallback for `LLM_API_KEY`. |
| `OPENROUTER_BASE_URL` | chat model fallback | `https://openrouter.ai/api/v1` | Fallback base URL for the chat model when `LLM_BASE_URL` or `OPENAI_BASE_URL` is unset. |
| `MODEL_NAME` | chat model (`llm` model, rethink planner, browser agent) | *(unset)* | Model identifier for the chat model (e.g. `google/gemini-2.5-flash` or `claude-fable-5-1`). |
| `MODEL_PROVIDER` | chat model | `openai` | Provider protocol for the chat model (`openai` or `anthropic`). |
| `OPENAI_API_KEY` | chat model | *(unset)* | API key for OpenAI-compatible endpoints; checked alongside alias `LLM_API_KEY`. |
| `OPENAI_BASE_URL` | chat model | `https://api.openai.com/v1` | Base endpoint URL for OpenAI-compatible chat models; checked alongside alias `LLM_BASE_URL`. |
| `LLM_API_KEY` | chat model | *(unset)* | Alias for `OPENAI_API_KEY`; falls back to `OPENROUTER_API_KEY` when unset. |
| `LLM_BASE_URL` | chat model | *(unset)* | Alias for `OPENAI_BASE_URL`; falls back to `OPENROUTER_BASE_URL` when unset. |
| `ANTHROPIC_WORKSPACE_ID` | chat model | *(unset)* | Organization workspace identifier passed in the `anthropic-workspace-id` header when `MODEL_PROVIDER=anthropic`. |
| `S1A_HOME` | runtime, console, logging | Repository root | Root directory where agent runs, logs, and artifacts are written (`runs/logs/`). |
| `PLAYWRIGHT_MCP_ARGS` | browser agents (`allrecipes`, `flights`), `scripts/browser_showcase.sh` | `npx -y @playwright/[email protected]` | Command arguments passed when launching Playwright MCP, or custom flags like `--cdp-endpoint`. |
| `PLAYWRIGHT_MCP_COMMAND` | `scripts/browser_showcase.sh` | `node` | Executable used to launch the Playwright MCP server process in browser showcase scripts. |
| `ALFWORLD_DATA` | `alfworld` agent, replay | *(unset)* | Directory containing the downloaded ALFWorld benchmark dataset and game files. |
| `CHAT_USD_PER_M_INPUT` | pricing / accounting | *(catalogue)* | Price in USD per million prompt tokens when `MODEL_NAME` is not in OpenRouter's public catalogue. |
| `CHAT_USD_PER_M_OUTPUT` | pricing / accounting | *(catalogue)* | Price in USD per million completion tokens when `MODEL_NAME` is not in OpenRouter's public catalogue. |
| `CHAT_USD_PER_M_CACHED_INPUT` | pricing / accounting | `CHAT_USD_PER_M_INPUT` | Price in USD per million cached input tokens for non-catalogue models. |
| `LAYA_MODEL` | `laya` model | `convaiinnovations/laya` | Hugging Face repository ID or local path for the resident Laya decision model checkpoint. |
| `LAYA_SUBFOLDER` | `laya` model | *(unset)* | Optional subfolder in the checkpoint repo (e.g. `multilingual` or `typed-decisions`). |
| `LAYA_DEVICE` | `laya` model | `(library default)` | PyTorch device for Laya model evaluation; passes None so the library selects CUDA, MPS, or CPU. |
| `LAYA_MAX_LEN` | `laya` model | `(checkpoint default)` | Maximum token sequence length for Laya state representation; overrides checkpoint window only when set. |
| `LAYA_HEAD_MAX_LEN` | `laya` model | `(checkpoint default)` | Maximum token sequence length for Laya decision head options; overrides checkpoint window only when set. |
| `CUA_S1_CHECKPOINT` | `cua` model | `cua-ai/cua-s1-nano-0.1` | Hugging Face checkpoint ID or local directory for Cua-S1 Nano option scorer. |
| `CUA_S1_SUBFOLDER` | `cua` model | `text` | Subfolder within checkpoint directory containing text option scoring weights. |
| `CUA_S1_DEVICE` | `cua` model | `auto` | PyTorch device used for Cua-S1 Nano evaluation (`auto`, `cpu`, `cuda`, or `mps`). |
| `CUA_DRIVER_BIN` | `desktop` agent | `cua-driver` | Path to the `cua-driver` executable on Windows or macOS when not located on `PATH`. |
| `CUA_DRIVER_PERMISSION_MODE` | `desktop` agent | `standard` | Permission mode passed to `cua-driver mcp` (`standard`, or `bounded` for restricted capability manifests). |
| `HF_HOME` | Hugging Face runtime | `~/.cache/huggingface` | Cache directory where Laya and Cua-S1 checkpoints are downloaded on first run. |
| `HF_HUB_OFFLINE` | Hugging Face runtime | `0` | When set to `1`, prevents network requests and forces models to load exclusively from local cache. |
10 changes: 5 additions & 5 deletions docs/decision-models.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,9 +42,9 @@ shorthands; `warm()` and `close()` open and release the backend.

| `--model` | class | `name` | notes |
|---|---|---|---|
| `jev` | `JevModel(transport)` | `jev` | the request body every front sent before the layer existed, byte for byte; `from_env` picks TypeSafe or the OpenRouter proxy |
| `laya` | `LayaModel(agent, model=)` | `laya` | one forward pass per call on a thread; `MODEL_SERVICE_CONFIG_ERROR` when `input_tokens` fills the window (Laya cuts the state silently; `LAYA_MAX_LEN`, `LAYA_HEAD_MAX_LEN` widen it); `ValueError` and `RuntimeError` from the library become `MODEL_CALL_FAILED` |
| `cua` | `CuaS1Model(scorer, collator, model=, context_bytes=, option_bytes=)` | `cua` | Cua-S1 Nano, one `score_elements` pass per request on a thread; choice questions only, text only, deterministic; the context is header, state and rules; the checkpoint reads its first 256 bytes, and the first overflowing request logs one warning; `from_env` reads `CUA_S1_CHECKPOINT`, `CUA_S1_SUBFOLDER`, `CUA_S1_DEVICE` |
| `jev` | `JevModel(transport)` | `jev` | the request body every front sent before the layer existed, byte for byte; `from_env` picks TypeSafe or the OpenRouter proxy (see [configuration.md](configuration.md)) |
| `laya` | `LayaModel(agent, model=)` | `laya` | one forward pass per call on a thread; `MODEL_SERVICE_CONFIG_ERROR` when `input_tokens` fills the window (Laya cuts the state silently; see `LAYA_MAX_LEN` and `LAYA_HEAD_MAX_LEN` in [configuration.md](configuration.md)); `ValueError` and `RuntimeError` from the library become `MODEL_CALL_FAILED` |
| `cua` | `CuaS1Model(scorer, collator, model=, context_bytes=, option_bytes=)` | `cua` | Cua-S1 Nano, one `score_elements` pass per request on a thread; choice questions only, text only, deterministic; the context is header, state and rules; the checkpoint reads its first 256 bytes, and the first overflowing request logs one warning; `from_env` reads `CUA_S1_*` (see [configuration.md](configuration.md)) |
| `random` | `RandomModel(seed)` | `random` | uniform over the offered keys, confidence 0, one seeded stream per episode; choice questions only |
| `rule` | `RuleModel(name, rule)` | the rule's name | one-hot, confidence 1; a key outside the menu raises `RuntimeError`, a bug in the rule |

Expand All @@ -53,8 +53,8 @@ shorthands; `warm()` and `close()` open and release the backend.
TypeSafe Jev answers `--model jev`. One request holds a `state` and one or more questions over options the caller
enumerates; the answer holds one option per question, a probability per option and a confidence, from one forward
pass, with no free text. Three heads: `choice` picks one key among the options, `noul` gives the probability that a
statement holds, `score` places the state on an ordered rubric. Input is capped at 32K tokens; the endpoint is
`api.typesafe.ai` with `TYPESAFE_API_KEY`, or OpenRouter's `typesafe/jev-1.13` on `/api/alpha/decisions`. Latency and
statement holds, `score` places the state on an ordered rubric. Input is capped at 32K tokens; endpoint configuration
and API keys are referenced in [configuration.md](configuration.md). Latency and
price: `benchmarks.md`. Laya (Convai Innovations, open weights, 0.4B parameters) and Cua-S1 Nano (Cua, 855K
parameters) answer the same `choice` question in process.

Expand Down
2 changes: 1 addition & 1 deletion docs/skills.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ claude plugin marketplace add ThinkFlowLab/system1-agents && claude plugin insta
The plugin's MCP server starts as `uv run --project <plugin root> s1a-mcp`, with `uv` on `PATH`; the first start runs
`uv sync` in the plugin folder and takes minutes to resolve the `openjiuwen` git pin. Export `TYPESAFE_API_KEY` (or
`OPENROUTER_API_KEY`) in the shell that launches Claude Code, since the plugin folder has no `.env`. Runs and logs
land under the plugin folder unless `S1A_HOME` names another root. Every host runs the command from a checkout of
land under the plugin folder unless `S1A_HOME` names another root ([configuration.md](configuration.md)). Every host runs the command from a checkout of
this repository.

### A ticket through the skill
Expand Down
15 changes: 5 additions & 10 deletions evals/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,14 +52,10 @@ the score statistics over the `scored` episodes only, and `evals.table` skips tr
## Setup

`uv sync`, plus `--extra blackjack` and `--extra alfworld` for those games (the README lists every extra), then
a `.env` with `TYPESAFE_API_KEY` or `OPENROUTER_API_KEY` for Jev.
`llm`, the rethink planner and the browser agent need the chat model: `OPENAI_API_KEY` (or `LLM_API_KEY`),
`OPENAI_BASE_URL` (or `LLM_BASE_URL`) and `MODEL_NAME`. `MODEL_PROVIDER=anthropic` talks Anthropic's own protocol, direct
(`OPENAI_BASE_URL=https://api.anthropic.com`, `MODEL_NAME=claude-fable-5-1`; an org-level key also needs
`ANTHROPIC_WORKSPACE_ID`) or through OpenRouter's `/v1/messages` (`MODEL_NAME=anthropic/claude-fable-5.1`). The browser agents launch a headless
Chromium through `@playwright/mcp` (Node); `--headed` shows it. ALFWorld needs `uv sync --extra alfworld` and
`ALFWORLD_DATA` in a Python 3.11 environment; the data comes from `python scripts/alfworld-download` in a clone
of alfworld/alfworld. The DeepAgent's workspace files land under `runs/evals/`.
a `.env` with `TYPESAFE_API_KEY` (or `OPENROUTER_API_KEY`) for Jev, and `MODEL_NAME` with provider credentials for the chat model.
ALFWorld needs `uv sync --extra alfworld` and `ALFWORLD_DATA` in a Python 3.11 environment; the data comes from
`python scripts/alfworld-download` in a clone of alfworld/alfworld. The DeepAgent's workspace files land under `runs/evals/`.
For the complete reference table of all environment variables, provider endpoints, and defaults, see [docs/configuration.md](../docs/configuration.md).

## Protocol

Expand All @@ -71,8 +67,7 @@ seeds (same loop, swap the brain); `jev` with `--rethink on` against `off`; the
act against a bare loop (loop overhead); and, later, Jev's top probability against the ALFWorld expert plan.
`summary.json` also holds decisions, chat calls, tokens (`chat_input_tokens`, `chat_output_tokens`,
`chat_cache_tokens`) and `cost_usd` (Jev at $0.042 per M input tokens; the chat model at OpenRouter's catalogue
price for `MODEL_NAME`, with cached input at the catalogue's cache-read rate; or `CHAT_USD_PER_M_INPUT`,
`CHAT_USD_PER_M_OUTPUT` and, optionally, `CHAT_USD_PER_M_CACHED_INPUT`). `python -m evals.table evals/results` prints one row per eval and model over every
price for `MODEL_NAME`, or custom rates from `CHAT_USD_PER_M_*`, see [docs/configuration.md](../docs/configuration.md)). `python -m evals.table evals/results` prints one row per eval and model over every
job folder. ALFWorld's game files sort by task type; `--stride 11` from offset 0 takes twelve games across
the six types. Every model plays the same tile draws because 2048 seeds the page's `Math.random`.

Expand Down