diff --git a/CHANGELOG.md b/CHANGELOG.md index bd2b2aa..f6f5d18 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,60 @@ All notable changes to this project are documented here. The format follows ## [Unreleased] +## [0.38.1] - 2026-09-05 + +Two defects that switched a feature off without saying so, found by driving the released 0.38.0 +the way a user drives it rather than by running its test suite. + +`io config set` wrote a quoted string for every key it did not know by name, and +`[app.io-cli.containment]` and `[run.context]` are two such sections — so `max_total_agents 4` +reached the file as `"4"`, io-harness refused to read the section back, and fan-out was simply +off behind one warning line. Meanwhile the in-session `/config` never went through that code at +all and wrote the operator's raw text as TOML source, so the same key typed the same way got two +different answers depending on which door it came through. There is now one speller and both +doors call it. + +A failing verification gate under `io exec` never ended the run. io-harness evaluates the +criterion after every step and hands the failure back, which is a reason for the agent to keep +going, so the only thing that stopped it was this crate's own step cap of a thousand — and that +constant is documented as safe precisely because three other bounds exist, every one of which a +gate removes. A gated headless run now ends, and exit `6` arrives. + +### Changed + +- **Behaviour change for scripts.** `io config get` on a key io-cli's catalogue does not name now + reads back an unquoted number or boolean where it read back a quoted string. A script that + matched on `"4"` will see `4`. This is the fix, not a side effect: the quoted form is what + io-harness refused to deserialize. +- **Behaviour change for CI.** A gated `io exec` that previously ran to the step cap now ends and + exits `6` (`UNVERIFIED`) once its criterion has failed and nothing else bounds the run. A job + that relied on a timeout to stop it will now fail fast instead, which is the documented + behaviour it should always have had. A gated run that genuinely needs more steps sets + `[run] max_steps`, `max_duration_secs` or `max_tokens` — any of the three wins over the new + bound. +- `/config` refuses a value the key does not admit at the moment it is typed, rather than opening + a picker asking which file to write a value into that will not be written. + +### Fixed + +- `io config set` types a bare value as it was written for keys outside io-cli's catalogue: an + integer as an integer, a float as a float, `true` and `false` as booleans, everything else as a + quoted string. `app.io-cli.containment.*` and `run.context.*` can be set from the shell again. +- `/config ` in a session goes through the same speller as `io config set`, so + `policy.defaults.net allow` is written as a string instead of being refused as invalid TOML. +- `io exec` refuses before the first step when `[app.io-cli]` will not deserialize while naming a + gate, instead of running with no criterion and reporting success. When the broken section names + no gate, the notice is printed rather than swallowed — headless never printed it at all. + +### Dependencies + +- io-harness `0.78` → `0.79`. No breaking change is declared and no line of its `src/` was + deleted. It adds `EventKind::Program` and the `run_program` tool name **unconditionally** rather + than behind its `codeact` feature, which this crate does not enable, so `triage::TRIAGE`, + `acp_map::MAPPING` and `acp_map::TOOL_KINDS` each take a row for something no run here can + currently emit. Three test assertions that counted event kinds by literal now derive the count + instead, so the next such addition names the new kind rather than a number. + ## [0.38.0] - 2026-09-04 The release that lets you install io the way you install everything else. Distribution was a @@ -3461,6 +3515,7 @@ client, tool, sandbox, policy engine or session store of its own. - No test in this release asserts on wall-clock time. [Unreleased]: https://github.com/initorigin/io-cli/compare/v0.38.0...HEAD +[0.38.1]: https://github.com/initorigin/io-cli/compare/v0.38.0...v0.38.1 [0.38.0]: https://github.com/initorigin/io-cli/compare/v0.37.0...v0.38.0 [0.37.0]: https://github.com/initorigin/io-cli/compare/v0.36.0...v0.37.0 [0.36.0]: https://github.com/initorigin/io-cli/compare/v0.35.0...v0.36.0 diff --git a/Cargo.lock b/Cargo.lock index 418deb9..5ec879e 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1850,7 +1850,7 @@ dependencies = [ [[package]] name = "io-cli" -version = "0.38.0" +version = "0.38.1" dependencies = [ "clap", "crossterm", @@ -1867,9 +1867,9 @@ dependencies = [ [[package]] name = "io-harness" -version = "0.78.0" +version = "0.79.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "dd8b7f233c0ef74d0da98f5c9983021688379c2652bfe825cfad44e5bd5aeb02" +checksum = "73599a2de1b1bf3630c72e24836f01354505d60522dc8f6d09e179687ad486f7" dependencies = [ "base64 0.23.1", "calamine", diff --git a/Cargo.toml b/Cargo.toml index ba51311..49cd8c2 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "io-cli" -version = "0.38.0" +version = "0.38.1" edition = "2021" # io-harness declares 1.95, set by `libsqlite3-sys` calling the std `cfg_select!` # macro from its build script. Nothing here can be lower than what the harness @@ -63,7 +63,7 @@ path = "src/lib.rs" # still correct about what this *renderer* can be handed; it is simply no longer # load-bearing on the build. Turning `barcode` back off is what would restore # it, and `documents` would then have to be spelled out as its other four. -io-harness = { version = "0.78", features = ["media", "browser", "documents"] } +io-harness = { version = "0.79", features = ["media", "browser", "documents"] } # **`tui-textarea` is gone as of 0.18.0, and it is why this pair was frozen.** # The comment that stood here said ratatui stayed at 0.29 and crossterm at 0.28 # because the composer was built out of that widget and the 0.30 line carried diff --git a/Formula/io.rb b/Formula/io.rb index c33b909..1721c39 100644 --- a/Formula/io.rb +++ b/Formula/io.rb @@ -32,13 +32,13 @@ class Io < Formula on_macos do if Hardware::CPU.arm? - url "https://github.com/initorigin/io-cli/releases/download/v0.37.0/io-0.37.0-aarch64-apple-darwin.tar.gz" - sha256 "93fa6ea5a85535457056207cf8ddde44c3a2b1f58dd644e78b26853f4169069f" + url "https://github.com/initorigin/io-cli/releases/download/v0.38.0/io-0.38.0-aarch64-apple-darwin.tar.gz" + sha256 "e38a9a4ac49b6149049d19b2d1398d42da6ed33f3b5d3c5f9f4f5933b725dc79" end if Hardware::CPU.intel? - url "https://github.com/initorigin/io-cli/releases/download/v0.37.0/io-0.37.0-x86_64-apple-darwin.tar.gz" - sha256 "f070a8d2afed9498a8a08bbf49683744b5741dda90319291430b99bcb98e08f8" + url "https://github.com/initorigin/io-cli/releases/download/v0.38.0/io-0.38.0-x86_64-apple-darwin.tar.gz" + sha256 "2f7f8923a6c918a4f8bae9fc33f6c31888a91456a2184daf2eb86978a6777b68" end end @@ -47,8 +47,8 @@ class Io < Formula # rather than a URL that 404s halfway through an install. on_linux do if Hardware::CPU.intel? - url "https://github.com/initorigin/io-cli/releases/download/v0.37.0/io-0.37.0-x86_64-unknown-linux-musl.tar.gz" - sha256 "1196630b787c343e96953a4091e78908d812c39a10876655c4afe6c3dbc2faf3" + url "https://github.com/initorigin/io-cli/releases/download/v0.38.0/io-0.38.0-x86_64-unknown-linux-musl.tar.gz" + sha256 "26a6e918526800047ada4050b5e6e296ee45c1a4ab5881d46741fc520b015a0c" end end diff --git a/bucket/io.json b/bucket/io.json index 19516b7..a557176 100644 --- a/bucket/io.json +++ b/bucket/io.json @@ -1,13 +1,13 @@ { - "version": "0.37.0", + "version": "0.38.0", "description": "Terminal agent that shows what it may do, what it spends and what it refused", "homepage": "https://github.com/initorigin/io-cli", "license": "Apache-2.0", "architecture": { "64bit": { - "url": "https://github.com/initorigin/io-cli/releases/download/v0.37.0/io-0.37.0-x86_64-pc-windows-msvc.zip", - "hash": "a6011df7775bdcc4d3a8b5e026690c0b010ee42c7d018c595196cb323d21ce57", - "extract_dir": "io-0.37.0-x86_64-pc-windows-msvc" + "url": "https://github.com/initorigin/io-cli/releases/download/v0.38.0/io-0.38.0-x86_64-pc-windows-msvc.zip", + "hash": "6ec1b0db26a25188e49c5c7d20084ef3c5538b50be18ab74692f06eaaf4859c3", + "extract_dir": "io-0.38.0-x86_64-pc-windows-msvc" } }, "bin": "io.exe", diff --git a/src/acp_map.rs b/src/acp_map.rs index a23c2e4..4296e79 100644 --- a/src/acp_map.rs +++ b/src/acp_map.rs @@ -300,6 +300,20 @@ pub const MAPPING: &[(&str, Update, &str)] = &[ Update::None, "as `sandbox` — the boundary is io-cli's status line", ), + // 0.79.0, and placed where `EventKind` declares it — between `contained` and + // `dialed` — because this table's ordering rule is that it can be read down + // the side of `observe.rs` when the pin moves. Declared unconditionally by + // the harness and emitted only behind its `codeact` feature, which this crate + // does not enable, so the row is here because the table is total over what + // the harness declares. + ( + "program", + Update::None, + "the acts a program took are not on this event — each re-enters dispatch \ + and arrives as its own `tool_call`, which is already translated; \ + announcing the program itself would put a tool call in the editor that \ + has no `call_id` and never completes", + ), ( "dialed", Update::None, @@ -396,6 +410,11 @@ pub const TOOL_KINDS: &[(&str, &str)] = &[ ("git_branch", "execute"), ("git_worktree", "execute"), ("spawn_agent", "execute"), + // 0.79.0, and `execute` is the only honest kind: it runs a program the model + // wrote, under the run's own exec mode. Declared unconditionally by + // `tools/mod.rs`, so this row is needed whether or not this crate enables + // io-harness's `codeact` feature — which today it does not. + ("run_program", "execute"), // Fetches — the browser reaches something outside the workspace. ("browser_navigate", "fetch"), ("browser_screenshot", "fetch"), diff --git a/src/adapt.rs b/src/adapt.rs index 6bd7579..ba9b06e 100644 --- a/src/adapt.rs +++ b/src/adapt.rs @@ -70,7 +70,7 @@ //! `self.root.join(d)` and `Path::join` of an absolute argument discards the base. //! io-harness 0.74.0 refuses that spelling for `skills` and for `templates` in //! **every** scope, `Scope::User` included -//! (`io-harness-0.78.0/src/plugin.rs:1097`), and the reason is the one this +//! (`io-harness-0.79.0/src/plugin.rs:1097`), and the reason is the one this //! product would give: every `*.md` under such a directory is read into the //! model's system prompt on every turn, so a manifest contributes a directory it //! *ships* rather than one it points at somewhere else on the machine. Copying is @@ -377,7 +377,7 @@ pub const SKILLS_DIR: &str = "skills"; /// io-harness calls the same thing a **template**, and the two formats are the /// same file: markdown, optional frontmatter, `$ARGUMENTS` for what the operator /// typed. `Templates::discover` takes every `*.md` in a directory -/// (`io-harness-0.78.0/src/template.rs:279`), so what is translated is the +/// (`io-harness-0.79.0/src/template.rs:279`), so what is translated is the /// directory and never the files in it — the bytes are copied verbatim into /// [`TEMPLATES_DIR`] and nothing in them is rewritten. pub const COMMANDS_DIR: &str = "commands"; @@ -578,7 +578,7 @@ pub fn at(root: &Path, owner: &str, repo: &str, name: &str) -> PathBuf { /// /// **The two directories are copied into `into` and named relative to it.** /// io-harness 0.74.0 refuses an absolute or a `..`-carrying `skills` or -/// `templates` in every scope (`io-harness-0.78.0/src/plugin.rs:1097`), because +/// `templates` in every scope (`io-harness-0.79.0/src/plugin.rs:1097`), because /// every `*.md` under one is read into the model's system prompt — so a bundle /// contributes a directory it ships. An adapter comes to ship one by copying it, /// and [`Adapter::copied`] reports which moved. The copy refuses a symbolic link @@ -714,7 +714,7 @@ pub fn generate(bundle: &Path, name: &str, into: &Path) -> Result Option { /// adapted, so expanding it reads nothing of the host — it writes down a path io /// already knew. `${env:…}`, `${file:…}`, `${cmd:…}` and every shell variable a /// bundle writes are the opposite: io-harness refuses each of them inside a -/// manifest in every scope (`io-harness-0.78.0/src/plugin.rs:977`), because a +/// manifest in every scope (`io-harness-0.79.0/src/plugin.rs:977`), because a /// bundle is a third party's directory even when the file naming it is the /// operator's own. /// @@ -1252,7 +1252,7 @@ fn unquoted(said: &str) -> &str { /// /// **A transport io-harness does not speak is a refusal naming the server**, not /// a server quietly left out. io-harness speaks stdio and streamable HTTP -/// (`io-harness-0.78.0/src/mcp.rs:301`), so a bundle declaring `sse` is declaring +/// (`io-harness-0.79.0/src/mcp.rs:301`), so a bundle declaring `sse` is declaring /// a capability no adapter can deliver — and a manifest carrying one fewer server /// than the bundle does is exactly the silent absence this module's docs exist to /// end. The whole generation stops, because a bundle is installed for what it diff --git a/src/commands.rs b/src/commands.rs index 8474ff9..d5e142e 100644 --- a/src/commands.rs +++ b/src/commands.rs @@ -1489,7 +1489,7 @@ pub const READ_MARK: &str = "+"; /// The case this page exists for. A file that is there and is not read looks, /// from everywhere else in this product, exactly like one that is read: nothing /// warns, and a missing or skipped instruction file is passed over in silence -/// (`read_instructions`, `io-harness-0.78.0/src/config.rs:2911`). +/// (`read_instructions`, `io-harness-0.79.0/src/config.rs:2997`). pub const UNREAD_MARK: &str = "-"; /// What one row of the memory page stands for. @@ -1705,8 +1705,8 @@ pub fn memory_notes(view: &crate::recall::View, glyphs: &crate::glyphs::Glyphs) /// writing: a file that exists and is not read. It names what would make it /// read rather than only reporting the fact, because `[instructions] files` /// **replaces** the default list rather than adding to it -/// (`DEFAULT_INSTRUCTIONS`, `io-harness-0.78.0/src/config.rs:190`, read at -/// `:2808`) and there is nothing on any +/// (`DEFAULT_INSTRUCTIONS`, `io-harness-0.79.0/src/config.rs:190`, read at +/// `:3006`) and there is nothing on any /// other surface to pull on. pub fn instruction_said( file: &crate::memory::Instruction, diff --git a/src/configure.rs b/src/configure.rs index 6119e89..c9d3847 100644 --- a/src/configure.rs +++ b/src/configure.rs @@ -255,7 +255,7 @@ pub enum Kind { /// /// **Both halves are the dependency's since io-harness 0.71.0, and neither is /// written here any more**: the list is `Effect::ALL` -/// (`io-harness-0.78.0/src/policy.rs:129`) and each spelling is `Effect::as_str` +/// (`io-harness-0.79.0/src/policy.rs:129`) and each spelling is `Effect::as_str` /// (`:145`), which is the word io-harness's own deserializer reads. /// /// Until this release io-cli held a copy of both — an array naming three variants @@ -282,7 +282,7 @@ pub fn effects() -> Vec { /// The `ExecMode` variants, spelled by io-harness itself. /// -/// **The list is `ExecMode::ALL` (`io-harness-0.78.0/src/sandbox.rs:453`) and the +/// **The list is `ExecMode::ALL` (`io-harness-0.79.0/src/sandbox.rs:453`) and the /// spellings are `ExecMode::as_str` (`:460`).** io-cli wrote the variant list out /// by hand until this release for a reason that was the dependency's and not a /// choice made here: `ExecMode` is `#[non_exhaustive]` (`sandbox.rs:407`), and @@ -326,12 +326,267 @@ pub fn exec_mode_label(mode: io_harness::ExecMode) -> String { } } +/// The dotted prefix of io-cli's own configuration section. +/// +/// Built from [`crate::settings::APP_KEY`] rather than spelled, so the two cannot +/// drift: that constant is what `Config::app` is asked for, and this is the same +/// name as it appears in a dotted key. See [`source_for`]'s `None` arms for why +/// the distinction between this section and io-harness's carries weight. +pub const APP_PREFIX: &str = "app.io-cli."; + +/// The TOML source for `words`, checked against what the key admits. +/// +/// **One speller, and both doors call it (0.38.1).** io-harness owns what a +/// setting *means*; [`kind_of`] answers only how a value is obtained, and this +/// turns a typed word into the source that expresses it. It lived in +/// `crate::manage` while `io config set` was the only door that spelled anything +/// — the session's `/config` handed `crate::main`'s `write_where` the operator's +/// raw text as TOML source with no speller at all. So the same key typed the same +/// way got two answers, and the 2026-09-05 field test found both of them: +/// `app.io-cli.containment.max_total_agents 4` was written `"4"` by the shell +/// door and `4` by the session, and `policy.defaults.net allow` was refused by +/// the session because `allow` is not a TOML value. +/// +/// **A key the catalogue does not name is no longer quoted unconditionally, and +/// that arm was the whole defect.** It quoted every unknown key's value, so every +/// number under `[app.io-cli.containment]` and `[run.context]` — sections +/// `kind_of` has never named — reached the file as a string, io-harness refused +/// to deserialize the section, and the session ran on defaults behind one warning +/// line. Reading the shape off the word the operator typed is not io-cli +/// inventing a schema: `4` typed at a command line means the number four in every +/// language a person has used before this one, and `configure::write`'s round +/// trip still refuses and rolls back a value io-harness will not read back, in +/// io-harness's own words. A refusal at write time is strictly better than a +/// silently wrong file. +pub fn source_for(key: &str, words: &[String]) -> Result { + let kind = kind_of(key); + if matches!(kind, Some(Kind::Machine)) { + return Err(format!( + "`{key}` is written by io itself rather than typed — it dates the price table, and a \ + date set by hand is a claim about a fetch that never happened; `/config` offers the \ + refresh that writes it" + )); + } + // The one key whose value is a list (`app.io-cli.gates.command`), so the + // remaining words are the value rather than a mistake. A scalar written there + // is a value io-harness cannot read back. + if matches!(kind, Some(Kind::List)) { + if words.is_empty() { + // **No command word in the sentence, because two doors reach it + // (0.38.1).** These refusals were written when `io config set` was + // the only caller and named it outright; the session's `/config` now + // reaches the same function, and an operator who typed a slash + // command was being told to run a shell one. Naming the value + // instead of the invocation is true on both surfaces and shorter. + return Err(format!( + "`{key}` is a command and its arguments, so it needs at least one word — the \ + program first, as in `cargo test`" + )); + } + let items: Vec<&str> = words.iter().map(String::as_str).collect(); + return Ok(crate::edit::array(&items)); + } + + let word = match words { + [word] => word.as_str(), + [] => { + return Err(format!( + "`{key}` has no value after it; {}", + offered(key, kind.as_ref()) + )) + } + _ => { + return Err(format!( + "`{key}` takes one value and {} words were given ({}); quote a value that \ + contains a space", + words.len(), + words.join(" ") + )) + } + }; + + match kind { + Some(Kind::Flag) => match word { + "true" | "false" => Ok(word.to_string()), + other => Err(format!( + "`{other}` is not a value `{key}` takes; it is on or off, written `true` or \ + `false`" + )), + }, + Some(Kind::Choice(options)) => { + if options.iter().any(|option| option == word) { + Ok(crate::servers::quoted(word)) + } else { + Err(format!( + "`{word}` is not a value `{key}` takes; the options are {}", + options + .iter() + .map(|option| format!("`{option}`")) + .collect::>() + .join(", ") + )) + } + } + Some(Kind::Number { signed }) => { + let number: i64 = word.parse().map_err(|_| { + format!("`{word}` is not a whole number, and `{key}` counts in whole numbers") + })?; + if !signed && number < 0 { + return Err(format!( + "`{key}` counts something and cannot be negative; `{word}` was not written" + )); + } + // Re-rendered from the parsed number rather than passed through, so + // that a form TOML would reject (`+5`, a leading zero run) becomes + // the value it meant instead of a refused write. + Ok(number.to_string()) + } + // A model name, a path and free text are strings in the file, and all + // three are escaped rather than wrapped in quotes by hand. + Some(Kind::Model | Kind::File | Kind::Text) => Ok(crate::servers::quoted(word)), + // A key the catalogue does not name. Its shape is read off the word — + // **except under `[app.io-cli]`, where it is quoted as it always was.** + // + // The asymmetry is not timidity, it is where the round trip reaches. + // `configure::write` re-discovers through io-harness, so a wrong shape + // written to one of *its* sections is refused and rolled back in its own + // words at the moment of writing. io-harness does not type `[app.io-cli]` + // at all — it reads the table as an opaque value and hands it over — so + // this crate's `CliSettings` is what fails, at the next session start, + // behind a single warning line. That is precisely how + // `max_total_agents = "4"` reached a file and switched fan-out off in + // silence, and inferring into the same section would buy the same failure + // with the types reversed: `app.io-cli.keys.accept 1` is a legitimate + // single-character binding whose map is `BTreeMap`, and a + // bare `1` there takes the whole section down with it. + // + // So io-cli's own section is typed by io-cli's own catalogue, above, and + // a key it does not name is left as a string. Adding a key to + // `[app.io-cli]` therefore means adding it to `CATALOGUE` and `kind_of`, + // which is a smaller obligation than it looks: this crate defines that + // schema, so it is the one place that can. + None if key.starts_with(APP_PREFIX) => Ok(crate::servers::quoted(word)), + None => Ok(inferred(word)), + // Both are answered above, before a single word was demanded, so this arm + // is unreachable — and it is written as a refusal rather than left to a + // wildcard or a `panic!`. A wildcard would swallow a `Kind` variant a + // later release adds and write it as a string; a panic would end the + // process on something an operator typed. `Kind` is io-cli's own enum, so + // the exhaustive match is a compile-time gate this crate can keep. + Some(Kind::List | Kind::Machine) => Err(format!( + "`{key}` was read as two different kinds in one parse, which is a defect in io \ + rather than in what you typed; nothing was written" + )), + } +} + +/// The TOML source for a word under a key no catalogue entry names. +/// +/// **Four shapes, in the order TOML itself resolves them, and a string last.** +/// A bare `true`/`false` is a boolean, an integer literal is an integer, a float +/// literal is a float, and everything else is a quoted string — which is what +/// every unknown key got unconditionally before 0.38.1. +/// +/// The integer is re-rendered from the parsed value for the reason +/// [`Kind::Number`] is above: `+5` and `007` are shapes a person types and TOML +/// rejects, and writing back what the number *meant* is better than writing a +/// line io-harness cannot read. The float is passed through instead, because +/// re-rendering an `f64` changes `1.10` to `1.1` and `1e3` to `1000` — a +/// round-trip that alters the operator's own text without being asked to. +/// +/// **`inf` and `nan` are strings here, deliberately.** They are valid TOML floats +/// and `f64::from_str` accepts both, so admitting them would mean a key whose +/// value is the word `nan` — a plausible model name, a plausible label — silently +/// becoming a float. No `[app.io-cli]` or io-harness key takes a non-finite +/// number, so the finite check costs nothing real and stops the one shape where +/// inference would surprise. +fn inferred(word: &str) -> String { + if matches!(word, "true" | "false") { + return word.to_string(); + } + if let Ok(number) = word.parse::() { + return number.to_string(); + } + if word.parse::().is_ok_and(f64::is_finite) { + return word.to_string(); + } + crate::servers::quoted(word) +} + +/// The words a composer line's value half is made of. +/// +/// **The composer is not a shell, and this is the whole of the difference +/// (0.38.1).** `io config set k a b` arrives at [`source_for`] as three argv +/// words with the shell's own quoting already applied; `/config k a b` arrives as +/// one string, because the composer does no splitting and there is no shell +/// behind it to do any. Handing that string to `source_for` whole would make +/// every multi-word value a single word containing spaces, which is right for a +/// rubric and wrong for `app.io-cli.gates.command`. +/// +/// So: one surrounding pair of double quotes is stripped and the inside is one +/// word — the operator quoted it precisely to say "this is one value" and there +/// was no shell to honour that — and anything else splits on whitespace, which is +/// what a shell would have done. +#[must_use] +pub fn composer_words(raw: &str) -> Vec { + let trimmed = raw.trim(); + // **One pair, and nothing quoted inside it.** `"cargo" "test"` also starts + // and ends with a quote, and stripping the outer pair would hand back the + // single word `cargo" "test` — a one-element command naming a binary that + // does not exist, which TOML accepts, `Vec` accepts and the round + // trip accepts. A wrong value written silently is worse than a refusal, and + // worse than the whitespace split this now falls through to. Found by the + // adversarial review before the merge. + // + // The interior check is on the slice *between* the outer quotes, so a value + // that legitimately contains an escaped quote is unaffected, and `""` is + // still one empty word. + let inner = trimmed + .strip_prefix('"') + .and_then(|rest| rest.strip_suffix('"')); + if let Some(inner) = inner { + if !inner.contains('"') { + return vec![inner.to_string()]; + } + } + trimmed + .split_whitespace() + .map(std::string::ToString::to_string) + .collect() +} + +/// What to say a key admits, for the refusal that has no value to name. +fn offered(key: &str, kind: Option<&Kind>) -> String { + match kind { + Some(Kind::Flag) => "it is on or off, written `true` or `false`".to_string(), + Some(Kind::Choice(options)) => format!( + "the options are {}", + options + .iter() + .map(|option| format!("`{option}`")) + .collect::>() + .join(", ") + ), + Some(Kind::Number { .. }) => "it takes a whole number".to_string(), + // Named without a command word for the same reason as the two refusals + // above: both doors reach this sentence. + _ => format!("reading `{key}` shows what it is set to now"), + } +} + /// The kind of a catalogue key, or `None` for a key no catalogue entry names. /// /// `None` is not a gap to fill in: [`settings`] deliberately lists keys an -/// operator wrote which this catalogue does not know about, and a kind guessed for -/// one of those would be io-cli inventing a schema. Such a key stays readable and -/// is edited as text. +/// operator wrote which this catalogue does not know about, and a kind invented +/// for one of those would be io-cli deciding a schema it does not own. Such a key +/// stays readable and is edited as text. +/// +/// **What [`source_for`] does with a `None` is a narrower question and it is not +/// this one (0.38.1).** It reads the *scalar shape* off the word the operator +/// typed — `4` is an integer, `true` is a boolean, everything else is a string — +/// which decides how the value is spelled in the file and says nothing about what +/// the key means or what values it admits. Those two remain io-harness's, and its +/// own round trip is still what refuses a value it cannot read back. #[must_use] pub fn kind_of(key: &str) -> Option { Some(match key { @@ -366,7 +621,22 @@ pub fn kind_of(key: &str) -> Option { | "app.io-cli.spawn_background_after_secs" | "app.io-cli.gates.retries" | "app.io-cli.routing.escalate_after.failures" - | "app.io-cli.routing.downshift_under.bytes" => Kind::Number { signed: false }, + | "app.io-cli.routing.downshift_under.bytes" + // **`[app.io-cli.containment]`, named here in 0.38.1 rather than left to + // inference.** These are `io_harness::Containment`'s own fields and the + // section is io-cli's, so the catalogue is where they belong — and it is + // the only place they can be typed *safely*. Nothing validates + // `[app.io-cli]` on the way back: io-harness reads it as an opaque value + // and this crate's `CliSettings` is what fails, at the next session + // start, behind one warning line. That is exactly how + // `max_total_agents = "4"` shipped in the first place, and it means + // `configure::write`'s round trip is not a safety net for this section + // the way it is for io-harness's own. + | "app.io-cli.containment.max_total_agents" + | "app.io-cli.containment.max_concurrent_agents" + | "app.io-cli.containment.max_depth" + | "app.io-cli.containment.max_total_tokens" + | "app.io-cli.containment.max_total_cost" => Kind::Number { signed: false }, "app.io-cli.gates.reviewer" | "app.io-cli.routing.escalate_after.model" | "app.io-cli.routing.downshift_under.model" => Kind::Model, @@ -386,7 +656,7 @@ pub fn kind_of(key: &str) -> Option { /// preference — but half of the old reason is now false and the correction is /// worth writing down.** io-harness 0.71.0 names its own defaults: /// `DEFAULT_MAX_STEPS` = 8, `DEFAULT_WORKSPACE_MAX_STEPS` = 12 and -/// `DEFAULT_MAX_RETRIES` = 2 (`io-harness-0.78.0/src/contract.rs:693,711,727`), +/// `DEFAULT_MAX_RETRIES` = 2 (`io-harness-0.79.0/src/contract.rs:704,722,738`), /// re-exported at the crate root. "There is nothing to read" was true when this /// was written and is not true now. What is still true is that none of it anchors /// *this* ladder: @@ -537,7 +807,7 @@ pub fn shape_of(key: &str, config: &Config) -> Option { /// The models `[prices.models]` names, across every scope, sorted and deduplicated. /// /// **Read from the dependency's own table since io-harness 0.71.0, not scraped -/// out of the files.** `PriceTable::models` (`io-harness-0.78.0/src/pricing.rs:268`) +/// out of the files.** `PriceTable::models` (`io-harness-0.79.0/src/pricing.rs:268`) /// lists every model the table can actually price, and [`Config::prices`] has /// always built that table out of the three scopes — so the merged question this /// used to hand-roll is precisely the one the accessor answers, and the gap filed @@ -563,7 +833,7 @@ pub fn shape_of(key: &str, config: &Config) -> Option { /// /// **This takes the `Config` the caller already holds, and must never re-discover /// one.** `Config::discover` resolves every `${env:}`, `${file:}` and `${cmd:}` as -/// it reads (`io-harness-0.78.0/src/config.rs:621`), so a second discovery re-runs +/// it reads (`io-harness-0.79.0/src/config.rs:627`), so a second discovery re-runs /// an operator's credential commands — which for a `${cmd:}` fetching a key out of /// a keychain means a Touch-ID prompt raised in order to draw a menu, every time /// the picker opens. Taking a `&Config` is not an optimisation; it is the @@ -641,7 +911,7 @@ pub fn destination(config: &Config, key: &str) -> (Scope, bool) { #[must_use] pub fn widens_workspace(key: &str, value: &str) -> bool { /// The clause io-harness's widening refusal always carries - /// (`io-harness-0.78.0/src/config.rs:2740`). Matched rather than the whole + /// (`io-harness-0.79.0/src/config.rs:2826`). Matched rather than the whole /// sentence, which interpolates the path, the key and the destination scope. const WIDENS: &str = "widens the boundary"; @@ -776,8 +1046,8 @@ fn is_credential(path: &str) -> bool { /// /// **There are three substitution forms and not two.** io-harness resolves /// `${env:...}`, `${file:...}` **and** `${cmd:...}` -/// (`substitute`, `io-harness-0.78.0/src/config.rs:2950`, the `cmd` arm at -/// `:3070`); this comment claimed two until +/// (`substitute`, `io-harness-0.79.0/src/config.rs:3036`, the `cmd` arm at +/// `:3156`); this comment claimed two until /// 0.21.0, and the sentence it claimed it in was the argument for which forms /// pass through here. The third is deliberately not one of them: a `${env:}` or /// `${file:}` reference is a *name*, and the name is the whole of what an diff --git a/src/context.rs b/src/context.rs index 33e91e1..5e6a390 100644 --- a/src/context.rs +++ b/src/context.rs @@ -128,7 +128,7 @@ pub const SKILLS_HEAD: &str = "Skills available to you"; /// **The directive is glued to the last catalogue line with no newline between /// them, so a line scan cannot see the boundary.** `compose` builds the prompt as /// `with_skill_catalog(..)` and then `out.push_str(&directive)` -/// (io-harness-0.78.0/src/run/prompts.rs:73-76); `Skills::catalog()` ends with its +/// (io-harness-0.79.0/src/run/prompts.rs:73-76); `Skills::catalog()` ends with its /// last `- name: description` line carrying **no trailing newline** /// (src/skills.rs:470-476); and `planning_directive` returns a string beginning /// with a space (src/run/gate.rs:282-286). So with a plan gate registered — which diff --git a/src/contract.rs b/src/contract.rs index 7461290..b047516 100644 --- a/src/contract.rs +++ b/src/contract.rs @@ -110,6 +110,82 @@ impl Capabilities { /// any of those, and it was standing in for all three. pub const MAX_STEPS: u32 = 1_000; +/// The cap a **gated headless** run takes instead, when nothing else bounds it. +/// +/// **A gate falsifies every one of the three fallbacks [`MAX_STEPS`] rests on, +/// and the 2026-09-05 field test is what proved it.** The paragraph above argues +/// that a thousand is safe because it is not the only bound in the system: the +/// harness stops an agent that stalls, `[run]`'s budgets stop one that spends, +/// and `Ctrl+C` stops one an operator has seen enough of. Configure a criterion +/// that cannot pass, run it under `io exec`, and all three are gone at once. The +/// harness does not stop a stalled agent — it evaluates the criterion after every +/// step and hands the failure back, which is precisely a reason to keep going. A +/// run with no `[run]` budget spends nothing it is measured against. And headless +/// has no operator to press anything. +/// +/// So the run walks to a thousand steps. The field test killed it at nine minutes +/// with no stdout at all, and exit `6` — which `docs/CONTRACT.md` publishes as +/// `UNVERIFIED` for a CI job to branch on — was never delivered. `exec::code` +/// computes it correctly; the run simply never ended. +/// +/// **Forty is a chosen number and it is named as one.** A gate exists to be +/// retried against, so the cap has to leave room for several write-then-check +/// cycles — io-harness's own twelve does not, which is the whole reason +/// [`MAX_STEPS`] exists — while still ending a hopeless run in the minutes an +/// unattended job can afford rather than the hours it had. It applies **only** +/// where nothing else bounds the run: any `[run] max_steps`, `max_duration_secs` +/// or `max_tokens` the operator wrote is a bound they chose, and it wins. +pub const GATED_MAX_STEPS: u32 = 40; + +/// The step cap for a headless run, given whether it is gated and what it budgets. +/// +/// Separate from [`configured`] because it is `io exec`'s alone: a session +/// applies the gate turn by turn, with `gates::may_retry` deciding whether to +/// send the agent back and an operator watching either way. Nothing headless has +/// either of those, which is why the bound has to be a property of the contract +/// there and need not be here. +/// +/// **The test for "the operator set no step cap" is that the floor survived.** +/// `configured` calls `with_max_steps(MAX_STEPS)` *before* `Config::apply_to`, so +/// a `[run] max_steps` in the file has already overwritten it by the time this +/// runs — the same reading `crate::status` takes of the same field, rather than a +/// second question put to the configuration. +/// +/// **It has one collision and it is stated rather than hidden: `[run] max_steps = +/// 1000` reads as "no cap set".** `Config::apply_to` calls `with_max_steps` +/// unconditionally when the key is present, so an explicit thousand is +/// indistinguishable from the floor, and a gated run configured that way takes +/// forty. io-harness's `run` section is private, so there is no way to ask +/// whether the key was written — only what it produced. The cost is bounded and +/// one-directional: an operator who meant a thousand gets forty and sees exit `3` +/// naming the ceiling, which is diagnosable and one edit from fixed; any other +/// number, including `1001`, behaves exactly as written. Asserted as a row in +/// `f3_a_gated_headless_run_with_no_budget_takes_the_gated_cap` so it is a chosen +/// cost rather than an unnoticed one. Found by the adversarial review. +#[must_use] +pub fn gated_bound(contract: TaskContract) -> TaskContract { + // **The contract's own criterion, never the configuration's.** The first + // draft asked `criterion_of(config)`, and the adversarial review found that + // it answers a different question than the one that matters: + // `configured` applies the criterion through `criterion_for`, which + // additionally returns `None` when there is no provider spec or when a + // review criterion's reviewer will not build. `criterion_for` is therefore a + // strict subset, and the gap is one-directional — a `rubric` whose reviewer + // fails leaves the run **ungated** (`exec` prints a notice saying exactly + // that) while a config-derived test would still have capped it at forty. An + // ungated run truncated to forty steps is a new failure mode invented for a + // misconfiguration that previously only cost the gate. + // + // Reading `contract.verify` cannot diverge: it is the gate that will + // actually run, put there by the same call that built everything else here. + let ungated = matches!(contract.verify, io_harness::Verification::None); + let budgeted = contract.max_duration.is_some() || contract.max_tokens.is_some(); + if ungated || budgeted || contract.max_steps != MAX_STEPS { + return contract; + } + contract.with_max_steps(GATED_MAX_STEPS) +} + /// What io-cli tells the agent about itself, appended to io-harness's own. /// /// **Every turn before 0.13.0 ran `SystemPrompt::Builtin`**, which the harness @@ -581,7 +657,7 @@ pub fn skills_dir(config: &Config, capabilities: &Capabilities, root: PathBuf) - /// **The existence test is not caution, it is the whole of what makes this /// default safe.** `Skills::discover` does not return early on a directory that /// is not there — it returns `Error::Config("skills directory … does not exist")` -/// (`io-harness-0.78.0/src/skills.rs`), and `TaskContract::discover_skills` +/// (`io-harness-0.79.0/src/skills.rs`), and `TaskContract::discover_skills` /// propagates it from `run.rs` at run start, before the first completion. A /// contract that named this directory unconditionally would therefore fail every /// turn of every operator who has never made one, which is almost all of them. @@ -606,7 +682,7 @@ fn default_skills() -> Option { /// /// **One expansion for two keys, applied after both have had their say.** /// io-harness substitutes `${env:…}`, `${file:…}` and `${cmd:…}` and nothing else -/// (`substitute`, `io-harness-0.78.0/src/config.rs:2950` — there is no tilde +/// (`substitute`, `io-harness-0.79.0/src/config.rs:3036` — there is no tilde /// branch anywhere in it, and 0.71.0 narrowed the forms rather than widening /// them: a plugin manifest now refuses all three), so a `~` an operator wrote in /// `[run] skills` or `[app.io-cli] skills` diff --git a/src/exec.rs b/src/exec.rs index 1afd327..272b5d3 100644 --- a/src/exec.rs +++ b/src/exec.rs @@ -691,6 +691,13 @@ pub fn contract( config, resolved.loaded(), ); + // **The gated bound, and it is here rather than in `configured` because it + // is this door's alone (0.38.1).** A session applies a gate turn by turn, + // with `gates::may_retry` deciding whether to send the agent back and an + // operator watching either way; headless has neither, so the bound has to + // ride the contract. See `contract::gated_bound` for why a gate takes away + // every fallback `contract::MAX_STEPS` is safe because of. + let contract = crate::contract::gated_bound(contract); // The flag last, so it beats the file, and applied with `with_exec_mode` // rather than by replacing the whole `SandboxConfig` — the limits the file // set are the operator's and are not this flag's to discard. @@ -793,6 +800,21 @@ pub async fn main( // a refused posture costs nothing and leaves no run behind. let posture = args.policy.map(posture_for).transpose()?; + // **Before the run, because the point is that the run does not happen + // (0.38.1).** A `[app.io-cli]` section that will not parse takes the gate + // down with it, and until this release `io exec` never called + // `settings::stored` at all — so a configured criterion was silently not + // applied and the run reported success without verifying anything. A session + // prints the notice to somebody who is looking at it; this door refuses. + if let Some(refusal) = settings::ungated_by_a_broken_section(&config) { + return Err(refusal); + } + // A section that broke without naming a gate is still worth a line here, for + // the same reason: the notice existed and headless never printed it. + if let (_, Some(notice)) = settings::stored(&config) { + eprintln!("io: {notice}"); + } + // And before those too: a template that does not resolve is a goal that does // not exist, and a run started on one would spend a provider call to say so. let (goal, templates_notice) = goal_for(&config, &args.goal)?; @@ -1364,6 +1386,19 @@ pub async fn resume_main( eprintln!("{notice}"); } + // **Both headless doors refuse an unreadable gate, not just the first.** A + // resume rebuilds the contract from the same configuration, so a broken + // `[app.io-cli]` naming a gate leaves the carried-on run with no criterion + // and `verified_code` with no standing to read — the same silent success + // `io exec` now refuses, reached by the command an operator runs after the + // first one stopped. + if let Some(refusal) = settings::ungated_by_a_broken_section(&config) { + return Err(refusal); + } + if let (_, Some(notice)) = settings::stored(&config) { + eprintln!("io: {notice}"); + } + let path = settings::store_path().ok_or("no place to keep the run store")?; let store = Store::open(&path).map_err(|error| error.to_string())?; @@ -1494,6 +1529,16 @@ impl WithProvider for Resuming { &self.config, resolved.loaded(), ); + // **The gated bound rides a resume too, and leaving it off was the whole + // shape of the 0.38.1 defect repeating itself (found by the adversarial + // review, before the merge).** `io exec` and `io resume` are the two + // headless doors, and a resume is what an operator reaches for *after* + // the first one ended badly — which for the 2026-09-05 field test means + // exactly this run: it was killed, a killed run is `Pending::Died`, and + // `Died` is carried on rather than refused. Bounding one door and not the + // other would have fixed the walk to a thousand steps only until somebody + // typed the obvious next command. + let contract = crate::contract::gated_bound(contract); // `None` on every arm: a containment is a fleet's shared budget, this // subcommand takes no flag that expresses one, and `crate::resume::recover` // refuses a contained run outright because io-harness publishes no @@ -1503,8 +1548,8 @@ impl WithProvider for Resuming { // other pause kind has both forms — `resume_tree_with_answer` beside // `resume_with_answer`, `resume_tree_with_plan_decision` beside its flat // one, `resume_tree_with_decision` beside `resume_with_decision` - // (`io-harness-0.78.0/src/run.rs:1806`, `:2140`, `:3184`). Recovery has - // `resume_with_recovery_observed` (`:2602`) and nothing tree-aware, so it + // (`io-harness-0.79.0/src/run.rs:1811`, `:2145`, `:3189`). Recovery has + // `resume_with_recovery_observed` (`:2607`) and nothing tree-aware, so it // is the one pause a contained run cannot be resumed from. Not an oversight // this crate can route around: a fleet's shared ceiling lives in the tree // entry points, and resuming through the flat one would drop it. diff --git a/src/home.rs b/src/home.rs index 7b5a831..62f938a 100644 --- a/src/home.rs +++ b/src/home.rs @@ -9,7 +9,7 @@ //! This module names one. It is deliberately **not** a second configuration //! system: io-harness resolves `$IO_CONFIG`, then `$IO_CONFIG_HOME`, then the //! platform's own place, reading the environment at call time -//! (`io-harness-0.78.0/src/config.rs:2241`, the Windows branch at `:2259`), and +//! (`io-harness-0.79.0/src/config.rs:2311`, the Windows branch at `:2329`), and //! there is no caller-supplied home //! anywhere in `Config`'s public surface — no `discover_in`, no builder. So the //! one lever is `$IO_CONFIG_HOME`, set once, before the first diff --git a/src/import.rs b/src/import.rs index 75544fe..a86c8c2 100644 --- a/src/import.rs +++ b/src/import.rs @@ -30,7 +30,7 @@ //! decision="allow")` and Claude's `permissions.ask` says `Bash(cargo yank *)` — //! both of which match a *command line*. io-harness's `Act::Exec` matches a //! **binary name and nothing else**; it has no argument matching at all -//! (`Act::Exec`, `io-harness-0.78.0/src/policy.rs:63`). So the nearest thing to a faithful +//! (`Act::Exec`, `io-harness-0.79.0/src/policy.rs:63`). So the nearest thing to a faithful //! import of `bun install` is a blanket allow on `bun`, which is a wider //! permission than the operator ever granted, written by a tool they were //! trusting to be careful. A boundary half-imported is worse than one left alone, diff --git a/src/main.rs b/src/main.rs index 4e7f565..e3bb9ca 100644 --- a/src/main.rs +++ b/src/main.rs @@ -5852,8 +5852,30 @@ async fn loop_over Result>( ), ); } + // **The value is spelled by the library, not passed through + // (0.38.1).** Until this release the operator's raw text went to + // `write_where` as TOML source, so `/config k 4` wrote the + // integer `4` while `io config set k 4` wrote the string `"4"` + // for the same key, and `/config policy.defaults.net allow` was + // refused because a bare `allow` is not a TOML value at all. + // `configure::source_for` is the one speller both doors call, and + // `composer_words` is the only thing that differs between them: + // a shell already split and unquoted the argv form, and nothing + // has done either to a composer line. Action::Config(Some((key, value))) => { - picker = Some(write_where(session.root(), key, value)); + match io_cli::configure::source_for( + &key, + &io_cli::configure::composer_words(&value), + ) { + Ok(source) => { + picker = Some(write_where(session.root(), key, source)); + } + // The refusal io-harness or the catalogue would have given + // at the far end, given here instead — before a picker + // asks which file to write a value into that will not be + // written. + Err(refusal) => app.record(Tone::Refused, refusal), + } } // A line typed with nothing after the word. Answered with what to // type rather than by opening a picker over three files it would diff --git a/src/manage.rs b/src/manage.rs index 9a9236b..0cf8cc6 100644 --- a/src/manage.rs +++ b/src/manage.rs @@ -79,7 +79,6 @@ use std::path::{Path, PathBuf}; use io_harness::config::Scope; use io_harness::{McpServer, McpTransport}; -use crate::configure::Kind; use crate::edit::Edit; /// One sentence an operator typed, on whichever surface they typed it. @@ -235,8 +234,9 @@ pub enum ConfigVerb { /// One key's value and what decided it. Get { key: String }, /// Write one key. `value` is TOML source, already checked against the key's - /// [`Kind`]: a choice arrives quoted, a number and a flag bare, so that what - /// is written is a value io-harness reads back as the one that was typed. + /// [`crate::configure::Kind`]: a choice arrives quoted, a number and a flag + /// bare, so that what is written is a value io-harness reads back as the one + /// that was typed. Set { key: String, value: String, @@ -1408,7 +1408,7 @@ fn mcp_add(args: &Args) -> Result { // Built by hand rather than through `McpServer::stdio(…).with_args(…)`, // because `env` and `headers` have no builder at all and `with_args` is a - // silent no-op on an HTTP server (`io-harness-0.78.0/src/mcp.rs:439-447`: the + // silent no-op on an HTTP server (`io-harness-0.79.0/src/mcp.rs:439-447`: the // body writes only into the `Stdio` arm) — a constructor chain here would drop // the arguments of half the servers it was handed and say nothing. // Asked of the harness rather than written as literals, the way `servers::add` @@ -1608,120 +1608,15 @@ fn config_set(args: &Args) -> Result { /// The TOML source for `words`, checked against what the key admits. /// -/// **The kind decides the shape and never the meaning.** io-harness owns what a -/// setting does; [`crate::configure::kind_of`] answers only how a value is -/// obtained, and this turns a typed word into the source that expresses it. A key -/// the catalogue does not name is written as a string and left to -/// `configure::write`'s round trip, which is io-harness reading its own file back -/// — inventing a kind for it here would be io-cli deciding a schema it does not -/// own. +/// **The body moved to [`crate::configure::source_for`] in 0.38.1, and the move +/// is the fix rather than a tidy-up.** It lived here while `io config set` was +/// the only door that spelled a value. The session's `/config` never reached it: +/// `Action::Config` carried the operator's raw text straight to `write_where` as +/// TOML source, so `4` landed as the integer `4` there and as the string `"4"` +/// here, and `allow` landed as a bare word TOML cannot parse at all. Two doors, +/// two spellings, neither of them right — and the module that owns the catalogue +/// is the one that should own the spelling. This name is kept because it is what +/// this module's own parse reads well as. fn config_value(key: &str, words: &[String]) -> Result { - let kind = crate::configure::kind_of(key); - if matches!(kind, Some(Kind::Machine)) { - return Err(format!( - "`{key}` is written by io itself rather than typed — it dates the price table, and a \ - date set by hand is a claim about a fetch that never happened; `/config` offers the \ - refresh that writes it" - )); - } - // The one key whose value is a list (`app.io-cli.gates.command`), so the - // remaining words are the value rather than a mistake. A scalar written there - // is a value io-harness cannot read back. - if matches!(kind, Some(Kind::List)) { - if words.is_empty() { - return Err(format!( - "`{key}` is a command and its arguments, so it needs at least one word, as in \ - `config set {key} cargo test`" - )); - } - let items: Vec<&str> = words.iter().map(String::as_str).collect(); - return Ok(crate::edit::array(&items)); - } - - let word = match words { - [word] => word.as_str(), - [] => { - return Err(format!( - "`config set {key}` has no value after it; {}", - offered(key, kind.as_ref()) - )) - } - _ => { - return Err(format!( - "`{key}` takes one value and {} words were given ({}); quote a value that \ - contains a space", - words.len(), - words.join(" ") - )) - } - }; - - match kind { - Some(Kind::Flag) => match word { - "true" | "false" => Ok(word.to_string()), - other => Err(format!( - "`{other}` is not a value `{key}` takes; it is on or off, written `true` or \ - `false`" - )), - }, - Some(Kind::Choice(options)) => { - if options.iter().any(|option| option == word) { - Ok(crate::servers::quoted(word)) - } else { - Err(format!( - "`{word}` is not a value `{key}` takes; the options are {}", - options - .iter() - .map(|option| format!("`{option}`")) - .collect::>() - .join(", ") - )) - } - } - Some(Kind::Number { signed }) => { - let number: i64 = word.parse().map_err(|_| { - format!("`{word}` is not a whole number, and `{key}` counts in whole numbers") - })?; - if !signed && number < 0 { - return Err(format!( - "`{key}` counts something and cannot be negative; `{word}` was not written" - )); - } - // Re-rendered from the parsed number rather than passed through, so - // that a form TOML would reject (`+5`, a leading zero run) becomes - // the value it meant instead of a refused write. - Ok(number.to_string()) - } - // A model name, a path, free text, and a key no catalogue entry names. - // All four are strings in the file, and all four are escaped rather than - // wrapped in quotes by hand. - Some(Kind::Model | Kind::File | Kind::Text) | None => Ok(crate::servers::quoted(word)), - // Both are answered above, before a single word was demanded, so this arm - // is unreachable — and it is written as a refusal rather than left to a - // wildcard or a `panic!`. A wildcard would swallow a `Kind` variant a - // later release adds and write it as a string; a panic would end the - // process on something an operator typed. `Kind` is io-cli's own enum, so - // the exhaustive match is a compile-time gate this crate can keep. - Some(Kind::List | Kind::Machine) => Err(format!( - "`{key}` was read as two different kinds in one parse, which is a defect in io \ - rather than in what you typed; nothing was written" - )), - } -} - -/// What to say a key admits, for the refusal that has no value to name. -fn offered(key: &str, kind: Option<&Kind>) -> String { - match kind { - Some(Kind::Flag) => "it is on or off, written `true` or `false`".to_string(), - Some(Kind::Choice(options)) => format!( - "the options are {}", - options - .iter() - .map(|option| format!("`{option}`")) - .collect::>() - .join(", ") - ), - Some(Kind::Number { .. }) => "it takes a whole number".to_string(), - _ => format!("`config get {key}` shows what it is set to now"), - } + crate::configure::source_for(key, words) } diff --git a/src/memory.rs b/src/memory.rs index 9a1ac3f..7997e35 100644 --- a/src/memory.rs +++ b/src/memory.rs @@ -27,7 +27,7 @@ //! //! Worth knowing before a surface promises otherwise. `read_instructions` joins //! each name in `[instructions] files` to the **discovery root** -//! (`io-harness-0.78.0/src/config.rs:2911`), and the default list is exactly +//! (`io-harness-0.79.0/src/config.rs:2997`), and the default list is exactly //! `["AGENTS.md"]` (`DEFAULT_INSTRUCTIONS`, `config.rs:190`). So `AGENTS.md` is read //! by every project with no configuration at all; `AGENTS.local.md` is read only //! where a file names it; and `IO.md` is not reachable by a bare name at all, @@ -464,7 +464,7 @@ fn bullet(line: &str) -> Option { /// The order the three files are named in, and it is the order they reach the /// model in: `read_instructions` pushes one constraint per name in list order -/// (`read_instructions`, `io-harness-0.78.0/src/config.rs:2911`). +/// (`read_instructions`, `io-harness-0.79.0/src/config.rs:2997`). /// /// Project, then local, then user — widest audience first, so the operator's own /// standing note is the last thing said. It also matches the order the three diff --git a/src/reload.rs b/src/reload.rs index 3e8fc29..70695ad 100644 --- a/src/reload.rs +++ b/src/reload.rs @@ -2,7 +2,7 @@ //! //! **io-harness takes the instruction files as a snapshot and never takes //! another.** `read_instructions` runs inside `Config::discover` -//! (`io-harness-0.78.0/src/config.rs:2911`), composes each named file into a +//! (`io-harness-0.79.0/src/config.rs:2997`), composes each named file into a //! constraint, and stores the result in a private `Config.instructions` field. //! There is no `Config::reload`, and `Config::with_profile` clones the field //! rather than going back to disk — so a `Config` is exactly as old as the call diff --git a/src/settings.rs b/src/settings.rs index 3e11e7d..b19c801 100644 --- a/src/settings.rs +++ b/src/settings.rs @@ -337,6 +337,58 @@ pub fn deprecated_max_steps(config: &io_harness::Config) -> Option { )) } +/// The refusal a headless run earns when `[app.io-cli]` will not parse **and it +/// names a gate**. +/// +/// **A gate that cannot be read is a gate that is not in force, and headless is +/// the one surface with nobody to notice (0.38.1).** `[app.io-cli]` has no +/// `deny_unknown_fields`, but it is still one section: a single mistyped value +/// anywhere in it — `max_total_agents = "4"`, which the 2026-09-05 field test +/// produced with `io config set` — fails the whole `Config::app` call, and +/// [`stored`] then answers `None` for every key including the gate. A session +/// prints the notice to a person who is looking at it. `io exec` did not call +/// [`stored`] at all, so the run proceeded with no criterion, no gate row and +/// exit `0`: a verification the operator configured, silently not applied, on the +/// surface whose whole purpose is to be trusted unattended. +/// +/// So the raw section is read for a gate key, by exactly the shape +/// [`deprecated_max_steps`] uses and for the same reason — the typed struct is +/// unavailable precisely when this question needs answering. +/// +/// **It refuses rather than warns**, and only here. A warning is the behaviour +/// that produced the finding; and the same broken section in a session is a +/// person reading a line, which is why this is `io exec`'s alone. +pub fn ungated_by_a_broken_section(config: &io_harness::Config) -> Option { + /// The gate table alone, and deliberately never constructed. + /// + /// `IgnoredAny` accepts any shape and keeps none of it, so this parse + /// succeeds on exactly the values that fail `CliSettings` — which is the + /// whole point: it answers "was a gate asked for", never "is the gate + /// valid". It is also the reason this file still names no configuration + /// type and parses no TOML, which `tests/dependencies.rs` holds it to; + /// `src/import.rs` reads a credential-shaped field the same way and for the + /// same reason. + #[derive(Deserialize)] + struct Gated { + #[serde(default)] + gates: Option, + } + + // Nothing to say when the section parses: `stored` returns the real settings + // and the gate is in force or genuinely absent. + if config.app::(APP_KEY).is_ok() { + return None; + } + config.app::(APP_KEY).ok()??.gates?; + Some( + "`[app.io-cli]` names a gate but the section could not be read, so no criterion is in \ + force and this run would report success without verifying anything. `io config get \ + app.io-cli` shows the section; a value written as a string where a number is expected \ + is the usual cause. Nothing was run." + .to_string(), + ) +} + /// Whether this session runs in plain mode: `--plain`, or `[app.io-cli] plain`. /// /// **A pure function, and it lives here rather than in `src/main.rs` on purpose.** diff --git a/src/triage.rs b/src/triage.rs index ab03b53..4cb158d 100644 --- a/src/triage.rs +++ b/src/triage.rs @@ -41,7 +41,7 @@ pub enum Disposition { Silent, } -/// Every kind io-harness 0.76 declares, in its own declaration order. +/// Every kind io-harness 0.79 declares, in its own declaration order. /// /// The order is the enum's rather than alphabetical so that this table can be /// read down the side of `observe.rs` when the pin moves. @@ -291,6 +291,36 @@ pub const TRIAGE: &[(&str, Disposition, &str)] = &[ the first step", ), ("contained", Disposition::Status, "the containment field"), + // **0.79.0 — a kind this crate cannot receive, dispositioned anyway.** + // Placed between `contained` and `dialed` because that is where + // `EventKind` declares it, and this table's whole ordering rule is that it + // can be read down the side of `observe.rs` when the pin moves. + // + // `EventKind::Program` is declared unconditionally in `observe.rs`, but + // every site that emits it is behind io-harness's `codeact` feature, which + // this crate does not enable. So the row exists because the table is total + // over what the harness *declares*, and the disposition is the one a run + // would get if the feature were ever turned on. + // + // Silent rather than a line, and the discovery half is why: the event is + // emitted once before the first step saying `available` or `withheld`, and + // once per program afterwards. A capability's availability is a fact about + // the run's configuration rather than about the conversation, and the acts a + // program takes are not on this event at all — each one re-enters dispatch + // and arrives as its own `tool_call`, which the transcript already draws. A + // line here would announce that a program ran and then say nothing about + // what it did. + // + // **Revisit when this crate enables `codeact`.** A program a turn wrote is + // the largest thing a model can do in one step, and the roadmap says it + // belongs in the transcript as source. That is a release, not a row. + ( + "program", + Disposition::Silent, + "`io exec --json`, which forwards it verbatim, and the durable trace; no run this crate \ + drives emits it, because every emitting site is behind io-harness's `codeact` feature and \ + this crate does not enable it", + ), ( "dialed", Disposition::Line, diff --git a/tests/adapt.rs b/tests/adapt.rs index fcc589f..9f6dd9a 100644 --- a/tests/adapt.rs +++ b/tests/adapt.rs @@ -510,7 +510,7 @@ fn loaded(into: &Path) -> Plugins { /// that io wrote until 0.35.0. The generator still runs and the fixture still /// builds — and `dropped()` carries the whole bundle, because io-harness 0.74.0 /// refuses an absolute `skills` or `templates` in every scope, `Scope::User` -/// included (`io-harness-0.78.0/src/plugin.rs:1097`): a manifest contributes a +/// included (`io-harness-0.79.0/src/plugin.rs:1097`): a manifest contributes a /// directory it ships rather than one it points at somewhere else on the machine, /// since every `*.md` under it is read into the model's system prompt. #[test] diff --git a/tests/configure.rs b/tests/configure.rs index 0bb2a92..a71926c 100644 --- a/tests/configure.rs +++ b/tests/configure.rs @@ -1816,3 +1816,139 @@ fn f9_the_profile_list_sees_every_scope_and_not_only_the_last() { rather than merely narrow", ); } + +/// One word, as it would be typed. +fn words(value: &str) -> Vec { + vec![value.to_string()] +} + +/// **F1 — a key the catalogue does not name is typed from the word, not quoted.** +/// +/// The 2026-09-05 field test's finding 3 and finding 28, at the one line that +/// caused both: `kind_of` answers `None` for every key under +/// `[app.io-cli.containment]` and `[run.context]`, and the `None` arm quoted its +/// word unconditionally. So `max_total_agents 4` reached the file as `"4"`, +/// io-harness refused to deserialize the section, and the session ran on defaults +/// behind a single warning line — a feature switched off in silence, which is the +/// worst shape a defect can have in a product whose claim is honesty. +/// +/// Sabotage: restore `None => Ok(crate::servers::quoted(word))` in `source_for` +/// and the first two rows fail. +#[test] +fn f1_an_unknown_key_takes_the_shape_of_the_word_that_was_typed() { + for (key, typed, written) in [ + // The two the field test found, by name. The first is now a `CATALOGUE` + // key — `[app.io-cli.containment]` is this crate's own schema and belongs + // there — and the second is io-harness's, so it reaches the inference. + ("app.io-cli.containment.max_total_agents", "4", "4"), + ("app.io-cli.containment.max_depth", "3", "3"), + ("run.context.max_tokens", "120000", "120000"), + // The other shapes, so the inference is asserted as a whole rather than + // at the one number that prompted it. + ("some.unknown.flag", "true", "true"), + ("some.unknown.flag", "false", "false"), + ("some.unknown.ratio", "1.5", "1.5"), + // Not a number, not a boolean: still a string, exactly as before. + ("some.unknown.label", "steady", "\"steady\""), + // A negative is a number too. Only `Kind::Number { signed: false }` + // refuses one, and that is a rule about a key the catalogue *does* name. + ("some.unknown.offset", "-3", "-3"), + ] { + assert_eq!( + configure::source_for(key, &words(typed)).as_deref(), + Ok(written), + "`{key} {typed}` should be written as `{written}`", + ); + } + + // **A form TOML rejects becomes the value it meant**, the same rule the + // catalogue's own number kind already keeps, and for the same reason: writing + // `+5` produces a line io-harness cannot read back. + assert_eq!( + configure::source_for("some.unknown.count", &words("+5")).as_deref(), + Ok("5"), + ); + + // **`inf` and `nan` are strings, deliberately.** Both are valid TOML floats + // and `f64::from_str` takes both, so admitting them would turn a key whose + // value is the word `nan` into a float nobody asked for. + for word in ["inf", "nan", "-inf", "NaN"] { + assert_eq!( + configure::source_for("some.unknown.label", &words(word)).as_deref(), + Ok(format!("\"{word}\"").as_str()), + "`{word}` is a word an operator typed, not a non-finite float", + ); + } + + // **The boundary the adversarial review put here: inference stops at + // `[app.io-cli]`.** Everywhere else, a wrong guess is refused and rolled back + // by `configure::write`'s round trip through io-harness. io-harness does not + // type io-cli's own section at all — which is how `max_total_agents = "4"` + // shipped — so a wrong guess there fails at the next session start behind one + // warning line instead. `app.io-cli.keys.accept` is the concrete case: a + // one-character binding is legitimate, the map is `BTreeMap`, + // and a bare `1` would take the whole section down. + // + // Sabotage: drop the `key.starts_with(APP_PREFIX)` arm from `source_for` and + // both rows below fail. + for typed in ["1", "true"] { + assert_eq!( + configure::source_for("app.io-cli.keys.accept", &words(typed)).as_deref(), + Ok(format!("\"{typed}\"").as_str()), + "an `app.io-cli.*` key the catalogue does not name is written as a \ + string, because nothing validates that section on the way back", + ); + } + + // And a key the catalogue DOES name is untouched by any of it: the shape + // still comes from the kind, which is what stops this inference from becoming + // a second opinion about a schema io-harness owns. + assert_eq!( + configure::source_for("sandbox.mode", &words("read-only")).as_deref(), + Ok("\"read-only\""), + ); + assert!( + configure::source_for("run.max_steps", &words("lots")).is_err(), + "a catalogue key still refuses a value its kind does not admit", + ); +} + +/// **F1 — the composer is not a shell, and that is the only difference between +/// the two doors.** +/// +/// `io config set k a b` arrives as three argv words the shell has already split +/// and unquoted. `/config k a b` arrives as one string, because nothing has split +/// or unquoted anything. So the composer's half needs the splitting a shell would +/// have done — and needs to honour a quoted value, since the operator typed those +/// quotes precisely to say "this is one value" and there was no shell to act on +/// it. +/// +/// Sabotage: return `vec![raw.to_string()]` and the multi-word row fails; drop +/// the quote-stripping branch and the quoted row fails. +#[test] +fn f1_the_composer_splits_a_value_the_way_a_shell_would_have() { + assert_eq!(configure::composer_words("4"), vec!["4".to_string()]); + assert_eq!( + configure::composer_words(" cargo test "), + vec!["cargo".to_string(), "test".to_string()], + ); + assert_eq!( + configure::composer_words("\"all green\""), + vec!["all green".to_string()], + "the quotes are the operator saying this is one value, and no shell ate them", + ); + assert!(configure::composer_words(" ").is_empty()); + + // The two doors agreeing is the whole criterion, so it is asserted directly + // rather than left to follow from the two halves above. + for typed in ["4", "true", "steady"] { + assert_eq!( + configure::source_for("app.io-cli.containment.max_total_agents", &words(typed)), + configure::source_for( + "app.io-cli.containment.max_total_agents", + &configure::composer_words(typed), + ), + "the shell door and the session door must spell `{typed}` the same way", + ); + } +} diff --git a/tests/context.rs b/tests/context.rs index 6f2bda9..d1ec2af 100644 --- a/tests/context.rs +++ b/tests/context.rs @@ -890,7 +890,7 @@ fn f9_allow_reports_a_no_op_and_a_clear_names_what_returned() { /// **The defect this replaces was the safety lever being silently inert.** /// io-harness keeps an unknown mask name rather than rejecting it, deliberately, /// so a mask stays portable across builds with different cargo features -/// (`io-harness-0.78.0/src/tools/mod.rs:55-58`). That means `mask_gate` matches on +/// (`io-harness-0.79.0/src/tools/mod.rs:55-58`). That means `mask_gate` matches on /// the exact string and a misspelling withholds nothing — while the operator was /// told "calling it will be refused before anything starts" and `/context` drew it /// on the withheld row. `/context withhold Docx_Write` and the file gets written. diff --git a/tests/contract.rs b/tests/contract.rs index fb3dd23..d198565 100644 --- a/tests/contract.rs +++ b/tests/contract.rs @@ -1829,7 +1829,7 @@ fn f5_servers_in_both_scopes_are_merged_and_a_collision_is_named() { /// **F7 — a tilde is a home directory, never a directory named `~`.** /// /// io-harness substitutes `${env:…}` and `${file:…}` and nothing else — there is -/// no tilde branch anywhere in `io-harness-0.78.0/src/config.rs` — so a `~` an +/// no tilde branch anywhere in `io-harness-0.79.0/src/config.rs` — so a `~` an /// operator writes in `[run] skills` reaches `Skills::discover` verbatim and the /// harness looks inside a directory whose name is one character long. The /// operator's skills sit exactly where they said they would, and the session diff --git a/tests/dependencies.rs b/tests/dependencies.rs index 1c6877e..e0a9058 100644 --- a/tests/dependencies.rs +++ b/tests/dependencies.rs @@ -2288,3 +2288,45 @@ fn f3_the_cost_columns_are_filled_by_the_driver_and_not_only_by_tests() { ); } } + +/// **F1 — the session door spells a configuration value through the library.** +/// +/// The criterion's production site is `src/main.rs`'s `Action::Config` arm, and +/// nothing under `tests/` links the driver, so without this gate the criterion is +/// unfalsifiable by construction — the exact shape 0.33.0 shipped three of before +/// a sabotage pass found them. +/// +/// What it holds is the half that cannot be asserted from a unit test: +/// `configure::source_for` is *reached* from the driver, and the driver builds no +/// TOML value of its own on the way. Before 0.38.1 the arm read +/// `write_where(session.root(), key, value)` with the operator's raw text as TOML +/// source — which is why `/config k 4` wrote an integer and `io config set k 4` +/// wrote a string for the same key. `source_for`'s own behaviour is asserted in +/// `tests/configure.rs`; that it is what the session uses is asserted here. +/// +/// Sabotage: put the raw `value` back into that `write_where` call and this fails +/// on the first needle. +#[test] +fn f1_the_session_door_spells_a_config_value_through_the_one_speller() { + let driver = std::fs::read_to_string("src/main.rs").expect("this crate's driver is readable"); + let code = code_of(&driver.replace("\r\n", "\n")); + + for needle in ["configure::source_for(", "configure::composer_words("] { + assert!( + code.contains(needle), + "`{needle}` has no call site in src/main.rs, so the session writes a \ + configuration value the shell door would have spelled differently — \ + two doors and two spellings, which is the defect 0.38.1 exists to \ + remove.", + ); + } + + // And the raw pass-through is gone. `write_where` takes TOML source; handing + // it the operator's own text is the pre-0.38.1 shape, and it is spelled here + // exactly as it was so the sabotage arm has something to restore. + assert!( + !code.contains("write_where(session.root(), key, value)"), + "the session still hands `write_where` the operator's raw text as TOML \ + source; that is what wrote `allow` as a bare word TOML cannot parse", + ); +} diff --git a/tests/docs.rs b/tests/docs.rs index ddb99f5..d67caae 100644 --- a/tests/docs.rs +++ b/tests/docs.rs @@ -2517,7 +2517,7 @@ fn f6_the_headless_guide_carves_out_the_provider_endpoint() { /// the risk is a sentence rather than a branch.** io-harness offers a masked turn a /// byte-identical catalogue on purpose: the tool array sits ahead of the provider's /// cache breakpoint, so dropping a definition would save its tokens once and pay a -/// cache *write* on every later turn (`io-harness-0.78.0/src/tools/mod.rs:33-42`). +/// cache *write* on every later turn (`io-harness-0.79.0/src/tools/mod.rs:33-42`). /// Withholding in fact makes the request marginally **larger**, by one sentence /// naming what is withheld (`src/run/prompts.rs:1133`). /// diff --git a/tests/events.rs b/tests/events.rs index 5da2a89..012c7c8 100644 --- a/tests/events.rs +++ b/tests/events.rs @@ -608,21 +608,25 @@ fn the_kind_name_is_the_serde_tag() { ); } -#[test] -fn f8_this_release_has_seen_every_kind_io_harness_emits() { - // The drift check itself moved to `tests/triage.rs`, which compares the - // locked harness's declared kinds against the disposition table rather than - // against a pair of lists nothing behind them had to agree with. What is - // left here is the count, because this file's own fixtures are written - // against it. - let declared = support::harness_event_kinds(); - assert_eq!( - declared.len(), - 53, - "the locked io-harness declares fifty-three event kinds; found {}", - declared.len(), - ); -} +// **`f8_this_release_has_seen_every_kind_io_harness_emits` was deleted in +// 0.38.1, and deleting it is the fix rather than bumping it.** +// +// Its own comment already said the drift check had moved to `tests/triage.rs`, +// and what it kept was a bare `53`. That number asserted nothing this suite does +// not assert better one file over: `the_table_names_every_kind_the_locked_harness_declares` +// compares the locked harness's declared kinds against the disposition table by +// NAME, so a kind io-harness adds fails there saying *which* kind is new. This +// one failed saying a number had moved — the least useful half of the same fact, +// and the half a reader repairs by editing the number. +// +// It is the fourth such literal this release found. Three were named by the +// dependency audit before the pin moved; this one and one in `tests/triage.rs` +// were found only by running the suite, which is the argument for running it +// rather than trusting an audit of what a pin *should* touch. +// +// The stated reason for keeping it — "this file's own fixtures are written +// against it" — is what a count cannot check. A fixture that stopped matching a +// kind fails on the fixture, not on a total. /// **F8.** The two facts no other core records, in the order a reader needs them, /// asserted by position. A `contains` assertion is green when the sentence is diff --git a/tests/gates.rs b/tests/gates.rs index 8c5e3ba..f15754e 100644 --- a/tests/gates.rs +++ b/tests/gates.rs @@ -846,3 +846,147 @@ fn the_gate_retry_admits_a_run_that_failed_its_verification() { firing for exactly the run it exists to retry", ); } + +// --------------------------------------------------------------------------- +// F3 — a gated headless run is bounded +// --------------------------------------------------------------------------- + +/// A configuration naming a command criterion, plus whatever else is given. +fn gated(extra: &str) -> Config { + let text = format!("[app.io-cli.gates]\ncommand = [\"false\"]\nexpect_exit = 0\n{extra}"); + Config::from_toml(&text).expect("the fixture parses") +} + +/// **F3 — a gate takes away every bound `MAX_STEPS` is safe because of.** +/// +/// `contract::MAX_STEPS` is a thousand, and its own documentation argues that is +/// safe because three other bounds exist: the harness stops an agent that stalls, +/// `[run]`'s budgets stop one that spends, and `Ctrl+C` stops one an operator has +/// seen enough of. Configure a criterion that cannot pass and run it headless and +/// all three are gone at once — io-harness evaluates the criterion after every +/// step and hands the failure back, which is a reason to keep going; a run with no +/// budget is measured against nothing; and there is no operator. The 2026-09-05 +/// field test killed such a run at nine minutes with no exit code. +/// +/// The four rows are the whole decision: bound it when nothing else does, and +/// leave it alone in every case where something already does. +/// +/// Sabotage: return `contract` unconditionally from `contract::gated_bound` and +/// the first row fails; drop either budget from the `budgeted` test and the row +/// naming it fails. +#[test] +fn f3_a_gated_headless_run_with_no_budget_takes_the_gated_cap() { + let workspace = PathBuf::from("."); + // **Through `contract::configured`, not a bare `TaskContract`.** `gated_bound` + // reads `contract.verify` — the criterion that will actually run — rather + // than re-deriving one from the configuration, so a fixture that skipped the + // builder would arrive with `Verification::None` and the cap would never + // apply. The first draft of this test did exactly that and would have passed + // over the feature being absent. + let build = |config: &Config| { + let contract = + io_cli::contract::configured("goal", workspace.clone(), config, &config.plugins()); + io_cli::contract::gated_bound(contract).max_steps + }; + + assert_eq!( + build(&gated("")), + io_cli::contract::GATED_MAX_STEPS, + "a gated run the operator gave no budget is what walks to a thousand steps", + ); + + // Ungated: untouched, because the three fallbacks the constant rests on are + // all still there. This is the row that stops the bound leaking onto every + // headless run. + assert_eq!( + build(&plain()), + io_cli::contract::MAX_STEPS, + "a run with no criterion is not gated and keeps the floor", + ); + + // Three budgets, one row each. Any of them is the operator saying how long + // this run may take, and none of them is this crate's to overrule. + for (extra, why) in [ + ("[run]\nmax_steps = 500\n", "a step cap they wrote"), + ("[run]\nmax_duration_secs = 60\n", "a wall clock they set"), + ("[run]\nmax_tokens = 50000\n", "a spend they set"), + ] { + let steps = build(&gated(extra)); + assert_ne!( + steps, + io_cli::contract::GATED_MAX_STEPS, + "{why} bounds the run already, so the gated cap must not replace it", + ); + } + + // **The one colliding value, asserted so it is a chosen cost.** `[run] + // max_steps = 1000` is the documented default written down, and + // `Config::apply_to` sets the field unconditionally — so it is + // indistinguishable from the floor `configured` applied, and the gated run + // takes forty. io-harness's `run` section is private, so nothing here can ask + // whether the key was present; this row exists so that the behaviour is + // recorded rather than discovered. Any other number, `1001` included, is + // honoured as written — which the row below proves, so this is a collision at + // exactly one value and not a rule. + assert_eq!( + build(&gated(&format!( + "[run]\nmax_steps = {}\n", + io_cli::contract::MAX_STEPS + ))), + io_cli::contract::GATED_MAX_STEPS, + "writing the default explicitly is indistinguishable from not writing it", + ); + assert_eq!( + build(&gated("[run]\nmax_steps = 1001\n")), + 1001, + "and one step either side of the collision is honoured as written", + ); +} + +/// **F3 — a gate that cannot be read refuses the headless run.** +/// +/// `[app.io-cli]` has no `deny_unknown_fields`, but it is still one section: a +/// single mistyped value anywhere in it fails the whole `Config::app` call, and +/// `settings::stored` then answers `None` for every key including the gate. The +/// field test produced exactly that with `io config set` writing +/// `max_total_agents = "4"`. A session prints the notice to somebody looking at +/// it; `io exec` never called `stored` at all, so the run proceeded with no +/// criterion and exit `0` — a verification the operator configured, silently not +/// applied, on the surface whose whole purpose is to be trusted unattended. +/// +/// The three rows are the three answers, and the middle one is the one a naive +/// implementation gets wrong: a broken section that names no gate is not a +/// refusal, because nothing about verification was asked for. +/// +/// Sabotage: return `None` unconditionally and the first row fails; drop the +/// `is_ok` early return and the third row fails. +#[test] +fn f3_a_gate_inside_a_section_that_will_not_parse_refuses_the_run() { + let broken_and_gated = Config::from_toml( + "[app.io-cli]\nplain = \"yes\"\n\n[app.io-cli.gates]\ncommand = [\"false\"]\n", + ) + .expect("the file itself is valid TOML; it is the section that will not type"); + let refusal = io_cli::settings::ungated_by_a_broken_section(&broken_and_gated) + .expect("a gate inside a section that will not parse is refused"); + assert!( + refusal.contains("no criterion is in force"), + "the refusal must say what is not happening rather than only that something \ + failed to parse: {refusal}", + ); + + // Broken, but nothing about a gate was asked for. Refusing here would stop + // runs over a key that has nothing to do with verification. + assert!( + io_cli::settings::ungated_by_a_broken_section( + &Config::from_toml("[app.io-cli]\nplain = \"yes\"\n").expect("valid TOML"), + ) + .is_none(), + "a section that will not parse and names no gate is not a gate defect", + ); + + // And a section that parses is never refused, however the gate is set. + assert!( + io_cli::settings::ungated_by_a_broken_section(&gated("")).is_none(), + "a readable gate is in force and must not be refused", + ); +} diff --git a/tests/hooks.rs b/tests/hooks.rs index 9ec8f3a..7d88157 100644 --- a/tests/hooks.rs +++ b/tests/hooks.rs @@ -410,7 +410,7 @@ fn a_hook_naming_an_event_that_does_not_exist_is_refused_rather_than_installed() // --------------------------------------------------------------------------- // // `io_harness::Hooks::new` creates every `append` path it is given, empty, as the -// value is constructed (`io-harness-0.78.0/src/hooks.rs`) — deliberately, so that +// value is constructed (`io-harness-0.79.0/src/hooks.rs`) — deliberately, so that // "the filter matched nothing" stays distinguishable from "the hook was never // installed". That makes *constructing* a `Hooks` a write to the filesystem, and // it resolves a relative `append` against the directory it was handed. Handed diff --git a/tests/instructions.rs b/tests/instructions.rs index ed77376..363c3ab 100644 --- a/tests/instructions.rs +++ b/tests/instructions.rs @@ -5,7 +5,7 @@ //! other half and `tests/memory.rs` covers it. This file covers the half that //! decides whether any of it reaches a model: `[instructions] files`, which is //! the only mechanism io-harness has for finding two of the three -//! (`io-harness-0.78.0/src/config.rs:190` — the automatic set is exactly +//! (`io-harness-0.79.0/src/config.rs:190` — the automatic set is exactly //! `["AGENTS.md"]`). //! //! Every test sets `IO_CONFIG_HOME`, because that is the only way to move the diff --git a/tests/live.rs b/tests/live.rs index 4569438..143489f 100644 --- a/tests/live.rs +++ b/tests/live.rs @@ -4276,3 +4276,106 @@ fn live_an_acp_client_that_denies_a_write_stops_it() { not reach the run", ); } + +/// **0.38.1 O3 — a gated run that cannot pass ends, and says `6`.** +/// +/// The defect this proves closed was found by running the released binary, not by +/// the suite, and it could only be found that way: every offline assertion about +/// the gate passed while a real `io exec` ran for nine minutes and was killed +/// without ever printing an exit code. So the evidence has to be a real turn. +/// +/// **The gate is a command that cannot succeed**, so the run's criterion can +/// never be met however well the model does the work. What is being measured is +/// therefore the *bound* rather than the model: before this release io-harness +/// kept handing the failure back and the agent kept stepping toward +/// `contract::MAX_STEPS`, a thousand. +/// +/// Three assertions, and the wall clock is the one that matters. An exit code +/// alone would have been produced eventually by the old build too — at a +/// thousand steps, which is what "never ends" means in practice. Bounding the +/// elapsed time is what distinguishes a run that ended because the gate bound it +/// from one that ended because the model gave up. +/// +/// `[run]` carries no budget on purpose: any of `max_steps`, `max_duration_secs` +/// or `max_tokens` would bound the run by itself and the arm would pass without +/// `GATED_MAX_STEPS` existing. +#[test] +#[ignore = "live: needs OPENROUTER_API_KEY"] +fn live_o3_a_gated_run_that_cannot_pass_exits_six() { + use std::process::Command; + + let config = tempfile::tempdir().expect("a config directory"); + let workspace = tempfile::tempdir().expect("a workspace"); + + // `false` is on every unix host and exits 1 whatever happens. No `[run]` + // section at all — see the doc comment. + std::fs::write( + config.path().join("io.toml"), + format!( + "[[provider]]\nkind = \"openrouter\"\nmodel = {:?}\n\n\ + [policy.defaults]\nread = \"allow\"\nwrite = \"allow\"\n\ + exec = \"allow\"\nnet = \"deny\"\n\n\ + [app.io-cli.gates]\ncommand = [\"false\"]\nexpect_exit = 0\n", + model() + ), + ) + .expect("the configuration is written"); + + let run = Command::new(env!("CARGO_BIN_EXE_io")) + .arg("-C") + .arg(workspace.path()) + .arg("exec") + .arg("--json") + .arg("Create gate.txt containing the word ok.") + .env("IO_CONFIG", config.path().join("io.toml")) + .env("OPENROUTER_API_KEY", key()) + .output() + .expect("the built binary runs"); + + let code = run.status.code(); + let stdout = String::from_utf8_lossy(&run.stdout); + let stderr = String::from_utf8_lossy(&run.stderr); + + // **The step count is the instrument, and a wall clock is not.** The first + // draft of this arm timed the run and asserted it finished inside ten + // minutes. `tests/timing.rs`'s N1 refused it — no test in this repository + // measures elapsed time — and N1 was right for a better reason than the one + // it states: elapsed time is a proxy for the bound, and a slow host or a + // chatty model moves it. The step count IS the bound. If `GATED_MAX_STEPS` + // fired, the run stopped at or under forty steps; before this release the + // same run walked toward `MAX_STEPS`, a thousand. + let steps = stdout + .lines() + .filter_map(|line| serde_json::from_str::(line).ok()) + .filter_map(|frame| frame.get("step").and_then(serde_json::Value::as_u64)) + .max() + .unwrap_or(0); + println!( + "exit {code:?} after {steps} steps\n--- stderr ---\n{}", + stderr.trim() + ); + + assert_eq!( + code, + Some(6), + "a run whose criterion cannot pass must report UNVERIFIED; stderr was:\n{stderr}", + ); + assert!( + steps > 0, + "no step was reported at all, so this arm is asserting nothing about a \ + run that happened; stdout was:\n{stdout}", + ); + assert!( + steps <= u64::from(io_cli::contract::GATED_MAX_STEPS), + "the run reached step {steps}, past the gated bound of {}. An exit code \ + eventually produced at a thousand steps is not the gate bounding \ + anything — it is the old behaviour with a slower ending", + io_cli::contract::GATED_MAX_STEPS, + ); + // The gate is reported rather than silently deciding the exit code. A `6` + // with nothing said about why is a CI failure nobody can act on. + assert!( + stderr.contains("gate"), + "stderr must say the gate is what ended this: {stderr}", + ); +} diff --git a/tests/manage.rs b/tests/manage.rs index 76bd8d5..5389c53 100644 --- a/tests/manage.rs +++ b/tests/manage.rs @@ -1392,3 +1392,76 @@ fn f3_a_mistyped_verb_names_the_verbs_that_surface_takes() { ); } } + +/// One `io` invocation against a scratch home, returning stdout and stderr. +/// +/// The home is redirected the way `tests/exec.rs` redirects it, and both +/// configuration variables are cleared, so the run cannot read this machine's own +/// `io.toml` and cannot write to it. +fn io_at(home: &Path, args: &[&str]) -> (String, String) { + let mut command = std::process::Command::new(env!("CARGO_BIN_EXE_io")); + command + .args(args) + .env_remove(io_harness::config::CONFIG_VAR) + .env_remove(io_harness::config::CONFIG_HOME_VAR); + #[cfg(windows)] + command + .env("USERPROFILE", home) + .env("APPDATA", home.join("AppData").join("Roaming")); + #[cfg(not(windows))] + command.env("HOME", home).env_remove("XDG_CONFIG_HOME"); + + let run = command.output().expect("the built binary runs"); + ( + String::from_utf8_lossy(&run.stdout).into_owned(), + String::from_utf8_lossy(&run.stderr).into_owned(), + ) +} + +/// **F2 — the shell door writes the value that was typed, through the real argv.** +/// +/// Driven against the built binary rather than against `manage::parse`, and that +/// is the point rather than thoroughness for its own sake: nothing under `tests/` +/// links `src/main.rs`, and this seam has caught something in four consecutive +/// releases. The 2026-09-05 field test drove exactly these commands at a real +/// command line and got `"4"` back. +/// +/// Sabotage: restore the unconditional quote in `configure::source_for`'s `None` +/// arm and the first two rows fail here as well as in `tests/configure.rs` — the +/// two are deliberately not the same assertion, because a unit test proves the +/// function and this proves the binary reaches it. +#[test] +fn f2_the_argv_door_writes_the_shape_that_was_typed() { + let fixture = tempfile::tempdir().expect("a temporary home"); + let home = fixture.path(); + + for (key, typed, read_back) in [ + ("app.io-cli.containment.max_total_agents", "4", "4"), + ("run.context.max_tokens", "120000", "120000"), + ("app.io-cli.plain", "true", "true"), + ("app.io-cli.gates.rubric", "be strict", "\"be strict\""), + ] { + let (_, stderr) = io_at(home, &["config", "set", key, typed, "--scope", "user"]); + let (out, _) = io_at(home, &["config", "get", key]); + + // **The value column, compared by equality — `contains` could not fail + // on the defect this test names, and the adversarial review caught it + // before the merge.** `config get` prints the raw TOML slice, so a + // regression writes `"4"` and `contains("4")` is still true: the only + // binary-level gate on this release's headline fix was passing over the + // very sabotage its doc comment described. The column is tab-separated + // (`key\tvalue\torigin`), so the value is taken by position. + let value = out + .lines() + .find(|line| line.starts_with(key)) + .and_then(|line| line.split('\t').nth(1)) + .map(str::trim) + .unwrap_or_else(|| panic!("`config get {key}` printed no row:\n{out}")); + assert_eq!( + value, read_back, + "`io config set {key} {typed}` then `get` should read back exactly \ + `{read_back}`; a quoted number here is the defect this release \ + exists to remove. The write said:\n{stderr}", + ); + } +} diff --git a/tests/reload.rs b/tests/reload.rs index f2116de..87c5f19 100644 --- a/tests/reload.rs +++ b/tests/reload.rs @@ -79,7 +79,7 @@ fn workspace(io_toml: &str, agents: &str) -> (tempfile::TempDir, Configuration) /// Whether any composed instruction carries `line`. /// /// io-harness words each instruction file as `Project instructions from -/// `AGENTS.md`:\n` (`io-harness-0.78.0/src/config.rs:2934-2937`), so the +/// `AGENTS.md`:\n` (`io-harness-0.79.0/src/config.rs:3020-3023`), so the /// assertion is a containment and not an equality — this file is not the place /// that pins the harness's wording. fn instructs(held: &Configuration, line: &str) -> bool { diff --git a/tests/triage.rs b/tests/triage.rs index d725eda..f2cd765 100644 --- a/tests/triage.rs +++ b/tests/triage.rs @@ -70,14 +70,27 @@ fn the_table_names_every_kind_the_locked_harness_declares() { "these names are no longer io-harness event kinds: {gone:?}", ); - // And the count last, which now catches only what the two sets above cannot: - // a harness that declared the same names twice, or a parser that stopped - // reading the enum and handed back a shorter list that happens to be a subset. + // And the degenerate-parse check last, which catches what the two sets above + // cannot: a harness that declared the same name twice, or a parser that read + // one line twice. A subset is already caught — a parser that handed back + // fewer names than the enum has puts every missing one in `gone`. + // + // **This was a literal `53` until 0.38.1, and the literal was the defect.** + // io-harness 0.79.0 added `Program` without declaring a break, so the count + // moved to 54 and this assertion failed with a number rather than with the + // name of the new kind — the least useful half of this test failing in place + // of the most useful one, which is the very thing the comment above says the + // ordering exists to prevent. Raising it to 54 would have repaired a count + // gate by raising its count, which weakens the gate to accommodate the change + // that fired it. The property the number was standing in for is + // *no duplicates*, and that is what is asserted now. + let unique: std::collections::BTreeSet<&String> = declared.iter().collect(); assert_eq!( + unique.len(), declared.len(), - 53, - "the locked io-harness declares fifty-three event kinds; found {}", + "the parser read {} names but only {} are distinct, so a kind was counted twice", declared.len(), + unique.len(), ); } @@ -150,7 +163,17 @@ fn the_table_has_no_duplicate_and_every_row_records_a_route() { "{name} has no route recorded, so nobody can check whether its fact reaches anyone", ); } - assert_eq!(seen.len(), 53); + // Derived from the locked harness rather than written as a literal (0.38.1). + // A row count is only ever interesting as "one per kind the harness + // declares", and a literal states that indirectly and goes stale on every + // pin that adds a variant. + assert_eq!( + seen.len(), + support::harness_event_kinds().len(), + "the table has {} rows for {} declared kinds", + seen.len(), + support::harness_event_kinds().len(), + ); } /// A `Line` kind with no arm behind it is the old defect wearing the new table's @@ -468,10 +491,27 @@ fn the_dispositions_are_the_three_the_contract_names() { .iter() .filter(|(_, d, _)| *d == Disposition::Silent) .count(); + // Against the table's own length rather than a literal (0.38.1). What this + // asserts is that the three dispositions **partition** the table — every row + // is exactly one of them, none is missed by all three — and a literal states + // that only for the length the table happened to have on the day it was + // written. `the_table_has_no_duplicate_and_every_row_records_a_route` is what + // ties `TRIAGE.len()` to the locked harness, so the two together still say + // "one of three, for every kind io-harness declares" without either one going + // stale on a pin. + // **What this can and cannot fail on, said plainly.** With three variants and + // no other way for a row to exist, the sum is a near-tautology: it fires only + // if `Disposition` grows a fourth variant and a row uses it. That is worth + // exactly what it costs, and the message says so rather than claiming to + // check "every kind" — the tie to the harness's declared set lives in + // `the_table_names_every_kind_the_locked_harness_declares` and in the row + // count below, which is where a reader should look for it. assert_eq!( lines + status + silent, - 53, - "every kind is exactly one of the three: {lines} lines, {status} status, {silent} silent", + TRIAGE.len(), + "a row carries a disposition outside the three this table counts — \ + {lines} lines, {status} status, {silent} silent, {} rows", + TRIAGE.len(), ); // The release's own claim, and the reason it exists: most kinds are not // worth a line, and until 0.11.0 thirty-seven of them got one anyway.