From 359c1f751e05bf36f9228dca861adea4c4d322fd Mon Sep 17 00:00:00 2001 From: mdsmithaustin <15272014+mdsmithaustin@users.noreply.github.com> Date: Mon, 12 Oct 2026 13:29:43 -0600 Subject: [PATCH 01/10] fix(arena): take upstream's intro cut and use a fresh scratch dir per run Closes #194. Arena's intro and Start list collapse to upstream's one worklist line, and Phase C takes upstream's wording. Phase A step 4 replaces the fixed /tmp/arena- fallback with a mktemp -d made per run and keeps the rubric outside candidate-readable paths. A retired-text rule bans the fixed path. poteto-help now says to link the repository path, such as skills/how/SKILL.md, so the printed links resolve. --- PORTING.md | 1 + skills/arena/SKILL.md | 17 +++-------------- skills/poteto-help/SKILL.md | 2 +- tools/skill-content-conventions.json | 3 ++- 4 files changed, 7 insertions(+), 16 deletions(-) diff --git a/PORTING.md b/PORTING.md index ea4b51eb..46066245 100644 --- a/PORTING.md +++ b/PORTING.md @@ -72,6 +72,7 @@ Upstream is the source of truth for the workflows themselves. This file records - A Codex lead waits instead of racing its children (port addition, 2026-10). A census of real poteto-mode sessions since 2026-09-14 found that in 106 of 339 non-blocking Codex spawns (31%), the lead read overlapping files while its child worked. The median was about 127K characters of tool output per window. Claude Code leads raced in 8 of 306 spawns. Claude's lead resumes on a completion notice, while a Codex lead holds its turn. The Codex agent-cap paragraph now has each brief name its scope. The lead stays out of a running child's scope, except for a read that a skill step says runs in parallel with the child, as arena's Phase C says of the Phase D read. It sends new facts to that child, and waits with `wait_agent` only once running children or queued arms own every remaining task. A fan-out still launches in full. - Personas announce themselves (port addition, 2026-10). A persona brief, native or pasted, tells the child to put `persona: ` on its own line in its first reply, unless the persona fixes its own first output. Comment Sicko's fixed first line, `Yes... Ha ha ha... Yes!`, is its signature. Codex encrypts spawn messages, so a pasted briefing was otherwise unprovable from a transcript. Child replies are plaintext. Setup also recommends a project-scope Codex install for wrappers that run Codex with their own `CODEX_HOME`, which never read `~/.codex/agents/`. Upstream has neither. On sync, keep both. - A rejected model is handled in one place, the **pstack-harness** skill: omit it, inherit the session model, and report the substitution. Upstream #422 gave `how`, `why`, `reflect`, `swarm`, `arena`, and `interrogate` their own fallbacks to a role's default, which the port does not carry. On sync, keep each skill's spawn sentence pointing at **Spawn a role**, which owns the rejection rule. +- Arena's scratch fallback is a fresh `mktemp -d` per run (port addition, 2026-10). Upstream's fixed `/tmp/arena-/` fallback let a later run read earlier candidates and the rubric. Phase A step 4 keeps the rubric outside candidate-readable paths, and `tools/skill-content-conventions.json` retires the fixed path. Keep it on sync and offer it upstream. - The models config is layered: a workspace `.agents/pstack-models.md` sits over `~/.agents/pstack-models.md`, and a harness section beats a flat line in either file; setup-pstack preserves values it cannot validate (another harness's choices) and has a write-fallback ladder (shell tool → workspace file → print) for harnesses that cannot write to the home directory. - The lower-ranked gaps from the same mechanism audit that still apply fail closed too (port addition, 2026-10, #120). The `/goal` gap no longer applies, because the 23e4138 sync removed `/goal` from the playbooks. `why` marks a PR it could not read through `gh` as unread. Eval grades only code shape when a transcript is unreadable. Orchestrate dispatches nothing new while `orch` cannot run. When `git worktree add` fails, Opening a PR branches off main in a current checkout with no unrelated changes, never committing to main. In an ungated repository, when the run has no built-in PR tool and neither forge CLI works, it reports a pushed branch with no PR. The persona contract never sends a generic delegate without the briefing. show-me-your-work says when the audit could not read the transcript. The GitNexus sites also need the CLI on `PATH`. Without `node`, a plan's hand-back says the check script did not run. A failed control-skill read leaves the surface with no control skill. Autopilot-stack and the multi-phase plan keep a tick running when the trunk fetch fails. On sync, keep these lines. diff --git a/skills/arena/SKILL.md b/skills/arena/SKILL.md index 21aa74be..c881a7ab 100644 --- a/skills/arena/SKILL.md +++ b/skills/arena/SKILL.md @@ -5,18 +5,7 @@ description: "Spawn N parallel candidates at the same task, pick a base, graft t # Arena -Fan out N parallel attempts at the same task. Read every candidate's rationale. Pick the strongest as the base. Graft the best ideas from the others into it. Verify the synthesized result. - -## Start - -Open a worklist with one entry per phase before launching anything. - -1. Frame -2. Fan out -3. Cross-judge -4. Pick -5. Graft -6. Verify +Open a worklist with one entry per phase (Frame, Fan out, Cross-judge, Pick, Graft, Verify) before launching anything. ## Phase A: Frame @@ -25,7 +14,7 @@ Every candidate gets the same prompt, so the prompt is the contract. 1. State the artifact each candidate is producing. 2. Derive the rubric, 3-6 concrete gradeable criteria for what success looks like on *this* task. For subjective criteria, use [Eval's criterion and calibration guidance](../poteto-mode/playbooks/eval.md). It is the picker's tool in Phase D. Candidates only see the task. 3. Pick the runners. Use the `arena runners` role, one arm per entry. Its default is one each on `fable`, `opus`, `sonnet`. A rejected entry also follows the required-arm rule below. Spawn more when the arena covers multiple design directions. Same model N times when the work is generation-bound rather than judgment-sensitive. Use the **pstack-harness** skill for the spawn. Its **Spawn a role** section resolves the role and builds this CLI's spawn call. -4. Give each candidate its own output location (a git worktree where possible, otherwise `/tmp/arena-/candidate-/`), per the **separate-before-serializing-shared-state** principle skill. +4. Give each candidate its own output location (a git worktree where possible, otherwise `candidate-/` inside a fresh `mktemp -d "${TMPDIR:-/tmp}/arena-.XXXXXX"` made for this run), per the **separate-before-serializing-shared-state** principle skill. Never reuse an earlier run's directory. Keep the rubric outside every directory a candidate can read, and give it to the cross-judge in its prompt. ## Phase B: Fan out @@ -37,7 +26,7 @@ An arm or exact model requested for this task remains an obligation. If it canno ## Phase C: Cross-judge -After all Phase B candidates complete, choose one arm of the `arena cross-judge pool` role. Use the **pstack-harness** skill's **Spawn a role** section to resolve it. Its default pool is `fable`, `opus`, `sonnet`. Prefer a different model family or capability tier from the parent's. Spawn one readonly judge subagent on that model. It sees the rubric and the candidates by path label, scores each criterion, and recommends a base with rationale. It runs in parallel with the parent's reading in Phase D, not with the candidates themselves. Don't spawn the judge while candidates are still writing. +After all Phase B candidates complete, choose one arm of the `arena cross-judge pool` role. Use the **pstack-harness** skill's **Spawn a role** section to resolve it. Its default pool is `fable`, `opus`, `sonnet`. Prefer a different model family or capability tier from the parent's. Spawn one readonly judge subagent on that model. It sees the rubric and the candidates by path label, scores each criterion, and recommends a base with rationale. It runs in parallel with your reading in Phase D. Don't spawn it while candidates are still writing. ## Phase D: Pick a base diff --git a/skills/poteto-help/SKILL.md b/skills/poteto-help/SKILL.md index d641143b..20c736d0 100644 --- a/skills/poteto-help/SKILL.md +++ b/skills/poteto-help/SKILL.md @@ -9,7 +9,7 @@ Answer the user's question about pstack, hand them a prompt they can send, and l A message that asks for work, such as "use pstack to fix this bug", is not a help question. Use the **poteto-mode** skill and do the work under it. -This file maps questions to the skills and guide pages that hold the answers. Those files own the details. Read the file you route to before you quote it, and trust it when it disagrees with this map. Read a sibling skill from the installed skills root. The README and guide pages don't ship with an installed skill, so read them from a pstack checkout when the session runs in one, and otherwise from their public copy. Give the user the file's public copy: `https://github.com/mdsmithaustin/pstack/blob/main/` followed by its path. +This file maps questions to the skills and guide pages that hold the answers. Those files own the details. Read the file you route to before you quote it, and trust it when it disagrees with this map. Read a sibling skill from the installed skills root. The README and guide pages don't ship with an installed skill, so read them from a pstack checkout when the session runs in one, and otherwise from their public copy. Give the user the file's public copy: `https://github.com/mdsmithaustin/pstack/blob/main/` followed by its repository path, such as `skills/how/SKILL.md` or `docs/guide/01-setup.md`. A skill's file sits under `skills/`, so a path relative to the skills root, such as `how/SKILL.md`, 404s. ## Find out what they need diff --git a/tools/skill-content-conventions.json b/tools/skill-content-conventions.json index fbab3d15..affd0341 100644 --- a/tools/skill-content-conventions.json +++ b/tools/skill-content-conventions.json @@ -7,7 +7,8 @@ "gpt-5.6": "name gpt-6 models; the resolver runs gpt-6.1 where Codex lists it", "environment: \"cloud\"": "workers are local; spawn a background agent in its own worktree", "cloud_base_branch": "give the worker a worktree at the branch's SHA and name it in the brief", - "cloud-sleeper": "arm the audit tick through the harness loop facility" + "cloud-sleeper": "arm the audit tick through the harness loop facility", + "/tmp/arena-": "use a fresh mktemp -d per arena run; a fixed /tmp path reuses stale candidates" }, "skill_prefixes": [ "principle-" From 6ab71761467b34e07e2e0e30eb64e120cdbca324 Mon Sep 17 00:00:00 2001 From: mdsmithaustin <15272014+mdsmithaustin@users.noreply.github.com> Date: Mon, 12 Oct 2026 14:03:16 -0600 Subject: [PATCH 02/10] fix(arena): delete a finished run's scratch dir and take upstream's step 3 ending Phase A step 3 ends with upstream #538's wording ("Spawn more runners when", "Use the same model N times when"). Step 4 now deletes the run's scratch directory when the run ends, after the synthesized artifact and note are saved outside it, and before any re-run from Phase E or F. Without that, a later run could still list and read an earlier run's sibling arena-.* directory. The PORTING note now claims only what the text prevents. --- PORTING.md | 2 +- skills/arena/SKILL.md | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/PORTING.md b/PORTING.md index 46066245..aecb3c47 100644 --- a/PORTING.md +++ b/PORTING.md @@ -72,7 +72,7 @@ Upstream is the source of truth for the workflows themselves. This file records - A Codex lead waits instead of racing its children (port addition, 2026-10). A census of real poteto-mode sessions since 2026-09-14 found that in 106 of 339 non-blocking Codex spawns (31%), the lead read overlapping files while its child worked. The median was about 127K characters of tool output per window. Claude Code leads raced in 8 of 306 spawns. Claude's lead resumes on a completion notice, while a Codex lead holds its turn. The Codex agent-cap paragraph now has each brief name its scope. The lead stays out of a running child's scope, except for a read that a skill step says runs in parallel with the child, as arena's Phase C says of the Phase D read. It sends new facts to that child, and waits with `wait_agent` only once running children or queued arms own every remaining task. A fan-out still launches in full. - Personas announce themselves (port addition, 2026-10). A persona brief, native or pasted, tells the child to put `persona: ` on its own line in its first reply, unless the persona fixes its own first output. Comment Sicko's fixed first line, `Yes... Ha ha ha... Yes!`, is its signature. Codex encrypts spawn messages, so a pasted briefing was otherwise unprovable from a transcript. Child replies are plaintext. Setup also recommends a project-scope Codex install for wrappers that run Codex with their own `CODEX_HOME`, which never read `~/.codex/agents/`. Upstream has neither. On sync, keep both. - A rejected model is handled in one place, the **pstack-harness** skill: omit it, inherit the session model, and report the substitution. Upstream #422 gave `how`, `why`, `reflect`, `swarm`, `arena`, and `interrogate` their own fallbacks to a role's default, which the port does not carry. On sync, keep each skill's spawn sentence pointing at **Spawn a role**, which owns the rejection rule. -- Arena's scratch fallback is a fresh `mktemp -d` per run (port addition, 2026-10). Upstream's fixed `/tmp/arena-/` fallback let a later run read earlier candidates and the rubric. Phase A step 4 keeps the rubric outside candidate-readable paths, and `tools/skill-content-conventions.json` retires the fixed path. Keep it on sync and offer it upstream. +- Arena's scratch fallback is a fresh `mktemp -d` per run (port addition, 2026-10). Upstream's fixed `/tmp/arena-/` fallback let a later run read earlier candidates and the rubric. Phase A step 4 gives each run its own directory, deletes it when the run ends or before a Phase E or F re-run, and keeps the rubric outside candidate-readable paths. A run that follows the text finds no earlier run's candidates. The text cannot stop a lead that skips the delete, and another run's directory left by an older copy of the skill stays readable. `tools/skill-content-conventions.json` retires the fixed path. Keep it on sync and offer it upstream. - The models config is layered: a workspace `.agents/pstack-models.md` sits over `~/.agents/pstack-models.md`, and a harness section beats a flat line in either file; setup-pstack preserves values it cannot validate (another harness's choices) and has a write-fallback ladder (shell tool → workspace file → print) for harnesses that cannot write to the home directory. - The lower-ranked gaps from the same mechanism audit that still apply fail closed too (port addition, 2026-10, #120). The `/goal` gap no longer applies, because the 23e4138 sync removed `/goal` from the playbooks. `why` marks a PR it could not read through `gh` as unread. Eval grades only code shape when a transcript is unreadable. Orchestrate dispatches nothing new while `orch` cannot run. When `git worktree add` fails, Opening a PR branches off main in a current checkout with no unrelated changes, never committing to main. In an ungated repository, when the run has no built-in PR tool and neither forge CLI works, it reports a pushed branch with no PR. The persona contract never sends a generic delegate without the briefing. show-me-your-work says when the audit could not read the transcript. The GitNexus sites also need the CLI on `PATH`. Without `node`, a plan's hand-back says the check script did not run. A failed control-skill read leaves the surface with no control skill. Autopilot-stack and the multi-phase plan keep a tick running when the trunk fetch fails. On sync, keep these lines. diff --git a/skills/arena/SKILL.md b/skills/arena/SKILL.md index c881a7ab..4a61d2cd 100644 --- a/skills/arena/SKILL.md +++ b/skills/arena/SKILL.md @@ -13,8 +13,8 @@ Every candidate gets the same prompt, so the prompt is the contract. 1. State the artifact each candidate is producing. 2. Derive the rubric, 3-6 concrete gradeable criteria for what success looks like on *this* task. For subjective criteria, use [Eval's criterion and calibration guidance](../poteto-mode/playbooks/eval.md). It is the picker's tool in Phase D. Candidates only see the task. -3. Pick the runners. Use the `arena runners` role, one arm per entry. Its default is one each on `fable`, `opus`, `sonnet`. A rejected entry also follows the required-arm rule below. Spawn more when the arena covers multiple design directions. Same model N times when the work is generation-bound rather than judgment-sensitive. Use the **pstack-harness** skill for the spawn. Its **Spawn a role** section resolves the role and builds this CLI's spawn call. -4. Give each candidate its own output location (a git worktree where possible, otherwise `candidate-/` inside a fresh `mktemp -d "${TMPDIR:-/tmp}/arena-.XXXXXX"` made for this run), per the **separate-before-serializing-shared-state** principle skill. Never reuse an earlier run's directory. Keep the rubric outside every directory a candidate can read, and give it to the cross-judge in its prompt. +3. Pick the runners. Use the `arena runners` role, one arm per entry. Its default is one each on `fable`, `opus`, `sonnet`. A rejected entry also follows the required-arm rule below. Spawn more runners when the arena covers multiple design directions. Use the same model N times when the work is generation-bound rather than judgment-sensitive. Use the **pstack-harness** skill for the spawn. Its **Spawn a role** section resolves the role and builds this CLI's spawn call. +4. Give each candidate its own output location (a git worktree where possible, otherwise `candidate-/` inside a fresh `mktemp -d "${TMPDIR:-/tmp}/arena-.XXXXXX"` made for this run), per the **separate-before-serializing-shared-state** principle skill. Never reuse an earlier run's directory. Delete the directory when the run ends, after saving the synthesized artifact and synthesis note outside it, and before any re-run from Phase E or F starts. Keep the rubric outside every directory a candidate can read, and give it to the cross-judge in its prompt. ## Phase B: Fan out From 5993d64ec6e105397a2d1d0401e4120fe2fd3fb1 Mon Sep 17 00:00:00 2001 From: mdsmithaustin <15272014+mdsmithaustin@users.noreply.github.com> Date: Mon, 12 Oct 2026 14:23:07 -0600 Subject: [PATCH 03/10] fix(arena): keep the scratch dir when Phase F goes back to Phase E to graft --- PORTING.md | 2 +- skills/arena/SKILL.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/PORTING.md b/PORTING.md index aecb3c47..a47d0989 100644 --- a/PORTING.md +++ b/PORTING.md @@ -72,7 +72,7 @@ Upstream is the source of truth for the workflows themselves. This file records - A Codex lead waits instead of racing its children (port addition, 2026-10). A census of real poteto-mode sessions since 2026-09-14 found that in 106 of 339 non-blocking Codex spawns (31%), the lead read overlapping files while its child worked. The median was about 127K characters of tool output per window. Claude Code leads raced in 8 of 306 spawns. Claude's lead resumes on a completion notice, while a Codex lead holds its turn. The Codex agent-cap paragraph now has each brief name its scope. The lead stays out of a running child's scope, except for a read that a skill step says runs in parallel with the child, as arena's Phase C says of the Phase D read. It sends new facts to that child, and waits with `wait_agent` only once running children or queued arms own every remaining task. A fan-out still launches in full. - Personas announce themselves (port addition, 2026-10). A persona brief, native or pasted, tells the child to put `persona: ` on its own line in its first reply, unless the persona fixes its own first output. Comment Sicko's fixed first line, `Yes... Ha ha ha... Yes!`, is its signature. Codex encrypts spawn messages, so a pasted briefing was otherwise unprovable from a transcript. Child replies are plaintext. Setup also recommends a project-scope Codex install for wrappers that run Codex with their own `CODEX_HOME`, which never read `~/.codex/agents/`. Upstream has neither. On sync, keep both. - A rejected model is handled in one place, the **pstack-harness** skill: omit it, inherit the session model, and report the substitution. Upstream #422 gave `how`, `why`, `reflect`, `swarm`, `arena`, and `interrogate` their own fallbacks to a role's default, which the port does not carry. On sync, keep each skill's spawn sentence pointing at **Spawn a role**, which owns the rejection rule. -- Arena's scratch fallback is a fresh `mktemp -d` per run (port addition, 2026-10). Upstream's fixed `/tmp/arena-/` fallback let a later run read earlier candidates and the rubric. Phase A step 4 gives each run its own directory, deletes it when the run ends or before a Phase E or F re-run, and keeps the rubric outside candidate-readable paths. A run that follows the text finds no earlier run's candidates. The text cannot stop a lead that skips the delete, and another run's directory left by an older copy of the skill stays readable. `tools/skill-content-conventions.json` retires the fixed path. Keep it on sync and offer it upstream. +- Arena's scratch fallback is a fresh `mktemp -d` per run (port addition, 2026-10). Upstream's fixed `/tmp/arena-/` fallback let a later run read earlier candidates and the rubric. Phase A step 4 gives each run its own directory, deletes it when the run ends or before a re-frame and re-run from Phase E or F, and keeps the rubric outside candidate-readable paths. A run that follows the text finds no earlier run's candidates. The text cannot stop a lead that skips the delete, and another run's directory left by an older copy of the skill stays readable. `tools/skill-content-conventions.json` retires the fixed path. Keep it on sync and offer it upstream. - The models config is layered: a workspace `.agents/pstack-models.md` sits over `~/.agents/pstack-models.md`, and a harness section beats a flat line in either file; setup-pstack preserves values it cannot validate (another harness's choices) and has a write-fallback ladder (shell tool → workspace file → print) for harnesses that cannot write to the home directory. - The lower-ranked gaps from the same mechanism audit that still apply fail closed too (port addition, 2026-10, #120). The `/goal` gap no longer applies, because the 23e4138 sync removed `/goal` from the playbooks. `why` marks a PR it could not read through `gh` as unread. Eval grades only code shape when a transcript is unreadable. Orchestrate dispatches nothing new while `orch` cannot run. When `git worktree add` fails, Opening a PR branches off main in a current checkout with no unrelated changes, never committing to main. In an ungated repository, when the run has no built-in PR tool and neither forge CLI works, it reports a pushed branch with no PR. The persona contract never sends a generic delegate without the briefing. show-me-your-work says when the audit could not read the transcript. The GitNexus sites also need the CLI on `PATH`. Without `node`, a plan's hand-back says the check script did not run. A failed control-skill read leaves the surface with no control skill. Autopilot-stack and the multi-phase plan keep a tick running when the trunk fetch fails. On sync, keep these lines. diff --git a/skills/arena/SKILL.md b/skills/arena/SKILL.md index 4a61d2cd..33072073 100644 --- a/skills/arena/SKILL.md +++ b/skills/arena/SKILL.md @@ -14,7 +14,7 @@ Every candidate gets the same prompt, so the prompt is the contract. 1. State the artifact each candidate is producing. 2. Derive the rubric, 3-6 concrete gradeable criteria for what success looks like on *this* task. For subjective criteria, use [Eval's criterion and calibration guidance](../poteto-mode/playbooks/eval.md). It is the picker's tool in Phase D. Candidates only see the task. 3. Pick the runners. Use the `arena runners` role, one arm per entry. Its default is one each on `fable`, `opus`, `sonnet`. A rejected entry also follows the required-arm rule below. Spawn more runners when the arena covers multiple design directions. Use the same model N times when the work is generation-bound rather than judgment-sensitive. Use the **pstack-harness** skill for the spawn. Its **Spawn a role** section resolves the role and builds this CLI's spawn call. -4. Give each candidate its own output location (a git worktree where possible, otherwise `candidate-/` inside a fresh `mktemp -d "${TMPDIR:-/tmp}/arena-.XXXXXX"` made for this run), per the **separate-before-serializing-shared-state** principle skill. Never reuse an earlier run's directory. Delete the directory when the run ends, after saving the synthesized artifact and synthesis note outside it, and before any re-run from Phase E or F starts. Keep the rubric outside every directory a candidate can read, and give it to the cross-judge in its prompt. +4. Give each candidate its own output location (a git worktree where possible, otherwise `candidate-/` inside a fresh `mktemp -d "${TMPDIR:-/tmp}/arena-.XXXXXX"` made for this run), per the **separate-before-serializing-shared-state** principle skill. Never reuse an earlier run's directory. Delete the directory when the run ends, after saving the synthesized artifact and synthesis note outside it, and before a re-frame and re-run starts, whether Phase E or Phase F sent you back. Going back to Phase E to graft keeps it. Keep the rubric outside every directory a candidate can read, and give it to the cross-judge in its prompt. ## Phase B: Fan out From 7e99de0df226d41e14bd8b3bb46807444231750f Mon Sep 17 00:00:00 2001 From: mdsmithaustin <15272014+mdsmithaustin@users.noreply.github.com> Date: Mon, 12 Oct 2026 15:04:29 -0600 Subject: [PATCH 04/10] fix(arena): name where outputs go and clean up candidate worktrees "Save the synthesized artifact and synthesis note outside it" named no place, and a Codex lead kept synthesis.md in its run folder. Outputs now puts the artifact where the calling task says, or in a fresh mktemp -d outside the project, and gives the synthesis note in the reply. The preferred git worktree branch was never cleaned up, so the next run saw earlier candidates through git worktree list. Cleanup now removes each candidate worktree and its branch as well as the scratch directory, and Phase E and Phase F repeat the rule where the lead re-frames or finishes. PORTING claims only what this prevents for a run that completes and names what it does not cover (#209). --- PORTING.md | 2 +- skills/arena/SKILL.md | 8 ++++---- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/PORTING.md b/PORTING.md index a47d0989..bfd065fe 100644 --- a/PORTING.md +++ b/PORTING.md @@ -72,7 +72,7 @@ Upstream is the source of truth for the workflows themselves. This file records - A Codex lead waits instead of racing its children (port addition, 2026-10). A census of real poteto-mode sessions since 2026-09-14 found that in 106 of 339 non-blocking Codex spawns (31%), the lead read overlapping files while its child worked. The median was about 127K characters of tool output per window. Claude Code leads raced in 8 of 306 spawns. Claude's lead resumes on a completion notice, while a Codex lead holds its turn. The Codex agent-cap paragraph now has each brief name its scope. The lead stays out of a running child's scope, except for a read that a skill step says runs in parallel with the child, as arena's Phase C says of the Phase D read. It sends new facts to that child, and waits with `wait_agent` only once running children or queued arms own every remaining task. A fan-out still launches in full. - Personas announce themselves (port addition, 2026-10). A persona brief, native or pasted, tells the child to put `persona: ` on its own line in its first reply, unless the persona fixes its own first output. Comment Sicko's fixed first line, `Yes... Ha ha ha... Yes!`, is its signature. Codex encrypts spawn messages, so a pasted briefing was otherwise unprovable from a transcript. Child replies are plaintext. Setup also recommends a project-scope Codex install for wrappers that run Codex with their own `CODEX_HOME`, which never read `~/.codex/agents/`. Upstream has neither. On sync, keep both. - A rejected model is handled in one place, the **pstack-harness** skill: omit it, inherit the session model, and report the substitution. Upstream #422 gave `how`, `why`, `reflect`, `swarm`, `arena`, and `interrogate` their own fallbacks to a role's default, which the port does not carry. On sync, keep each skill's spawn sentence pointing at **Spawn a role**, which owns the rejection rule. -- Arena's scratch fallback is a fresh `mktemp -d` per run (port addition, 2026-10). Upstream's fixed `/tmp/arena-/` fallback let a later run read earlier candidates and the rubric. Phase A step 4 gives each run its own directory, deletes it when the run ends or before a re-frame and re-run from Phase E or F, and keeps the rubric outside candidate-readable paths. A run that follows the text finds no earlier run's candidates. The text cannot stop a lead that skips the delete, and another run's directory left by an older copy of the skill stays readable. `tools/skill-content-conventions.json` retires the fixed path. Keep it on sync and offer it upstream. +- Arena's scratch fallback is a fresh `mktemp -d` per run (port addition, 2026-10). Upstream's fixed `/tmp/arena-/` fallback let a later run read earlier candidates and the rubric. Phase A step 4 gives each run its own directory and keeps the rubric outside candidate-readable paths. Phase A step 4, Phase E, and Phase F remove the candidate worktrees and their branches and delete the scratch directory when the run ends and before a re-frame and re-run. Outputs puts the synthesized artifact where the calling task says, or in a fresh `mktemp -d` outside the project, and the synthesis note in the reply. After a run that completes, the next run finds none of its candidate files, `arena-.*` directories, worktrees, branches, or judge scores, and nothing of it in the project. The saved artifact stays where Outputs put it. The text does not cover a run cut off before the cleanup, as in the Claude Code headless turn end and the Grok Build 2400 s cap recorded in `evals/promises/results/2026-10-08-sync-df58112.md`, runs that happen at the same time, a directory left by an older copy of the skill, or a candidate that searches the temp directory and finds a saved artifact there. Stronger isolation needs a sandbox per candidate or per run (#209). `tools/skill-content-conventions.json` retires the fixed path. Keep it on sync and offer it upstream. - The models config is layered: a workspace `.agents/pstack-models.md` sits over `~/.agents/pstack-models.md`, and a harness section beats a flat line in either file; setup-pstack preserves values it cannot validate (another harness's choices) and has a write-fallback ladder (shell tool → workspace file → print) for harnesses that cannot write to the home directory. - The lower-ranked gaps from the same mechanism audit that still apply fail closed too (port addition, 2026-10, #120). The `/goal` gap no longer applies, because the 23e4138 sync removed `/goal` from the playbooks. `why` marks a PR it could not read through `gh` as unread. Eval grades only code shape when a transcript is unreadable. Orchestrate dispatches nothing new while `orch` cannot run. When `git worktree add` fails, Opening a PR branches off main in a current checkout with no unrelated changes, never committing to main. In an ungated repository, when the run has no built-in PR tool and neither forge CLI works, it reports a pushed branch with no PR. The persona contract never sends a generic delegate without the briefing. show-me-your-work says when the audit could not read the transcript. The GitNexus sites also need the CLI on `PATH`. Without `node`, a plan's hand-back says the check script did not run. A failed control-skill read leaves the surface with no control skill. Autopilot-stack and the multi-phase plan keep a tick running when the trunk fetch fails. On sync, keep these lines. diff --git a/skills/arena/SKILL.md b/skills/arena/SKILL.md index 33072073..17463129 100644 --- a/skills/arena/SKILL.md +++ b/skills/arena/SKILL.md @@ -14,7 +14,7 @@ Every candidate gets the same prompt, so the prompt is the contract. 1. State the artifact each candidate is producing. 2. Derive the rubric, 3-6 concrete gradeable criteria for what success looks like on *this* task. For subjective criteria, use [Eval's criterion and calibration guidance](../poteto-mode/playbooks/eval.md). It is the picker's tool in Phase D. Candidates only see the task. 3. Pick the runners. Use the `arena runners` role, one arm per entry. Its default is one each on `fable`, `opus`, `sonnet`. A rejected entry also follows the required-arm rule below. Spawn more runners when the arena covers multiple design directions. Use the same model N times when the work is generation-bound rather than judgment-sensitive. Use the **pstack-harness** skill for the spawn. Its **Spawn a role** section resolves the role and builds this CLI's spawn call. -4. Give each candidate its own output location (a git worktree where possible, otherwise `candidate-/` inside a fresh `mktemp -d "${TMPDIR:-/tmp}/arena-.XXXXXX"` made for this run), per the **separate-before-serializing-shared-state** principle skill. Never reuse an earlier run's directory. Delete the directory when the run ends, after saving the synthesized artifact and synthesis note outside it, and before a re-frame and re-run starts, whether Phase E or Phase F sent you back. Going back to Phase E to graft keeps it. Keep the rubric outside every directory a candidate can read, and give it to the cross-judge in its prompt. +4. Give each candidate its own output location (a git worktree where possible, otherwise `candidate-/` inside a fresh `mktemp -d "${TMPDIR:-/tmp}/arena-.XXXXXX"` made for this run), per the **separate-before-serializing-shared-state** principle skill. Never reuse an earlier run's directory. Clean up the run when it ends, after saving its outputs per Outputs, and before a re-frame and re-run starts, whether Phase E or Phase F sent you back. To clean up, remove each candidate worktree with `git worktree remove --force `, delete any branch made for it with `git branch -D`, and delete the scratch directory. Going back to Phase E to graft keeps them. Keep the rubric outside every directory a candidate can read, and give it to the cross-judge in its prompt. ## Phase B: Fan out @@ -40,12 +40,12 @@ Pick the base a future maintainer can extend most easily without breaking invari Walk each losing candidate's rationale and the cross-judge's scores once more for what is worth porting into the base, usually one or two things per candidate, not most of it. Read only the parts of a losing artifact you will graft. Fold each graft in by hand, per the **redesign-from-first-principles** principle skill, so the result stays coherent under one mental model. Don't paste mechanically. -If the candidates converge on one shape, that is strong agreement. Note it and ship the consensus shape. No graft is needed. If they wildly diverge, Phase A was under-specified. Reframe and re-run rather than averaging the divergence. +If the candidates converge on one shape, that is strong agreement. Note it and ship the consensus shape. No graft is needed. If they wildly diverge, Phase A was under-specified. Reframe and re-run rather than averaging the divergence. Before the re-run, remove this run's candidate worktrees and their branches and delete its scratch directory. ## Phase F: Verify -Verify the synthesized artifact like any other output, per the **prove-it-works** principle skill. If verification finds a problem the arena missed, either Phase A was wrong (re-frame and re-run) or a candidate caught it and you missed the graft (go back to Phase E). Don't paper over it. +Verify the synthesized artifact like any other output, per the **prove-it-works** principle skill. If verification finds a problem the arena missed, either Phase A was wrong (re-frame and re-run) or a candidate caught it and you missed the graft (go back to Phase E). Don't paper over it. Before a re-frame and re-run, remove this run's candidate worktrees and their branches and delete its scratch directory. Going back to Phase E keeps them. Once verification passes, save the outputs, then clean up the same way. ## Outputs -One synthesized artifact, with a short synthesis note alongside naming the base and why, the cross-judge's verdict, each graft with its source candidate, what was rejected and why, any convergence or dropouts, and the verification result. +One synthesized artifact, saved where the calling task says, or, if it names no place, in a fresh `mktemp -d` outside the project. Give a short synthesis note in your reply rather than as a file, naming the base and why, the cross-judge's verdict, each graft with its source candidate, what was rejected and why, any convergence or dropouts, and the verification result. From 162f169fe3d2b2ed64c8fe32d30bdb9a6d3247c8 Mon Sep 17 00:00:00 2001 From: mdsmithaustin <15272014+mdsmithaustin@users.noreply.github.com> Date: Mon, 12 Oct 2026 15:14:22 -0600 Subject: [PATCH 05/10] =?UTF-8?q?no-mistakes(review):=20Route=20arena=20co?= =?UTF-8?q?de=20outputs=20onto=20the=20calling=20task=E2=80=99s=20branch?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- PORTING.md | 2 +- skills/arena/SKILL.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/PORTING.md b/PORTING.md index bfd065fe..2cbc5a19 100644 --- a/PORTING.md +++ b/PORTING.md @@ -72,7 +72,7 @@ Upstream is the source of truth for the workflows themselves. This file records - A Codex lead waits instead of racing its children (port addition, 2026-10). A census of real poteto-mode sessions since 2026-09-14 found that in 106 of 339 non-blocking Codex spawns (31%), the lead read overlapping files while its child worked. The median was about 127K characters of tool output per window. Claude Code leads raced in 8 of 306 spawns. Claude's lead resumes on a completion notice, while a Codex lead holds its turn. The Codex agent-cap paragraph now has each brief name its scope. The lead stays out of a running child's scope, except for a read that a skill step says runs in parallel with the child, as arena's Phase C says of the Phase D read. It sends new facts to that child, and waits with `wait_agent` only once running children or queued arms own every remaining task. A fan-out still launches in full. - Personas announce themselves (port addition, 2026-10). A persona brief, native or pasted, tells the child to put `persona: ` on its own line in its first reply, unless the persona fixes its own first output. Comment Sicko's fixed first line, `Yes... Ha ha ha... Yes!`, is its signature. Codex encrypts spawn messages, so a pasted briefing was otherwise unprovable from a transcript. Child replies are plaintext. Setup also recommends a project-scope Codex install for wrappers that run Codex with their own `CODEX_HOME`, which never read `~/.codex/agents/`. Upstream has neither. On sync, keep both. - A rejected model is handled in one place, the **pstack-harness** skill: omit it, inherit the session model, and report the substitution. Upstream #422 gave `how`, `why`, `reflect`, `swarm`, `arena`, and `interrogate` their own fallbacks to a role's default, which the port does not carry. On sync, keep each skill's spawn sentence pointing at **Spawn a role**, which owns the rejection rule. -- Arena's scratch fallback is a fresh `mktemp -d` per run (port addition, 2026-10). Upstream's fixed `/tmp/arena-/` fallback let a later run read earlier candidates and the rubric. Phase A step 4 gives each run its own directory and keeps the rubric outside candidate-readable paths. Phase A step 4, Phase E, and Phase F remove the candidate worktrees and their branches and delete the scratch directory when the run ends and before a re-frame and re-run. Outputs puts the synthesized artifact where the calling task says, or in a fresh `mktemp -d` outside the project, and the synthesis note in the reply. After a run that completes, the next run finds none of its candidate files, `arena-.*` directories, worktrees, branches, or judge scores, and nothing of it in the project. The saved artifact stays where Outputs put it. The text does not cover a run cut off before the cleanup, as in the Claude Code headless turn end and the Grok Build 2400 s cap recorded in `evals/promises/results/2026-10-08-sync-df58112.md`, runs that happen at the same time, a directory left by an older copy of the skill, or a candidate that searches the temp directory and finds a saved artifact there. Stronger isolation needs a sandbox per candidate or per run (#209). `tools/skill-content-conventions.json` retires the fixed path. Keep it on sync and offer it upstream. +- Arena's scratch fallback is a fresh `mktemp -d` per run (port addition, 2026-10). Upstream's fixed `/tmp/arena-/` fallback let a later run read earlier candidates and the rubric. Phase A step 4 gives each run its own directory and keeps the rubric outside candidate-readable paths. Phase A step 4, Phase E, and Phase F remove the candidate worktrees and their branches and delete the scratch directory when the run ends and before a re-frame and re-run. Outputs puts the synthesized artifact where the calling task says, code on the branch the task works on, and any other artifact with no named place in a fresh `mktemp -d` outside the project, and the synthesis note in the reply. After a run that completes, the next run finds none of its candidate files, `arena-.*` directories, worktrees, branches, or judge scores, and nothing of it in the project. The saved artifact stays where Outputs put it. The text does not cover a run cut off before the cleanup, as in the Claude Code headless turn end and the Grok Build 2400 s cap recorded in `evals/promises/results/2026-10-08-sync-df58112.md`, runs that happen at the same time, a directory left by an older copy of the skill, or a candidate that searches the temp directory and finds a saved artifact there. Stronger isolation needs a sandbox per candidate or per run (#209). `tools/skill-content-conventions.json` retires the fixed path. Keep it on sync and offer it upstream. - The models config is layered: a workspace `.agents/pstack-models.md` sits over `~/.agents/pstack-models.md`, and a harness section beats a flat line in either file; setup-pstack preserves values it cannot validate (another harness's choices) and has a write-fallback ladder (shell tool → workspace file → print) for harnesses that cannot write to the home directory. - The lower-ranked gaps from the same mechanism audit that still apply fail closed too (port addition, 2026-10, #120). The `/goal` gap no longer applies, because the 23e4138 sync removed `/goal` from the playbooks. `why` marks a PR it could not read through `gh` as unread. Eval grades only code shape when a transcript is unreadable. Orchestrate dispatches nothing new while `orch` cannot run. When `git worktree add` fails, Opening a PR branches off main in a current checkout with no unrelated changes, never committing to main. In an ungated repository, when the run has no built-in PR tool and neither forge CLI works, it reports a pushed branch with no PR. The persona contract never sends a generic delegate without the briefing. show-me-your-work says when the audit could not read the transcript. The GitNexus sites also need the CLI on `PATH`. Without `node`, a plan's hand-back says the check script did not run. A failed control-skill read leaves the surface with no control skill. Autopilot-stack and the multi-phase plan keep a tick running when the trunk fetch fails. On sync, keep these lines. diff --git a/skills/arena/SKILL.md b/skills/arena/SKILL.md index 17463129..381c0f38 100644 --- a/skills/arena/SKILL.md +++ b/skills/arena/SKILL.md @@ -48,4 +48,4 @@ Verify the synthesized artifact like any other output, per the **prove-it-works* ## Outputs -One synthesized artifact, saved where the calling task says, or, if it names no place, in a fresh `mktemp -d` outside the project. Give a short synthesis note in your reply rather than as a file, naming the base and why, the cross-judge's verdict, each graft with its source candidate, what was rejected and why, any convergence or dropouts, and the verification result. +One synthesized artifact, saved where the calling task says. Code goes onto the branch the calling task works on. Any other artifact whose task names no place goes in a fresh `mktemp -d` outside the project. Give a short synthesis note in your reply rather than as a file, naming the base and why, the cross-judge's verdict, each graft with its source candidate, what was rejected and why, any convergence or dropouts, and the verification result. From 7fb99d71e05240aec581af052d51f80ff8cf12b4 Mon Sep 17 00:00:00 2001 From: mdsmithaustin <15272014+mdsmithaustin@users.noreply.github.com> Date: Mon, 12 Oct 2026 15:15:53 -0600 Subject: [PATCH 06/10] no-mistakes(review): Limit arena branch outputs to code the task ships --- PORTING.md | 2 +- skills/arena/SKILL.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/PORTING.md b/PORTING.md index 2cbc5a19..05654bfa 100644 --- a/PORTING.md +++ b/PORTING.md @@ -72,7 +72,7 @@ Upstream is the source of truth for the workflows themselves. This file records - A Codex lead waits instead of racing its children (port addition, 2026-10). A census of real poteto-mode sessions since 2026-09-14 found that in 106 of 339 non-blocking Codex spawns (31%), the lead read overlapping files while its child worked. The median was about 127K characters of tool output per window. Claude Code leads raced in 8 of 306 spawns. Claude's lead resumes on a completion notice, while a Codex lead holds its turn. The Codex agent-cap paragraph now has each brief name its scope. The lead stays out of a running child's scope, except for a read that a skill step says runs in parallel with the child, as arena's Phase C says of the Phase D read. It sends new facts to that child, and waits with `wait_agent` only once running children or queued arms own every remaining task. A fan-out still launches in full. - Personas announce themselves (port addition, 2026-10). A persona brief, native or pasted, tells the child to put `persona: ` on its own line in its first reply, unless the persona fixes its own first output. Comment Sicko's fixed first line, `Yes... Ha ha ha... Yes!`, is its signature. Codex encrypts spawn messages, so a pasted briefing was otherwise unprovable from a transcript. Child replies are plaintext. Setup also recommends a project-scope Codex install for wrappers that run Codex with their own `CODEX_HOME`, which never read `~/.codex/agents/`. Upstream has neither. On sync, keep both. - A rejected model is handled in one place, the **pstack-harness** skill: omit it, inherit the session model, and report the substitution. Upstream #422 gave `how`, `why`, `reflect`, `swarm`, `arena`, and `interrogate` their own fallbacks to a role's default, which the port does not carry. On sync, keep each skill's spawn sentence pointing at **Spawn a role**, which owns the rejection rule. -- Arena's scratch fallback is a fresh `mktemp -d` per run (port addition, 2026-10). Upstream's fixed `/tmp/arena-/` fallback let a later run read earlier candidates and the rubric. Phase A step 4 gives each run its own directory and keeps the rubric outside candidate-readable paths. Phase A step 4, Phase E, and Phase F remove the candidate worktrees and their branches and delete the scratch directory when the run ends and before a re-frame and re-run. Outputs puts the synthesized artifact where the calling task says, code on the branch the task works on, and any other artifact with no named place in a fresh `mktemp -d` outside the project, and the synthesis note in the reply. After a run that completes, the next run finds none of its candidate files, `arena-.*` directories, worktrees, branches, or judge scores, and nothing of it in the project. The saved artifact stays where Outputs put it. The text does not cover a run cut off before the cleanup, as in the Claude Code headless turn end and the Grok Build 2400 s cap recorded in `evals/promises/results/2026-10-08-sync-df58112.md`, runs that happen at the same time, a directory left by an older copy of the skill, or a candidate that searches the temp directory and finds a saved artifact there. Stronger isolation needs a sandbox per candidate or per run (#209). `tools/skill-content-conventions.json` retires the fixed path. Keep it on sync and offer it upstream. +- Arena's scratch fallback is a fresh `mktemp -d` per run (port addition, 2026-10). Upstream's fixed `/tmp/arena-/` fallback let a later run read earlier candidates and the rubric. Phase A step 4 gives each run its own directory and keeps the rubric outside candidate-readable paths. Phase A step 4, Phase E, and Phase F remove the candidate worktrees and their branches and delete the scratch directory when the run ends and before a re-frame and re-run. Outputs puts the synthesized artifact where the calling task says, code the task ships on the branch it works on, and any other artifact with no named place in a fresh `mktemp -d` outside the project, and the synthesis note in the reply. After a run that completes, the next run finds none of its candidate files, `arena-.*` directories, worktrees, branches, or judge scores, and nothing of it in the project. The saved artifact stays where Outputs put it. The text does not cover a run cut off before the cleanup, as in the Claude Code headless turn end and the Grok Build 2400 s cap recorded in `evals/promises/results/2026-10-08-sync-df58112.md`, runs that happen at the same time, a directory left by an older copy of the skill, or a candidate that searches the temp directory and finds a saved artifact there. Stronger isolation needs a sandbox per candidate or per run (#209). `tools/skill-content-conventions.json` retires the fixed path. Keep it on sync and offer it upstream. - The models config is layered: a workspace `.agents/pstack-models.md` sits over `~/.agents/pstack-models.md`, and a harness section beats a flat line in either file; setup-pstack preserves values it cannot validate (another harness's choices) and has a write-fallback ladder (shell tool → workspace file → print) for harnesses that cannot write to the home directory. - The lower-ranked gaps from the same mechanism audit that still apply fail closed too (port addition, 2026-10, #120). The `/goal` gap no longer applies, because the 23e4138 sync removed `/goal` from the playbooks. `why` marks a PR it could not read through `gh` as unread. Eval grades only code shape when a transcript is unreadable. Orchestrate dispatches nothing new while `orch` cannot run. When `git worktree add` fails, Opening a PR branches off main in a current checkout with no unrelated changes, never committing to main. In an ungated repository, when the run has no built-in PR tool and neither forge CLI works, it reports a pushed branch with no PR. The persona contract never sends a generic delegate without the briefing. show-me-your-work says when the audit could not read the transcript. The GitNexus sites also need the CLI on `PATH`. Without `node`, a plan's hand-back says the check script did not run. A failed control-skill read leaves the surface with no control skill. Autopilot-stack and the multi-phase plan keep a tick running when the trunk fetch fails. On sync, keep these lines. diff --git a/skills/arena/SKILL.md b/skills/arena/SKILL.md index 381c0f38..e79bdf81 100644 --- a/skills/arena/SKILL.md +++ b/skills/arena/SKILL.md @@ -48,4 +48,4 @@ Verify the synthesized artifact like any other output, per the **prove-it-works* ## Outputs -One synthesized artifact, saved where the calling task says. Code goes onto the branch the calling task works on. Any other artifact whose task names no place goes in a fresh `mktemp -d` outside the project. Give a short synthesis note in your reply rather than as a file, naming the base and why, the cross-judge's verdict, each graft with its source candidate, what was rejected and why, any convergence or dropouts, and the verification result. +One synthesized artifact, saved where the calling task says. Code the calling task ships goes onto the branch it works on. Any other artifact whose task names no place goes in a fresh `mktemp -d` outside the project. Give a short synthesis note in your reply rather than as a file, naming the base and why, the cross-judge's verdict, each graft with its source candidate, what was rejected and why, any convergence or dropouts, and the verification result. From 788e7cd605cce76a47ed04e95d66f5f30d5f4f1a Mon Sep 17 00:00:00 2001 From: mdsmithaustin <15272014+mdsmithaustin@users.noreply.github.com> Date: Mon, 12 Oct 2026 15:31:12 -0600 Subject: [PATCH 07/10] no-mistakes(document): Clarify arena cleanup and synthesis note documentation --- PORTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/PORTING.md b/PORTING.md index 05654bfa..aedf6f5a 100644 --- a/PORTING.md +++ b/PORTING.md @@ -72,7 +72,7 @@ Upstream is the source of truth for the workflows themselves. This file records - A Codex lead waits instead of racing its children (port addition, 2026-10). A census of real poteto-mode sessions since 2026-09-14 found that in 106 of 339 non-blocking Codex spawns (31%), the lead read overlapping files while its child worked. The median was about 127K characters of tool output per window. Claude Code leads raced in 8 of 306 spawns. Claude's lead resumes on a completion notice, while a Codex lead holds its turn. The Codex agent-cap paragraph now has each brief name its scope. The lead stays out of a running child's scope, except for a read that a skill step says runs in parallel with the child, as arena's Phase C says of the Phase D read. It sends new facts to that child, and waits with `wait_agent` only once running children or queued arms own every remaining task. A fan-out still launches in full. - Personas announce themselves (port addition, 2026-10). A persona brief, native or pasted, tells the child to put `persona: ` on its own line in its first reply, unless the persona fixes its own first output. Comment Sicko's fixed first line, `Yes... Ha ha ha... Yes!`, is its signature. Codex encrypts spawn messages, so a pasted briefing was otherwise unprovable from a transcript. Child replies are plaintext. Setup also recommends a project-scope Codex install for wrappers that run Codex with their own `CODEX_HOME`, which never read `~/.codex/agents/`. Upstream has neither. On sync, keep both. - A rejected model is handled in one place, the **pstack-harness** skill: omit it, inherit the session model, and report the substitution. Upstream #422 gave `how`, `why`, `reflect`, `swarm`, `arena`, and `interrogate` their own fallbacks to a role's default, which the port does not carry. On sync, keep each skill's spawn sentence pointing at **Spawn a role**, which owns the rejection rule. -- Arena's scratch fallback is a fresh `mktemp -d` per run (port addition, 2026-10). Upstream's fixed `/tmp/arena-/` fallback let a later run read earlier candidates and the rubric. Phase A step 4 gives each run its own directory and keeps the rubric outside candidate-readable paths. Phase A step 4, Phase E, and Phase F remove the candidate worktrees and their branches and delete the scratch directory when the run ends and before a re-frame and re-run. Outputs puts the synthesized artifact where the calling task says, code the task ships on the branch it works on, and any other artifact with no named place in a fresh `mktemp -d` outside the project, and the synthesis note in the reply. After a run that completes, the next run finds none of its candidate files, `arena-.*` directories, worktrees, branches, or judge scores, and nothing of it in the project. The saved artifact stays where Outputs put it. The text does not cover a run cut off before the cleanup, as in the Claude Code headless turn end and the Grok Build 2400 s cap recorded in `evals/promises/results/2026-10-08-sync-df58112.md`, runs that happen at the same time, a directory left by an older copy of the skill, or a candidate that searches the temp directory and finds a saved artifact there. Stronger isolation needs a sandbox per candidate or per run (#209). `tools/skill-content-conventions.json` retires the fixed path. Keep it on sync and offer it upstream. +- Arena's scratch fallback is a fresh `mktemp -d` per run (port addition, 2026-10). Upstream's fixed `/tmp/arena-/` fallback let a later run read earlier candidates and the rubric. Phase A step 4 gives each run its own directory and keeps the rubric outside candidate-readable paths. Phase A step 4, Phase E, and Phase F remove the candidate worktrees and their branches and delete the scratch directory when the run ends and before a re-frame and re-run. Outputs puts the synthesized artifact where the calling task says, code the task ships on the branch it works on, and any other artifact with no named place in a fresh `mktemp -d` outside the project. The synthesis note goes in the reply. After a run that completes, the next run finds none of its candidate files, `arena-.*` directories, worktrees, branches, or judge scores, and nothing of it in the project beyond the code the task ships. The saved artifact stays where Outputs put it. The text does not cover a run cut off before the cleanup, as in the Claude Code headless turn end and the Grok Build 2400 s cap recorded in `evals/promises/results/2026-10-08-sync-df58112.md`, runs that happen at the same time, a directory left by an older copy of the skill, or a candidate that searches the temp directory and finds a saved artifact there. Stronger isolation needs a sandbox per candidate or per run (#209). `tools/skill-content-conventions.json` retires the fixed path. Keep it on sync and offer it upstream. - The models config is layered: a workspace `.agents/pstack-models.md` sits over `~/.agents/pstack-models.md`, and a harness section beats a flat line in either file; setup-pstack preserves values it cannot validate (another harness's choices) and has a write-fallback ladder (shell tool → workspace file → print) for harnesses that cannot write to the home directory. - The lower-ranked gaps from the same mechanism audit that still apply fail closed too (port addition, 2026-10, #120). The `/goal` gap no longer applies, because the 23e4138 sync removed `/goal` from the playbooks. `why` marks a PR it could not read through `gh` as unread. Eval grades only code shape when a transcript is unreadable. Orchestrate dispatches nothing new while `orch` cannot run. When `git worktree add` fails, Opening a PR branches off main in a current checkout with no unrelated changes, never committing to main. In an ungated repository, when the run has no built-in PR tool and neither forge CLI works, it reports a pushed branch with no PR. The persona contract never sends a generic delegate without the briefing. show-me-your-work says when the audit could not read the transcript. The GitNexus sites also need the CLI on `PATH`. Without `node`, a plan's hand-back says the check script did not run. A failed control-skill read leaves the surface with no control skill. Autopilot-stack and the multi-phase plan keep a tick running when the trunk fetch fails. On sync, keep these lines. From 19b6bf6ff1f8e2c5ed535bf947f6a8048c7b1613 Mon Sep 17 00:00:00 2001 From: mdsmithaustin <15272014+mdsmithaustin@users.noreply.github.com> Date: Mon, 12 Oct 2026 19:05:27 -0600 Subject: [PATCH 08/10] fix(arena): make candidate worktrees real git worktrees on every harness Grok Build's isolation: worktree makes a standalone clone under ~/.grok/worktrees/, which git worktree list omits and git worktree remove refuses with exit 128, so arena's cleanup left every Grok candidate behind (grok arena-run relay-go6dzbwe left five). Arena step 4 now counts a native isolation option only when git rev-parse --git-common-dir on its path prints the project's. Every other candidate worktree is made with git worktree add --detach inside the run's mktemp -d, so cleanup is the same on every harness. pstack-harness states the same rule and corrects its Grok Build row. Arena's Outputs keeps a sketch awaiting sign-off outside the project (#214 item 2). PORTING's arena note now holds on all four harnesses. --- PORTING.md | 2 +- skills/arena/SKILL.md | 4 ++-- skills/pstack-harness/SKILL.md | 4 ++-- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/PORTING.md b/PORTING.md index aedf6f5a..d3608859 100644 --- a/PORTING.md +++ b/PORTING.md @@ -72,7 +72,7 @@ Upstream is the source of truth for the workflows themselves. This file records - A Codex lead waits instead of racing its children (port addition, 2026-10). A census of real poteto-mode sessions since 2026-09-14 found that in 106 of 339 non-blocking Codex spawns (31%), the lead read overlapping files while its child worked. The median was about 127K characters of tool output per window. Claude Code leads raced in 8 of 306 spawns. Claude's lead resumes on a completion notice, while a Codex lead holds its turn. The Codex agent-cap paragraph now has each brief name its scope. The lead stays out of a running child's scope, except for a read that a skill step says runs in parallel with the child, as arena's Phase C says of the Phase D read. It sends new facts to that child, and waits with `wait_agent` only once running children or queued arms own every remaining task. A fan-out still launches in full. - Personas announce themselves (port addition, 2026-10). A persona brief, native or pasted, tells the child to put `persona: ` on its own line in its first reply, unless the persona fixes its own first output. Comment Sicko's fixed first line, `Yes... Ha ha ha... Yes!`, is its signature. Codex encrypts spawn messages, so a pasted briefing was otherwise unprovable from a transcript. Child replies are plaintext. Setup also recommends a project-scope Codex install for wrappers that run Codex with their own `CODEX_HOME`, which never read `~/.codex/agents/`. Upstream has neither. On sync, keep both. - A rejected model is handled in one place, the **pstack-harness** skill: omit it, inherit the session model, and report the substitution. Upstream #422 gave `how`, `why`, `reflect`, `swarm`, `arena`, and `interrogate` their own fallbacks to a role's default, which the port does not carry. On sync, keep each skill's spawn sentence pointing at **Spawn a role**, which owns the rejection rule. -- Arena's scratch fallback is a fresh `mktemp -d` per run (port addition, 2026-10). Upstream's fixed `/tmp/arena-/` fallback let a later run read earlier candidates and the rubric. Phase A step 4 gives each run its own directory and keeps the rubric outside candidate-readable paths. Phase A step 4, Phase E, and Phase F remove the candidate worktrees and their branches and delete the scratch directory when the run ends and before a re-frame and re-run. Outputs puts the synthesized artifact where the calling task says, code the task ships on the branch it works on, and any other artifact with no named place in a fresh `mktemp -d` outside the project. The synthesis note goes in the reply. After a run that completes, the next run finds none of its candidate files, `arena-.*` directories, worktrees, branches, or judge scores, and nothing of it in the project beyond the code the task ships. The saved artifact stays where Outputs put it. The text does not cover a run cut off before the cleanup, as in the Claude Code headless turn end and the Grok Build 2400 s cap recorded in `evals/promises/results/2026-10-08-sync-df58112.md`, runs that happen at the same time, a directory left by an older copy of the skill, or a candidate that searches the temp directory and finds a saved artifact there. Stronger isolation needs a sandbox per candidate or per run (#209). `tools/skill-content-conventions.json` retires the fixed path. Keep it on sync and offer it upstream. +- Arena's scratch fallback is a fresh `mktemp -d` per run (port addition, 2026-10). Upstream's fixed `/tmp/arena-/` fallback let a later run read earlier candidates and the rubric. Phase A step 4 gives each run its own directory and keeps the rubric outside candidate-readable paths. Every candidate worktree is a git worktree of the project, so `git worktree remove` deletes it on every harness. Step 4 counts a native isolation option only when it makes one, as Claude Code's `isolation: "worktree"` does. Grok Build's `isolation: worktree` makes a standalone clone under `~/.grok/worktrees/` that `git worktree remove` refuses (grok arena-run relay-go6dzbwe left five, and `grok worktree create` on Grok 1.0.52 makes the same kind). There, and on any harness with no native option that makes one, the lead runs `git worktree add --detach` in the run's `mktemp -d`. Where `git worktree add` fails, as in Codex's `workspace-write` sandbox, the candidates get plain directories there. Phase A step 4, Phase E, and Phase F remove the candidate worktrees and their branches and delete the scratch directory when the run ends and before a re-frame and re-run. Outputs puts the synthesized artifact where the calling task says, code the task ships on the branch it works on, and any other artifact with no named place in a fresh `mktemp -d` outside the project. A sketch awaiting the caller's sign-off stays outside the project until sign-off. The synthesis note goes in the reply. After a run that completes, the next run finds none of its candidate files, `arena-.*` directories, worktrees, branches, or judge scores, and nothing of it in the project beyond the code the task ships. The saved artifact stays where Outputs put it. The text does not cover a run cut off before the cleanup, as in the Claude Code headless turn end and the Grok Build 2400 s cap recorded in `evals/promises/results/2026-10-08-sync-df58112.md`, runs that happen at the same time, a directory left by an older copy of the skill, or a candidate that searches the temp directory and finds a saved artifact there. Stronger isolation needs a sandbox per candidate or per run (#209). `tools/skill-content-conventions.json` retires the fixed path. Keep it on sync and offer it upstream. - The models config is layered: a workspace `.agents/pstack-models.md` sits over `~/.agents/pstack-models.md`, and a harness section beats a flat line in either file; setup-pstack preserves values it cannot validate (another harness's choices) and has a write-fallback ladder (shell tool → workspace file → print) for harnesses that cannot write to the home directory. - The lower-ranked gaps from the same mechanism audit that still apply fail closed too (port addition, 2026-10, #120). The `/goal` gap no longer applies, because the 23e4138 sync removed `/goal` from the playbooks. `why` marks a PR it could not read through `gh` as unread. Eval grades only code shape when a transcript is unreadable. Orchestrate dispatches nothing new while `orch` cannot run. When `git worktree add` fails, Opening a PR branches off main in a current checkout with no unrelated changes, never committing to main. In an ungated repository, when the run has no built-in PR tool and neither forge CLI works, it reports a pushed branch with no PR. The persona contract never sends a generic delegate without the briefing. show-me-your-work says when the audit could not read the transcript. The GitNexus sites also need the CLI on `PATH`. Without `node`, a plan's hand-back says the check script did not run. A failed control-skill read leaves the surface with no control skill. Autopilot-stack and the multi-phase plan keep a tick running when the trunk fetch fails. On sync, keep these lines. diff --git a/skills/arena/SKILL.md b/skills/arena/SKILL.md index e79bdf81..b64228ae 100644 --- a/skills/arena/SKILL.md +++ b/skills/arena/SKILL.md @@ -14,7 +14,7 @@ Every candidate gets the same prompt, so the prompt is the contract. 1. State the artifact each candidate is producing. 2. Derive the rubric, 3-6 concrete gradeable criteria for what success looks like on *this* task. For subjective criteria, use [Eval's criterion and calibration guidance](../poteto-mode/playbooks/eval.md). It is the picker's tool in Phase D. Candidates only see the task. 3. Pick the runners. Use the `arena runners` role, one arm per entry. Its default is one each on `fable`, `opus`, `sonnet`. A rejected entry also follows the required-arm rule below. Spawn more runners when the arena covers multiple design directions. Use the same model N times when the work is generation-bound rather than judgment-sensitive. Use the **pstack-harness** skill for the spawn. Its **Spawn a role** section resolves the role and builds this CLI's spawn call. -4. Give each candidate its own output location (a git worktree where possible, otherwise `candidate-/` inside a fresh `mktemp -d "${TMPDIR:-/tmp}/arena-.XXXXXX"` made for this run), per the **separate-before-serializing-shared-state** principle skill. Never reuse an earlier run's directory. Clean up the run when it ends, after saving its outputs per Outputs, and before a re-frame and re-run starts, whether Phase E or Phase F sent you back. To clean up, remove each candidate worktree with `git worktree remove --force `, delete any branch made for it with `git branch -D`, and delete the scratch directory. Going back to Phase E to graft keeps them. Keep the rubric outside every directory a candidate can read, and give it to the cross-judge in its prompt. +4. Give each candidate its own output location (a git worktree where possible, otherwise `candidate-/` inside a fresh `mktemp -d "${TMPDIR:-/tmp}/arena-.XXXXXX"` made for this run), per the **separate-before-serializing-shared-state** principle skill. Never reuse an earlier run's directory. A native isolation option counts only when it makes a git worktree, where `git -C rev-parse --path-format=absolute --git-common-dir` prints the project's. Grok Build's `isolation: worktree` makes a standalone clone, so spawn there without it. Make every other candidate worktree yourself with `git worktree add --detach /candidate-`, where `` is that fresh `mktemp -d`, and name the path in the candidate's brief. Clean up the run when it ends, after saving its outputs per Outputs, and before a re-frame and re-run starts, whether Phase E or Phase F sent you back. To clean up, remove each candidate worktree with `git worktree remove --force `, delete any branch made for it with `git branch -D`, and delete the scratch directory. Going back to Phase E to graft keeps them. Keep the rubric outside every directory a candidate can read, and give it to the cross-judge in its prompt. ## Phase B: Fan out @@ -48,4 +48,4 @@ Verify the synthesized artifact like any other output, per the **prove-it-works* ## Outputs -One synthesized artifact, saved where the calling task says. Code the calling task ships goes onto the branch it works on. Any other artifact whose task names no place goes in a fresh `mktemp -d` outside the project. Give a short synthesis note in your reply rather than as a file, naming the base and why, the cross-judge's verdict, each graft with its source candidate, what was rejected and why, any convergence or dropouts, and the verification result. +One synthesized artifact, saved where the calling task says. Code the calling task ships goes onto the branch it works on. A sketch awaiting the caller's sign-off is not yet code the task ships, so it stays outside the project until sign-off. Any other artifact whose task names no place goes in a fresh `mktemp -d` outside the project. Give a short synthesis note in your reply rather than as a file, naming the base and why, the cross-judge's verdict, each graft with its source candidate, what was rejected and why, any convergence or dropouts, and the verification result. diff --git a/skills/pstack-harness/SKILL.md b/skills/pstack-harness/SKILL.md index 0f459c85..5b883049 100644 --- a/skills/pstack-harness/SKILL.md +++ b/skills/pstack-harness/SKILL.md @@ -56,7 +56,7 @@ Every pstack spawn runs a role from the models config, such as `feature` or `how 2. No such tool → invoke your own CLI non-interactively as a subprocess (its help names the command and flags), one invocation per arm, run concurrently in background shells, each arm's report collected from stdout or a file path named in its brief. 3. No subprocesses either → run the arms sequentially inline, one at a time, each writing its report to a file before the next starts, then synthesize. Keep the configured arm count. A verdict or review that runs inline is not independent. Label it `self-review` in the verdict and in the reply, and never count it as an independent verdict. -Each writer gets its own git worktree, whichever mechanism spawns it. Keep the location a native isolation option picks. When you run `git worktree add` yourself, never place the worktree beside the repository: +Each writer gets its own git worktree, whichever mechanism spawns it. Keep the location a native isolation option picks when the option makes a git worktree, that is, when `git -C rev-parse --path-format=absolute --git-common-dir` prints the project's. Grok Build's `isolation: worktree` makes a standalone clone instead, so a writer there that needs a git worktree gets one from `git worktree add`. When you run `git worktree add` yourself, never place the worktree beside the repository: - A throwaway checkout (verify, review, replay, eval arm) goes under `mktemp -d "${TMPDIR:-/tmp}/pstack-.XXXXXX"`. Remove it with `git worktree remove` when its run ends. - A worktree that holds unpushed work goes under `.worktrees/` in the main checkout. If the repository does not ignore `.worktrees/`, append it to `"$(git rev-parse --git-common-dir)/info/exclude"` first. @@ -92,7 +92,7 @@ Observed circa 2026-09. Treat as starting points, not contracts — verify again | Claude Code | `Task` tool (custom agents from `.claude/agents/` spawn by name); `model` takes short aliases | Prefer any exposed structured task-tracking capability; otherwise use normal progress updates. When the task tools are off, setup-pstack step 0b names the switch. With tool search on, the default, they arrive deferred, so the discovery step in **Track a worklist** applies | no per-call field; a `pstack-effort-` agent type carries it (see **Spawn a role**). An `inherit-parent` effort, or a session whose catalog lacks the agent, runs at the session effort (`--effort`, `effortLevel`) | JSONL under `~/.claude/projects//`, `` = the resolved workspace path with every character other than a letter or digit turned into `-` | | Codex | `spawn_agent` (the multi-agent feature, stable in 0.152) takes a model and a reasoning effort per spawn, with the `fork_turns` rule in **Spawn a role**; custom agents use TOML in `~/.codex/agents/` or `.codex/agents/` with `name`, `description`, and `developer_instructions`; generated pstack personas leave model and effort unset; `codex exec` is the subprocess route | Prefer any exposed structured plan-tracking capability; otherwise use normal progress updates in the commentary channel. Goal lifecycle tools are not work tracking unless the user explicitly asked to create a goal. When the plan tool is off, setup-pstack step 0b names the switch | the reasoning-effort field on `spawn_agent`; `-c model_reasoning_effort=` on `codex exec`. A model set without an effort gets that model's default effort (low on `gpt-6.1-sol`, medium on the other gpt-6 models), not the parent's, so always pass one | JSONL under `~/.codex/sessions/` by date | | Hermes | a delegation toolset when enabled; `hermes -z` for one-shot subprocess runs | Prefer any exposed structured task-planning capability; otherwise use normal progress updates | `--reasoning ` on `hermes -z`; config `agent.reasoning_effort` and per-model `agent.reasoning_overrides`. Whether the delegation toolset takes an effort field is unverified: check its schema in session | SQLite store; `hermes sessions` subcommands list and export | -| Grok Build | `spawn_subagent`. On 1.0.41 its schema has no agent type and no argument names a persona, so pass a pstack persona's full briefing in `prompt`, prefix `description` with that persona's name in brackets, such as `[poteto-agent]` or `[comment-sicko]`, so the subagent label shows it, and on `resume_from` keep the tag but skip the briefing because the child already has it. Grok loads the `~/.claude/agents/` wrappers as agent types, and its source honors a `subagent_type` key that arrives, but the schema hides the key and a model told to send it dropped it in a 2026-09 probe, so do not rely on naming the type. A `model` argument appears unless subagent model inheritance is on (default off, settable remotely) and every catalog model is xAI. Only the top-level session spawns, so a delegate cannot fan out. `isolation: worktree` makes a full clone under `~/.grok/worktrees/`, and `grok worktree gc --max-age ` reclaims it. `grok -p` is the subprocess route | Prefer any exposed structured task-tracking capability; otherwise use normal progress updates | no per-spawn field; `reasoning_effort` on a Grok role or persona, otherwise the session effort, so the effort is `inherit-parent` here. `--effort ` on `grok -p` | per-session directories under `~/.grok/sessions///`; each spawn records `subagents//meta.json` | +| Grok Build | `spawn_subagent`. On 1.0.41 its schema has no agent type and no argument names a persona, so pass a pstack persona's full briefing in `prompt`, prefix `description` with that persona's name in brackets, such as `[poteto-agent]` or `[comment-sicko]`, so the subagent label shows it, and on `resume_from` keep the tag but skip the briefing because the child already has it. Grok loads the `~/.claude/agents/` wrappers as agent types, and its source honors a `subagent_type` key that arrives, but the schema hides the key and a model told to send it dropped it in a 2026-09 probe, so do not rely on naming the type. A `model` argument appears unless subagent model inheritance is on (default off, settable remotely) and every catalog model is xAI. Only the top-level session spawns, so a delegate cannot fan out. `isolation: worktree` makes a standalone clone under `~/.grok/worktrees/`, not a git worktree, so `git worktree list` omits it and `git worktree remove` refuses it. `grok worktree gc --max-age ` reclaims it. A writer that needs a git worktree gets one from `git worktree add`. `grok -p` is the subprocess route | Prefer any exposed structured task-tracking capability; otherwise use normal progress updates | no per-spawn field; `reasoning_effort` on a Grok role or persona, otherwise the session effort, so the effort is `inherit-parent` here. `--effort ` on `grok -p` | per-session directories under `~/.grok/sessions///`; each spawn records `subagents//meta.json` | **Codex agent cap.** Codex caps subagents, and a spawn past the cap fails with `agent thread limit reached` and creates nothing. If your tools include `close_agent`, by default at most 6 agents stay open per session and a finished agent holds its slot until closed, so launch at most 6 at once and close each agent once you have read its result. If they do not, by default at most 3 subagents run at once and finished ones free their slots, so launch parallel arms in waves of 3 or fewer. Either limit overrides a skill's "spawn all in one message". The arm count stays the same either way. After a limit error, never wait for the refused arm: spawn it when a slot frees. Run it yourself only when the arm needs no other model, and report it as a substitution. Without `close_agent`, `wait_agent` takes no target and blocks until its timeout when no child is running, so call it only while a spawned child is still working. Name each child's scope in its brief: the question and the files or areas it owns. Do not read or search a scope that a running child owns, unless a skill step says that read runs in parallel with the child, as arena's Phase C says of the Phase D read. When a new fact changes a child's question, send the fact to that child. Wait with `wait_agent` when each remaining task belongs either to a running child or to an arm that waits for a free slot. Spawn each waiting arm as soon as a slot frees. After a child reports, spot-check its claims. From 2b988ec22eb8510c2fbce4c5b9f9565b1ff81261 Mon Sep 17 00:00:00 2001 From: mdsmithaustin <15272014+mdsmithaustin@users.noreply.github.com> Date: Mon, 12 Oct 2026 20:07:10 -0600 Subject: [PATCH 09/10] no-mistakes(test): Remove contradicted Grok CLI checkout claim --- PORTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/PORTING.md b/PORTING.md index d3608859..57c538d9 100644 --- a/PORTING.md +++ b/PORTING.md @@ -72,7 +72,7 @@ Upstream is the source of truth for the workflows themselves. This file records - A Codex lead waits instead of racing its children (port addition, 2026-10). A census of real poteto-mode sessions since 2026-09-14 found that in 106 of 339 non-blocking Codex spawns (31%), the lead read overlapping files while its child worked. The median was about 127K characters of tool output per window. Claude Code leads raced in 8 of 306 spawns. Claude's lead resumes on a completion notice, while a Codex lead holds its turn. The Codex agent-cap paragraph now has each brief name its scope. The lead stays out of a running child's scope, except for a read that a skill step says runs in parallel with the child, as arena's Phase C says of the Phase D read. It sends new facts to that child, and waits with `wait_agent` only once running children or queued arms own every remaining task. A fan-out still launches in full. - Personas announce themselves (port addition, 2026-10). A persona brief, native or pasted, tells the child to put `persona: ` on its own line in its first reply, unless the persona fixes its own first output. Comment Sicko's fixed first line, `Yes... Ha ha ha... Yes!`, is its signature. Codex encrypts spawn messages, so a pasted briefing was otherwise unprovable from a transcript. Child replies are plaintext. Setup also recommends a project-scope Codex install for wrappers that run Codex with their own `CODEX_HOME`, which never read `~/.codex/agents/`. Upstream has neither. On sync, keep both. - A rejected model is handled in one place, the **pstack-harness** skill: omit it, inherit the session model, and report the substitution. Upstream #422 gave `how`, `why`, `reflect`, `swarm`, `arena`, and `interrogate` their own fallbacks to a role's default, which the port does not carry. On sync, keep each skill's spawn sentence pointing at **Spawn a role**, which owns the rejection rule. -- Arena's scratch fallback is a fresh `mktemp -d` per run (port addition, 2026-10). Upstream's fixed `/tmp/arena-/` fallback let a later run read earlier candidates and the rubric. Phase A step 4 gives each run its own directory and keeps the rubric outside candidate-readable paths. Every candidate worktree is a git worktree of the project, so `git worktree remove` deletes it on every harness. Step 4 counts a native isolation option only when it makes one, as Claude Code's `isolation: "worktree"` does. Grok Build's `isolation: worktree` makes a standalone clone under `~/.grok/worktrees/` that `git worktree remove` refuses (grok arena-run relay-go6dzbwe left five, and `grok worktree create` on Grok 1.0.52 makes the same kind). There, and on any harness with no native option that makes one, the lead runs `git worktree add --detach` in the run's `mktemp -d`. Where `git worktree add` fails, as in Codex's `workspace-write` sandbox, the candidates get plain directories there. Phase A step 4, Phase E, and Phase F remove the candidate worktrees and their branches and delete the scratch directory when the run ends and before a re-frame and re-run. Outputs puts the synthesized artifact where the calling task says, code the task ships on the branch it works on, and any other artifact with no named place in a fresh `mktemp -d` outside the project. A sketch awaiting the caller's sign-off stays outside the project until sign-off. The synthesis note goes in the reply. After a run that completes, the next run finds none of its candidate files, `arena-.*` directories, worktrees, branches, or judge scores, and nothing of it in the project beyond the code the task ships. The saved artifact stays where Outputs put it. The text does not cover a run cut off before the cleanup, as in the Claude Code headless turn end and the Grok Build 2400 s cap recorded in `evals/promises/results/2026-10-08-sync-df58112.md`, runs that happen at the same time, a directory left by an older copy of the skill, or a candidate that searches the temp directory and finds a saved artifact there. Stronger isolation needs a sandbox per candidate or per run (#209). `tools/skill-content-conventions.json` retires the fixed path. Keep it on sync and offer it upstream. +- Arena's scratch fallback is a fresh `mktemp -d` per run (port addition, 2026-10). Upstream's fixed `/tmp/arena-/` fallback let a later run read earlier candidates and the rubric. Phase A step 4 gives each run its own directory and keeps the rubric outside candidate-readable paths. Every candidate worktree is a git worktree of the project, so `git worktree remove` deletes it on every harness. Step 4 counts a native isolation option only when it makes one, as Claude Code's `isolation: "worktree"` does. Grok Build's `isolation: worktree` makes a standalone clone under `~/.grok/worktrees/` that `git worktree remove` refuses (grok arena-run relay-go6dzbwe left five). There, and on any harness with no native option that makes one, the lead runs `git worktree add --detach` in the run's `mktemp -d`. Where `git worktree add` fails, as in Codex's `workspace-write` sandbox, the candidates get plain directories there. Phase A step 4, Phase E, and Phase F remove the candidate worktrees and their branches and delete the scratch directory when the run ends and before a re-frame and re-run. Outputs puts the synthesized artifact where the calling task says, code the task ships on the branch it works on, and any other artifact with no named place in a fresh `mktemp -d` outside the project. A sketch awaiting the caller's sign-off stays outside the project until sign-off. The synthesis note goes in the reply. After a run that completes, the next run finds none of its candidate files, `arena-.*` directories, worktrees, branches, or judge scores, and nothing of it in the project beyond the code the task ships. The saved artifact stays where Outputs put it. The text does not cover a run cut off before the cleanup, as in the Claude Code headless turn end and the Grok Build 2400 s cap recorded in `evals/promises/results/2026-10-08-sync-df58112.md`, runs that happen at the same time, a directory left by an older copy of the skill, or a candidate that searches the temp directory and finds a saved artifact there. Stronger isolation needs a sandbox per candidate or per run (#209). `tools/skill-content-conventions.json` retires the fixed path. Keep it on sync and offer it upstream. - The models config is layered: a workspace `.agents/pstack-models.md` sits over `~/.agents/pstack-models.md`, and a harness section beats a flat line in either file; setup-pstack preserves values it cannot validate (another harness's choices) and has a write-fallback ladder (shell tool → workspace file → print) for harnesses that cannot write to the home directory. - The lower-ranked gaps from the same mechanism audit that still apply fail closed too (port addition, 2026-10, #120). The `/goal` gap no longer applies, because the 23e4138 sync removed `/goal` from the playbooks. `why` marks a PR it could not read through `gh` as unread. Eval grades only code shape when a transcript is unreadable. Orchestrate dispatches nothing new while `orch` cannot run. When `git worktree add` fails, Opening a PR branches off main in a current checkout with no unrelated changes, never committing to main. In an ungated repository, when the run has no built-in PR tool and neither forge CLI works, it reports a pushed branch with no PR. The persona contract never sends a generic delegate without the briefing. show-me-your-work says when the audit could not read the transcript. The GitNexus sites also need the CLI on `PATH`. Without `node`, a plan's hand-back says the check script did not run. A failed control-skill read leaves the surface with no control skill. Autopilot-stack and the multi-phase plan keep a tick running when the trunk fetch fails. On sync, keep these lines. From 6456188818e1b2f7486a5ca8af6817c8617de4f6 Mon Sep 17 00:00:00 2001 From: mdsmithaustin <15272014+mdsmithaustin@users.noreply.github.com> Date: Mon, 12 Oct 2026 20:48:12 -0600 Subject: [PATCH 10/10] no-mistakes(test): Qualify Grok isolation claims and remove validation artifacts --- PORTING.md | 2 +- skills/arena/SKILL.md | 2 +- skills/pstack-harness/SKILL.md | 4 ++-- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/PORTING.md b/PORTING.md index 57c538d9..e64dd84b 100644 --- a/PORTING.md +++ b/PORTING.md @@ -72,7 +72,7 @@ Upstream is the source of truth for the workflows themselves. This file records - A Codex lead waits instead of racing its children (port addition, 2026-10). A census of real poteto-mode sessions since 2026-09-14 found that in 106 of 339 non-blocking Codex spawns (31%), the lead read overlapping files while its child worked. The median was about 127K characters of tool output per window. Claude Code leads raced in 8 of 306 spawns. Claude's lead resumes on a completion notice, while a Codex lead holds its turn. The Codex agent-cap paragraph now has each brief name its scope. The lead stays out of a running child's scope, except for a read that a skill step says runs in parallel with the child, as arena's Phase C says of the Phase D read. It sends new facts to that child, and waits with `wait_agent` only once running children or queued arms own every remaining task. A fan-out still launches in full. - Personas announce themselves (port addition, 2026-10). A persona brief, native or pasted, tells the child to put `persona: ` on its own line in its first reply, unless the persona fixes its own first output. Comment Sicko's fixed first line, `Yes... Ha ha ha... Yes!`, is its signature. Codex encrypts spawn messages, so a pasted briefing was otherwise unprovable from a transcript. Child replies are plaintext. Setup also recommends a project-scope Codex install for wrappers that run Codex with their own `CODEX_HOME`, which never read `~/.codex/agents/`. Upstream has neither. On sync, keep both. - A rejected model is handled in one place, the **pstack-harness** skill: omit it, inherit the session model, and report the substitution. Upstream #422 gave `how`, `why`, `reflect`, `swarm`, `arena`, and `interrogate` their own fallbacks to a role's default, which the port does not carry. On sync, keep each skill's spawn sentence pointing at **Spawn a role**, which owns the rejection rule. -- Arena's scratch fallback is a fresh `mktemp -d` per run (port addition, 2026-10). Upstream's fixed `/tmp/arena-/` fallback let a later run read earlier candidates and the rubric. Phase A step 4 gives each run its own directory and keeps the rubric outside candidate-readable paths. Every candidate worktree is a git worktree of the project, so `git worktree remove` deletes it on every harness. Step 4 counts a native isolation option only when it makes one, as Claude Code's `isolation: "worktree"` does. Grok Build's `isolation: worktree` makes a standalone clone under `~/.grok/worktrees/` that `git worktree remove` refuses (grok arena-run relay-go6dzbwe left five). There, and on any harness with no native option that makes one, the lead runs `git worktree add --detach` in the run's `mktemp -d`. Where `git worktree add` fails, as in Codex's `workspace-write` sandbox, the candidates get plain directories there. Phase A step 4, Phase E, and Phase F remove the candidate worktrees and their branches and delete the scratch directory when the run ends and before a re-frame and re-run. Outputs puts the synthesized artifact where the calling task says, code the task ships on the branch it works on, and any other artifact with no named place in a fresh `mktemp -d` outside the project. A sketch awaiting the caller's sign-off stays outside the project until sign-off. The synthesis note goes in the reply. After a run that completes, the next run finds none of its candidate files, `arena-.*` directories, worktrees, branches, or judge scores, and nothing of it in the project beyond the code the task ships. The saved artifact stays where Outputs put it. The text does not cover a run cut off before the cleanup, as in the Claude Code headless turn end and the Grok Build 2400 s cap recorded in `evals/promises/results/2026-10-08-sync-df58112.md`, runs that happen at the same time, a directory left by an older copy of the skill, or a candidate that searches the temp directory and finds a saved artifact there. Stronger isolation needs a sandbox per candidate or per run (#209). `tools/skill-content-conventions.json` retires the fixed path. Keep it on sync and offer it upstream. +- Arena's scratch fallback is a fresh `mktemp -d` per run (port addition, 2026-10). Upstream's fixed `/tmp/arena-/` fallback let a later run read earlier candidates and the rubric. Phase A step 4 gives each run its own directory and keeps the rubric outside candidate-readable paths. Every candidate worktree is a git worktree of the project, so `git worktree remove` deletes it on every harness. Step 4 counts a native isolation option only when it makes one, as Claude Code's `isolation: "worktree"` does. Grok Build's `isolation: worktree` makes a standalone clone under `~/.grok/worktrees/` when `cli.worktree_type` is `standalone` (Grok can set it remotely), and a linked git worktree otherwise. `git worktree remove` refuses a standalone clone; grok arena-run relay-go6dzbwe's account had `worktree_type` set to `standalone` and left five clones. There, and on any harness with no native option that makes one, the lead runs `git worktree add --detach` in the run's `mktemp -d`. Where `git worktree add` fails, as in Codex's `workspace-write` sandbox, the candidates get plain directories there. Phase A step 4, Phase E, and Phase F remove the candidate worktrees and their branches and delete the scratch directory when the run ends and before a re-frame and re-run. Outputs puts the synthesized artifact where the calling task says, code the task ships on the branch it works on, and any other artifact with no named place in a fresh `mktemp -d` outside the project. A sketch awaiting the caller's sign-off stays outside the project until sign-off. The synthesis note goes in the reply. After a run that completes, the next run finds none of its candidate files, `arena-.*` directories, worktrees, branches, or judge scores, and nothing of it in the project beyond the code the task ships. The saved artifact stays where Outputs put it. The text does not cover a run cut off before the cleanup, as in the Claude Code headless turn end and the Grok Build 2400 s cap recorded in `evals/promises/results/2026-10-08-sync-df58112.md`, runs that happen at the same time, a directory left by an older copy of the skill, or a candidate that searches the temp directory and finds a saved artifact there. Stronger isolation needs a sandbox per candidate or per run (#209). `tools/skill-content-conventions.json` retires the fixed path. Keep it on sync and offer it upstream. - The models config is layered: a workspace `.agents/pstack-models.md` sits over `~/.agents/pstack-models.md`, and a harness section beats a flat line in either file; setup-pstack preserves values it cannot validate (another harness's choices) and has a write-fallback ladder (shell tool → workspace file → print) for harnesses that cannot write to the home directory. - The lower-ranked gaps from the same mechanism audit that still apply fail closed too (port addition, 2026-10, #120). The `/goal` gap no longer applies, because the 23e4138 sync removed `/goal` from the playbooks. `why` marks a PR it could not read through `gh` as unread. Eval grades only code shape when a transcript is unreadable. Orchestrate dispatches nothing new while `orch` cannot run. When `git worktree add` fails, Opening a PR branches off main in a current checkout with no unrelated changes, never committing to main. In an ungated repository, when the run has no built-in PR tool and neither forge CLI works, it reports a pushed branch with no PR. The persona contract never sends a generic delegate without the briefing. show-me-your-work says when the audit could not read the transcript. The GitNexus sites also need the CLI on `PATH`. Without `node`, a plan's hand-back says the check script did not run. A failed control-skill read leaves the surface with no control skill. Autopilot-stack and the multi-phase plan keep a tick running when the trunk fetch fails. On sync, keep these lines. diff --git a/skills/arena/SKILL.md b/skills/arena/SKILL.md index b64228ae..7880b67f 100644 --- a/skills/arena/SKILL.md +++ b/skills/arena/SKILL.md @@ -14,7 +14,7 @@ Every candidate gets the same prompt, so the prompt is the contract. 1. State the artifact each candidate is producing. 2. Derive the rubric, 3-6 concrete gradeable criteria for what success looks like on *this* task. For subjective criteria, use [Eval's criterion and calibration guidance](../poteto-mode/playbooks/eval.md). It is the picker's tool in Phase D. Candidates only see the task. 3. Pick the runners. Use the `arena runners` role, one arm per entry. Its default is one each on `fable`, `opus`, `sonnet`. A rejected entry also follows the required-arm rule below. Spawn more runners when the arena covers multiple design directions. Use the same model N times when the work is generation-bound rather than judgment-sensitive. Use the **pstack-harness** skill for the spawn. Its **Spawn a role** section resolves the role and builds this CLI's spawn call. -4. Give each candidate its own output location (a git worktree where possible, otherwise `candidate-/` inside a fresh `mktemp -d "${TMPDIR:-/tmp}/arena-.XXXXXX"` made for this run), per the **separate-before-serializing-shared-state** principle skill. Never reuse an earlier run's directory. A native isolation option counts only when it makes a git worktree, where `git -C rev-parse --path-format=absolute --git-common-dir` prints the project's. Grok Build's `isolation: worktree` makes a standalone clone, so spawn there without it. Make every other candidate worktree yourself with `git worktree add --detach /candidate-`, where `` is that fresh `mktemp -d`, and name the path in the candidate's brief. Clean up the run when it ends, after saving its outputs per Outputs, and before a re-frame and re-run starts, whether Phase E or Phase F sent you back. To clean up, remove each candidate worktree with `git worktree remove --force `, delete any branch made for it with `git branch -D`, and delete the scratch directory. Going back to Phase E to graft keeps them. Keep the rubric outside every directory a candidate can read, and give it to the cross-judge in its prompt. +4. Give each candidate its own output location (a git worktree where possible, otherwise `candidate-/` inside a fresh `mktemp -d "${TMPDIR:-/tmp}/arena-.XXXXXX"` made for this run), per the **separate-before-serializing-shared-state** principle skill. Never reuse an earlier run's directory. A native isolation option counts only when it makes a git worktree, where `git -C rev-parse --path-format=absolute --git-common-dir` prints the project's. Grok Build's `isolation: worktree` can make a standalone clone, so spawn there without it. Make every other candidate worktree yourself with `git worktree add --detach /candidate-`, where `` is that fresh `mktemp -d`, and name the path in the candidate's brief. Clean up the run when it ends, after saving its outputs per Outputs, and before a re-frame and re-run starts, whether Phase E or Phase F sent you back. To clean up, remove each candidate worktree with `git worktree remove --force `, delete any branch made for it with `git branch -D`, and delete the scratch directory. Going back to Phase E to graft keeps them. Keep the rubric outside every directory a candidate can read, and give it to the cross-judge in its prompt. ## Phase B: Fan out diff --git a/skills/pstack-harness/SKILL.md b/skills/pstack-harness/SKILL.md index 5b883049..e98152bb 100644 --- a/skills/pstack-harness/SKILL.md +++ b/skills/pstack-harness/SKILL.md @@ -56,7 +56,7 @@ Every pstack spawn runs a role from the models config, such as `feature` or `how 2. No such tool → invoke your own CLI non-interactively as a subprocess (its help names the command and flags), one invocation per arm, run concurrently in background shells, each arm's report collected from stdout or a file path named in its brief. 3. No subprocesses either → run the arms sequentially inline, one at a time, each writing its report to a file before the next starts, then synthesize. Keep the configured arm count. A verdict or review that runs inline is not independent. Label it `self-review` in the verdict and in the reply, and never count it as an independent verdict. -Each writer gets its own git worktree, whichever mechanism spawns it. Keep the location a native isolation option picks when the option makes a git worktree, that is, when `git -C rev-parse --path-format=absolute --git-common-dir` prints the project's. Grok Build's `isolation: worktree` makes a standalone clone instead, so a writer there that needs a git worktree gets one from `git worktree add`. When you run `git worktree add` yourself, never place the worktree beside the repository: +Each writer gets its own git worktree, whichever mechanism spawns it. Keep the location a native isolation option picks when the option makes a git worktree, that is, when `git -C rev-parse --path-format=absolute --git-common-dir` prints the project's. Grok Build's `isolation: worktree` can make a standalone clone instead, so a writer there that needs a git worktree gets one from `git worktree add`. When you run `git worktree add` yourself, never place the worktree beside the repository: - A throwaway checkout (verify, review, replay, eval arm) goes under `mktemp -d "${TMPDIR:-/tmp}/pstack-.XXXXXX"`. Remove it with `git worktree remove` when its run ends. - A worktree that holds unpushed work goes under `.worktrees/` in the main checkout. If the repository does not ignore `.worktrees/`, append it to `"$(git rev-parse --git-common-dir)/info/exclude"` first. @@ -92,7 +92,7 @@ Observed circa 2026-09. Treat as starting points, not contracts — verify again | Claude Code | `Task` tool (custom agents from `.claude/agents/` spawn by name); `model` takes short aliases | Prefer any exposed structured task-tracking capability; otherwise use normal progress updates. When the task tools are off, setup-pstack step 0b names the switch. With tool search on, the default, they arrive deferred, so the discovery step in **Track a worklist** applies | no per-call field; a `pstack-effort-` agent type carries it (see **Spawn a role**). An `inherit-parent` effort, or a session whose catalog lacks the agent, runs at the session effort (`--effort`, `effortLevel`) | JSONL under `~/.claude/projects//`, `` = the resolved workspace path with every character other than a letter or digit turned into `-` | | Codex | `spawn_agent` (the multi-agent feature, stable in 0.152) takes a model and a reasoning effort per spawn, with the `fork_turns` rule in **Spawn a role**; custom agents use TOML in `~/.codex/agents/` or `.codex/agents/` with `name`, `description`, and `developer_instructions`; generated pstack personas leave model and effort unset; `codex exec` is the subprocess route | Prefer any exposed structured plan-tracking capability; otherwise use normal progress updates in the commentary channel. Goal lifecycle tools are not work tracking unless the user explicitly asked to create a goal. When the plan tool is off, setup-pstack step 0b names the switch | the reasoning-effort field on `spawn_agent`; `-c model_reasoning_effort=` on `codex exec`. A model set without an effort gets that model's default effort (low on `gpt-6.1-sol`, medium on the other gpt-6 models), not the parent's, so always pass one | JSONL under `~/.codex/sessions/` by date | | Hermes | a delegation toolset when enabled; `hermes -z` for one-shot subprocess runs | Prefer any exposed structured task-planning capability; otherwise use normal progress updates | `--reasoning ` on `hermes -z`; config `agent.reasoning_effort` and per-model `agent.reasoning_overrides`. Whether the delegation toolset takes an effort field is unverified: check its schema in session | SQLite store; `hermes sessions` subcommands list and export | -| Grok Build | `spawn_subagent`. On 1.0.41 its schema has no agent type and no argument names a persona, so pass a pstack persona's full briefing in `prompt`, prefix `description` with that persona's name in brackets, such as `[poteto-agent]` or `[comment-sicko]`, so the subagent label shows it, and on `resume_from` keep the tag but skip the briefing because the child already has it. Grok loads the `~/.claude/agents/` wrappers as agent types, and its source honors a `subagent_type` key that arrives, but the schema hides the key and a model told to send it dropped it in a 2026-09 probe, so do not rely on naming the type. A `model` argument appears unless subagent model inheritance is on (default off, settable remotely) and every catalog model is xAI. Only the top-level session spawns, so a delegate cannot fan out. `isolation: worktree` makes a standalone clone under `~/.grok/worktrees/`, not a git worktree, so `git worktree list` omits it and `git worktree remove` refuses it. `grok worktree gc --max-age ` reclaims it. A writer that needs a git worktree gets one from `git worktree add`. `grok -p` is the subprocess route | Prefer any exposed structured task-tracking capability; otherwise use normal progress updates | no per-spawn field; `reasoning_effort` on a Grok role or persona, otherwise the session effort, so the effort is `inherit-parent` here. `--effort ` on `grok -p` | per-session directories under `~/.grok/sessions///`; each spawn records `subagents//meta.json` | +| Grok Build | `spawn_subagent`. On 1.0.41 its schema has no agent type and no argument names a persona, so pass a pstack persona's full briefing in `prompt`, prefix `description` with that persona's name in brackets, such as `[poteto-agent]` or `[comment-sicko]`, so the subagent label shows it, and on `resume_from` keep the tag but skip the briefing because the child already has it. Grok loads the `~/.claude/agents/` wrappers as agent types, and its source honors a `subagent_type` key that arrives, but the schema hides the key and a model told to send it dropped it in a 2026-09 probe, so do not rely on naming the type. A `model` argument appears unless subagent model inheritance is on (default off, settable remotely) and every catalog model is xAI. Only the top-level session spawns, so a delegate cannot fan out. `isolation: worktree` makes a standalone clone under `~/.grok/worktrees/` when `cli.worktree_type` is `standalone` (Grok can set it remotely), and a linked git worktree otherwise. For a standalone clone, `git worktree list` omits it and `git worktree remove` refuses it. `grok worktree gc --max-age ` reclaims it. A writer that needs a git worktree gets one from `git worktree add`. `grok -p` is the subprocess route | Prefer any exposed structured task-tracking capability; otherwise use normal progress updates | no per-spawn field; `reasoning_effort` on a Grok role or persona, otherwise the session effort, so the effort is `inherit-parent` here. `--effort ` on `grok -p` | per-session directories under `~/.grok/sessions///`; each spawn records `subagents//meta.json` | **Codex agent cap.** Codex caps subagents, and a spawn past the cap fails with `agent thread limit reached` and creates nothing. If your tools include `close_agent`, by default at most 6 agents stay open per session and a finished agent holds its slot until closed, so launch at most 6 at once and close each agent once you have read its result. If they do not, by default at most 3 subagents run at once and finished ones free their slots, so launch parallel arms in waves of 3 or fewer. Either limit overrides a skill's "spawn all in one message". The arm count stays the same either way. After a limit error, never wait for the refused arm: spawn it when a slot frees. Run it yourself only when the arm needs no other model, and report it as a substitution. Without `close_agent`, `wait_agent` takes no target and blocks until its timeout when no child is running, so call it only while a spawned child is still working. Name each child's scope in its brief: the question and the files or areas it owns. Do not read or search a scope that a running child owns, unless a skill step says that read runs in parallel with the child, as arena's Phase C says of the Phase D read. When a new fact changes a child's question, send the fact to that child. Wait with `wait_agent` when each remaining task belongs either to a running child or to an arm that waits for a free slot. Spawn each waiting arm as soon as a slot frees. After a child reports, spot-check its claims.