Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions PORTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ Upstream is the source of truth for the workflows themselves. This file records
- A Codex lead waits instead of racing its children (port addition, 2026-10). A census of real poteto-mode sessions since 2026-09-14 found that in 106 of 339 non-blocking Codex spawns (31%), the lead read overlapping files while its child worked. The median was about 127K characters of tool output per window. Claude Code leads raced in 8 of 306 spawns. Claude's lead resumes on a completion notice, while a Codex lead holds its turn. The Codex agent-cap paragraph now has each brief name its scope. The lead stays out of a running child's scope, except for a read that a skill step says runs in parallel with the child, as arena's Phase C says of the Phase D read. It sends new facts to that child, and waits with `wait_agent` only once running children or queued arms own every remaining task. A fan-out still launches in full.
- Personas announce themselves (port addition, 2026-10). A persona brief, native or pasted, tells the child to put `persona: <id>` on its own line in its first reply, unless the persona fixes its own first output. Comment Sicko's fixed first line, `Yes... Ha ha ha... Yes!`, is its signature. Codex encrypts spawn messages, so a pasted briefing was otherwise unprovable from a transcript. Child replies are plaintext. Setup also recommends a project-scope Codex install for wrappers that run Codex with their own `CODEX_HOME`, which never read `~/.codex/agents/`. Upstream has neither. On sync, keep both.
- A rejected model is handled in one place, the **pstack-harness** skill: omit it, inherit the session model, and report the substitution. Upstream #422 gave `how`, `why`, `reflect`, `swarm`, `arena`, and `interrogate` their own fallbacks to a role's default, which the port does not carry. On sync, keep each skill's spawn sentence pointing at **Spawn a role**, which owns the rejection rule.
- Arena's scratch fallback is a fresh `mktemp -d` per run (port addition, 2026-10). Upstream's fixed `/tmp/arena-<slug>/` fallback let a later run read earlier candidates and the rubric. Phase A step 4 gives each run its own directory and keeps the rubric outside candidate-readable paths. Every candidate worktree is a git worktree of the project, so `git worktree remove` deletes it on every harness. Step 4 counts a native isolation option only when it makes one, as Claude Code's `isolation: "worktree"` does. Grok Build's `isolation: worktree` makes a standalone clone under `~/.grok/worktrees/` when `cli.worktree_type` is `standalone` (Grok can set it remotely), and a linked git worktree otherwise. `git worktree remove` refuses a standalone clone; grok arena-run relay-go6dzbwe's account had `worktree_type` set to `standalone` and left five clones. There, and on any harness with no native option that makes one, the lead runs `git worktree add --detach` in the run's `mktemp -d`. Where `git worktree add` fails, as in Codex's `workspace-write` sandbox, the candidates get plain directories there. Phase A step 4, Phase E, and Phase F remove the candidate worktrees and their branches and delete the scratch directory when the run ends and before a re-frame and re-run. Outputs puts the synthesized artifact where the calling task says, code the task ships on the branch it works on, and any other artifact with no named place in a fresh `mktemp -d` outside the project. A sketch awaiting the caller's sign-off stays outside the project until sign-off. The synthesis note goes in the reply. After a run that completes, the next run finds none of its candidate files, `arena-<slug>.*` directories, worktrees, branches, or judge scores, and nothing of it in the project beyond the code the task ships. The saved artifact stays where Outputs put it. The text does not cover a run cut off before the cleanup, as in the Claude Code headless turn end and the Grok Build 2400 s cap recorded in `evals/promises/results/2026-10-08-sync-df58112.md`, runs that happen at the same time, a directory left by an older copy of the skill, or a candidate that searches the temp directory and finds a saved artifact there. Stronger isolation needs a sandbox per candidate or per run (#209). `tools/skill-content-conventions.json` retires the fixed path. Keep it on sync and offer it upstream.
- The models config is layered: a workspace `.agents/pstack-models.md` sits over `~/.agents/pstack-models.md`, and a harness section beats a flat line in either file; setup-pstack preserves values it cannot validate (another harness's choices) and has a write-fallback ladder (shell tool → workspace file → print) for harnesses that cannot write to the home directory.
- The lower-ranked gaps from the same mechanism audit that still apply fail closed too (port addition, 2026-10, #120). The `/goal` gap no longer applies, because the 23e4138 sync removed `/goal` from the playbooks. `why` marks a PR it could not read through `gh` as unread. Eval grades only code shape when a transcript is unreadable. Orchestrate dispatches nothing new while `orch` cannot run. When `git worktree add` fails, Opening a PR branches off main in a current checkout with no unrelated changes, never committing to main. In an ungated repository, when the run has no built-in PR tool and neither forge CLI works, it reports a pushed branch with no PR. The persona contract never sends a generic delegate without the briefing. show-me-your-work says when the audit could not read the transcript. The GitNexus sites also need the CLI on `PATH`. Without `node`, a plan's hand-back says the check script did not run. A failed control-skill read leaves the surface with no control skill. Autopilot-stack and the multi-phase plan keep a tick running when the trunk fetch fails. On sync, keep these lines.

Expand Down
25 changes: 7 additions & 18 deletions skills/arena/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,27 +5,16 @@ description: "Spawn N parallel candidates at the same task, pick a base, graft t

# Arena

Fan out N parallel attempts at the same task. Read every candidate's rationale. Pick the strongest as the base. Graft the best ideas from the others into it. Verify the synthesized result.

## Start

Open a worklist with one entry per phase before launching anything.

1. Frame
2. Fan out
3. Cross-judge
4. Pick
5. Graft
6. Verify
Open a worklist with one entry per phase (Frame, Fan out, Cross-judge, Pick, Graft, Verify) before launching anything.

## Phase A: Frame

Every candidate gets the same prompt, so the prompt is the contract.

1. State the artifact each candidate is producing.
2. Derive the rubric, 3-6 concrete gradeable criteria for what success looks like on *this* task. For subjective criteria, use [Eval's criterion and calibration guidance](../poteto-mode/playbooks/eval.md). It is the picker's tool in Phase D. Candidates only see the task.
3. Pick the runners. Use the `arena runners` role, one arm per entry. Its default is one each on `fable`, `opus`, `sonnet`. A rejected entry also follows the required-arm rule below. Spawn more when the arena covers multiple design directions. Same model N times when the work is generation-bound rather than judgment-sensitive. Use the **pstack-harness** skill for the spawn. Its **Spawn a role** section resolves the role and builds this CLI's spawn call.
4. Give each candidate its own output location (a git worktree where possible, otherwise `/tmp/arena-<slug>/candidate-<n>/`), per the **separate-before-serializing-shared-state** principle skill.
3. Pick the runners. Use the `arena runners` role, one arm per entry. Its default is one each on `fable`, `opus`, `sonnet`. A rejected entry also follows the required-arm rule below. Spawn more runners when the arena covers multiple design directions. Use the same model N times when the work is generation-bound rather than judgment-sensitive. Use the **pstack-harness** skill for the spawn. Its **Spawn a role** section resolves the role and builds this CLI's spawn call.
4. Give each candidate its own output location (a git worktree where possible, otherwise `candidate-<n>/` inside a fresh `mktemp -d "${TMPDIR:-/tmp}/arena-<slug>.XXXXXX"` made for this run), per the **separate-before-serializing-shared-state** principle skill. Never reuse an earlier run's directory. A native isolation option counts only when it makes a git worktree, where `git -C <its path> rev-parse --path-format=absolute --git-common-dir` prints the project's. Grok Build's `isolation: worktree` can make a standalone clone, so spawn there without it. Make every other candidate worktree yourself with `git worktree add --detach <run dir>/candidate-<n>`, where `<run dir>` is that fresh `mktemp -d`, and name the path in the candidate's brief. Clean up the run when it ends, after saving its outputs per Outputs, and before a re-frame and re-run starts, whether Phase E or Phase F sent you back. To clean up, remove each candidate worktree with `git worktree remove --force <path>`, delete any branch made for it with `git branch -D`, and delete the scratch directory. Going back to Phase E to graft keeps them. Keep the rubric outside every directory a candidate can read, and give it to the cross-judge in its prompt.

## Phase B: Fan out

Expand All @@ -37,7 +26,7 @@ An arm or exact model requested for this task remains an obligation. If it canno

## Phase C: Cross-judge

After all Phase B candidates complete, choose one arm of the `arena cross-judge pool` role. Use the **pstack-harness** skill's **Spawn a role** section to resolve it. Its default pool is `fable`, `opus`, `sonnet`. Prefer a different model family or capability tier from the parent's. Spawn one readonly judge subagent on that model. It sees the rubric and the candidates by path label, scores each criterion, and recommends a base with rationale. It runs in parallel with the parent's reading in Phase D, not with the candidates themselves. Don't spawn the judge while candidates are still writing.
After all Phase B candidates complete, choose one arm of the `arena cross-judge pool` role. Use the **pstack-harness** skill's **Spawn a role** section to resolve it. Its default pool is `fable`, `opus`, `sonnet`. Prefer a different model family or capability tier from the parent's. Spawn one readonly judge subagent on that model. It sees the rubric and the candidates by path label, scores each criterion, and recommends a base with rationale. It runs in parallel with your reading in Phase D. Don't spawn it while candidates are still writing.

## Phase D: Pick a base

Expand All @@ -51,12 +40,12 @@ Pick the base a future maintainer can extend most easily without breaking invari

Walk each losing candidate's rationale and the cross-judge's scores once more for what is worth porting into the base, usually one or two things per candidate, not most of it. Read only the parts of a losing artifact you will graft. Fold each graft in by hand, per the **redesign-from-first-principles** principle skill, so the result stays coherent under one mental model. Don't paste mechanically.

If the candidates converge on one shape, that is strong agreement. Note it and ship the consensus shape. No graft is needed. If they wildly diverge, Phase A was under-specified. Reframe and re-run rather than averaging the divergence.
If the candidates converge on one shape, that is strong agreement. Note it and ship the consensus shape. No graft is needed. If they wildly diverge, Phase A was under-specified. Reframe and re-run rather than averaging the divergence. Before the re-run, remove this run's candidate worktrees and their branches and delete its scratch directory.

## Phase F: Verify

Verify the synthesized artifact like any other output, per the **prove-it-works** principle skill. If verification finds a problem the arena missed, either Phase A was wrong (re-frame and re-run) or a candidate caught it and you missed the graft (go back to Phase E). Don't paper over it.
Verify the synthesized artifact like any other output, per the **prove-it-works** principle skill. If verification finds a problem the arena missed, either Phase A was wrong (re-frame and re-run) or a candidate caught it and you missed the graft (go back to Phase E). Don't paper over it. Before a re-frame and re-run, remove this run's candidate worktrees and their branches and delete its scratch directory. Going back to Phase E keeps them. Once verification passes, save the outputs, then clean up the same way.

## Outputs

One synthesized artifact, with a short synthesis note alongside naming the base and why, the cross-judge's verdict, each graft with its source candidate, what was rejected and why, any convergence or dropouts, and the verification result.
One synthesized artifact, saved where the calling task says. Code the calling task ships goes onto the branch it works on. A sketch awaiting the caller's sign-off is not yet code the task ships, so it stays outside the project until sign-off. Any other artifact whose task names no place goes in a fresh `mktemp -d` outside the project. Give a short synthesis note in your reply rather than as a file, naming the base and why, the cross-judge's verdict, each graft with its source candidate, what was rejected and why, any convergence or dropouts, and the verification result.
2 changes: 1 addition & 1 deletion skills/poteto-help/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Answer the user's question about pstack, hand them a prompt they can send, and l

A message that asks for work, such as "use pstack to fix this bug", is not a help question. Use the **poteto-mode** skill and do the work under it.

This file maps questions to the skills and guide pages that hold the answers. Those files own the details. Read the file you route to before you quote it, and trust it when it disagrees with this map. Read a sibling skill from the installed skills root. The README and guide pages don't ship with an installed skill, so read them from a pstack checkout when the session runs in one, and otherwise from their public copy. Give the user the file's public copy: `https://github.com/mdsmithaustin/pstack/blob/main/` followed by its path.
This file maps questions to the skills and guide pages that hold the answers. Those files own the details. Read the file you route to before you quote it, and trust it when it disagrees with this map. Read a sibling skill from the installed skills root. The README and guide pages don't ship with an installed skill, so read them from a pstack checkout when the session runs in one, and otherwise from their public copy. Give the user the file's public copy: `https://github.com/mdsmithaustin/pstack/blob/main/` followed by its repository path, such as `skills/how/SKILL.md` or `docs/guide/01-setup.md`. A skill's file sits under `skills/`, so a path relative to the skills root, such as `how/SKILL.md`, 404s.

## Find out what they need

Expand Down
Loading
Loading