feat(runtime): let agents record roadblocks in a run-local friction log

[mrthankyou]

Refs #172

Summary

Adds an opt-in run.friction_log_enabled setting (default false). It lets agents running under Ultrafuzz record roadblocks they hit with Ultrafuzz, tooling, or their instructions. Entries are recorded with Frog and stay local to the run until an operator reviews them.

Now uses Frog. Following @aviggiano's preference for a maintained library over more Ultrafuzz code, this PR goes back to Frog ([email protected], exact-pinned in @ultrafuzz/runtime) and replaces the hand-written entry format of the previous revision. The four problems found in the first Frog version are fixed; see below.

How it works

• Agent command. Agents run <run>/friction-bin/ultrafuzz-friction-log, a generated wrapper around the pinned Frog CLI. The prompt tells them to run … list first, then … log '<title>' --severity … --body '…' with Frog's five sections.
• Wrapper limits. It accepts only log and list with their local options (--body/-b, --severity/-s, --label, --force, --format), and checks each option's value. Everything else is refused with exit 2: publish, sync, --publish, --update, --mcp, --llms, --cwd, --target/-t, --token, and --.
• Where entries land. The wrapper passes --cwd <run>/friction before every agent argument and sets GIT_CEILING_DIRECTORIES=<run>. Entries therefore land in <run>/friction/.agents/friction-log/<id>/friction.md, never in the target repository, and are never validated against the target's issue forms. It also unsets GITHUB_TOKEN, GH_TOKEN, GITHUB_API_URL, GIT_DIR and GIT_WORK_TREE.
• No environment variables. The wrapper finds the run root from its own location, so its bytes are identical for every run. The generated workflow resolves the command and entry directory from each task's own runRoot, so start, resume, replay and fork all resolve the same paths.
• Best-effort preparation. During task preparation the workflow creates <run>/friction (0700) and installs the wrapper (0700) by atomic rename, rewriting it only when its bytes change. A wrapper path that isn't a regular file is left alone, not followed. Any preparation failure goes to stderr and never fails the task.
• Write access. Agents get <run>/friction as an addDir, never the command's directory.
• Failure guidance. The prompt tells agents that recording friction never blocks their task: if a command fails, they continue. They are also told never to create, edit or delete entries by hand. In testing, a single malformed friction.md makes Frog refuse every later log and list with MALFORMED_ENTRY, even with --force.
• Operator review. GIT_CEILING_DIRECTORIES=<run> npx frog list --cwd <run>/friction, or read the Markdown files directly. Review entries before publishing anything, because they can describe private targets.

Fixes to the first Frog version (@aviggiano's review)

1. Resume blanked ULTRAFUZZ_FRICTION_LOG*. There are no environment variables now. Paths come from task.runRoot, and the prompt carries the absolute command path. PATH was not an option either, because continuation drops every target-local PATH entry except the Forge guard's safe-bin.
2. A trailing value-taking flag swallowed --cwd. --cwd now comes before all agent arguments, --cwd itself is refused, and a trailing option without its value is refused.
3. --update / --mcp. Refused, along with every other global except --format.
4. Agent-writable wrapper. The wrapper sits outside the agent's write root, is written best-effort, and is never followed if replaced, so a replaced wrapper can't make resume, replay or fork throw.

This revision also fixes Greptile's two P1s, both introduced when main removed per-node cloud execution. frictionLogDirectory read task.execution.mode, which no longer exists, and the addDir spread split the directory path into single characters. Greptile's P2 about colliding entries is also resolved: Frog reserves each entry directory with an atomic mkdir, so concurrent entries get distinct ids and none is overwritten.

Cost

• Always installed. Frog adds 20 packages (about 36 MB installed) to @ultrafuzz/runtime, and they are installed and sealed even when the friction log is disabled. This is the trade-off for fewer lines of code in Ultrafuzz.
• Smithers install paths move. Frog depends on postgres, which satisfies Smithers' optional postgres peer, so pnpm re-resolves the @smthrs/* packages under a new peer suffix. Patch hashes are unchanged and node scripts/smithers-patches.mjs --check passes. The engine still runs with SMITHERS_BACKEND=sqlite. This does move the engine's install directory, which main's CHANGELOG warns breaks running campaigns in the same checkout.
• Disabled runs. The workflow bakes in null and prompts are byte-identical to main.

Testing

End to end. I ran a smoke run of 9e0e4532 on ultrafuzz_simple_test. Setup: ultrafuzz init --force from this build, friction_log_enabled = true, --audit-profile smoke, Claude via subscription.

• Result: done (succeeded), 24/24 nodes, 0 failed, 21m 41s; the report is COMPLETE and verified. There were 0 failed run events and no friction log unavailable warnings in any engine or agent log.
• Setup: agents received the Friction Log section with this run's absolute command path and --add-dir <run>/friction. The wrapper was installed at <run>/friction-bin/ultrafuzz-friction-log (0700).
• Agent calls: agents called the command 3 times, and all 3 succeeded:
  1. smoke-context ran list, which returned an empty log.
  2. smoke-context ran log, which wrote 20260930170102-forge-wrapper-aborts.
  3. final-report ran list, which showed that entry as pending.
• Entry: the entry is well-formed Frog front matter with all five sections. It records the real macOS safe-bin/forge ulimit -v failure.
• Isolation: no .agents/ appeared in the target root or in any task worktree.
• Direct check: from inside a task worktree, I ran the sealed command's log and list, and both worked. --publish was refused with exit 2. The documented operator command, GIT_CEILING_DIRECTORIES=<run> frog list --cwd <run>/friction, listed the entries. I removed my test entry afterwards.

Automated

• friction-log.test.ts (new) runs the generated wrapper against the real pinned Frog:
  • it logs and lists entries, including a title containing - and a body value starting with -;
  • Frog's DUPLICATE_FRICTION comes back as a clean exit-1 error;
  • the target root stays untouched;
  • every refused surface above exits 2.
• generated-workflow-verifier:
  • Prompt placement and shell quoting of the command.
  • Preparation behavior: install, no rewrite when the bytes are unchanged, atomic replacement, no leftover staging file, a planted symlink is neither followed nor overwritten, an unwritable run root is reported rather than thrown, and a disabled run gets no extra addDir.
  • 117 of 119 pass and 1 is skipped. The one failure is the existing pinned source proof counts hidden unreachable commits, which assumes Git's default branch is master and passes with init.defaultBranch=master.
• runtime.test.ts: compileSmithersWorkflow applies group execution defaults checks the disabled null and the sealed { instructions, entriesPath, commandPath, wrapper }. The startRun … submits through Smithers CLI test checks the new addDir.
• workspace-preparation-lifecycle passes, as does the prompt fragment test.
• format:check, lint, lint:strict, typecheck, knip, docs:check, size, security:dependency-advisories, smithers-patches.mjs --check and pnpm install --frozen-lockfile are clean.
• The full runtime suite and validate:release were not run locally (macOS, Runtime suite is 2-63x slower per test on macOS than Linux; a full local run does not finish in 7 hours #1060); CI covers them.

Not in this PR

• Friction entries are not yet included in report bundles. That touches the bundle manifest's included_roots, which is better as a follow-up.
• Publishing reviewed entries upstream stays an operator step.
• An upstream Frog issue: log could skip unreadable entries during its duplicate check instead of refusing to write.

🤖 Generated with Claude Code

Confidence Score: 3/5

The PR is not yet safe to merge because a run path containing a colon can defeat the wrapper’s Git discovery boundary.

Findings

1.   Git ceiling fails on colons ▶
2.  Friction entries can collide ▶
3.  Operator listing guidance is missing ▶

Fix with agent prompt

### Issue 1
packages/runtime/src/friction-log.ts:undefined-74
If the target project path contains a colon, the wrapper puts the raw run path in `GIT_CEILING_DIRECTORIES`, which Git treats as a colon-separated list. Git can then discover the enclosing target repository instead of stopping at the run root. A friction entry may land in the target repository rather than the run-local log, bypassing the operator’s review location.

**How this was verified:** The wrapper exports the unescaped run path as Git’s ceiling value before invoking Frog, which uses Git discovery to locate its entry root.

### Issue 2
.ultrafuzz/prompts/_templates/agent-preamble/friction-log.mdx:undefined-5
If two local agents record a problem in the same second and their titles start with the same three words, both use the same `friction.md` path. The existing-title check is not atomic, so both can pass it before either writes. One entry may overwrite the other or leave an incomplete file, losing a useful roadblock report.

### Issue 3
docs/reference/configuration.md:188-190
The docs ask operators to review friction entries before publishing, but remove the only documented way to list them. Runs sit inside the target Git repository, so a plain `frog list --cwd <run>/friction` looks at the repository root and misses the entries. Please restore the Git-ceiling command or explain how to list and read the Markdown files directly; otherwise operators may overlook roadblocks.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

Adds an opt-in, run-local Frog friction log for agents.

• Generates a restricted log/list wrapper and grants agents write access only to the entry directory.
• Adds configuration, prompt guidance, operator documentation, and tests.
• The earlier entry-collision and task-rendering findings are fixed.

_{Reviews (9) · Last reviewed commit: "Merge remote-tracking branch 'origin/mai..."}

[aviggiano]

Thanks for this — the opt-in flag, the static preamble fragment, and the real-Frog integration tests are well done. Before this merges I think it needs a rework, mainly because of cost and a few wrapper holes found while reviewing:

Cost when disabled. [email protected] adds 32 third-party packages to @ultrafuzz/runtime, and every run copies and seals the runtime's transitive closure (trusted-CLI closure: ~343 → 374 packages / ~15.7k → 16.8k files, roughly +20 MB per run, ~10 MB of it @octokit/openapi-types; the same packages also land in the execution snapshot). So friction_log_enabled = false is not a no-op, and it makes the #921 snapshot cost worse.

Bugs found:

1. After a plain ultrafuzz resume (no --refresh-controller), workflowControlChildEnvironment blanks any env value under the project root, so ULTRAFUZZ_FRICTION_LOG / ULTRAFUZZ_FRICTION_LOG_DIR become "" while the prompt still tells agents to use them.
2. A trailing value-taking flag (e.g. ... log --label) swallows the wrapper's pinned --cwd, and Frog then writes .agents/ into the task worktree — the same contamination the first end-to-end run hit.
3. The log/list allowlist misses incur's global --update (runs pnpm add --global frog@latest) and --mcp (exposes a publish tool).
4. The wrapper lives in an agent-writable addDir; if it becomes a symlink the next resume/replay/fork throws ArtifactPathError.

Suggested shape (my recommendation): keep the flag and the static fragment, drop the runtime dependency, and have agents write Frog-format files directly (<run>/friction/.agents/friction-log/<UTC-ts>-<slug>/friction.md with the front matter), deriving the directory from task.runRoot rather than inherited env vars. The operator can run npx frog over the directory after review. That keeps the value of #172 without adding a dependency, a shell allowlist, or git-discovery fencing.

Leaving the final call to the maintainer since #172 is a product decision.

[mrthankyou]

@aviggiano thanks, all four bugs reproduced from the code as you described. I went with your suggested design. It's in 3020a9d, rebased on current main.

Cost when disabled: the frog dependency is gone. packages/runtime/package.json and pnpm-lock.yaml are byte-identical to main, so nothing is added to the trusted-CLI closure or execution snapshot. A disabled run bakes in null and renders byte-identical prompts.

Bugs:

1. Resume blanking the env paths: there are no env vars now. The workflow resolves <runRoot>/friction/.agents/friction-log from each local task's own task.runRoot, as you suggested. Start, resume, replay and fork therefore all resolve the same directory, and the three start-run.ts hooks are reverted.
2. Trailing flag swallowing --cwd: gone with the wrapper. No Frog command runs inside a campaign.
3. --update / --mcp: same, there is no CLI surface to allowlist.
4. Agent-writable wrapper: gone. The directory is created best-effort (mkdir -p, 0700) during task preparation, and no lifecycle command inspects it. A replaced or missing directory can't make resume/replay/fork throw, and a failure to create it goes to stderr instead of failing the task.

Shape: agents write <run>/friction/.agents/friction-log/<UTC YYYYMMDDHHMMSS>-<slug>/friction.md with Frog's single-quoted front matter and its five sections. The fragment's only variable is that directory, which is shared by every task in a run. Cloud tasks get no friction log.

One operator detail: run roots sit inside the target repo and Frog anchors at the Git root, so review needs GIT_CEILING_DIRECTORIES=<run> npx frog list --cwd <run>/friction. Without it Frog lists nothing. This is in the docs and CHANGELOG.

Verified with a clean smoke run of this branch on a small Foundry target (Claude via subscription):

• done (succeeded), 24/24 nodes, report COMPLETE/verified;
• agents wrote 3 entries, and Frog 1.1.0 lists them as pending;
• nothing appeared in the target root.

The entries were real findings about Ultrafuzz itself:

• the safe-bin/forge guard aborts every forge call on macOS, because ulimit -v fails on Darwin under set -eu;
• lib/ submodules were empty in a task worktree, so forge test cloned dependencies over the network.

I'll open issues for both unless you already track them.

Other checks:

• Format, lint, strict lint, typecheck, knip (with the new exports/types/duplicates flags) and docs:check are clean.
• The friction-related runtime and workflow-verifier tests pass.
• The full runtime suite is left to CI.

The PR description is updated with the details.

[mrthankyou]

Small correction to my note on point 4: a directory that can't be created is reported on stderr and doesn't fail task preparation. The path is still passed to the agent as an addDir, and I haven't verified how each CLI treats a missing addDir. That only matters if mkdir -p fails inside the run's own directory. 289aec5 makes the code comment say exactly that.

[greptile-apps]

Friction entries can collide

If two local agents record a problem in the same second and their titles start with the same three words, both use the same friction.md path. The existing-title check is not atomic, so both can pass it before either writes. One entry may overwrite the other or leave an incomplete file, losing a useful roadblock report.

Prompt To Fix With AI

This is a comment left during a code review.
Path: .ultrafuzz/prompts/_templates/agent-preamble/friction-log.mdx
Line: 5

Comment:
**Friction entries can collide**

If two local agents record a problem in the same second and their titles start with the same three words, both use the same `friction.md` path. The existing-title check is not atomic, so both can pass it before either writes. One entry may overwrite the other or leave an incomplete file, losing a useful roadblock report.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

[greptile-apps]

Operator listing guidance is missing

The docs ask operators to review friction entries before publishing, but remove the only documented way to list them. Runs sit inside the target Git repository, so a plain frog list --cwd <run>/friction looks at the repository root and misses the entries. Please restore the Git-ceiling command or explain how to list and read the Markdown files directly; otherwise operators may overlook roadblocks.

Prompt To Fix With AI

This is a comment left during a code review.
Path: docs/reference/configuration.md
Line: 188-190

Comment:
**Operator listing guidance is missing**

The docs ask operators to review friction entries before publishing, but remove the only documented way to list them. Runs sit inside the target Git repository, so a plain `frog list --cwd <run>/friction` looks at the repository root and misses the entries. Please restore the Git-ceiling command or explain how to list and read the Markdown files directly; otherwise operators may overlook roadblocks.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

[aviggiano]

@mrthankyou the idea to remove frog was actually claude's not mine
I didn't respond because I was still thinking about their suggestion 😆 😆 😆 😆 😆 😆

I noticed frog is quite active and maintained, so I wouldn't mind having an external dependency rather than growing ultrafuzz's codebase

sorry for not having said that earlier

what do you think? I'm fine with both options tbh, with a slight preference for using a lib (reducing LOC is always good)

[mrthankyou]

Let me look at frog again. I'm fine with either approach as long as frog can gracefully handle errors along the way.

Reducing LoC is a valid reason as well.

[mrthankyou]

@aviggiano went with Frog, as you preferred: 9e0e453 replaces the hand-written entries with [email protected], exact-pinned in @ultrafuzz/runtime. Agents call a generated <run>/friction-bin/ultrafuzz-friction-log wrapper that addresses your four earlier findings:

1. No environment variables. Paths come from task.runRoot, and the prompt carries the absolute command path.
2. --cwd is pinned ahead of all agent arguments, and every option's value is checked.
3. Only log/list with their local options are allowed; --update, --mcp, publishing and the rest are refused.
4. The wrapper sits outside the agent's write root and is installed best-effort, so a replaced wrapper can't make resume throw.

It also fixes Greptile's two P1s, which came from main removing per-node cloud execution.

Two things worth knowing:

• Frog refuses every later log and list once a single entry is malformed, so the prompt tells agents to continue on any failure and never to edit entries by hand.
• Frog's postgres dependency satisfies Smithers' optional peer, which moves the @smthrs/* install paths. Patch hashes are unchanged and the engine still runs on SQLite.

A smoke run of this commit succeeded 24/24, and an agent recorded a real entry through Frog. Details are in the updated description.

[socket-security]

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	frog@​1.1.0

View full report

[greptile-apps]

Git ceiling fails on colons

If the target project path contains a colon, the wrapper puts the raw run path in GIT_CEILING_DIRECTORIES, which Git treats as a colon-separated list. Git can then discover the enclosing target repository instead of stopping at the run root. A friction entry may land in the target repository rather than the run-local log, bypassing the operator’s review location.

How this was verified: The wrapper exports the unescaped run path as Git’s ceiling value before invoking Frog, which uses Git discovery to locate its entry root.

Prompt To Fix With AI

This is a comment left during a code review.
Path: packages/runtime/src/friction-log.ts
Line: 74

Comment:
**Git ceiling fails on colons**

If the target project path contains a colon, the wrapper puts the raw run path in `GIT_CEILING_DIRECTORIES`, which Git treats as a colon-separated list. Git can then discover the enclosing target repository instead of stopping at the run root. A friction entry may land in the target repository rather than the run-local log, bypassing the operator’s review location.

**How this was verified:** The wrapper exports the unescaped run path as Git’s ceiling value before invoking Frog, which uses Git discovery to locate its entry root.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

[aviggiano]

Thanks @mrthankyou, this is careful work. The rework fixes all four bugs from the earlier review and both of Greptile's cloud-removal P1s. A disabled run renders byte-identical prompts, with no directory or addDir, and nothing in verification, reports or bundles reads <run>/friction. CI is green and the branch merges cleanly with main, so we're merging it as is. No blockers.

A follow-up PR before v0.1.3 hardens the Frog wrapper. Nothing is needed from you.

1. Wrapper gaps found in review.
   • incur honours its built-in flags anywhere in argv, so log x --body --llms and log x --body --mcp get past the value skip, and --mcp exposes publish. Both reproduced. The wrapper will refuse an option value that is itself a built-in flag.
   • Greptile's colon P1 reproduces. GIT_CEILING_DIRECTORIES is a :-separated list, so under an a:b-target/ project the entry landed in the target root. The wrapper will fence Git discovery with a GIT_DIR that doesn't exist instead.
   • Frog falls back to gh auth token, and HOME reaches agents, so unsetting GITHUB_TOKEN/GH_TOKEN doesn't remove GitHub credentials. The follow-up stops that fallback, or drops the claim if it can't.
   • Preparation will rewrite the wrapper by rename every time, so a replaced wrapper path gets replaced instead of being left in place and later run.
2. Cost when disabled. Frog's ~32 packages (~20 MB) are copied into every run's trusted-CLI closure and execution snapshot, even with the log off. The wrapper runs the checkout's Frog, so the follow-up skips frog in both copies.
3. Docs and CHANGELOG. We'll align the wording with docs/security.md. Agents run unsandboxed, so <run>/friction is where they are told to write, not a boundary. We'll also note that entries aren't secret-scanned, and replace the npx frog review command, which can pick up a target's own frog bin.
4. Small items.
   • Replacer functions for the two replaceAll substitutions; today a $$ or $& in the run path gets expanded.
   • A test that task preparation creates the directory when the log is enabled.

Thanks also for the smoke-run findings: the macOS ulimit -v Forge-guard abort and the empty lib/ submodules. Please do open issues for those.

[mrthankyou]

@aviggiano I opened issues for both smoke-run findings:

• Forge guard aborts every forge call on macOS: ulimit -v fails on Darwin under set -eu #1250: the Forge guard stops every forge call on macOS, because ulimit -v can't be set on Darwin under set -eu.
• Task worktrees of unpinned runs have uninitialized lib/ submodules, so forge clones dependencies over the network #1251: lib/ submodules are empty in the task worktrees of unpinned runs. Per-task hydration (fix(runtime): hydrate pinned submodules in local task worktrees #473) only runs when the launch HEAD is ultrafuzz-pinned, so forge clones dependencies over the network.