docs: explain OpenRouter prompt-injection guardrail rejections

[aviggiano]

Problem

#1149 reports that OpenRouter rejects requests from the OpenRouter-backed OpenCode adapter with HTTP 403 Request blocked: prompt injection patterns detected. It blames transcript-like examples in Ultrafuzz's own prompt templates and proposes four things: a provider-specific prompt transform, snapshot pinning for that transform, a normalization record, and a live-provider fixture.

Root cause

The 403 comes from OpenRouter's opt-in prompt-injection guardrail. It fires when a guardrail that covers the operator's key uses the Block action. Ultrafuzz's prompt sources and the rendered prompts of a local run match none of OpenRouter's 33 documented exact patterns. That check does not cover OpenRouter's typoglycemia, misspelling and encoding layers, which its docs describe but do not publish as exact rules.

• What OpenRouter documents (prompt injection, guardrails, errors, guardrail API):

  • Block rejects "the entire request ... with a 403 before it reaches the model". The error message is Request blocked: prompt injection patterns detected.
  • "Every incoming message is scanned", and base64/hex content is decoded and checked for injection keywords. The API's scan_scope for this detector "Defaults to all_messages" and can be set to user_only.
  • When several guardrails apply to one request, "the most restrictive action wins" (block > redact > flag).

• Ultrafuzz's text does not match the exact patterns. I copied the 33 exact regexes from OpenRouter's documentation on 2026-09-28 and ran them over:

  • Ultrafuzz's prompt sources: .ultrafuzz/prompts, packages/prompts/src and packages/runtime/src/templates, 88 files.
  • The 8 rendered prompt snapshots and 5 dynamic templates from a local run.
  • Smithers 0.35.0's worktree-isolation notice.

  None of them matched.

• Text Ultrafuzz does not write does match:

  • OpenCode 1.18.18's built-in prompts, checked at tag v1.18.18 in packages/opencode/src/session/prompt/*.txt. default.txt and trinity.txt each match role_delimiter_injection: the line assistant: [runs ls and sees foo.c, bar.c, baz.c] followed by a user: line. SystemPrompt.provider() picks default.txt for any model ID that is not Muse, GPT/o1/o3, gemini-, Claude, Trinity or Kimi, for example DeepSeek, Qwen or GLM. llm/request.ts puts it into the system prompt of every request, and Ultrafuzz never sees it.
  • Ordinary target files. Vyper (balances: HashMap[address, uint256] then user: address) and YAML (roles: [admin] then user: alice) both match role_delimiter_injection. I wrote these snippets myself; I did not find them in a real target.

So rewording Ultrafuzz's prompts cannot prevent this. It is an operator setting.

Correction to the analysis this work item started from. anthropic.txt is the prompt OpenCode uses for openrouter/anthropic/claude-opus-4.8, the example OpenCode profile in docs/config.md (the default config ships no OpenCode model profile). It does not match any documented exact regex: its lines that end in ] are followed by </example>, not by a role line. I do not know what tripped the reporter's request. The issue does not name the model.

Change

Docs plus a CHANGELOG entry. In docs/config.md:

• A new ## OpenRouter guardrails section, placed after the Pi agent section because it covers OpenRouterAgent, PiAgent, and OpenCodeAgent with openrouter/ models. It:
  • quotes the 403 message and says where it comes from;
  • says the match can come from the harness's own system prompt or from target source and test output, since OpenRouter scans every message by default, using the verified OpenCode 1.18.18 example;
  • tells operators to set prompt-injection detection to Flag, or turn it off, on every guardrail that covers the key: the workspace default, member and API-key guardrails. The most restrictive action wins, so fixing only one of them is not enough. It also tells them not to use Redact, which forwards the request with each match replaced by [PROMPT_INJECTION];
  • says what relaxing costs. The workspace default covers every key in its workspace, and a member guardrail covers every key of that member. A workspace used only for the Ultrafuzz key keeps the workspace-default change to that key. In an organization account, only an organization admin can change guardrails;
  • says Ultrafuzz has no special handling for the rejection. The attempt fails like any other agent error and follows the [retry] policy. A retry on the same profile is rejected again when the match is in the task prompt or in the harness's system prompt.
• The OpenCode agent section no longer says "The default root config includes an opt-in OpenCode profile". packages/config/defaults.toml and ultrafuzz.toml have [agents.OpenCodeAgent] but no [models.opencode], and git log -G '\[models\.opencode\]' over both files is empty. The text now says to add a model profile. The doctor paragraph's "keeping the shipped [models.opencode] profile" is left alone, because perf(runtime): halve launch fsync work, and doctor stops failing on unused agent profiles #1179 rewrites that paragraph and removes the claim.

I departed from the work-item spec in two places:

1. Every guardrail, not just the key's. The spec said to set "the guardrail on the key" to Flag. Because OpenRouter applies the most restrictive action across all guardrails that cover a request, a workspace-default Block would stay in force. The doc names all three kinds.
2. No promise that the fallback runs. The spec said to state "fresh-session retry, then the [retry] agents fallback". On current main that is not what happens when the 403 repeats identically. The agent Task's retry policy (smithers.ts:7917) sets no maxIdenticalFailures, so Smithers 0.35.0 falls back to DEFAULT_MAX_IDENTICAL_FAILURES = 3 (@smthrs/scheduler/src/errorSignature.js). It marks the node stalled on the third failure with the same error signature (makeWorkflowSession.js:878), before any later attempt or fallback profile runs. I read this in the pinned source; I did not run it. The new section points to the [retry] paragraph instead of restating it. That paragraph is itself wrong about identical failures on main, and fix(runtime): give agent retries a real wait, stop retrying deterministic failures, and label timeouts by code #1171 fixes it: it sets maxIdenticalFailures: 0 and states that "repeated identical failures do not end it before later attempts or fallback profiles run". The pointer is accurate only after fix(runtime): give agent retries a real wait, stop retrying deterministic failures, and label timeouts by code #1171 lands, so merge this PR after fix(runtime): give agent retries a real wait, stop retrying deterministic failures, and label timeouts by code #1171.

Deliberately not built (and why)

• Prompt transform or delimiter rewriting, as the issue proposes: Ultrafuzz's prompts have no matches. The matching text is in OpenCode's system prompt and in tool history, which Ultrafuzz does not own. Rewriting target content would also corrupt code and findings evidence.
• Snapshot pinning and a normalization record: more sealing and provenance machinery for a transform that should not exist.
• Live-provider fixture: it would need a paid OpenRouter workspace with a Block guardrail in CI, and it would test OpenRouter's pattern set rather than Ultrafuzz.
• Classifying the 403 as non-retryable AGENT_CONFIG_INVALID: that contradicts Add bounded error-agnostic agent retries with backoff and optional model fallback #572, which rules out matchers on provider error text. It would also stop the retry chain. Retrying can still succeed when the match came from content the failed session happened to read.
• A committed test that pins OpenRouter's regex list: OpenRouter changes its patterns over time, so such a test would pin their docs, not Ultrafuzz behavior.
• Recommending OpenRouter's phrase allowlist instead of Flag: entries are exact substrings, so they can exempt a known harness line but cannot anticipate target source or tool output. An allowlisted phrase also does not exempt a separate encoding or misspelling detector hit.

Verification

There is no behavior change, so there is no discriminating test. A test that pins doc prose would prove nothing.

Run for the first commit:

• The regex scans described above. The script is in the scratchpad and is not committed.
• Smithers 0.35.0 classification. In BaseCliAgent, classifyQuotaError and the 8 non-retryable patterns (loaded from the pinned source) match neither the Codex-shaped message (unexpected status 403 Forbidden: {"error":{...}}) nor the bare message. Both therefore become a generic AGENT_CLI_ERROR.
• Ultrafuzz's failure normalizer. A scratch test through captureAgentFailure in packages/runtime/test/generated-workflow-verifier.test.ts, reverted afterwards. It covered both message shapes, with and without AGENT_CLI_ERROR, on both generate and preflight. Every case came out with no code or details and the provider message intact. In the same run, 402 gateway failures are promoted to Smithers quota parking controls and agent failure normalization preserves only validated Smithers recovery controls passed.
• OpenCode 1.18.18 session/retry.ts. A 403 with OpenRouter's documented body is not retryable there, and its internal retries are capped at 5, so OpenCode does not loop on it.

Run for the review follow-up commit:

• Re-ran role_delimiter_injection, taken from OpenRouter's current page, over OpenCode v1.18.18 default.txt, trinity.txt and anthropic.txt. It hits default.txt:43 and trinity.txt:45 once each and anthropic.txt not at all. Re-read SystemPrompt.provider() at that tag.
• Checked the new claims against OpenRouter's docs, fetched 2026-09-29:
  • scan_scope "Defaults to all_messages" (PromptInjectionScanScope in the guardrail API reference).
  • "Each workspace has a default guardrail that applies to all traffic in that workspace".
  • "All of an organization member's created API keys will implicitly follow that user's guardrail assignment".
  • "If you're using an organization account, you must be an organization admin to create and manage guardrails".
• Read the Smithers stall path in the pinned 0.35.0 source (departure 2). I did not run it.
• On the final tree, all of these pass: npx prettier --check docs/config.md CHANGELOG.md, node scripts/docs-check.mjs, node scripts/audit-profile-docs.mjs --check, node scripts/prompt-catalog-docs.mjs --check, CI=1 ESLINT_PLUGIN_DIFF_COMMIT=origin/main pnpm -w lint:strict:ci (a no-op, since no code files changed) and pnpm -w knip.
• git merge-tree against fix(runtime): give agent retries a real wait, stop retrying deterministic failures, and label timeouts by code #1171, fix(runtime): agent adapters never hang or reroute on telemetry and environment quirks #1173, refactor: delete dead prompt rename, config helpers, and CI scripts orphaned by #1131 #1175, perf(runtime): halve launch fsync work, and doctor stops failing on unused agent profiles #1179 and ci: validate every package on PRs, stop cancelling main runs, and add a global complexity ceiling #1184. docs/config.md merges cleanly with each. CHANGELOG.md conflicts with all of them, because every PR adds to the same list.

CI is red for a reason outside this PR's content.

• External static analysis fails, and release-gates fails because it depends on it. The cause is Super-Linter's markdownlint MD013 (lines over 400 characters) on CHANGELOG.md.
• There are 43 errors: 42 on lines this PR does not touch, plus this PR's entry. docs/config.md has no markdownlint errors.
• Every PR in this batch that edits CHANGELOG.md fails the same way. ci: validate every package on PRs, stop cancelling main runs, and add a global complexity ceiling #1184 turns MD013 off.
• A local markdownlint-cli 0.45.0 run with ci: validate every package on PRs, stop cancelling main runs, and add a global complexity ceiling #1184's .github/linters/.markdown-lint.yml passes on both files. Merge after ci: validate every package on PRs, stop cancelling main runs, and add a global complexity ceiling #1184, or re-run CI once it lands.

Not verified:

• I did not reproduce this against live OpenRouter; there is no key or Block-mode workspace here. How each harness (Codex, OpenCode, pi) surfaces the 403 comes from reading source, not from watching it happen.
• OpenRouter does not document whether it scans the Responses API instructions field. That field matters because OpenRouterAgent runs Codex with wire_api = "responses".

Risk / compatibility

• Merge order: after fix(runtime): give agent retries a real wait, stop retrying deterministic failures, and label timeouts by code #1171 (the [retry] pointer, departure 2) and after ci: validate every package on PRs, stop cancelling main runs, and add a global complexity ceiling #1184 (CI).
• Only docs and the CHANGELOG change. There is no runtime, config or schema change.
• The OpenCode example is tied to 1.18.18, the version the adapter's state roots were enumerated against. It can go stale when OpenCode changes its prompts.
• Flag gives up OpenRouter's prompt-injection enforcement for every key the relaxed guardrails cover, not only Ultrafuzz's: the whole workspace for the workspace default, and all of that member's keys for a member guardrail. Detections are still recorded. The doc says this and suggests a workspace used only for the Ultrafuzz key. Ultrafuzz's trust model already treats agents as trusted local execution.

Closes #1149

🤖 Generated with Claude Code

Confidence Score: 5/5

The PR appears safe to merge based on the reviewed changes.

Summary

The PR documents how OpenRouter prompt-injection guardrails can reject agent requests and clarifies that operators must add an OpenCode model profile.

• Explains which guardrails operators need to check and the scope of relaxing them.
• Describes the effect of a rejection on Ultrafuzz retries.

_{Reviews (3) · Last reviewed commit: "chore: move the changelog entry to the c..."}