fix(security): stop the publication secret gate rejecting public test keys and JWT-shaped identifiers

[aviggiano]

Problem

The publication secret gate (assertArtifactPublicationsContainNoSecrets, run from the workflow verify step and from verified-output.ts) scans each published file in positive-only mode. It fails the node on any hit and does not rewrite bytes. On origin/main, three positive rules fire on ordinary Foundry campaign output:

Output	Rule that fires on main
`ReentrancyGuardUpgradeable.nonReentrantModifier.lockedStateCheck` reverts on re-entry (report text)	supplemental JWT rule
auditProfileResolution.settingOrigins.property_priority_threshold (a JSON path)	supplemental JWT rule
uint256 privateKey = 0xac0974…ff80; (Anvil account 0)	context-labeled private-key rule
string memory mnemonic = "test test … junk";	BIP39 mnemonic rule

The last two are the canonical Foundry idiom. forge-std's own test_DeriveRememberKey (lib/forge-std/test/StdCheats.t.sol) quotes the mnemonic and asserts key 0, and main's detector rejects that file. The gate runs after the agent has finished, so a generated test or report shaped like these fails its node.

These failures are shown at the detector and gate level (see Verification). I have not reproduced a full campaign killed by them.

Root cause

• JWT rule. /\b[A-Za-z0-9_-]{20,}\.[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}\b/ identifies a JWT by shape alone. Three long dotted identifier segments have the same shape.
• Private-key and mnemonic rules. These rules cannot tell a secret key from a published one. Anvil prints the test test test test test test test test test test test junk mnemonic and its first 10 keys (m/44'/60'/0'/0/0-9) on every startup. The mnemonic has a valid BIP39 checksum, and privateKey = <64 hex> satisfies the label rule.

Change

All source edits are in packages/security/src/sensitive-redaction.ts:

1. JWT rule. The first segment gets a (?=eyJ) lookahead, and the old 20/10/10 minimum lengths stay. A JWT header in compact JSON that opens with {" and a letter base64url-encodes to eyJ. The later segments are not constrained, because a JWE's second segment is its encrypted key.
2. Public dev material.
   • PUBLIC_DEVELOPMENT_PRIVATE_KEYS holds the 10 keys anvil prints. The labeled-key rule skips a candidate in this set, compared case-insensitively with or without 0x.
   • PUBLIC_DEVELOPMENT_MNEMONIC: when a valid window is exactly this phrase, the mnemonic rule ends the word run as it does for any detected mnemonic but records no redaction range. The words after the phrase are therefore checked exactly as on main.

As a yes/no property, detection only narrows. When main's version of a changed pass matches nothing in a string, the new version matches nothing there either. Every match of the new JWT rule also satisfies the old pattern, and the key and mnemonic changes only drop matches; the mnemonic pass ends its word runs exactly where main's does. If main leaves a string unchanged, every pass here therefore receives that same string and also leaves it unchanged. So containsSensitiveSecrets can go from true to false but not from false to true, and main's fixed points (the checks in modal/src/worker-diagnostics.ts and runner.ts) are kept. Redaction spans can differ: for abc-<jwt>, main's JWT rule redacts from abc- and this rule redacts from eyJ.

Tests:

• sensitive-redaction.test.ts, one test per rule:
  • Must publish: the 10 keys in 3 label forms, the mnemonic, prose after the mnemonic, and 3 dotted identifiers.
  • Must still be rejected: a real mnemonic on the line after the phrase (it must be redacted in full), a different valid mnemonic that shares the first 11 words, key 0 with one digit changed, an HS256 JWT, an RSA-OAEP compact JWE, and a JWS whose payload is space-padded JSON.
• artifacts.test.ts: the gate test gets a must-publish block with a Foundry test file (mnemonic, key 0, vm.deriveKey) and a report line with a dotted identifier. Existing must-reject fixtures k01-k15 are unchanged and still rejected.

Since the first revision (review follow-up)

• The mnemonic exemption used continue, which never reset the word run. Two consequences, reproduced on 247c75a:

  • The gate still rejected the phrase when a whitespace-separated BIP39 word followed it (… junk one account per index, and two copies on consecutive lines). 128 of the 2048 words trigger this.
  • Redaction weakened next to a real mnemonic: <phrase>\n<abandon ×11 about> redacted to test <redacted> abandon … about, leaving 11 of the real mnemonic's 12 words in the clear. Main redacted both.

  The phrase now ends the run. This also resolves Greptile's second finding (repeated copies growing one run).

• The payload (?=eyJ) lookahead is gone. It stopped none of the false positives above: all three identifier fixtures publish with the header lookahead alone. What it did was drop detection of compact JWE and of JWS tokens with a non-compact payload, both of which main detects (Greptile's first finding).

• The CHANGELOG no longer says that real JWTs, other keys and other mnemonics are all still rejected, and it now carries the cloud-run note below.

Deliberately not built (and why)

• Exact-value matching of configured env values. The workflow gate also byte-matches sensitiveEnvironmentValues(process.env, …), and that includes any variable named like *PRIVATE_KEY*. An operator who exported PRIVATE_KEY=<Anvil key 0> will still have artifacts containing that value rejected. I left this alone for three reasons: those values come from the operator's environment, the same helper also feeds diagnostic redaction in start-run.ts, workflow-sync.ts and agent-failure normalization, and I have no reproduction. It is a follow-up if it shows up.

• forge-std's public Infura key. The provider-keyed RPC URL rule still rejects https://…infura.io/v3/b9794ad1… in StdChains.sol. Exempting a vendor API key is a separate decision.

• The sk- rule matching kebab-case words such as sk-learn-pipeline. The ak-/as- vendor pattern matches kebab-case English, silently skipping goal-directed audit lanes #822 fix (exclude - from the suffix) doesn't transfer, because OpenRouter (sk-or-v1-…) and OpenAI project keys (sk-proj-…) are themselves hyphenated.

• Hardhat's accounts 10-19. Hardhat's documented default derives 20 accounts from the same mnemonic. I did not run Hardhat, and only Anvil's 10 are exempted, as the spec asked.

• JWT-shaped tokens this rule no longer matches. Main matched these by shape:

  • a header that does not begin eyJ, such as a space-padded header (eyA…) or a first member name that starts with a digit (eyI…);
  • a token glued to a preceding word character (token_eyJ…), which has no \b before eyJ.

  JWT libraries emit compact headers. A Bearer <token> is still caught by the Bearer rule, and configured values are still matched byte for byte. The spec asked for eyJ on both the header and the payload. The header alone stops every false positive shown here and gives up less detection.

• Longer mnemonics that contain the phrase. A valid 15- to 24-word mnemonic containing the public phrase as 12 consecutive words is not flagged, because the phrase ends the run before the longer window completes. Main flagged such a mnemonic only through the public phrase, and it redacted just those 12 words, which left the words carrying the secret bits in the clear. I verified this with a constructed valid 24-word mnemonic that begins with the phrase. Random entropy reproduces the phrase's 132 bits with probability 2^-132, so such a mnemonic arises only by construction. Catching it would need lookahead past the phrase.

• Dotted run ids. I first suspected the old JWT rule also broke run-link verification for a dotted --run-id, and calling prepareWorkflowRunLink and finalizeWorkflowRunLink directly does fail on main. But startRun rejects such ids earlier (the workflow runner id must match ^[a-z0-9_-]{1,64}$), so that path is unreachable, and this PR makes no claim about it.

Verification

Discriminating tests. I swapped in packages/security/src/sensitive-redaction.ts from each revision under the current test files:

• Previous head (247c75a): both changed tests fail.

  • Anvil test: containsSensitiveSecrets("<phrase> one account per index", [], "positive-only") returns true.
  • JWT test: the JWE is not detected.

  I also ran each new assertion on its own. The four that are new since the first revision fail on 247c75a and pass now: prose after the phrase, the real mnemonic on the next line, the JWE, and the space-padded-payload JWS.

• origin/main: both tests fail. The phrase is flagged, and ReentrancyGuardUpgradeable.nonReentrantModifier.lockedStateCheck is flagged. Main detects the JWE and the padded-payload JWS, so those two assertions guard against losing detection rather than discriminate against main.

• Artifacts gate test with main's security source: it fails with artifact publication contains sensitive data: generated-tests/AnvilSigner.t.sol (re-run for this revision).

• This head: security passes 25/25, and all artifacts test files pass 354/354.

Differential fuzz against origin/main. Both builds' @ultrafuzz/security dist ran on 6 seeds × 4,000 generated inputs × 2 modes, 48,000 runs in total. The inputs mix the phrase, the keys, labels, random 64-hex, BIP39 words, a real mnemonic, eyJ-prefixed, ey-prefixed and random dotted segments, and punctuation. Half of them are whitespace-joined word streams. Results:

• 0 inputs that main leaves unchanged and this branch changes.
• 0 inputs where main's output is idempotent and this branch's is not.
• 15,649 positive-only runs had no JWT-shaped pieces. In every one, this branch's output equals main's output with some occurrences of the phrase or the keys restored. The same check fails on 247c75a in 1,203 of 2,616 runs on seed 1, so it does detect the run-reset bug.

Keys. The 10 keys match cast wallet private-key --mnemonic "test … junk" --mnemonic-index 0..9, in order, which I re-ran for this revision.

Corpus check. I scanned a local target checkout (tiny-vault) with both builds in positive-only mode. The scan covered the 69,783 files under 1 MB with no NUL byte, skipping .git, node_modules, out and cache directories. It includes the target's .ultrafuzz/runs outputs, execution snapshots and trusted-CLI closures. Results:

• main rejects 341 files and this head rejects 222. No file is rejected here that main accepts.
• Outside execution snapshots (133) and trusted-CLI closures (71), the only files still rejected are forge-std's StdChains.sol and StdChains.t.sol copies (9 each), because of the Infura key.

Suites run on this head:

• security: all tests, 25/25
• artifacts: all test files, 354/354
• runtime:
  • final-report-markdown.test.js: 35/35
  • smithers-diagnostic.test.js: 2/2
  • data-governance.test.js: 10/10
  • generated-workflow-verifier.test.js and verified-output.test.js with --test-name-pattern='secret|redact|credential|sensitive': 3/3 and 1/1
  • runtime.test.js with --test-name-pattern='allowlist|redact|sensitive|classif|cloud agent credential|credential rotation|secret': 5/5
• modal vitest (worker-diagnostics, public-bundle, recovery-lifecycle, public-eval-diagnostics, worker-result, public-worker): 166/166. An earlier run under heavy machine load timed out 3 public-worker tests at 30 s, and they timed out the same way on origin/main at that point.
• config vitest (config.test.ts, stateful-profiles.test.ts): 59/59

Static checks:

• npx prettier --check and npx eslint on the changed files
• CI=1 ESLINT_PLUGIN_DIFF_COMMIT=<merge base> pnpm -w lint:strict:ci
• pnpm --filter @ultrafuzz/security typecheck and pnpm --filter @ultrafuzz/artifacts typecheck
• pnpm -w knip
• node scripts/docs-check.mjs

Not run: the full runtime and CLI suites, or an end-to-end campaign.

Risk / compatibility

• Only narrows detection as a yes/no property; see the argument and the fuzz above.
• No validator identity change. Only packages/security source changes. No schema, contract description or validator module is touched, so the validator build identity does not rotate.
• In-flight runs: attempt ledger. From now on, events, state and attempt-ledger diagnostics keep these values in the clear where main wrote <redacted>.
  • Sync re-derives each recorded attempt and reconciles a changed failure message through the redaction spans recorded with it. The ledger records no spans when redactedTextSpanCodePointLengths returns none, which happens for a message over 16,384 characters or when span inference is ambiguous.
  • So consider an attempt recorded before the upgrade whose raw failure message is that long and contains one of these values. After the upgrade, sync throws node attempt … was already recorded with different immutable data for that run.
  • I reproduced this at function level, with main's createNodeAttemptLedgerEntry and this branch's reconcileNodeAttemptLedgerEntry, on three messages: a 22,093-character message with the dotted identifier, an 18,089-character one with signing key: <Anvil key 0>, and a 22,089-character one quoting VM::deriveKey("<phrase>", 0). The short versions of the same messages reconcile through their recorded spans.
  • fix(runtime): attempt and usage ledgers never block run synchronization #1186 (w02) removes this re-derivation: it returns a recorded attempt as stored.
• In-flight cloud runs: credential classification. At resume, replay and fork, assertCurrentCloudAgentCredentialEnvironment recomputes the classification of ULTRAFUZZ_AGENT_ENV_ALLOWLIST variables and compares it with the list sealed at compile time.
  • Consider a variable whose name is not credential-like and whose value is the public mnemonic, or a three-segment dotted value without an eyJ header (for example anvil-fork-node-production.monad-foundation.cloudfront-cdn). Main classified it sensitive and dropped it; this branch forwards it.
  • Such a run therefore refuses with cloud agent credential classification changed after workflow compilation …; start a new run. I reproduced this by compiling with origin/main's compileSmithersWorkflow and checking with this branch.
  • Removing the variable from ULTRAFUZZ_AGENT_ENV_ALLOWLIST, or unsetting it, passes the check (verified). The CHANGELOG says so.
  • A JWE-shaped value keeps its classification.
• Security. The exempted values are published, so exempting them hides nothing. JWT-shaped tokens without an eyJ header, and constructed longer mnemonics that contain the phrase, are no longer caught by these rules (see above).
• CI. "External static analysis" fails markdownlint MD013 (400 characters) on CHANGELOG.md for the whole file. origin/main's copy already has 43 longer lines, up to 1,777 characters, and 42 of them are flagged, so any PR that edits the file fails the job. This entry is 398 characters and is not flagged (CI run 36515120221 reports 42 MD013 errors, all on pre-existing lines; gitleaks reports no leaks). This PR does not fix the existing ones. Because that job fails, release-gates fails too and the full release validation lanes are skipped, so the runtime release lanes have not run on this PR in CI. "PR build and Node.js 24 runtime smoke", which includes the diff-limited strict lint, passes.

🤖 Generated with Claude Code

Confidence Score: 4/5

The PR does not appear safe to merge while a constructed longer private mnemonic beginning with the public phrase can pass the publication gate.

Findings

1.   Longer mnemonics can pass ▶

Fix with agent prompt

### Issue 1
packages/security/src/sensitive-redaction.ts:388-390
A valid 24-word private mnemonic can start with the public Anvil phrase. This branch clears the word run after those first 12 words, so the scanner never checks all 24 together. If the remaining 12 words are not a valid mnemonic on their own, the publication gate accepts the private phrase. The previous code kept scanning after the public phrase. **How this was verified:** The public-prefix branch clears the word run before the 24-word window can be assembled, and the publication gate relies on this scan for mnemonic detection.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

The PR narrows publication-secret detection to allow Anvil’s published mnemonic and ten development keys, and to avoid treating long dotted identifiers as JWTs. Security and artifact tests cover the intended exemptions and retained detection.

_{Reviews (4) · Last reviewed commit: "Merge remote-tracking branch 'origin/mai..."}

[greptile-apps]

Longer mnemonics can pass A valid 24-word private mnemonic can start with the public Anvil phrase. This branch clears the word run after those first 12 words, so the scanner never checks all 24 together. If the remaining 12 words are not a valid mnemonic on their own, the publication gate accepts the private phrase. The previous code kept scanning after the public phrase. How this was verified: The public-prefix branch clears the word run before the 24-word window can be assembled, and the publication gate relies on this scan for mnemonic detection.

Prompt To Fix With AI

This is a comment left during a code review.
Path: packages/security/src/sensitive-redaction.ts
Line: 388-390

Comment:
**Longer mnemonics can pass** A valid 24-word private mnemonic can start with the public Anvil phrase. This branch clears the word run after those first 12 words, so the scanner never checks all 24 together. If the remaining 12 words are not a valid mnemonic on their own, the publication gate accepts the private phrase. The previous code kept scanning after the public phrase. **How this was verified:** The public-prefix branch clears the word run before the 24-word window can be assembled, and the publication gate relies on this scan for mnemonic detection.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.