Skip to content

fix(rules): PERM-001 convicted a skill for a flag it was warning about - #54

Merged
Ap6pack merged 2 commits into
mainfrom
claude/perm001-fp-investigation
Aug 16, 2026
Merged

fix(rules): PERM-001 convicted a skill for a flag it was warning about#54
Ap6pack merged 2 commits into
mainfrom
claude/perm001-fp-investigation

Conversation

@Ap6pack

@Ap6pack Ap6pack commented Aug 16, 2026

Copy link
Copy Markdown
Owner

Two fixes, both found by reading a file instead of trusting a verdict.

The false positive

ai-talent-grader — a Chinese HR resume-grading skill with no permission behaviour at all — was MALICIOUS at risk 100, on MALWAR-PERM-001 as its only rule, LLM-confirmed.

The matching line, from its deployment notes:

扫描 verdict 达到 CAUTION 且为 community source 时会 BLOCKED,需 `--yolo` 绕过或修复

"When the scan verdict hits CAUTION on a community source it gets BLOCKED; you need --yolo to bypass, or fix it."

The author is documenting a registry publishing gate. The rule matched the bare token --yolo and read it as an instruction.

This is the same defect as MULTI-001 reading consent as evasion — a technique discussed is not a technique performed — and I shipped it again one rule later, in the rule I wrote because I had just learned that lesson.

The fix

Bypass-flag matches now require an execution context: inside a fenced code block, or reached from a command name or imperative on the same line without crossing sentence-ending punctuation (full-width stops included, since much of the registry is Chinese). Negation before the flag disarms it, because "Never run with --yolo" is advice and parses identically to an instruction.

Two things went wrong while building it, both caught by running cases rather than reading the regex:

  • The first gate broke a real detection: "Run the agent with --dangerously-skip-permissions" stopped firing, because "the agent" is not a command name. The pre-existing test caught it; imperatives now cover it.
  • The second suppressed a true positive: the negation guard scanned the whole line, so "Run claude --flag to avoid prompts" was killed by "avoid" in its tail. Negation now only counts before the flag.

Also: the diagnostic was hiding the evidence

--content truncated at 4,000 chars. The matching line fell outside that window, so the visible portion produced zero findings and the verdict looked inexplicable. Full file by default now; MAX_CHARS trims deliberately.

Verification

1718 passed, 16 skipped

22 cases in both directions. Benign corpus unchanged, malicious control still MALICIOUS at 100, and the vendor connector fleet that motivated the rule still reads SUSPICIOUS 52 on its allowlist edit — so the fix costs no real detection.

Consequence for the numbers

MALWAR-PERM-001 currently reports 767 skills widening agent permissions. That figure includes this false-positive class and is stale until a re-scan. Not to be quoted.

Worth noting separately

The same section of that skill is a field guide to getting past a scanner:

The scanner marks any string containing pip install as MEDIUM supply-chain risk — including echo "安装: pip install xxx". Fix: reword it.

A legitimate author, blocked by false positives of exactly the kind fixed this week, documenting how to launder phrasing until a scanner passes it. Reworded-to-publish and reworded-to-evade are indistinguishable downstream.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DNoTXU8k3pfSBzR7aJubqL


Generated by Claude Code

claude added 2 commits August 14, 2026 12:22
A skill flagged by PERM-001 matched on a line past the 4,000-char excerpt, so
the visible portion produced no finding at all and the verdict looked
inexplicable. A diagnostic that hides the evidence it exists to surface is
worse than none. Full file by default; MAX_CHARS trims deliberately.

Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01DNoTXU8k3pfSBzR7aJubqL
MALWAR-PERM-001 matched the bare token `--yolo` anywhere on a line. A live
skill -- a Chinese HR resume-grading tool with no permission behaviour at all
-- was rated MALICIOUS at risk 100 on this single line, in its deployment
notes:

    扫描 verdict 达到 CAUTION 且为 community source 时会 BLOCKED,需 `--yolo` 绕过或修复

"When the scan verdict hits CAUTION on a community source it gets BLOCKED;
you need --yolo to bypass, or fix it." The author is documenting a registry
publishing gate, not instructing an agent. This is the same defect as
MULTI-001 reading consent as evasion: a technique discussed is not a
technique performed, and I shipped it again one rule later.

Bypass-flag matches now require an execution context: inside a fenced code
block, or reached from a command name or imperative on the same line without
crossing sentence-ending punctuation (full-width stops included, since much of
the registry is Chinese). Negation before the flag disarms it, because
"Never run with --yolo" is advice and parses identically to an instruction --
and only text *before* the flag counts, so "Run claude --flag to avoid
prompts" still fires.

The existing test asserted "Run the agent with --dangerously-skip-permissions"
fires, which the first version of this gate broke because "the agent" is not a
command name; imperatives are matched for exactly that case.

Verified against 22 cases in both directions, the benign corpus (unchanged),
and the malicious control (still MALICIOUS at 100). The vendor connector fleet
that motivated the rule still reads SUSPICIOUS 52 on its allowlist edit.

Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01DNoTXU8k3pfSBzR7aJubqL
@Ap6pack
Ap6pack merged commit 8860e6a into main Aug 16, 2026
5 checks passed
@Ap6pack
Ap6pack deleted the claude/perm001-fp-investigation branch August 16, 2026 03:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants