fix(rules): PERM-001 convicted a skill for a flag it was warning about - #54
Merged
Conversation
A skill flagged by PERM-001 matched on a line past the 4,000-char excerpt, so the visible portion produced no finding at all and the verdict looked inexplicable. A diagnostic that hides the evidence it exists to surface is worse than none. Full file by default; MAX_CHARS trims deliberately. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01DNoTXU8k3pfSBzR7aJubqL
MALWAR-PERM-001 matched the bare token `--yolo` anywhere on a line. A live
skill -- a Chinese HR resume-grading tool with no permission behaviour at all
-- was rated MALICIOUS at risk 100 on this single line, in its deployment
notes:
扫描 verdict 达到 CAUTION 且为 community source 时会 BLOCKED,需 `--yolo` 绕过或修复
"When the scan verdict hits CAUTION on a community source it gets BLOCKED;
you need --yolo to bypass, or fix it." The author is documenting a registry
publishing gate, not instructing an agent. This is the same defect as
MULTI-001 reading consent as evasion: a technique discussed is not a
technique performed, and I shipped it again one rule later.
Bypass-flag matches now require an execution context: inside a fenced code
block, or reached from a command name or imperative on the same line without
crossing sentence-ending punctuation (full-width stops included, since much of
the registry is Chinese). Negation before the flag disarms it, because
"Never run with --yolo" is advice and parses identically to an instruction --
and only text *before* the flag counts, so "Run claude --flag to avoid
prompts" still fires.
The existing test asserted "Run the agent with --dangerously-skip-permissions"
fires, which the first version of this gate broke because "the agent" is not a
command name; imperatives are matched for exactly that case.
Verified against 22 cases in both directions, the benign corpus (unchanged),
and the malicious control (still MALICIOUS at 100). The vendor connector fleet
that motivated the rule still reads SUSPICIOUS 52 on its allowlist edit.
Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01DNoTXU8k3pfSBzR7aJubqL
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two fixes, both found by reading a file instead of trusting a verdict.
The false positive
ai-talent-grader— a Chinese HR resume-grading skill with no permission behaviour at all — was MALICIOUS at risk 100, onMALWAR-PERM-001as its only rule, LLM-confirmed.The matching line, from its deployment notes:
"When the scan verdict hits CAUTION on a community source it gets BLOCKED; you need
--yoloto bypass, or fix it."The author is documenting a registry publishing gate. The rule matched the bare token
--yoloand read it as an instruction.This is the same defect as MULTI-001 reading consent as evasion — a technique discussed is not a technique performed — and I shipped it again one rule later, in the rule I wrote because I had just learned that lesson.
The fix
Bypass-flag matches now require an execution context: inside a fenced code block, or reached from a command name or imperative on the same line without crossing sentence-ending punctuation (full-width stops included, since much of the registry is Chinese). Negation before the flag disarms it, because
"Never run with --yolo"is advice and parses identically to an instruction.Two things went wrong while building it, both caught by running cases rather than reading the regex:
"Run the agent with --dangerously-skip-permissions"stopped firing, because "the agent" is not a command name. The pre-existing test caught it; imperatives now cover it."Run claude --flag to avoid prompts"was killed by "avoid" in its tail. Negation now only counts before the flag.Also: the diagnostic was hiding the evidence
--contenttruncated at 4,000 chars. The matching line fell outside that window, so the visible portion produced zero findings and the verdict looked inexplicable. Full file by default now;MAX_CHARStrims deliberately.Verification
22 cases in both directions. Benign corpus unchanged, malicious control still MALICIOUS at 100, and the vendor connector fleet that motivated the rule still reads SUSPICIOUS 52 on its allowlist edit — so the fix costs no real detection.
Consequence for the numbers
MALWAR-PERM-001currently reports 767 skills widening agent permissions. That figure includes this false-positive class and is stale until a re-scan. Not to be quoted.Worth noting separately
The same section of that skill is a field guide to getting past a scanner:
A legitimate author, blocked by false positives of exactly the kind fixed this week, documenting how to launder phrasing until a scanner passes it. Reworded-to-publish and reworded-to-evade are indistinguishable downstream.
🤖 Generated with Claude Code
https://claude.ai/code/session_01DNoTXU8k3pfSBzR7aJubqL
Generated by Claude Code