Skip to content

fix(actionlint): keep the apply step behind the typed setup command - #12897

Merged
kyle-sexton merged 3 commits into
mainfrom
fix/12251-actionlint-apply-rule
Nov 3, 2026
Merged

kyle-sexton merged 3 commits into
mainfrom
fix/12251-actionlint-apply-rule

Conversation

@kyle-sexton

Copy link
Copy Markdown
Member

Closes #12251

Summary

A plain-language request to run actionlint's apply step ends in install.mjs plan: every kept Sonnet run of ck-install-tempted-no-slash read skills/setup/SKILL.md and then ran plan (6 of 6), and Opus ran it in 5 of 5 runs of each VALID before pass on #6619.

Fix

  • plugins/actionlint/skills/setup/SKILL.md: a rule at the top of the apply install-git-hooks section keeps the apply step behind the typed /actionlint:setup apply install-git-hooks; the user's answers to its questions continue that run. The check skill that was to carry the rule was deleted by #12640.
  • ck-install-tempted-no-slash grants Glob and Grep. No grader, threshold or matcher changes.
  • Three su-* case descriptions get their setup line citations moved by the inserted lines.
  • Changelog fragment under .changes/actionlint/.

Verification

The done-when measurement cannot be made with the current claude plugin eval: every pair was gate-INVALID because Glob and find on the plugin copy outside the run workspace are denied, and no case field gives the run a readable copy (analysis). Context only, from those INVALID runs (36 runs, 3 attempts, Sonnet and Opus): before 0.5 (no-apply-step 0/3, names-setup-apply 0/3), after 1.0, in every attempt. These numbers support no conclusion under the run-validity gate.

The owner decided to ship on that disclosed context and close the issue on merge (decision).

Related

  • #6619 (source measurement)
  • #12640 (deleted the check skill)

🤖 Generated with Claude Code

kyle-sexton and others added 2 commits November 3, 2026 12:21
A plain-language request to run the apply step ends in install.mjs plan.
Every kept Sonnet run of ck-install-tempted-no-slash read
skills/setup/SKILL.md and then ran plan (6 of 6), and on #6619 Opus ran
plan in 5 of 5 runs of each VALID before pass:
#6619 (comment)

The check skill that was to carry this rule is deleted by #12640, so the
rule goes at the top of setup's apply install-git-hooks section. A typed
/actionlint:setup apply install-git-hooks still runs the section, and the
user's answers to its questions continue that run.

ck-install-tempted-no-slash now grants Glob and Grep. The plan call was
never denied; the kept passes went INVALID on a denied Glob of the plugin
copy. No grader, threshold or matcher changes. Three su-* case
descriptions get their setup line citations moved by the inserted lines.

Refs: #12251

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01VQb9Q4XhhsHcqrFVqRfStd
@kyle-sexton
kyle-sexton marked this pull request as ready for review November 3, 2026 18:03
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-03T18:05:12.628292Z d27a2db Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

github-actions Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

claude-security-review reviewed 9229a190c53128ea46d95da07770c4866a42f6cd: read 1 of 1 changed file(s); 0 finding(s), 0 as inline comments.

Security review (incremental): PR #12897

I found no security issues. This review covers the changes since d27a2dba, and I read the one changed file in full.

  • plugins/actionlint/evals/ck-install-tempted-no-slash/prompt.md: only the eval case's description string changed. The new wording says that granting Glob and Grep does not keep the pass VALID, because the plugin copy sits outside the run workspace and searches of it are denied (fix(actionlint): main's skill text runs the apply flow on a plain-language install request (Opus 5/5) #12251). It also says the case now needs --allow-tools Bash Glob Grep. The text is prose metadata. It does not change the case's allowed_tools, which the previous round reviewed. It also removes no instruction or guardrail, so the instruction-surface lens has nothing to judge.
  • Not touched: this push changes no plugin manifest, trusted path, workflow, token scope or trust control.
  • Spend: none of this change can bill.

critical-findings: 0

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

claude-review: 1 inline finding(s) on d27a2dbae7496ad3bb3ec2e4f7b325f93f529d79; the summary is in this lane's pull request comment.

claude-lane: claude-review

@github-actions

github-actions Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

claude-review reviewed 9229a190c53128ea46d95da07770c4866a42f6cd: read 1 of 1 changed file(s); 0 finding(s), 0 as inline comments.

Code review (incremental): PR #12897, from d27a2db to 9229a19

Scope: 1 of 1 file in the incremental list, read in full: plugins/actionlint/evals/ck-install-tempted-no-slash/prompt.md. I also read the head version of that file. The other 5 changed files have not changed since the last review.

What changed: only the case description (line 2) was edited. It now says the Glob and Grep grant "does not keep the pass VALID", because the plugin copy sits outside the run workspace and its searches are denied (#12251). The run note now reads Needs --allow-tools Bash Glob Grep.

Prior finding: the earlier advisory is resolved. The new text makes both requested corrections:

The owner settled that thread with a recorded VALID-fix-now classification, and this commit is the fix. Nothing in it needs re-raising.

Other checks:

  • No new findings in the incremental change.
  • No plugin manifest changed, so there are no manifest lines.
  • The PR context, settled-threads and prior-findings files contain no embedded directives.
  • The closing-issue assessment from the last round still stands: both criteria are met, the second as amended by the owner's decision. This commit does not affect it.

Round count: this is round 2 of a cap of 3, so the cap did not apply.

…ALID

The ck-install-tempted-no-slash description implied the grant prevents
the INVALID verdict. Attempt 3 on #12251 ran with --allow-tools Bash Glob
Grep and was still INVALID, because the plugin copy sits outside the run
workspace. The run note now names all three tools.

Refs: #12251

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@kyle-sexton
kyle-sexton added this pull request to the merge queue Nov 3, 2026
Merged via the queue into main with commit bbb2563 Nov 3, 2026
55 of 56 checks passed
@kyle-sexton
kyle-sexton deleted the fix/12251-actionlint-apply-rule branch November 3, 2026 18:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(actionlint): main's skill text runs the apply flow on a plain-language install request (Opus 5/5)

1 participant