Skip to content

feat(session-flow): workflow probes git for missing files before asking - #12680

Merged
kyle-sexton merged 5 commits into
mainfrom
fix/12630-replacement
Nov 3, 2026
Merged

kyle-sexton merged 5 commits into
mainfrom
fix/12630-replacement

Conversation

@kyle-sexton

Copy link
Copy Markdown
Member

Closes #12594

Summary

Replaces #12630, opened by the work-resolve-issue lane. Its pr-unblock run stopped with feedback-unadjudicated: claude-review's thread asked for an eval case covering the explore stage's missing-file gate, and the lane could not add one because the case needs a scaffold.sh, which is outside the lane's write scope. This branch carries #12630's change unchanged (merged in from its head a4d96230) plus that eval case.

The /session-flow:workflow explore stage now probes git status and git log --diff-filter=D for a referenced file that is not on disk before asking: a staged or committed delete is skipped and cited, a working-tree-only delete is investigated, and it asks only when neither probe settles the path.

The review's other finding (the wording must match the explore skill's slice for the same gate) is met on main: #12607 merged, and plugins/discovery/skills/explore/reference/workflow.md:41 gives the same two commands and the same three outcomes.

Fix

  • plugins/session-flow/skills/workflow/context/steps.md: the explore stage's missing-file rule (from feat(session-flow): workflow probes git for missing files before asking #12630).
  • .changes/session-flow/…-93e14bf0.md: feat(session-flow): workflow probes git for missing files before asking #12630's changelog fragment, kept as is.
  • plugins/session-flow/evals/wf-explore-probes-git-before-asking/: new case. The scaffold builds a repository whose src/report.py imports three exporters that are off disk: one removed by a commit, one staged for removal (git rm), one removed only from the working tree (rm). Graders check that the run calls git status and git log --diff-filter=D, investigates the working-tree-only removal with git show/diff, cites the staged removal and the deleting commit, and asks no question about investigating any of the three.

Verification

  • python3 plugins/evals/skills/validate/scripts/validate-cases.py plugins/session-flow/evals: no FAIL; the new case's only WARN is the Bash-needs---allow-tools note every sibling Bash case carries. All six graders sort their samples correctly under Python and JavaScript.
  • The scaffold run in a scratch directory gave D src/exporters/csv_out.py, D src/exporters/xml_out.py, and git log --diff-filter=D named the json commit.
  • shfmt -d and shellcheck on scaffold.sh: clean. markdownlint-cli2 on steps.md and the fragment: 0 issues.

Not verified: the case was checked statically only; no paid claude plugin eval run.

Related

🤖 Generated with Claude Code

https://claude.ai/code/session_01Vcupom3BKF6QuC5U3sRJPj

melodic-automation-lanes Bot and others added 3 commits November 3, 2026 03:08
Lane-Activity: work-resolve-issue/resolve-issue
Add wf-explore-probes-git-before-asking: a scaffolded repository whose
src/report.py imports three exporters that are off disk, one removed by
a commit, one staged for removal and one removed only from the working
tree. Graders check that the run probes git status and git log
--diff-filter=D, cites the commit and the staged removal, investigates
the working-tree-only removal, and asks no question about the three.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Vcupom3BKF6QuC5U3sRJPj
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-03T04:18:02.696620Z ecd54e8 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

claude-review: 1 inline finding(s) on ecd54e85c635646319c0aec4201fa2074867fbe6; the summary is in this lane's pull request comment.

claude-lane: claude-review

@github-actions

github-actions Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

claude-review reviewed a4dea37b59de4ff0d07aab6a9e74d1a619f786dc: read 2 of 2 changed file(s); 1 finding(s), 0 as inline comments.

Code review: PR #12680 (incremental since 2877c332)

I read every hunk of both files on the incremental list. I also read the head case.yaml and the head samples/no-question-on-missing-files.json.

What changed. This push fixes the inline finding from the last round (case.yaml:49-53: a correct one-line yaml_out question that also named a removed exporter failed the case).

  • no-question-on-missing-files now begins with a (?![^\n]*yaml_out) lookahead.
  • The pass samples gain the two one-line examples from that finding.

Correctness, checked by reading only. I have no shell, so I did not run the graders.

  • Both new pass samples contain yaml_out, so the negative lookahead blocks a match and the not_contains grader passes them.
  • The three fail samples do not name yaml_out, so they still match.
  • The four earlier pass samples still behave as before. In json_out.py was removed on purpose.\nShould I investigate yaml_out.py…, the first line has no ? and the second line names yaml_out.
  • untracked-import-asked (^(?=[^\n]*yaml_out)[^\n]*\?) still matches both new one-line samples, so the two graders no longer conflict on them.

The prior inline finding is fixed, so I did not post it again.

Note, not counted as a finding. This push took the lookahead option the last finding offered. As a result, any line that names yaml_out is exempt from no-question-on-missing-files, including a line that also asks to investigate a removed exporter. For example, Should I investigate xml_out.py, csv_out.py and yaml_out.py? now passes this grader. No fail sample records that trade-off. It is an accepted design choice, not a defect.

Findings

  1. (restated summary finding from earlier rounds, advisory) Closing issue feat(session-flow): workflow probes git for missing files instead of asking before investigating #12594 says: "The wording matches the explore skill's slice for the same gate." Judged by reading only, the wording meets it. But no test in the PR or the repository compares steps.md#L34 with plugins/discovery/skills/explore/reference/workflow.md:41, so either file can drift and nothing fails. This push does not change that.

Spend. This push adds no graders and changes no turn or timeout settings, so the case's prepaid API credit use is unchanged. That is not a hold.

The PR context, the settled threads and the prior findings contain no embedded instructions. Prior-findings round count: 2 of 3 before this review, not past the cap.

@github-actions

github-actions Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

claude-security-review reviewed a4dea37b59de4ff0d07aab6a9e74d1a619f786dc: read 2 of 2 changed file(s); 0 finding(s), 0 as inline comments.

Security review (incremental since 2877c332): no security issues found

I read every change in both files on the incremental list. Neither file matches a trusted-path glob. This push doesn't touch any plugin manifest, workflow, token, trusted-actor list, merge hold or instruction file.

What changed.

  • case.yaml: the no-question-on-missing-files grader's regex gains a negative lookahead, (?![^\n]*yaml_out), so a line that asks about the never-tracked yaml_out no longer counts as a question about the three deleted exporters. Graders are data that the eval harness matches against model output, and they run nothing.
  • samples/no-question-on-missing-files.json: two pass samples are added. They are static grader test data.

The eval's execution block, including its tool allowances, is unchanged in this push.

Spend: none is added. The change only adjusts how existing output is graded.

Untrusted text: the PR context contains no instructions aimed at this review.

Prior findings: the lane's only earlier post was a no-findings summary, so nothing carries over.

critical-findings: 0

…robe eval

report.py now also imports exporters.yaml_out, which git never tracked, so
neither probe names it. Two graders cover the "Not settled" branch: a
question naming yaml_out, and no git show/diff/cat-file or log -p on it
before that question.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Vcupom3BKF6QuC5U3sRJPj

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

claude-review: 1 inline finding(s) on 2877c3320cb3e442e10357da36c707eacc0be728; the summary is in this lane's pull request comment.

claude-lane: claude-review

…on line

A line asking about the never-tracked yaml_out may also name the three
probed exporters; no-question-on-missing-files now skips lines naming
yaml_out, with both one-line summaries as pass samples.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Vcupom3BKF6QuC5U3sRJPj
@kyle-sexton
kyle-sexton added this pull request to the merge queue Nov 3, 2026
Merged via the queue into main with commit a87baac Nov 3, 2026
43 checks passed
@kyle-sexton
kyle-sexton deleted the fix/12630-replacement branch November 3, 2026 06:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(session-flow): workflow probes git for missing files instead of asking before investigating

1 participant