Skip to content

reflect: sibling-transcript check, template-identical caution, commit-as-you-go - #3

Open
EdbertChan wants to merge 1 commit into
mainfrom
reflect-a71f331-thrash-findings
Open

reflect: sibling-transcript check, template-identical caution, commit-as-you-go#3
EdbertChan wants to merge 1 commit into
mainfrom
reflect-a71f331-thrash-findings

Conversation

@EdbertChan

@EdbertChan EdbertChan commented Aug 17, 2026

Copy link
Copy Markdown
Owner

Summary

Three findings from running /reflect against Invoker's fleet / a71f331 (12 jobs) CI-repair incident (2026-08-17), where at least 4 uncoordinated Invoker workflows independently worked the same failing job over ~7 hours (none landed), and two prior reflect passes on that same incident crashed non-zero before their own drafted skill edits were ever committed.

  • Step 3: check for a sibling reflect's surviving transcript before fanning out lenses. A crashed sibling's real synthesis work is only visible in its raw JSONL (never committed), so without this check a later reflect re-derives the same facts from zero instead of reading it as prior art.
  • Step 3 (lens table): caution for History (and any lens comparing multiple fix attempts) that agreement between commits isn't corroboration when both were rendered from the same static formula/template against the same anchor commit — directly observed as two commits diffing to nothing but SHA/timestamp/task-ID, converging on the same wrong answer (a regression that reversed an already-merged prior fix, uncited by either commit).
  • Step 6: commit each applied skill edit immediately, not batched to the end. Directly observed as the cause of losing a sibling reflect's two drafted CLAUDE.md findings — its closing-summary turn was blocked by an unrelated Stop hook with no further turn, and the edits were never committed.

The step-6 addition also folds in the still-unmerged fix from branch reflect-check-for-unlanded-prior-drafts (check git history for an unlanded prior reflect commit before drafting a new one) rather than leaving it to bit-rot separately, and explicitly notes the gap it doesn't cover (an uncommitted sibling draft with no git trace at all) — which the new step-3 transcript check closes.

Review Claim

Each addition traces to a directly-observed, cited fact from this reflect pass's own investigation (git commits, PR numbers, and a sibling transcript's own tail), not a speculative rewrite.

Test plan

  • Read the full resulting skills/reflect/SKILL.md diff for tone/structure consistency with the rest of the doc.
  • Confirmed via gh pr list that no existing open PR already covers this before opening.
  • No script/test changes in this PR (prose-only); skills/reflect/scripts/tests/ untouched.

🤖 Generated with Claude Code


Note

Low Risk
Prose-only updates to the reflect skill workflow; no runtime, auth, or script changes.

Overview
Process hardening for reflect so lessons aren’t re-derived or lost when passes crash or duplicate each other.

Step 3 adds a pre–lens-fan-out check for sibling reflect transcripts on the same incident (ls + keyword grep under ~/.claude/projects/), with guidance to read a crashed sibling’s tail as prior art—covering cases where nothing was ever committed (complementing step 6’s git-based unlanded-draft search).

A new History lens caution warns that two commits agreeing on the same fix is not independent corroboration when both were rendered from the same static template against the same anchor; reviewers should diff commit bodies (ignoring SHA/timestamp/task IDs) before treating agreement as evidence.

Step 6 instructs checking git for unlanded prior reflect commits on the same skill section before drafting again, and committing each skill edit immediately instead of batching at the end, so Stop-hook blocks or late crashes don’t leave drafted edits only in the transcript.

Reviewed by Cursor Bugbot for commit f6df334. Bugbot is set up for automated code reviews on this repo. Configure here.

…-as-you-go

Three findings from a reflect pass on Invoker's fleet/a71f331 CI-repair
incident (2026-08-17), where at least 4 uncoordinated workflows worked
the same failing job over ~7 hours and two prior reflect passes on the
same incident crashed non-zero before committing their own findings:

- Step 3: check for a sibling reflect's surviving transcript before
  fanning out lenses, so a crashed sibling's real synthesis work (only
  visible in its raw JSONL, never committed) is read as prior art
  instead of re-derived from zero.
- Step 3 (lens table): warn History (and any multi-attempt comparison)
  that commit agreement is not corroboration when both attempts were
  rendered from the same static template against the same anchor --
  directly observed as two commits diffing to nothing but SHA/
  timestamp/ID, converging on the same wrong answer.
- Step 6: commit each applied skill edit immediately, not batched to
  the end -- directly observed as the cause of losing a sibling
  reflect's two drafted CLAUDE.md edits when its closing-summary turn
  was blocked by an unrelated Stop hook with no further turn.

The step 6 addition also folds in the still-unlanded fix from branch
`reflect-check-for-unlanded-prior-drafts` (checking git history for an
unlanded prior reflect commit before drafting a new one) rather than
leaving that finding to bit-rot on its own separate branch, and notes
the residual gap it doesn't cover -- an uncommitted sibling draft --
which the new step 3 transcript check closes.

Co-Authored-By: Claude Sonnet 5 <[email protected]>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit f6df334. Configure here.

Comment thread skills/reflect/SKILL.md

### 3. Spawn parallel reviewers

Before fanning out, check whether a sibling `reflect` already ran on the same incident and might still hold usable work: `ls ~/.claude/projects/ | grep -F <incident-or-branch-keyword>`. This catches a case the step-6 unlanded-draft check (below) structurally can't: a sibling reflect that crashed *before* committing anything (e.g. denied by an unrelated Stop hook, killed mid-turn) leaves no git-log trace at all, but its transcript can still survive worktree teardown if the transcript directory itself wasn't torn down (see the degraded-mode note in step 1). If a sibling transcript exists, read its tail first — a crashed sibling that got as far as drafting synthesis findings (visible in its last assistant turn even with no commit to show for it) is prior art worth extending, not a reason to re-run all 5 lenses from zero. This was directly observed: two independent reflect passes on the same CI-repair incident (Invoker, 2026-08-17) crashed non-zero after completing real synthesis, and a third, unaware of either, re-derived the same facts from scratch before one of the crashed transcripts was found and read.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sibling check searches wrong paths

High Severity

The prescribed sibling check ls ~/.claude/projects/ | grep -F &lt;incident-or-branch-keyword&gt; only matches encoded project directory names. Per step 1 and the skill’s own scanners, transcripts are *.jsonl files inside those directories, and incident keywords usually appear in file content—not directory names. Sibling reflects on the same cwd share one project dir, so this command systematically misses the prior-art transcripts the step exists to find.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit f6df334. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant