Repository navigation
feat(pstack): add /create-formal-verification-skill - #709
Merged
Merged
Conversation
Generates a project-local skill that formally verifies one property of the code where a bug costs the most and tests reach the least. The skill picks the verification tool from the question instead of prescribing one.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit feaf2fd. Configure here.
…ings The proof step said everything must pass again after the scratch change is dropped, and to fix the skill or the verification when a step fails. When the current code already breaks the property, that pointed the agent at weakening the check. Accept a baseline that fails only on findings the agent will report, and limit the fix-it fallback to a missed break or a step that does not run.
poteto
marked this pull request as ready for review
October 18, 2026 06:18
This was referenced Oct 19, 2026
leegmoore
added a commit
to liminal-ai/pstack-lim
that referenced
this pull request
Oct 22, 2026
* fix(pstack): make a brand with a type guard, not as Upstream 0317eb8 (cursor#700), taken unchanged. Co-Authored-By: Claude Opus 5.5 <[email protected]> * docs(pstack): port upstream's docs refresh and docs-vs-tree check Port of upstream 019a6a2 (cursor#704). The guide and README take upstream's plain-language pass, and scripts/check-pstack-docs.mjs comes in as is. Left out: the Claude Code plugin install text and custom-mode wording (Install row), and the generated harness tree. The check found three skills the fork bundles that the README never listed (control-cli, control-ui, deslop, copied from cursor-team-kit), so the README table now lists them. Co-Authored-By: Claude Opus 5.5 <[email protected]> * fix(pstack): list every waiting item so each ketchup call is decidable Port of upstream 16541c6 (cursor#692) and bc14437 (cursor#708). Every item waiting on the user is listed with its pick, evidence and risk, and items are never cut to stay under a word count. Adapted for T3: screenshots and recordings stay inline, and diagrams and charts stay on the html_render page, in place of Mermaid and /visualize (Visuals row). The description keeps the named-only ending without disable-model-invocation (Skill visibility row). Co-Authored-By: Claude Opus 5.5 <[email protected]> * feat(pstack): add the formal verification skills and offer them in setup Port of upstream 4382e68 (cursor#709), f9be7aa (cursor#715), 6b78e9f (cursor#721), 7b353f7 (cursor#716) and the guide line from d457567 (cursor#724). create-formal-verification-skill and maintain-formal-verification-skill come in with upstream's text. setup-pstack gains step 8, the formal verification offer. poteto-help, its recipes, the README and the guide mention both skills. Adapted for T3: both descriptions end with the named-only sentence and drop disable-model-invocation (Skill visibility row). poteto-help keeps the fork's per-user install steps and T3 note, with upstream's new setup offer added to its step 2 (Install row). The guide's Claude Code install line is left out (Install row). Version bumps are left out. Co-Authored-By: Claude Opus 5.5 <[email protected]> * fix(pstack): restore /sharpen to rewriting the prompt Port of upstream 75fabac (cursor#712) and 88d4751 (cursor#729), taken over the fork's do-the-work version at Lee's call in the 2026-10-20 retro. /sharpen asks four questions, then hands back the rewritten prompt and leaves the user's choices open. README, guide and poteto-help follow. Adapted for T3: the description keeps the named-only ending without disable-model-invocation (Skill visibility row). Version bump left out. Co-Authored-By: Claude Opus 5.5 <[email protected]> * docs(fork): log sync 2 and move the sync point to f2bad52 Records each upstream commit from 101b274 to f2bad52 as take, adapt or skip, and adds check-pstack-docs.mjs to the checks a sync runs. Co-Authored-By: Claude Opus 5.5 <[email protected]> * fix(pstack): allow the negative-control break and name the project skill path in the formal verification skills Codex review on PR 24: the maintain skill's edit scope forbade the scratch break its own step 5 requires, and the create skill never said where the project skill lives. Co-Authored-By: Claude Opus 5.5 <[email protected]> * fix(pstack): let the create skill's handoff break the code, and let maintain report updated and bug together Codex review on PR 24 (r2): the create skill forbade the scratch break its own handoff check needs, and the maintain skill asked for one outcome while allowing two. Co-Authored-By: Claude Opus 5.5 <[email protected]> --------- Co-authored-by: Claude Opus 5.5 <[email protected]>
leegmoore
added a commit
to liminal-ai/pstack-lim
that referenced
this pull request
Oct 22, 2026
…te, workflow setup, three contradictions (#25) * sync: upstream cursor/plugins 101b274 to f2bad52 (pstack 0.15.29) (#24) * fix(pstack): make a brand with a type guard, not as Upstream 0317eb8 (cursor#700), taken unchanged. Co-Authored-By: Claude Opus 5.5 <[email protected]> * docs(pstack): port upstream's docs refresh and docs-vs-tree check Port of upstream 019a6a2 (cursor#704). The guide and README take upstream's plain-language pass, and scripts/check-pstack-docs.mjs comes in as is. Left out: the Claude Code plugin install text and custom-mode wording (Install row), and the generated harness tree. The check found three skills the fork bundles that the README never listed (control-cli, control-ui, deslop, copied from cursor-team-kit), so the README table now lists them. Co-Authored-By: Claude Opus 5.5 <[email protected]> * fix(pstack): list every waiting item so each ketchup call is decidable Port of upstream 16541c6 (cursor#692) and bc14437 (cursor#708). Every item waiting on the user is listed with its pick, evidence and risk, and items are never cut to stay under a word count. Adapted for T3: screenshots and recordings stay inline, and diagrams and charts stay on the html_render page, in place of Mermaid and /visualize (Visuals row). The description keeps the named-only ending without disable-model-invocation (Skill visibility row). Co-Authored-By: Claude Opus 5.5 <[email protected]> * feat(pstack): add the formal verification skills and offer them in setup Port of upstream 4382e68 (cursor#709), f9be7aa (cursor#715), 6b78e9f (cursor#721), 7b353f7 (cursor#716) and the guide line from d457567 (cursor#724). create-formal-verification-skill and maintain-formal-verification-skill come in with upstream's text. setup-pstack gains step 8, the formal verification offer. poteto-help, its recipes, the README and the guide mention both skills. Adapted for T3: both descriptions end with the named-only sentence and drop disable-model-invocation (Skill visibility row). poteto-help keeps the fork's per-user install steps and T3 note, with upstream's new setup offer added to its step 2 (Install row). The guide's Claude Code install line is left out (Install row). Version bumps are left out. Co-Authored-By: Claude Opus 5.5 <[email protected]> * fix(pstack): restore /sharpen to rewriting the prompt Port of upstream 75fabac (cursor#712) and 88d4751 (cursor#729), taken over the fork's do-the-work version at Lee's call in the 2026-10-20 retro. /sharpen asks four questions, then hands back the rewritten prompt and leaves the user's choices open. README, guide and poteto-help follow. Adapted for T3: the description keeps the named-only ending without disable-model-invocation (Skill visibility row). Version bump left out. Co-Authored-By: Claude Opus 5.5 <[email protected]> * docs(fork): log sync 2 and move the sync point to f2bad52 Records each upstream commit from 101b274 to f2bad52 as take, adapt or skip, and adds check-pstack-docs.mjs to the checks a sync runs. Co-Authored-By: Claude Opus 5.5 <[email protected]> * fix(pstack): allow the negative-control break and name the project skill path in the formal verification skills Codex review on PR 24: the maintain skill's edit scope forbade the scratch break its own step 5 requires, and the create skill never said where the project skill lives. Co-Authored-By: Claude Opus 5.5 <[email protected]> * fix(pstack): let the create skill's handoff break the code, and let maintain report updated and bug together Codex review on PR 24 (r2): the create skill forbade the scratch break its own handoff check needs, and the maintain skill asked for one outcome while allowing two. Co-Authored-By: Claude Opus 5.5 <[email protected]> --------- Co-authored-by: Claude Opus 5.5 <[email protected]> * fix(retro): record each model a thread ran on, and stop the backup when gitleaks is missing register takes --model <provider>/<model>@<from>, repeatable, where <from> is a run ordinal from t3_thread_read's recentRuns or a date. Every model is kept in order, so a thread that changed lead models shows each one. REGISTRY.md and the round README list them. The backup used to treat a missing gitleaks as no hits and push. It now fails the scan step, pushes nothing, says so on stderr and heads REGISTRY.md with the failure until a write backs up cleanly. Co-Authored-By: Claude Opus 5.5 <[email protected]> * fix(reflect): record what happened and apply nothing Lee's decision in the 2026-10-20 retro: reflect saves and summarizes in a few lines, surfaces only product or ops decisions that can't wait, and applies nothing. Approval, filing and skill edits move to the retro round's open and close. The three reviewer briefs now ask for what happened, with evidence and a classification (bad port, bad application, contradiction, genuine gap), not principles routed to skills. The synthesizer checks and merges those observations by incident instead of sorting proposals into Accepted, Rejected and Backlog. The save step records each model the thread ran on. README, guide and poteto-help follow. Co-Authored-By: Claude Opus 5.5 <[email protected]> * fix(poteto-mode): wake on schedule_task in every lead, and keep resume notes in the store Three installed instructions contradicted the role file and the resume-note hook (2026-10-20 retro, step 1): - autonomous-run step 2, autopilot-full and multi-phase-plan told a Claude lead to use /loop or Monitor. The role file's Waking and polling section says those die when the session restarts, so every lead now uses T3's schedule_task. check-plan.mjs stops requiring the literal /loop 1h marker and requires schedule_task instead. - pause-safely wrote the resume note to /tmp. It now goes to ~/.local/state/pstack/resume/<session>.md, which ~/lim-v2/bin/resume-note-hook reads after a compaction, and session-pickup looks there. autonomous-run also says each heartbeat retests the blocker it reports before repeating it. Co-Authored-By: Claude Opus 5.5 <[email protected]> * feat(poteto-mode): make the plan the record of what done means From the 2026-10-20 retro. The plan already held the verify boxes, the fix bar and the scope, but later stages did not read it back, so deferrals piled into tickets and scope cuts reached Lee as his own discovery. - multi-phase-plan keeps the per-PR verify boxes and adds an optional Verify, model block for a PR whose risk is the order of events (queues, Stop, retries, restarts), run with the project's formal verification skill. A new Appendix E lists scope cuts and splits later work into decided no, each with its reason, and owed. Close the program files tickets only from owed and names every scope cut. - autopilot-full closes out from that list and names the scope cuts in its reply. - bugbot-triage: a deferral that says a race cannot happen carries the model check's result when the plan has a Verify, model block, and review cost alone is not a reason to defer. - check-plan.mjs checks the new blocks and appendix, and prints each PR's verdict box ids. plan-boxes.mjs holds the fence and box rules so the merge gate reads plans the same way. Co-Authored-By: Claude Opus 5.5 <[email protected]> * feat(merge-gate): refuse a verdict that omits a plan box or has an unsigned skip Lee's merge-gate decision in the 2026-10-20 retro: each PR gets a short must-verify list from its plan, the approver may waive an item with a reason on the PR, and findings outside the list stay the approver's call. - shipping step 1: under the pstack-verdict line, the verdict lists each required box of the PR's plan section as done, n/a with a reason, or skipped. A skip needs the lead's signed pstack-skip line on the PR, and until then the verdict is not a pass. - merge-gate takes --plan <file> --plan-pr <id> and adds a Plan boxes requirement. It refuses a verdict that omits a required box, a box line it cannot read, n/a or skipped with no reason, and a skip no one with a role in the repo signed outside the verdict comment. Skips are checked with no plan given too. - Tested on GC2's real verdict (liminal-ai/t3code-v2#18, issuecomment-6262808190), kept verbatim in fixtures/gc2-verdict.md, against a plan fixture with GC2's 16 boxes. The gate refuses it. Run against the real plan in the pstack store, merge-gate --check refuses it the same way. Co-Authored-By: Claude Opus 5.5 <[email protected]> * feat(poteto-mode): hand single-task work back through ketchup The autopilots already end by catching the operator up per the ketchup skill. The build playbooks that hand finished work back did not: autonomous-run, feature, bug-fix, refactoring, perf-issue, hillclimb, visual-parity and shipping now do, keeping each playbook's own content. Read-only and planning playbooks (investigation, the forensics, eval, prototype, multi-phase-plan) keep their replies, since the answer is the reply. Co-Authored-By: Claude Opus 5.5 <[email protected]> * feat(poteto-mode): set up a thread workflow right after picking the playbook Lee's decision in the 2026-10-20 retro: once the playbook is picked and the agent has decided how to use it, the agent sets up a T3 thread workflow from a template or open-ended, fitted to the actual work in a handful of consolidated steps, and tells the user it is set up. A step that waits on the user is asked in chat while the user is there, and the question tool is for getting their attention when they are away. Every workflow ends with a cleanup step. The lead lists what it remembers creating and the scope it worked in, a cleanup subagent scours that scope and deletes what is clearly from the run, and the user is asked only about the rest. The pstack-store todo file stays as the fallback where workflows are off. session-pickup reads the thread's workflow, and FORK.md's Todo list row says so. Co-Authored-By: Claude Opus 5.5 <[email protected]> * test(retro): the backup finds gitleaks through GITLEAKS_PATH or ~/lim/service The verifier's FAIL on #25 showed the installed scanner at ~/lim/service/gitleaks/8.30.1/bin is not on PATH, and GITLEAKS_PATH had no effect. These tests fail at d59628c. Co-Authored-By: Claude Opus 5.5 <[email protected]> * fix(retro): resolve gitleaks from GITLEAKS_PATH, PATH, then ~/lim/service The backup ran bare `gitleaks`, so on a host where the scanner is installed but not on PATH every push stopped at the scan step. A GITLEAKS_PATH that does not run fails closed rather than falling back, and the not-found error names each place it looked. Co-Authored-By: Claude Opus 5.5 <[email protected]> * test(merge-gate): a skip the verifier signs in a separate comment is refused Reproduces the verifier's finding on #25: a Codex PASS+NOTES verdict with perf skipped, then a separate 'pstack-skip: perf by=codex/gpt-6.1-sol' comment, exits 0 with WOULD MERGE at d59628c. Every agent posts as one GitHub account, so only by= tells the verifier from the lead. Also pins that a skip line with no signer never counts. Co-Authored-By: Claude Opus 5.5 <[email protected]> * fix(merge-gate): a skip signed with the verifier's by= does not count findSignedSkips excluded only the verdict comment itself, so the verifier could accept its own skip by posting a second comment. Co-Authored-By: Claude Opus 5.5 <[email protected]> * test(check-plan): plans written before the record of done keep their checks At d59628c the running limhealth-next port plan fails the new Owed, scope cut, schedule_task and Later work requirements, and an existing appendix whose title mentions later work is taken for the new one. These tests fail there: a plan without the template's opt-in keeps the checks it had, the template opts in, and only the exact new title counts. Co-Authored-By: Claude Opus 5.5 <[email protected]> * fix(check-plan): the later-work checks apply only to plans that opt in The template's How to read this now writes "This plan is the record of done.", and only a plan with that line must carry the Owed, scope cut and schedule_task close-out and the Later work appendix. Other plans keep the checks they were written under, /loop 1h included. The appendix is "## Appendix. Later work and scope cuts", found by that exact title, so a plan's own Appendix E, such as the operator's reserved calls, is never read as it. Co-Authored-By: Claude Opus 5.5 <[email protected]> * test(merge-gate): plan box ids, stale box lines and duplicate boxes Codex's four findings on acda474, each failing here: lanes indented in the plan get no ids, bold text inside a verify block drops the boxes after it, box lines above the counted verdict line still count, and a box listed twice passes. Co-Authored-By: Claude Opus 5.5 <[email protected]> * fix(check-plan): check-plan and the merge gate read blocks and boxes with one parser plan-boxes.mjs now owns the block and box reading both use. A block changes only at a recognized block heading, so bold text such as **Linux.** inside a verify block keeps the boxes after it. A box is nested only when it sits inside the list item above it, so lanes indented one to three spaces get their live ids, and check-plan checks lanes, perf items and empty blocks against the same top-level boxes the gate requires. Co-Authored-By: Claude Opus 5.5 <[email protected]> * fix(merge-gate): count only the box lines under the verdict line, each box once Box lines above the counted verdict line belong to an older verdict an edit left behind, so they no longer count toward the current one. A box listed more than once is refused, since the verdict then does not say which status holds. Co-Authored-By: Claude Opus 5.5 <[email protected]> * fix(merge-gate): the lead waives a skipped box on the gate run, not in a PR comment Every agent posts as the same GitHub account, so no PR comment can show that the lead accepted a skip. Two fixes that read a by= field in a comment both failed: a verifier could sign its own skip, and then an older verifier's comment could sign a later verifier's skip. The lead is the only agent that runs the gate to merge, so the waiver moves onto that run: --accept-skip <id>=<reason>, repeatable. The gate accepts a skipped box only when the run waives it with a reason. Before it merges, it posts one PR comment listing each waived box and reason, and a new waived column in runs.tsv records them. An empty reason, a bad id, a repeated id, or a box the verdict did not skip is a usage error. pstack-skip: parsing is gone, and shipping and multi-phase-plan tell the lead to use the flag. Co-Authored-By: Claude Opus 5.5 <[email protected]> --------- Co-authored-by: Claude Opus 5.5 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Why
A test checks the cases that someone thought to write. Races, retries, state machines, parsers, and unsafe code often break on a case that nobody wrote. pstack already has
/create-verification-skill, which proves app behavior with scripted runs. It has no skill that checks a property across every case in a stated scope.What the skill does for you
Call
/create-formal-verification-skillin your repo. If you name an area or a bug, the skill starts there. The skill then does these steps in order:At handoff, the skill tells you what it verified, which tool it used and why, the bounds, what the check found, how to rerun it, and what it does not cover. If the current code breaks the property, you get the counterexample as a short sequence of steps in the code's words. The skill does not weaken the property or the model to get a pass, and it does not change product code unless you ask.
Why the skill stays goal-level
The skill says what a good result looks like. It leaves the choice of tool and model design to the agent, because the right choice depends on the question and the repo. A question about events in a bad order usually suits a model checker over a small model of the design. A question about the code for every input, such as memory safety or overflow, usually suits a verifier that runs on the code. A property that must hold at every size can need a proof. The tool must also be one that the repo's developers can install and run.
One prescribed tool or model shape fits some repos and gives the wrong check in the others. So the skill names the tool families, suggests TLA+ as a common start for concurrent and distributed designs, and asks the agent to explain its choice. What the skill fixes is the bar that every result must meet, whatever the tool:
What changed
pstack/skills/create-formal-verification-skill/SKILL.md. It setsdisable-model-invocation: true, so it runs only when you call it by name.pstack/README.mdand one line inpstack/skills/poteto-help/SKILL.md.harness/pstack/claude-code/.Blast radius
This PR changes only prompt text and docs. No script, runtime code, or existing skill changes.
Verification
node scripts/validate-plugins.mjs,node scripts/check-pstack-docs.mjs, andnode scripts/generate-pstack-harness-trees.mjs --checkpass on the head commit and on a merge with currentmain.pstack/and its copy underharness/pstack/claude-code/are identical.pstack/docs/guidedoes not name the new command yet. Three existing commands get the same note.SKILL.mdon six open-source repos that it never saw:timgit/pg-boss,vercel/swr,brettwooldridge/HikariCP,kubernetes/client-go,DaveGamble/cJSON, andbluss/arrayvec. It compared the skill with an earlier draft that prescribed TLA+ and came with file templates and a worked example. Blinded judges scored each attempt out of 30 on six criteria, such as target choice, tool fit, and checks.Note
Low Risk
Docs and prompt text only; no runtime, scripts, or changes to existing skills.
Reviewed by Cursor Bugbot for commit 3da6ed8. Bugbot is set up for automated code reviews on this repo. Configure here.