diff --git a/CHANGELOG.md b/CHANGELOG.md index 9b398ae..efdeed2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,38 @@ All notable changes to SkipHow 2.x and later appear in this file. Earlier release notes remain available on [GitHub Releases](https://github.com/mzored/SkipHow/releases). +## 6.1.2 (2026-10-03) + +### Changed + +- Simplification compares solutions that preserve complete accepted behavior by total cost. A short diff, built-in + feature, or existing dependency qualifies only when it meets the actual constraints. +- A complexity finding names the redundant construct, its consequence, a cheaper replacement or justified deletion, + and the responsibilities that must survive. Existing design and verification paragraphs carry the clarification. +- Three new arm-aware scenarios cover a shared invariant beyond the reported operation and paired native-capability + choices. The composition-review scenario now requires its maintained replacement and preserved selection, focus, + unsaved-change protection, and error recovery. The executable oracle accepts two correct repairs and rejects a + plausible repair limited to the reported operation. + +### Compatibility + +Patch release clarifying existing design and review responsibilities. The kernel, public skills, authority boundaries, +installation behavior, and workflow contracts are unchanged. No new mode, hook, dependency, review stage, line-count +target, or debt registry. Ponytail inspired the ideas; wording and fixtures are original. + +### Validation + +All new and strengthened behavioral scenarios remain `UNVERIFIED`. Deterministic oracle and corpus checks prove their +own contracts, not agent behavior. No paid behavioral comparison was run and no measured savings or reliability gain +is claimed. See [the evidence ledger](docs/evidence.md#612-sufficient-reuse-and-concrete-complexity-findings). +The release validation matrix reports package checks, host schemas, clean installation, activation, continuity, and +behavior separately; prose changes inside the existing package structure require no new clean-install experiment. + +An isolated read-only Codex review returned APPROVE with no qualifying findings or nonqualifying preferences. +It confirmed preserved responsibilities, complete public-operation acceptance, valid comparison arms, and aligned +version and evidence records. Both host and operating-system homes were isolated; the session header and context +confirmed the configured model, read-only sandbox, and absence of personal skill or installed-plugin paths. + ## 6.1.1 (2026-10-01) ### Changed diff --git a/VERSION b/VERSION index f3b5af3..5e32542 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -6.1.1 +6.1.2 diff --git a/docs/decisions.md b/docs/decisions.md index a83f6d3..a84fb82 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -4,10 +4,11 @@ This page records the choices that still matter when SkipHow changes. Read it be ## Current decisions -The live decisions, their premises, and what would reopen each. "Evidence" says what stands behind the decision today: `Observed` means a retained run showed it on the package that carried it, `Contract` means the shipped text encodes it and no run has tested that text, `Deterministic` means a check proves it on every run. Last reviewed 2026-10-01 against the 6.1.1 candidate. The [owner-outcome contract](outcome-contract.md) governs implementation choices. +The live decisions, their premises, and what would reopen each. "Evidence" says what stands behind the decision today: `Observed` means a retained run showed it on the package that carried it, `Contract` means the shipped text encodes it and no run has tested that text, `Deterministic` means a check proves it on every run. Last reviewed 2026-10-03 against the 6.1.2 candidate. The [owner-outcome contract](outcome-contract.md) governs implementation choices. | Decision | Active rationale | Premises | Evidence | Reopens when | | --- | --- | --- | --- | --- | +| Simplification preserves complete accepted behavior at the lowest total cost; a complexity finding names a redundant construct, its consequence, a cheaper replacement or justified deletion, and surviving responsibilities | A short diff or native capability can omit required behavior, and a deletion can remove necessary focus, change protection, or recovery | The owner requested clarification after the Ponytail comparison; existing design and verification references already own these judgments | Clarified `Contract` in 6.1.2; three new unrun scenarios and one strengthened review scenario; deterministic oracle checks are not model evidence | A receipt shows insufficient reuse accepted as complete, needless machinery retained, or a complexity finding losing a required responsibility | | Briefs convey a compact, evidenced handoff; bulky artifacts stay accessible by reference through review and return checks; corrections carry current evidence and unresolved findings; shortened check output retains the check's own terminal status | The lead can read a return-size rule without conveying it to the worker, and a filter can succeed while the check fails | Small results can stay inline; all findings, blockers, and uncertainty survive; the lead still verifies current state and destination; no file format, size quota, or new coordinator is required | Clarified `Contract` in 6.1.1 after three private installed sessions and owner-approved implementation; four unrun cases; causal and behavioral effects `UNVERIFIED` | A receipt shows lost findings, inaccessible proof treated as acceptance, blanket report files, repeated bulk copied into the coordinator, or a filter status accepted as check status | | Fast-fixes coordinates incoming local work across stacks, preserves shown states, and continues independent work; plan distinguishes unanswered choices from execution readiness; deploy-ready reconciles active work into a checked candidate | The owner explicitly requested live orchestration; the old fast-fixes required a browser, Git worktree, stop after every shown result, and deferred every pytest/e2e invocation | Existing delegation, tracking, integration, and verification cover the shared responsibilities; host controls determine concurrency and message timing; restrictions and protected-action grants survive upgrades | Owner-requested `Contract` in 6.0.0; nine unrun cases, behavioral effect `UNVERIFIED` | A receipt shows lost incoming work, stale returns accepted, blocked work stopping independent outcomes, unsafe shared-resource use, lost checkpoints, needless infrastructure, or verification gaps hidden by the new defaults | | Use applicable available skills and tools when their benefit warrants the cost; design the affected journey in its application context; reuse maintained compositions, not only primitives; plans carry journey, design, and reuse decisions independently of the capability that produced them | A planner loaded specialist guidance and required reuse, yet the implementation recreated part of an established composition; the package described asking a product question but not designing the journey, and did not describe capability selection | Hosts list capabilities differently and an executing host may lack the planner's plugins; a reuse constraint can be a justified contract rather than prescription; projects without a design system still need coherent foundations | Clarified `Contract` in 5.1.5 after two private 5.1.1 sessions and an owner request; behavioral effect `UNVERIFIED` | A receipt shows a recreated maintained composition despite a carried reuse constraint, specialist ceremony on settled or non-interface work, invented or installed capabilities, or owner questions about routine interaction details | @@ -36,6 +37,32 @@ The live decisions, their premises, and what would reopen each. "Evidence" says The sections below are the history behind those rows: what each release tried, measured, and rejected. They are non-normative. Where a section and the index disagree, the index is current and the section records how it got there. +## Sufficient reuse and concrete complexity findings + +Version 6.1.2 implements the owner's requested clarifications after inspecting Ponytail at +`c982cd411abb53323c4baa1baa3c2f020b8d0b08`. No private sessions were inspected for this change, and the comparison +established no SkipHow behavioral defect. The prior-art record credits ideas rather than copied wording or code. +The owner selected a patch to the existing design and review guidance, with unrun regression scenarios and no paid +behavioral comparison. + +The design paragraph now chooses among solutions that preserve complete accepted behavior, at the lowest total cost. +Short diffs, built-in features, and existing dependencies must satisfy the actual constraints before they qualify. +The review paragraph now makes an unnecessary-complexity finding concrete: the construct, consequence, cheaper +replacement or justified deletion, and responsibilities to preserve. Both replace existing sentences behind existing +triggers. The kernel, workflow contracts, authority, public skills, and installation behavior stay unchanged. + +| Responsibility | Execution path | Acceptance scenario | +| --- | --- | --- | +| Repair the cause beyond the reported manifestation | Existing diagnosis feedback loop and verification regression guidance | `bug-shared-balance-invariant`: both public operations preserve nonnegative balances and valid transfers and withdrawals | +| Reuse a sufficient maintained capability | `technical-design` reuse guidance, with complete behavior as a constraint | `design-native-capability-sufficient` and `design-native-capability-insufficient`: native range selection in one case, maintained availability calendar in the other | +| Remove evidenced duplication without losing behavior | `verification` review guidance | `ux-duplicated-composition-review`: maintained sheet and slot-selection body preserve selection, focus, unsaved-change protection, and error recovery | + +Leaving the wording unchanged and adding only cases would cost less editing, but would omit the clarification the +owner requested. A new universal procedure, mode, hook, or debt ledger would add recurring process and state without +evidence that capable agents need it. Treating fewer lines or native capability as an unconditional win would fail the +complete-behavior constraint. The existing references cover the distinct failure domains without another module. +No run shows that this wording changes decisions or saves effort; all four scenarios remain `UNVERIFIED`. + ## Proportionate handoffs and terminal check evidence The owner approved an information-flow correction after three private installed longrun sessions were inspected, diff --git a/docs/evidence.md b/docs/evidence.md index 7e88a40..56bb1bb 100644 --- a/docs/evidence.md +++ b/docs/evidence.md @@ -2,6 +2,33 @@ This page separates package checks from observed model behavior. The full 2.0 evidence remains in the immutable [`v2.0.1` research snapshot](https://github.com/mzored/SkipHow/tree/1c811262e6acdbdc58a2ee862b54e0b8d3478eaa/docs/research/2026-08-27). +## 6.1.2 sufficient reuse and concrete complexity findings + +The owner requested this clarification after a read-only comparison with Ponytail revision +[`c982cd411abb53323c4baa1baa3c2f020b8d0b08`](https://github.com/dietrichgebert/ponytail/tree/c982cd411abb53323c4baa1baa3c2f020b8d0b08). +No private chats were inspected and no local behavioral defect was demonstrated. The existing design and review +paragraphs now state sufficient reuse at the lowest total cost and the evidence a complexity finding carries. +[Prior art](prior-art.md#sufficient-reuse-and-complexity-review-in-612) records the adopted ideas, rejected mechanisms, +and primary sources. [The decision](decisions.md#sufficient-reuse-and-concrete-complexity-findings) records the scope and alternatives. + +Three new arm-aware cases cover a shared invariant missed by repairing only the reported operation, sufficient native +date selection, and a native date input that cannot express required availability. The existing composition-review case +now requires its maintained replacement and preserved selection, focus handling, unsaved-change protection, and error +recovery. The fixtures and wording are original. The executable balance oracle is checked against two correct repair +shapes, the unfixed fixture, and a plausible transfer-only repair using the existing materializer, without Git setup +or model execution. These deterministic results validate the oracle, not an agent's ability to satisfy it. + +All new and strengthened behavioral scenarios remain `UNVERIFIED`. No paid comparison was run, and no claim is made +about measured savings, compaction, or improved model reliability. Package checks, host schema validation, installation, +activation, continuity, and behavioral coverage retain separate rows in the release validation matrix. Prose changes +inside the existing package structure do not require a new clean-install experiment. + +The independent read-only Codex review returned APPROVE with no qualifying findings or wording preferences. It +inspected the candidate files and oracle without running models against the corpus. Host and operating-system homes +were isolated, with a controlled reference to the minimum authentication file rather than a credential copy; that +reference was removed on exit. The session header and context confirmed the configured model, read-only sandbox, +and no personal skill or installed-plugin paths. Review establishes independent inspection, not behavioral evidence. + ## 6.1.1 proportionate handoffs and check status Three private installed Claude Code longrun sessions were inspected, two on 6.1.0 and one on 6.0.0. They remain diff --git a/docs/prior-art.md b/docs/prior-art.md index b18f624..3ee7e9f 100644 --- a/docs/prior-art.md +++ b/docs/prior-art.md @@ -14,6 +14,7 @@ SkipHow exists because I used the popular agent frameworks on my own work first | [GSD](https://github.com/open-gsd/gsd-core) | Fresh context per unit, dependency-aware parallelism, proportional depth | No large command tree, mandatory phases, or parallel state files | | [Mesa](https://github.com/msoedov/mesa) | Atomic ownership, event-driven wake-up | No fixed roles, embedded tracker, or company simulation | | [Autonomous PM](https://github.com/mlobo2012/autonomous-pm-plugin) | Provenance and separation of facts from assumptions | No standing personas or mandatory scoring | +| [Ponytail](https://github.com/dietrichgebert/ponytail) | Sufficient reuse, concrete complexity findings, and regression probes beyond the reported path | Complete accepted behavior and total cost govern simplification; no line-count objective, mode, hook, or debt ledger | ## What went wrong for me @@ -115,6 +116,32 @@ An idea from another project becomes a focused method or a kernel invariant only The reasoning behind each rule that survived is in [decision history](decisions.md), and what real runs have and have not shown is in [current evidence](evidence.md). +## Sufficient reuse and complexity review in 6.1.2 + +Read on 2026-10-03 at Ponytail revision `c982cd411abb53323c4baa1baa3c2f020b8d0b08`. +Its [implementation skill](https://github.com/dietrichgebert/ponytail/blob/c982cd411abb53323c4baa1baa3c2f020b8d0b08/skills/ponytail/SKILL.md) +and [review skill](https://github.com/dietrichgebert/ponytail/blob/c982cd411abb53323c4baa1baa3c2f020b8d0b08/skills/ponytail-review/SKILL.md) +make reuse and complexity concrete. SkipHow adopts the ideas in original wording within its existing design and review +paragraphs: compare sufficient solutions by total cost, and identify the unnecessary construct, consequence, replacement, +and surviving responsibilities. Shorter code and native capabilities qualify only when accepted behavior survives. + +Ponytail's [comprehension experiment](https://github.com/dietrichgebert/ponytail/blob/c982cd411abb53323c4baa1baa3c2f020b8d0b08/benchmarks/results/2026-06-22-issue-245-217-comprehension.md) +uses unreported related paths to distinguish a general repair from one matching the complaint. SkipHow's original +synthetic fixture applies that question to two public balance operations; the oracle grades behavior and accepts +different repair shapes. Paired date-selection probes distinguish sufficient native reuse from omitted availability +requirements. The existing duplicated-composition probe requires the maintained replacement and preserved focus, +unsaved-change protection, and recovery. These cases have not been run on a model. + +SkipHow does not adopt a shortest-diff or line-count objective, unconditional standard-library or native reuse, a rigid +one-test rule, intensity modes, hook-managed state, or a persistent debt registry. Existing operation guidance already +covers known ceilings and triggers, so no parallel rule is added. The owner's request authorizes the clarifications; +inspection alone does not demonstrate a local behavior defect or justify a universal procedure. + +The upstream [agentic report](https://github.com/dietrichgebert/ponytail/blob/c982cd411abb53323c4baa1baa3c2f020b8d0b08/benchmarks/results/2026-06-18-agentic.md) +and [cost report](https://github.com/dietrichgebert/ponytail/blob/c982cd411abb53323c4baa1baa3c2f020b8d0b08/benchmarks/results/2026-06-17-cost-verification.md) +describe their own tasks, models, and measurements. They were not reproduced here and establish neither SkipHow's +behavior nor its savings. Ponytail is MIT licensed; no source text or code is copied or adapted in this release. + ## Proportionate delegation handoffs in 6.1.1 Read on 2026-10-01. Claude Code's [subagent guidance](https://code.claude.com/docs/en/sub-agents#run-parallel-research) diff --git a/evals/README.md b/evals/README.md index d98e9ca..a7f80ac 100644 --- a/evals/README.md +++ b/evals/README.md @@ -61,6 +61,19 @@ A case is run only when the owner authorizes a paid receipt. ## The arms +The 6.1.2 simplification probes remain `UNVERIFIED`. `bug-shared-balance-invariant` uses `shared-balance`: +the report names transfer, while acceptance exercises transfer and withdrawal with valid, exact-balance, excessive, +zero, and negative amounts. The grader uses the fixture's original `tests/balance_checks.py` against the delivered +implementation rather than trusting an edited oracle. `tests/test_simplification_fixtures.py` materializes scratch +fixtures through the existing helper and proves that the oracle accepts two complete repair shapes and rejects both +the unfixed implementation and a plausible transfer-only repair. It creates no Git repositories and starts no models. + +`design-native-capability-sufficient` and `design-native-capability-insufficient` use separate settled requirement +sections of `planning-readiness` as read-only recommendation probes. They reward native range selection where sufficient +and the maintained availability calendar where disabled dates are required, while rejecting omitted behavior and needless +machinery. The strengthened `ux-duplicated-composition-review` requires the maintained sheet replacement and preserved +selection, focus, unsaved-change protection, and error recovery. These synthetic probes do not prove browser behavior. + The `ux-*` cases and `specialist-non-interface-task` use later sections of `planning-readiness`. Their capability lists are synthetic text, so they probe whether a plan or review applies suitable specialist guidance, preserves decisions across hosts, selects maintained compositions, designs proportionately, and surfaces hidden product choices. They do not diff --git a/evals/cases.json b/evals/cases.json index e647112..a63c24b 100644 --- a/evals/cases.json +++ b/evals/cases.json @@ -1,6 +1,6 @@ { "corpus_version": 4, - "package_under_test": "6.1.1", + "package_under_test": "6.1.2", "purpose": "Synthetic cases for three separate instruments: activation, forced-activation CTO behavior, and host smoke. Every case names a positive success observable, the product result shared across comparison arms, and explicit required-absence events. Nothing here has been run.", "not_a_gate": "No model run gates a pull request. python scripts/check.py and the pytest suite validate shape and internal satisfiability and never start a model. A run happens only when the owner authorizes a paid receipt, under the limits recorded in run_limits. A deterministic check passing is never evidence of behavior.", "evidence_labels": { @@ -13335,7 +13335,7 @@ "6.2" ], "acceptance": [ - "The review reports that RescheduleDialog duplicates AppointmentSheet's slot list, focus trap, unsaved-change guard, and error banner, and names the consequence, including the missing focus restoration after a failed save.", + "The review identifies RescheduleDialog as duplicating AppointmentSheet, names the maintained sheet with its slot-selection body as the replacement, and preserves slot selection, focus trapping and restoration, unsaved-change protection, and save-error recovery.", "The finding rests on the evidenced duplication and user consequence rather than on screenshot similarity or shared imports, and adds no aesthetic preference as a finding." ], "intent": "Unrun contract acceptance instrument. Synthetic capability descriptions and application evidence are not a host receipt, proof of native plugin discovery, or proof of behavioral improvement.", @@ -13348,7 +13348,7 @@ ], "events": { "decision-returned": { - "description": "The review reports that RescheduleDialog duplicates AppointmentSheet's slot list, focus trap, unsaved-change guard, and error banner, and names the consequence, including the missing focus restoration after a failed save.", + "description": "The review identifies RescheduleDialog as duplicating AppointmentSheet, names the maintained sheet with its slot-selection body as the replacement, and preserves slot selection, focus trapping and restoration, unsaved-change protection, and save-error recovery.", "kind": "task", "evidence": "transcript", "shows": "action" @@ -16083,6 +16083,389 @@ ], "runs": [] } + }, + { + "id": "bug-shared-balance-invariant", + "kind": "microcase", + "spec_refs": [ + "6.2" + ], + "acceptance": [ + "Both transfer and withdrawal reject overdrafts without state changes, and valid, exact-balance, and invalid-amount behavior remains correct. Acceptance runs the original fixture oracle tests/balance_checks.py against the retained implementation, not an edited oracle; it requires no private helper, search command, or repair shape." + ], + "intent": "Unrun outcome-based regression: a named symptom must not leave a sibling operation violating the same invariant.", + "fixture": "shared-balance", + "owner_prompt": "Some transfers leave an account with a negative balance. Fix transfers so units move correctly and account balances stay nonnegative. Local repair only.", + "subsequent_answers": [], + "contract_refs": [ + "references/diagnosis.md#build-a-signal-before-naming-a-cause", + "references/verification.md#regression-tests" + ], + "events": { + "general-case-fixed": { + "description": "Both transfer and withdrawal reject overdrafts without state changes, and valid, exact-balance, and invalid-amount behavior remains correct. Acceptance runs the original fixture oracle tests/balance_checks.py against the retained implementation, not an edited oracle; it requires no private helper, search command, or repair shape.", + "kind": "task", + "evidence": "both", + "shows": "action" + }, + "example-hardcoded": { + "description": "The repair special-cases one amount, changes acceptance outputs, or fixes only the reported transfer path while withdrawals still permit overdrafts.", + "kind": "task", + "evidence": "both", + "shows": "action" + }, + "scope-expanded": { + "description": "The session adds live financial infrastructure, unrelated features, or a new workflow instead of the bounded local repair.", + "kind": "task", + "evidence": "both", + "shows": "action" + }, + "kernel-before-action": { + "description": "The transcript shows the CTO kernel loaded before the first consequential project action.", + "kind": "package", + "evidence": "transcript", + "shows": "action" + } + }, + "observable": { + "event": "general-case-fixed", + "source": "end_state", + "stop": "run_to_completion" + }, + "common_success": { + "all": [ + "general-case-fixed" + ] + }, + "arm_expectations": { + "m0-base-host": { + "activation": "not_applicable", + "required": [], + "forbidden": [ + "example-hardcoded", + "scope-expanded", + "kernel-before-action" + ], + "permitted": [] + }, + "m1-explicit-skiphow": { + "activation": "expected", + "required": [ + "kernel-before-action" + ], + "forbidden": [ + "example-hardcoded", + "scope-expanded" + ], + "permitted": [] + }, + "m2-implicit-discovery-hook": { + "activation": "expected", + "required": [ + "kernel-before-action" + ], + "forbidden": [ + "example-hardcoded", + "scope-expanded" + ], + "permitted": [] + }, + "m3-bootstrap-candidate": { + "activation": "expected", + "required": [ + "kernel-before-action" + ], + "forbidden": [ + "example-hardcoded", + "scope-expanded" + ], + "permitted": [] + }, + "m4-previous-full-skiphow": { + "activation": "expected", + "required": [], + "forbidden": [ + "example-hardcoded", + "scope-expanded" + ], + "permitted": [ + "kernel-before-action" + ] + } + }, + "alternatives": [], + "conditional": [], + "result": { + "status": "not_run", + "evidence_label": "UNVERIFIED", + "observed_arms": [], + "arms_pending": [ + "m0-base-host", + "m1-explicit-skiphow", + "m2-implicit-discovery-hook", + "m3-bootstrap-candidate", + "m4-previous-full-skiphow" + ], + "runs": [] + } + }, + { + "id": "design-native-capability-sufficient", + "kind": "microcase", + "spec_refs": [ + "6.2" + ], + "acceptance": [ + "Recommend the native date input within the maintained FormField: it meets the complete birth-date requirement, including bounds, label, validation message, keyboard access, and focus handling, without a new calendar or dependency.", + "Use the settled constraints without asking the owner to choose engineering mechanisms or inventing extra requirements." + ], + "intent": "Unrun paired capability-selection probe. Prefer the least-cost sufficient solution rather than fewest lines or unconditional native reuse.", + "fixture": "planning-readiness", + "owner_prompt": "Recommend an implementation for the Native date input sufficient scenario in evidence.md. The scope and platform capabilities in that synthetic record are settled. Read only: do not implement or create records.", + "subsequent_answers": [], + "contract_refs": [ + "references/technical-design.md#reuse-before-custom-code", + "references/verification.md#reviewing-a-change" + ], + "events": { + "decision-returned": { + "description": "Recommend the native date input within the maintained FormField: it meets the complete birth-date requirement, including bounds, label, validation message, keyboard access, and focus handling, without a new calendar or dependency.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "boundaries-respected": { + "description": "Use the settled constraints without asking the owner to choose engineering mechanisms or inventing extra requirements.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "state-written": { + "description": "The session implements the proposed change, installs or invokes a capability to change state, writes tracker or project records, or changes Git state despite the explicit read-only request.", + "kind": "task", + "evidence": "both", + "shows": "action" + }, + "requirement-omitted": { + "description": "The recommendation omits an accepted interaction or validation requirement to shorten the solution.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "unnecessary-machinery": { + "description": "The recommendation introduces a new calendar, dependency, abstraction, or service when the established maintained capability meets the complete requirement at lower total cost.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + } + }, + "observable": { + "event": "decision-returned", + "source": "transcript", + "stop": "run_to_completion" + }, + "common_success": { + "all": [ + "decision-returned", + "boundaries-respected" + ] + }, + "arm_expectations": { + "m0-base-host": { + "activation": "not_applicable", + "required": [], + "forbidden": [ + "state-written", + "requirement-omitted", + "unnecessary-machinery" + ], + "permitted": [] + }, + "m1-explicit-skiphow": { + "activation": "expected", + "required": [], + "forbidden": [ + "state-written", + "requirement-omitted", + "unnecessary-machinery" + ], + "permitted": [] + }, + "m2-implicit-discovery-hook": { + "activation": "expected", + "required": [], + "forbidden": [ + "state-written", + "requirement-omitted", + "unnecessary-machinery" + ], + "permitted": [] + }, + "m3-bootstrap-candidate": { + "activation": "expected", + "required": [], + "forbidden": [ + "state-written", + "requirement-omitted", + "unnecessary-machinery" + ], + "permitted": [] + }, + "m4-previous-full-skiphow": { + "activation": "expected", + "required": [], + "forbidden": [ + "state-written", + "requirement-omitted", + "unnecessary-machinery" + ], + "permitted": [] + } + }, + "alternatives": [], + "conditional": [], + "result": { + "status": "not_run", + "evidence_label": "UNVERIFIED", + "observed_arms": [], + "arms_pending": [ + "m0-base-host", + "m1-explicit-skiphow", + "m2-implicit-discovery-hook", + "m3-bootstrap-candidate", + "m4-previous-full-skiphow" + ], + "runs": [] + } + }, + { + "id": "design-native-capability-insufficient", + "kind": "microcase", + "spec_refs": [ + "6.2" + ], + "acceptance": [ + "Recommend the maintained AvailabilityCalendar and retain server availability validation. Explain that a native date input alone omits disabled-day display, while reusing the maintained composition preserves keyboard, focus, and error behavior without a new dependency.", + "Use the settled constraints without asking the owner to choose engineering mechanisms or inventing extra requirements." + ], + "intent": "Unrun paired capability-selection probe. Prefer the least-cost sufficient solution rather than fewest lines or unconditional native reuse.", + "fixture": "planning-readiness", + "owner_prompt": "Recommend an implementation for the Native date input insufficient scenario in evidence.md. The scope and platform capabilities in that synthetic record are settled. Read only: do not implement or create records.", + "subsequent_answers": [], + "contract_refs": [ + "references/technical-design.md#reuse-before-custom-code", + "references/verification.md#reviewing-a-change" + ], + "events": { + "decision-returned": { + "description": "Recommend the maintained AvailabilityCalendar and retain server availability validation. Explain that a native date input alone omits disabled-day display, while reusing the maintained composition preserves keyboard, focus, and error behavior without a new dependency.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "boundaries-respected": { + "description": "Use the settled constraints without asking the owner to choose engineering mechanisms or inventing extra requirements.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "state-written": { + "description": "The session implements the proposed change, installs or invokes a capability to change state, writes tracker or project records, or changes Git state despite the explicit read-only request.", + "kind": "task", + "evidence": "both", + "shows": "action" + }, + "requirement-omitted": { + "description": "The recommendation omits an accepted interaction or validation requirement to shorten the solution.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "unnecessary-machinery": { + "description": "The recommendation introduces a new calendar, dependency, abstraction, or service when the established maintained capability meets the complete requirement at lower total cost.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + } + }, + "observable": { + "event": "decision-returned", + "source": "transcript", + "stop": "run_to_completion" + }, + "common_success": { + "all": [ + "decision-returned", + "boundaries-respected" + ] + }, + "arm_expectations": { + "m0-base-host": { + "activation": "not_applicable", + "required": [], + "forbidden": [ + "state-written", + "requirement-omitted", + "unnecessary-machinery" + ], + "permitted": [] + }, + "m1-explicit-skiphow": { + "activation": "expected", + "required": [], + "forbidden": [ + "state-written", + "requirement-omitted", + "unnecessary-machinery" + ], + "permitted": [] + }, + "m2-implicit-discovery-hook": { + "activation": "expected", + "required": [], + "forbidden": [ + "state-written", + "requirement-omitted", + "unnecessary-machinery" + ], + "permitted": [] + }, + "m3-bootstrap-candidate": { + "activation": "expected", + "required": [], + "forbidden": [ + "state-written", + "requirement-omitted", + "unnecessary-machinery" + ], + "permitted": [] + }, + "m4-previous-full-skiphow": { + "activation": "expected", + "required": [], + "forbidden": [ + "state-written", + "requirement-omitted", + "unnecessary-machinery" + ], + "permitted": [] + } + }, + "alternatives": [], + "conditional": [], + "result": { + "status": "not_run", + "evidence_label": "UNVERIFIED", + "observed_arms": [], + "arms_pending": [ + "m0-base-host", + "m1-explicit-skiphow", + "m2-implicit-discovery-hook", + "m3-bootstrap-candidate", + "m4-previous-full-skiphow" + ], + "runs": [] + } } ], "historical_evidence": { diff --git a/evals/cto-cases.json b/evals/cto-cases.json index fb8620b..61fcde6 100644 --- a/evals/cto-cases.json +++ b/evals/cto-cases.json @@ -1,6 +1,6 @@ { "instrument": "forced_activation_behavior", - "package_under_test": "6.1.1", + "package_under_test": "6.1.2", "suite_status": "not_run", "minimum_coverage": { "case_ids": [ @@ -96,7 +96,8 @@ "runs": [] }, "regression_cases": [ - "audit-small-visible-two-file" + "audit-small-visible-two-file", + "bug-shared-balance-invariant" ] }, { @@ -199,7 +200,11 @@ "status": "not_run", "evidence_label": "UNVERIFIED", "runs": [] - } + }, + "regression_cases": [ + "design-native-capability-sufficient", + "design-native-capability-insufficient" + ] }, { "id": "cto-large-programme", diff --git a/evals/fixtures/planning-readiness/evidence.md b/evals/fixtures/planning-readiness/evidence.md index dca528d..7a389e3 100644 --- a/evals/fixtures/planning-readiness/evidence.md +++ b/evals/fixtures/planning-readiness/evidence.md @@ -67,6 +67,9 @@ like the cancellation flow. Its new `RescheduleDialog` reimplements the detail s unsaved-change guard, and error banner in 220 lines. The maintained `AppointmentSheet` already provides those behaviors and accepts a slot-selection body. Screenshots of the two flows are nearly identical. The candidate's error banner does not restore focus after a failed save, which `AppointmentSheet` does. +The review must identify the maintained sheet with its slot-selection body as the replacement and preserve +slot selection, focus trapping and restoration, the unsaved-change guard, and save-error recovery. Deleting +the dialog without preserving those responsibilities does not satisfy the accepted reschedule flow. ## Requirement beyond the existing composition @@ -75,6 +78,25 @@ The accepted outcome lets clinic staff reschedule several appointments at once f primitives exist. Partial failure must report which appointments moved and which did not; that acceptance condition is recorded as an owner decision. +## Native date input sufficient + +The accepted outcome collects one birth date as an ISO calendar date within a continuous minimum/maximum range. +The project's supported platform already provides native date entry, keyboard access, and those range constraints. +Its maintained `FormField` accepts a native input and supplies the required label, validation message, and focus +handling. These capabilities are established synthetic evidence, not a claim about every browser. There is no +date-range selection, unavailable-day display, or other calendar requirement. No new dependency is needed. +The owner requests a read-only implementation recommendation for this settled scope. + +## Native date input insufficient + +The accepted outcome lets patients select an available appointment date. Unavailable dates must appear disabled +before selection, including isolated unavailable days inside the otherwise allowed range. In this fixture's +supported platform, the native input supplies continuous minimum/maximum bounds but no disabled-day display. +The maintained `AvailabilityCalendar` already consumes the service's available-date list and supplies disabled +days, keyboard navigation, and the required error and focus behavior. The server still validates availability +when saving. It costs less to reuse this maintained composition than to reproduce it. Accepting any date and +showing a save error omits an accepted requirement. The owner requests a read-only implementation recommendation. + ## Partial design A new internal tool lets clinic managers review weekly no-show counts. The repository has a small token file diff --git a/evals/fixtures/planning-readiness/fixture.json b/evals/fixtures/planning-readiness/fixture.json index 4f17258..e4ff86a 100644 --- a/evals/fixtures/planning-readiness/fixture.json +++ b/evals/fixtures/planning-readiness/fixture.json @@ -9,7 +9,8 @@ "A single-message repair has no useful parallel boundary.", "Synthetic capability descriptions, some suitable and some not, stand in for a host's skill list; they do not prove native discovery.", "Maintained and legacy compositions, a duplicated composition, a requirement the maintained composition cannot meet, partial design, and a product choice presented as an algorithm.", - "A settled one-string interaction change and a non-interface task with specialist capabilities listed." + "A settled one-string interaction change and a non-interface task with specialist capabilities listed.", + "Paired date-entry requirements distinguish a sufficient native input from a maintained calendar needed for disabled-day display." ], "setup": [ "Use the existing capture preparation path to copy the fixture into an owned isolated session.", diff --git a/evals/fixtures/shared-balance/fixture.json b/evals/fixtures/shared-balance/fixture.json new file mode 100644 index 0000000..9042301 --- /dev/null +++ b/evals/fixtures/shared-balance/fixture.json @@ -0,0 +1,26 @@ +{ + "id": "shared-balance", + "summary": "A named transfer defect also affects withdrawals through a shared balance invariant.", + "kind": "checked-in skeleton", + "synthetic": true, + "planted": [ + "Both public operations permit an overdraft. The report names transfer only.", + "A transfer-only guard passes the reported path but leaves withdrawal broken.", + "Positive amount validation and valid operations already work and must survive the repair." + ], + "absent_on_purpose": [ + "No tracker, remote, credentials, network, or live financial data.", + "No prescribed private helper, search command, or implementation shape." + ], + "setup": [ + "Use scripts/capture_eval.py materialize to copy this fixture into a new owned scratch directory.", + "Keep the owner prompt unchanged across arms. Local repair only; no Git setup is needed.", + "Grade the delivered implementation with the original fixture oracle; edits to acceptance checks do not establish success." + ], + "run_checks": "python -m pytest tests/balance_checks.py", + "end_state_signals": [ + "Both public operations reject overdrafts without changing either balance.", + "Exact-balance operations, valid transfers and withdrawals, and invalid-amount rejection retain their behavior." + ], + "privacy": "Invented accounts and integer units; no customer data, personal paths, credentials, or external effects." +} diff --git a/evals/fixtures/shared-balance/ledger.py b/evals/fixtures/shared-balance/ledger.py new file mode 100644 index 0000000..71039f1 --- /dev/null +++ b/evals/fixtures/shared-balance/ledger.py @@ -0,0 +1,18 @@ +"""Synthetic account operations, using integer units and no external systems.""" + + +def _debit(balances: dict[str, int], account: str, amount: int) -> None: + if amount <= 0: + raise ValueError("amount must be positive") + balances[account] -= amount + + +def transfer(balances: dict[str, int], source: str, target: str, amount: int) -> None: + if target not in balances: + raise KeyError(target) + _debit(balances, source, amount) + balances[target] += amount + + +def withdraw(balances: dict[str, int], account: str, amount: int) -> None: + _debit(balances, account, amount) diff --git a/evals/fixtures/shared-balance/tests/balance_checks.py b/evals/fixtures/shared-balance/tests/balance_checks.py new file mode 100644 index 0000000..cc54179 --- /dev/null +++ b/evals/fixtures/shared-balance/tests/balance_checks.py @@ -0,0 +1,49 @@ +"""Observable acceptance for both public operations, independent of repair shape.""" + +import pytest + +from ledger import transfer, withdraw + + +def test_valid_transfer_conserves_units() -> None: + balances = {"alpha": 100, "beta": 20} + transfer(balances, "alpha", "beta", 40) + assert balances == {"alpha": 60, "beta": 60} + + +def test_valid_withdrawal_changes_only_the_requested_account() -> None: + balances = {"alpha": 100, "beta": 20} + withdraw(balances, "alpha", 40) + assert balances == {"alpha": 60, "beta": 20} + + +@pytest.mark.parametrize("operation", ["transfer", "withdraw"]) +@pytest.mark.parametrize("amount", [100, 101, 150]) +def test_balance_boundary_is_shared_by_public_operations(operation: str, amount: int) -> None: + balances = {"alpha": 100, "beta": 20} + if amount == 100: + if operation == "transfer": + transfer(balances, "alpha", "beta", amount) + assert balances == {"alpha": 0, "beta": 120} + else: + withdraw(balances, "alpha", amount) + assert balances == {"alpha": 0, "beta": 20} + else: + with pytest.raises(ValueError): + if operation == "transfer": + transfer(balances, "alpha", "beta", amount) + else: + withdraw(balances, "alpha", amount) + assert balances == {"alpha": 100, "beta": 20} + + +@pytest.mark.parametrize("amount", [0, -1]) +def test_invalid_amount_preserves_state(amount: int) -> None: + for operation in (transfer, withdraw): + balances = {"alpha": 100, "beta": 20} + with pytest.raises(ValueError): + if operation is transfer: + operation(balances, "alpha", "beta", amount) + else: + operation(balances, "alpha", amount) + assert balances == {"alpha": 100, "beta": 20} diff --git a/evals/host-smoke.json b/evals/host-smoke.json index b87e2c2..0de31d2 100644 --- a/evals/host-smoke.json +++ b/evals/host-smoke.json @@ -1,6 +1,6 @@ { "instrument": "host_smoke", - "package_under_test": "6.1.1", + "package_under_test": "6.1.2", "scope": "external_candidate_receipts", "checks": { "clean_install": { diff --git a/plugins/skiphow/.claude-plugin/plugin.json b/plugins/skiphow/.claude-plugin/plugin.json index a06dd3e..f217186 100644 --- a/plugins/skiphow/.claude-plugin/plugin.json +++ b/plugins/skiphow/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "skiphow", - "version": "6.1.1", + "version": "6.1.2", "description": "Adaptive virtual CTO for founders and product owners using Claude Code or Codex. Describe the product outcome; SkipHow owns the technical lifecycle through verified completion.", "author": { "name": "mzored", diff --git a/plugins/skiphow/.codex-plugin/plugin.json b/plugins/skiphow/.codex-plugin/plugin.json index 4589b46..78d3a7d 100644 --- a/plugins/skiphow/.codex-plugin/plugin.json +++ b/plugins/skiphow/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "skiphow", - "version": "6.1.1", + "version": "6.1.2", "description": "Adaptive virtual CTO for founders and product owners using Claude Code or Codex. Describe the product outcome; SkipHow owns the technical lifecycle through verified completion.", "author": { "name": "mzored", diff --git a/plugins/skiphow/skills/skiphow/references/technical-design.md b/plugins/skiphow/skills/skiphow/references/technical-design.md index 07fd8e7..5e16f8f 100644 --- a/plugins/skiphow/skills/skiphow/references/technical-design.md +++ b/plugins/skiphow/skills/skiphow/references/technical-design.md @@ -16,7 +16,7 @@ Security, reliability, operability, performance, cost, and reversibility are len ## Reuse before custom code -Prefer existing, maintained capabilities over custom machinery. Before introducing a subsystem, abstraction, dependency, infrastructure component, or service, look outward roughly in this order — and again whenever existing custom code looks like it duplicates something mature: +Choose the lowest total cost among solutions that preserve the complete accepted behavior. A short diff, built-in feature, or installed dependency qualifies only when it meets the real constraints. Prefer maintained capabilities over custom machinery. Before introducing a subsystem, abstraction, dependency, infrastructure component, or service, look outward roughly in this order, and again when custom code duplicates something mature: - Capabilities the repository already has. - Primitives in the language, framework, or platform. diff --git a/plugins/skiphow/skills/skiphow/references/verification.md b/plugins/skiphow/skills/skiphow/references/verification.md index ad65bae..4dd5eaa 100644 --- a/plugins/skiphow/skills/skiphow/references/verification.md +++ b/plugins/skiphow/skills/skiphow/references/verification.md @@ -44,7 +44,7 @@ Establish the exact candidate revision under review and the request, issue, or s For changed interactions, verify the journey in the surrounding product, including consequential transitions and recovery paths. Compare it with accepted intent and applicable interaction and visual conventions. Inspect the implementation separately to confirm required shared behavior and compositions are reused. Rendered similarity does not establish reuse, and shared imports do not establish a coherent experience. A prototype or visual comparison earns its cost where it resolves a specific uncertainty. -Look for incorrect behavior, missing cases, scope creep, security or data risks, broken compatibility, weak error handling, misleading tests, and violations of documented project rules. Verify a suspected issue before reporting it when a focused check can settle it. A finding names a concrete defect; a reviewer who cannot point at what breaks, such as a user consequence, a broken contract, or evidenced duplication, is reporting taste. State each actionable finding with its location, triggering scenario, and impact, most consequential first. If there are no material findings, say so and name any important area left unverified. +Look for incorrect behavior, missing cases, scope creep, security or data risks, broken compatibility, weak error handling, misleading tests, and violations of documented project rules. Verify a suspected issue with a focused check when one can settle it. A finding names a concrete defect; a reviewer who cannot point at a consequence, broken contract, or evidenced duplication is reporting taste. For unnecessary complexity, name the redundant construct, its consequence, a cheaper replacement or justified deletion, and the responsibilities that must survive. State each actionable finding with its location, triggering scenario, and impact, most consequential first. If there are no material findings, say so and name any important area left unverified. On a read-only review, report confirmed defects without modifying the project; urgency, including a security finding, does not widen the request, and a sensitive finding stays private unless disclosure is granted. When repair is authorized, fix confirmed in-scope defects before completion, re-review the changed parts, and verify the repaired final state; do not carry an important defect forward as accepted. Stop when the remaining items are taste, lack evidence, or are reported as unresolved; use another broad reviewer only to resolve a high-consequence disagreement or contradictory evidence. diff --git a/site/evidence/index.html b/site/evidence/index.html index 5ddcae7..25cd2f0 100644 --- a/site/evidence/index.html +++ b/site/evidence/index.html @@ -56,11 +56,11 @@

Claims stop where the receipts stop.

-
+

01 · Historical observations

What controlled 2.x runs showed.

-

Historical observations below: 2.x only. Current package: 6.1.1. Retained current-package CTO scenarios with Observed receipts: 0 of 12. Eight historical 4.0.1 Claude run records remain incomplete. Current CTO scenario behavior is UNVERIFIED; configuration and deterministic checks do not establish model behavior.

+

Historical observations below: 2.x only. Current package: 6.1.2. Retained current-package CTO scenarios with Observed receipts: 0 of 12. Eight historical 4.0.1 Claude run records remain incomplete. Current CTO scenario behavior is UNVERIFIED; configuration and deterministic checks do not establish model behavior.

diff --git a/tests/test_simplification_fixtures.py b/tests/test_simplification_fixtures.py new file mode 100644 index 0000000..bb9bc09 --- /dev/null +++ b/tests/test_simplification_fixtures.py @@ -0,0 +1,53 @@ +"""Validate the shared-invariant oracle, not model behavior or repair topology.""" + +from pathlib import Path +import subprocess +import sys + +import pytest + +ROOT = Path(__file__).resolve().parents[1] +sys.path.insert(0, str(ROOT / "scripts")) +from capture_eval import materialize + + +@pytest.mark.parametrize("variant", ["unfixed", "named_path_only", "shared_guard", "public_guards"]) +def test_shared_balance_oracle_distinguishes_complete_repairs(tmp_path: Path, variant: str) -> None: + fixture = tmp_path / "shared-balance" + materialize("shared-balance", fixture) + source = fixture / "ledger.py" + original = source.read_text(encoding="utf-8") + candidate = original + if variant == "shared_guard": + candidate = candidate.replace( + " balances[account] -= amount\n", + " if amount > balances[account]:\n" + " raise ValueError('insufficient balance')\n" + " balances[account] -= amount\n", + ) + if variant in {"named_path_only", "public_guards"}: + candidate = candidate.replace( + " _debit(balances, source, amount)\n", + " if amount > balances[source]:\n" + " raise ValueError('insufficient balance')\n" + " _debit(balances, source, amount)\n", + ) + if variant == "public_guards": + candidate = candidate.replace( + " _debit(balances, account, amount)\n", + " if amount > balances[account]:\n" + " raise ValueError('insufficient balance')\n" + " _debit(balances, account, amount)\n", + ) + source.write_text(candidate, encoding="utf-8") + result = subprocess.run( + [sys.executable, "-B", "-m", "pytest", "-q", "-p", "no:cacheprovider", "tests/balance_checks.py"], + cwd=fixture, capture_output=True, text=True, check=False, timeout=20, + ) + if variant in {"unfixed", "named_path_only"}: + assert result.returncode == 1, result.stdout + result.stderr + assert "withdraw" in result.stdout + if variant == "named_path_only": + assert "FAILED tests/balance_checks.py::test_balance_boundary_is_shared_by_public_operations[101-transfer]" not in result.stdout + else: + assert result.returncode == 0, result.stdout + result.stderr