From af8223e72f3f64352cbe2bdc18bb769dd5ce1071 Mon Sep 17 00:00:00 2001 From: mzored Date: Sat, 3 Oct 2026 22:34:20 +0400 Subject: [PATCH 1/2] Prepare workload-bounded parallel plans and short launches for 6.2.0 --- CHANGELOG.md | 31 + README.md | 16 + SECURITY.md | 4 +- VERSION | 2 +- docs/decisions.md | 44 +- docs/evidence.md | 24 + docs/outcome-contract.md | 2 +- docs/prior-art.md | 26 + evals/README.md | 8 + evals/cases.json | 1281 ++++++++++++++++- evals/cto-cases.json | 2 +- .../fixtures/plan-execution/accepted-plan.md | 17 + evals/fixtures/plan-execution/evidence.md | 100 ++ evals/fixtures/plan-execution/fixture.json | 25 + .../plan-execution/incomplete-plan.md | 7 + evals/fixtures/plan-execution/owner-notes.md | 12 + .../samples/automated-label-checks.md | 6 + .../samples/automated-labels.md | 7 + .../samples/manual-guidance-checks.md | 7 + .../plan-execution/samples/manual-guidance.md | 8 + evals/host-smoke.json | 2 +- plugins/skiphow/.claude-plugin/plugin.json | 2 +- plugins/skiphow/.codex-plugin/plugin.json | 2 +- plugins/skiphow/skills/skiphow-plan/SKILL.md | 17 +- site/evidence/index.html | 4 +- 25 files changed, 1640 insertions(+), 16 deletions(-) create mode 100644 evals/fixtures/plan-execution/accepted-plan.md create mode 100644 evals/fixtures/plan-execution/evidence.md create mode 100644 evals/fixtures/plan-execution/fixture.json create mode 100644 evals/fixtures/plan-execution/incomplete-plan.md create mode 100644 evals/fixtures/plan-execution/owner-notes.md create mode 100644 evals/fixtures/plan-execution/samples/automated-label-checks.md create mode 100644 evals/fixtures/plan-execution/samples/automated-labels.md create mode 100644 evals/fixtures/plan-execution/samples/manual-guidance-checks.md create mode 100644 evals/fixtures/plan-execution/samples/manual-guidance.md diff --git a/CHANGELOG.md b/CHANGELOG.md index efdeed2..9455dda 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,37 @@ All notable changes to SkipHow 2.x and later appear in this file. Earlier release notes remain available on [GitHub Releases](https://github.com/mzored/SkipHow/releases). +## 6.2.0 (2026-10-03) + +### Changed + +- Plan's existing independent task review assesses context, heterogeneous implementation, manual output, checks, + review, and integration together. A single outcome is insufficient proof of one-agent feasibility; supported size + findings preserve acceptance and dependencies in smaller independently verifiable slices. Large uniform automated + changes can stay whole, and implementation, checks, and handoff steps without separate outcomes remain one task. +- Work lacking an established component path starts with a thin real slice and only its necessary structure, preserving + safety, integrity, and compatibility. Mature paths are reused. The whole accepted scope remains prepared in advance, + with actual blockers, independent preparation, shared resources, and assumptions to recheck at dispatch. +- Ready plans return an explicit host-native longrun invocation and one delivery sentence referencing accepted records + and destination. An owner-selected workflow survives. Only otherwise-missing project, subset, or run constraints are + added, with authorization provenance; task content and SkipHow policy stay in their sources. +- Arm-aware scenarios cover workload boundaries and sampling, useful grouping, first-path and mature planning, + parallel readiness and resources, short launches, selected workflows, and blocked sources or preparation. + +### Compatibility + +Compatible MINOR capability in the optional plan workflow. Public skill names, kernel, authority boundaries, package +layout, installation, and existing execution workflows are unchanged. No new stage, module, scheduler, size score, +quota, inventory gate, or sizing ledger. Unfinished preparation remains blocked, and launch text grants no new actions. +Tracer bullets and to-tickets inspired the ideas; wording and fixtures are original. + +### Validation + +New behavioral scenarios and improvement claims remain `UNVERIFIED`. No paid behavioral comparison was run, and no +context-fit, cost, or parallel-execution gain is claimed. See [the evidence ledger](docs/evidence.md#620-workload-based-planning-and-short-launches). +The release matrix separates deterministic checks, host schemas, installation, activation, continuity, and behavior. +Prose changes within the existing package structure require no fresh clean-install experiment. + ## 6.1.2 (2026-10-03) ### Changed diff --git a/README.md b/README.md index f5c86df..757b5f8 100644 --- a/README.md +++ b/README.md @@ -132,6 +132,22 @@ Use these when you know the work pattern you want. Ordinary requests still need Explicit `skiphow-plan` invocation requests durable planning records within the authorized audience. A read-only constraint overrides that default. Automatic selection for an ordinary planning question does not grant tracker writes. Unavailable tracking leaves complete issue drafts and a blocker. +Plan reviews each assignment's actual context, implementation, verification, review, and integration load. It prepares +the whole accepted scope as bounded vertical slices with real dependencies and parallel work. A new component path +starts with a thin working slice; a mature project reuses its established path. The planner keeps large checkable +automated changes whole and avoids steps that have no independently verifiable result. + +Ready plans end with a short launch using the target host's explicit invocation syntax. Longrun is the default unless +you already chose another workflow. For example, with a concrete accepted plan and destination: + +```text +$skiphow-longrun +Deliver the accepted implementation scope in issue #42 to the development branch. +``` + +Tasks carry requirements and dependencies, so the launch adds only otherwise-missing project, subset, or run constraints. +Preparation and inaccessible sources remain explicit blockers, and the prompt creates no new authorization. + Fast-fixes accepts new changes, revisions, and cancellations while accepted work continues. The lead handles tiny fixes directly and delegates independent work when that pays, with verified writer isolation and coordination of shared resources. It shows an inspected preview, demonstrated behavior, or an artifact as appropriate to the project; a browser and Git are not prerequisites. Before replacing a shown state, it preserves a recoverable checkpoint, using local commits in Git projects. Focused tests may run, including pytest or browser tests, while full delivery gates wait. Explicit requests to wait or restrict tests still govern. Acceptance keeps fast-fixes in local iteration. Say `deploy-ready` or `clean` to reconcile the delivery set, including running assignments, and begin preparation. Corrections receive affected review and revalidation; later work stays accounted for outside that delivery. Say `deploy prod` to authorize the specified production release. Preparation and contextual clean alone do not authorize production or arbitrary deletion. Hosts determine when messages arrive and which delegation or continuation controls exist; the plugin adds no background scheduler. diff --git a/SECURITY.md b/SECURITY.md index 5933ae9..6bf06a4 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -4,8 +4,8 @@ | Version | Supported | | --- | --- | -| 6.1.x | Yes | -| 6.0.x, 5.1.x, 5.0.x, 4.8.x, 4.7.x, 4.6.x, 4.5.x, 4.4.x, 4.3.x, 4.2.x, 4.1.x, and 4.0.x | No; upgrade to 6.1.x, reviewing the 6.0 iteration defaults when coming from 5.x or earlier; existing authorization stays unchanged | +| 6.2.x | Yes | +| 6.1.x, 6.0.x, 5.1.x, 5.0.x, 4.8.x, 4.7.x, 4.6.x, 4.5.x, 4.4.x, 4.3.x, 4.2.x, 4.1.x, and 4.0.x | No; upgrade to 6.2.x, reviewing the 6.0 iteration defaults when coming from 5.x or earlier; existing authorization stays unchanged | | 3.0.x and earlier | No | Security review covers the packaged owner skill, its linked playbooks, host manifests, diff --git a/VERSION b/VERSION index 5e32542..6abaeb2 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -6.1.2 +6.2.0 diff --git a/docs/decisions.md b/docs/decisions.md index a84fb82..d9ecd53 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -4,10 +4,11 @@ This page records the choices that still matter when SkipHow changes. Read it be ## Current decisions -The live decisions, their premises, and what would reopen each. "Evidence" says what stands behind the decision today: `Observed` means a retained run showed it on the package that carried it, `Contract` means the shipped text encodes it and no run has tested that text, `Deterministic` means a check proves it on every run. Last reviewed 2026-10-03 against the 6.1.2 candidate. The [owner-outcome contract](outcome-contract.md) governs implementation choices. +The live decisions, their premises, and what would reopen each. "Evidence" says what stands behind the decision today: `Observed` means a retained run showed it on the package that carried it, `Contract` means the shipped text encodes it and no run has tested that text, `Deterministic` means a check proves it on every run. Last reviewed 2026-10-03 against the 6.2.0 candidate. The [owner-outcome contract](outcome-contract.md) governs implementation choices. | Decision | Active rationale | Premises | Evidence | Reopens when | | --- | --- | --- | --- | --- | +| Plan reviews actual one-agent workload, prepares bounded vertical slices across the accepted scope, proves an absent component path with a thin real slice, and returns an explicit short longrun launch unless the owner selected another workflow | One common outcome can conceal heterogeneous work; a generic short-prompt instruction permits missing invocation or repeated task and policy content | The owner requested these optional plan behaviors; existing delegation and longrun own execution, reslicing, writer isolation, and resource coordination; mature paths and uniform automated work need no artificial foundation or quotas | Owner-requested `Contract` in 6.2.0; two private 6.1.1 planning sessions show differing handoffs, not causal improvement; new scenarios and behavioral benefit `UNVERIFIED` | A receipt shows oversized assignments passing review, unnecessary fragmentation, invented blocking edges, safety deferred from a first slice, repeated launch content, or authority lost or expanded | | Simplification preserves complete accepted behavior at the lowest total cost; a complexity finding names a redundant construct, its consequence, a cheaper replacement or justified deletion, and surviving responsibilities | A short diff or native capability can omit required behavior, and a deletion can remove necessary focus, change protection, or recovery | The owner requested clarification after the Ponytail comparison; existing design and verification references already own these judgments | Clarified `Contract` in 6.1.2; three new unrun scenarios and one strengthened review scenario; deterministic oracle checks are not model evidence | A receipt shows insufficient reuse accepted as complete, needless machinery retained, or a complexity finding losing a required responsibility | | Briefs convey a compact, evidenced handoff; bulky artifacts stay accessible by reference through review and return checks; corrections carry current evidence and unresolved findings; shortened check output retains the check's own terminal status | The lead can read a return-size rule without conveying it to the worker, and a filter can succeed while the check fails | Small results can stay inline; all findings, blockers, and uncertainty survive; the lead still verifies current state and destination; no file format, size quota, or new coordinator is required | Clarified `Contract` in 6.1.1 after three private installed sessions and owner-approved implementation; four unrun cases; causal and behavioral effects `UNVERIFIED` | A receipt shows lost findings, inaccessible proof treated as acceptance, blanket report files, repeated bulk copied into the coordinator, or a filter status accepted as check status | | Fast-fixes coordinates incoming local work across stacks, preserves shown states, and continues independent work; plan distinguishes unanswered choices from execution readiness; deploy-ready reconciles active work into a checked candidate | The owner explicitly requested live orchestration; the old fast-fixes required a browser, Git worktree, stop after every shown result, and deferred every pytest/e2e invocation | Existing delegation, tracking, integration, and verification cover the shared responsibilities; host controls determine concurrency and message timing; restrictions and protected-action grants survive upgrades | Owner-requested `Contract` in 6.0.0; nine unrun cases, behavioral effect `UNVERIFIED` | A receipt shows lost incoming work, stale returns accepted, blocked work stopping independent outcomes, unsafe shared-resource use, lost checkpoints, needless infrastructure, or verification gaps hidden by the new defaults | @@ -37,6 +38,47 @@ The live decisions, their premises, and what would reopen each. "Evidence" says The sections below are the history behind those rows: what each release tried, measured, and rejected. They are non-normative. Where a section and the index disagree, the index is current and the section records how it got there. +## Workload-based planning and a short execution launch + +Version 6.2.0 implements the owner's accepted planning contract. Two private Codex sessions carrying the full 6.1.1 +plan text returned different handoffs: a short launch omitted explicit longrun, while another supplied a longer one +after an owner follow-up and repeated execution guidance. The previous workflow required only a short prompt; this +establishes freedom in its shape, not violation of a prior explicit-longrun default or a general model failure rate. +The workload complaint motivates an explicitly requested review clarification; no transcript establishes its cause. + +The existing mandatory review of every published task now weighs context, heterogeneous changes, manual output, +verification, review, and integration together. A common outcome does not establish one-agent feasibility. Existing +measurements, comparable work, or a representative sample can settle material uncertainty. Supported size findings +name the load, credible smaller independently verifiable cuts, and surviving acceptance and dependencies. Splits +must repay repeated context and coordination costs. Tools and review corrections remain part of one implementation +assignment. Large uniform automated changes and small cohesive work can stay whole. No score, inventory gate, +universal file or context limit, sizing registry, or new review stage is added. + +For a path not yet established through the components, plan selects a thin first slice with real behavior and only +the structure it needs, preserving required safety, integrity, and compatibility. The entire accepted scope is still +prepared as bounded tasks, with the earlier result and later assumptions named for rechecking at dispatch. Independent +preparation can start sooner. Mature projects reuse their path; compatible migrations and mechanical changes retain +their existing treatment. Actual blocking results differ from shared-write, runtime-resource, and capacity constraints. +Two consumers of a settled API need no dependency edge solely for that reason. + +Ready scope gets one explicit workflow invocation and one delivery sentence pointing to the concrete accepted records +and agreed destination. Longrun is the default, using the target host's syntax; an owner-selected workflow survives. +Only otherwise-missing project, subset, and run constraints are added. Existing records carry acceptance, dependencies, +and authorization provenance; the launch copies no task content or CTO policy and creates no grant. Inaccessible +sources and unfinished preparation remain blockers to affected scope. This is a compatible optional-workflow capability, +so it receives a MINOR version without changing the kernel, authority, package structure, or installation. + +Alternatives rejected: a prompt generator or new scheduler for a prose-only gap, an unconditional skeleton stage in +every project, horizontal setup/implementation/test/handoff tickets with no standalone outcome, numeric size quotas, +and leaving accepted later tasks for decomposition during longrun. Existing longrun and delegation already preserve +an outgrown assignment's work and evidence and return its remaining scope for reassessment. + +[Prior art](prior-art.md#bounded-plans-and-first-working-paths-in-620) records the primary sources and rejected mechanisms. +The new acceptance scenarios are arm-aware and unrun. Deterministic validation and independent instruction review +do not prove model reliability, context fit, parallel execution, or lower token cost. Reopen this decision on receipts +showing oversized accepted tasks, microtask overhead, invented dependencies, omitted first-slice protections, or lost +launch constraints and authority. + ## Sufficient reuse and concrete complexity findings Version 6.1.2 implements the owner's requested clarifications after inspecting Ponytail at diff --git a/docs/evidence.md b/docs/evidence.md index 56bb1bb..35f0926 100644 --- a/docs/evidence.md +++ b/docs/evidence.md @@ -2,6 +2,30 @@ This page separates package checks from observed model behavior. The full 2.0 evidence remains in the immutable [`v2.0.1` research snapshot](https://github.com/mzored/SkipHow/tree/1c811262e6acdbdc58a2ee862b54e0b8d3478eaa/docs/research/2026-08-27). +## 6.2.0 workload-based planning and short launches + +Two private installed Codex planning sessions from 2026-10-02 carried the full 6.1.1 plan text. One returned a short +launch without explicit longrun. Another returned a longer launch after an owner follow-up requesting longrun and +reference-only task content, and repeated execution guidance. No transcript, project identity, private issue content, +or personal path is retained here. These observations establish differing handoffs under the prior generic prompt +requirement, not a failure rate or the cause of oversized tasks. The owner explicitly requested the new plan capability. + +The existing task review now assesses complete one-agent workload and requires supported smaller cuts when a task +is too large. A conditional thin first working path, real parallel readiness and resource constraints, and a two-line +launch are added inside plan. The kernel, delegation, and longrun retain their existing responsibilities, including +outgrown-assignment recovery. [Prior art](prior-art.md#bounded-plans-and-first-working-paths-in-620) and +[the decision](decisions.md#workload-based-planning-and-a-short-execution-launch) record the sources, scope, and alternatives. + +New arm-aware scenarios cover heterogeneous umbrella tasks, uncertain workload sampling, uniform automated changes, +artificial microtasks, a first working path and mature reuse, real blocking edges and shared resources, host-native +short launches and selected workflows, constraint provenance, and blocked preparation or sources. Existing migration, +localization, small-change, and isolation cases remain. Every new scenario is `UNVERIFIED`; no paid behavioral +comparison was run. Corpus checks establish scenario semantics, not successful agents, context savings, or parallelism. + +The release matrix keeps the deterministic package gate, each host schema, each clean install, explicit invocation, +implicit activation, continuity, and behavioral suite separate. No package structure or installation mechanism changes, +so fresh installation and activation experiments are not required and remain visible as `UNVERIFIED`. + ## 6.1.2 sufficient reuse and concrete complexity findings The owner requested this clarification after a read-only comparison with Ponytail revision diff --git a/docs/outcome-contract.md b/docs/outcome-contract.md index 50382f0..8110a83 100644 --- a/docs/outcome-contract.md +++ b/docs/outcome-contract.md @@ -18,7 +18,7 @@ Ordinary-language operation has an inspectable, reversible installation path on The lead delivers to the authorized project and destination using an established owner-authorized workflow, including routine non-production tracking, branches, commits, CI, push, pull requests, and integration where covered. It checks actual downstream effects. Production, public publication, new financial commitments, access expansion, material deletion, and disclosure require applicable explicit owner authorization. An upgrade grants nothing and overrides no existing restriction. Read-only requests leave the protected project and external records unchanged. Retiring the lead's own recoverable engineering state, including what earlier sessions left, is routine engineering rather than material deletion; material deletion covers data, history others share, and anything unrecoverable. -Explicit invocation of the planning workflow requests durable planning records, subject to any read-only constraint and the authorized audience. An execution-ready result includes a reviewed specification, bounded vertical slices, and an execution prompt, with no known unresolved product choices in its accepted scope. Unanswered choices leave preparation explicitly incomplete. Later discoveries and known human prerequisites remain possible; planning grants no implementation or protected action. Missing tracker access leaves complete drafts and a blocker. Longrun keeps implementation and independent review in a fix/review loop, with the lead owning routing, reslicing, anomaly diagnosis, integration, and continuation. Unavailable writer isolation leaves the lead as the only writer and does not remove independent review. Unavailable delegates leave safe direct implementation possible and required independent review blocked. Human-dependent work retains the action needed, evidence, dependencies, and resume condition; recording it neither completes it nor schedules resumption. +Explicit invocation of the planning workflow requests durable planning records, subject to any read-only constraint and the authorized audience. An execution-ready result includes a reviewed specification, vertical slices bounded by actual one-agent workload across implementation and delivery, real dependencies and parallel opportunities, and a short execution prompt, with no known unresolved product choices in its accepted scope. Where no working component path is established, the plan starts with a thin real slice and extends it; mature projects reuse their path. The launch explicitly invokes longrun by default, preserves an owner-selected workflow, and references accepted records and destination without repeating task content or CTO policy. Unanswered choices leave preparation explicitly incomplete. Later discoveries and known human prerequisites remain possible; planning grants no implementation or protected action. Missing tracker access leaves complete drafts and a blocker. Longrun keeps implementation and independent review in a fix/review loop, with the lead owning routing, reslicing, anomaly diagnosis, integration, and continuation. Unavailable writer isolation leaves the lead as the only writer and does not remove independent review. Unavailable delegates leave safe direct implementation possible and required independent review blocked. Human-dependent work retains the action needed, evidence, dependencies, and resume condition; recording it neither completes it nor schedules resumption. While the owner keeps sending changes, or has said they will look and send them, the work stays in iteration with its feedback and checkpoint agreement. Delivery starts when they request it, name a destination, or accept with nothing further to send. Fast-fixes coordinates incoming local fixes across stacks, showing checked results without stopping independent accepted work. It preserves a shown state before superseding it, using local commits in Git projects or an available safe preservation mechanism elsewhere. Focused verification follows behavior and actual effects rather than tool names; full delivery gates and shared integration remain deferred, and explicit restrictions and mandatory project checks govern. In such a session acceptance alone does not start delivery. Deploy-ready and contextual clean request preparation of the reconciled delivery set, including in-flight work and explicit blockers. Late corrections enter the candidate only after review and affected revalidation; unrelated additions remain accounted for without silently enlarging it. Production requires an applicable explicit grant. These optional workflows do not change the delivery boundary of ordinary SkipHow requests. diff --git a/docs/prior-art.md b/docs/prior-art.md index 3ee7e9f..5556358 100644 --- a/docs/prior-art.md +++ b/docs/prior-art.md @@ -116,6 +116,32 @@ An idea from another project becomes a focused method or a kernel invariant only The reasoning behind each rule that survived is in [decision history](decisions.md), and what real runs have and have not shown is in [current evidence](evidence.md). +## Bounded plans and first working paths in 6.2.0 + +Read on 2026-10-03. The [Hunt and Thomas interview on tracer bullets](https://www.artima.com/articles/tracer-bullets-and-prototypes) +describes a thin path through the real components, followed by complete use cases that grow the running result. +SkipHow adopts that idea only where the accepted work lacks an established path. The first slice creates the structure +it needs and preserves required safety, integrity, and compatibility. A mature project can extend its existing path. +A disposable uncertainty probe and an evolving implementation remain different engineering choices. + +Matt Pocock's [to-tickets skill at `d81f3a1`](https://github.com/mattpocock/skills/blob/d81f3a183412e71a5b1e84ca21bc1a35eea03a60/skills/engineering/to-tickets/SKILL.md) +uses demonstrable vertical tickets with explicit blocking edges. SkipHow keeps those ideas, distinguishes true +prerequisites from shared-write and runtime-capacity constraints, and sizes the whole accepted task set for dispatch. +It rejects a setup interview and owner approval of technical granularity, and treats one-agent feasibility as an +evidence-based workload judgment rather than a guarantee about fresh context. + +GSD's [MVP planner at `84b4dfe`](https://github.com/open-gsd/gsd-core/blob/84b4dfe84f35e58d4b35d57a54bd92af1b21fbce/gsd-core/references/planner-mvp-mode.md) +provides another example of starting with a working tracer and expanding it. SkipHow does not adopt its per-phase +tracer requirement, mandatory skeleton record, or task-count guidance. The selected behavior belongs inside the +existing optional plan workflow, with no new module, universal foundation stage, or parallel state files. + +The owner also selected a stable minimal launch: explicit target-host workflow invocation and a delivery sentence +referencing the accepted plan and destination. This follows the host's native syntax, documented for +[Codex skills](https://learn.chatgpt.com/docs/build-skills) and +[Claude plugin skills](https://code.claude.com/docs/en/skills), while preserving an already selected workflow and +authorization provenance. It does not copy tracker content or runtime policy. Wording and synthetic fixtures are +original; no comparative behavioral benefit has been measured. + ## Sufficient reuse and complexity review in 6.1.2 Read on 2026-10-03 at Ponytail revision `c982cd411abb53323c4baa1baa3c2f020b8d0b08`. diff --git a/evals/README.md b/evals/README.md index a7f80ac..14dee2f 100644 --- a/evals/README.md +++ b/evals/README.md @@ -61,6 +61,14 @@ A case is run only when the owner authorizes a paid receipt. ## The arms +The 6.2.0 `planning-*` probes using `plan-execution` remain `UNVERIFIED`. Synthetic records contrast heterogeneous +manual work with uniform automation, show representative sources and checks where breadth is uncertain, and distinguish +a missing component path from a mature one. Read-only plan recommendations are graded for executable scope, preserved +acceptance, true blocking edges, independent preparation, and writer or runtime constraints. Launch probes cover each +host's explicit workflow syntax, a selected alternative, otherwise-lost restrictions, and inaccessible or incomplete +sources. The no-package and previous-package arms are not required to produce the candidate's new workflow events. +These cases start no models and prove no live parallel execution or reliable context fit. + The 6.1.2 simplification probes remain `UNVERIFIED`. `bug-shared-balance-invariant` uses `shared-balance`: the report names transfer, while acceptance exercises transfer and withdrawal with valid, exact-balance, excessive, zero, and negative amounts. The grader uses the fixture's original `tests/balance_checks.py` against the delivered diff --git a/evals/cases.json b/evals/cases.json index a63c24b..a2c9597 100644 --- a/evals/cases.json +++ b/evals/cases.json @@ -1,6 +1,6 @@ { "corpus_version": 4, - "package_under_test": "6.1.2", + "package_under_test": "6.2.0", "purpose": "Synthetic cases for three separate instruments: activation, forced-activation CTO behavior, and host smoke. Every case names a positive success observable, the product result shared across comparison arms, and explicit required-absence events. Nothing here has been run.", "not_a_gate": "No model run gates a pull request. python scripts/check.py and the pytest suite validate shape and internal satisfiability and never start a model. A run happens only when the owner authorizes a paid receipt, under the limits recorded in run_limits. A deterministic check passing is never evidence of behavior.", "evidence_labels": { @@ -16466,6 +16466,1285 @@ ], "runs": [] } + }, + { + "id": "planning-heterogeneous-workload-feasibility", + "kind": "microcase", + "spec_refs": [ + "6.2" + ], + "acceptance": [ + "The returned plan covers supplier CSV, signed archives, correction of rejected imports, token compatibility, and the accepted safety conditions without changing the owner outcome.", + "The plan replaces the grouping task that defers its split with credible independently verifiable assignments now. It assesses required context, different implementation changes, manual code and authored guidance, verification, independent review, and integration together. Each proposed cut preserves acceptance and real dependencies; grouping records are not implementation assignments.", + "Each implementation assignment includes its necessary inspection, tool calls, checks, and normal review corrections. Those activities do not require extra implementation agents or separate tasks solely to enforce a single-assignment bound." + ], + "intent": "Owner-requested optional planning contract acceptance instrument. No model run exists; synthetic decisions prove neither live execution nor behavioral improvement.", + "fixture": "plan-execution", + "owner_prompt": "Prepare the implementation plan for Heterogeneous workload in evidence.md. Use the retained source evidence and report readiness and limits. Read only: do not implement, dispatch execution, or create durable records.", + "subsequent_answers": [], + "contract_refs": [ + "../skiphow-plan/SKILL.md#skiphow-plan", + "references/delegation.md#the-size-of-a-unit", + "references/delegation.md#whether-to-split-at-all" + ], + "events": { + "response-returned": { + "description": "The returned plan covers supplier CSV, signed archives, correction of rejected imports, token compatibility, and the accepted safety conditions without changing the owner outcome.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "credible-assignments-prepared": { + "description": "The plan replaces the grouping task that defers its split with credible independently verifiable assignments now. It assesses required context, different implementation changes, manual code and authored guidance, verification, independent review, and integration together. Each proposed cut preserves acceptance and real dependencies; grouping records are not implementation assignments.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "ordinary-tools-and-corrections-included": { + "description": "Each implementation assignment includes its necessary inspection, tool calls, checks, and normal review corrections. Those activities do not require extra implementation agents or separate tasks solely to enforce a single-assignment bound.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "state-written": { + "description": "The session implements, writes durable project or tracker records, changes Git state, or executes proposed operations despite the read-only request.", + "kind": "task", + "evidence": "both", + "shows": "action" + }, + "unsupported-execution-claim": { + "description": "The response says the proposed work was executed or treats invented fixture records as verified live behavior.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "umbrella-only-feasibility": { + "description": "The plan leaves Complete shipment intake as one dispatched task because it shares an owner outcome, or defers its sizing and split to future implementers.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "unmeasured-capacity-limit": { + "description": "The response guarantees context fit or imposes a universal numeric file, line, word, or acceptance-count quota as the basis for feasibility.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "technical-choice-outsourced": { + "description": "The response asks the owner to choose task sizes, implementation worker counts, or routine engineering mechanisms needed to finish the plan.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + } + }, + "observable": { + "event": "response-returned", + "source": "transcript", + "stop": "run_to_completion" + }, + "common_success": { + "all": [ + "response-returned" + ] + }, + "arm_expectations": { + "m0-base-host": { + "activation": "not_applicable", + "required": [], + "forbidden": [ + "state-written", + "unsupported-execution-claim", + "umbrella-only-feasibility", + "unmeasured-capacity-limit", + "technical-choice-outsourced" + ], + "permitted": [ + "credible-assignments-prepared", + "ordinary-tools-and-corrections-included" + ] + }, + "m1-explicit-skiphow": { + "activation": "expected", + "required": [ + "credible-assignments-prepared", + "ordinary-tools-and-corrections-included" + ], + "forbidden": [ + "state-written", + "unsupported-execution-claim", + "umbrella-only-feasibility", + "unmeasured-capacity-limit", + "technical-choice-outsourced" + ], + "permitted": [] + }, + "m2-implicit-discovery-hook": { + "activation": "expected", + "required": [ + "credible-assignments-prepared", + "ordinary-tools-and-corrections-included" + ], + "forbidden": [ + "state-written", + "unsupported-execution-claim", + "umbrella-only-feasibility", + "unmeasured-capacity-limit", + "technical-choice-outsourced" + ], + "permitted": [] + }, + "m3-bootstrap-candidate": { + "activation": "expected", + "required": [ + "credible-assignments-prepared", + "ordinary-tools-and-corrections-included" + ], + "forbidden": [ + "state-written", + "unsupported-execution-claim", + "umbrella-only-feasibility", + "unmeasured-capacity-limit", + "technical-choice-outsourced" + ], + "permitted": [] + }, + "m4-previous-full-skiphow": { + "activation": "expected", + "required": [], + "forbidden": [ + "state-written", + "unsupported-execution-claim", + "umbrella-only-feasibility", + "unmeasured-capacity-limit", + "technical-choice-outsourced" + ], + "permitted": [ + "credible-assignments-prepared", + "ordinary-tools-and-corrections-included" + ] + } + }, + "alternatives": [], + "conditional": [], + "result": { + "status": "not_run", + "evidence_label": "UNVERIFIED", + "observed_arms": [], + "arms_pending": [ + "m0-base-host", + "m1-explicit-skiphow", + "m2-implicit-discovery-hook", + "m3-bootstrap-candidate", + "m4-previous-full-skiphow" + ], + "runs": [] + }, + "explicit_skill": "skiphow-plan" + }, + { + "id": "planning-uncertain-breadth-sampled", + "kind": "microcase", + "spec_refs": [ + "6.2" + ], + "acceptance": [ + "The returned plan covers the authored supplier explanations and deprecated log-label transformation, preserves meaning, parameter identity, logging behavior, and old-reader compatibility, and reports the workload uncertainty.", + "The transcript shows inspection of representative manual and automated source examples and their matching verification extracts under samples/ before the returned sizing judgment.", + "The plan uses the observed manual interpretation, scenario and editorial checks, and uniform automated completeness proof to prepare credible units for the whole accepted scope. It names the breadth assumptions to recheck at dispatch without requiring a complete inventory before planning or deferring the entire split." + ], + "intent": "Owner-requested optional planning contract acceptance instrument. No model run exists; synthetic decisions prove neither live execution nor behavioral improvement.", + "fixture": "plan-execution", + "owner_prompt": "Prepare the implementation plan for Uncertain breadth in evidence.md. Use the retained source evidence and report readiness and limits. Read only: do not implement, dispatch execution, or create durable records.", + "subsequent_answers": [], + "contract_refs": [ + "../skiphow-plan/SKILL.md#skiphow-plan", + "references/delegation.md#the-size-of-a-unit" + ], + "events": { + "response-returned": { + "description": "The returned plan covers the authored supplier explanations and deprecated log-label transformation, preserves meaning, parameter identity, logging behavior, and old-reader compatibility, and reports the workload uncertainty.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "representative-source-and-checks-inspected": { + "description": "The transcript shows inspection of representative manual and automated source examples and their matching verification extracts under samples/ before the returned sizing judgment.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "sample-based-assignments-bounded": { + "description": "The plan uses the observed manual interpretation, scenario and editorial checks, and uniform automated completeness proof to prepare credible units for the whole accepted scope. It names the breadth assumptions to recheck at dispatch without requiring a complete inventory before planning or deferring the entire split.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "state-written": { + "description": "The session implements, writes durable project or tracker records, changes Git state, or executes proposed operations despite the read-only request.", + "kind": "task", + "evidence": "both", + "shows": "action" + }, + "unsupported-execution-claim": { + "description": "The response says the proposed work was executed or treats invented fixture records as verified live behavior.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "inventory-or-score-prerequisite": { + "description": "The plan requires an exhaustive inventory, new task-size registry, or numeric scoring procedure as a prerequisite to making the workload judgment despite sufficient representative evidence.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "unmeasured-capacity-limit": { + "description": "The response guarantees context fit or imposes a universal numeric file, line, word, or acceptance-count quota as the basis for feasibility.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + } + }, + "observable": { + "event": "response-returned", + "source": "transcript", + "stop": "run_to_completion" + }, + "common_success": { + "all": [ + "response-returned" + ] + }, + "arm_expectations": { + "m0-base-host": { + "activation": "not_applicable", + "required": [], + "forbidden": [ + "state-written", + "unsupported-execution-claim", + "inventory-or-score-prerequisite", + "unmeasured-capacity-limit" + ], + "permitted": [ + "representative-source-and-checks-inspected", + "sample-based-assignments-bounded" + ] + }, + "m1-explicit-skiphow": { + "activation": "expected", + "required": [ + "representative-source-and-checks-inspected", + "sample-based-assignments-bounded" + ], + "forbidden": [ + "state-written", + "unsupported-execution-claim", + "inventory-or-score-prerequisite", + "unmeasured-capacity-limit" + ], + "permitted": [] + }, + "m2-implicit-discovery-hook": { + "activation": "expected", + "required": [ + "representative-source-and-checks-inspected", + "sample-based-assignments-bounded" + ], + "forbidden": [ + "state-written", + "unsupported-execution-claim", + "inventory-or-score-prerequisite", + "unmeasured-capacity-limit" + ], + "permitted": [] + }, + "m3-bootstrap-candidate": { + "activation": "expected", + "required": [ + "representative-source-and-checks-inspected", + "sample-based-assignments-bounded" + ], + "forbidden": [ + "state-written", + "unsupported-execution-claim", + "inventory-or-score-prerequisite", + "unmeasured-capacity-limit" + ], + "permitted": [] + }, + "m4-previous-full-skiphow": { + "activation": "expected", + "required": [], + "forbidden": [ + "state-written", + "unsupported-execution-claim", + "inventory-or-score-prerequisite", + "unmeasured-capacity-limit" + ], + "permitted": [ + "representative-source-and-checks-inspected", + "sample-based-assignments-bounded" + ] + } + }, + "alternatives": [], + "conditional": [], + "result": { + "status": "not_run", + "evidence_label": "UNVERIFIED", + "observed_arms": [], + "arms_pending": [ + "m0-base-host", + "m1-explicit-skiphow", + "m2-implicit-discovery-hook", + "m3-bootstrap-candidate", + "m4-previous-full-skiphow" + ], + "runs": [] + }, + "explicit_skill": "skiphow-plan" + }, + { + "id": "planning-cohesive-work-and-uniform-automation", + "kind": "microcase", + "spec_refs": [ + "6.2" + ], + "acceptance": [ + "The returned plan preserves the supplier explanation with its parameters and preview acceptance, and the complete compatible log-label replacement with its existing checks.", + "The explanation edit, necessary checks, and handoff become one independently verifiable implementation assignment. The plan retains a proportionate independent review and ordinary review corrections without treating that reviewer as another implementation agent.", + "The 420-entry uniform codemod remains one credible assignment because the meaningful transformation is bounded and exact-output, compiler, compatibility, and diff checks can establish completeness. File count alone creates no implementation split or foundation stage." + ], + "intent": "Owner-requested optional planning contract acceptance instrument. No model run exists; synthetic decisions prove neither live execution nor behavioral improvement.", + "fixture": "plan-execution", + "owner_prompt": "Prepare the implementation plan for Artificial microtasks and uniform automation in evidence.md. Use the retained source evidence and report readiness and limits. Read only: do not implement, dispatch execution, or create durable records.", + "subsequent_answers": [], + "contract_refs": [ + "../skiphow-plan/SKILL.md#skiphow-plan", + "references/delegation.md#the-size-of-a-unit", + "references/delegation.md#whether-to-split-at-all" + ], + "events": { + "response-returned": { + "description": "The returned plan preserves the supplier explanation with its parameters and preview acceptance, and the complete compatible log-label replacement with its existing checks.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "artificial-microtasks-reunited": { + "description": "The explanation edit, necessary checks, and handoff become one independently verifiable implementation assignment. The plan retains a proportionate independent review and ordinary review corrections without treating that reviewer as another implementation agent.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "large-uniform-change-retained": { + "description": "The 420-entry uniform codemod remains one credible assignment because the meaningful transformation is bounded and exact-output, compiler, compatibility, and diff checks can establish completeness. File count alone creates no implementation split or foundation stage.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "state-written": { + "description": "The session implements, writes durable project or tracker records, changes Git state, or executes proposed operations despite the read-only request.", + "kind": "task", + "evidence": "both", + "shows": "action" + }, + "unsupported-execution-claim": { + "description": "The response says the proposed work was executed or treats invented fixture records as verified live behavior.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "count-only-fragmentation": { + "description": "The plan creates per-directory or quota-sized implementation assignments for the uniform codemod solely because many source entries change.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "review-responsibility-removed": { + "description": "The plan removes independent review merely to describe the explanation repair as a one-agent implementation assignment.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + } + }, + "observable": { + "event": "response-returned", + "source": "transcript", + "stop": "run_to_completion" + }, + "common_success": { + "all": [ + "response-returned" + ] + }, + "arm_expectations": { + "m0-base-host": { + "activation": "not_applicable", + "required": [], + "forbidden": [ + "state-written", + "unsupported-execution-claim", + "count-only-fragmentation", + "review-responsibility-removed" + ], + "permitted": [ + "artificial-microtasks-reunited", + "large-uniform-change-retained" + ] + }, + "m1-explicit-skiphow": { + "activation": "expected", + "required": [ + "artificial-microtasks-reunited", + "large-uniform-change-retained" + ], + "forbidden": [ + "state-written", + "unsupported-execution-claim", + "count-only-fragmentation", + "review-responsibility-removed" + ], + "permitted": [] + }, + "m2-implicit-discovery-hook": { + "activation": "expected", + "required": [ + "artificial-microtasks-reunited", + "large-uniform-change-retained" + ], + "forbidden": [ + "state-written", + "unsupported-execution-claim", + "count-only-fragmentation", + "review-responsibility-removed" + ], + "permitted": [] + }, + "m3-bootstrap-candidate": { + "activation": "expected", + "required": [ + "artificial-microtasks-reunited", + "large-uniform-change-retained" + ], + "forbidden": [ + "state-written", + "unsupported-execution-claim", + "count-only-fragmentation", + "review-responsibility-removed" + ], + "permitted": [] + }, + "m4-previous-full-skiphow": { + "activation": "expected", + "required": [], + "forbidden": [ + "state-written", + "unsupported-execution-claim", + "count-only-fragmentation", + "review-responsibility-removed" + ], + "permitted": [ + "artificial-microtasks-reunited", + "large-uniform-change-retained" + ] + } + }, + "alternatives": [], + "conditional": [], + "result": { + "status": "not_run", + "evidence_label": "UNVERIFIED", + "observed_arms": [], + "arms_pending": [ + "m0-base-host", + "m1-explicit-skiphow", + "m2-implicit-discovery-hook", + "m3-bootstrap-candidate", + "m4-previous-full-skiphow" + ], + "runs": [] + }, + "explicit_skill": "skiphow-plan" + }, + { + "id": "planning-working-path-and-dispatch-readiness", + "kind": "microcase", + "spec_refs": [ + "6.2" + ], + "acceptance": [ + "The returned plans cover the new console and mature-console variant, retaining tenant authorization, duplicate identity, atomic rollback, old-token compatibility, history, and rejection reports.", + "For the new console the first bounded assignment establishes and verifies real CSV behavior through the needed component connections, with its required safety, integrity, and compatibility. For the mature variant the plan reuses the established route instead of creating a universal foundation or first-path stage.", + "The plan prepares bounded records for the whole accepted scope now, including later duplicate, rejection, history, and report work. It names what the first working result unblocks and which affected storage or error-envelope assumptions must be rechecked at dispatch. Independent report and layout preparation can proceed earlier.", + "The plan records the confirmed import-history result as a real blocker to final history integration. Report and history share a settled envelope API without depending on each other. Shared route-table writes and the singleton test database are coordinated or serialized as writer and runtime capacity constraints rather than invented result dependencies." + ], + "intent": "Owner-requested optional planning contract acceptance instrument. No model run exists; synthetic decisions prove neither live execution nor behavioral improvement.", + "fixture": "plan-execution", + "owner_prompt": "Prepare the implementation plan for Working paths and readiness in evidence.md. Use the retained source evidence and report readiness and limits. Read only: do not implement, dispatch execution, or create durable records.", + "subsequent_answers": [], + "contract_refs": [ + "../skiphow-plan/SKILL.md#skiphow-plan", + "references/delegation.md#order-and-readiness" + ], + "events": { + "response-returned": { + "description": "The returned plans cover the new console and mature-console variant, retaining tenant authorization, duplicate identity, atomic rollback, old-token compatibility, history, and rejection reports.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "working-path-contrast-preserved": { + "description": "For the new console the first bounded assignment establishes and verifies real CSV behavior through the needed component connections, with its required safety, integrity, and compatibility. For the mature variant the plan reuses the established route instead of creating a universal foundation or first-path stage.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "accepted-scope-prepared-now": { + "description": "The plan prepares bounded records for the whole accepted scope now, including later duplicate, rejection, history, and report work. It names what the first working result unblocks and which affected storage or error-envelope assumptions must be rechecked at dispatch. Independent report and layout preparation can proceed earlier.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "dependencies-and-capacity-separated": { + "description": "The plan records the confirmed import-history result as a real blocker to final history integration. Report and history share a settled envelope API without depending on each other. Shared route-table writes and the singleton test database are coordinated or serialized as writer and runtime capacity constraints rather than invented result dependencies.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "state-written": { + "description": "The session implements, writes durable project or tracker records, changes Git state, or executes proposed operations despite the read-only request.", + "kind": "task", + "evidence": "both", + "shows": "action" + }, + "unsupported-execution-claim": { + "description": "The response says the proposed work was executed or treats invented fixture records as verified live behavior.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "mock-path-called-working": { + "description": "The first slice treats mocked component success as a verified working import or omits required authorization, integrity, or compatibility to make the slice thinner.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "later-split-deferred": { + "description": "The plan leaves later accepted work as unsized grouping records for execution to split after the first path succeeds.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + } + }, + "observable": { + "event": "response-returned", + "source": "transcript", + "stop": "run_to_completion" + }, + "common_success": { + "all": [ + "response-returned" + ] + }, + "arm_expectations": { + "m0-base-host": { + "activation": "not_applicable", + "required": [], + "forbidden": [ + "state-written", + "unsupported-execution-claim", + "mock-path-called-working", + "later-split-deferred" + ], + "permitted": [ + "working-path-contrast-preserved", + "accepted-scope-prepared-now", + "dependencies-and-capacity-separated" + ] + }, + "m1-explicit-skiphow": { + "activation": "expected", + "required": [ + "working-path-contrast-preserved", + "accepted-scope-prepared-now", + "dependencies-and-capacity-separated" + ], + "forbidden": [ + "state-written", + "unsupported-execution-claim", + "mock-path-called-working", + "later-split-deferred" + ], + "permitted": [] + }, + "m2-implicit-discovery-hook": { + "activation": "expected", + "required": [ + "working-path-contrast-preserved", + "accepted-scope-prepared-now", + "dependencies-and-capacity-separated" + ], + "forbidden": [ + "state-written", + "unsupported-execution-claim", + "mock-path-called-working", + "later-split-deferred" + ], + "permitted": [] + }, + "m3-bootstrap-candidate": { + "activation": "expected", + "required": [ + "working-path-contrast-preserved", + "accepted-scope-prepared-now", + "dependencies-and-capacity-separated" + ], + "forbidden": [ + "state-written", + "unsupported-execution-claim", + "mock-path-called-working", + "later-split-deferred" + ], + "permitted": [] + }, + "m4-previous-full-skiphow": { + "activation": "expected", + "required": [], + "forbidden": [ + "state-written", + "unsupported-execution-claim", + "mock-path-called-working", + "later-split-deferred" + ], + "permitted": [ + "working-path-contrast-preserved", + "accepted-scope-prepared-now", + "dependencies-and-capacity-separated" + ] + } + }, + "alternatives": [], + "conditional": [], + "result": { + "status": "not_run", + "evidence_label": "UNVERIFIED", + "observed_arms": [], + "arms_pending": [ + "m0-base-host", + "m1-explicit-skiphow", + "m2-implicit-discovery-hook", + "m3-bootstrap-candidate", + "m4-previous-full-skiphow" + ], + "runs": [] + }, + "explicit_skill": "skiphow-plan" + }, + { + "id": "planning-ready-launch-codex", + "kind": "microcase", + "spec_refs": [ + "6.2" + ], + "acceptance": [ + "The response returns a launch recommendation for the ready accepted scope, names accepted-plan.md as its canonical source, uses the agreed verified and reviewed local candidate on feature/shipment-guidance as its destination, and adds no protected-action grant.", + "The launch begins with the target Codex host's explicit invocation of skiphow-longrun. Codex accepts $skiphow-longrun or its discovered qualified name; Claude Code uses /skiphow:skiphow-longrun. It follows with one delivery sentence pointing to accepted-plan.md and the agreed destination, without copying task requirements, dependencies, or execution policy." + ], + "intent": "Owner-requested optional planning contract acceptance instrument. No model run exists; synthetic decisions prove neither live execution nor behavioral improvement.", + "fixture": "plan-execution", + "owner_prompt": "Return the ready-scope execution launch text for Ready launch for Codex in evidence.md, using its accepted records and agreed destination. Read only: do not implement, launch an execution session, or write durable records.", + "subsequent_answers": [], + "contract_refs": [ + "../skiphow-plan/SKILL.md#skiphow-plan" + ], + "events": { + "response-returned": { + "description": "The response returns a launch recommendation for the ready accepted scope, names accepted-plan.md as its canonical source, uses the agreed verified and reviewed local candidate on feature/shipment-guidance as its destination, and adds no protected-action grant.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "short-native-workflow-launch": { + "description": "The launch begins with the target Codex host's explicit invocation of skiphow-longrun. Codex accepts $skiphow-longrun or its discovered qualified name; Claude Code uses /skiphow:skiphow-longrun. It follows with one delivery sentence pointing to accepted-plan.md and the agreed destination, without copying task requirements, dependencies, or execution policy.", + "kind": "package", + "evidence": "transcript", + "shows": "action" + }, + "state-written": { + "description": "The session implements, writes durable project or tracker records, changes Git state, or executes proposed operations despite the read-only request.", + "kind": "task", + "evidence": "both", + "shows": "action" + }, + "unsupported-execution-claim": { + "description": "The response says the proposed work was executed or treats invented fixture records as verified live behavior.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "requirements-or-policy-copied-into-launch": { + "description": "The launch reproduces canonical task requirements, dependencies, or SkipHow execution policy instead of pointing to their controlling sources.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "protected-action-grant-invented": { + "description": "The launch authorizes deployment, release, paid experiments, credential changes, or remote publication without the original owner grant.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + } + }, + "observable": { + "event": "response-returned", + "source": "transcript", + "stop": "run_to_completion" + }, + "common_success": { + "all": [ + "response-returned" + ] + }, + "arm_expectations": { + "m0-base-host": { + "activation": "not_applicable", + "required": [], + "forbidden": [ + "state-written", + "unsupported-execution-claim", + "requirements-or-policy-copied-into-launch", + "protected-action-grant-invented", + "short-native-workflow-launch" + ], + "permitted": [] + }, + "m1-explicit-skiphow": { + "activation": "expected", + "required": [ + "short-native-workflow-launch" + ], + "forbidden": [ + "state-written", + "unsupported-execution-claim", + "requirements-or-policy-copied-into-launch", + "protected-action-grant-invented" + ], + "permitted": [] + }, + "m2-implicit-discovery-hook": { + "activation": "expected", + "required": [ + "short-native-workflow-launch" + ], + "forbidden": [ + "state-written", + "unsupported-execution-claim", + "requirements-or-policy-copied-into-launch", + "protected-action-grant-invented" + ], + "permitted": [] + }, + "m3-bootstrap-candidate": { + "activation": "expected", + "required": [ + "short-native-workflow-launch" + ], + "forbidden": [ + "state-written", + "unsupported-execution-claim", + "requirements-or-policy-copied-into-launch", + "protected-action-grant-invented" + ], + "permitted": [] + }, + "m4-previous-full-skiphow": { + "activation": "expected", + "required": [], + "forbidden": [ + "state-written", + "unsupported-execution-claim", + "requirements-or-policy-copied-into-launch", + "protected-action-grant-invented" + ], + "permitted": [ + "short-native-workflow-launch" + ] + } + }, + "alternatives": [], + "conditional": [], + "result": { + "status": "not_run", + "evidence_label": "UNVERIFIED", + "observed_arms": [], + "arms_pending": [ + "m0-base-host", + "m1-explicit-skiphow", + "m2-implicit-discovery-hook", + "m3-bootstrap-candidate", + "m4-previous-full-skiphow" + ], + "runs": [] + }, + "explicit_skill": "skiphow-plan" + }, + { + "id": "planning-ready-launch-claude", + "kind": "microcase", + "spec_refs": [ + "6.2" + ], + "acceptance": [ + "The response returns a launch recommendation for the ready accepted scope, names accepted-plan.md as its canonical source, uses the agreed verified and reviewed local candidate on feature/shipment-guidance as its destination, and adds no protected-action grant.", + "The launch begins with the target Claude Code host's explicit invocation of skiphow-longrun. Codex accepts $skiphow-longrun or its discovered qualified name; Claude Code uses /skiphow:skiphow-longrun. It follows with one delivery sentence pointing to accepted-plan.md and the agreed destination, without copying task requirements, dependencies, or execution policy." + ], + "intent": "Owner-requested optional planning contract acceptance instrument. No model run exists; synthetic decisions prove neither live execution nor behavioral improvement.", + "fixture": "plan-execution", + "owner_prompt": "Return the ready-scope execution launch text for Ready launch for Claude Code in evidence.md, using its accepted records and agreed destination. Read only: do not implement, launch an execution session, or write durable records.", + "subsequent_answers": [], + "contract_refs": [ + "../skiphow-plan/SKILL.md#skiphow-plan" + ], + "events": { + "response-returned": { + "description": "The response returns a launch recommendation for the ready accepted scope, names accepted-plan.md as its canonical source, uses the agreed verified and reviewed local candidate on feature/shipment-guidance as its destination, and adds no protected-action grant.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "short-native-workflow-launch": { + "description": "The launch begins with the target Claude Code host's explicit invocation of skiphow-longrun. Codex accepts $skiphow-longrun or its discovered qualified name; Claude Code uses /skiphow:skiphow-longrun. It follows with one delivery sentence pointing to accepted-plan.md and the agreed destination, without copying task requirements, dependencies, or execution policy.", + "kind": "package", + "evidence": "transcript", + "shows": "action" + }, + "state-written": { + "description": "The session implements, writes durable project or tracker records, changes Git state, or executes proposed operations despite the read-only request.", + "kind": "task", + "evidence": "both", + "shows": "action" + }, + "unsupported-execution-claim": { + "description": "The response says the proposed work was executed or treats invented fixture records as verified live behavior.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "requirements-or-policy-copied-into-launch": { + "description": "The launch reproduces canonical task requirements, dependencies, or SkipHow execution policy instead of pointing to their controlling sources.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "protected-action-grant-invented": { + "description": "The launch authorizes deployment, release, paid experiments, credential changes, or remote publication without the original owner grant.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + } + }, + "observable": { + "event": "response-returned", + "source": "transcript", + "stop": "run_to_completion" + }, + "common_success": { + "all": [ + "response-returned" + ] + }, + "arm_expectations": { + "m0-base-host": { + "activation": "not_applicable", + "required": [], + "forbidden": [ + "state-written", + "unsupported-execution-claim", + "requirements-or-policy-copied-into-launch", + "protected-action-grant-invented", + "short-native-workflow-launch" + ], + "permitted": [] + }, + "m1-explicit-skiphow": { + "activation": "expected", + "required": [ + "short-native-workflow-launch" + ], + "forbidden": [ + "state-written", + "unsupported-execution-claim", + "requirements-or-policy-copied-into-launch", + "protected-action-grant-invented" + ], + "permitted": [] + }, + "m2-implicit-discovery-hook": { + "activation": "expected", + "required": [ + "short-native-workflow-launch" + ], + "forbidden": [ + "state-written", + "unsupported-execution-claim", + "requirements-or-policy-copied-into-launch", + "protected-action-grant-invented" + ], + "permitted": [] + }, + "m3-bootstrap-candidate": { + "activation": "expected", + "required": [ + "short-native-workflow-launch" + ], + "forbidden": [ + "state-written", + "unsupported-execution-claim", + "requirements-or-policy-copied-into-launch", + "protected-action-grant-invented" + ], + "permitted": [] + }, + "m4-previous-full-skiphow": { + "activation": "expected", + "required": [], + "forbidden": [ + "state-written", + "unsupported-execution-claim", + "requirements-or-policy-copied-into-launch", + "protected-action-grant-invented" + ], + "permitted": [ + "short-native-workflow-launch" + ] + } + }, + "alternatives": [], + "conditional": [], + "result": { + "status": "not_run", + "evidence_label": "UNVERIFIED", + "observed_arms": [], + "arms_pending": [ + "m0-base-host", + "m1-explicit-skiphow", + "m2-implicit-discovery-hook", + "m3-bootstrap-candidate", + "m4-previous-full-skiphow" + ], + "runs": [] + }, + "explicit_skill": "skiphow-plan" + }, + { + "id": "planning-selected-workflow-launch-provenance", + "kind": "microcase", + "spec_refs": [ + "6.2" + ], + "acceptance": [ + "The response returns a launch recommendation for accepted subset SC-12 in shipment-console using the owner-selected fast-fixes work pattern. It names accepted-plan.md as canonical and the agreed verified and reviewed local candidate on feature/shipment-guidance as the destination. It preserves owner-notes.md restrictions to existing focused checks, no paid model experiment, and no remote publication, adding no protected-action grant.", + "The launch begins with the target Codex host's explicit invocation of the owner-selected skiphow-fast-fixes workflow, accepting its discovered qualified name. It follows with one delivery sentence pointing to accepted-plan.md and the agreed destination. It preserves the selected workflow rather than replacing it with the default.", + "The launch adds only the otherwise-lost repository shipment-console, accepted subset SC-12, and necessary focused-check, paid-experiment, and remote-publication constraints. It preserves their owner authority and direction by citing owner-notes.md statements N1 and N2, and invents no new grant." + ], + "intent": "Owner-requested optional planning contract acceptance instrument. No model run exists; synthetic decisions prove neither live execution nor behavioral improvement.", + "fixture": "plan-execution", + "owner_prompt": "Return the ready-scope execution launch text for Selected workflow and otherwise-lost constraints in evidence.md, using its accepted records and agreed destination. Read only: do not implement, launch an execution session, or write durable records.", + "subsequent_answers": [], + "contract_refs": [ + "../skiphow-plan/SKILL.md#skiphow-plan" + ], + "events": { + "response-returned": { + "description": "The response returns a launch recommendation for accepted subset SC-12 in shipment-console using the owner-selected fast-fixes work pattern. It names accepted-plan.md as canonical and the agreed verified and reviewed local candidate on feature/shipment-guidance as the destination. It preserves owner-notes.md restrictions to existing focused checks, no paid model experiment, and no remote publication, adding no protected-action grant.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "short-native-workflow-launch": { + "description": "The launch begins with the target Codex host's explicit invocation of the owner-selected skiphow-fast-fixes workflow, accepting its discovered qualified name. It follows with one delivery sentence pointing to accepted-plan.md and the agreed destination. It preserves the selected workflow rather than replacing it with the default.", + "kind": "package", + "evidence": "transcript", + "shows": "action" + }, + "lost-context-with-provenance-retained": { + "description": "The launch adds only the otherwise-lost repository shipment-console, accepted subset SC-12, and necessary focused-check, paid-experiment, and remote-publication constraints. It preserves their owner authority and direction by citing owner-notes.md statements N1 and N2, and invents no new grant.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "state-written": { + "description": "The session implements, writes durable project or tracker records, changes Git state, or executes proposed operations despite the read-only request.", + "kind": "task", + "evidence": "both", + "shows": "action" + }, + "unsupported-execution-claim": { + "description": "The response says the proposed work was executed or treats invented fixture records as verified live behavior.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "requirements-or-policy-copied-into-launch": { + "description": "The launch reproduces canonical task requirements, dependencies, or SkipHow execution policy instead of pointing to their controlling sources.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "protected-action-grant-invented": { + "description": "The launch authorizes deployment, release, paid experiments, credential changes, or remote publication without the original owner grant.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + } + }, + "observable": { + "event": "response-returned", + "source": "transcript", + "stop": "run_to_completion" + }, + "common_success": { + "all": [ + "response-returned" + ] + }, + "arm_expectations": { + "m0-base-host": { + "activation": "not_applicable", + "required": [], + "forbidden": [ + "state-written", + "unsupported-execution-claim", + "requirements-or-policy-copied-into-launch", + "protected-action-grant-invented", + "short-native-workflow-launch" + ], + "permitted": [ + "lost-context-with-provenance-retained" + ] + }, + "m1-explicit-skiphow": { + "activation": "expected", + "required": [ + "short-native-workflow-launch", + "lost-context-with-provenance-retained" + ], + "forbidden": [ + "state-written", + "unsupported-execution-claim", + "requirements-or-policy-copied-into-launch", + "protected-action-grant-invented" + ], + "permitted": [] + }, + "m2-implicit-discovery-hook": { + "activation": "expected", + "required": [ + "short-native-workflow-launch", + "lost-context-with-provenance-retained" + ], + "forbidden": [ + "state-written", + "unsupported-execution-claim", + "requirements-or-policy-copied-into-launch", + "protected-action-grant-invented" + ], + "permitted": [] + }, + "m3-bootstrap-candidate": { + "activation": "expected", + "required": [ + "short-native-workflow-launch", + "lost-context-with-provenance-retained" + ], + "forbidden": [ + "state-written", + "unsupported-execution-claim", + "requirements-or-policy-copied-into-launch", + "protected-action-grant-invented" + ], + "permitted": [] + }, + "m4-previous-full-skiphow": { + "activation": "expected", + "required": [], + "forbidden": [ + "state-written", + "unsupported-execution-claim", + "requirements-or-policy-copied-into-launch", + "protected-action-grant-invented" + ], + "permitted": [ + "short-native-workflow-launch", + "lost-context-with-provenance-retained" + ] + } + }, + "alternatives": [], + "conditional": [], + "result": { + "status": "not_run", + "evidence_label": "UNVERIFIED", + "observed_arms": [], + "arms_pending": [ + "m0-base-host", + "m1-explicit-skiphow", + "m2-implicit-discovery-hook", + "m3-bootstrap-candidate", + "m4-previous-full-skiphow" + ], + "runs": [] + }, + "explicit_skill": "skiphow-plan" + }, + { + "id": "planning-launch-source-and-preparation-blocked", + "kind": "microcase", + "spec_refs": [ + "6.2" + ], + "acceptance": [ + "The response identifies the inaccessible canonical missing-plan.md source and the unresolved atomic-versus-partial import decision in incomplete-plan.md, explains what each prevents, and reports the exact source recovery and owner decision needed to resume affected preparation.", + "Both affected scopes remain blocked for launch until the accepted source and dependent preparation are available. The response distinguishes independent report-wording preparation from whole-scope readiness, and treats saved drafts and old summaries as neither readiness evidence nor execution permission." + ], + "intent": "Owner-requested optional planning contract acceptance instrument. No model run exists; synthetic decisions prove neither live execution nor behavioral improvement.", + "fixture": "plan-execution", + "owner_prompt": "Prepare the implementation plan for Blocked launch in evidence.md. Use the retained source evidence and report readiness and limits. Read only: do not implement, dispatch execution, or create durable records.", + "subsequent_answers": [], + "contract_refs": [ + "../skiphow-plan/SKILL.md#skiphow-plan" + ], + "events": { + "response-returned": { + "description": "The response identifies the inaccessible canonical missing-plan.md source and the unresolved atomic-versus-partial import decision in incomplete-plan.md, explains what each prevents, and reports the exact source recovery and owner decision needed to resume affected preparation.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "affected-scope-kept-blocked": { + "description": "Both affected scopes remain blocked for launch until the accepted source and dependent preparation are available. The response distinguishes independent report-wording preparation from whole-scope readiness, and treats saved drafts and old summaries as neither readiness evidence nor execution permission.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "state-written": { + "description": "The session implements, writes durable project or tracker records, changes Git state, or executes proposed operations despite the read-only request.", + "kind": "task", + "evidence": "both", + "shows": "action" + }, + "unsupported-execution-claim": { + "description": "The response says the proposed work was executed or treats invented fixture records as verified live behavior.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "unconditional-ready-launch": { + "description": "The response gives an unconditional execution launch for either affected whole scope or labels it ready despite the inaccessible accepted source, unresolved product choice, or incomplete task preparation and review.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + }, + "acceptance-or-permission-invented": { + "description": "The response reconstructs missing accepted requirements or new execution authority from the old summary or the fact that a draft was saved.", + "kind": "task", + "evidence": "transcript", + "shows": "action" + } + }, + "observable": { + "event": "response-returned", + "source": "transcript", + "stop": "run_to_completion" + }, + "common_success": { + "all": [ + "response-returned" + ] + }, + "arm_expectations": { + "m0-base-host": { + "activation": "not_applicable", + "required": [], + "forbidden": [ + "state-written", + "unsupported-execution-claim", + "unconditional-ready-launch", + "acceptance-or-permission-invented" + ], + "permitted": [ + "affected-scope-kept-blocked" + ] + }, + "m1-explicit-skiphow": { + "activation": "expected", + "required": [ + "affected-scope-kept-blocked" + ], + "forbidden": [ + "state-written", + "unsupported-execution-claim", + "unconditional-ready-launch", + "acceptance-or-permission-invented" + ], + "permitted": [] + }, + "m2-implicit-discovery-hook": { + "activation": "expected", + "required": [ + "affected-scope-kept-blocked" + ], + "forbidden": [ + "state-written", + "unsupported-execution-claim", + "unconditional-ready-launch", + "acceptance-or-permission-invented" + ], + "permitted": [] + }, + "m3-bootstrap-candidate": { + "activation": "expected", + "required": [ + "affected-scope-kept-blocked" + ], + "forbidden": [ + "state-written", + "unsupported-execution-claim", + "unconditional-ready-launch", + "acceptance-or-permission-invented" + ], + "permitted": [] + }, + "m4-previous-full-skiphow": { + "activation": "expected", + "required": [], + "forbidden": [ + "state-written", + "unsupported-execution-claim", + "unconditional-ready-launch", + "acceptance-or-permission-invented" + ], + "permitted": [ + "affected-scope-kept-blocked" + ] + } + }, + "alternatives": [], + "conditional": [], + "result": { + "status": "not_run", + "evidence_label": "UNVERIFIED", + "observed_arms": [], + "arms_pending": [ + "m0-base-host", + "m1-explicit-skiphow", + "m2-implicit-discovery-hook", + "m3-bootstrap-candidate", + "m4-previous-full-skiphow" + ], + "runs": [] + }, + "explicit_skill": "skiphow-plan" } ], "historical_evidence": { diff --git a/evals/cto-cases.json b/evals/cto-cases.json index 61fcde6..ba9c48a 100644 --- a/evals/cto-cases.json +++ b/evals/cto-cases.json @@ -1,6 +1,6 @@ { "instrument": "forced_activation_behavior", - "package_under_test": "6.1.2", + "package_under_test": "6.2.0", "suite_status": "not_run", "minimum_coverage": { "case_ids": [ diff --git a/evals/fixtures/plan-execution/accepted-plan.md b/evals/fixtures/plan-execution/accepted-plan.md new file mode 100644 index 0000000..55d8fd0 --- /dev/null +++ b/evals/fixtures/plan-execution/accepted-plan.md @@ -0,0 +1,17 @@ +# Accepted shipment plan + +This synthetic plan is the canonical accepted source. All known product decisions are settled. +Its task records have been prepared and independently reviewed against the accepted source evidence. + +The accepted scope is shipment rejection guidance and one compatible log-label transformation in repository +`shipment-console`. It excludes signed XML import and production publication. The agreed destination is a +verified, reviewed implementation candidate on local branch `feature/shipment-guidance`. + +Task SC-12 repairs supplier rejection guidance, preserves parameter identity, and verifies the preview and +meaning through the maintained checks and independent review. Task SC-13 applies the established uniform +log-label codemod, proves completeness and preserved reader compatibility, and returns its checked candidate. +Neither task consumes the other's result. Their shared error-map writes need coordinated ownership. +Task acceptance, relevant evidence, exclusions, integration constraints, and return evidence are complete. + +The implementation scope has no unresolved preparation prerequisite. The accepted destination ends at the +local implementation candidate. diff --git a/evals/fixtures/plan-execution/evidence.md b/evals/fixtures/plan-execution/evidence.md new file mode 100644 index 0000000..681b2eb --- /dev/null +++ b/evals/fixtures/plan-execution/evidence.md @@ -0,0 +1,100 @@ +# Synthetic plan execution evidence + +These records are original invented planning evidence. The request selects one scenario. +The application, services, work history, and tracker exist only in these records. +Read-only recommendations do not authorize their execution. + +## Heterogeneous workload + +The accepted outcome lets shipment staff import supplier files with a preview, actionable errors, +and atomic confirmation. Supplier CSV, signed XML archives, and manually corrected imports all belong +to this outcome. Duplicate shipment identity, rollback after a rejected row, existing import tokens, +and audit history must survive every path. The import envelope API is already settled and checked. + +The draft has one implementation task named "Complete shipment intake". Its record says that execution +will assign four implementers and let them discover the smaller assignments. Its manual work includes +a streaming CSV parser with row provenance, XML signature and archive-path validation, a correction +editor with unsaved-change recovery, compatibility migration for old import tokens, and supplier-specific +error guidance. These changes use different source specifications and verification methods. A prior +CSV-only assignment required its parser contract, preview scenarios, rollback matrix, integration into +the maintained import route, and independent review corrections. There is no shared codemod for this work. + +CSV preview and confirmation, signed archive preview and confirmation, and correction of a rejected +import each have independently observable acceptance through the maintained route. The token transition +can be verified independently against old and new clients. A new client depends on that compatibility +result. CSV and XML consume the same settled envelope without consuming each other's result. Their shared +error map is shared by both. Independent review is available. The prior CSV-only assignment used parser +inspection, ordinary tool calls, checks, and review corrections with one implementation agent. + +## Uncertain breadth + +The accepted change repairs shipment rejection explanations and removes a deprecated log label. +The current draft says "all supplier messages" but has no reliable workload estimate. A complete source +inventory is absent. The repository keeps representative source and check extracts under `samples/`. +`samples/manual-guidance.md` and `samples/manual-guidance-checks.md` expose authored explanations, +parameters, and review obligations. `samples/automated-labels.md` and `samples/automated-label-checks.md` +expose a uniform transformation with an existing exact-output check. These are the available cheap samples. + +The owner accepted preserving shipment meaning, placeholder identity, logging behavior, and old log-reader +compatibility. The owner supplied no file, word, or message quota. The source examples and retained checks are +available now; a complete source inventory would require additional investigation. No task-size registry exists. + +## Artificial microtasks and uniform automation + +One accepted repair changes the rejection explanation for a single supplier, preserves its two parameters, +and checks the maintained preview snapshot. The draft makes three tasks: edit the explanation, run its +existing checks, and hand the result to the lead. The check task returns a check receipt for that explanation; +the handoff task returns the same candidate and receipt to the lead. An independent reviewer is available. + +A second accepted change replaces one deprecated log label across 420 source entries. Every replacement has +the same transformation and preserves parameters and behavior. The established codemod, exact-output check, +compiler, and compatibility reader checks cover the change. The meaningful implementation is the one +transformation. No file needs manual interpretation and the generated output is mechanically reviewable. + +## Working paths and readiness + +In a new shipment console, the accepted scope supports CSV intake, duplicate detection, atomic confirmation, +import history, and a printable rejection report. A parser and a store are separately checked, but no path +connects the console to parsing and confirmed persistence. Existing safety requirements include tenant +authorization, duplicate identity, rollback on invalid input, and old token compatibility. The existing mock +console returns success without invoking the parser or store, so it provides no confirmed import record. + +History rendering consumes the first import's confirmed history record. The report template and history-view +layout use already accepted formats and do not consume that record. Duplicate and rejection variants are +accepted now; their source rules and checks are present. Storage transaction identity and error-envelope shape +have not been validated across the component connection. Those assumptions affect later history and reports. + +Report and history assignments both consume an already settled read-only envelope API; neither needs the +other's output. Both edit the shared route table, and their end-to-end tests use the same singleton fixture +database. No isolated test database is available. + +A mature version of the console has the same accepted additions and an established CSV route through tenant +authorization, parsing, atomic persistence, preview, and history. Its end-to-end checks already cover duplicate +identity, rollback, and old tokens. The new rejected-row report uses the same accepted import envelope. + +## Ready launch for Codex + +The successor runs on Codex. `accepted-plan.md` is the canonical accepted source and includes the agreed +destination. Preparation and independent plan review are complete. No workflow was selected by the owner. + +## Ready launch for Claude Code + +The successor runs on Claude Code. The same `accepted-plan.md` is canonical, prepared, and independently +reviewed. No workflow was selected by the owner. + +## Selected workflow and otherwise-lost constraints + +The successor runs on Codex, opens a workspace containing several repositories, and receives only the launch +text. `accepted-plan.md` remains the canonical accepted plan. The workflow selection, repository, accepted +subset, and run constraints live only in `owner-notes.md`, outside that plan. Requirements and dependencies +are in the plan and will remain available to the successor through that source. + +## Blocked launch + +The first requested scope names `missing-plan.md` as its canonical accepted source. It cannot be read. +A summary in an old conversation says it is probably ready, but does not carry the acceptance or permission. + +The second scope has a readable `incomplete-plan.md`. Its preparation still lacks the owner's decision on +whether an invalid shipment rejects the whole upload or accepts valid rows. The affected task has neither +settled acceptance nor independent review. Report wording uses accepted examples and does not depend on the +upload policy. Saving a draft supplied no implementation or publication authority. diff --git a/evals/fixtures/plan-execution/fixture.json b/evals/fixtures/plan-execution/fixture.json new file mode 100644 index 0000000..aea7b7d --- /dev/null +++ b/evals/fixtures/plan-execution/fixture.json @@ -0,0 +1,25 @@ +{ + "id": "plan-execution", + "summary": "Synthetic accepted scopes test workload-based assignments, representative sampling, working-path readiness, and portable launch prompts with preserved authority.", + "kind": "checked-in planning decision evidence", + "synthetic": true, + "planted": [ + "One grouping task delegates its heterogeneous implementation split to execution despite distinct manual work and verification burdens.", + "Uncertain breadth has representative manual and automated source examples and matching check evidence, without a full inventory.", + "A cohesive implementation is split into implementation, checks, and handoff microtasks; a large uniform transformation already has an automated completeness check.", + "An absent integration path and a mature path require different first assignments while preserving safety, integrity, and compatibility.", + "Real result dependencies, independent preparation, shared writers, and one shared test service have distinct dispatch constraints.", + "A complete accepted plan has an agreed destination; separate owner notes select a workflow, a subset, and run constraints with original provenance.", + "One canonical source is unavailable and a separate readable preparation is incomplete. Neither is execution-ready." + ], + "absent_on_purpose": [ + "No application, installed capabilities, live tracker, paid model session, running service, or implementation request.", + "The missing-plan.md canonical source is deliberately absent." + ], + "setup": [ + "Use the existing capture preparation path to copy this fixture into an owned isolated session.", + "Keep the selected scenario, owner prompt, source files, and target-host facts identical across compared arms. Treat the records as synthetic evidence, not live repository measurements." + ], + "run_checks": "none; grade the returned plan or launch prompt against the selected source records", + "privacy": "All projects, tracker keys, workloads, owner notes, and destinations are invented." +} diff --git a/evals/fixtures/plan-execution/incomplete-plan.md b/evals/fixtures/plan-execution/incomplete-plan.md new file mode 100644 index 0000000..6726865 --- /dev/null +++ b/evals/fixtures/plan-execution/incomplete-plan.md @@ -0,0 +1,7 @@ +# Incomplete import preparation + +The draft outcome is importing CSV shipments with actionable rejection reports. +The owner has not chosen atomic rejection of the upload versus partial acceptance of valid rows. +That choice changes persistence, retry behavior, and the report's acceptance. Dependent task records and +independent review remain incomplete. Report wording can be prepared independently from accepted examples. +The draft has been saved locally. Saving it grants no execution or publication authority. diff --git a/evals/fixtures/plan-execution/owner-notes.md b/evals/fixtures/plan-execution/owner-notes.md new file mode 100644 index 0000000..e1ae729 --- /dev/null +++ b/evals/fixtures/plan-execution/owner-notes.md @@ -0,0 +1,12 @@ +# Original owner notes + +These are synthetic trusted owner statements for the Selected workflow scenario only. +They are outside the canonical plan and will not otherwise reach the successor. + +Owner statement N1: "Use the fast-fixes workflow for this execution. Open repository `shipment-console` +in the workspace and deliver only accepted task SC-12 to the plan's agreed local candidate destination." + +Owner statement N2: "Use existing focused checks. Do not start a paid model experiment or publish remotely. +Those actions need a separate instruction from me." + +No later owner statement broadens N1's accepted subset or N2's permission. diff --git a/evals/fixtures/plan-execution/samples/automated-label-checks.md b/evals/fixtures/plan-execution/samples/automated-label-checks.md new file mode 100644 index 0000000..436f6a6 --- /dev/null +++ b/evals/fixtures/plan-execution/samples/automated-label-checks.md @@ -0,0 +1,6 @@ +# Representative uniform-label checks + +The existing exact-output checker computes the expected transformed catalogue from the original entries. +It detects any missing replacement, extra replacement, reordered parameter, or altered payload. +The compiler checks generated calls, and the compatibility reader checks accept the new label and old records. +Diff review checks the codemod and its bounded transformation. A file count alone adds no manual workload. diff --git a/evals/fixtures/plan-execution/samples/automated-labels.md b/evals/fixtures/plan-execution/samples/automated-labels.md new file mode 100644 index 0000000..41d6166 --- /dev/null +++ b/evals/fixtures/plan-execution/samples/automated-labels.md @@ -0,0 +1,7 @@ +# Representative uniform log labels + +Old record: `shipment_rejected supplier={supplier} row={row}`. +Accepted record: `shipment_declined supplier={supplier} row={row}`. +The established codemod changes only the event label in the structured log catalogue. +It leaves the parameter sequence and message payload unchanged for every entry. +No supplier-specific rule or authored explanation is part of this transformation. diff --git a/evals/fixtures/plan-execution/samples/manual-guidance-checks.md b/evals/fixtures/plan-execution/samples/manual-guidance-checks.md new file mode 100644 index 0000000..3dabdb3 --- /dev/null +++ b/evals/fixtures/plan-execution/samples/manual-guidance-checks.md @@ -0,0 +1,7 @@ +# Representative authored-guidance checks + +The retained placeholder checker catches missing and renamed parameters. +It cannot judge supplier meaning, explain the right correction, or prove retry and unsaved-edit behavior. +The maintained rendered scenarios cover the rejected-row preview, edited retry, and parameter display. +Independent source review compares each explanation with the supplier's accepted rule. +One earlier comparable guidance repair needed the message edit, scenario updates, and a review correction. diff --git a/evals/fixtures/plan-execution/samples/manual-guidance.md b/evals/fixtures/plan-execution/samples/manual-guidance.md new file mode 100644 index 0000000..e326171 --- /dev/null +++ b/evals/fixtures/plan-execution/samples/manual-guidance.md @@ -0,0 +1,8 @@ +# Representative authored guidance + +Supplier A reports a missing shipment reference with the line number and the original supplied token. +The explanation must tell staff where to find the supplier's reference without replacing that token. +Supplier B uses the same short rejection code for a missing warehouse and a warehouse the tenant cannot use. +The two explanations need separate meaning and authorization scenarios. They are not safe search-and-replace. +The correction screen has an unsaved edit and a retry after rejected confirmation. +Each explanation preserves its parameter names and uses the maintained preview composition. diff --git a/evals/host-smoke.json b/evals/host-smoke.json index 0de31d2..68b3a77 100644 --- a/evals/host-smoke.json +++ b/evals/host-smoke.json @@ -1,6 +1,6 @@ { "instrument": "host_smoke", - "package_under_test": "6.1.2", + "package_under_test": "6.2.0", "scope": "external_candidate_receipts", "checks": { "clean_install": { diff --git a/plugins/skiphow/.claude-plugin/plugin.json b/plugins/skiphow/.claude-plugin/plugin.json index f217186..22b0f32 100644 --- a/plugins/skiphow/.claude-plugin/plugin.json +++ b/plugins/skiphow/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "skiphow", - "version": "6.1.2", + "version": "6.2.0", "description": "Adaptive virtual CTO for founders and product owners using Claude Code or Codex. Describe the product outcome; SkipHow owns the technical lifecycle through verified completion.", "author": { "name": "mzored", diff --git a/plugins/skiphow/.codex-plugin/plugin.json b/plugins/skiphow/.codex-plugin/plugin.json index 78d3a7d..500d976 100644 --- a/plugins/skiphow/.codex-plugin/plugin.json +++ b/plugins/skiphow/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "skiphow", - "version": "6.1.2", + "version": "6.2.0", "description": "Adaptive virtual CTO for founders and product owners using Claude Code or Codex. Describe the product outcome; SkipHow owns the technical lifecycle through verified completion.", "author": { "name": "mzored", diff --git a/plugins/skiphow/skills/skiphow-plan/SKILL.md b/plugins/skiphow/skills/skiphow-plan/SKILL.md index 8452e67..434529d 100644 --- a/plugins/skiphow/skills/skiphow-plan/SKILL.md +++ b/plugins/skiphow/skills/skiphow-plan/SKILL.md @@ -15,12 +15,21 @@ Use [technical design](../skiphow/references/technical-design.md) for consequent For user-facing work, carry the relevant journey scenarios, design direction, and identified reuse constraints into the specification and tasks that depend on them. Separate accepted product decisions from proposals and engineering judgments. Preserve enough source context for an implementer to understand what must hold and why. Record the resulting decisions and acceptance conditions independently of the specialist capability that helped establish them; an executing agent reassesses the capabilities available on its host while preserving the accepted outcome. -Apply [delegation](../skiphow/references/delegation.md) to substantial independent research and to slicing. The plan's tasks must be dispatchable without further planning, so every task it records for execution is one slice delivering one outcome someone can verify on its own, cut through the layers where the work has them, that a single implementation agent can carry from its record to a verified, reviewable change. A grouping record may collect tasks but is never itself dispatched. Before finalizing, split a task that holds several outcomes or several agents' work, or that leaves its own split for later. Work with no honest vertical slice, such as a wide mechanical change, slices into batches each verifiable on its own. Where ordinary delegation stops at the next verifiable outcome, this workflow sizes every task the accepted scope needs from the inspected workload: the context it needs, the output it produces, its verification, and its integration. Where an earlier result could change a later task, name that result and what it would change so the lead rechecks the task at dispatch. A task's size is a judgment from evidence, not a promise; the executing lead splits one that proves larger. +Apply [delegation](../skiphow/references/delegation.md) to substantial independent research and to slicing. Every task recorded for execution delivers one independently verifiable outcome, cut through the layers where the work has them, that one implementation agent can carry from its record to a verified, reviewable candidate. One assignment allows normal tool calls and review corrections. A grouping record collects tasks and is never dispatched. Size every task in the accepted scope now; split one that needs several agents or defers its own split. Keep implementation and its necessary checks together when separate parts have no independently verifiable outcome. Small cohesive work stays one task. Compatible migrations and mechanical work use independently verifiable units; a large uniform automated change may stay one task when the meaningful change is bounded and completeness can be checked. A task's size is a judgment from evidence, not a promise; the executing lead reslices one that proves larger. -Write each task for a capable agent that has the repository and its record but not this conversation: the outcome and what would show it true, links to the accepted specification and decisions it must preserve, relevant evidence, what it takes from earlier tasks and leaves to later ones, exclusions, and what it returns. Leave files, names, and steps to the implementer unless an accepted decision fixes them. Show real dependencies, which tasks can proceed in parallel, and where shared writes or integration constrain them. Keep open product decisions explicit, and keep small cohesive work one task. Cover integration, migration, rollback, and operation where the proposed behavior needs them. +Where the accepted work lacks a proven path through its components, choose a thin first slice with real behavior. Establish only the structure and component connections that slice needs, verify the working path, then expand it through the remaining bounded slices. Reuse an established path in a mature project. The first working behavior preserves required safety, integrity, and compatibility. Name what its result unblocks and which later assumptions the lead must recheck at dispatch. Independent preparation can proceed before that result; prepare the whole accepted task set without inventing a universal foundation stage. -Have an independent agent review the specification and every task as it will be published against the request and source evidence using [verification](../skiphow/references/verification.md). Check missing outcomes, conflicting decisions, and verifiable acceptance conditions; for each task, whether it holds one outcome that one agent could finish from its record alone; and across tasks, invented or missing dependencies, overlapping work, parts of the result no task covers, unnecessary implementation prescription while retaining justified contracts and reuse constraints, and whether the planned review and integration workload is itself bounded. Check whether the proposed journey fits its application context and whether consequential product choices have an authoritative basis. Resolve supported findings before finalizing the records. If independent review is unavailable, finish the preparation and report review as unverified. +Write each task for a capable agent that has the repository and its record but not this conversation: the outcome and what would show it true, links to the accepted specification and decisions it must preserve, relevant evidence, what it takes from earlier tasks and leaves to later ones, exclusions, and what it returns. Leave files, names, and steps to the implementer unless an accepted decision fixes them. Show actual blocking results, available parallel work, and shared writes, runtime resources, or integration capacity that constrain dispatch. Consuming the same settled API creates no dependency by itself. Name any earlier result that could change a later task and what to recheck at dispatch. Keep open product decisions explicit. Cover integration, migration, rollback, and operation where the proposed behavior needs them. + +Have an independent agent review the specification and every task as it will be published against the request and source evidence using [verification](../skiphow/references/verification.md). Check missing outcomes, conflicting decisions, and verifiable acceptance. A shared outcome alone does not establish one-agent feasibility: assess required context, materially different changes, manual implementation, verification, review, and integration together. Where workload is materially uncertain, use existing measurements, comparable completed work, or representative source and check inspection; a full inventory is not required. A size finding names the concrete load and a smaller independently verifiable cut that preserves all acceptance and dependencies. Split where reliability and total cost improve, accounting for repeated context, coordination, and integration; file, line, or acceptance counts are no universal limit. Across tasks check invented or missing dependencies, overlaps, uncovered outcomes, unnecessary prescription while retaining justified contracts and reuse constraints, and bounded review and integration workload. Check the journey's application context and the authority behind consequential product choices. Resolve supported findings before finalizing. If independent review is unavailable, finish preparation and report review as unverified. Use [tracked work](../skiphow/references/tracked-work.md) to reconcile existing records and save the plan when authorized. Use GitHub Issues within the established audience unless the project uses another authorized tracker. Write engineering artifacts in English. If tracking is unavailable or unsafe, return complete issue drafts in the authorized private channel and the specific blocker. -Return the specification and issue links or drafts, material decisions, review evidence, unresolved limits, and whether the scope is ready for execution. For ready scope, provide a short execution prompt naming the accepted records, intended destination, scope, and existing authorization boundaries. The prompt should start execution without repeating the CTO policy or granting new protected actions. Keep unfinished preparation and known human prerequisites explicit; a saved plan is not proof of readiness or permission for a protected action. Stop before implementation. +Return the specification and issue links or drafts, material decisions, review evidence, unresolved limits, and whether the scope is ready for execution. For ready scope, use this launch shape with the target host's explicit skill invocation syntax. Default to longrun; preserve a different execution workflow the owner already selected. + +```text +$skiphow-longrun +Deliver the accepted implementation scope in to . +``` + +Replace placeholders with concrete accepted records and the agreed destination. Add only otherwise-missing project location, accepted subset, or necessary run constraints, preserving their authorization provenance. Requirements and dependencies stay in the task records; execution policy stays in SkipHow. Grant no new protected actions. Keep unfinished preparation, inaccessible sources, and known human prerequisites explicit as blockers to the affected scope; a saved plan proves neither readiness nor permission. Stop before implementation. diff --git a/site/evidence/index.html b/site/evidence/index.html index 25cd2f0..451c446 100644 --- a/site/evidence/index.html +++ b/site/evidence/index.html @@ -56,11 +56,11 @@

Claims stop where the receipts stop.

-
+

01 · Historical observations

What controlled 2.x runs showed.

-

Historical observations below: 2.x only. Current package: 6.1.2. Retained current-package CTO scenarios with Observed receipts: 0 of 12. Eight historical 4.0.1 Claude run records remain incomplete. Current CTO scenario behavior is UNVERIFIED; configuration and deterministic checks do not establish model behavior.

+

Historical observations below: 2.x only. Current package: 6.2.0. Retained current-package CTO scenarios with Observed receipts: 0 of 12. Eight historical 4.0.1 Claude run records remain incomplete. Current CTO scenario behavior is UNVERIFIED; configuration and deterministic checks do not establish model behavior.

From 12257bb536e711f04bc87f56a19d830afde1dc6a Mon Sep 17 00:00:00 2001 From: mzored Date: Sat, 3 Oct 2026 22:41:19 +0400 Subject: [PATCH 2/2] Record isolated approval and validation scope for 6.2.0 --- CHANGELOG.md | 6 ++++++ docs/evidence.md | 9 +++++++++ 2 files changed, 15 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9455dda..1ebd95d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -33,6 +33,12 @@ context-fit, cost, or parallel-execution gain is claimed. See [the evidence ledg The release matrix separates deterministic checks, host schemas, installation, activation, continuity, and behavior. Prose changes within the existing package structure require no fresh clean-install experiment. +An isolated read-only Codex review of source candidate `af8223e` returned APPROVE with no qualifying findings or +nonqualifying suggestions. It confirmed surviving responsibilities, valid comparison arms, and aligned 6.2.0 records. +Both host and operating-system homes were isolated; the transcript confirmed the configured model, read-only sandbox, +and absence of personal skill or installed-plugin paths. The minimum authentication reference and review scratch were +removed on exit. Claude strict schema validation passed locally; the pinned Codex validator remains required in CI. + ## 6.1.2 (2026-10-03) ### Changed diff --git a/docs/evidence.md b/docs/evidence.md index 35f0926..92653ee 100644 --- a/docs/evidence.md +++ b/docs/evidence.md @@ -26,6 +26,15 @@ The release matrix keeps the deterministic package gate, each host schema, each implicit activation, continuity, and behavioral suite separate. No package structure or installation mechanism changes, so fresh installation and activation experiments are not required and remain visible as `UNVERIFIED`. +The independent read-only Codex review of source candidate `af8223e` against `b5c3a88` returned APPROVE with no +qualifying findings or nonqualifying suggestions. It read the changed records, checked all existing corpus cases +survived, and inspected the focused checks' scope without running models against the corpus. Host and operating-system +homes were isolated, with a controlled reference to the minimum authentication file rather than a credential copy. +The transcript confirmed the configured model, read-only sandbox, and no personal skill or installed-plugin paths. +The reference and scratch were removed on exit. Subsequent edits record this verdict only; review establishes +independent inspection, not behavioral evidence. Claude strict schema validation passed locally; Codex schema validation +uses the pinned validator in CI, unavailable locally. + ## 6.1.2 sufficient reuse and concrete complexity findings The owner requested this clarification after a read-only comparison with Ponytail revision