diff --git a/CHANGELOG.md b/CHANGELOG.md index 5deb79c..470989c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,35 @@ All notable changes to SkipHow 2.x and later appear in this file. Earlier release notes remain available on [GitHub Releases](https://github.com/mzored/SkipHow/releases). +## 6.3.3 (2026-10-08) + +Delegates, leads, and reviewers reuse a verifiable passing result when the command, tree fingerprint, +and relevant environment are unchanged. Checking a return, reviewing, integrating, or reporting does +not require executing it again. Changed inputs, missing or unverifiable evidence, or a concrete named +doubt justify another run. A different fingerprint never establishes a pass for the current tree. + +Verification defines the minimal record in existing logs or handoffs: command and arguments, checked +fingerprint, terminal exit, completion time, checkout, accessible evidence, and relevant environment. +Ignored configuration, scratch inputs, service state, and uncertain capture still limit applicability. +Required coverage, independent review, destination inspection, and `agent-verify inherited` remain intact. + +The fixed host-log sample found 28 same-label, same-fingerprint repeats in 116 completed runs, 24.1% +of runs and 9.6% of execution time. Command identity was absent, so this is a proxy, not a measured +same-command or avoidable-work share. No wording cause, behavioral improvement, or savings is claimed. +Five new arm-aware decision probes remain `UNVERIFIED`. No paid behavioral experiment or clean-install +session was run. Package structure and installation are unchanged; no host configuration was edited. +See [current evidence](docs/evidence.md#633-verification-receipt-reuse). + +Independent isolated Codex source review approved the candidate with no qualifying material findings. +It inspected runtime responsibilities, the five probes and arm expectations, version metadata, canonical +documentation, and site evidence. No finding required correction or was rejected. Source review proves +neither model adherence nor the private measurement records; the maintainer reproduced the aggregate. + +The local deterministic gate passed on the reviewed source. Its non-blocking headroom lint required +a bounded editorial tightening of delegation; focused package/document validation covers that correction. +Revision-bound CI supplies full evidence for the integrated candidate. Clean installs and behavioral +sessions stay `UNVERIFIED` in the release validation matrix. + ## 6.3.2 (2026-10-08) Contributor checks now use the optional installed `agent-verify` host queue before their subprocess diff --git a/VERSION b/VERSION index 91e4a9f..7849b73 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -6.3.2 +6.3.3 diff --git a/docs/decisions.md b/docs/decisions.md index 6078a4c..2bd2ebf 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -25,7 +25,7 @@ The live decisions, their premises, and what would reopen each. "Evidence" says | Read-only analysis writes nothing; established owner-authorized non-production workflows cover routine delivery and tracking, settling what may travel and not how often, so the lead groups related accepted results into reviewable delivery units without holding independent finished work for unrelated pending work; assignment handoff and final delivery can have different destinations, and local combination grants no publication; a series of owner-reviewed changes stays in iteration until the owner requests delivery or accepts with nothing further to send, and an invoked iteration workflow keeps its own boundary until delivery is requested | Explicit planning-skill invocation requests durable records, but automatic selection grants nothing; fast-fixes acceptance stays inside iteration; preparation and production remain separate grants | Owner-selected workflow contracts preserve ordinary SkipHow behavior, existing grants, audience boundaries, and downstream-effect checks | `Contract`; assignment and delivery wording aligned in 5.1.3 by inspection; workflow behavior `UNVERIFIED` | A receipt shows unauthorized effects, missing records requested by invocation, unnecessary permission questions, per-assignment publication, premature closure after local handoff, or unrelated blocked work delaying delivery | | Every branch, worktree, checkout, checkpoint, running service, and uncommitted change in a project is the lead's own engineering state, made by this session, an earlier one, or a delegate, and completion is relative to the authorized destination; on a request that changes the project, integrated leftovers of earlier sessions that nothing holds are retired as routine hygiene, a superseded or abandoned attempt with unique commits is retired on the lead's judgment with its commit named, uncommitted changes the run did not make and a live session's lane stay untouched, and none of it is attributed to the owner or a third party or handed back to them | The owner writes no code and no third party does, so state reported as someone else's is a false report and a decision handed to the one person who cannot make it; a named destination is incomplete until verified there; retiring one's own recoverable state is engineering, not the material deletion the exact-grant rule protects | Liveness is readable from changes still moving, an open review, a live record, or a session the host lists, and not from a name, a timestamp, or a running process alone; Git recovers a deleted branch's commit and nothing recovers an uncommitted file; a read-only request grants nothing | `Contract` (5.0.0; destination-relative completion 4.0; naming duty 4.8); integration and custody cases in `evals/`, not run | Receipts show a run retiring a live session's lane or an uncommitted file, retiring as abandoned an attempt whose defect is still present, retiring anything under a read-only request, a small change growing a sweep material beside it or custody ceremony in every report, a project where a human writes code put under SkipHow, or the owner asking for a retired branch back or for report-and-wait again | | No technical choice reaches the owner, and a running result shown to them names the branch or revision and the address it serves | The owner never handling engineering is the boundary this product exists to hold; a recommendation the owner can decline is not a question that stops the work | Reversible technical uncertainty is settled by taking the option the agent would defend; several checkouts of one project answer on the same default address | `Contract` (4.8); the corpus forbids technical approval, tracker interviews, and test-selection questions in five cases, none run | Receipts show the recommendation channel being used to hand product choices back, or an owner asking for the technical options they no longer see | -| Verification proves each property at the narrowest stable boundary that retains the needed fidelity; distinct high-fidelity evidence survives; recurring disproportionate cost is measured at its responsible layer; equivalent inputs reuse valid evidence | The 4.6.0 release repeated equivalent gates, while the new verification-health cases distinguish coupling, setup cost, unique cross-boundary confidence, and proportional iteration | The relevant code, dependencies, configuration, environment, artifact, and destination determine whether evidence applies; test category, suite size, and speed alone establish neither cause nor adequacy | One installed 4.6.0 release session supports evidence reuse; the 4.7.0 placement and diagnosis rules are `Contract` with deterministic corpus coverage; behavior is `UNVERIFIED` | A run drops unique evidence, normalizes disproportionate churn, optimizes an unmeasured cause, or makes ordinary work heavier without preserving delivery confidence | +| Verification proves each property at the narrowest stable boundary that retains the needed fidelity; distinct high-fidelity evidence survives; recurring disproportionate cost is measured at its responsible layer; matching command, tree fingerprint, and relevant environment reuse verifiable passing records across roles; changed inputs, unverifiable records, or named concrete doubts justify another execution | The 4.6.0 release repeated equivalent gates, while the new verification-health cases distinguish coupling, setup cost, unique cross-boundary confidence, and proportional iteration | The checked fingerprint identifies the actual state exercised; ignored or external inputs and destination remain relevant; a different fingerprint needs current-candidate evidence, including revision-bound CI; test category, suite size, and speed alone establish neither cause nor adequacy | One installed 4.6.0 release session supports evidence reuse; the 6.3.3 clarification is owner-requested `Contract`, with unrun arm-aware probes; causal and behavioral effects remain `UNVERIFIED` | A run drops unique evidence, normalizes disproportionate churn, optimizes an unmeasured cause, or makes ordinary work heavier without preserving delivery confidence | | Broad project-state reconciliation reconstructs before it mutates; status reports each leftover with the disposition a change request would give it, record correction changes records and retires nothing, and any request that changes the project retires integrated unheld leftovers; only a non-reconstructible outcome-to-workspace association becomes durable | Tracker state, branch names, merge status, and CI can each be stale or refer to the wrong outcome, revision, or destination; a parallel lifecycle registry would create another claim to reconcile | Live Git, reviews, destinations, and revision-specific validation establish every transition in the current cases; workspace ownership cannot always be reconstructed after a session ends | `Contract` and deterministic corpus validation (4.6.0; amended 5.0.0); behavior `UNVERIFIED` | A receipt finds live evidence insufficient to recover a material transition, the ownership association insufficient for safe recovery, or the ordering adding ceremony without preventing a failure | | Delegates are read-only without verified distinct isolation; the root serializes writes; lead and delegate model and effort are independent choices minimizing expected total cost within owner constraints; a route already shown adequate for comparable work survives; an unset control still resolves to some default, often the parent's route, so leaving it unset is a routing decision held to the same test; a brief states whether and how widely its delegate may dispatch delegates and at what route; a delegate receives the kernel's path and acts as CTO of its brief within the owner's authority the brief relays, fixing what it finds inside its surface and returning the rest, while the lead keeps owner questions, dispositions, and records; settings use actual host controls | One shared checkout has one index and one branch; a delegate's own account of its isolation is not proof; host controls change independently of this package; a review's independence and framing matter more than its level; a delegate may not have this guidance in context and its own delegates often inherit its route, so the brief is where nested fan-out is bounded | The active host supplies the controls a run can use; no portable writer-isolation or absolute-level interface exists across supported hosts, so unverifiable isolation takes the safe fallback | Failures `Observed` on 2.x (five lanes in one checkout; a worktree that reported success into the shared tree); the portable fallback is `Contract`, not run; writer lanes remain host-specific and must be verified; inherited delegate routes and unbriefed nested fan-out are readable in private 5.0.0 Claude sessions, and the 5.0.1 wording is `Contract`; one private 6.0.0 Claude session motivates the 6.1.0 delegate reading, which is `Contract` | A portable capability interface appears, a host makes isolation verifiable and default, or paired runs settle the routing cost question | | Prefer host-native execution; admit thin bindings when a demonstrated gap justifies them | Duplicated runtime state increases cost; optional skills reuse one kernel without a service or custom workers | Required outcomes survive implementation changes; new components need host schema, safety, and compatibility checks | Current package shape `Deterministic`; adapter benefits require receipts | A controlled check shows a host binding is needed to preserve an outcome | @@ -40,6 +40,36 @@ The live decisions, their premises, and what would reopen each. "Evidence" says The sections below are the history behind those rows: what each release tried, measured, and rejected. They are non-normative. Where a section and the index disagree, the index is current and the section records how it got there. +## Verification receipts across roles + +Version 6.3.3 clarifies existing evidence reuse after the owner identified repeated verification as material +shared-host load. A fixed host-log sample found 28 same-label, same-fingerprint candidates in 116 completed +runs. Command differences, scratch inputs, environment changes, and lost output explain why those counts +cannot establish redundant passing checks or a wording cause. The [evidence ledger](evidence.md#633-verification-receipt-reuse) +records the measurement boundary and uncertainties. + +The selected change tightens the existing kernel and verification reference. A delegate, lead, or reviewer +can reuse an inspectable passing record for the same command, fingerprint, and relevant environment. +The record carries command and arguments, checked fingerprint, terminal exit, completion time, checkout, +and an accessible evidence location, plus relevant environment context. Existing logs and handoffs suffice. +Changed inputs, missing or unverifiable evidence, or a concrete named doubt justify execution. A different +fingerprint needs evidence for that candidate; relevant-file similarity cannot silently substitute for equality. +This is deliberately conservative when a HEAD-inclusive fingerprint changes after a content-preserving commit. +Revision-bound CI can establish the integrated state. Role and stage changes alone invalidate nothing. + +Required property coverage, risk-scaled independent review, scrutiny of returns, and destination verification +survive. Fingerprint equality covers no ignored or external input by itself. A capture that cannot identify +the stable state exercised establishes no reusable pass. The contributor `agent-verify inherited` protocol, +host queue ownership, package structure, and installation path are unchanged. + +A reference-only clarification would cost fewer kernel characters but leave the invariant absent when that +reference is not opened. A host result cache would add state, command/environment key design, and host coupling +without evidence that this package needs an execution service. The selected prose rule uses existing evidence, +adds no tool, file format, persistent ledger, or model experiment, and retains current host fallback behavior. +It clarifies the promised runtime contract, so the release is PATCH. Behavioral improvement and savings remain +`UNVERIFIED`. Reopen if a receipt shows stale evidence accepted, duplicated execution despite a matching record, +missing required coverage, or evidence capture too costly to repay reuse. + ## Planning boundaries and shared review continuation Version 6.3.1 clarifies the existing planning and independent-review contracts. One private installed Claude diff --git a/docs/evidence.md b/docs/evidence.md index d4b7f10..0640fec 100644 --- a/docs/evidence.md +++ b/docs/evidence.md @@ -2,6 +2,55 @@ This page separates package checks from observed model behavior. The full 2.0 evidence remains in the immutable [`v2.0.1` research snapshot](https://github.com/mzored/SkipHow/tree/1c811262e6acdbdc58a2ee862b54e0b8d3478eaa/docs/research/2026-08-27). +## 6.3.3 verification receipt reuse + +On 2026-10-08, read-only inspection used the host admission events from 10:28:44.019 through +12:18:20.126 UTC. The fixed cutoff excludes later events from the live append-only journal. +There were 116 completed runs, with 2,191.686 total execution seconds. Count a repeat when an earlier +completed run in the same checkout has the same label and non-null tree fingerprint, was admitted +within the preceding 30 minutes, and completed before the later admission. Count each later run once. +This gives 28 repeats, 24.1% of completed runs, consuming 209.539 seconds, 9.6% of execution time. +These are same-label, same-fingerprint candidates, not a measured same-command repeat rate. + +The host journal has no command arguments or session/role correlation. Private transcript inspection +confirmed at least two same-command, same-fingerprint repeats: a delegate retrieving truncated failure +output, and a lead recovering an explicit terminal exit status before reporting. A pre-merge lead also +repeated checks inside a differently composed batch. Other candidates ran different test selections or +diagnostics, changed scratch fixtures, or changed environment settings. Those distinctions prevent treating +the aggregate as redundant passing checks. Reviewer and lead-return duplicate rates, the exact same-command +share, and avoidable execution time remain `UNVERIFIED`. The owner's earlier 81-hour, label-only estimate +cannot be retroactively classified by tree. Private transcripts, host identifiers, and raw logs are not published. + +Source inspection found an ambiguity rather than a demonstrated cause. The kernel required checking returns +against current state, fresh review, and final verification after the last edit, while verification and integration +already permitted input-bound evidence reuse. None defined a minimal reusable check record in the kernel's +execution path. No controlled before/after model receipt connects those sentences to the measured executions. + +The owner requested a clarification. Delegates, leads, and reviewers inspect existing passing records for the +same command, tree fingerprint, and relevant environment. Changed inputs, missing or unverifiable evidence, +or a concrete named doubt justify another execution. A different fingerprint requires current-candidate evidence, +including revision-bound CI where available. Verification owns the record fields; delegation carries references +and integration applies that rule. Required coverage, independent review, destination checks, and contributor +admission remain intact. The asynchronous host fingerprint alone cannot establish that source inputs stayed +stable throughout a check. Relevant ignored and external inputs remain part of applicability. + +Five original synthetic decision probes cover role transitions, tracked/untracked/history changes, changed +commands and environments, scratch inputs outside the fingerprint, incomplete or failed records, uncertain +capture, and conflicting results. They remain `UNVERIFIED`; deterministic corpus checks validate their shape +and arm satisfiability only. No behavioral experiment or savings claim is attached to this release. +The [decision](decisions.md#verification-receipts-across-roles) records alternatives and surviving responsibilities. + +Exact future telemetry would need command identity and session/role correlation. Reusable host receipts would +also need an inspectable fingerprint query and evidence of the state actually exercised. These are external +host requirements, not package dependencies. No host configuration or admission implementation was changed. + +Independent read-only Codex source review approved the runtime, corpus, metadata, canonical documentation, +and site changes with no qualifying material findings. Isolated operating-system and host homes used a +temporary minimal authentication reference. Control and review transcripts showed no personal skill or +installed-plugin paths. The reviewer ran no verification commands or behavioral experiments and did not +inspect private measurement records. The maintainer reproduced the fixed aggregate separately. Owned +review resources and the authentication reference were removed after completion. + ## 6.3.2 optional contributor-check admission [PR #143](https://github.com/mzored/SkipHow/pull/143) integrated optional host admission into both diff --git a/docs/outcome-contract.md b/docs/outcome-contract.md index 7fc5e35..2e8736b 100644 --- a/docs/outcome-contract.md +++ b/docs/outcome-contract.md @@ -28,7 +28,7 @@ Broad project-state requests reconstruct each relevant outcome from its promised One lead remains accountable for each accepted outcome. A delegate applies the same standards to its assignment, within the owner's authority its brief relays, and returns what it finds outside it for the lead to dispose of. Assignments and status provide visibility, not exclusive session ownership. Parallel writers need verified distinct workspaces and explicit lanes. Every branch, worktree, checkout, checkpoint, running service, and uncommitted change in the project is the lead's own engineering state or another session's under the same responsibility; none of it is attributed to the owner or a third party, and no decision about it is handed back to them. Failed delegates, interruptions, and resumed sessions reconcile live state and retain decisions and outstanding obligations. A live session's lane and uncommitted changes the run did not make stay untouched; a leftover of a finished session is finished, integrated, recorded, or retired, with a retired commit named so it can be restored. Missing host capabilities lead to a safe available strategy and an accurate limitation. -Every change receives fresh evidence appropriate to its consequences. Each required property is proved at the narrowest stable boundary that retains the needed fidelity, and unique cross-boundary evidence is preserved. Repeated disproportionate verification cost is diagnosed at its responsible layer without making ordinary requests heavier or weakening delivery checks. Independent challenge addresses material risks and correlated mistakes. Small clear low-risk changes may use cold self-review, including visible corrections across more than one file. Findings receive a checked disposition; unsupported cosmetic objections do not cause endless review. The lead diagnoses broken fixtures, CI, and tools at the responsible layer and verifies the integrated result at its authorized destination. +Every change has evidence appropriate to the delivered state and its consequences. Reuse verifiable passing checks while the command, tree fingerprint, and relevant environment match; changed inputs, missing or unverifiable records, or a concrete named doubt justify another execution. Each required property is proved at the narrowest stable boundary that retains the needed fidelity, and unique cross-boundary evidence is preserved. Repeated disproportionate verification cost is diagnosed at its responsible layer without making ordinary requests heavier or weakening delivery checks. Independent challenge addresses material risks and correlated mistakes. Small clear low-risk changes may use cold self-review, including visible corrections across more than one file. Findings receive a checked disposition; unsupported cosmetic objections do not cause endless review. The lead diagnoses broken fixtures, CI, and tools at the responsible layer and verifies the integrated result at its authorized destination. Completion names what was delivered, the evidence that applies, and material unresolved work. A delegate's claim, local marker, configured setting, or passing schema is not evidence of a different destination or actual model behavior. diff --git a/evals/README.md b/evals/README.md index a6433db..14e3915 100644 --- a/evals/README.md +++ b/evals/README.md @@ -19,6 +19,12 @@ the receipt every future run has to leave behind. ## What is here +The 6.3.3 `verification-reuse-*` probes use invented read-only records to distinguish matching receipts +across roles from changed fingerprints, command/environment inputs, inaccessible or incomplete evidence, +unstable capture, and concrete conflicting results. Scratch inputs may change outside the fingerprint. +All arms share the decision outcomes; only candidate-package arms require kernel loading. These are +unrun instruments, not proof that checks executed, models reuse evidence, or verification load fell. + The 6.2.1 supervision probes remain `UNVERIFIED`. The original `project-supervision` fixture tests bounded discovery beyond a small passing change, complete export behavior, and legitimate fixed examples. Existing coupling, measured setup, and live-continuation probes cover old checks, alternative remedies, and a compacted iteration stream. Their diff --git a/evals/cases.json b/evals/cases.json index 9f270ea..20c1562 100644 --- a/evals/cases.json +++ b/evals/cases.json @@ -1,6 +1,6 @@ { "corpus_version": 4, - "package_under_test": "6.3.2", + "package_under_test": "6.3.3", "purpose": "Synthetic cases for three separate instruments: activation, forced-activation CTO behavior, and host smoke. Every case names a positive success observable, the product result shared across comparison arms, and explicit required-absence events. Nothing here has been run.", "not_a_gate": "No model run gates a pull request. python scripts/check.py and the pytest suite validate shape and internal satisfiability and never start a model. A run happens only when the owner authorizes a paid receipt, under the limits recorded in run_limits. A deterministic check passing is never evidence of behavior.", "evidence_labels": { @@ -19695,6 +19695,681 @@ ], "runs": [] } + }, + { + "id": "verification-reuse-across-roles", + "kind": "microcase", + "spec_refs": [ + "6.2" + ], + "acceptance": [ + "The response explicitly reuses P at delegate self-check, lead return acceptance, independent review, pre-merge preparation, and pre-report, because command, fingerprint, environment, and inspectable passing evidence match.", + "The response preserves independent candidate and expectation review and separate destination inspection, and names command with arguments, fingerprint, terminal exit, completion time, checkout, evidence location, and relevant environment as the minimal record." + ], + "intent": "Original unrun receipt-applicability decision probe. Deterministic corpus validity proves no model behavior or savings.", + "fixture": "verification-reuse", + "owner_prompt": "Decide which verification evidence applies and what further checking, if any, is justified for Matching receipt across roles in evidence.md. These are synthetic records. Read only; do not execute commands, dispatch, or change state.", + "subsequent_answers": [], + "contract_refs": [ + "SKILL.md#verification-and-reporting", + "references/verification.md#reusing-a-verification-record", + "references/delegation.md#what-comes-back", + "references/integration.md#verifying-the-integrated-state" + ], + "events": { + "applicability-decided": { + "description": "The response explicitly reuses P at delegate self-check, lead return acceptance, independent review, pre-merge preparation, and pre-report, because command, fingerprint, environment, and inspectable passing evidence match.", + "kind": "task", + "evidence": "both", + "shows": "action" + }, + "evidence-boundaries-preserved": { + "description": "The response preserves independent candidate and expectation review and separate destination inspection, and names command with arguments, fingerprint, terminal exit, completion time, checkout, evidence location, and relevant environment as the minimal record.", + "kind": "task", + "evidence": "both", + "shows": "action" + }, + "state-written": { + "description": "The session executes synthetic commands, dispatches, or writes project or external state despite the read-only request.", + "kind": "task", + "evidence": "both", + "shows": "action" + }, + "synthetic-execution-claimed": { + "description": "The response claims these invented records prove actual host execution, live passing checks, or measured savings.", + "kind": "task", + "evidence": "both", + "shows": "action" + }, + "kernel-before-action": { + "description": "The transcript shows the CTO kernel before the consequential decision or action; explicit workflow invocation does not separately inject the kernel.", + "kind": "package", + "evidence": "transcript", + "shows": "action" + } + }, + "observable": { + "event": "applicability-decided", + "source": "transcript", + "stop": "run_to_completion" + }, + "common_success": { + "all": [ + "applicability-decided", + "evidence-boundaries-preserved" + ] + }, + "arm_expectations": { + "m0-base-host": { + "activation": "not_applicable", + "required": [], + "forbidden": [ + "state-written", + "synthetic-execution-claimed", + "kernel-before-action" + ], + "permitted": [] + }, + "m1-explicit-skiphow": { + "activation": "expected", + "required": [ + "kernel-before-action" + ], + "forbidden": [ + "state-written", + "synthetic-execution-claimed" + ], + "permitted": [] + }, + "m2-implicit-discovery-hook": { + "activation": "expected", + "required": [ + "kernel-before-action" + ], + "forbidden": [ + "state-written", + "synthetic-execution-claimed" + ], + "permitted": [] + }, + "m3-bootstrap-candidate": { + "activation": "expected", + "required": [ + "kernel-before-action" + ], + "forbidden": [ + "state-written", + "synthetic-execution-claimed" + ], + "permitted": [] + }, + "m4-previous-full-skiphow": { + "activation": "expected", + "required": [], + "forbidden": [ + "state-written", + "synthetic-execution-claimed" + ], + "permitted": [ + "kernel-before-action" + ] + } + }, + "alternatives": [], + "conditional": [], + "result": { + "status": "not_run", + "evidence_label": "UNVERIFIED", + "observed_arms": [], + "arms_pending": [ + "m0-base-host", + "m1-explicit-skiphow", + "m2-implicit-discovery-hook", + "m3-bootstrap-candidate", + "m4-previous-full-skiphow" + ], + "runs": [] + } + }, + { + "id": "verification-reuse-different-tree", + "kind": "microcase", + "spec_refs": [ + "6.2" + ], + "acceptance": [ + "The response rejects P as current-tree evidence for T, U, H, and M, including unchanged HEAD with a changed tracked file and identical file content with a changed HEAD-inclusive fingerprint.", + "The response requires evidence for each current candidate, allowing revision-bound CI for the integrated candidate, without inventing such a result or bypassing required coverage." + ], + "intent": "Original unrun receipt-applicability decision probe. Deterministic corpus validity proves no model behavior or savings.", + "fixture": "verification-reuse", + "owner_prompt": "Decide which verification evidence applies and what further checking, if any, is justified for Different trees in evidence.md. These are synthetic records. Read only; do not execute commands, dispatch, or change state.", + "subsequent_answers": [], + "contract_refs": [ + "SKILL.md#verification-and-reporting", + "references/verification.md#reusing-a-verification-record", + "references/delegation.md#what-comes-back", + "references/integration.md#verifying-the-integrated-state" + ], + "events": { + "applicability-decided": { + "description": "The response rejects P as current-tree evidence for T, U, H, and M, including unchanged HEAD with a changed tracked file and identical file content with a changed HEAD-inclusive fingerprint.", + "kind": "task", + "evidence": "both", + "shows": "action" + }, + "evidence-boundaries-preserved": { + "description": "The response requires evidence for each current candidate, allowing revision-bound CI for the integrated candidate, without inventing such a result or bypassing required coverage.", + "kind": "task", + "evidence": "both", + "shows": "action" + }, + "state-written": { + "description": "The session executes synthetic commands, dispatches, or writes project or external state despite the read-only request.", + "kind": "task", + "evidence": "both", + "shows": "action" + }, + "synthetic-execution-claimed": { + "description": "The response claims these invented records prove actual host execution, live passing checks, or measured savings.", + "kind": "task", + "evidence": "both", + "shows": "action" + }, + "kernel-before-action": { + "description": "The transcript shows the CTO kernel before the consequential decision or action; explicit workflow invocation does not separately inject the kernel.", + "kind": "package", + "evidence": "transcript", + "shows": "action" + } + }, + "observable": { + "event": "applicability-decided", + "source": "transcript", + "stop": "run_to_completion" + }, + "common_success": { + "all": [ + "applicability-decided", + "evidence-boundaries-preserved" + ] + }, + "arm_expectations": { + "m0-base-host": { + "activation": "not_applicable", + "required": [], + "forbidden": [ + "state-written", + "synthetic-execution-claimed", + "kernel-before-action" + ], + "permitted": [] + }, + "m1-explicit-skiphow": { + "activation": "expected", + "required": [ + "kernel-before-action" + ], + "forbidden": [ + "state-written", + "synthetic-execution-claimed" + ], + "permitted": [] + }, + "m2-implicit-discovery-hook": { + "activation": "expected", + "required": [ + "kernel-before-action" + ], + "forbidden": [ + "state-written", + "synthetic-execution-claimed" + ], + "permitted": [] + }, + "m3-bootstrap-candidate": { + "activation": "expected", + "required": [ + "kernel-before-action" + ], + "forbidden": [ + "state-written", + "synthetic-execution-claimed" + ], + "permitted": [] + }, + "m4-previous-full-skiphow": { + "activation": "expected", + "required": [], + "forbidden": [ + "state-written", + "synthetic-execution-claimed" + ], + "permitted": [ + "kernel-before-action" + ] + } + }, + "alternatives": [], + "conditional": [], + "result": { + "status": "not_run", + "evidence_label": "UNVERIFIED", + "observed_arms": [], + "arms_pending": [ + "m0-base-host", + "m1-explicit-skiphow", + "m2-implicit-discovery-hook", + "m3-bootstrap-candidate", + "m4-previous-full-skiphow" + ], + "runs": [] + } + }, + { + "id": "verification-reuse-command-and-environment", + "kind": "microcase", + "spec_refs": [ + "6.2" + ], + "acceptance": [ + "The response identifies the invalidating input in each of C, E, I, S, D, and L and does not accept P for the changed command, dependency, ignored configuration, external scratch input, database state, or service destination.", + "The response selects verification appropriate to the changed inputs while retaining required coverage and explains why a matching repository fingerprint alone cannot settle these rows." + ], + "intent": "Original unrun receipt-applicability decision probe. Deterministic corpus validity proves no model behavior or savings.", + "fixture": "verification-reuse", + "owner_prompt": "Decide which verification evidence applies and what further checking, if any, is justified for Changed command or environment in evidence.md. These are synthetic records. Read only; do not execute commands, dispatch, or change state.", + "subsequent_answers": [], + "contract_refs": [ + "SKILL.md#verification-and-reporting", + "references/verification.md#reusing-a-verification-record", + "references/delegation.md#what-comes-back", + "references/integration.md#verifying-the-integrated-state" + ], + "events": { + "applicability-decided": { + "description": "The response identifies the invalidating input in each of C, E, I, S, D, and L and does not accept P for the changed command, dependency, ignored configuration, external scratch input, database state, or service destination.", + "kind": "task", + "evidence": "both", + "shows": "action" + }, + "evidence-boundaries-preserved": { + "description": "The response selects verification appropriate to the changed inputs while retaining required coverage and explains why a matching repository fingerprint alone cannot settle these rows.", + "kind": "task", + "evidence": "both", + "shows": "action" + }, + "state-written": { + "description": "The session executes synthetic commands, dispatches, or writes project or external state despite the read-only request.", + "kind": "task", + "evidence": "both", + "shows": "action" + }, + "synthetic-execution-claimed": { + "description": "The response claims these invented records prove actual host execution, live passing checks, or measured savings.", + "kind": "task", + "evidence": "both", + "shows": "action" + }, + "kernel-before-action": { + "description": "The transcript shows the CTO kernel before the consequential decision or action; explicit workflow invocation does not separately inject the kernel.", + "kind": "package", + "evidence": "transcript", + "shows": "action" + } + }, + "observable": { + "event": "applicability-decided", + "source": "transcript", + "stop": "run_to_completion" + }, + "common_success": { + "all": [ + "applicability-decided", + "evidence-boundaries-preserved" + ] + }, + "arm_expectations": { + "m0-base-host": { + "activation": "not_applicable", + "required": [], + "forbidden": [ + "state-written", + "synthetic-execution-claimed", + "kernel-before-action" + ], + "permitted": [] + }, + "m1-explicit-skiphow": { + "activation": "expected", + "required": [ + "kernel-before-action" + ], + "forbidden": [ + "state-written", + "synthetic-execution-claimed" + ], + "permitted": [] + }, + "m2-implicit-discovery-hook": { + "activation": "expected", + "required": [ + "kernel-before-action" + ], + "forbidden": [ + "state-written", + "synthetic-execution-claimed" + ], + "permitted": [] + }, + "m3-bootstrap-candidate": { + "activation": "expected", + "required": [ + "kernel-before-action" + ], + "forbidden": [ + "state-written", + "synthetic-execution-claimed" + ], + "permitted": [] + }, + "m4-previous-full-skiphow": { + "activation": "expected", + "required": [], + "forbidden": [ + "state-written", + "synthetic-execution-claimed" + ], + "permitted": [ + "kernel-before-action" + ] + } + }, + "alternatives": [], + "conditional": [], + "result": { + "status": "not_run", + "evidence_label": "UNVERIFIED", + "observed_arms": [], + "arms_pending": [ + "m0-base-host", + "m1-explicit-skiphow", + "m2-implicit-discovery-hook", + "m3-bootstrap-candidate", + "m4-previous-full-skiphow" + ], + "runs": [] + } + }, + { + "id": "verification-reuse-missing-record", + "kind": "microcase", + "spec_refs": [ + "6.2" + ], + "acceptance": [ + "The response distinguishes N, A, X, F, Q, and O, rejecting unsupported or failed pass claims and an asynchronous capture that cannot identify the state actually checked.", + "The response names the missing or contradictory evidence to recover or obtain for each candidate, preserves the check's terminal status, and reports a blocker if that evidence cannot be established." + ], + "intent": "Original unrun receipt-applicability decision probe. Deterministic corpus validity proves no model behavior or savings.", + "fixture": "verification-reuse", + "owner_prompt": "Decide which verification evidence applies and what further checking, if any, is justified for Missing or unverifiable records in evidence.md. These are synthetic records. Read only; do not execute commands, dispatch, or change state.", + "subsequent_answers": [], + "contract_refs": [ + "SKILL.md#verification-and-reporting", + "references/verification.md#reusing-a-verification-record", + "references/delegation.md#what-comes-back", + "references/integration.md#verifying-the-integrated-state" + ], + "events": { + "applicability-decided": { + "description": "The response distinguishes N, A, X, F, Q, and O, rejecting unsupported or failed pass claims and an asynchronous capture that cannot identify the state actually checked.", + "kind": "task", + "evidence": "both", + "shows": "action" + }, + "evidence-boundaries-preserved": { + "description": "The response names the missing or contradictory evidence to recover or obtain for each candidate, preserves the check's terminal status, and reports a blocker if that evidence cannot be established.", + "kind": "task", + "evidence": "both", + "shows": "action" + }, + "state-written": { + "description": "The session executes synthetic commands, dispatches, or writes project or external state despite the read-only request.", + "kind": "task", + "evidence": "both", + "shows": "action" + }, + "synthetic-execution-claimed": { + "description": "The response claims these invented records prove actual host execution, live passing checks, or measured savings.", + "kind": "task", + "evidence": "both", + "shows": "action" + }, + "kernel-before-action": { + "description": "The transcript shows the CTO kernel before the consequential decision or action; explicit workflow invocation does not separately inject the kernel.", + "kind": "package", + "evidence": "transcript", + "shows": "action" + } + }, + "observable": { + "event": "applicability-decided", + "source": "transcript", + "stop": "run_to_completion" + }, + "common_success": { + "all": [ + "applicability-decided", + "evidence-boundaries-preserved" + ] + }, + "arm_expectations": { + "m0-base-host": { + "activation": "not_applicable", + "required": [], + "forbidden": [ + "state-written", + "synthetic-execution-claimed", + "kernel-before-action" + ], + "permitted": [] + }, + "m1-explicit-skiphow": { + "activation": "expected", + "required": [ + "kernel-before-action" + ], + "forbidden": [ + "state-written", + "synthetic-execution-claimed" + ], + "permitted": [] + }, + "m2-implicit-discovery-hook": { + "activation": "expected", + "required": [ + "kernel-before-action" + ], + "forbidden": [ + "state-written", + "synthetic-execution-claimed" + ], + "permitted": [] + }, + "m3-bootstrap-candidate": { + "activation": "expected", + "required": [ + "kernel-before-action" + ], + "forbidden": [ + "state-written", + "synthetic-execution-claimed" + ], + "permitted": [] + }, + "m4-previous-full-skiphow": { + "activation": "expected", + "required": [], + "forbidden": [ + "state-written", + "synthetic-execution-claimed" + ], + "permitted": [ + "kernel-before-action" + ] + } + }, + "alternatives": [], + "conditional": [], + "result": { + "status": "not_run", + "evidence_label": "UNVERIFIED", + "observed_arms": [], + "arms_pending": [ + "m0-base-host", + "m1-explicit-skiphow", + "m2-implicit-discovery-hook", + "m3-bootstrap-candidate", + "m4-previous-full-skiphow" + ], + "runs": [] + } + }, + { + "id": "verification-reuse-named-doubt", + "kind": "microcase", + "spec_refs": [ + "6.2" + ], + "acceptance": [ + "The response names intermittent order dependence for J, retains both conflicting records, and proposes focused investigation and evidence without treating a later green run as a repair.", + "The response reuses K's matching pass because changing reviewer or writing a report supplies no concrete doubt, while preserving independent review and destination verification." + ], + "intent": "Original unrun receipt-applicability decision probe. Deterministic corpus validity proves no model behavior or savings.", + "fixture": "verification-reuse", + "owner_prompt": "Decide which verification evidence applies and what further checking, if any, is justified for Concrete doubt and unchanged transitions in evidence.md. These are synthetic records. Read only; do not execute commands, dispatch, or change state.", + "subsequent_answers": [], + "contract_refs": [ + "SKILL.md#verification-and-reporting", + "references/verification.md#reusing-a-verification-record", + "references/delegation.md#what-comes-back", + "references/integration.md#verifying-the-integrated-state" + ], + "events": { + "applicability-decided": { + "description": "The response names intermittent order dependence for J, retains both conflicting records, and proposes focused investigation and evidence without treating a later green run as a repair.", + "kind": "task", + "evidence": "both", + "shows": "action" + }, + "evidence-boundaries-preserved": { + "description": "The response reuses K's matching pass because changing reviewer or writing a report supplies no concrete doubt, while preserving independent review and destination verification.", + "kind": "task", + "evidence": "both", + "shows": "action" + }, + "state-written": { + "description": "The session executes synthetic commands, dispatches, or writes project or external state despite the read-only request.", + "kind": "task", + "evidence": "both", + "shows": "action" + }, + "synthetic-execution-claimed": { + "description": "The response claims these invented records prove actual host execution, live passing checks, or measured savings.", + "kind": "task", + "evidence": "both", + "shows": "action" + }, + "kernel-before-action": { + "description": "The transcript shows the CTO kernel before the consequential decision or action; explicit workflow invocation does not separately inject the kernel.", + "kind": "package", + "evidence": "transcript", + "shows": "action" + } + }, + "observable": { + "event": "applicability-decided", + "source": "transcript", + "stop": "run_to_completion" + }, + "common_success": { + "all": [ + "applicability-decided", + "evidence-boundaries-preserved" + ] + }, + "arm_expectations": { + "m0-base-host": { + "activation": "not_applicable", + "required": [], + "forbidden": [ + "state-written", + "synthetic-execution-claimed", + "kernel-before-action" + ], + "permitted": [] + }, + "m1-explicit-skiphow": { + "activation": "expected", + "required": [ + "kernel-before-action" + ], + "forbidden": [ + "state-written", + "synthetic-execution-claimed" + ], + "permitted": [] + }, + "m2-implicit-discovery-hook": { + "activation": "expected", + "required": [ + "kernel-before-action" + ], + "forbidden": [ + "state-written", + "synthetic-execution-claimed" + ], + "permitted": [] + }, + "m3-bootstrap-candidate": { + "activation": "expected", + "required": [ + "kernel-before-action" + ], + "forbidden": [ + "state-written", + "synthetic-execution-claimed" + ], + "permitted": [] + }, + "m4-previous-full-skiphow": { + "activation": "expected", + "required": [], + "forbidden": [ + "state-written", + "synthetic-execution-claimed" + ], + "permitted": [ + "kernel-before-action" + ] + } + }, + "alternatives": [], + "conditional": [], + "result": { + "status": "not_run", + "evidence_label": "UNVERIFIED", + "observed_arms": [], + "arms_pending": [ + "m0-base-host", + "m1-explicit-skiphow", + "m2-implicit-discovery-hook", + "m3-bootstrap-candidate", + "m4-previous-full-skiphow" + ], + "runs": [] + } } ], "historical_evidence": { diff --git a/evals/cto-cases.json b/evals/cto-cases.json index a424584..c2b08b2 100644 --- a/evals/cto-cases.json +++ b/evals/cto-cases.json @@ -1,6 +1,6 @@ { "instrument": "forced_activation_behavior", - "package_under_test": "6.3.2", + "package_under_test": "6.3.3", "suite_status": "not_run", "minimum_coverage": { "case_ids": [ diff --git a/evals/fixtures/verification-reuse/evidence.md b/evals/fixtures/verification-reuse/evidence.md new file mode 100644 index 0000000..c34ec6e --- /dev/null +++ b/evals/fixtures/verification-reuse/evidence.md @@ -0,0 +1,79 @@ +# Synthetic verification decisions + +These records are invented decision inputs, not evidence that any check ran. Fingerprints are symbolic +identities supplied by this fixture. All paths are relative to its synthetic workspace. No command below +is an instruction to execute anything. + +## Matching receipt across roles + +Receipt P records command `python -m pytest tests/order_total_checks.py -q`, tree fingerprint +`1111111111111111`, terminal exit `0`, completion at `2026-09-01T10:00:00Z`, checkout `workspace/orders`, +and accessible evidence at `logs/order-check-P.txt`. Its retained terminal output says `6 passed`. +The reviewed command covers the required six order-total assertions. + +Current inspection establishes the same command, fingerprint, dependency versions, configuration, +ignored inputs, scratch files, service state, and destination. No writer changed the checked inputs +during execution or capture. No contradictory evidence or concrete doubt exists. + +Decide separately for the delegate's final self-check, the lead accepting its return, an independent +reviewer, pre-merge preparation without a merge yet, and the final report. The reviewer still owes +independent inspection of the candidate and expectations. The lead still owes inspection of delivery +at the authorized destination. No role change has altered the checked state or invalidated P. + +## Different trees + +Every row otherwise retains P's command, environment, accessible evidence, and successful exit. + +| Row | Recorded fingerprint | Current fingerprint | What changed | +| --- | --- | --- | --- | +| T | `1111111111111111` | `2222222222222222` | A tracked file changed after the pass. HEAD stayed the same. | +| U | `1111111111111111` | `3333333333333333` | An untracked test helper changed after the pass. | +| H | `1111111111111111` | `4444444444444444` | A commit changed the host's HEAD-inclusive fingerprint, with identical visible file content. | +| M | `1111111111111111` | `5555555555555555` | Integration produced a different fingerprint; no CI result for it exists yet. | + +Decide whether P establishes a pass for each current tree and what current-candidate evidence is needed. +Neither a branch name nor equal visible file content makes different supplied fingerprints equal. + +## Changed command or environment + +All rows keep P's repository fingerprint. The named difference is independently confirmed. + +| Row | Difference from P | +| --- | --- | +| C | The required command now selects `tests/order_total_checks.py -q -k refunds`, rather than P's full command. | +| E | The installed dependency version changed, while the tracked lockfile stayed the same. | +| I | The command reads an ignored configuration file whose contents changed. | +| S | The command reads `scratch/rates.json`; its contents changed outside the fingerprinted repository. | +| D | The command exercises a database whose schema or test data changed. | +| L | The requested check now targets another service destination. | + +Decide applicability for each row without treating repository-fingerprint equality as sufficient. + +## Missing or unverifiable records + +Each row names a separate candidate. Do not combine evidence from different candidates. + +| Row | Available evidence | +| --- | --- | +| N | A delegate says the check passed, but there is no record. | +| A | Command, fingerprint, exit, time, and checkout are listed; the only evidence location is inaccessible. | +| X | A filter exited zero, but the check's own terminal status was not retained. | +| F | A complete matching record has terminal exit `1`. | +| Q | The host captured a fingerprint asynchronously while a writer edited source. No evidence identifies the stable state actually exercised. | +| O | The only record omits both the command arguments and the fingerprint. | + +State what remains unverified and what evidence must be recovered or obtained. A failure is not a +reusable pass; an unavailable command remains a blocker rather than permission to report success. + +## Concrete doubt and unchanged transitions + +Candidate J has a complete matching passing receipt. An earlier completed run on the same command, +fingerprint, and environment failed the order-independence assertion. The conflicting terminal records +are accessible. The named doubt is intermittent order dependence. Decide how to preserve both results, +investigate that doubt, and obtain focused evidence without reporting the intermittent defect fixed +merely because another run passes. + +Candidate K has P's complete matching receipt and no conflicting result or named risk. The only proposed +reason for another execution is that a reviewer replaced the lead or that the report is now being written. +Decide whether this transition justifies another execution. Neither candidate's receipt proves delivery +to a remote destination or substitutes for independent review. diff --git a/evals/fixtures/verification-reuse/fixture.json b/evals/fixtures/verification-reuse/fixture.json new file mode 100644 index 0000000..88e4a22 --- /dev/null +++ b/evals/fixtures/verification-reuse/fixture.json @@ -0,0 +1,21 @@ +{ + "id": "verification-reuse", + "summary": "Read-only decisions about synthetic verification receipts and their applicability across roles.", + "kind": "checked-in overlay with decision evidence", + "synthetic": true, + "derives_from": "orders-service", + "planted": [ + "A delegate's complete passing receipt still matches the command, tree, and environment at every subsequent role and reporting boundary.", + "Other receipts differ in tracked or untracked content, history, command, environment, scratch inputs, or service state.", + "Incomplete evidence, an unstable fingerprint capture, and contradictory results must not become current passes." + ], + "absent_on_purpose": [ + "No real verification receipts, host fingerprint implementation, service, reviewer, or executed model session." + ], + "setup": [ + "Materialize the inherited orders-service skeleton with the existing capture preparation path.", + "For decision probes, read only the selected section of evidence.md; all commands, identifiers, timestamps, and locations there are invented and must not be executed or treated as live evidence." + ], + "run_checks": "Decision cases are graded from the response, without executing the synthetic commands.", + "privacy": "Original invented records with no private transcript text, paths, identifiers, credentials, or customer data." +} diff --git a/evals/host-smoke.json b/evals/host-smoke.json index adfdb6e..5215f19 100644 --- a/evals/host-smoke.json +++ b/evals/host-smoke.json @@ -1,6 +1,6 @@ { "instrument": "host_smoke", - "package_under_test": "6.3.2", + "package_under_test": "6.3.3", "scope": "external_candidate_receipts", "checks": { "clean_install": { diff --git a/plugins/skiphow/.claude-plugin/plugin.json b/plugins/skiphow/.claude-plugin/plugin.json index 02f9e5e..570b2a2 100644 --- a/plugins/skiphow/.claude-plugin/plugin.json +++ b/plugins/skiphow/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "skiphow", - "version": "6.3.2", + "version": "6.3.3", "description": "Adaptive virtual CTO for founders and product owners using Claude Code or Codex. Describe the product outcome; SkipHow owns the technical lifecycle through verified completion.", "author": { "name": "mzored", diff --git a/plugins/skiphow/.codex-plugin/plugin.json b/plugins/skiphow/.codex-plugin/plugin.json index d8d21ba..90e026a 100644 --- a/plugins/skiphow/.codex-plugin/plugin.json +++ b/plugins/skiphow/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "skiphow", - "version": "6.3.2", + "version": "6.3.3", "description": "Adaptive virtual CTO for founders and product owners using Claude Code or Codex. Describe the product outcome; SkipHow owns the technical lifecycle through verified completion.", "author": { "name": "mzored", diff --git a/plugins/skiphow/skills/skiphow/SKILL.md b/plugins/skiphow/skills/skiphow/SKILL.md index 00ca9e2..0e46364 100644 --- a/plugins/skiphow/skills/skiphow/SKILL.md +++ b/plugins/skiphow/skills/skiphow/SKILL.md @@ -27,15 +27,15 @@ The owner decides only unresolved choices materially changing visible behavior, ## Adaptive technical leadership -Use skills and tools when their benefit warrants the cost. Assess material risks and accumulating costs across the authorized project, even when checks pass; question existing solutions' fit. Keep work proportional to evidence, consequence, and total cost, prioritizing the owner's outcome. Resolve feasible engineering findings within authority in this run at safe boundaries, including iteration. A record never replaces a feasible repair. Defer only for a concrete owner decision, protected action, blocker, or evidenced cost that would materially compromise the owner's outcome; record why and what resumes it, or reject with a reason. Stop improving when further benefit does not justify cost. A second failure of one kind is a class: fix its other members before returning to the loop. Never mask a defect with longer timeouts, retries, disabled checks, or weaker assertions. Review the final state fresh, scaled to risk. +Use skills and tools when their benefit warrants the cost. Assess material risks and accumulating costs across the authorized project, even when checks pass; question existing solutions' fit. Keep work proportional to evidence, consequence, and total cost, prioritizing the owner's outcome. Resolve feasible engineering findings within authority in this run at safe boundaries, including iteration. A record never replaces a feasible repair. Defer only for a concrete owner decision, protected action, blocker, or evidenced cost that would materially compromise the owner's outcome; record why and what resumes it, or reject with a reason. Stop improving when further benefit does not justify cost. A second failure of one kind is a class: fix its other members before returning to the loop. Never mask a defect with longer timeouts, retries, disabled checks, or weaker assertions. Review current-state risks. ## Custody of engineering state and delegates -Every branch, worktree, checkout, checkpoint, service, and uncommitted change in the project came from this session, an earlier one, or a delegate: account for all of it as your own, never the owner's or another's, and never hand its decision back. Keep working state in the project, host area, or ignored location. A session is live while its changes move, its review is open, its record is live, or the host lists it, and a name, timestamp, or running process alone proves nothing; never overwrite, reset, publish, delete, or absorb a live lane. Settle a finished session's leftover and say how: a change request retires integrated or superseded leftovers, naming unique commits; a read-only request only reports. Uncommitted changes this run did not make stay in place, outside its commits. Retiring recoverable engineering state is not material deletion. Before dispatch, have [delegation](references/delegation.md) in context and set each delegate's model and effort through host controls to the cheapest route likely to meet its acceptance in one pass; justify an unset control's default, often the parent's. Brief one outcome, observable proof, allowed surface, starting revision, authority and prohibitions, when to return a blocking unknown, a compact handoff of status, identity, evidence, all findings and blockers, nested delegation's permission, route and bound, and this file's path to read. Verify a distinct workspace and starting state before delegate writes. Coordinate shared mutable resources outside that isolation. Check each return against current state; the lead keeps owner questions, dispositions, records, sensitive context, synthesis, integration, final verification, and the completion claim. +Every branch, worktree, checkout, checkpoint, service, and uncommitted change in the project came from this session, an earlier one, or a delegate: account for all of it as your own, never the owner's or another's, and never hand its decision back. Keep working state in the project, host area, or ignored location. A session is live while its changes move, its review is open, its record is live, or the host lists it, and a name, timestamp, or running process alone proves nothing; never overwrite, reset, publish, delete, or absorb a live lane. Settle a finished session's leftover and say how: a change request retires integrated or superseded leftovers, naming unique commits; a read-only request only reports. Uncommitted changes this run did not make stay in place, outside its commits. Retiring recoverable engineering state is not material deletion. Before dispatch, keep [delegation](references/delegation.md) in context and set each delegate's model and effort through host controls to the cheapest route likely to meet acceptance in one pass. Justify an unset control's default, often the parent's. Brief one outcome, observable proof, allowed surface, starting revision, authority and restrictions, blocking-unknown return conditions, a compact status and evidence handoff with identity, all findings and blockers, nested delegation's permission, route and bound, and this file's path. Verify a distinct workspace and starting state before delegate writes. Coordinate shared mutable resources outside that isolation. Check each return and its evidence against current state; the lead keeps owner questions, dispositions, records, sensitive context, synthesis, integration, final verification, and the completion claim. ## Verification and reporting -Continue while a safe authorized step advances the result. Stop only at verified completion, a requested pause, an owner decision, a protected or human-only step, or a real external blocker. While blocked, do only independent work demonstrably advancing acceptance, invent no prerequisites to fill capacity, and report blockers together. Verify the final state after the last edit, including rendered output and effects where they land; name the branch and address of any running result you show. Reasoning, dry runs, and silence do not prove a check passed. Before reporting, reconcile every requested part, lane, branch, worktree, finding, and blocker this run touched, including unachieved effects such as refused cleanup; success with a part never started is false completion. Lead with the result, then evidence, decisions, `UNVERIFIED` or blocked parts, and protected actions outside authority. +Continue safe authorized work until verified completion, a requested pause, an owner decision, a protected or human-only step, or an external blocker. While blocked, advance independent acceptance only and report blockers together. Verify the final state after the last edit, including rendered output and destination effects. Delegates, leads, and reviewers reuse verifiable passes for the same command, tree fingerprint, and relevant environment. Rerun only for changed inputs, missing or unverifiable records, or a concrete named doubt. A different tree proves no current pass. Name branch and address of running results. Reconcile every requested part, lane, branch, worktree, finding, and blocker touched, including unachieved effects, before reporting. An unstarted part precludes completion. Lead with result, evidence, decisions, `UNVERIFIED` or blocked parts, and protected actions outside authority. ## Focused guidance diff --git a/plugins/skiphow/skills/skiphow/references/delegation.md b/plugins/skiphow/skills/skiphow/references/delegation.md index 7b43f45..a24d3c0 100644 --- a/plugins/skiphow/skills/skiphow/references/delegation.md +++ b/plugins/skiphow/skills/skiphow/references/delegation.md @@ -60,9 +60,9 @@ Prefer host-managed worktrees and cleanup. Otherwise verify an ignored in-projec ## What comes back -Convey the handoff contract in the brief: status, checked result identity, acceptance-evidence summary, every finding and blocker, and accessible detail references. Keep decision-relevant facts inline and bulky reports, logs, inventories, and attempt histories in the host's working area. Reuse artifacts; small results need no report file. Preserve every finding and uncertainty. +Convey status, checked result identity, acceptance summary with [verification records](verification.md#reusing-a-verification-record), every finding and blocker, and accessible detail references in the brief. Keep decision facts inline and bulk reports, logs, inventories, and histories in the host's working area. Reuse artifacts; small results need no report file. Preserve all findings and uncertainty. -Carry this boundary through briefs, review inputs, and return checks, passing bulk directly by reference to whoever needs it. The lead checks evidence, widening inspection for risk, contradiction, or missing proof; a verdict or pointer alone is not acceptance. Corrections return changes, current evidence, and all unresolved findings and blockers rather than the whole history. +Carry this boundary through briefs, review inputs, and return checks; pass bulk directly by reference. The lead inspects reusable verification records, widening for risk, contradiction, or missing proof; a verdict or pointer alone is not acceptance. Corrections return changes, current evidence, and all unresolved findings and blockers rather than the whole history. Settle returned technical questions from project evidence and your judgment, without passing them to the owner. A problem a delegate returns is one you found. diff --git a/plugins/skiphow/skills/skiphow/references/integration.md b/plugins/skiphow/skills/skiphow/references/integration.md index 0e55aff..aca93f9 100644 --- a/plugins/skiphow/skills/skiphow/references/integration.md +++ b/plugins/skiphow/skills/skiphow/references/integration.md @@ -18,7 +18,7 @@ Where the request or the repository's workflow puts this work in a tracker, [tra ## Verifying the integrated state -Verify the integrated state. Reuse earlier evidence when revision-bound CI tested that result, or when the relevant tree, dependencies, and configuration prove that integration preserved the checked inputs. A merge does not by itself invalidate a check. Rerun only evidence whose inputs changed. Inspect the destination separately, because test equivalence does not prove that delivery happened. +Establish evidence for the integrated state under [verification record reuse](verification.md#reusing-a-verification-record). Reuse a matching record or obtain evidence for the integrated candidate, including revision-bound CI. A mismatched local fingerprint proves no current-tree pass. Integration as a named stage requires no repeat by itself. Inspect the destination separately, because passing checks do not prove that delivery happened. If the merged result fails, preserve the evidence and any unrelated work first, then choose the recovery by consequence. A failing local merge or disposable branch can stay in place for diagnosis; nothing is gained by unwinding a merge you are about to redo. A failing shared target that other work, CI, or a deployment path depends on is contained or restored to its last good state while the diagnosis continues, whenever that is safer than leaving it broken. A revert on a covered non-production destination needs no new blanket approval; restoring production still needs the applicable production grant. In either case the failed state is not reported as delivered. diff --git a/plugins/skiphow/skills/skiphow/references/verification.md b/plugins/skiphow/skills/skiphow/references/verification.md index b19ebde..32d4dd7 100644 --- a/plugins/skiphow/skills/skiphow/references/verification.md +++ b/plugins/skiphow/skills/skiphow/references/verification.md @@ -4,11 +4,11 @@ Open this for tests, final review, security, privacy, reliability, migration, ro ## Choosing the test -For a read-only design or coverage request, propose the tests without changing the project. Test observable behavior rather than internal shape such as call order or private state, and follow the repository's existing test layout and vocabulary. +For read-only design or coverage, propose tests without edits. Test observable behavior, not call order or private state, using the repository's test layout and vocabulary. -For every durable check, name the property it proves and place it at the narrowest stable boundary that provides the required fidelity. Use a broader check when its environment contributes evidence a narrower boundary cannot establish reliably, such as real component integration, browser or runtime behavior, persistence, rendering, infrastructure wiring, an external protocol, or an important cross-boundary product outcome. Preserve that evidence. Do not repeat a business invariant at increasingly expensive boundaries unless each protects a distinct failure mode. +Name what each durable check proves and use the narrowest stable boundary with the required fidelity. Broader checks earn their cost when real integration, browser or runtime behavior, persistence, rendering, infrastructure, external protocols, or important cross-boundary outcomes provide evidence a narrower check cannot. Preserve that evidence. Repeat a business invariant at costlier boundaries only for distinct failure modes. -Prefer stable product-facing contracts over incidental presentation or implementation details. Widespread unrelated test rewrites after a behavior-preserving change are evidence of coupling; find and repair the responsible boundary instead of mechanically updating every affected test. Introduce mocks or internal seams only where they materially improve isolation, determinism, cost, or safety. External systems, time, and randomness are common cases, and a legacy or tightly coupled system may need more. +Prefer stable product contracts over incidental presentation or implementation. Unrelated test rewrites after behavior-preserving changes indicate coupling; repair the responsible boundary. Use mocks or internal seams where they materially improve isolation, determinism, cost, or safety, often for external systems, time, randomness, legacy or tightly coupled code. When setup or an earlier journey is not under test, establish the required state through an existing reliable cheaper path instead of replaying it through an expensive one. Keep direct coverage of a setup journey that is itself important. @@ -24,17 +24,17 @@ A regression test should close the class of bug. Assert the rule the defect brok ## How much to run -Preserve required behavioral evidence. Within authority, consolidate, replace, or remove pre-existing checks when evidence shows they are redundant, obsolete, or coupled to an incidental implementation; retain or establish equivalent reliable proof of every required property and distinct failure mode. A test's age or cost alone does not justify deletion. +Preserve proof of every required property and distinct failure mode. Within authority, consolidate, replace, or remove checks shown redundant, obsolete, or coupled to incidental implementation, retaining equivalent reliable coverage. Age or cost alone justifies no deletion. -The CTO owns test selection. During implementation and iteration, use the smallest reliable evidence covering the changed behavior, widening by reachable behavior, crossed boundaries, uncertainty, and consequence. Frequent cheap checks can support safe refactoring. Inspect what a command selects, costs, and changes; repair material friction through [operations](operations.md). Do not rerun an unchanged expensive gate when narrower evidence answers the question. +The CTO selects the smallest reliable evidence for changed behavior, widening for reachable behavior, crossed boundaries, uncertainty, and consequence. Cheap checks can support refactoring. Inspect command selection, cost, and effects; use [operations](operations.md) for material friction. Satisfy required delivery evidence before integration or release. Prefer reliable native affected-test or dependency selection; improving missing selection is legitimate engineering when verification cost is material. Brittle path heuristics and skipped required gates provide no substitute. -Before integration or release, satisfy the broader evidence the project's delivery contract and risk require. Use reliable native affected-test, dependency, project, tagging, or equivalent selection when available. When repeated verification cost is material and reliable selection is missing, improving it is legitimate engineering work. Do not replace reliable selection with brittle filename or path heuristics, or skip a required integration or release gate to reduce latency. +### Reusing a verification record -Bind each result to the code, dependencies, configuration, environment, and destination it exercised. Reuse it while those inputs remain equivalent. A commit, rebase, merge, tag, or named stage does not invalidate evidence by itself. Establish equivalence from revision-bound CI, the relevant tree and configuration, or an immutable artifact, and rerun only the checks whose inputs changed. +Keep the command and arguments, checked tree fingerprint, terminal exit status, completion time, checkout, and accessible evidence location in an existing log or handoff. Include relevant environment context: dependencies, configuration, external or ignored inputs, services, and destination. No new file or ledger is required. The fingerprint must identify the state actually checked; source changes during execution or an uncertain capture leave applicability unverified. -When filtering, redirecting, or summarizing output, preserve the check's own terminal status and relevant failures. A filter's success does not prove the check passed; an unknown status or inaccessible evidence stays `UNVERIFIED` until established. +Delegates, leads, and reviewers inspect and reuse a recorded pass for the same command and fingerprint while the relevant environment is unchanged. Accepting a return, reviewing, integrating, or reporting requires inspecting that record and current inputs, not another execution. Rerun only for changed inputs, missing or unverifiable evidence, or a concrete named doubt. Scratch files or service state may change without changing the repository fingerprint. A different tree requires evidence for that candidate, including revision-bound CI where available. Role and stage transitions invalidate nothing by themselves. Required coverage, independent review, and destination checks remain necessary. -An intermittent test is a defect or an explicit blocker until it is classified; [diagnosis](diagnosis.md) covers that. +Preserve the check's own terminal status and relevant failures when filtering or shortening output. A filter's success proves no check passed. Unknown status or inaccessible evidence stays `UNVERIFIED` until established. An intermittent test remains a defect or explicit blocker until classified under [diagnosis](diagnosis.md). ## Reviewing a change @@ -50,7 +50,7 @@ Check behavior, missing cases, scope, security and data risks, compatibility, er Read-only review reports defects without edits. Urgency grants no repair authority; sensitive findings stay private unless disclosure is granted. When repair is authorized, the lead confirms findings against evidence and has qualifying in-scope defects repaired. Retain implementer and reviewer through corrections where supported; replacements receive findings, dispositions, candidate identity, and evidence. The lead corrects plans. -After independent review, obtain targeted independent review of material corrections and affected consequences. Supply the prior candidate, changes, and dispositions. Reuse still-applicable evidence; widen for a concrete risk or invalidated evidence, not repeated unchanged investigation. Revalidate affected checks on the final candidate. Stop when no supported material defect remains, or report unresolved work and affected readiness limits. Another broad reviewer needs high-consequence disagreement or contradictory evidence. Incomplete review never establishes reviewed readiness. +After independent review, obtain targeted independent review of material corrections and affected consequences. Supply the prior candidate, changes, and dispositions. Reuse still-applicable evidence; widen for a concrete risk or invalidated evidence, not repeated unchanged investigation. Establish applicable evidence for affected checks on the final candidate. Stop when no supported material defect remains, or report unresolved work and affected readiness limits. Another broad reviewer needs high-consequence disagreement or contradictory evidence. Incomplete review never establishes reviewed readiness. ## Security, reliability, and operations diff --git a/site/evidence/index.html b/site/evidence/index.html index 9ddde93..bbbe480 100644 --- a/site/evidence/index.html +++ b/site/evidence/index.html @@ -56,11 +56,11 @@

Claims stop where the receipts stop.

-
+

01 · Historical observations

What controlled 2.x runs showed.

-

Historical observations below: 2.x only. Current package: 6.3.2. Retained current-package CTO scenarios with Observed receipts: 0 of 12. Eight historical 4.0.1 Claude run records remain incomplete. Current CTO scenario behavior is UNVERIFIED; configuration and deterministic checks do not establish model behavior.

+

Historical observations below: 2.x only. Current package: 6.3.3. Retained current-package CTO scenarios with Observed receipts: 0 of 12. Eight historical 4.0.1 Claude run records remain incomplete. Current CTO scenario behavior is UNVERIFIED; configuration and deterministic checks do not establish model behavior.