Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,8 @@ The shipped text and every delegate brief follow one prompt standard. State the

## Checks

`scripts/check.py` and `scripts/check_hosts.py` use `agent-verify` when that optional host capability is on PATH. The adapter validates a live inherited grant or queues the entire check before any subprocess execution timeout begins. Admission status remains visible, and adapter errors stop the check. Hosts without that capability run the same portable checks directly. This integration adds no dependency to the shipped plugin.

Install the pinned dependencies with `python -m pip install -r requirements-dev.txt`; the checks install nothing and stop on an interpreter that misses a pin. Run focused tests through `python scripts/check.py --pytest <pytest-arguments>`. Before completion, obtain one full `python scripts/check.py` result and one clean `git diff --check` result for the final candidate. If the full run has bounded failures, repair them and rerun only the affected signals when inspection proves no other inputs changed; the full result and those later passes form the completion evidence. Bind each result to the code, dependencies, configuration, and environment it covered, and reuse it while those inputs remain equivalent. A commit, clean merge, tag, or release stage does not by itself invalidate evidence. After integration, use revision-bound CI or establish relevant tree equivalence; rerun only checks whose inputs changed. `tests/conftest.py` keeps a direct `pytest tests` run from writing bytecode into `plugins/skiphow/`, whose every file the package identity hashes. For changes that affect manifest shape, package layout, installation, or host validation, also run `python scripts/check_hosts.py`, which reports each host capability on its own row, and report an unavailable host as `UNVERIFIED`. A version value or new runtime prose inside an unchanged package structure does not by itself require a clean-install smoke. `scripts/check.py` validates the required CTO skill, sibling workflow links to it, reachable modules and resources, the safety shape of any shipped hook, aligned versions, and portability boundaries such as personal paths and versioned model IDs. Change the check and `docs/decisions.md` together when those invariants change.

Every deterministic assertion belongs to one class: a package or host contract, a security or release invariant, or the semantic validity of the behavioral corpus, all of which fail the run; or an editorial or layout preference, which is at most a non-blocking lint; or an assumption about the current shape, which is deleted unless it is promoted into a stated contract with a reason. Do not pin a marketing sentence, a method roster or count, a role set, a hook matcher topology, a site presentation detail, or a prose budget with no measured limit behind it.
Expand Down
2 changes: 2 additions & 0 deletions docs/decisions.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ This page records the choices that still matter when SkipHow changes. Read it be

## Current decisions

The deterministic and host contributor verification entrypoints optionally discover `agent-verify` on PATH and enters its host queue before starting check subprocesses. A live inherited grant avoids a second reservation; a stale environment does not. Hosts without the capability retain direct execution. The adapter owns queue policy, process cancellation and the whole-run watchdog, while the checker retains its existing subprocess budgets. No host path, dependency, queue implementation or admission requirement is shipped in the plugin. Deterministic adapter tests cover this repository integration; they do not prove model behavior.

The live decisions, their premises, and what would reopen each. "Evidence" says what stands behind the decision today: `Observed` means a retained run showed it on the package that carried it, `Contract` means the shipped text encodes it and no run has tested that text, `Deterministic` means a check proves it on every run. Last reviewed 2026-10-07 against the 6.3.1 candidate. The [owner-outcome contract](outcome-contract.md) governs implementation choices.

| Decision | Active rationale | Premises | Evidence | Reopens when |
Expand Down
10 changes: 10 additions & 0 deletions scripts/check.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@
from urllib.parse import unquote, urlsplit
import xml.etree.ElementTree as ET

from host_admission import enter_host_admission


ROOT = Path(__file__).resolve().parents[1]
PLUGIN_ROOT = ROOT / "plugins/skiphow"
Expand Down Expand Up @@ -2203,6 +2205,11 @@ def report_missing_requirements() -> None:
)


def host_admission(raw_args: list[str]) -> int | None:
"""Enter an installed host queue before starting any check execution budget."""
return enter_host_admission(raw_args, Path(__file__), "skiphow-check")


def main(argv: list[str] | None = None) -> int:
raw_args = list(sys.argv[1:] if argv is None else argv)
if not requirements_satisfied():
Expand All @@ -2217,6 +2224,9 @@ def main(argv: list[str] | None = None) -> int:
help="run pytest with the remaining arguments",
)
args = parser.parse_args(raw_args)
admission_result = host_admission(raw_args)
if admission_result is not None:
return admission_result
if args.pytest is not None:
environment = os.environ.copy()
environment["PYTHONDONTWRITEBYTECODE"] = "1"
Expand Down
9 changes: 8 additions & 1 deletion scripts/check_hosts.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@
import tempfile
from typing import Sequence

from host_admission import enter_host_admission


ROOT = Path(__file__).resolve().parents[1]
PLUGIN_ROOT = ROOT / "plugins/skiphow"
Expand Down Expand Up @@ -784,6 +786,7 @@ def render_matrix(rows: Sequence[tuple[str, str, str]]) -> str:


def main(argv: Sequence[str] | None = None) -> int:
raw_args = list(sys.argv[1:] if argv is None else argv)
parser = argparse.ArgumentParser(
description=(
"Report each host capability separately: the deterministic gate, each "
Expand Down Expand Up @@ -815,14 +818,18 @@ def main(argv: Sequence[str] | None = None) -> int:
help="directory that receives one privacy-safe JSON receipt per smoked host",
)
parser.add_argument("--matrix-out", help="also write the matrix to this file")
args = parser.parse_args(argv)
args = parser.parse_args(raw_args)
if args.skip_install and (args.require_codex_install or args.require_claude_install):
parser.error("--skip-install cannot satisfy --require-codex-install or --require-claude-install")
if args.smoke and args.skip_install:
parser.error("--smoke cannot be combined with --skip-install")
if args.smoke and not args.receipt_dir:
parser.error("--smoke requires --receipt-dir")

admission_result = enter_host_admission(raw_args, Path(__file__), "skiphow-host-check")
if admission_result is not None:
return admission_result

errors: list[str] = []
rows: list[tuple[str, str, str]] = []
codex = shutil.which("codex")
Expand Down
29 changes: 29 additions & 0 deletions scripts/host_admission.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
"""Connect contributor verification to an optional installed host queue."""

import os
from pathlib import Path
import shutil
import subprocess
import sys


def enter_host_admission(raw_args: list[str], entrypoint: Path, label: str) -> int | None:
"""Replace the entrypoint before its subprocess execution budgets begin."""
adapter = shutil.which("agent-verify")
if adapter is None:
return None
try:
inherited = subprocess.run([adapter, "inherited", "--memory", "4", "--cpus", "4"],
check=False).returncode
if inherited == 0:
return None
if inherited != 1:
print("host admission could not validate the inherited grant", file=sys.stderr)
return 78
os.execv(adapter, [adapter, "--memory", "4", "--cpus", "4", "--timeout", "2700",
"--label", label, "--", sys.executable, str(entrypoint.resolve()),
*raw_args])
except OSError as error:
print(f"host admission could not start: {error}", file=sys.stderr)
return 78
raise AssertionError("host admission exec returned")
2 changes: 2 additions & 0 deletions tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,10 @@
from __future__ import annotations

import os
from pathlib import Path
import sys


sys.dont_write_bytecode = True
os.environ["PYTHONDONTWRITEBYTECODE"] = "1"
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "scripts"))
7 changes: 7 additions & 0 deletions tests/test_hosts.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,13 @@ def load(name: str, relative: str):
hosts = load("skiphow_check_hosts_isolated", "scripts/check_hosts.py")


@pytest.fixture(autouse=True)
def portable_host_checks():
"""Schema/install tests do not depend on an optional host queue installation."""
with patch.object(hosts, "enter_host_admission", return_value=None):
yield


def test_skip_install_cannot_satisfy_a_required_install() -> None:
for required in ("--require-codex-install", "--require-claude-install"):
with pytest.raises(SystemExit) as raised:
Expand Down
119 changes: 119 additions & 0 deletions tests/test_verification_admission.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
"""Host queue integration remains optional and precedes check execution budgets."""

import importlib.util
import os
from pathlib import Path
import signal
import subprocess
import sys
import tempfile
from unittest.mock import patch

import pytest

import host_admission as admission


ROOT = Path(__file__).resolve().parents[1]
spec = importlib.util.spec_from_file_location("admitted_check", ROOT / "scripts/check.py")
check = importlib.util.module_from_spec(spec)
spec.loader.exec_module(check)
spec = importlib.util.spec_from_file_location("admitted_hosts", ROOT / "scripts/check_hosts.py")
hosts = importlib.util.module_from_spec(spec)
spec.loader.exec_module(hosts)


def test_off_host_runs_without_admission():
with patch.object(admission.shutil, "which", return_value=None), \
patch.object(check.subprocess, "run", side_effect=AssertionError("unexpected subprocess")):
assert check.host_admission([]) is None


@pytest.mark.parametrize("probe, expected", [(0, None), (78, 78), (-15, 78)])
def test_only_a_validated_live_grant_skips_the_queue(probe, expected):
with patch.object(admission.shutil, "which", return_value="adapter"), \
patch.object(check.subprocess, "run", return_value=subprocess.CompletedProcess([], probe)) as run, \
patch.object(check.os, "execv", side_effect=AssertionError("unexpected re-exec")):
assert check.host_admission([]) == expected
assert run.call_args.args[0] == ["adapter", "inherited", "--memory", "4", "--cpus", "4"]


def test_queue_replaces_the_process_before_execution_budgets_begin():
class ReplacedProcess(BaseException):
pass

arguments = ["--pytest", "tests/test_checks.py", "-k", "two words"]
with patch.object(admission.shutil, "which", return_value="adapter"), \
patch.object(check.subprocess, "run", return_value=subprocess.CompletedProcess([], 1)) as run, \
patch.object(check.os, "execv", side_effect=ReplacedProcess) as execute:
with pytest.raises(ReplacedProcess):
check.main(arguments)
# No pytest subprocess, capture buffer or execution deadline exists while queued.
assert run.call_count == 1
command = execute.call_args.args[1]
assert command[-len(arguments):] == arguments
assert command[:9] == ["adapter", "--memory", "4", "--cpus", "4", "--timeout", "2700",
"--label", "skiphow-check"]


def test_discovered_broken_adapter_does_not_bypass_admission():
with patch.object(admission.shutil, "which", return_value="adapter"), \
patch.object(check.subprocess, "run", side_effect=OSError("unavailable")):
assert check.host_admission([]) == 78


@pytest.fixture
def adapter_directory():
with tempfile.TemporaryDirectory(prefix="verification-admission-") as directory:
yield Path(directory)


@pytest.mark.skipif(os.name != "posix", reason="POSIX exec and signal lifecycle")
@pytest.mark.parametrize("entrypoint", ["check.py", "check_hosts.py"])
def test_queued_adapter_status_and_cancellation_reach_the_caller(adapter_directory, entrypoint):
"""A real exec has no checker parent left to swallow status or retain a child."""
adapter = adapter_directory / "agent-verify"
marker = adapter_directory / "started"
adapter.write_text(f"#!{sys.executable}\nimport os, pathlib, sys, time\n"
"if sys.argv[1] == 'inherited': sys.exit(1)\n"
"if os.environ.get('REFUSE'):\n print('queued', flush=True)\n sys.exit(75)\n"
f"pathlib.Path({str(marker)!r}).write_text(str(os.getpid()))\n"
"print('queued', flush=True)\n"
"time.sleep(60)\nraise AssertionError('queued command started')\n")
adapter.chmod(0o755)
environment = dict(os.environ, PATH=str(adapter_directory), REFUSE="1")
arguments = ["--pytest", "unused"] if entrypoint == "check.py" else ["--package-gate", "--skip-install"]
command = [sys.executable, str(ROOT / "scripts" / entrypoint), *arguments]
result = subprocess.run(command, env=environment, capture_output=True, text=True, timeout=10)
assert result.returncode == 75
assert "queued" in result.stdout
assert not marker.exists()
environment.pop("REFUSE")
process = subprocess.Popen(command, env=environment, stdout=subprocess.PIPE, text=True)
try:
assert process.stdout.readline().strip() == "queued"
assert int(marker.read_text()) == process.pid
process.send_signal(signal.SIGTERM)
assert process.wait(timeout=10) == -signal.SIGTERM
finally:
if process.poll() is None:
process.kill()
process.wait()
process.stdout.close()


def test_host_check_queues_before_its_captured_package_gate_budget():
class ReplacedProcess(BaseException):
pass

arguments = ["--package-gate", "--skip-install"]
with patch.object(admission.shutil, "which", return_value="adapter"), \
patch.object(admission.subprocess, "run", return_value=subprocess.CompletedProcess([], 1)) as run, \
patch.object(admission.os, "execv", side_effect=ReplacedProcess) as execute, \
patch.object(hosts, "checked", side_effect=AssertionError("captured child started before admission")):
with pytest.raises(ReplacedProcess):
hosts.main(arguments)
assert run.call_count == 1
command = execute.call_args.args[1]
assert command[8] == "skiphow-host-check"
assert command[-3:] == [str(ROOT / "scripts/check_hosts.py"), *arguments]
Loading