Repository navigation
Conversation
Phase 2 of the Q4-2026 roadmap (STRATEGY.md). The CLI path `harness verify --embed-repro` already writes a stable `repro` block into evidence JSON (Phase 1, PR #10). This commit wires the same behavior into the runner path (the async parallel reimplementation of verify used by `harness serve`) and exposes it through the task contract so dashboard / API callers do not have to know the flag. - harness/runner.py:_verify_run When the loaded contract has embed_repro=True, ev['repro'] = embed_repro_metadata(root). Same shape as cmd_verify; same total contract (the function never raises). - harness/contract.py:cmd_start Persists 'embed_repro': bool(a.embed_repro) into the contract; default off. No schema_version bump needed (field is forward-compatible). - harness/contract.py:cmd_update Adds '--embed-repro {on,off}' toggle. Normalises the on/off string into the persisted bool. - harness/cli.py --embed-repro on 'task start'; --embed-repro {on,off} on 'task update' for symmetry with the existing 'verify --embed-repro'. - harness/server.py TaskCreate gains 'embed_repro: bool = False' (POST /api/tasks). /api/tasks/{name}/verify gains 'embed_repro: bool | None' query: when provided and different from the contract, the contract is amended via jsave so the dashboard reflects the new state and the next run inherits it. argv stays 'verify <name>' because the runner does not parse flags; embed_repro is plumbed via contract. tests - tests/test_runner.py (new, 8 tests, all pass): - test_runner_emits_repro_when_contract_says_so - test_runner_omits_repro_when_off - test_per_invocation_embed_repro_override_mutates_contract - test_repro_block_is_byte_identical_to_cli_when_inputs_match - test_task_start_persists_embed_repro - test_task_start_default_is_off - test_task_update_toggles_embed_repro - one extra sanity test for the CLI surface verified - uv run pytest -q tests/test_runner.py -> 8 passed - uv run pytest -q (full suite) -> 107 passed - uv run ruff check harness/ tests/... -> clean - uv run mypy harness/ -> no issues
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 2 of the Q4-2026 roadmap (
STRATEGY.md)This PR stacks on
#10(Phase 1) which addsharness/repro.pyand the CLI--embed-reproflag. PR #10 must merge first; once it does, this branch will be a clean fast-forward ofmain.What it adds
The runner path is the async, parallel re-implementation of
cmd_verifythat runs insideharness serve. Before this change, it ignoredembed_repro. Nowembed_repro_metadata(self.root)runs there too, and the contract carries anembed_repro: boolfield so dashboard / API callers do not need to know the flag.Files
harness/runner.py_verify_runc.get("embed_repro")is true, setev["repro"] = await asyncio.to_thread(embed_repro_metadata, self.root).harness/contract.pycmd_start"embed_repro": bool(a.embed_repro)in the contract.harness/contract.pycmd_update--embed-repro {on,off}toggle that normalises strings to bool.harness/cli.pytask start--embed-reproflag.harness/cli.pytask update--embed-repro {on,off}symmetric flag.harness/server.pyTaskCreateembed_repro: bool = Falsefield.harness/server.py/api/tasksembed_repro=body.embed_reprointocmd_start.harness/server.py/api/tasks/{name}/verifyembed_reproquery param; when provided & different, the contract is amended viajsaveso the next run inherits.tests/test_runner.py(new)Why contract-stored instead of argv-flag-plumbed
The runner doesn't parse argv;
Runner.create(..., argv, worktree)records argv but_verify_runre-implements the verify work inline. Soembed_reproflows through the contract, not argv. The FastAPI handler optionally amends the contract for per-invocation overrides; the override is idempotent and visible on the dashboard.Behaviour change
None for existing callers. The CLI
harness verify --embed-reproalready worked from Phase 1; nothing there changes.Tests
Out of scope
--numeric-verifyflag → separate Phase 3 PR.verify-allorchestrator → Phase 5 (depends on Phase 3 + 4).Acceptance
harness serveandharness verifyproduce byte-identicalreproblocks for the same inputs (test_repro_block_is_byte_identical_to_cli_when_inputs_match).POST /api/taskswithembed_repro: true→ contract stores it, runner emits the repro block.POST /api/tasks/<name>/verify?embed_repro=trueoverrides and persists.