Repository navigation
Conversation
harness verify --embed-repro now attaches a stable 'repro' block to the
evidence it writes:
{
'captured_at': '...',
'harness_sha': '<40-hex>',
'harness_version': 'X.Y.Z',
'python_version': '3.11.9',
'python_implementation': 'CPython',
'platform': '...',
'host': '...',
'packages_sha256': '<sha256 or null>',
'packages_count': N,
'github_sha': '<sha or null>',
'github_run_id': '... or null',
'github_run_attempt': '... or null'
}
Stable on the same machine: harness_sha, harness_version, python_version,
platform, host, packages_sha256, packages_count. captured_at is the only
wall-clock field. github_* fields pick up GITHUB_* env vars when running
under Actions.
implementation
- harness/repro.py (new): embed_repro_metadata(root, *, harness_sha=None).
Total contract: never raises; missing data lands as None. harness_sha
resolves by walking up from the installed harness module's origin
looking for '.git' and 'git rev-parse HEAD'. packages_sha256 derives
from 'python -m pip freeze'.
- harness/verify.py cmd_verify: when --embed-repro is on, ev['repro']
is set before write_evidence. Same code path as run-direct and
runner.py:_verify_run is a follow-up (tracked separately).
- harness/cli.py: --embed-repro flag on the verify subparser (default off).
- harness/evidence.py render: optional 'Reproducibility' section in the
HTML evidence so reviewers can scan it without opening JSON.
tests
- tests/test_repro.py (new, 10 tests, all pass): shape, well-typedness,
field stability across consecutive calls, override harness_sha,
github env propagation, total/never-raise contract, root arg symmetry.
verified
- uv run pytest -q tests/test_repro.py → 10 passed
- uv run pytest -q (full suite) → ~104 passed
- uv run ruff check (4 touched files) → clean
- uv run mypy harness/repro.py → no issues
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR adds
harness verify --embed-repronow writes a stable reproducibility block into the evidence JSON it produces, alongside the existingtask,goal,accept,results,files,stat,verdict,timefields.{ "captured_at": "2026-10-11T10:23:32+00:00", "harness_sha": "cd10f245519756198fe9c5e37baf414cdeedb07f", "harness_version": "1.1.0", "python_version": "3.11.9", "python_implementation": "CPython", "platform": "Linux-7.2.9-zen1-1-zen-x86_64-with-glibc2.44", "host": "omarchy", "packages_sha256": null, "packages_count": null, "github_sha": null, "github_run_id": null, "github_run_attempt": null }Stable across consecutive invocations on the same machine:
harness_sha,harness_version,python_version,platform,host,packages_sha256,packages_count. The only wall-clock field iscaptured_at. The GitHub-Actions fields (github_sha,github_run_id,github_run_attempt) come fromos.environand pick up automatically in CI.Why it matters
Evidence is the single artifact that flows from
harness verifyto reviewers, dashboards, and PR-merge gates. Today, an evidence JSON says what passed; tomorrow's reviewer also needs which build of harness, on which Python, on which host passed. The repro block makes evidence self-describing — six months later, anyone holding the JSON can prove what runtime produced it. This is the smallest possible cross-cutting substrate for the larger cross-repo verification story in the org's Q4 roadmap (STRATEGY.mdinQuattro-Commas.github.io).Files
harness/repro.py(new)embed_repro_metadata(root, *, harness_sha=None) -> dict. Total contract — never raises; missing data lands asNone.harness/verify.pycmd_verifywritesev["repro"] = embed_repro_metadata(r)whena.embed_reprois set.harness/cli.py--embed-reproflag on theverifysubparser (default off, no behaviour change for existing callers).harness/evidence.pyrenderadds an optionalReproducibilitysection to the HTML evidence.tests/test_repro.py(new)Tests
uv run pytest -q tests/test_repro.py→ 10 passeduv run pytest -q(full suite) → ~104 passeduv run ruff check(4 touched files) → cleanuv run mypy harness/repro.py→ no issuesBehaviour change
None for existing callers.
--embed-reprois opt-in; default off. A re-run of any currentharness verifyinvocation produces byte-identical evidence.Follow-ups (out of scope here)
runner.py:_verify_runparity — the same flag wired throughserve/the API. Today, only the CLI path emitsrepro; the dashboard inherits nothing until that lands.harness task start --embed-repromakesrepropart of the contract (no longer passed per-invocation).