Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
e729fa3
feat(code-review): deliver portable project runtime (#473)
djm81 Sep 14, 2026
13300fb
fix(code-review): preserve project context and isolate runtime builds…
djm81 Sep 15, 2026
61e5d98
chore(code-review): retain CI publisher signature for portable runtim…
djm81 Sep 15, 2026
76f6ad9
fix(code-review): isolate build diagnostics from runtime payloads
djm81 Sep 15, 2026
0b5bdb2
chore(code-review): retain CI signature for diagnostic isolation
djm81 Sep 15, 2026
0a63a91
fix(code-review): preserve runtime diagnostics and nested import paths
djm81 Sep 15, 2026
ce96307
chore(code-review): retain CI signature for portable runtime fixes
djm81 Sep 15, 2026
1293aad
feat(code-review): deliver portable project runtimes (#473) (#475)
djm81 Sep 15, 2026
01ea1a1
chore(registry): publish changed modules [skip ci]
github-actions[bot] Sep 15, 2026
23410c2
chore(registry): publish changed modules (#476)
djm81 Sep 15, 2026
dad3257
Merge branch 'main' into dev
djm81 Sep 15, 2026
f6ca19d
fix(code-review): preserve native pytest outcomes and source coverage
djm81 Sep 15, 2026
b21e89e
chore(code-review): retain CI publisher signature for 0.50.1
djm81 Sep 15, 2026
1f480e4
fix(code-review): resolve pytest finding paths through native root
djm81 Sep 15, 2026
be1421b
chore(code-review): retain CI signature for pytest path correction
djm81 Sep 15, 2026
efe7303
fix(code-review): preserve pytest startup failure evidence
djm81 Sep 15, 2026
a9bea49
fix(code-review): retain project runtime and native coverage
djm81 Sep 16, 2026
38402b9
chore(code-review): import CI signature for 0.50.1
djm81 Sep 16, 2026
2cd9375
fix(code-review): measure single modules and execute corpus defects
djm81 Sep 16, 2026
8ef33d6
chore(code-review): import follow-up CI signature for 0.50.1
djm81 Sep 16, 2026
046d366
fix(code-review): preserve portable runtime and native pytest coverag…
djm81 Sep 16, 2026
da006b4
chore(registry): publish changed modules [skip ci]
github-actions[bot] Sep 16, 2026
3d6e984
chore(registry): publish changed modules (#479)
djm81 Sep 16, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions .github/workflows/capsule-customer-execution.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,8 @@ jobs:
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065
with:
python-version: ${{ matrix.python }}
- name: Install declared external corpus native requirements
run: sudo apt-get update && sudo apt-get install -y libodbc2 bubblewrap
- name: Prepare fresh customer-owned installation
shell: bash
run: |
Expand Down Expand Up @@ -107,13 +109,21 @@ jobs:
set -euo pipefail
# Administrator prerequisite only: all installation and analysis runs as the runner user.
# Keep Ubuntu's global AppArmor namespace restriction enabled.
sudo install -d -o root -g root -m 0755 /specfact-corpus-offline
sudo install -o root -g root -m 0555 /usr/bin/bwrap /specfact-corpus-offline/bwrap
sha256sum /usr/bin/bwrap | cut -d ' ' -f 1 | sudo tee /specfact-corpus-offline/bwrap.sha256 > /dev/null
sudo chmod 0444 /specfact-corpus-offline/bwrap.sha256
echo "SPECFACT_CORPUS_OFFLINE_LAUNCHER=/specfact-corpus-offline/bwrap" >> "$GITHUB_ENV"
profile="$CUSTOMER_ROOT/capsule-userns.apparmor"
cat > "$profile" <<EOF
abi <abi/4.0>,
include <tunables/global>
profile specfact-customer-capsule "$CUSTOMER_ROOT/cache/**/opt/specfact/bin/bwrap-static" flags=(unconfined) {
userns,
}
profile specfact-customer-offline-network "/specfact-corpus-offline/bwrap" flags=(unconfined) {
userns,
}
profile specfact-customer-capsule-umask "$CUSTOMER_ROOT/cache-077/**/opt/specfact/bin/bwrap-static" flags=(unconfined) {
userns,
}
Expand All @@ -127,13 +137,20 @@ jobs:
python scripts/capsule_customer_gate.py \
--repository "$GITHUB_WORKSPACE" \
--evidence "$CUSTOMER_ROOT/evidence" --mode "$MODE"
- name: Validate pinned external Python corpus with automatic runtimes
shell: bash
run: |
set -euo pipefail
unset GITHUB_TOKEN GH_TOKEN PYTHONPATH
python scripts/external_capsule_corpus.py --workspace "$CUSTOMER_ROOT/external"
- name: Upload reports and exact installation evidence
if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: capsule-${{ env.MODE }}-cp${{ matrix.python }}-${{ github.run_attempt }}
path: |
${{ env.CUSTOMER_ROOT }}/evidence/
${{ env.CUSTOMER_ROOT }}/external/evidence/
${{ env.CUSTOMER_ROOT }}/denied-evidence/
${{ env.CUSTOMER_ROOT }}/installation-evidence/
${{ env.CUSTOMER_ROOT }}/installed-*.txt
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/pr-orchestrator.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,8 @@ jobs:
- "packages/specfact-code-review/**"
- "registry/**"
- "scripts/capsule_customer_gate.py"
- "scripts/external_capsule_corpus.py"
- "tests/fixtures/portable-runtime/**"
- "scripts/link_dev_module.py"
- "publish_bundle_selection.py"
- "tests/unit/test_publish_bundle_selection.py"
Expand Down
3 changes: 3 additions & 0 deletions docs/_data/nav.yml
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,9 @@
- title: Run
url: /bundles/code-review/run/
expertise: [intermediate, advanced]
- title: External Python runtimes
url: /guides/portable-project-runtime/
expertise: [intermediate, advanced]
- title: Ledger
url: /bundles/code-review/ledger/
expertise: [intermediate, advanced]
Expand Down
6 changes: 6 additions & 0 deletions docs/bundles/code-review/run.md
Original file line number Diff line number Diff line change
Expand Up @@ -199,6 +199,12 @@ specfact code review run --scope changed --enforcement shadow --focus simplify -

Simplify-focused reports keep advisory `ai_bloat` findings plus high-confidence `dry` and `kiss` findings that include deterministic simplification metadata. Metadata fields such as `rewrite_hint`, `canonical_pattern`, `intent_key`, `estimated_deletion_lines`, `related_locations`, `signal_trace`, `preserve_reasons`, and `remediation_packet` are additive; legacy consumers can keep reading the original finding fields. The report-level `cleanup_forecast` summarizes reviewed LOC, estimated deletion ranges, guidance-kind totals, normalized AI-bloat density, weighted bloat points, and cleanup-yield LOC per KLOC. Simplification findings remain score-neutral; enforce mode blocks only unresolved safe-mechanical cleanup candidates.

## External project runtimes

Dependency-sensitive review automatically prepares an isolated project runtime. Use `--project-config PATH` to select a package manager, environment, extras, or test groups, and `--project-runtime PATH` to attach the descriptor returned by `specfact code review runtime prepare --json`. Inspect discovery without installation using `specfact code review runtime inspect --json`.

See [Review external Python projects](/guides/portable-project-runtime/) for pip, Hatch, uv, Poetry, offline reuse, and incomplete-evidence diagnostics. Local runtime provenance cannot authorize protected `pr_range` evidence.

## Related

- [Code review ledger](/bundles/code-review/ledger/)
Expand Down
123 changes: 123 additions & 0 deletions docs/guides/portable-project-runtime.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
---
layout: default
title: Review external Python projects
permalink: /guides/portable-project-runtime/
description: Prepare repository dependencies in an isolated sidecar runtime for capsule review.
keywords: [capsule, review, Python, pip, Hatch, uv, Poetry]
audience: [users, developers]
expertise_level: [intermediate]
doc_owner: specfact-cli-modules
tracks:
- packages/specfact-code-review/**
last_reviewed: 2026-09-14
exempt: false
exempt_reason: ""
---

# Review external Python projects

Portable project runtimes let the code review capsule use a repository's own dependencies and test configuration. Runtime acquisition runs in a disposable builder; analysis runs offline in separate target workers. The signed supervisor keeps its own dependencies. Analyzer-owned imports follow the recorded member graph and resolve from the verified analyzer installation before project paths or startup imports are considered; project-owned dependencies keep their selected versions. Member startup preserves ordinary editable hooks, restores analyzer lookup precedence afterward, and reports hooks that replace analyzer dispatch or protected import machinery, or add paths outside the copied runtime. Startup rejection remains incomplete analysis even when a hook prints output that resembles analyzer results. These startup checks do not sandbox arbitrary Python code within one interpreter. A namespace shared between analyzer-owned and project-owned portions is not currently combined; that configuration reports an origin mismatch and incomplete analysis.

Execution requires Linux x86-64 with the capsule's supported Python 3.11, 3.12, or 3.13 ABI and working unprivileged user namespaces. A local descriptor does not grant protected pull-request authority.

Windows and macOS host-to-worker setup is not implemented or validated by this change. Runtime inspection reads project metadata, but automatic preparation and capsule execution require the Linux environment above. A future container or VM handoff must validate source transfer, architecture, native dependencies, and returned evidence. Linux execution cannot establish Windows- or macOS-specific test behavior.

## Inspect and review

From the repository root, inspect the selected manager and input identities without installing dependencies:

```bash
specfact code review runtime inspect --json
```

Existing review commands prepare the target runtime automatically before dependency-sensitive analysis:

```bash
specfact code review run src/example.py tests/test_example.py --bug-hunt --json --out /tmp/review.json
```

A single declared test dependency group or test extra is selected automatically. Explicit groups and extras suppress automatic selection of the other category; competing choices require configuration. Hatchling alone is a build backend, so it does not imply a Hatch environment. Competing environment choices produce a diagnostic; select the intended environment explicitly. Discovery validates the metadata tables it consumes before choosing an environment, including when a manager is explicitly selected; malformed tables identify the source file and section.

## Select an environment

Store selection outside the reviewed source when possible. For a detached Hatch environment:

```toml
manager = "hatch"
environment = "review"
```

For a uv project with a test group:

```toml
manager = "uv"
groups = ["tests"]
```

For a Hatch test matrix, `environment = "hatch-test"` selects its unique native environment matching the selected Python version. Multiple matching matrix entries require a concrete name such as `hatch-test.py3.12-feature`. Selection uses Hatch's exported configuration.

The `python` setting overrides `.python-version`; both are checked against `requires-python` and the exact signed interpreter version. A root `.python-version` also activates automatic preparation for source-only projects without packaging metadata. A compatible controller ABI is reused; an incompatible controller does not force that ABI onto the project. Base and head reviews select their workers independently.

Poetry Python constraints are intersected with project `requires-python` before choosing a worker. PEP 440 ranges are supported; Poetry caret, tilde, union and table syntax currently produce an explicit unsupported-constraint diagnostic. Express the declaration as an equivalent PEP 440 range to proceed.

Supported manager names are `pip`, `hatch`, `uv`, and `poetry`. Optional fields are `python`, `groups`, `extras`, `requirements`, `constraints`, `source_roots`, `native_libraries`, and `native_tools`. Explicit `requirements` and `constraints` fields apply to `pip`; other managers reject these fields with a diagnostic rather than ignore them. Paths are repository-relative. Native libraries use ELF names such as `libodbc.so.2`; unavailable libraries are named in the preparation diagnostic. Hatch extras and dependencies belong in the selected native Hatch environment.

Repositories whose tests invoke external programs can explicitly declare the supported Linux capabilities `git`, `uname`, and `sed`:

```toml
[tool.specfact.code-review]
native_tools = ["git", "uname", "sed"]
```

Alternatively, put this in an external `/tmp/review-runtime.toml` without editing the repository:

```toml
native_tools = ["git", "uname", "sed"]
```

An explicit `native_tools` field in `--project-config` overrides the repository declaration; a configuration without that field retains it. Preparation captures the declared executables, Git HTTP and local transport helpers, and their ELF library closure before dependency builds. Their hashes and generated launchers are recorded in the runtime inventory and cache identity. Tools are installed into the target runtime after the build; conflicting package-provided command names fail explicitly. Git uses the selected capsule's sealed `/bin/sh`. Other programs and Git transports requiring additional external clients, such as SSH, remain unsupported unless separately available through declared project dependencies; host PATH directories are never imported. Target execution remains offline even when Git HTTP helpers are present. Workers retain the signed capsule's public certificate bundle so dependencies can initialize TLS; host trust stores and credentials are not imported. Missing capabilities and incompatible native binaries produce preparation diagnostics.

```bash
specfact code review runtime prepare --project-config /tmp/review-runtime.toml --json
specfact code review run src/example.py tests/test_example.py --project-config /tmp/review-runtime.toml
```

Plain package-index dependencies do not require Git. Preparation stages available Git capabilities; a dependency that actually needs a missing Git executable or HTTP transport fails with a preparation diagnostic and installation remedy. A newly initialized repository without a first commit has no VCS version context; ordinary dependency discovery can still proceed. Missing requested revisions or corrupt Git state remain errors.

Preparation returns the path to `project-runtime.json`. Generated environments, resolutions, and inventories stay in the private cache, outside customer source. Keep descriptors private: they contain project dependency and path metadata. Index credentials are excluded from published descriptor inventory; private failed-build logs are owner-readable.

## Reuse and attach

```bash
specfact code review runtime prepare --project-config /tmp/review-runtime.toml --offline --json
specfact code review run src/example.py tests/test_example.py --project-config /tmp/review-runtime.toml --project-runtime /path/from/prepare/project-runtime.json
```

Attachment verifies the payload, project inputs, ABI, and analyzer worker identity. Changed workspace source invalidates built project packages as well as changed dependency files. Git commits, tags, and shallow-history boundaries also participate in cache identity for dynamically versioned packages. Staged reviews bind the captured Git tree separately from its HEAD commit, so private build copies retain the staged index and sanitized VCS metadata even if the live index later changes. Private Git metadata contains only the object closure recorded by the selected commit, staged tree, tags, and shallow boundary; unrelated branch refs and unreachable objects are excluded. A corrupt cache is rejected; remove only the identified invalid artifact and prepare again. Interrupted preparations do not publish partial descriptors.

## Interpret incomplete evidence

Runtime failure preserves independent static findings. Affected analyzers reference the preparation diagnostic and remain incomplete. A real missing import after successful preparation remains a finding. A report can contain real failures while still showing incomplete required evidence.

Runtime attachment adds import roots only when declared through pytest `pythonpath` or explicit review `source_roots`. It does not infer `src/` or repository-root overrides from layout: installed packages, build-generated contents, editable hooks, and native pytest import modes retain their normal precedence.

Pytest configuration and selection controls remain active. Full reviews leave collection roots to pytest, including its configured path expansion and fallback when configured test paths are missing. Targeted reviews retain explicit test selection. Deselected tests are recorded separately from the selected execution inventory; distributed deselection visibility is labelled as controller-observed. Quoted pytest paths and file patterns preserve spaces using native argument rules. Malformed tables, invalid path/pattern value shapes, and unclosed quotes identify the section or option and configuration file. The report records collection, execution, setup and internal errors, configured options, versions, and coverage. Distribution-owned commands are available inside target workers. Localhost coordination for plugins stays inside the offline namespace. Collection-only runs, early stopping that leaves collected tests unexecuted, and absent required coverage do not become successful test evidence.

## Compatibility evidence

The checked-in corpus pins Requests, Hatch, Flask, and Poetry commits and their initial source/test slices. The customer workflow exercises supported Python ABIs, records host tests separately, checks cold preparation and offline attachment, and injects controlled defects. Untouched upstream acceptance requires completed applicable analysis; it does not require zero findings.

Track release acceptance in [User Story #473](https://github.com/nold-ai/specfact-cli-modules/issues/473) and the original [customer bug #472](https://github.com/nold-ai/specfact-cli-modules/issues/472). The reconstructed Hatch fixture is not proof that the inaccessible original customer reproduction is resolved.

Python subprocess options `-I`, `-E`, and `-S` disable the startup mechanism used to attach the selected runtime. The worker rejects these exact options with `project_python_option_unsupported` rather than running without project dependencies. Ordinary `-c`, `-m`, `-u`, and script invocations retain native Python argument handling.

Source aliases must resolve to included repository content. Aliases into excluded `.env`, virtual environments, or Git metadata are rejected before dependency building. Valid internal links are preserved in the private source copy, including absolute links rebased into that copy; the copied bytes are verified before build hooks execute.

Pip discovery recognizes `requirements.txt`, pip-tools `requirements.in`, and `pylock.toml`. Multiple lock/requirements alternatives require explicit selection. Standard lock handling is delegated to pip; pip 26.2.1 documents `pylock.toml` support as experimental ([pip install reference](https://pip.pypa.io/en/stable/cli/pip_install/), accessed 2026-09-14). Static `setup.cfg` Python constraints are imported when PEP 621 metadata does not provide them.

Portable analysis uses a verified private source copy with local environments and excluded secret files removed. The corpus records artifact bytes and Linux host-interface transfer counters; those counters include concurrent host traffic and are not exact package download sizes. Its warm preparation and attachment commands additionally run in a non-root network namespace, independently of the offline option.


Portable reviews retain skipped, XFAIL, and XPASS outcomes as non-passing test evidence, even when other tests pass. Each reviewed production Python source needs coverage evidence of at least 80 percent, or the effective project threshold when higher. A package directory used for pytest discovery does not exempt its production files from coverage. Missing coverage remains incomplete evidence.

Coverage provenance distinguishes native pytest-cov requests from review-only measurement. When the repository does not request coverage, the review collects private evidence through the supported pytest-cov lifecycle without activating its dormant aggregate `fail_under` policy. The per-reviewed-file floor above still applies. Native `--cov`, `--cov-reset`, `--cov-config`, source/include/omit settings, report destinations, threshold and precision retain their effect; actual native aggregate failures remain blocking findings. The report records both configured and effective measurement scopes. Explicit `--no-cov`, blocked coverage plugins, `--no-cov-on-fail`, excluded sources, missing distributed-worker data and unsupported plugin contracts leave required evidence incomplete. Native report write failures remain visible rather than replacing the requested output. The current lifecycle contract supports pytest-cov7.0.0 and7.1.0; xdist and configured Coverage subprocess collection use its native worker lifecycle.
Loading
Loading