Skip to content

feat: import bounded external test evidence - #199

Open
YuzeJ21 wants to merge 24 commits into
mainfrom
codex/junit-evidence-adapter
Open

feat: import bounded external test evidence#199
YuzeJ21 wants to merge 24 commits into
mainfrom
codex/junit-evidence-adapter

Conversation

@YuzeJ21

@YuzeJ21 YuzeJ21 commented Aug 21, 2026

Copy link
Copy Markdown
Owner

Summary

  • add a bounded, local, bytes-only JUnit XML parser that never executes target-repository code, follows artifact references, or fetches remote resources
  • require explicit human criterion mappings and bind every validated import to repository, PR, exact 40-character head, ordered confirmed-criteria digest, and criteria-source provenance
  • persist imported test results as a versioned Pydantic envelope separate from E1/E2 candidates, E3/E4 runtime evidence, GitHub CI observations, human resolutions, final acceptance, and deterministic gate truth
  • cap each review at 20 retained imports and each warning or limitation at 1,000 characters; a new analysis cannot inherit imports from an earlier criteria revision
  • expose one shared core workflow through CLI and Streamlit, with atomic saved-review mutation, reopen validation, safe exports, and comparison-aware provenance
  • cover the installed-wheel import, inspect, save, reopen, and safe-export round trip in loopback-only Chromium
  • retain the exact 1 MiB widget/parser boundary while allowing multipart transport overhead; require Streamlit 1.53+, the first compatible per-widget implementation

Evidence boundary

Imported JUnit results are externally supplied, non-gating context. ScopeProof did not execute the tests or target-repository code. The artifact digest identifies imported bytes only. Importer identity is asserted, not authenticated. Explicit criterion mapping is organizational context, not proof. An import never changes a finding or decision, satisfies E3/E4, creates or preserves Ready, records final acceptance, or carries a decision to another head.

This pull request is engineering evidence only. It does not establish correctness, customer validation, accessibility conformance, authenticated identity, broad platform support, demand, adoption, or Stage 3 activity.

Verification

  • Ruff: passed
  • complete suite: 2,495 passed, 2 intentional skips
  • combined coverage: 95.11%
  • repository contracts: 125 passed
  • final re-analysis lifecycle regression: passed red-to-green; rejected state remained unmodified
  • deterministic acceptance benchmark: 12 cases, zero mismatches, zero must-have False Ready outcomes, zero false blockers
  • comparison benchmark: 2 cases, zero mismatches
  • two byte-identical wheels: SHA-256 f6d888b5b033a045d2d8069b43cafafc3179436729d393e41a792e98f9bb051b
  • artifact inventory: 104 expected entries, 1,425,047 uncompressed bytes, 306,631-byte wheel; .coverage 2, local state, worktrees, caches, and raw JUnit XML absent
  • clean final-wheel Python 3.12.0 install: dependencies compatible, source/installed version 0.2.4.dev0, both CLIs correct, and both installed benchmarks returned zero mismatches; final-head hosted Python 3.11 and 3.13 compatibility lanes and the Windows package/storage/CLI lane are green
  • installed-wheel Chromium: final repaired-head run passed 5/5; the preceding synchronization repair also passed three consecutive fresh 5/5 runs, with loopback-only networking and no captured external requests, console errors, or page errors
  • exact installed loopback workbench health returned ok; the server then stopped and the endpoint was unreachable
  • final diff and commit audit: clean

Review

Automated review surfaced four P2 findings after initially green heads: a criteria-revision reset could preserve stale imports, individual limitation strings were unbounded, accumulated imports were unbounded, and re-analysis attachment could preload imports outside the append-only lifecycle. All four were reproduced with failing regressions and repaired in the shared core. No unresolved Critical or Important findings remain; all hosted checks are terminal green except the two intentional skips.

Product stage truth

  • Stage 1: closed_not_pursued_by_owner
  • 0/5 qualifying reviews
  • 0/3 independent practitioners
  • 0/3 public repositories
  • 0/3 independently observed under-ten-minute completions
  • 0/2 reuse-intent signals
  • Stage 2: owner_led_productization_active
  • Stage 3 has not begun

Unsupported environments

Local installed-browser evidence is Chromium on macOS/POSIX. Hosted compatibility lanes provide Python and package/storage evidence, not real desktop workflow evidence. No real Windows or Linux desktop workflow, real screen-reader workflow, or accessibility-conformance claim is included.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1c6f4b75ab

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scopeproof_core/schemas/models.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dc4c34e082

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scopeproof_core/schemas/models.py

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1a527ee777

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scopeproof_core/schemas/models.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 40ad63eb08

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scopeproof_core/reviews/lifecycle.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant