Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 74 additions & 0 deletions .github/workflows/automation-documentation-quality-ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
name: Automation Documentation Quality CI

on:
pull_request:
branches: [main]
paths:
- ".github/workflows/**"
- "docs/**"
- "*.md"
- "scripts/ci/**"
- "tests/**"
- "requirements-opencode-review-ci-hashes.txt"
- "pyproject.toml"
push:
branches: [main]
paths:
- ".github/workflows/**"
- "docs/**"
- "*.md"
- "scripts/ci/**"
- "tests/**"
- "requirements-opencode-review-ci-hashes.txt"
- "pyproject.toml"

concurrency:
group: automation-documentation-quality-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

permissions:
contents: read

env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: true
PYTHONDONTWRITEBYTECODE: "1"

jobs:
docs-quality:
name: Full suite and documentation contracts
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- name: Harden runner
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit

- name: Checkout exact event revision
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
fetch-depth: 1
persist-credentials: false

- name: Set up Python
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.14"
cache: pip
cache-dependency-path: requirements-opencode-review-ci-hashes.txt

- name: Install hash-pinned test dependencies
run: >-
python -m pip install --disable-pip-version-check
--require-hashes
-r requirements-opencode-review-ci-hashes.txt

- name: Run full repository suite with permanent coverage gate
run: |
python -m coverage erase
python -m coverage run -m pytest tests -q
python -m coverage report --fail-under=100

- name: Compile test contracts
run: python -m compileall -q tests
2 changes: 2 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,5 @@

Materialize accepts only exact SHA-256 pins or a bounded relative `-r` include (no `.`/`..`); a lone `--require-hashes` directive is not trust evidence. See [`docs/doctoring/hourly-nvidia-nim-autofix.md`](docs/doctoring/hourly-nvidia-nim-autofix.md).
Conflict-scope roots fail closed when the immediate parent directory is a symbolic link.

Automation control-plane specifications: [`docs/automation/README.md`](docs/automation/README.md).
6 changes: 5 additions & 1 deletion ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,4 +107,8 @@ tests pin workflow structure and governance prose so drift fails closed.
- [`docs/doctoring/hourly-nvidia-nim-autofix.md`](docs/doctoring/hourly-nvidia-nim-autofix.md)
— current increment's repair-worker decision and APA 7th citations.
- [`docs/doctoring/fast-mlsirm-hourly-review-caller.md`](docs/doctoring/fast-mlsirm-hourly-review-caller.md)
— product-specific psychometric repair heartbeat and scientific gates.
— product-specific psychometric repair heartbeat and scientific gates.

## Automation control plane

See [docs/automation/README.md](docs/automation/README.md).
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ Semantic Versioning where the repository publishes a release.

### Added

- Added authoritative automation control-plane specifications, ADRs, and documentation-quality CI.

- Added a trusted pull-request comment router for `@cwl-noema-review` and review-only `@opencode-agent` dispatches, with an organization sweep, exact-head receipts, repository allowlisting, fixed runners, immutable checkout pins, and a permanent 100% statement/branch/docstring quality gate.
- Added exact-base `uv.lock` materialization that reconstructs standalone nested projects with a checksum-pinned official `uv` exporter, isolated frozen/offline execution, strict exact-pin and SHA-256 output validation, and complete Python 3.10/3.14 quality evidence.
- Added a permanent exact-head contract workflow for the hourly review-repair scheduler, immutable reusable-workflow source, NVIDIA NIM model boundary, credential isolation, and fail-closed unattended-agent permissions.
Expand Down
2 changes: 2 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,3 +128,5 @@ repeatable compile command.
cross-repo references as `owner/repo#num` or full URLs; durable knowledge in the repo/Project, not
private memory; one roadmap phase at a time) are defined in `docs/CWL-MASTER-CONTEXT.md` §7 and
apply here.

Automation control-plane specifications: [`docs/automation/README.md`](docs/automation/README.md).
6 changes: 6 additions & 0 deletions PR_GOVERNANCE_AUDIT.md
Original file line number Diff line number Diff line change
Expand Up @@ -428,3 +428,9 @@ PR #381: wait: OpenCode review is already in progress
- `.github` PR #42 same-head OpenCode run `28070438305` exposed a second decode gap: model output reading tolerated invalid UTF-8, but approval-summary repair still read `OPENCODE_APPROVAL_REPAIR_EVIDENCE_FILE` as strict UTF-8. DeepSeek produced a repairable control block, then normalization failed on byte `0xea` in bounded evidence. Evidence repair now reads lossy UTF-8 so a damaged transcript byte cannot prevent source-backed normalization.
- `codec-carver` PR #98 already has base `opencode.jsonc`. PR #98 now pins the central scheduler instead of downloading from `main`; same-head Strix run `28030439830` and OpenCode runs `28030438605`/`28030439065` were still in progress at the 2026-06-23 22:48 KST snapshot.
- `.github` PR #38 exposed two central gaps after PR #37 merged: the `review_dispatch` reason lost the `same-head Strix and OpenCode dispatched` contract string, and `failed_status_checks()` treated failed PR-target Strix check runs as blockers even when a later manual `strix` status could supersede them. Commit `7be2d99` restores the reason string, materializes PR-head scheduler policy as non-executed data for Strix self-test, and ignores stale Strix check-run failures when the same head has a successful `strix` status context. Manual Strix run `28030448032` had passed self-test and was still running `Run Strix (quick)` at the 2026-06-23 22:48 KST snapshot.

## Current executable policy corrections

The hourly repair caller uses `cancel-in-progress: true`. Queue starvation is
tracked in https://github.com/ContextualWisdomLab/.github/issues/889. The
central autofix worker remains source-actionable current-head review repair only.
6 changes: 6 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,3 +149,9 @@ Operational cases folded into the central policy:
reduce maintainer, reviewer, CI-operator, contributor, user, or reader
friction. Noisy automation, repeated waiting, false failures, misleading
statuses, and URL-only diagnostics are treated as review-experience defects.

## Automation control plane

Authoritative specifications live in [docs/automation/README.md](docs/automation/README.md).
Protected `main` requires exactly two approving reviews. The scheduled repair
worker cannot invoke Bash, task/subagents, or webfetch.
98 changes: 98 additions & 0 deletions docs/automation/ARCHITECTURE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
# Architecture — CWL automation control plane

Status: accepted baseline
Last reviewed: 2026-08-09

## 1. Architecture description approach

This description follows stakeholder/viewpoint separation from ISO/IEC/IEEE 42010:2022. It documents context, component, information, security, execution, and operations viewpoints. Standards and primary sources are listed in [`../doctoring/automation-control-plane-standards.md`](../doctoring/automation-control-plane-standards.md).

## 2. Context and bounded responsibilities

```mermaid
flowchart TB
Human["Maintainer / independent reviewer"]
GitHub["GitHub API, rulesets, checks, reviews"]
Central["CWL central automation control plane"]
Product["Independent product repositories"]
Provider["Review and security providers"]

Human --> GitHub
GitHub --> Central
Central --> GitHub
Product --> Central
Central --> Provider
Provider --> Central
Central --> Product
```

The central control plane owns shared governance policy, trusted workflow source, dispatch validation, evidence normalization, and safe scheduling. GitHub owns repository state, rule enforcement, formal review identities, checks/status APIs, and merge primitives. Product repositories own product code, product data, domain tests, release/deployment, and thin central integrations. Providers supply advisory or configured machine evidence but do not acquire merge or human-review authority.

## 3. Logical components

```mermaid
flowchart TB
Entry["Trusted entry workflows"]
Identity["Identity and revision validator"]
Evidence["Evidence collectors and normalizers"]
Decision["Review / merge / fix schedulers"]
Sandbox["Sandbox and redaction boundary"]

Entry --> Identity
Identity --> Evidence
Evidence --> Sandbox
Sandbox --> Decision
Decision --> Entry
```

| Component | Current implementation examples | Responsibility |
|---|---|---|
| Trusted entry workflows | `opencode-review.yml`, `opencode-review-dispatch.yml`, `noema-review.yml`, `strix.yml` | Materialize stable contexts; keep privileged execution on trusted default-branch source. |
| Identity validator | workflow metadata checks, `pr_head_replay_guard.py`, dispatch validators | Bind repository, PR, head, base, actor, workflow source, and run attempt. |
| Evidence collectors | `collect_failed_check_evidence.sh`, review context and normalizers | Gather bounded current evidence without merging authority classes. |
| Sandbox/redaction | `sandboxed_verify.py`, `sandboxed_web_e2e.py`, `redact_sensitive_log.py` | Execute bounded proof paths and publish safe, useful diagnostics. |
| Decision/schedulers | `pr_review_merge_scheduler.py`, `pr_review_fix_scheduler.py`, autofix workflows | Select allowed actions under expected-head, policy, and writer constraints. |
| Mention router | `agent_mention_router.py`, `agent_mention_sweep.py`, exact-name artifact ledger | Authenticate, deduplicate, and forward explicit review-agent requests. |
| Security/supply-chain gates | CodeQL, Semgrep, OSV, Scorecard, SBOM, secret scan, Strix | Produce independent security evidence. |

## 4. Control plane and data plane

The **control plane** reads metadata, classifies evidence, selects work, validates authority, and requests GitHub operations. The **evidence data plane** carries source archives-as-data, test output, logs, findings, review bodies, status/check records, and artifacts. Product application data is outside both planes.

Untrusted PR content may enter the evidence data plane only through a bounded low-privilege execution or inert inspection path. It must not cross into a privileged `pull_request_target` or default-branch write path as executable code. A privileged action consumes validated identity and bounded evidence, not arbitrary PR shell text.

## 5. Trust boundaries

| Boundary | Untrusted side | Trusted side | Required control |
|---|---|---|---|
| PR to base workflow | PR source, filenames, metadata, artifacts | protected workflow source | No untrusted checkout/execution in privileged event; strict input validation. |
| Dispatch caller to central worker | payload and actor claims | default-branch dispatcher | Actor allowlist, canonical schema, live PR comparison, replay/idempotency receipt. |
| Provider to review gate | model text and tool output | normalized gate record | Schema validation, head/run binding, adversarial receipts, no raw authority transfer. |
| Sandbox to public evidence | child stdout/stderr/logs/commands | CI log, summary, comment, result JSON | Complete-boundary redaction, canonicalization, bounded output, stable schema. |
| Central to product repository | central App/token and mutation request | product branch/ruleset | Least privilege, expected head, branch-local lease, protected review. |
| Check/review evidence to merge | heterogeneous evidence | GitHub merge primitive | Authority separation, required gate inventory, qualifying approval, thread state. |

## 6. Failure domains

- **GitHub control-plane failure:** API, Actions queue, artifact service, or ruleset availability. Actions remain non-passing; maintenance rotates to other safe work.
- **Provider failure:** OpenCode, Noema, Strix, or upstream model capacity/transport. Provider evidence remains absent or failed; deterministic work continues.
- **Runner/toolchain failure:** DNS, package index, compiler, container, or toolchain. Classified transient failures may retry within a budget; integrity failures do not.
- **Revision race:** head/base changes after evidence collection. Expected-head checks abort mutations and invalidate stale acceptance.
- **Evidence failure:** malformed, oversized, secret-bearing, ambiguous, or contradictory data. Fail closed while preserving bounded non-sensitive diagnosis.
- **Governance failure:** no eligible independent reviewer or required permission. Record the exact external prerequisite and rotate work; never weaken policy.
- **Documentation failure:** implementation and authoritative contracts diverge. Treat as repository debt and repair with machine-checked links/terms.

## 7. Deployment topology

The central repository is the organization policy/source repository. Organization required workflows and thin `workflow_call` consumers materialize stable check contexts in target repositories. Default-branch `repository_dispatch` workers perform validated privileged operations. GitHub-hosted runners are disposable execution environments; durable state is GitHub repository/PR/check/review/artifact state and, conceptually, the evidence entities in [DATA_MODEL.md](DATA_MODEL.md).

No new always-on service or database is required by this architecture baseline. Persisting the conceptual evidence model would be a separate architecture decision.

## 8. Evolution rules

- Add central capability behind a versioned thin contract; do not copy a thick workflow into every product repository.
- Preserve independent product operation and product-owned gates.
- Prefer immutable action/workflow pins and short-lived identity.
- Separate deterministic gates from optional model execution so missing model credentials do not prevent unrelated validation.
- Change one authority boundary at a time and require consumer proof after protected integration.
- Keep diagrams, ADRs, tests, operations, and traceability synchronized with source changes.
Loading
Loading