-
Notifications
You must be signed in to change notification settings - Fork 0
fix(coverage): restore trusted LLVM 19 producer pin #1057
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
f57bd93
bcd17db
594f35a
c68313b
d9167b6
adca662
7612a7a
a47d239
c443048
3bcc08a
2fbe4ce
0d1ea22
a4768bf
fb38faf
2a4c998
1cdc97e
b4ab001
d51496d
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,58 @@ | ||
| name: OpenCode Rust Coverage Toolchain Quality CI | ||
|
|
||
| on: | ||
| pull_request: | ||
| paths: | ||
| - ".github/workflows/opencode-review-dispatch.yml" | ||
| - ".github/workflows/opencode-rust-coverage-toolchain-quality-ci.yml" | ||
| - "scripts/ci/ensure_rust_llvm19.sh" | ||
| - "tests/test_opencode_rust_coverage_toolchain_contract.py" | ||
| - "tests/test_pr_review_autofix_nvidia_nim_contract.py" | ||
| - "docs/doctoring/opencode-rust-coverage-runtime-boundary.md" | ||
| - "CHANGELOG.md" | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| concurrency: | ||
| group: opencode-rust-coverage-toolchain-quality-${{ github.event.pull_request.number || github.ref }} | ||
| cancel-in-progress: true | ||
|
|
||
| jobs: | ||
| quality: | ||
| name: quality | ||
| runs-on: ubuntu-24.04 | ||
| timeout-minutes: 15 | ||
| env: | ||
| FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: true | ||
| steps: | ||
| - name: Harden runner | ||
| uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0 | ||
| with: | ||
| egress-policy: audit | ||
|
|
||
| - name: Checkout exact pull request head | ||
| uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | ||
| with: | ||
| ref: ${{ github.event.pull_request.head.sha }} | ||
| fetch-depth: 0 | ||
| persist-credentials: false | ||
|
|
||
| - name: Set up Python | ||
| uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 | ||
| with: | ||
| python-version: "3.14" | ||
| cache: pip | ||
| cache-dependency-path: requirements-opencode-review-ci-hashes.txt | ||
|
|
||
| - name: Install exact hash-locked test tooling | ||
| run: >- | ||
| python -m pip install --disable-pip-version-check --require-hashes | ||
| -r requirements-opencode-review-ci-hashes.txt | ||
|
|
||
| - name: Run permanent LLVM runtime-boundary contract | ||
| run: | | ||
| set -euo pipefail | ||
| python -m pytest -q tests/test_opencode_rust_coverage_toolchain_contract.py | ||
| python -m compileall -q tests/test_opencode_rust_coverage_toolchain_contract.py | ||
| git diff --check "${{ github.event.pull_request.base.sha }}...${{ github.event.pull_request.head.sha }}" |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,100 @@ | ||
| name: Repair PR 827 CodeRabbit comments | ||
|
|
||
| on: | ||
| pull_request: | ||
| types: [synchronize, reopened, ready_for_review] | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| concurrency: | ||
| group: repair-pr827-coderabbit-comments | ||
| cancel-in-progress: true | ||
|
|
||
| jobs: | ||
| repair: | ||
| if: >- | ||
| github.event.pull_request.number == 827 && | ||
| github.event.pull_request.head.repo.full_name == github.repository && | ||
| github.event.pull_request.head.ref == 'fix/opencode-rust-coverage-runtime-boundary-main' | ||
| runs-on: ubuntu-24.04 | ||
| timeout-minutes: 45 | ||
| permissions: | ||
| contents: write | ||
| env: | ||
| FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: true | ||
| steps: | ||
| - name: Harden runner | ||
| uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0 | ||
| with: | ||
| egress-policy: audit | ||
|
|
||
| - name: Checkout exact PR branch | ||
| uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | ||
| with: | ||
| ref: fix/opencode-rust-coverage-runtime-boundary-main | ||
| fetch-depth: 0 | ||
| persist-credentials: true | ||
|
|
||
| - name: Set up Python 3.14 | ||
| uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 | ||
| with: | ||
| python-version: '3.14' | ||
| cache: pip | ||
| cache-dependency-path: requirements-opencode-review-ci-hashes.txt | ||
|
|
||
| - name: Install exact hash-locked test tooling | ||
| run: >- | ||
| python -m pip install --disable-pip-version-check --require-hashes | ||
| -r requirements-opencode-review-ci-hashes.txt | ||
|
|
||
| - name: Apply bounded non-workflow repairs | ||
| run: | | ||
| set -euo pipefail | ||
| python - <<'PY' | ||
| from pathlib import Path | ||
|
|
||
| repair = Path('scripts/ci/repair_pr827_coderabbit_comments.py') | ||
| repair_text = repair.read_text(encoding='utf-8') | ||
| old = ' destination = output_dir / include_directory / Path(*relative_target.parts)\n' | ||
| new = ' destination = output_dir / include_directory / pathlib.Path(*relative_target.parts)\n' | ||
| if repair_text.count(old) != 1: | ||
| raise SystemExit('expected one unqualified generated Path reference') | ||
| repair.write_text(repair_text.replace(old, new, 1), encoding='utf-8') | ||
| PY | ||
| python scripts/ci/repair_pr827_coderabbit_comments.py | ||
| # The ordinary Actions token cannot update workflow files. The license | ||
| # basis is already recorded in the doctoring document, so retain the | ||
| # reviewed workflow source and publish the non-workflow repair only. | ||
| git checkout -- .github/workflows/opencode-review-dispatch.yml | ||
| rm -f scripts/ci/repair_pr827_coderabbit_comments.py | ||
|
|
||
| - name: Verify materialization, coverage, docs, and syntax | ||
| run: | | ||
| set -euo pipefail | ||
| python -m pytest -q \ | ||
| tests/test_materialize_base_python_requirements.py \ | ||
| tests/test_opencode_rust_coverage_toolchain_contract.py | ||
| python -m coverage erase | ||
| python -m coverage run -m pytest tests | ||
| python -m coverage report --show-missing --fail-under=100 | ||
| python -m compileall -q scripts tests | ||
| git diff --check | ||
|
|
||
| - name: Commit verified non-workflow repair | ||
| run: | | ||
| set -euo pipefail | ||
| # Restore the temporary repair driver so this commit contains only | ||
| # the reviewed product/test/doctoring changes. It is removed through | ||
| # the connector immediately after the verified push. | ||
| git checkout -- scripts/ci/repair_pr827_coderabbit_comments.py | ||
| git config user.name 'github-actions[bot]' | ||
| git config user.email '41898282+github-actions[bot]@users.noreply.github.com' | ||
| git add \ | ||
| scripts/ci/materialize_base_python_requirements.py \ | ||
| tests/test_materialize_base_python_requirements.py \ | ||
| CHANGELOG.md \ | ||
| docs/doctoring/opencode-rust-coverage-runtime-boundary.md | ||
| git diff --cached --check | ||
| git commit -m 'fix(coverage): preserve bounded requirement includes' | ||
| git push origin HEAD:fix/opencode-rust-coverage-runtime-boundary-main | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,128 @@ | ||
| # OpenCode Rust coverage LLVM runtime boundary | ||
|
|
||
| ## Decision | ||
|
|
||
| The trusted OpenCode coverage sandbox binds Rust coverage to the reviewed LLVM | ||
| 19 executables shipped by Debian's `llvm-19` package: | ||
|
|
||
| - `LLVM_COV=/usr/bin/llvm-cov-19` | ||
| - `LLVM_PROFDATA=/usr/bin/llvm-profdata-19` | ||
|
|
||
| These are compatibility and trust-boundary constants, not caller-selectable | ||
| configuration. The reviewed helper `scripts/ci/ensure_rust_llvm19.sh` binds both | ||
| exact paths and fails closed unless the live `LLVM_COV` / `LLVM_PROFDATA` | ||
| values match and are executable before Rust coverage evidence is admitted. The | ||
| independent OpenCode review-dispatch workflow stays byte-for-byte so the | ||
| review-agent key system is not rewritten to carry this runtime check. | ||
|
Comment on lines
+11
to
+16
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win 도구 바인딩과 blob pin의 담당자를 정확히 문서화하십시오.
또한 이 PR은 🤖 Prompt for AI Agents |
||
|
|
||
| The runtime MUST NOT fall back to unversioned `llvm-cov` or `llvm-profdata`, a | ||
| host-runner tool, a pull-request-selected path, or a dynamically downloaded LLVM | ||
| binary. Missing, changed, or non-executable reviewed paths are coverage-evidence | ||
| failures rather than reasons to measure a different toolchain. | ||
|
|
||
| NIST SP 800-218 PW.4.1 requires third-party software to come from expected, | ||
| trusted sources with integrity verification (Souppaya et al., 2022). The exact | ||
| `/usr/bin/llvm-cov-19` and `/usr/bin/llvm-profdata-19` bindings are | ||
| producer-selection controls: they select reviewed paths and `test -x` verifies | ||
| executability. They do not hash or signature-verify the Debian package or binary. | ||
| Package/image hashes, signatures, repository metadata, and attestations are | ||
| separate integrity controls and must not be inferred from path equality. | ||
|
Comment on lines
+23
to
+29
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win NIST control 인용을 수정하십시오. 현재 문구는 🤖 Prompt for AI Agents |
||
|
|
||
| ## Why the boundary exists | ||
|
|
||
| `cargo-llvm-cov` is a wrapper around Rust's LLVM source-based coverage and | ||
| explicitly supports `LLVM_COV` and `LLVM_PROFDATA` as path overrides. Its | ||
| current project documentation states that the LLVM tools must be compatible | ||
| with the LLVM version used by `rustc`. Allowing ambient `PATH` discovery would | ||
| therefore make a runner-image change capable of silently changing the coverage | ||
| producer. | ||
|
|
||
| Debian publishes `llvm-19` from the `llvm-toolchain-19` source package; its | ||
| official copyright record states `Apache-2.0 WITH LLVM-exception`. Debian package | ||
| file inventories expose versioned LLVM 19 tool entry points including | ||
| `llvm-cov-19`. Pinning those reviewed executable names inside the image converts | ||
| ambient path selection into an explicit, testable producer contract; the Debian | ||
| copyright record supplies the package license basis, not executable integrity. | ||
|
|
||
| ## Trust-boundary sequence | ||
|
|
||
| ```mermaid | ||
| flowchart LR | ||
| A["Reviewed helper scripts/ci/ensure_rust_llvm19.sh"] --> B["Default LLVM_COV_PATH / LLVM_PROFDATA_PATH"] | ||
| B --> C["Require live LLVM_COV and LLVM_PROFDATA equality"] | ||
| C --> D["Require both paths executable"] | ||
| D --> E["Fail closed before cargo llvm-cov"] | ||
| F["Hashed opencode-review-dispatch.yml"] --> G["Unchanged review-agent key blob"] | ||
| ``` | ||
|
|
||
| Each arrow is fail-closed. A later stage does not repair or broaden an earlier | ||
| stage's failed trust decision. | ||
|
|
||
| ## Security and supply-chain implications | ||
|
|
||
| The reviewed paths are fixed in trusted central workflow source. Pull-request | ||
| content cannot choose an LLVM package, executable path, download origin, or | ||
| runtime environment value. The existing coverage sandbox retains | ||
| `--network=none`, credential/Git isolation, exact-head/base materialization, | ||
| and the separately checksum-pinned `cargo-llvm-cov` archive. | ||
|
|
||
| This binding narrows reproducibility risk but does not by itself attest Debian's | ||
| whole package supply chain or prove a future Rust toolchain is compatible with | ||
| LLVM 19. A future rustc or base-image upgrade must revalidate compatibility and | ||
| update this contract, its tests, and CHANGELOG in one reviewed change rather | ||
| than silently selecting a different binary. | ||
|
|
||
| ## Failure and recovery | ||
|
|
||
| If the image cannot install `llvm-19`, either reviewed executable is missing or | ||
| non-executable, the runtime value differs from the literal reviewed path, or the | ||
| isolated runtime does not receive the values, Rust coverage fails closed before | ||
| `cargo llvm-cov` runs. The operator should identify whether the failure comes | ||
| from Debian package availability, the pinned image/base generation, a central | ||
| workflow regression, or an intentional Rust/LLVM compatibility change. | ||
|
|
||
| Do not work around the failure by removing the exact-value check, using an | ||
| unversioned executable, adding network access to the PR runtime, or accepting a | ||
| host-provided path. A deliberate toolchain migration requires fresh authoritative | ||
| compatibility evidence and the same RED→GREEN exact-head verification sequence. | ||
|
|
||
| ## Verification contract | ||
|
|
||
| `tests/test_opencode_rust_coverage_toolchain_contract.py` proves that: | ||
|
|
||
| 1. the helper defaults both reviewed LLVM 19 executable paths; | ||
| 2. the helper requires live `LLVM_COV` / `LLVM_PROFDATA` equality with those | ||
| paths; | ||
| 3. the helper requires both paths to be executable and exits `1` on mismatch; | ||
| 4. the helper does not mention unversioned `llvm-cov` / `llvm-profdata`; and | ||
| 5. every exact path named by the permanent quality workflow's | ||
| `pull_request.paths` filter resolves to a repository file, including the | ||
| helper, preventing a dangling documentation trigger from becoming | ||
| invisible debt. | ||
|
|
||
| The permanent quality workflow runs on Python 3.14, checks out the exact PR head, | ||
| executes the focused contract, compiles the test, and applies `git diff --check`. | ||
| Repository security and supply-chain workflows remain separate authorities. | ||
|
|
||
| ## References | ||
|
|
||
| Debian Project. (2026). *Package: llvm-19 (1:19.1.7-3~deb12u1), bookworm*. | ||
| Debian Packages. Retrieved August 10, 2026, from | ||
| https://packages.debian.org/bookworm/llvm-19 | ||
|
|
||
| Debian Project. (2026). *File list of package llvm-19*. Debian Packages. | ||
| Retrieved August 10, 2026, from | ||
| https://packages.debian.org/bookworm/amd64/llvm-19/filelist | ||
|
|
||
| Debian Project. (2026). *Copyright file for llvm-toolchain-19 19.1.7-20*. | ||
| Debian FTP Masters. Retrieved August 15, 2026, from | ||
| https://metadata.ftp-master.debian.org/changelogs/main/l/llvm-toolchain-19/llvm-toolchain-19_19.1.7-20_copyright | ||
|
|
||
| Souppaya, M., Scarfone, K., & Dodson, D. (2022). *Secure Software Development | ||
| Framework (SSDF) version 1.1: Recommendations for mitigating the risk of | ||
| software vulnerabilities* (NIST Special Publication 800-218). National | ||
| Institute of Standards and Technology. https://doi.org/10.6028/NIST.SP.800-218 | ||
|
|
||
| Taiki Endo. (2026). *cargo-llvm-cov: Cargo subcommand to use LLVM source-based | ||
| code coverage*. GitHub. Retrieved August 10, 2026, from | ||
| https://github.com/taiki-e/cargo-llvm-cov | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
일회성 복구 workflow가 재실행에 안전하지 않습니다. 공통 근본 원인은 하나입니다. 이 workflow는 자신이 push한 커밋으로 다시 트리거될 수 있고, 어떤 단계도 멱등하지 않습니다. 재실행 시 마커 불일치와 빈 커밋 때문에 job이 실패하며, PR 827에 영구적으로 실패한 체크가 남습니다.
.github/workflows/repair-pr827-coderabbit-comments.yml#L84-L100:git commit앞에git diff --cached --quiet검사를 추가하고, 변경이 없으면 성공으로 종료하십시오..github/workflows/repair-pr827-coderabbit-comments.yml#L3-L19: job 조건에 마지막 커밋 작성자가github-actions[bot]인 경우를 제외하는 가드를 추가하십시오.📍 Affects 1 file
.github/workflows/repair-pr827-coderabbit-comments.yml#L84-L100(this comment).github/workflows/repair-pr827-coderabbit-comments.yml#L3-L19🤖 Prompt for AI Agents