Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion WORKFLOW.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,7 @@ The required behaviors, organized by domain. Each is a **MUST**, stated as input
- **D1.3 Smoke never publishes and never uploads.** Input: `smoke: true`. Output: full compile/lint/test, but no registry/image push, no release, and **no** artifact uploads (every `upload-artifact`, including any aggregation job, is gated `!smoke`). *Prevents: a PR publishing, and orphaned artifacts churning the storage quota.*
- **D1.4 Workflow-file changes are not smoke-built.** Input: a PR changing only `.github/workflows/**`. Output: the paths-filter excludes workflow files, so smoke-build skips. *Implication: a workflow-only change is not smoke-built, but actionlint still validates it in CI.*
- **D1.5 One required aggregator gates merge.** Input: any PR. Output: a single aggregator job must **succeed**, `needs:` the changes job and the validation job, treat a **skipped** smoke build as pass, and **block** on `failure`/`cancelled`. Its name is ruleset-bound: the job `name:` and the ruleset `context:` are the same string and MUST be renamed together, never independently. *Prevents: a paths-filter error letting a target-changing PR merge unbuilt.*
- **D1.6 Coverage is reported to Codecov (C# and Python).** Input: a C# or Python repo's validation/test job. Output: tests run with coverage collection (`dotnet test --coverage --coverage-output-format cobertura --results-directory ./coverage` or `pytest --cov-report=xml`) and a `codecov/codecov-action` step uploads it, **best-effort** (`continue-on-error` and/or `fail_ci_if_error: false`, so a Codecov outage or an absent token never reds the gate). `CODECOV_TOKEN` lives in the repo's **actions** and **dependabot** secret stores, the second because a run triggered by a Dependabot PR reads the Dependabot store and the upload would otherwise skip silently on every bot PR, and the caller maps it to the reusable validator by name (`secrets:` with `CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}`), the way every hub task's declared secrets are passed. Required for **every** C# and Python repo that has tests. That C# invocation runs under **Microsoft.Testing.Platform**, which an MTP-based test project on the .NET 10 SDK and later requires, since running one through the VSTest target fails outright. A repo whose test project is MTP-based, in practice any repo on xunit.v3 4.0.0 or later, therefore also ships a root **`global.json`** declaring `{"test": {"runner": "Microsoft.Testing.Platform"}}`, references **`Microsoft.Testing.Extensions.CodeCoverage`** at **18.9.0 or later** in place of `coverlet.collector`, whose VSTest data collector MTP ignores without failing, and drops `xunit.runner.visualstudio`, the VSTest adapter MTP replaces. A repo whose test project is not yet MTP-based keeps the VSTest collector and its existing pin on the reusable validator, and that lagging state is a migration still owed rather than drift, until its own bump makes the project MTP-based and forces the move. The version floor is load-bearing rather than cautionary. Below 18.1.0 the extension is built against Microsoft.Testing.Platform 1.x, and an 18.0.x resolution, which is what a `>= 18.0.0` range picks, throws a `TypeLoadException` against the 2.x platform xunit.v3 4.0.0 carries, runs zero tests, and still writes a well-formed Cobertura file reporting full coverage, so only the non-zero exit says the run reported nothing. 18.9.0 is the first release on Microsoft.Testing.Platform 2.3.x, where every test project writes into the one shared `--results-directory` the invocation names rather than resolving that relative path per project. Two details of the invocation are equally load-bearing, and neither failure reds the job on its own. `--coverage-output` stays unset, because pinning one filename gives every test project in the solution the same path and a repo with more than one then keeps only whichever ran last. Leaving it unset produces the default name `<guid>.cobertura.xml`, which `codecov-cli`'s own file finder does not match. The validator therefore prefixes each report to `coverage-<guid>.cobertura.xml` before the upload step reads the directory. Where this guarantee does not apply (a `lint-only` profile for that type, per the hub's `registry/repos.json`), the hub's `spec/secrets.json` `typeMechanisms` mapping is not claimed for that repo, and the absence is not drift. The repo also ships a **`codecov.yml`** that sets the project and patch statuses to **`informational: true`** so a coverage delta never gates a PR (a distinct knob from `fail_ci_if_error`, which only guards the upload step), and excludes intentionally-untested, non-shipped code (an example/demo or benchmark project) from the coverage denominator via `ignore`. A repo may override this to enforce a coverage threshold where its quality bar requires it. Coverage output is a build artifact, so `.gitignore` excludes it (e.g. `coverage/` and `*.cobertura.xml`, with `.gitignore` the full source of truth) so a blanket `git add -A` won't stage the untracked output. *Prevents: coverage silently going unreported, a test project stranded on a runner the current SDK refuses, a stale and unused token, a coverage regression blocking an unrelated PR, and a coverage artifact committed by a blanket add.*
- **D1.6 Coverage is reported to Codecov (C# and Python).** Input: a C# or Python repo that has tests for that type. Output: the validation job runs those tests under coverage collection (`dotnet test --coverage --coverage-output-format cobertura --results-directory ./coverage`, leaving `--coverage-output` unset so each test project writes its own report rather than overwriting a shared one, or `pytest --cov-report=xml` over a repo whose own `pyproject.toml` selects what to measure) and a `codecov/codecov-action` step uploads the report, **best-effort** (`continue-on-error` and/or `fail_ci_if_error: false`, so a Codecov outage or an absent token never reds the gate). The Python leg **fails its test step when no report was written**, since `--cov-report=xml` alone selects nothing to measure. The C# leg renames each report to `coverage-<guid>.cobertura.xml` before the upload step reads the directory, `codecov-cli`'s own finder not matching the default name, and a repo owning its validator rather than calling the hub's owes that rename itself. `CODECOV_TOKEN` lives in the repo's **actions** and **dependabot** secret stores, the second because a run triggered by a Dependabot pull request reads the Dependabot store and the upload would otherwise skip silently on every bot pull request. A caller reaching the reusable validator across repositories names the secret it passes (`secrets:` with `CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}`), on its pull request path and its publisher path alike, because `secrets: inherit` is documented for a caller in the same organization or enterprise, which a personal account is not. A call by local path stays inside one repository, where the caller's own store is the one the callee reads, so `secrets: inherit` is available there instead of naming each secret. The repo ships a **`codecov.yml`** setting the project and patch statuses to **`informational: true`** so a coverage delta never gates a pull request (a repo whose quality bar requires a threshold may turn that off), and excluding intentionally-untested, non-shipped code (an example or benchmark project) from the denominator via `ignore`. Coverage output is a build artifact, so `.gitignore` excludes it. The C# invocation runs under **Microsoft.Testing.Platform**, and the runner declaration, package references, and version floor an MTP-based test project needs are `CODESTYLE.md`'s .NET side. The Python invocation needs **`pytest-cov`** and a coverage selector, which are `CODESTYLE.md`'s Python side. N/A for a repo carrying no tests for that type, and for a `lint-only` profile for it (per the hub's `registry/repos.json`). *Prevents: coverage silently going unreported, and a coverage regression blocking an unrelated pull request.*
Comment thread
ptr727 marked this conversation as resolved.

### D2 - Input/State Validation at Entry

Expand Down
Loading