diff --git a/.github/actions/codegen/action.yml b/.github/actions/codegen/action.yml new file mode 100644 index 0000000..3c91252 --- /dev/null +++ b/.github/actions/codegen/action.yml @@ -0,0 +1,14 @@ +# The codegen hook the hub's run-codegen-pull-request-task.yml runs on each of main and develop. +# The hub task owns the .NET setup, the CSharpier format, and the pull request, so this carries only the generator invocation. +name: NxWitness codegen hook +description: Refresh Make/Version.json and Make/Matrix.json from the upstream product releases. + +runs: + using: composite + steps: + - name: Run CreateMatrix codegen step + shell: bash + run: | + set -Eeuo pipefail + dotnet run --project ./CreateMatrix/CreateMatrix.csproj -- \ + matrix --versionpath=./Make/Version.json --matrixpath=./Make/Matrix.json --updateversion diff --git a/.github/actions/docker-build-base/action.yml b/.github/actions/docker-build-base/action.yml new file mode 100644 index 0000000..43be6ab --- /dev/null +++ b/.github/actions/docker-build-base/action.yml @@ -0,0 +1,78 @@ +# The docker-build-base hook the hub's build-docker-task.yml runs before the product images when build-base is set. +# It builds the two shared base images every product Dockerfile starts from, nx-base and nx-base-lsio. +# The base carries one branch-agnostic tag, so only a main publish pushes it, and a develop publish reuses main's. +# The publisher's own build-base job runs it for that push, and the smoke build reaches it through the hub without one. +# No branch reaches the hook, so push stands in for it: a push is a main publish, built multi-arch with a main cache. +# Anything else is a smoke build, amd64 only and never pushed. +name: NxWitness docker-build-base hook +description: Build and optionally push the shared nx-base and nx-base-lsio images. + +inputs: + push: + description: Push the base images, which only a main publish sets. + required: true + ref: + description: Accepted because the hub passes it, and unused, since the hub job has already checked it out. + required: false + default: '' + +runs: + using: composite + steps: + - name: Select platforms step + id: platforms + shell: bash + env: + PUSH: ${{ inputs.push }} + run: | + set -Eeuo pipefail + if [[ "$PUSH" == "true" ]]; then + echo "platforms=linux/amd64,linux/arm64" >> "$GITHUB_OUTPUT" + else + echo "platforms=linux/amd64" >> "$GITHUB_OUTPUT" + fi + + # An arm64 build is non-native on the amd64 runner, so its QEMU emulator is installed only when the build includes it. + - name: Setup QEMU step + if: ${{ contains(steps.platforms.outputs.platforms, 'arm64') }} + uses: docker/setup-qemu-action@99012661954931238ded8c8b007157a8430204e1 # v4.4.0 + with: + platforms: arm64 + + - name: Setup Buildx step + uses: docker/setup-buildx-action@594f3bf4285d9ea8dc53c9a0c9c4092420091003 # v4.4.0 + with: + platforms: ${{ steps.platforms.outputs.platforms }} + + # The hook holds no login, since a composite action cannot read secrets, so the job running it with push set logs in first. + # The hub's own docker-build-base job carries no login, which is why only the smoke build reaches this hook through the hub. + # The inline cache on the pushed tag is what the first cache-from entry reads back on the next publish. + - name: Build and push nx-base step + uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7.4.0 + with: + push: ${{ inputs.push == 'true' }} + context: Docker + file: Docker/NxBase.Dockerfile + platforms: ${{ steps.platforms.outputs.platforms }} + tags: docker.io/ptr727/nx-base:ubuntu-noble + cache-from: | + type=registry,ref=docker.io/ptr727/nx-base:ubuntu-noble + type=registry,ref=docker.io/ptr727/nx-base:buildcache-main + cache-to: | + ${{ inputs.push == 'true' && 'type=registry,ref=docker.io/ptr727/nx-base:buildcache-main,mode=max,ignore-error=true' || '' }} + ${{ inputs.push == 'true' && 'type=inline' || '' }} + + - name: Build and push nx-base-lsio step + uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7.4.0 + with: + push: ${{ inputs.push == 'true' }} + context: Docker + file: Docker/NxBase-LSIO.Dockerfile + platforms: ${{ steps.platforms.outputs.platforms }} + tags: docker.io/ptr727/nx-base-lsio:ubuntu-noble + cache-from: | + type=registry,ref=docker.io/ptr727/nx-base-lsio:ubuntu-noble + type=registry,ref=docker.io/ptr727/nx-base-lsio:buildcache-main + cache-to: | + ${{ inputs.push == 'true' && 'type=registry,ref=docker.io/ptr727/nx-base-lsio:buildcache-main,mode=max,ignore-error=true' || '' }} + ${{ inputs.push == 'true' && 'type=inline' || '' }} diff --git a/.github/actions/docker-prepare/action.yml b/.github/actions/docker-prepare/action.yml new file mode 100644 index 0000000..fc6b0a6 --- /dev/null +++ b/.github/actions/docker-prepare/action.yml @@ -0,0 +1,56 @@ +# The docker-prepare hook the hub's build-docker-task.yml runs in place of its single-image default. +# It turns the Make/Matrix.json rows for one branch into the hub's {name, tags, build-args, context, dockerfile, cache-repo} matrix shape. +# The hub core adds LABEL_VERSION from semver2 itself, so the entries carry only the product tags and download args codegen wrote. +name: NxWitness docker-prepare hook +description: Emit the build-docker-task.yml matrix from the Make/Matrix.json rows of one branch. + +inputs: + # This hook reads image as an optional filter rather than as a Docker Hub repository. + # The smoke build names the one Ubuntu and one LSIO product it builds, and a publish leaves it empty to build every row. + image: + description: Comma-separated Make/Matrix.json image names to keep, one row per name, or empty for every row of the branch. + required: false + default: '' + branch: + description: The Make/Matrix.json Branch value whose rows to build. + required: true + semver2: + description: Accepted because the hub passes it, and unused, since the hub core adds LABEL_VERSION itself. + required: false + default: '' + +outputs: + matrix: + description: Compact JSON array of {name, tags, build-args, context, dockerfile, cache-repo} entries. + value: ${{ steps.emit.outputs.matrix }} + +runs: + using: composite + steps: + - name: Emit Make/Matrix.json matrix step + id: emit + shell: bash + env: + NAMES: ${{ inputs.image }} + BRANCH: ${{ inputs.branch }} + run: | + set -Eeuo pipefail + # The cache repository is the first tag without its :tag suffix, the image's own Docker Hub repository. + # A name filter keeps one row per name, since the rows of one product differ only in the upstream version they pin. + # shellcheck disable=SC2016 # $b and $n are jq variables, not shell expansions + matrix=$(jq --compact-output --arg b "$BRANCH" --arg n "$NAMES" ' + [.Images[] | select(.Branch == $b)] + | if $n == "" then . else map(select(.Name as $name | $n | split(",") | index($name))) | unique_by(.Name) end + | map({ + name: .Name, + tags: .Tags, + "build-args": .Args, + context: "Docker", + dockerfile: "Docker/\(.Name).Dockerfile", + "cache-repo": (.Tags[0] | sub(":[^:]*$"; "")) + })' ./Make/Matrix.json) + if [[ "$(jq 'length' <<<"$matrix")" == "0" ]]; then + echo "::error::Make/Matrix.json holds no rows for branch '$BRANCH' and names '$NAMES'." + exit 1 + fi + echo "matrix=$matrix" >> "$GITHUB_OUTPUT" diff --git a/.github/dependabot.yml b/.github/dependabot.yml index 3eb8f80..0ea58c4 100644 --- a/.github/dependabot.yml +++ b/.github/dependabot.yml @@ -15,18 +15,18 @@ # via the next develop -> main release, those channels would ship # outdated code in the interim. # - Codegen workflows take the same dual-target shape for the same -# reason - see .github/workflows/run-codegen-pull-request-task.yml. +# reason - see .github/workflows/run-periodic-codegen-pull-request.yml. # -# The merge-bot's `case` statement in -# .github/workflows/merge-bot-pull-request.yml dispatches the merge -# method per base ref (squash on develop, merge on main) so both bases +# The hub merge-bot task that +# .github/workflows/merge-bot-pull-request.yml calls dispatches the +# merge method per base ref (squash on develop, merge on main) so both bases # auto-merge cleanly. `develop` remains strictly forward-only: there # are no main -> develop back-merges; each branch absorbs its own # Dependabot PRs and codegen PRs independently. # # Security update PRs (CVE-driven) are opened by Dependabot against # the repo default branch (`main`) regardless of any `target-branch` -# config - the `case` statement handles them in the same code path. +# config - the merge-bot handles them in the same code path. version: 2 updates: diff --git a/.github/workflows/build-base-images-task.yml b/.github/workflows/build-base-images-task.yml deleted file mode 100644 index 3c57d4c..0000000 --- a/.github/workflows/build-base-images-task.yml +++ /dev/null @@ -1,89 +0,0 @@ -name: Build base images task - -on: - workflow_call: - inputs: - push: - required: false - type: boolean - default: true - platforms: - required: false - type: string - default: linux/amd64,linux/arm64 - # Branch to check out. The publisher passes main so the shared - # nx-base tag is always built from the release branch regardless of - # the dispatch ref. Empty falls back to the triggering ref. - ref: - required: false - type: string - default: '' - -jobs: - - build-base: - name: Build base image job - runs-on: ubuntu-latest - - strategy: - matrix: - base: - - name: nx-base - dockerfile: Docker/NxBase.Dockerfile - cache_tag: docker.io/ptr727/nx-base:ubuntu-noble - tags: | - docker.io/ptr727/nx-base:ubuntu-noble - - name: nx-base-lsio - dockerfile: Docker/NxBase-LSIO.Dockerfile - cache_tag: docker.io/ptr727/nx-base-lsio:ubuntu-noble - tags: | - docker.io/ptr727/nx-base-lsio:ubuntu-noble - - steps: - - - name: Checkout step - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - ref: ${{ inputs.ref || github.ref }} - - # arm64 is non-native on the amd64 runner, so install its QEMU emulator only when the build includes it. - - name: Setup QEMU step - if: ${{ contains(inputs.platforms, 'arm64') }} - uses: docker/setup-qemu-action@99012661954931238ded8c8b007157a8430204e1 # v4.4.0 - with: - platforms: arm64 - - - name: Setup Buildx step - uses: docker/setup-buildx-action@594f3bf4285d9ea8dc53c9a0c9c4092420091003 # v4.4.0 - with: - platforms: ${{ inputs.platforms }} - - # Log in on every build (including smoke), for higher pull/cache rate limits against the branch-scoped - # registry buildcache. DOCKER_HUB_USERNAME / DOCKER_HUB_ACCESS_TOKEN must be in both the Actions and - # Dependabot secret stores so a Dependabot-triggered push CI run can log in too. Forks cannot push here. - - name: Login to Docker Hub step - uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0 - with: - registry: docker.io - username: ${{ secrets.DOCKER_HUB_USERNAME }} - password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }} - - - name: Docker build and push base step - uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7.4.0 - with: - # Push is controlled by the caller, not the trigger ref: the - # publisher passes push: true with ref: main, so it must publish - # the shared base tag even when dispatched from another branch. - # The registry buildcache tag keys on the built ref, not github.ref_name. - push: ${{ inputs.push }} - context: Docker - file: ${{ matrix.base.dockerfile }} - platforms: ${{ inputs.platforms }} - tags: ${{ matrix.base.tags }} - cache-from: | - type=registry,ref=${{ matrix.base.cache_tag }} - type=registry,ref=docker.io/ptr727/${{ matrix.base.name }}:buildcache-develop - type=registry,ref=docker.io/ptr727/${{ matrix.base.name }}:buildcache-main - cache-to: | - ${{ inputs.push && format('type=registry,ref=docker.io/ptr727/{0}:buildcache-{1},mode=max,ignore-error=true', matrix.base.name, inputs.ref != '' && inputs.ref || github.ref_name) || '' }} - ${{ inputs.push && 'type=inline' || '' }} diff --git a/.github/workflows/build-docker-task.yml b/.github/workflows/build-docker-task.yml deleted file mode 100644 index 615fac3..0000000 --- a/.github/workflows/build-docker-task.yml +++ /dev/null @@ -1,186 +0,0 @@ -name: Build Docker image task - - -on: - workflow_call: - inputs: - push: - required: false - type: boolean - default: false - # Smoke mode: build a single Ubuntu + single LSIO variant on amd64 - # only, never push. Used for fast PR feedback in place of the full - # matrix. See test-pull-request.yml. - smoke: - required: false - type: boolean - default: false - # Logical branch whose Matrix.json rows to build/push and whose - # registry buildcache tag to use. Decoupled from `ref` so `ref` can be pinned to an - # immutable commit (e.g. the publisher pins main to the versioned SHA) - # while this still selects the right branch's rows. Empty in smoke mode - # builds both branches' rows. The publisher passes main and develop so - # both branches' tags are produced from one scheduled run. Empty in a - # non-smoke build falls back to github.ref_name for the buildcache tag. - branch: - required: false - type: string - default: '' - # Immutable git ref to check out / version, decoupled from branch/tag - # selection (see `branch`). Empty uses the triggering ref (PR context). - # The publisher pins this to the exact versioned commit so the image's - # embedded version matches the GitHub release tag even if the branch - # advances mid-run. - ref: - required: false - type: string - default: '' - # When false the caller is expected to have built the base images - # already (the publisher builds them once, see publish-release.yml), - # so the internal base build is skipped. - build_base: - required: false - type: boolean - default: true - # SemVer2 threaded from the orchestrator's single NBGV run (get-version-task), baked into the image as - # LABEL_VERSION. NBGV is never re-run here, so one classification feeds every product leg (avoids a - # second NBGV run reclassifying and producing a wrong/colliding tag). Smoke callers omit it and bake - # the placeholder default below (smoke images are never pushed, so the label is informational only). - semver2: - required: false - type: string - default: '0.0.0-smoke' - -jobs: - - get-matrix: - name: Get matrix job - runs-on: ubuntu-latest - outputs: - matrix: ${{ steps.getmatrix.outputs.matrix }} - - steps: - - - name: Checkout step - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - ref: ${{ inputs.ref || github.ref }} - - - name: Load matrix.json step - id: getmatrix - env: - SMOKE: ${{ inputs.smoke }} - BRANCH: ${{ inputs.branch }} - run: | - # $b below is a jq variable (passed via --arg), not a shell - # variable, so it must stay single-quoted / unexpanded. - # shellcheck disable=SC2016 - if [[ "$SMOKE" == "true" ]]; then - # One Ubuntu + one LSIO variant exercises the shared Dockerfile - # build logic and the branch's build args. Tags are irrelevant - # since smoke never pushes. Restrict to the PR base branch ($b) - # so a PR onto develop validates the develop rows and a PR onto - # main validates the main rows. Empty $b builds both branches. - FILTER='.Images |= (map(select((.Name == "NxMeta" or .Name == "NxMeta-LSIO") and ($b == "" or .Branch == $b))) | unique_by([.Name, .Branch]))' - elif [[ -n "$BRANCH" ]]; then - # Publish: build only the rows targeting the branch being built - # (avoids building the other branch's rows just to discard them). - FILTER='.Images |= map(select(.Branch == $b))' - else - FILTER='.' - fi - echo "matrix=$(jq --arg b "$BRANCH" --compact-output "$FILTER" ./Make/Matrix.json)" >> "$GITHUB_OUTPUT" - - build-base: - name: Build base image job - if: ${{ inputs.build_base }} - uses: ./.github/workflows/build-base-images-task.yml - with: - push: ${{ inputs.push }} - platforms: ${{ (!inputs.smoke && inputs.branch == 'main') && 'linux/amd64,linux/arm64' || 'linux/amd64' }} - ref: ${{ inputs.ref }} - secrets: inherit - - build-docker: - name: Build Docker image job - runs-on: ubuntu-latest - needs: [get-matrix, build-base] - # always() so the job still runs when build-base is intentionally - # skipped (build_base == false). Fail only if a prerequisite failed. - if: >- - ${{ always() - && needs.get-matrix.result == 'success' - && (needs.build-base.result == 'success' || needs.build-base.result == 'skipped') }} - - strategy: - max-parallel: 4 - matrix: - images: ${{ fromJson(needs.get-matrix.outputs.matrix).Images }} - - env: - # Multi-arch (amd64+arm64) only on a full main publish. Smoke and develop build amd64 only. - PLATFORMS: ${{ (!inputs.smoke && inputs.branch == 'main') && 'linux/amd64,linux/arm64' || 'linux/amd64' }} - - steps: - - - name: Checkout step - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - ref: ${{ inputs.ref || github.ref }} - - # arm64 is non-native on the amd64 runner, so install its QEMU emulator only when the build includes it. - - name: Setup QEMU step - if: ${{ contains(env.PLATFORMS, 'arm64') }} - uses: docker/setup-qemu-action@99012661954931238ded8c8b007157a8430204e1 # v4.4.0 - with: - platforms: arm64 - - - name: Setup Buildx step - uses: docker/setup-buildx-action@594f3bf4285d9ea8dc53c9a0c9c4092420091003 # v4.4.0 - with: - platforms: ${{ env.PLATFORMS }} - - # Log in on every build (including smoke), for higher pull/cache rate limits against the branch-scoped - # registry buildcache. DOCKER_HUB_USERNAME / DOCKER_HUB_ACCESS_TOKEN must be in both the Actions and - # Dependabot secret stores so a Dependabot-triggered push CI run can log in too. Forks cannot push here. - - name: Login to Docker Hub step - uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0 - with: - registry: docker.io - username: ${{ secrets.DOCKER_HUB_USERNAME }} - password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }} - - - name: Create tags and args step - id: tagsargs - run: | - set -Eeuo pipefail - TAGS=$(jq -r '.[]' <<< '${{ toJson(matrix.images.Tags) }}') - ARGS=$(jq -r '.[]' <<< '${{ toJson(matrix.images.Args) }}') - CACHE_REPO=$(jq -r '.[0]' <<< '${{ toJson(matrix.images.Tags) }}' | sed 's/:[^:]*$//') - { - echo "tags<> "$GITHUB_OUTPUT" - - - name: Docker build and push step - uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7.4.0 - with: - # Matrix is already filtered to the target branch, so push when asked. - # Smoke callers pass push:false. - push: ${{ inputs.push }} - context: Docker - file: Docker/${{ matrix.images.Name }}.Dockerfile - platforms: ${{ env.PLATFORMS }} - tags: ${{ steps.tagsargs.outputs.tags }} - build-args: ${{ steps.tagsargs.outputs.args }} - cache-from: | - type=registry,ref=${{ steps.tagsargs.outputs.cache_repo }}:buildcache-develop - type=registry,ref=${{ steps.tagsargs.outputs.cache_repo }}:buildcache-main - cache-to: | - ${{ inputs.push && format('type=registry,ref={0}:buildcache-{1},mode=min,ignore-error=true', steps.tagsargs.outputs.cache_repo, inputs.branch != '' && inputs.branch || github.ref_name) || '' }} diff --git a/.github/workflows/get-version-task.yml b/.github/workflows/get-version-task.yml deleted file mode 100644 index 09a6a44..0000000 --- a/.github/workflows/get-version-task.yml +++ /dev/null @@ -1,54 +0,0 @@ -name: Get version information task - -on: - workflow_call: - inputs: - # Git ref to check out and version. Empty falls back to the caller's default checkout ref (`github.ref`). The - # publisher passes an explicit branch so a scheduled run can still compute versions for `develop`. - ref: - required: false - type: string - default: '' - outputs: - SemVer2: - value: ${{ jobs.get-version.outputs.SemVer2 }} - AssemblyVersion: - value: ${{ jobs.get-version.outputs.AssemblyVersion }} - AssemblyFileVersion: - value: ${{ jobs.get-version.outputs.AssemblyFileVersion }} - AssemblyInformationalVersion: - value: ${{ jobs.get-version.outputs.AssemblyInformationalVersion }} - # Full SHA of the commit the version was computed from, used to pin the release tag to the exact built commit. - GitCommitId: - value: ${{ jobs.get-version.outputs.GitCommitId }} - -jobs: - - get-version: - name: Get version information job - runs-on: ubuntu-latest - outputs: - SemVer2: ${{ steps.nbgv.outputs.SemVer2 }} - AssemblyVersion: ${{ steps.nbgv.outputs.AssemblyVersion }} - AssemblyFileVersion: ${{ steps.nbgv.outputs.AssemblyFileVersion }} - AssemblyInformationalVersion: ${{ steps.nbgv.outputs.AssemblyInformationalVersion }} - GitCommitId: ${{ steps.nbgv.outputs.GitCommitId }} - - steps: - - - name: Setup .NET SDK step - uses: actions/setup-dotnet@a98b56852c35b8e3190ac28c8c2271da59106c68 # v6.0.0 - with: - dotnet-version: 10.x - - - name: Checkout code step - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - ref: ${{ inputs.ref }} - fetch-depth: 0 - - # nbgv is floated on @master: its tag stream lags master, so Dependabot tag-tracking would propose a downgrade. - # Revisit if dotnet/nbgv resumes regular tagged releases. - - name: Run Nerdbank.GitVersioning tool step - id: nbgv - uses: dotnet/nbgv@master diff --git a/.github/workflows/merge-bot-pull-request.yml b/.github/workflows/merge-bot-pull-request.yml index d8cb6cc..f9ec61c 100644 --- a/.github/workflows/merge-bot-pull-request.yml +++ b/.github/workflows/merge-bot-pull-request.yml @@ -1,146 +1,29 @@ name: Merge bot pull request action -# Enable auto-merge once per PR on opened/reopened. Disable it when a maintainer pushes to a bot branch. Merge -# method by base branch (develop = squash, main = merge). App token so the merge fires downstream workflows -# (GITHUB_TOKEN pushes don't) and so the disable job has write access on read-only Dependabot PRs. - -# `pull_request_target` (not `pull_request`): these jobs hold the App private key, so the workflow definition and -# its action SHAs must resolve from the trusted base branch, not the PR head. Safe because no job checks out PR -# code - each only runs `gh pr merge` against the PR by URL. +# Thin caller: the merge-bot is the hub's reusable merge-bot-task.yml, which every fleet repo reaches rather than carries. +# The trigger is pull_request_target so the called workflow resolves from the trusted base rather than the PR head, and no job checks out PR code. on: pull_request_target: types: [opened, reopened, synchronize] -# Per-PR group: under `pull_request_target` `github.ref` is the base branch, which would serialize every bot PR -# against that base. Key on the PR number so each PR's events queue independently. `cancel-in-progress: false` so a -# follow-up synchronize doesn't cancel an in-flight `opened` run before it enables auto-merge. +# Concurrency keys on the PR number rather than on github.ref, which under pull_request_target is the base branch and would serialize every bot PR against it, so each PR queues independently. +# The cancel-in-progress setting is false so a follow-up synchronize does not cancel an in-flight opened run before it enables auto-merge. concurrency: group: ${{ github.workflow }}-${{ github.event.pull_request.number }} cancel-in-progress: false -jobs: - - merge-dependabot: - name: Merge dependabot pull request job - runs-on: ubuntu-latest - # Dependabot PRs from this repo (not forks). Only on opened/reopened so the disable job stays sticky. - if: >- - (github.event.action == 'opened' || github.event.action == 'reopened') && - github.event.pull_request.user.login == 'dependabot[bot]' && - github.event.pull_request.head.repo.full_name == github.repository - permissions: - contents: write - pull-requests: write - - steps: - - - name: Generate GitHub App token step - id: app-token - uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0 - with: - client-id: ${{ secrets.CODEGEN_APP_CLIENT_ID }} - private-key: ${{ secrets.CODEGEN_APP_PRIVATE_KEY }} - - - name: Get dependabot metadata step - id: metadata - uses: dependabot/fetch-metadata@25dd0e34f4fe68f24cc83900b1fe3fe149efef98 # v3.1.0 - with: - github-token: "${{ secrets.GITHUB_TOKEN }}" - - # Skip semver-major NuGet bumps so they land via human review. Other ecosystems' majors auto-merge. - - name: Merge pull request step - if: >- - (steps.metadata.outputs.package-ecosystem != 'nuget') || - (steps.metadata.outputs.update-type != 'version-update:semver-major') - run: | - set -Eeuo pipefail - case "${{ github.event.pull_request.base.ref }}" in - develop) method=--squash ;; - main) method=--merge ;; - *) - echo "::error::Unsupported base branch: ${{ github.event.pull_request.base.ref }}" - exit 1 - ;; - esac - # --delete-branch removes the merged bot branch (the repo-wide auto-delete setting stays off so a - # develop -> main promotion never deletes develop). - gh pr merge --auto --delete-branch "$method" "$PR_URL" - env: - PR_URL: ${{ github.event.pull_request.html_url }} - GH_TOKEN: ${{ steps.app-token.outputs.token }} +# Every write in the called workflow uses the App token, so GITHUB_TOKEN gets no scope. +permissions: {} - merge-codegen: - name: Merge codegen pull request job - runs-on: ubuntu-latest - # Codegen PRs from this repo. Head/base pairing is enforced strictly (codegen-main->main, codegen-develop-> - # develop). Only on opened/reopened so the disable job stays sticky. - if: >- - (github.event.action == 'opened' || github.event.action == 'reopened') && - github.event.pull_request.user.login == 'ptr727-codegen[bot]' && - github.event.pull_request.head.repo.full_name == github.repository && - ( - (github.event.pull_request.head.ref == 'codegen-main' && github.event.pull_request.base.ref == 'main') || - (github.event.pull_request.head.ref == 'codegen-develop' && github.event.pull_request.base.ref == 'develop') - ) - permissions: - contents: write - pull-requests: write - - steps: - - - name: Generate GitHub App token step - id: app-token - uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0 - with: - client-id: ${{ secrets.CODEGEN_APP_CLIENT_ID }} - private-key: ${{ secrets.CODEGEN_APP_PRIVATE_KEY }} - - - name: Merge pull request step - run: | - set -Eeuo pipefail - case "${{ github.event.pull_request.base.ref }}" in - develop) method=--squash ;; - main) method=--merge ;; - *) - echo "::error::Unsupported base branch: ${{ github.event.pull_request.base.ref }}" - exit 1 - ;; - esac - # --delete-branch removes the merged codegen branch (the repo-wide auto-delete setting stays off so a - # develop -> main promotion never deletes develop). - gh pr merge --auto --delete-branch "$method" "$PR_URL" - env: - PR_URL: ${{ github.event.pull_request.html_url }} - GH_TOKEN: ${{ steps.app-token.outputs.token }} - - disable-auto-merge-on-maintainer-push: - name: Disable auto-merge on maintainer push job - runs-on: ubuntu-latest - # Fires when a maintainer pushes to a bot's branch (synchronize, actor != bot). Disables auto-merge so the - # maintainer's commits don't merge with the bot's. They re-enable it manually. The disable call is idempotent. - if: >- - github.event.action == 'synchronize' && - github.event.pull_request.head.repo.full_name == github.repository && - ( - github.event.pull_request.user.login == 'dependabot[bot]' || - github.event.pull_request.user.login == 'ptr727-codegen[bot]' - ) && - github.actor != github.event.pull_request.user.login - permissions: - pull-requests: write - - steps: - - - name: Generate GitHub App token step - # App token because a Dependabot PR's GITHUB_TOKEN is read-only regardless of who triggered the event. - id: app-token - uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0 - with: - client-id: ${{ secrets.CODEGEN_APP_CLIENT_ID }} - private-key: ${{ secrets.CODEGEN_APP_PRIVATE_KEY }} +jobs: - - name: Disable auto-merge step - run: gh pr merge --disable-auto "$PR_URL" - env: - PR_URL: ${{ github.event.pull_request.html_url }} - GH_TOKEN: ${{ steps.app-token.outputs.token }} + merge-bot: + name: Merge bot pull request job + uses: ptr727/ProjectTemplate/.github/workflows/merge-bot-task.yml@456694681ae76ff162a9bd784b5376901957feae # 2.0.685 + secrets: + CODEGEN_APP_CLIENT_ID: ${{ secrets.CODEGEN_APP_CLIENT_ID }} + CODEGEN_APP_PRIVATE_KEY: ${{ secrets.CODEGEN_APP_PRIVATE_KEY }} + # A repo with a tracker outside the built-in codegen and upstream-version pairs adds a with: block carrying a rules JSON array of head or head-prefix plus base. + # A repo that keeps the repository-wide auto-delete off and still wants bot branches gone sets delete-branch true in the same block. + with: + delete-branch: true diff --git a/.github/workflows/publish-plan-task.yml b/.github/workflows/publish-plan-task.yml deleted file mode 100644 index ee1c4fa..0000000 --- a/.github/workflows/publish-plan-task.yml +++ /dev/null @@ -1,84 +0,0 @@ -name: Publish plan task - -# Single source of truth for the release-gate decision, reused by every publish-release.yml job so the policy -# lives here, not scattered across job `if:` conditions. A human PR merge never auto-publishes. A release is a -# deliberate dispatch, a bot (Dependabot/codegen) code-merge to main, or the Docker weekly schedule. -# -# Outputs: -# publish - 'true' when this run should publish: a bot-authored push (the codegen App merges every bot PR, so -# its identity - or dependabot[bot] - is the gate), a schedule, or a workflow_dispatch of main/develop. -# A human push (a merge/promotion to main) or a dispatch from any other branch is 'false'. -# stable - 'true' when the target branch is main (stable channel). Main-only jobs gate on publish && stable. -# Both outputs are the strings 'true'/'false' - gate with == 'true'. A bare `if: ${{ needs.plan.outputs.publish }}` -# is always truthy (a non-empty string is truthy in an Actions expression). -# -# Shared across repo types: a library/package repo triggers only push + dispatch and uses `publish`. A -# Docker/wrapper repo also triggers the weekly schedule and gates main-only jobs on `stable`. A case a given -# caller never triggers (e.g. schedule for a library) is simply inert for it - expected of a single-source task. - -on: - workflow_call: - inputs: - event_name: - description: The triggering event (github.event_name). - required: true - type: string - actor: - description: The actor that triggered the run (github.actor). - required: true - type: string - ref_name: - description: The short ref name (github.ref_name). - required: true - type: string - outputs: - publish: - description: "'true' when this run should publish." - value: ${{ jobs.plan.outputs.publish }} - stable: - description: "'true' when the target branch is main (stable channel)." - value: ${{ jobs.plan.outputs.stable }} - -jobs: - - plan: - name: Plan release job - runs-on: ubuntu-latest - outputs: - publish: ${{ steps.decide.outputs.publish }} - stable: ${{ steps.decide.outputs.stable }} - - steps: - - - name: Decide release plan step - id: decide - env: - EVENT: ${{ inputs.event_name }} - ACTOR: ${{ inputs.actor }} - REF: ${{ inputs.ref_name }} - run: | - set -Eeuo pipefail - publish=false - case "$EVENT" in - workflow_dispatch) - # A human release: only the long-lived branches publish (a stray feature-branch dispatch is a no-op). - [[ "$REF" == "main" || "$REF" == "develop" ]] && publish=true - ;; - schedule) - # Docker weekly refresh (main-only by schedule config). - publish=true - ;; - push) - # A human merge never auto-publishes. Only a bot merge to main does. The codegen App merges every - # Dependabot/codegen PR, so github.actor is its identity (dependabot[bot] allowed defensively). The - # ref==main guard keeps the task self-contained even if a caller's push trigger is not main-only. - if [[ "$REF" == "main" ]] && { [[ "$ACTOR" == "ptr727-codegen[bot]" ]] || [[ "$ACTOR" == "dependabot[bot]" ]]; }; then - publish=true - fi - ;; - esac - stable=false - [[ "$REF" == "main" ]] && stable=true - echo "publish=$publish" >> "$GITHUB_OUTPUT" - echo "stable=$stable" >> "$GITHUB_OUTPUT" - echo "Release plan: event=$EVENT actor=$ACTOR ref=$REF -> publish=$publish stable=$stable" diff --git a/.github/workflows/publish-release.yml b/.github/workflows/publish-release.yml index ca5f7ba..a8e612d 100644 --- a/.github/workflows/publish-release.yml +++ b/.github/workflows/publish-release.yml @@ -1,239 +1,133 @@ name: Publish project release action -# Triggered-Docker publisher, one run = one branch (the trigger ref). The weekly schedule rebuilds `main` -# only (full product matrix + shared base refresh for CVEs + a versioned GitHub release). A path-scoped push -# publishes `main` immediately when codegen commits a new `Make/Matrix.json` pin, so new upstream Nx product -# versions ship at once. A manual dispatch publishes the branch it is started from (main => latest/stable, -# develop => :develop). Building only the trigger branch keeps github.ref aligned with the branch being -# versioned, so NBGV classifies it natively with no cross-branch ref leak. +# Thin caller onto the hub's reusable plan, validate, build-release, and Docker Hub readme tasks, per ptr727/ProjectTemplate docs/reusable-workflows.md "Adopting the Release Chain". +# One run publishes one branch, the trigger ref, so github.ref stays aligned with the branch NBGV versions. +# The weekly schedule rebuilds main, refreshing the shared base images for CVEs and cutting a versioned GitHub release. +# A path-scoped push publishes main as soon as the codegen App commits a new Make/Matrix.json pin, so a new upstream product version ships at once. +# A manual dispatch publishes the branch it is started from, main as the stable channel and develop as the develop tags. # -# Ordinary code merges do NOT publish: only a matrix pin change committed by the codegen App (push, branch- -# filtered to main, gated to the App identity so a human pin edit does not publish) or a schedule/dispatch does. -# develop's daily codegen pin update is sync-only (the push trigger is main-only). -# Refresh :develop by dispatching from develop. CI/validation runs separately on push (test-pull-request). -# -# The publish/stable decision is computed once by the plan job (publish-plan-task.yml). Every job gates on its -# outputs instead of re-testing event/actor/branch. +# Ordinary code merges do not publish: the plan task publishes a push only when a bot made it, so a human pin edit does not. +# The daily develop codegen pin update is sync-only, since the push trigger is main-only. +# Refresh the develop tags by dispatching from develop. +# The Docker matrix comes from Make/Matrix.json through the repo's own docker-prepare hook. on: workflow_dispatch: schedule: - # Weekly rebuild/publish of main (Mon 02:00 UTC) to pick up shared base-image updates. + # Weekly rebuild and publish of main, Monday 02:00 UTC, to pick up shared base-image updates. - cron: '0 2 * * MON' - # Publish main immediately when the codegen matrix changes. The daily codegen commits the new pin here. push: branches: [main] paths: [Make/Matrix.json] -# Single global group (not ref-scoped): this workflow pushes shared tags (base images, product images, -# readme), so a schedule + a dispatch + a pin push (or back-to-back dispatches) must not race on those -# shared tags. cancel-in-progress: false queues the next run instead of cancelling an in-flight publish, -# which could otherwise leave a partially pushed tag set. +# One global group rather than a ref-scoped one, since every run pushes shared tags (the base images, the product images, the readme). +# A schedule, a dispatch, and a pin push must queue rather than race on them, and cancel-in-progress is false so a queued run never cancels a partial push. concurrency: group: ${{ github.workflow }} cancel-in-progress: false +# GITHUB_TOKEN gets no scope by default, and each job below grants only what its task writes with. +permissions: {} + jobs: - # Single source of the release-gate decision (publish? stable?) - every job below gates on its outputs - # instead of re-testing event/actor/branch. See publish-plan-task.yml for the policy. + # Single source of the release-gate decision (publish or not, stable or not), reused by every job below. plan: name: Plan release job - uses: ./.github/workflows/publish-plan-task.yml + uses: ptr727/ProjectTemplate/.github/workflows/publish-plan-task.yml@456694681ae76ff162a9bd784b5376901957feae # 2.0.685 with: event_name: ${{ github.event_name }} actor: ${{ github.actor }} ref_name: ${{ github.ref_name }} - # Compute the version once from the trigger branch and thread it to every consumer. github.ref already - # equals the built branch (one branch per run), so NBGV classifies natively - main publishes clean, a - # develop dispatch publishes a prerelease. - get-version: - name: Get version information job - # Only the long-lived branches publish. A stray dispatch from a feature branch is a no-op. + # The same reusable gate the push CI runs, on the triggering commit, running only when a publish will happen. + validate: + name: Validate job needs: [plan] if: ${{ needs.plan.outputs.publish == 'true' }} - uses: ./.github/workflows/get-version-task.yml - secrets: inherit - with: - ref: ${{ github.ref_name }} + uses: ptr727/ProjectTemplate/.github/workflows/validate-task.yml@456694681ae76ff162a9bd784b5376901957feae # 2.0.685 + permissions: + contents: read + secrets: + CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }} - # Base images share a single tag (nx-base:ubuntu-noble) across branches. The main schedule/push/dispatch - # builds and refreshes it. A develop dispatch sets build_base: false (see build-docker below) and reuses - # main's published base so it can't overwrite the shared tag. + # The shared base images carry one branch-agnostic tag, so only a main publish builds and pushes them, and a develop publish reuses main's. + # This job rather than the hub's build-base leg pushes them, since that leg holds no Docker Hub login and a composite hook cannot read secrets. build-base: name: Build base image job - needs: [plan] - if: ${{ needs.plan.outputs.publish == 'true' && needs.plan.outputs.stable == 'true' }} - uses: ./.github/workflows/build-base-images-task.yml - secrets: inherit - with: - push: true - # The shared base is branch-agnostic and its branch-scoped buildcache tag (`buildcache-`) keys on this - # ref, so it uses the branch ref, not the versioned commit: the base is not part of the per-commit versioned - # product set, and a commit SHA here would split the cache into per-commit tags. - ref: ${{ github.ref_name }} - - # Validate the published tree (lint + dotnet test) before building, so a publish can never ship a tree that - # would fail the same gate CI enforces on push. Pinned to the versioned commit on main so it validates the - # exact commit the images and release are built from. - validate: - name: Validate job - needs: [plan, get-version] - if: ${{ needs.plan.outputs.publish == 'true' }} - uses: ./.github/workflows/validate-task.yml - secrets: inherit - with: - ref: ${{ github.ref_name == 'main' && needs.get-version.outputs.GitCommitId || github.ref_name }} - - build-docker: - name: Build Docker image job - needs: [plan, get-version, build-base, validate] - # always() so the job still runs when build-base is intentionally skipped on a develop dispatch. Fail only - # if a prerequisite actually failed. - if: >- - ${{ always() - && needs.plan.outputs.publish == 'true' - && needs.get-version.result == 'success' - && needs.validate.result == 'success' - && (needs.build-base.result == 'success' || needs.build-base.result == 'skipped') }} - uses: ./.github/workflows/build-docker-task.yml - secrets: inherit - with: - push: true - branch: ${{ github.ref_name }} - # Pin main to the exact commit get-version versioned, not the moving `main` ref, so the image's embedded - # version matches the GitHub release tag even if main advances mid-run. develop has no versioned release, - # so the moving ref is fine. - ref: ${{ github.ref_name == 'main' && needs.get-version.outputs.GitCommitId || github.ref_name }} - # The publisher's own build-base job already built the shared base on the main run (and a develop - # dispatch reuses main's published base), so the build task never rebuilds it - this avoids a double - # base build and keeps the branch-agnostic nx-base tag owned by the main run. - build_base: false - # Single NBGV classification threaded down (no nested re-run in the build task). - semver2: ${{ needs.get-version.outputs.SemVer2 }} - - # main-only: a develop dispatch publishes images + the :develop tag but cuts no versioned GitHub release. - github-release: - name: Publish GitHub release job - needs: [plan, get-version, build-docker] - if: ${{ needs.plan.outputs.publish == 'true' && needs.plan.outputs.stable == 'true' }} + needs: [plan, validate] + if: ${{ needs.plan.outputs.stable == 'true' && needs.validate.result == 'success' }} runs-on: ubuntu-latest permissions: - contents: write - + contents: read steps: - # Check out the exact built commit so the uploaded release files match the tag even if main advances mid-run. - name: Checkout code step uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: - ref: ${{ needs.get-version.outputs.GitCommitId }} - - # Backstop (main only): a public release must not carry a prerelease '-', guarding against NBGV mis-versioning the - # public ref into a malformed "Latest" release. Strip '+buildmetadata' first - a '-' there is legitimate. Only a - # '-' in the core/prerelease segment marks a prerelease. - - name: Verify public release version step - env: - SEMVER2: ${{ needs.get-version.outputs.SemVer2 }} - run: | - set -Eeuo pipefail - CORE_AND_PRE="${SEMVER2%%+*}" # drop +buildmetadata; a '-' here is the genuine prerelease separator - if [[ "$CORE_AND_PRE" == *-* ]]; then - echo "::error::Public (main) release version '$SEMVER2' carries a prerelease suffix; refusing to publish." - exit 1 - fi + ref: ${{ github.sha }} - # The weekly publisher re-runs even with no new commits, so the version may already be released. Skip the release - # step when a release for this tag already exists to avoid a no-op republish. - - name: Check for existing release step - id: release-exists - env: - GH_TOKEN: ${{ github.token }} - TAG: ${{ needs.get-version.outputs.SemVer2 }} - run: | - set -Eeuo pipefail - if gh release view "$TAG" --repo "$GITHUB_REPOSITORY" >/dev/null 2>&1; then - echo "exists=true" >> "$GITHUB_OUTPUT" - if [[ "${{ github.event_name }}" == "workflow_dispatch" ]]; then - echo "Release $TAG already exists; workflow_dispatch will refresh it." - else - echo "Release $TAG already exists; skipping release creation (no-op republish)." - fi - else - echo "exists=false" >> "$GITHUB_OUTPUT" - fi - - # `target_commitish` must be set explicitly: otherwise GitHub's REST API tags the release on the default branch. - # Pin it to `GitCommitId` so the tag is on the exact built commit, consistent with the SemVer2 tag and artifacts. - # Skip when the release already exists, but always let a manual `workflow_dispatch` through to refresh it. - # Every release is a tag on the built commit plus the auto-attached source zip, README, and LICENSE. This is a - # Docker-only repo: it ships no `release-asset-*` binaries/packages, so the release carries no extra files and - # `fail_on_unmatched_files` is omitted (see Template Adaptations in ARCHITECTURE.md). The image is the published artifact - # on Docker Hub. - - name: Create GitHub release step - if: ${{ steps.release-exists.outputs.exists == 'false' || github.event_name == 'workflow_dispatch' }} - uses: softprops/action-gh-release@efb35369e0ad2afab669f228072c1b0d510eae64 # v3.0.3 - with: - generate_release_notes: true - tag_name: ${{ needs.get-version.outputs.SemVer2 }} - target_commitish: ${{ needs.get-version.outputs.GitCommitId }} - # This run's release represents the main publish. - prerelease: false - files: | - LICENSE - README.md - - # main-only: resolve the Docker Hub repository list (Docker Hub does not read the GitHub README). The image - # set is repo-specific, so derive it from main's Matrix.json (lowercased ptr727/ plus the shared base - # repos) for the readme matrix below. - docker-readme-repos: - name: Get docker hub readme repositories job - needs: [plan, get-version, build-docker] - if: ${{ needs.plan.outputs.publish == 'true' && needs.plan.outputs.stable == 'true' }} - runs-on: ubuntu-latest - outputs: - repositories: ${{ steps.list.outputs.repositories }} - - steps: - - - name: Checkout code step - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - ref: ${{ needs.get-version.outputs.GitCommitId }} - - - name: Resolve repository list step - id: list - run: | - set -Eeuo pipefail - # shellcheck disable=SC2016 # the jq program is a literal, not a shell expansion - REPOS=$(jq --compact-output \ - '[.Images[].Name | ascii_downcase | "ptr727/\(.)"] + ["ptr727/nx-base","ptr727/nx-base-lsio"] | sort | unique' \ - ./Make/Matrix.json) - echo "repositories=$REPOS" >> "$GITHUB_OUTPUT" - - # main-only: push the repository overview (README.md) to each Docker Hub repo derived above. - docker-readme: - name: Publish docker hub readme job - needs: [plan, get-version, docker-readme-repos] - if: ${{ needs.plan.outputs.publish == 'true' && needs.plan.outputs.stable == 'true' }} - runs-on: ubuntu-latest - strategy: - matrix: - repository: ${{ fromJSON(needs.docker-readme-repos.outputs.repositories) }} - - steps: - - - name: Checkout code step - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - ref: ${{ needs.get-version.outputs.GitCommitId }} - - - name: Publish Docker Hub readme step - uses: peter-evans/dockerhub-description@1b9a80c056b620d92cedb9d9b5a223409c68ddfa # v5.0.0 + - name: Login to Docker Hub step + uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0 with: username: ${{ secrets.DOCKER_HUB_USERNAME }} password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }} - repository: ${{ matrix.repository }} - short-description: ${{ github.event.repository.description }} - readme-filepath: ./README.md + + - name: Build and push base images step + uses: ./.github/actions/docker-build-base + with: + push: true + + # Version, build, and push every Make/Matrix.json row of the trigger branch, then cut the GitHub release on main. + # build-base is an optional dependency, skipped on a develop publish, so its result is allowlisted as success or skipped explicitly. + # The !cancelled() replaces the implicit success() that would otherwise skip this job whenever build-base is skipped. + publish: + name: Publish project release job + needs: [plan, validate, build-base] + if: >- + ${{ !cancelled() + && needs.plan.outputs.publish == 'true' + && needs.validate.result == 'success' + && (needs.build-base.result == 'success' || needs.build-base.result == 'skipped') }} + uses: ptr727/ProjectTemplate/.github/workflows/build-release-task.yml@456694681ae76ff162a9bd784b5376901957feae # 2.0.685 + permissions: + contents: write + # Lets the hub github-release job delete its transfer artifacts (D5.1), though a Docker-only release uploads none. + actions: write + secrets: + DOCKER_HUB_USERNAME: ${{ secrets.DOCKER_HUB_USERNAME }} + DOCKER_HUB_ACCESS_TOKEN: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }} + with: + # Pin to the triggering commit rather than the moving branch tip, so the version, the images, and the release tag share one commit. + ref: ${{ github.sha }} + branch: ${{ github.ref_name }} + smoke: false + # A develop publish pushes images and the develop tags but cuts no GitHub release. + github: ${{ needs.plan.outputs.stable == 'true' }} + dockerhub: true + enable_docker: true + enable_dotnet_publish: false + enable_nuget: false + enable_pypi: false + # The images are the published artifact, so the release carries only the tag, the source zip, README, and LICENSE. + expect_release_assets: false + # The build-base job above already built the shared base, or a develop publish reuses main's. + docker_build_base: false + + # Docker Hub does not read the GitHub README, so the overview is pushed to every product repository and both base repositories. + # The hub task pushes only on main, and publishes Docker/README.md, the Docker Hub overview. + publish-docker-readme: + name: Publish Docker Hub readme job + needs: [plan, publish] + if: ${{ needs.plan.outputs.stable == 'true' && needs.publish.result == 'success' }} + permissions: + contents: read + uses: ptr727/ProjectTemplate/.github/workflows/publish-docker-readme-task.yml@456694681ae76ff162a9bd784b5376901957feae # 2.0.685 + with: + ref: ${{ github.sha }} + branch: ${{ github.ref_name }} + manifest: ./Make/Matrix.json + manifest-jq: '[.Images[].Name | ascii_downcase | "ptr727/\(.)"] + ["ptr727/nx-base","ptr727/nx-base-lsio"] | sort | unique' + secrets: + DOCKER_HUB_USERNAME: ${{ secrets.DOCKER_HUB_USERNAME }} + DOCKER_HUB_ACCESS_TOKEN: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }} diff --git a/.github/workflows/run-codegen-pull-request-task.yml b/.github/workflows/run-codegen-pull-request-task.yml deleted file mode 100644 index cac3628..0000000 --- a/.github/workflows/run-codegen-pull-request-task.yml +++ /dev/null @@ -1,79 +0,0 @@ -name: Run codegen and pull request task - -# Runs codegen against `main` and `develop` in parallel via a matrix and opens a PR against each base -# (codegen-main -> main, codegen-develop -> develop), which the merge-bot auto-merges independently. - -on: - workflow_call: - secrets: - CODEGEN_APP_CLIENT_ID: - required: true - CODEGEN_APP_PRIVATE_KEY: - required: true - -jobs: - - codegen: - name: Run ${{ matrix.target.ref }} codegen and pull request job - runs-on: ubuntu-latest - permissions: - contents: write - pull-requests: write - strategy: - # Each branch gets its own parallel codegen run + PR. One branch's failure doesn't affect the other. - fail-fast: false - matrix: - target: - - ref: main - branch: codegen-main - - ref: develop - branch: codegen-develop - - steps: - - - name: Generate GitHub App token step - # App token so the PR open fires `pull_request` workflow events (GITHUB_TOKEN opens don't). - id: app-token - uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0 - with: - client-id: ${{ secrets.CODEGEN_APP_CLIENT_ID }} - private-key: ${{ secrets.CODEGEN_APP_PRIVATE_KEY }} - - - name: Setup .NET SDK step - uses: actions/setup-dotnet@a98b56852c35b8e3190ac28c8c2271da59106c68 # v6.0.0 - with: - dotnet-version: 10.x - - - name: Checkout code step - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - ref: ${{ matrix.target.ref }} - token: ${{ steps.app-token.outputs.token }} - - - name: Run codegen step - run: | - set -Eeuo pipefail - dotnet run --project ./CreateMatrix/CreateMatrix.csproj -- \ - matrix --versionpath=./Make/Version.json --matrixpath=./Make/Matrix.json --updateversion - - - name: Format code step - run: | - set -Eeuo pipefail - dotnet tool restore - dotnet husky install - dotnet csharpier format --log-level=debug . - git status - - - name: Create pull request step - uses: peter-evans/create-pull-request@5f6978faf089d4d20b00c7766989d076bb2fc7f1 # v8.1.1 - id: cpr - with: - # App token: triggers pull_request workflow events and creates verified commits as the app. - token: ${{ steps.app-token.outputs.token }} - base: ${{ matrix.target.ref }} - branch: ${{ matrix.target.branch }} - title: 'Update codegen files' - body: 'This PR updates the codegen files.' - commit-message: 'Update codegen files' - delete-branch: true - sign-commits: true diff --git a/.github/workflows/run-periodic-codegen-pull-request.yml b/.github/workflows/run-periodic-codegen-pull-request.yml index f22d91b..dcb49ef 100644 --- a/.github/workflows/run-periodic-codegen-pull-request.yml +++ b/.github/workflows/run-periodic-codegen-pull-request.yml @@ -1,23 +1,23 @@ -name: Run daily CodeGen and Pull Request action +name: Run daily codegen and pull request action +# Thin caller: the codegen cycle is the hub's reusable run-codegen-pull-request-task.yml, which every fleet repo generating checked-in files from an external source reaches rather than carries. on: workflow_dispatch: schedule: - # Daily at 04:00 UTC, staggered after the weekly publish so the two don't start together on Mondays. - cron: '0 4 * * *' +# Workflow-only group (no -${{ github.ref }}): the task writes to the fixed codegen-main/codegen-develop branches regardless of the triggering ref, so a dispatch and the scheduled run must not race on them. concurrency: - # Workflow-only group (no `-${{ github.ref }}`): the task writes the fixed `codegen-main`/`codegen-develop` - # branches regardless of triggering ref, so a dispatch and the scheduled run must not race on them. group: ${{ github.workflow }} cancel-in-progress: true +permissions: {} + jobs: run-codegen: name: Run codegen and pull request job - uses: ./.github/workflows/run-codegen-pull-request-task.yml - secrets: inherit - permissions: - contents: write - pull-requests: write + uses: ptr727/ProjectTemplate/.github/workflows/run-codegen-pull-request-task.yml@456694681ae76ff162a9bd784b5376901957feae # 2.0.685 + secrets: + CODEGEN_APP_CLIENT_ID: ${{ secrets.CODEGEN_APP_CLIENT_ID }} + CODEGEN_APP_PRIVATE_KEY: ${{ secrets.CODEGEN_APP_PRIVATE_KEY }} diff --git a/.github/workflows/test-pull-request.yml b/.github/workflows/test-pull-request.yml index 9089c1e..5d647ef 100644 --- a/.github/workflows/test-pull-request.yml +++ b/.github/workflows/test-pull-request.yml @@ -1,7 +1,6 @@ name: Test pull request action -# CI for every branch. Runs on push so the reusable tasks resolve from the pushed head: a PR that edits a -# workflow tests its own copy, and the aggregator (the ruleset required check) is produced on the head SHA by +# CI for every branch. Runs on push so the aggregator (the ruleset required check) is produced on the head SHA by # the sole producing run. validate (unit tests + lint) runs on every push. The image smoke build runs only # when image files changed (inline git diff change-gate), since a full product smoke is heavier than a # doc-only push warrants. CI never publishes (the publisher runs on schedule/push-on-Matrix.json/dispatch). @@ -27,8 +26,9 @@ permissions: {} jobs: # Inline change-gate (no third-party action): diff the push against its before-commit to decide whether the - # image smoke build is needed. image => any Docker/Matrix/Version file changed. Base => a base Dockerfile - # changed. On a non-push event (workflow_dispatch) or a new branch (no before-commit), build everything. + # image smoke build is needed. image => any Docker/Matrix/Version file, Docker hook, or workflow changed, the last + # covering a hub pin bump, whose tasks a pull request otherwise never exercises. Base => a base + # Dockerfile or the base hook changed. On a non-push event (workflow_dispatch) or a new branch (no before-commit), build everything. # !github.event.deleted skips a branch-deletion push (github.sha is all-zeros, so checkout would fail). changes: name: Detect image changes job @@ -65,49 +65,57 @@ jobs: echo "base=true" >> "$GITHUB_OUTPUT" exit 0 fi - if grep -qE '^(Docker/|Make/Matrix\.json$|Make/Version\.json$)' <<<"$DIFF"; then + if grep -qE '^(Docker/|Make/Matrix\.json$|Make/Version\.json$|\.github/actions/docker-|\.github/workflows/)' <<<"$DIFF"; then echo "image=true" >> "$GITHUB_OUTPUT" else echo "image=false" >> "$GITHUB_OUTPUT" fi - if grep -qE '^Docker/(NxBase\.Dockerfile|NxBase-LSIO\.Dockerfile)$' <<<"$DIFF"; then + if grep -qE '^(Docker/(NxBase\.Dockerfile|NxBase-LSIO\.Dockerfile)$|\.github/actions/docker-build-base/)' <<<"$DIFF"; then echo "base=true" >> "$GITHUB_OUTPUT" else echo "base=false" >> "$GITHUB_OUTPUT" fi - # The same unit-test + lint gate the publisher runs, so the PR gate and the publish gate are identical. + # The hub's reusable gate, the same one the publisher runs, so the push gate and the publish gate are identical. validate: name: Validate job if: ${{ !github.event.deleted }} + uses: ptr727/ProjectTemplate/.github/workflows/validate-task.yml@456694681ae76ff162a9bd784b5376901957feae # 2.0.685 permissions: contents: read - uses: ./.github/workflows/validate-task.yml - secrets: inherit + secrets: + CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }} - # Fast smoke build in place of the full matrix: only runs when image files changed, and builds just NxMeta + - # NxMeta-LSIO on amd64 (no push). branch targets the pushed branch so a push to develop validates the develop - # image rows and a push to main validates the main rows. ref is left unset so checkout uses the pushed head. - # build_base is enabled only when a base Dockerfile changed (otherwise the product smoke pulls the published - # base from Docker Hub). It also `needs: validate`, and the `if` carries no `always()`, so an unsuccessful - # validate skips this job via GitHub's implicit needs-success rule - the smoke build is not spent when - # validation already failed. + # Fast smoke build in place of the full matrix, only when image files changed, through the hub's release chain with smoke set, which never pushes. + # It builds one NxMeta and one NxMeta-LSIO row on amd64, the image input naming them as the docker-prepare hook's filter. + # The base images build first only when a base Dockerfile changed, and otherwise the product smoke pulls the published base from Docker Hub. + # It needs validate with no always() in its if, so an unsuccessful validate skips it through the implicit needs-success rule. + # The Docker Hub secrets are mapped even on smoke, since the hub task logs in on every build for the higher rate limit. smoke-build: name: Smoke build job needs: [changes, validate] if: ${{ !github.event.deleted && needs.changes.outputs.image == 'true' }} + uses: ptr727/ProjectTemplate/.github/workflows/build-release-task.yml@456694681ae76ff162a9bd784b5376901957feae # 2.0.685 permissions: contents: read - uses: ./.github/workflows/build-docker-task.yml - secrets: inherit + secrets: + DOCKER_HUB_USERNAME: ${{ secrets.DOCKER_HUB_USERNAME }} + DOCKER_HUB_ACCESS_TOKEN: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }} with: - push: false smoke: true - # Map the push ref to a Matrix.json .Branch value: a main push checks main's rows. Every other branch - # (develop, and feature branches that merge via develop) FALLS BACK to develop's rows. Passing - # github.ref_name directly would be a feature-branch name matching no .Branch row -> empty smoke matrix. + github: false + dockerhub: false + enable_docker: true + enable_dotnet_publish: false + enable_nuget: false + enable_pypi: false + expect_release_assets: false + docker_image: NxMeta,NxMeta-LSIO + docker_build_base: ${{ needs.changes.outputs.base == 'true' }} + # Map the push ref to a Make/Matrix.json Branch value: a main push checks main's rows. + # Every other branch, develop and the feature branches that merge through it, falls back to develop's rows. + # Passing github.ref_name directly would name a feature branch that matches no row, an empty smoke matrix. branch: ${{ github.ref_name == 'main' && 'main' || 'develop' }} - build_base: ${{ needs.changes.outputs.base == 'true' }} # Single required status check. Its name is the ruleset-bound context in the live branch ruleset. # Do not rename it without updating the ruleset in lockstep. diff --git a/.github/workflows/validate-task.yml b/.github/workflows/validate-task.yml deleted file mode 100644 index fa82838..0000000 --- a/.github/workflows/validate-task.yml +++ /dev/null @@ -1,87 +0,0 @@ -name: Validate task - -# The single validation gate, reused by test-pull-request (the required check) and the publisher, so CI and -# publish run the identical definition. Does not build images. -# - Lint: Husky (CSharpier + dotnet format style), plus Markdown, spelling, and workflow YAML via pinned action -# wrappers, and line endings via editorconfig-checker (Docker). -# - Test: dotnet test with coverage, uploaded to Codecov (report-only). - -on: - workflow_call: - inputs: - # Git ref to validate (empty = default checkout ref). A publisher caller passes its trigger branch so the - # publish run validates the branch it publishes. - ref: - required: false - type: string - default: '' - -jobs: - - validate: - name: Validate job - runs-on: ubuntu-latest - permissions: - contents: read - - steps: - - - name: Setup .NET SDK step - uses: actions/setup-dotnet@a98b56852c35b8e3190ac28c8c2271da59106c68 # v6.0.0 - with: - dotnet-version: 10.x - - - name: Checkout code step - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - ref: ${{ inputs.ref }} - - # Husky runs the same CSharpier + dotnet format style checks the editor and the pre-commit hook run. - - name: Check code style step - run: | - set -Eeuo pipefail - dotnet tool restore - dotnet husky install - dotnet husky run - - # Doc linters run as pinned action wrappers. editorconfig-checker's action is install-only, so it runs via Docker. - - name: Lint Markdown step - uses: DavidAnson/markdownlint-cli2-action@21c1be1b93ad9ed58fa840aacc3f279cde2a72ff # v24.2.0 - with: - globs: '**/*.md' - - - name: Spell check step - uses: streetsidesoftware/cspell-action@6f3c77c1406bc930f944ba97e9801a22e42caf58 # v9.1.0 - with: - files: | - README.md - HISTORY.md - incremental_files_only: false - - - name: Lint workflows step - uses: raven-actions/actionlint@3d39aea434753780c3b3d4a1a31c854b4dbf49d7 # v2.2.0 - - - name: Check EditorConfig step - run: docker run --rm -v "$PWD":/check --workdir /check mstruebing/editorconfig-checker:latest - - # global.json opts into the native Microsoft.Testing.Platform runner, so the VSTest bridge - # (and its --collect data collector) is gone. - # --coverage-output stays unset because pinning one filename gives every test project in the solution the same path, and the last to finish overwrites the rest. - # The default .cobertura.xml written instead is a name codecov-cli's finder does not match, so each report is prefixed rather than renamed, keeping the guid that makes it unique. - - name: Run unit tests step - run: | - set -Eeuo pipefail - dotnet test --coverage --coverage-output-format cobertura --results-directory ./coverage - for report in ./coverage/*.cobertura.xml; do - [ -e "$report" ] || continue - mv "$report" "./coverage/coverage-$(basename "$report")" - done - - # Report-only: fail_ci_if_error is false so a Codecov hiccup or an absent token never fails the gate. - - name: Upload coverage to Codecov step - uses: codecov/codecov-action@303a32d7a59b442fa8d48b6a1cc6825c09c847a5 # v7.1.1 - with: - directory: ./coverage - fail_ci_if_error: false - env: - CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }} diff --git a/.husky/pre-commit b/.husky/pre-commit index e9a75d5..7557df6 100755 --- a/.husky/pre-commit +++ b/.husky/pre-commit @@ -1,4 +1,5 @@ #!/bin/sh +# shellcheck disable=SC1091 # Generated by Husky install, not present at lint time. . "$(dirname "$0")/_/husky.sh" # Local pre-commit: language formatting and style only (no Docker). Full lint runs in CI and the VS Code Lint tasks. diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index eb5d5c2..1af58a7 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -38,40 +38,39 @@ Because `Docker/` and the Compose files under `Make/` are generated, a change to The full CI/CD contract, meaning triggers, jobs, the one-branch publish model, versioning, and the multi-image build layer, is specified in [WORKFLOW.md][workflow], the canonical guide. The summary below is a pointer and does not duplicate those rules. -- CI runs on **push to every branch** ([test-pull-request.yml][test-pull-request]): it validates ([validate-task.yml][validate-task]) on every push, and runs a fast smoke build ([build-docker-task.yml][build-docker-task] with `smoke: true`, meaning NxMeta and NxMeta-LSIO, amd64, no push) only when image files (`Docker/**`, `Make/Matrix.json`, `Make/Version.json`) change, via an inline `git diff` change-gate. One aggregator job, `Check pull request workflow status job`, is the ruleset-bound required check. -- Publishing is **triggered-Docker, one branch per run** ([publish-release.yml][publish-release]): the triggers are the weekly schedule (rebuilds `main` only), a path-scoped push to `main` on `Make/Matrix.json` (publishes a new codegen product pin at once), and manual dispatch (publishes the started-from branch). One run computes the version once ([get-version-task.yml][get-version-task]), builds the shared base once (main only, with a develop dispatch reusing it via `build_base: false`), builds the full product matrix from `Make/Matrix.json`, and on `main` cuts the GitHub release and pushes the Docker Hub overviews. +- Every task is hub-hosted in `ptr727/ProjectTemplate` and reached by a SHA-pinned `uses:`, so this repo carries only its entry workflows and three hooks under [`.github/actions/`][actions]: `docker-prepare` maps `Make/Matrix.json` onto the hub's Docker matrix, `docker-build-base` builds the shared bases, and `codegen` runs the generator. +- CI runs on **push to every branch** ([test-pull-request.yml][test-pull-request]): it validates (the hub's `validate-task.yml`) on every push, and runs a fast smoke build (the hub's `build-release-task.yml` with `smoke: true`, meaning NxMeta and NxMeta-LSIO, amd64, no push) only when image files (`Docker/**`, `Make/Matrix.json`, `Make/Version.json`, the Docker hooks, the workflows) change, via an inline `git diff` change-gate. One aggregator job, `Check pull request workflow status job`, is the ruleset-bound required check. +- Publishing is **triggered-Docker, one branch per run** ([publish-release.yml][publish-release]): the triggers are the weekly schedule (rebuilds `main` only), a path-scoped push to `main` on `Make/Matrix.json` (publishes a new codegen product pin at once), and manual dispatch (publishes the started-from branch). One run builds the shared base once (main only, with a develop dispatch reusing it), then the hub's `build-release-task.yml` computes the version once, builds the branch's product matrix from `Make/Matrix.json`, and on `main` cuts the GitHub release, and the hub's `publish-docker-readme-task.yml` pushes the Docker Hub overviews. - Merges to `main` or `develop` do not build or publish images by themselves. Only the matrix-pin push to `main`, a schedule, or a dispatch publishes. Auto-merged Dependabot and codegen pull requests land commits the next publish picks up. -- Do not reintroduce a two-branch publish matrix, a nested `get-version` in the build task, the date-badge or standalone docker-readme workflows, or `dorny/paths-filter`. +- Do not reintroduce a two-branch publish matrix, a carried copy of a hub task, the date-badge workflow, or `dorny/paths-filter`. ## Versioning Labels and Tags Two version streams reach the images, and confusing them is the common error: -- **The NBGV version labels the image.** The orchestrator's single `get-version` run computes the Nerdbank.GitVersioning version once and threads its `semver2` output down into `build-docker-task.yml` as the `LABEL_VERSION` build arg, so one classification labels every product leg and no second NBGV run can reclassify it. The same version is the GitHub release tag on `main`. +- **The NBGV version labels the image.** The hub release chain's single `get-version` run computes the Nerdbank.GitVersioning version once and threads its `semver2` output down into the hub's `build-docker-task.yml` as the `LABEL_VERSION` build arg, so one classification labels every product leg and no second NBGV run can reclassify it. The same version is the GitHub release tag on `main`. - **The Nx product version tags the image.** The Docker Hub tags come from `Make/Matrix.json`, not from NBGV. A row's `Tags` carry the upstream product version and its channel alias (`stable`, `latest`, `rc`, `beta`, and the `develop-` prefixed forms). The NBGV version appears only as the image label and the release tag, never as a Docker tag. ## Template Adaptations This repo derives its CI and conventions from the fleet template, `ptr727/ProjectTemplate`. Carried artifacts are taken by full-file replacement, and the deliberate deviations below are documented so they are not mistaken for drift. -- **Triggered-Docker publisher, one branch per run.** `publish-release.yml` is `workflow_dispatch` plus a weekly `schedule` (main only) plus a path-scoped `push` to `main` on `Make/Matrix.json`. It builds exactly one branch, the trigger ref (`github.ref_name`), so NBGV classifies natively with no cross-branch leg and no `IGNORE_GITHUB_REF`. The jobs are a single `get-version` -> `build-base` (main only) -> `build-docker` -> `github-release` (main only) -> `docker-readme` (main only) -> `cleanup-artifacts` chain. A develop dispatch refreshes the `:develop` images only, with no GitHub release, and there is no two-leg `build-main` / `build-develop` combined run. -- **Multi-image, shared-base build layer.** The shared `nx-base` and `nx-base-lsio` images are built once on the `main` run and reused by a develop dispatch (`build_base: false`, so it never overwrites the branch-agnostic `nx-base` tag), and `build-docker-task.yml` builds every product image from `Make/Matrix.json` with `max-parallel: 4`. The template's single-target branch matrix cannot express the shared-base fan-out, so the build layer stays repo-owned. -- **Single NBGV run threaded to the build task.** `build-docker-task.yml` has no nested `get-version`. The orchestrator's single `get-version` run threads `semver2` down as the image `LABEL_VERSION`, so one classification feeds every product leg and no second NBGV run can reclassify or collide a tag. -- **Docker-only GitHub release, with no `release-asset-*` files.** The `github-release` job follows the template's generic release semantics (tag on the built commit, auto source zip plus README and LICENSE, `target_commitish` pinned, skip-existing guard, main-only `Verify public release version` backstop), but this repo ships no binary or package release assets because the published artifacts are the Docker Hub images. So there is no `release-asset-*` download step, and `fail_on_unmatched_files` is omitted since it has no files to guard. -- **Folded Docker Hub readme.** There is no standalone docker-readme task. `publish-release.yml` carries `docker-readme-repos` and `docker-readme` jobs gated to `main` that derive the repository list inline from `Make/Matrix.json` (lowercased `ptr727/` plus the shared base repos) and matrix `peter-evans/dockerhub-description` over it. +- **Triggered-Docker publisher, one branch per run.** `publish-release.yml` is `workflow_dispatch` plus a weekly `schedule` (main only) plus a path-scoped `push` to `main` on `Make/Matrix.json`, where the hub's Docker shape is dispatch plus schedule alone. It builds exactly one branch, the trigger ref (`github.ref_name`), so NBGV classifies natively with no cross-branch leg and no `IGNORE_GITHUB_REF`. The jobs are a `plan` -> `validate` -> `build-base` (main only) -> `publish` -> `publish-docker-readme` (main only) chain. A develop dispatch refreshes the `:develop` images only, with no GitHub release. +- **Multi-image, shared-base build layer through hooks.** This repo is the hub Docker family's matrix and `build-base` case. The `docker-prepare` hook emits the branch's `Make/Matrix.json` rows as the hub's matrix, and takes the smoke build's `docker_image` input as a name filter rather than as a Docker Hub repository. The hub core owns the cache policy, the platform selection, and the login, and runs the matrix without the `max-parallel: 4` cap the carried copy had. +- **Repo-owned pushing base build.** The hub's `build-base` leg holds no Docker Hub login, and a composite hook cannot read secrets, so the publisher's own `build-base` job logs in and runs the `docker-build-base` hook to push the shared `nx-base` and `nx-base-lsio` images, and `publish` sets `docker_build_base: false`. Only the non-pushing smoke build reaches the hook through the hub. The hook takes `push` as its branch signal, since no branch reaches it: a push is a `main` publish, built multi-arch against `buildcache-main` plus the inline cache on the base tag. +- **Docker-only GitHub release, with no `release-asset-*` files.** The publisher sets `expect_release_assets: false`, so the hub's release carries the tag, the auto source zip, README, and LICENSE, and sets `github: true` only on `main`. +- **Docker Hub readme from `Docker/README.md`.** The hub's `publish-docker-readme-task.yml` pushes [`Docker/README.md`][docker-readme], its default's first choice, to every product and base repository, the list derived from `Make/Matrix.json` by its `manifest-jq` input. - **No date badge.** The repo ships no `build-datebadge-task.yml` workflow and no publisher job for it, and `README.md` carries no "Last Build" badge pointing at a BYOB gist. -- **Husky.Net pre-commit hooks.** This repo runs its local hook through Husky.Net, configured in `.husky/task-runner.json`, rather than through the Python `pre-commit` framework. The hook runs the same CSharpier and `dotnet format style` checks CI enforces, surfaced earlier, and `validate-task.yml`, which stands in for the template's `test-release-task.yml`, runs that same Husky lint plus `dotnet test` in CI as the required check's quality gate. -- **Repo-owned build-layer leaves.** The build-layer leaves (`build-docker-task.yml`, `build-base-images-task.yml`) own their per-image Dockerfiles, build args, and target matrix, which the template's build layer does not express. Owning those specifics is not an exemption from the shared workflow conventions: their actions are SHA-pinned like the orchestration layer, and their Docker layer cache uses per-image registry-tag caches (`docker.io/ptr727/:buildcache-`, plus the base image's own tag and inline cache). -- **No `merge-upstream-version` merge-bot job.** This repo tracks the upstream Nx version through codegen, with `run-codegen-pull-request-task.yml` updating `Make/Version.json` and `Make/Matrix.json`, so the merge bot keeps `merge-codegen` and omits the template's `merge-upstream-version` job, which uses the separate `check-upstream-version-task.yml` mechanism this repo does not ship. +- **Husky.Net pre-commit hooks.** This repo runs its local hook through Husky.Net, configured in `.husky/task-runner.json`, rather than through the Python `pre-commit` framework. The hook runs the same CSharpier and `dotnet format style` checks the hub's `validate-task.yml` enforces in CI, surfaced earlier. +- **No upstream-version tracker.** This repo tracks the upstream Nx version through codegen, the `codegen` hook updating `Make/Version.json` and `Make/Matrix.json`, so it carries no `check-upstream-version-task.yml` caller, and the hub merge-bot's built-in `codegen-main` and `codegen-develop` rules cover its App pull requests. The `.vscode` task-set deviation is recorded in [OPERATIONS.md][operations], with the tooling it belongs to. -[build-docker-task]: ./.github/workflows/build-docker-task.yml -[get-version-task]: ./.github/workflows/get-version-task.yml +[actions]: ./.github/actions/ +[docker-readme]: ./Docker/README.md [operations]: ./OPERATIONS.md [publish-release]: ./.github/workflows/publish-release.yml [test-pull-request]: ./.github/workflows/test-pull-request.yml -[validate-task]: ./.github/workflows/validate-task.yml [workflow]: ./WORKFLOW.md diff --git a/Docker/download.sh b/Docker/download.sh index 6021d5c..bce7377 100644 --- a/Docker/download.sh +++ b/Docker/download.sh @@ -24,29 +24,29 @@ echo "Download Filename: ${DOWNLOAD_FILENAME}" wget --no-verbose --tries=5 --timeout=30 --retry-connrefused "${DOWNLOAD_URL}" case "${DOWNLOAD_FILENAME}" in - *.zip) - echo "Downloaded ZIP: ${DOWNLOAD_FILENAME}" - DOWNLOAD_DIR="./download_zip" - rm -rf "${DOWNLOAD_DIR}" - mkdir -p "${DOWNLOAD_DIR}" - unzip -q -d "${DOWNLOAD_DIR}" "${DOWNLOAD_FILENAME}" - DEB_ZIP_FILE="$(find "${DOWNLOAD_DIR}" -maxdepth 1 -type f -name "*.deb" -print -quit)" - if [ -z "${DEB_ZIP_FILE}" ]; then - echo "No .deb found in ${DOWNLOAD_DIR}" >&2 - exit 1 - fi - echo "DEB in ZIP: ${DEB_ZIP_FILE}" - mv "${DEB_ZIP_FILE}" "${DEB_FILE}" - rm -rf "${DOWNLOAD_DIR}" "${DOWNLOAD_FILENAME}" - ;; - *.deb) - echo "Downloaded DEB: ${DOWNLOAD_FILENAME}" - mv "${DOWNLOAD_FILENAME}" "${DEB_FILE}" - ;; - *) - echo "Unsupported download type: ${DOWNLOAD_FILENAME}" >&2 +*.zip) + echo "Downloaded ZIP: ${DOWNLOAD_FILENAME}" + DOWNLOAD_DIR="./download_zip" + rm -rf "${DOWNLOAD_DIR}" + mkdir -p "${DOWNLOAD_DIR}" + unzip -q -d "${DOWNLOAD_DIR}" "${DOWNLOAD_FILENAME}" + DEB_ZIP_FILE="$(find "${DOWNLOAD_DIR}" -maxdepth 1 -type f -name "*.deb" -print -quit)" + if [ -z "${DEB_ZIP_FILE}" ]; then + echo "No .deb found in ${DOWNLOAD_DIR}" >&2 exit 1 - ;; + fi + echo "DEB in ZIP: ${DEB_ZIP_FILE}" + mv "${DEB_ZIP_FILE}" "${DEB_FILE}" + rm -rf "${DOWNLOAD_DIR}" "${DOWNLOAD_FILENAME}" + ;; +*.deb) + echo "Downloaded DEB: ${DOWNLOAD_FILENAME}" + mv "${DOWNLOAD_FILENAME}" "${DEB_FILE}" + ;; +*) + echo "Unsupported download type: ${DOWNLOAD_FILENAME}" >&2 + exit 1 + ;; esac echo "DEB File: ${DEB_FILE}" diff --git a/Docker/entrypoint.sh b/Docker/entrypoint.sh index 4829f37..767c66b 100755 --- a/Docker/entrypoint.sh +++ b/Docker/entrypoint.sh @@ -2,7 +2,7 @@ # Launch the root-tool as root in the background using & echo "Launching root-tool" -sudo /opt/${COMPANY_NAME}/mediaserver/bin/root-tool & +sudo "/opt/${COMPANY_NAME}/mediaserver/bin/root-tool" & # Tell mediaserver it is running under Docker so it reports its OS variant correctly # https://github.com/networkoptix/nxvms-docker/commit/54bbd16 @@ -14,10 +14,8 @@ sudo /opt/${COMPANY_NAME}/mediaserver/bin/root-tool & # ${COMPANY_NAME} user, so the bind-mounted etc directory must be writable by # that user (same requirement as mediaserver itself writes its own conf). MEDIASERVER_CONF="/opt/${COMPANY_NAME}/mediaserver/etc/mediaserver.conf" -if ! grep -q "^currentOsVariantOverride=" "${MEDIASERVER_CONF}" 2>/dev/null -then - if echo "currentOsVariantOverride=docker" >> "${MEDIASERVER_CONF}" - then +if ! grep -q "^currentOsVariantOverride=" "${MEDIASERVER_CONF}" 2>/dev/null; then + if echo "currentOsVariantOverride=docker" >>"${MEDIASERVER_CONF}"; then echo "Added currentOsVariantOverride=docker to ${MEDIASERVER_CONF}" else echo "Warning: failed to write ${MEDIASERVER_CONF} — check that the bind-mounted etc directory is writable by user ${COMPANY_NAME}" >&2 @@ -26,4 +24,4 @@ fi # Launch the mediaserver using exec so it receives shutdown commands echo "Launching mediaserver" -exec /opt/${COMPANY_NAME}/mediaserver/bin/mediaserver -e +exec "/opt/${COMPANY_NAME}/mediaserver/bin/mediaserver" -e diff --git a/Docker/lsio-rename-user.sh b/Docker/lsio-rename-user.sh index d04e9ec..949cf8c 100644 --- a/Docker/lsio-rename-user.sh +++ b/Docker/lsio-rename-user.sh @@ -18,11 +18,28 @@ COMPANY_NAME="${1:?company name is required}" ADDUSER_RUN="/etc/s6-overlay/s6-rc.d/init-adduser/run" # Verify the LSIO base still matches the expected signature before patching -getent passwd abc >/dev/null || { echo "ERROR: expected LSIO user abc not found" >&2; exit 1; } -getent group abc >/dev/null || { echo "ERROR: expected LSIO group abc not found" >&2; exit 1; } -test -f "${ADDUSER_RUN}" || { echo "ERROR: ${ADDUSER_RUN} not found" >&2; exit 1; } -grep -q 'groupmod -o -g "${PGID}" abc' "${ADDUSER_RUN}" || { echo "ERROR: init-adduser groupmod signature changed" >&2; exit 1; } -grep -q 'usermod -o -u "${PUID}" abc' "${ADDUSER_RUN}" || { echo "ERROR: init-adduser usermod signature changed" >&2; exit 1; } +getent passwd abc >/dev/null || { + echo "ERROR: expected LSIO user abc not found" >&2 + exit 1 +} +getent group abc >/dev/null || { + echo "ERROR: expected LSIO group abc not found" >&2 + exit 1 +} +test -f "${ADDUSER_RUN}" || { + echo "ERROR: ${ADDUSER_RUN} not found" >&2 + exit 1 +} +# shellcheck disable=SC2016 # The LSIO script's own literal text is matched, not an expansion. +grep -qF 'groupmod -o -g "${PGID}" abc' "${ADDUSER_RUN}" || { + echo "ERROR: init-adduser groupmod signature changed" >&2 + exit 1 +} +# shellcheck disable=SC2016 # The LSIO script's own literal text is matched, not an expansion. +grep -qF 'usermod -o -u "${PUID}" abc' "${ADDUSER_RUN}" || { + echo "ERROR: init-adduser usermod signature changed" >&2 + exit 1 +} # Rename abc to the mediaserver account and repoint init-adduser at the new name usermod -l "${COMPANY_NAME}" abc @@ -30,8 +47,23 @@ groupmod -n "${COMPANY_NAME}" abc sed -i "s/abc/${COMPANY_NAME}/g" "${ADDUSER_RUN}" # Verify the rename took effect and no stray abc reference remains -getent passwd "${COMPANY_NAME}" >/dev/null || { echo "ERROR: rename to ${COMPANY_NAME} user failed" >&2; exit 1; } -getent group "${COMPANY_NAME}" >/dev/null || { echo "ERROR: rename to ${COMPANY_NAME} group failed" >&2; exit 1; } -if getent passwd abc >/dev/null; then echo "ERROR: abc user still present after rename" >&2; exit 1; fi -grep -q "usermod -o -u \"\${PUID}\" ${COMPANY_NAME}" "${ADDUSER_RUN}" || { echo "ERROR: init-adduser not repointed to ${COMPANY_NAME}" >&2; exit 1; } -if grep -qw abc "${ADDUSER_RUN}"; then echo "ERROR: stray abc token remains in init-adduser" >&2; exit 1; fi +getent passwd "${COMPANY_NAME}" >/dev/null || { + echo "ERROR: rename to ${COMPANY_NAME} user failed" >&2 + exit 1 +} +getent group "${COMPANY_NAME}" >/dev/null || { + echo "ERROR: rename to ${COMPANY_NAME} group failed" >&2 + exit 1 +} +if getent passwd abc >/dev/null; then + echo "ERROR: abc user still present after rename" >&2 + exit 1 +fi +grep -qF "usermod -o -u \"\${PUID}\" ${COMPANY_NAME}" "${ADDUSER_RUN}" || { + echo "ERROR: init-adduser not repointed to ${COMPANY_NAME}" >&2 + exit 1 +} +if grep -qw abc "${ADDUSER_RUN}"; then + echo "ERROR: stray abc token remains in init-adduser" >&2 + exit 1 +fi diff --git a/Make/Build.sh b/Make/Build.sh index d911e2f..fc28921 100755 --- a/Make/Build.sh +++ b/Make/Build.sh @@ -6,7 +6,6 @@ set -euo pipefail # sudo apt update && sudo apt upgrade --yes # sudo apt install dotnet-sdk-10.0 docker-compose docker-buildx --yes - ## Test installing in container: # docker run -it --rm ubuntu:noble /bin/bash # docker run -it --rm lsiobase/ubuntu:noble /bin/bash @@ -37,13 +36,12 @@ set -euo pipefail # Test.sh : Create and build and launch compose Test.yml compose stack. # Clean.sh : Shutdown compose stack and delete images. - # Build Dockerfile function BuildDockerfile { # Build x64 and ARM64 targets - docker buildx build --platform linux/amd64,linux/arm64 --tag test_${1,,} --file ../Docker/$1.Dockerfile ../Docker + docker buildx build --platform linux/amd64,linux/arm64 --tag "test_${1,,}" --file "../Docker/$1.Dockerfile" ../Docker # Build and load x64 target - docker buildx build --platform linux/amd64 --load --tag test_${1,,} --file ../Docker/$1.Dockerfile ../Docker + docker buildx build --platform linux/amd64 --load --tag "test_${1,,}" --file "../Docker/$1.Dockerfile" ../Docker } # Build base Dockerfile @@ -54,12 +52,12 @@ function BuildBaseDockerfile { RegistryCacheFrom="--cache-from=type=registry,ref=$2" # Build x64 and ARM64 targets if [[ "${PushBaseImages}" == "true" ]]; then - docker buildx build --platform linux/amd64,linux/arm64 --push --cache-to=type=inline ${RegistryCacheFrom} --tag $2 --file ../Docker/$1.Dockerfile ../Docker + docker buildx build --platform linux/amd64,linux/arm64 --push --cache-to=type=inline "${RegistryCacheFrom}" --tag "$2" --file "../Docker/$1.Dockerfile" ../Docker else - docker buildx build --platform linux/amd64,linux/arm64 ${RegistryCacheFrom} --tag $2 --file ../Docker/$1.Dockerfile ../Docker + docker buildx build --platform linux/amd64,linux/arm64 "${RegistryCacheFrom}" --tag "$2" --file "../Docker/$1.Dockerfile" ../Docker fi # Build and load x64 target - docker buildx build --platform linux/amd64 --load ${RegistryCacheFrom} --tag $2 --file ../Docker/$1.Dockerfile ../Docker + docker buildx build --platform linux/amd64 --load "${RegistryCacheFrom}" --tag "$2" --file "../Docker/$1.Dockerfile" ../Docker } # Create and use multi platform build environment diff --git a/Make/Clean.sh b/Make/Clean.sh index 103e305..42d5f67 100755 --- a/Make/Clean.sh +++ b/Make/Clean.sh @@ -4,7 +4,7 @@ set -euo pipefail # Delete images function DeleteImage { - docker image rm test_${1,,} || true + docker image rm "test_${1,,}" || true } # Down stack diff --git a/NxWitness.slnx b/NxWitness.slnx index 01a2a84..c74200b 100644 --- a/NxWitness.slnx +++ b/NxWitness.slnx @@ -1,14 +1,12 @@ - - - + + + - - diff --git a/OPERATIONS.md b/OPERATIONS.md index d5d64b6..d04efbb 100644 --- a/OPERATIONS.md +++ b/OPERATIONS.md @@ -6,9 +6,9 @@ How this repo is run: what verifying a change requires before it is pushed, the Verifying a change here is two things: running the gates, and then running by hand the one part of the contract the gates never reach. -The gates are the .NET clean-compile (the `.NET Format` VS Code task, per [CODESTYLE.md](./CODESTYLE.md)), the unit tests, and the document linters. `GOVERNANCE.md` "Running the Linters Locally (Known-Working Invocations)", a hub-only section read in a hub checkout rather than carried here, carries the known-working invocation of each linter, and CI runs the same set in [validate-task.yml](./.github/workflows/validate-task.yml), so a local lint run buys an earlier failure rather than a different one. **Linting is not editor-only here**: `validate-task.yml` runs markdownlint, CSpell, `actionlint`, and `editorconfig-checker` alongside the Husky style checks and `dotnet test`, all inside the required check. +The gates are the .NET clean-compile (the `.NET Format` VS Code task, per [CODESTYLE.md](./CODESTYLE.md)), the unit tests, and the document linters. `GOVERNANCE.md` "Running the Linters Locally (Known-Working Invocations)", a hub-only section read in a hub checkout rather than carried here, carries the known-working invocation of each linter, and CI runs the same set in the hub's `validate-task.yml`, so a local lint run buys an earlier failure rather than a different one. **Linting is not editor-only here**: `validate-task.yml` runs markdownlint, CSpell, `actionlint`, `editorconfig-checker`, `shellcheck`, `shfmt`, the composite-action schema check, and the hub's repo gates alongside `dotnet csharpier check`, `dotnet format style`, and `dotnet test`, all inside the required check. -**What CI structurally cannot exercise is the product image matrix.** The pull-request pipeline builds a deliberate smoke subset: NxMeta and NxMeta-LSIO, amd64 only, never pushed, and only when `Docker/**`, `Make/Matrix.json`, or `Make/Version.json` changed. Eight of the ten product images, the arm64 leg of every image, the base image push path, and a container that actually starts and serves its web UI are all unbuilt at merge time. The publish run is the first thing that builds them, and a workflow-only edit is deliberately not smoke-built at all. So a change to a Dockerfile, a build arg, a base image, or the generated matrix is verified locally by running `./Create.sh` and `./Build.sh` from inside `Make/`, which build every product image for both `linux/amd64` and `linux/arm64`, and, when the change can affect a running server, `./Test.sh` followed by `./Instructions.sh` there to reach each product's web UI. Reading a green pipeline as coverage of the full matrix is the mistake this section exists to name. +**What CI structurally cannot exercise is the product image matrix.** The pull-request pipeline builds a deliberate smoke subset: NxMeta and NxMeta-LSIO, amd64 only, never pushed, and only when `Docker/**`, `Make/Matrix.json`, `Make/Version.json`, a Docker hook, or a workflow changed. Eight of the ten product images, the arm64 leg of every image, the base image push path, and a container that actually starts and serves its web UI are all unbuilt at merge time. The publish run is the first thing that builds them, and a workflow edit reaches only the same smoke subset. So a change to a Dockerfile, a build arg, a base image, or the generated matrix is verified locally by running `./Create.sh` and `./Build.sh` from inside `Make/`, which build every product image for both `linux/amd64` and `linux/arm64`, and, when the change can affect a running server, `./Test.sh` followed by `./Instructions.sh` there to reach each product's web UI. Reading a green pipeline as coverage of the full matrix is the mistake this section exists to name. ## Runbooks @@ -55,7 +55,7 @@ A failure that appears only in a built image needs a running container instead. **Husky.Net runs the pre-commit hook.** `.husky/task-runner.json` defines the task set: `CSharpier Format` over the staged `.cs` files, then `.NET Format` running `dotnet format style --verify-no-changes`. `dotnet husky run` runs that set by hand. Matching this tooling, `.vscode/tasks.json` carries a Husky.Net Run task where the fleet template carries a Benchmark task. -**Workflow files get an editor check and a CLI check.** The GitHub Actions extension covers schema and expression checks while editing; run the `actionlint` CLI for the deeper checks, including shellcheck over `run:` steps. This matters more here than the lint gate alone suggests, because a workflow-only change is not smoke-built. +**Workflow files get an editor check and a CLI check.** The GitHub Actions extension covers schema and expression checks while editing; run the `actionlint` CLI for the deeper checks, including shellcheck over `run:` steps. This matters more here than the lint gate alone suggests, because the smoke build a workflow change triggers exercises only the pull request path, never the publisher's own jobs. ## Configuration Layout diff --git a/WORKFLOW.md b/WORKFLOW.md index 8db3ffa..3c1fe60 100644 --- a/WORKFLOW.md +++ b/WORKFLOW.md @@ -2,8 +2,8 @@ The single guide for this repo's CI/CD **workflows** (GitHub Actions): **code style**, **architecture**, a **behavioral contract** (expected inputs and outputs), and a **test methodology**. Source code style lives -in [`CODESTYLE.md`](./CODESTYLE.md). This file covers everything under -[`.github/workflows/`](./.github/workflows/). +in [`CODESTYLE.md`][codestyle]. This file covers everything under +[`.github/workflows/`][workflows]. It **describes required outcomes, not a required implementation.** A workflow is correct when it satisfies the contract (section 4), whatever shape its YAML takes. Section 2 keeps workflows legible. Section 3 is @@ -37,7 +37,11 @@ once their checks pass. - **Entry workflow** - has `push` / `schedule` / `workflow_dispatch` triggers. The orchestrator that an event or a person starts. - **Reusable workflow (task)** - a `workflow_call` workflow invoked through a `uses:` reference, never - triggered directly. File ends in `-task.yml`. + triggered directly. File ends in `-task.yml`. Every task this repo runs is **hub-hosted** in + `ptr727/ProjectTemplate` and reached by a SHA-pinned `uses:`, so this repo carries entry workflows only. +- **Hook** - a composite action under [`.github/actions/`][actions] that a hub task runs from this + repo's checkout for the repo-specific step: `docker-prepare` (the image matrix), `docker-build-base` (the + shared bases), and `codegen` (the generator invocation). - **Product image** - one shipped image built from a `Make/Matrix.json` row's Dockerfile (e.g. `NxMeta`, `NxMeta-LSIO`), pushed to its own Docker Hub repo (`docker.io/ptr727/`). - **Shared base image** - `nx-base` / `nx-base-lsio`, built once and reused as the `FROM` for the product @@ -48,8 +52,9 @@ once their checks pass. prove the Dockerfiles still build, publishing and pushing nothing. Driven by a `smoke: true` input. - **Transfer artifact** - a workflow artifact handing data between jobs of one run. The durable copy lives on the GitHub release / Docker Hub. -- **Threaded version** - the single NBGV `SemVer2` (plus `GitCommitId`) computed once in `get-version-task` - and passed down as `semver2` / `ref` inputs to every consumer, never recomputed in a build task. +- **Threaded version** - the single NBGV `SemVer2` (plus `GitCommitId`) computed once in the hub's + `get-version-task.yml`, which `build-release-task.yml` calls, and passed down as `semver2` / `ref` inputs to + every consumer, never recomputed in a build task. - **GitHub App token** - a short-lived installation token from `actions/create-github-app-token`, minted from the App credentials (`CODEGEN_APP_CLIENT_ID` / `CODEGEN_APP_PRIVATE_KEY`). The merge-bot and the codegen PR-opener use it, not `GITHUB_TOKEN`: a `GITHUB_TOKEN` push does not trigger downstream workflows, and that @@ -77,11 +82,15 @@ Legibility rules. Necessary but not sufficient: a perfectly styled workflow can - **Action pinning.** Pin every action to a commit SHA with a trailing `# vX.Y.Z` comment. Use `# vX` only when the upstream floating major tag has no specific patch SHA. **Sole exception: `dotnet/nbgv@master`** is consumed via the floating `@master` ref, never SHA-pinned - its tag stream lags `master` substantially, so - Dependabot tag-tracking would only propose downgrades to stale tags. The rationale lives in an inline - comment in [`get-version-task.yml`](./.github/workflows/get-version-task.yml); leave that comment intact. A - tool an action *installs* (not a `uses:` ref) is left unpinned to track latest. + Dependabot tag-tracking would only propose downgrades to stale tags. That `uses:` lives in the hub's + `get-version-task.yml`, with its rationale inline. A tool an action *installs* (not a `uses:` ref) is left + unpinned to track latest. +- **Hub task pinning.** A hub task is reached as + `ptr727/ProjectTemplate/.github/workflows/.yml@ # `, every call at the same + hub release, and Dependabot bumps the pin. A hub-side change reaches this repo only through a pin bump. - **Filename.** Reusable workflows end in `-task.yml`; entry workflows end in what they do - (`-pull-request.yml`, `-release.yml`). A `-task.yml` is `uses:`-d, never triggered directly. + (`-pull-request.yml`, `-release.yml`). A `-task.yml` is `uses:`-d, never triggered directly, and none is + carried here. - **Workflow `name:`.** Reusable names end in **"task"**, entry names in **"action"**. - **Job and step `name:`.** Every job `name:` ends in **"job"**, every step `name:` in **"step"**, the aggregator included (`Check pull request workflow status job`). A job name also bound as a ruleset @@ -100,33 +109,34 @@ Legibility rules. Necessary but not sufficient: a perfectly styled workflow can job needs valid permissions. Grant least privilege; a callee's extra scope is granted by the caller. - **Allowlist `success` and `skipped` explicitly** across an optional dependency: use `(needs.X.result == 'success' || needs.X.result == 'skipped')`, not `!= 'failure'`. -- **Line endings.** Workflow YAML is LF, per [`.editorconfig`](./.editorconfig)'s `[*]` default (Actions and Dependabot rewrite it that way). Preserve endings on every edit. +- **Line endings.** Workflow YAML is LF, per [`.editorconfig`][editorconfig]'s `[*]` default (Actions and Dependabot rewrite it that way). Preserve endings on every edit. ## 3. Architecture ### Three workflows: CI on push, publishing on schedule/pin-push/dispatch, codegen daily -CI ([`test-pull-request.yml`](./.github/workflows/test-pull-request.yml)) and the publisher -([`publish-release.yml`](./.github/workflows/publish-release.yml)) are separate workflows with separate +CI ([`test-pull-request.yml`][test-pull-request]) and the publisher +([`publish-release.yml`][publish-release]) are separate workflows with separate concurrency, so they never race. CI re-tests every pushed tree and never publishes; the publisher releases on its own triggers and never runs on an ordinary merge. Codegen -([`run-periodic-codegen-pull-request.yml`](./.github/workflows/run-periodic-codegen-pull-request.yml) -> -[`run-codegen-pull-request-task.yml`](./.github/workflows/run-codegen-pull-request-task.yml)) keeps the -version/matrix data current. *Prevents a merge from silently cutting a release, and a CI run from racing a +([`run-periodic-codegen-pull-request.yml`][run-periodic-codegen-pull-request] -> the +hub's `run-codegen-pull-request-task.yml` -> the [`codegen`][actions-codegen] hook) keeps +the version/matrix data current. *Prevents a merge from silently cutting a release, and a CI run from racing a publish on the same ref.* ### The publisher builds one branch: the trigger ref A publish builds exactly **one** branch - the run's trigger ref. The **schedule** and the **pin push** both run on `main`; a **dispatch** runs on the branch it is started from (`main` or `develop`). The jobs pass -`github.ref_name` as both `ref` and `branch`, so the branch built, versioned, and tagged is always the run's -own ref. *No matrix and no cross-branch ref mixing - `github.ref` is the branch being published.* The jobs -are guarded to the long-lived branches (`main` / `develop`); a stray dispatch from a feature branch is a -no-op. To refresh `:develop`, dispatch the workflow from `develop`. +`github.ref_name` as `branch` and the triggering commit `github.sha` as `ref`, so the branch built, versioned, +and tagged is always the run's own ref, pinned to one commit even if the branch advances mid-run. *No matrix and no cross-branch ref mixing - `github.ref` is the branch being published.* The hub's +`publish-plan-task.yml` publishes only the long-lived branches (`main` / `develop`); a dispatch from a feature +branch fails the plan job with an `::error::` and publishes nothing. To refresh `:develop`, dispatch the +workflow from `develop`. -Because the run's ref **is** the built branch, GitHub resolves the local `uses: ./...` reusable workflows -from that same branch's commit - so a `develop` dispatch runs develop's own task definitions, and the -schedule runs main's. +Because the run's ref **is** the built branch, the hooks run from that same branch's commit - so a `develop` +dispatch builds develop's own `Make/Matrix.json` rows with develop's hooks, and the schedule runs main's. The +hub tasks themselves resolve at the pinned hub commit on every branch. ### The publisher's pin-push trigger @@ -139,21 +149,25 @@ base image for CVEs. ### The multi-image build layer -The publisher decomposes into a single `get-version` -> `build-base` -> `build-docker` -> -`github-release` -> `docker-readme` -> `cleanup-artifacts` chain (a multi-product Docker repo, not the -template's single-target branch matrix). [`build-base-images-task.yml`](./.github/workflows/build-base-images-task.yml) -builds the two shared bases; [`build-docker-task.yml`](./.github/workflows/build-docker-task.yml) builds the -product matrix from `Make/Matrix.json` (`max-parallel: 4`). The shared base is built **once** (on the `main` -run) and reused: a `develop` dispatch sets `build_base: false` and pulls main's published base, so it never -overwrites the branch-agnostic `nx-base` tag. The product build reads both branches' registry buildcaches -(`buildcache-main`, `buildcache-develop`) and writes only its own branch's cache, only when pushing. +The publisher is a `plan` -> `validate` -> `build-base` -> `publish` -> `publish-docker-readme` chain (a +multi-product Docker repo, the matrix and `build-base` case of the hub's Docker family). The `build-base` job +logs in to Docker Hub and runs the [`docker-build-base`][actions-docker-build-base] hook +to build and push the two shared bases. It is this repo's own job rather than the hub's `build-base` leg, +since that leg carries no Docker Hub login and a composite hook cannot read secrets. The `publish` job calls +the hub's `build-release-task.yml`, whose Docker leg (the hub's `build-docker-task.yml`) takes the product +matrix from the [`docker-prepare`][actions-docker-prepare] hook, which maps the +branch's `Make/Matrix.json` rows onto the hub's matrix shape. The shared base is built **once** (on the +`main` run) and reused: a `develop` dispatch skips `build-base` and pulls main's published base, so it never +overwrites the branch-agnostic `nx-base` tag. The hub core owns the cache policy: the product build reads both +branches' registry buildcaches (`buildcache-main`, `buildcache-develop`) and writes only its own branch's +cache, only when pushing. ### Versioning: compute once, thread everywhere -NBGV runs once (in [`get-version-task.yml`](./.github/workflows/get-version-task.yml)), classifying from +NBGV runs once (in the hub's `get-version-task.yml`, called by `build-release-task.yml`), classifying from `github.ref`, and its outputs (`SemVer2`, `GitCommitId`) thread to every consumer via `outputs:` / `needs:` / -`semver2` inputs. `build-docker-task` accepts the threaded `semver2` as the image `LABEL_VERSION` and never -re-runs NBGV - one classification feeds every product leg, so no second NBGV run can reclassify or collide a +`semver2` inputs. The hub's `build-docker-task.yml` accepts the threaded `semver2` as the image +`LABEL_VERSION` and never re-runs NBGV - one classification feeds every product leg, so no second NBGV run can reclassify or collide a tag. A build job may check out a specific commit to compile it (main pins to `GitCommitId`) but consumes the threaded version. `main` (the public ref, `publicReleaseRefSpec = ^refs/heads/main$`) builds a clean `X.Y.`; every other branch a prerelease `X.Y.-g`. *Keeps each image's embedded version @@ -171,41 +185,48 @@ NBGV build version. ### Validate at entry -A run that carries a cross-input invariant (`main` must not carry a prerelease suffix) asserts it once with -`::error::` before the release is published. The `github-release` job `needs:` the version job and runs the -backstop step first. +A run that carries a cross-input invariant (`main` must not carry a prerelease suffix, and any other branch +must carry one) asserts it once with `::error::` before anything builds. The hub's `build-release-task.yml` +runs that check in its `validate-release` job, which every build job `needs:`. ### Fast CI feedback, head-resolved -CI runs on push to every branch, so GitHub head-resolves the reusable `./...` workflows from the pushed head: -a pull request that edits a reusable task tests its own copy. CI validates (the reusable `validate-task`: -Husky lint + `dotnet test`) on every push, and smoke-builds a representative image subset only when image -files changed (an inline `git diff` change-gate, no `dorny/paths-filter`), uploading and pushing nothing. One +CI runs on push to every branch, and the hooks resolve from the pushed head, so a pull request that edits a +hook tests its own copy. The hub tasks resolve at their pinned commit, so a hub-side change is tested by the +pull request that bumps the pin, which touches `.github/workflows/` and so runs the smoke build too. CI validates (the hub's `validate-task.yml`: the document linters, CSharpier +and `dotnet format style`, the repo gates, and `dotnet test`) on every push, and smoke-builds a +representative image subset only when image files, the Docker hooks, or the workflows changed (an inline `git diff` +change-gate, no `dorny/paths-filter`), through the hub's `build-release-task.yml` with `smoke: true`, +uploading and pushing nothing. One aggregator job, the ruleset-bound required check, gates the merge. A branch-deletion push (all-zeros `github.sha`) is skipped by a `!github.event.deleted` guard on every job, so a deletion never runs a failing build. ### The Docker-only release -The `github-release` job tags the built commit and creates the GitHub release (auto source zip + README + -LICENSE; `target_commitish` pinned to `GitCommitId`; skip-existing guard; main-only). This repo ships **no** -`release-asset-*` binaries or packages - the published artifacts are the Docker Hub images - so there is no -release-asset download step and `fail_on_unmatched_files` is omitted. The GitHub release exists only as the -version anchor / tag. The Docker Hub repository overview (the repo `README.md`) is pushed to every product + -base repo on a `main` Docker publish (the `docker-readme` jobs), since Docker Hub does not read the GitHub -README; the repo list is derived inline from `Make/Matrix.json`. +The hub's `github-release` job tags the built commit and creates the GitHub release (auto source zip + +README + LICENSE; `target_commitish` pinned to `GitCommitId`; skip-existing guard). The caller sets +`github: true` only on `main`, so a `develop` dispatch cuts no release. This repo ships **no** +`release-asset-*` binaries or packages - the published artifacts are the Docker Hub images - so it sets +`expect_release_assets: false`, which skips the release-asset download and relaxes +`fail_on_unmatched_files`. The GitHub release exists only as the version anchor / tag. The Docker Hub +repository overview ([`Docker/README.md`][docker-readme], the hub default's first choice) is pushed to +every product + base repo on a `main` publish by the hub's `publish-docker-readme-task.yml`, since Docker +Hub does not read the GitHub README; the repo list is derived from `Make/Matrix.json` by the task's +`manifest-jq` input. ### Resource lifecycle -Workflow artifacts are an intra-run handoff; the durable copy lives on the GitHub release / Docker Hub. The -publisher and CI both run a terminal `cleanup-artifacts` job that deletes the run's transfer artifacts so -they do not accumulate against the small account-wide storage quota; it is `continue-on-error` so housekeeping -never reds the run. +Workflow artifacts are an intra-run handoff; the durable copy lives on the GitHub release / Docker Hub. This +repo uploads none: the images go straight to Docker Hub and the release carries no `release-asset-*` files. +The hub's `github-release` job still owns the consume-then-delete cleanup, `continue-on-error` so +housekeeping never reds the run, should a target that uploads one ever be enabled. ### Self-sufficiency: automatic updates -Every Dependabot pull request, any ecosystem and any tier, auto-merges once the required checks pass, except -a **semver-major NuGet** bump, which waits for human review. Codegen opens a `codegen-main` -> `main` and a +Every Dependabot pull request, any ecosystem and any tier, auto-merges once the required checks pass, a +semver-major NuGet bump included, since the required checks are the gate (D8.2). The merge-bot is the hub's +`merge-bot-task.yml`. Codegen opens a `codegen-main` -> `main` and a `codegen-develop` -> `develop` PR daily; the merge-bot auto-merges each independently (`--delete-branch`). A merged dependency bump does not itself publish; a merged matrix pin on `main` does (the pin-push trigger). A person steps in only for a breaking change (a red check) or to dispatch a release. @@ -218,8 +239,7 @@ the same outcomes that the section 4 contract specifies, drawn from the workflow a guarantee disagree, one of them is a defect. Triggers are blue, gates yellow, durable/published outputs green, and stop/skip outcomes red. -**Pull request (CI) - `test-pull-request.yml`.** Every push head-resolves the reusable tasks, runs the -validate gate, smoke-builds a representative image subset only when image files changed, and a single +**Pull request (CI) - `test-pull-request.yml`.** Every push runs the hub's validate gate, smoke-builds a representative image subset only when image files changed, and a single aggregator produces the ruleset-bound required check (D1, D6). ```mermaid @@ -228,14 +248,14 @@ flowchart TD T --> D{"github.event.deleted?"}:::gate D -- "yes: branch deletion" --> X(["all jobs + aggregator skip
no failed run, no pending check"]):::stop D -- "no" --> CH["changes job
inline git diff change-gate
image? base?"] - D -- "no" --> V["validate job
(validate-task.yml)"] - subgraph VT ["validate-task.yml"] - VU["Husky lint (CSharpier,
dotnet format style)
+ dotnet test"] + D -- "no" --> V["validate job
(hub validate-task.yml)"] + subgraph VT ["hub validate-task.yml"] + VU["doc linters, CSharpier,
dotnet format style, repo gates
+ dotnet test"] end V --> VT - CH --> SG{"image files changed?
(Docker/**, Make/Matrix.json, Make/Version.json)"}:::gate + CH --> SG{"image files changed?
(Docker/**, Make/Matrix.json, Make/Version.json,
.github/actions/docker-*, .github/workflows/**)"}:::gate SG -- "no" --> SS(["smoke-build skipped
(aggregator allows skip)"]):::stop - SG -- "yes" --> S["smoke-build job
build-docker-task.yml
smoke: true, push: false
NxMeta + NxMeta-LSIO, amd64"] + SG -- "yes" --> S["smoke-build job
hub build-release-task.yml
smoke: true, never pushes
NxMeta + NxMeta-LSIO, amd64"] CH --> A VT --> A S --> A @@ -250,37 +270,37 @@ flowchart TD ``` **Publish - `publish-release.yml`.** A weekly schedule (main), a `Make/Matrix.json` pin push (main), or a -dispatch versions once with NBGV, builds the shared base (main only), validates, builds the 12-image -product matrix with the threaded SemVer2, then cuts the main-only GitHub release and refreshes the Docker -Hub overviews (D0, D2, D3, D4). +dispatch plans, validates, builds the shared base (main only), then calls the hub release chain, which +versions once with NBGV, builds the branch's product matrix with the threaded SemVer2, and cuts the main-only +GitHub release; the Docker Hub overviews refresh last (D0, D2, D3, D4). ```mermaid flowchart TD - P1(["schedule: weekly Mon 02:00 UTC
(main only)"]):::trig --> GV - P2(["push: main
paths = Make/Matrix.json (codegen pin)"]):::trig --> GV - P3(["workflow_dispatch
(main or develop)"]):::trig --> GV - GV{"get-version job
ref_name in (main, develop)?"}:::gate - GV -- "feature branch" --> GVS(["all jobs skip
no publish"]):::stop - GV -- "yes" --> GVR["get-version job
(get-version-task.yml)
NBGV @master, runs once
SemVer2 + GitCommitId"] - GVR --> BB{"ref_name == main?"}:::gate + P1(["schedule: weekly Mon 02:00 UTC
(main only)"]):::trig --> PL + P2(["push: main
paths = Make/Matrix.json (codegen pin)"]):::trig --> PL + P3(["workflow_dispatch
(main or develop)"]):::trig --> PL + PL{"plan job (hub publish-plan-task.yml)
publish? stable?"}:::gate + PL -- "no: human push, or
feature-branch dispatch" --> PLS(["all jobs skip or plan fails
no publish"]):::stop + PL -- "publish" --> VAL["validate job
(hub validate-task.yml)"] + VAL --> BB{"stable (main)?"}:::gate BB -- "develop dispatch" --> BBS(["build-base skipped
reuse main's nx-base"]):::stop - BB -- "main" --> BBJ["build-base job
(build-base-images-task.yml)
nx-base + nx-base-lsio
amd64 + arm64, branch ref (github.ref_name)"] - GVR --> VAL["validate job
(validate-task.yml)
main: pinned to GitCommitId"] - VAL --> BD - BBJ --> BD - BBS --> BD - BD["build-docker job
(build-docker-task.yml, build_base: false)
12-image matrix from Make/Matrix.json
amd64 + arm64, max-parallel 4
LABEL_VERSION = threaded SemVer2"] - BD --> DH[("Docker Hub
10 product repos (branch tags)
+ 2 shared base repos")]:::pub - BD --> RG{"ref_name == main?"}:::gate + BB -- "main" --> BBJ["build-base job
docker login + docker-build-base hook
nx-base + nx-base-lsio
amd64 + arm64, buildcache-main"] + BBJ --> PUB + BBS --> PUB + subgraph PUB ["publish job: hub build-release-task.yml"] + GVR["get-version
NBGV @master, runs once
SemVer2 + GitCommitId"] --> VR{"validate-release
main: no prerelease '-'
other: has one"}:::gate + VR -- "clean" --> BD["build-docker (hub build-docker-task.yml)
matrix from the docker-prepare hook
= the branch's Make/Matrix.json rows
amd64 + arm64 on main
LABEL_VERSION = threaded SemVer2"] + BD --> RG{"github: true (main)?"}:::gate + end + VR -- "mismatch" --> VRX(["fail ::error::
refuse to publish"]):::stop + BD --> DH[("Docker Hub
10 product repos (branch tags)")]:::pub + BBJ --> DHB[("Docker Hub
2 shared base repos")]:::pub RG -- "develop" --> RGS(["no GitHub release
(:develop images only)"]):::stop - RG -- "main" --> VPR{"github-release job
SemVer2 has no prerelease '-'?
(strip +buildmetadata)"}:::gate - VPR -- "prerelease suffix" --> VPRX(["fail ::error::
refuse to publish"]):::stop - VPR -- "clean" --> EX{"tag exists AND not dispatch?"}:::gate + RG -- "main" --> EX{"tag exists AND not dispatch?"}:::gate EX -- "yes" --> EXS(["skip release create
(no-op republish)"]):::stop EX -- "no" --> REL[("GitHub release
tag = SemVer2 at GitCommitId
prerelease: false, source zip + README + LICENSE")]:::pub - BD --> DRR["docker-readme-repos job
derive repo list from Matrix.json"] - DRR --> DRM["docker-readme job (matrix)
push README to each Docker Hub repo"] - DRM --> DRO[("Docker Hub overviews
10 product + 2 base repos")]:::pub + PUB --> DRM["publish-docker-readme job (main)
hub publish-docker-readme-task.yml
repo list from Make/Matrix.json"] + DRM --> DRO[("Docker Hub overviews (Docker/README.md)
10 product + 2 base repos")]:::pub classDef trig fill:#dbeafe,stroke:#2563eb,color:#1e3a8a classDef gate fill:#fef9c3,stroke:#ca8a04,color:#713f12 classDef pub fill:#dcfce7,stroke:#16a34a,color:#14532d @@ -294,21 +314,19 @@ merge-bot enables auto-merge (or disables it on a maintainer push); the required ```mermaid flowchart TD SCH(["schedule daily 04:00 UTC
(or workflow_dispatch)"]):::trig --> CG - subgraph CGT ["run-codegen-pull-request-task.yml (matrix: main, develop)"] - CG["codegen job per branch
regenerate Version.json + Matrix.json
(deterministic, forward-only guard)"] --> CGC{"data changed?"}:::gate + subgraph CGT ["hub run-codegen-pull-request-task.yml (matrix: main, develop)"] + CG["codegen job per branch
codegen hook: regenerate Version.json + Matrix.json
(deterministic, forward-only guard)"] --> CGC{"data changed?"}:::gate CGC -- "no" --> CGN(["no PR"]):::stop CGC -- "yes" --> CPR["open codegen-<branch> PR
(App token)"] end DEP(["Dependabot opens PR
any ecosystem/tier"]):::trig --> MB CPR --> MB - subgraph MBT ["merge-bot-pull-request.yml (pull_request_target)"] + subgraph MBT ["merge-bot-pull-request.yml -> hub merge-bot-task.yml (pull_request_target)"] MB{"event / author"}:::gate MB -- "opened/reopened
bot author" --> EN["enable auto-merge --delete-branch
squash develop / merge main"] MB -- "synchronize by maintainer" --> DIS["disable auto-merge"] end - EN --> SM{"semver-major NuGet?"}:::gate - SM -- "yes" --> HUM(["wait for human review"]):::stop - SM -- "no" --> CK{"required check passes?"}:::gate + EN --> CK{"required check passes?"}:::gate CK -- "yes" --> MRG(["PR merges (App token)"]):::pub CK -- "no" --> BLK(["merge blocked
maintainer notified"]):::stop MRG -. "codegen-main Matrix.json change" .-> PUBR(["publisher pin-push auto-publishes main"]):::pub @@ -332,7 +350,7 @@ flowchart TD DCH -- "yes: develop" --> PRD["codegen-develop -> develop PR
(merge-bot auto-merges)"] PRD --> SYNC(["develop Matrix.json updated
sync-only, push trigger is main-only
:develop refreshed by dispatch"]):::stop PRM --> PUSH(["push to main
paths = Make/Matrix.json"]):::trig - PUSH --> PUB["publish-release.yml
pin-push: build base + 12-image matrix"] + PUSH --> PUB["publish-release.yml
pin-push: build base + main's product matrix"] PUB --> SINK[("Docker Hub product + base images
+ main GitHub release")]:::pub SCHED(["weekly schedule (main)
base refresh for CVEs"]):::trig --> PUB DISP(["workflow_dispatch (main or develop)
force publish that branch"]):::trig --> PUB @@ -351,10 +369,12 @@ Each is a **MUST**, stated as input -> output plus the failure it prevents. - **D0.1 CI is one run, one branch.** Input: any push. Output: `test-pull-request` builds/validates exactly `github.ref_name` and publishes nothing. *Prevents cross-branch ref mixing in CI.* - **D0.2 The publisher builds one branch: the trigger ref.** Output: the publisher passes `github.ref_name` - as `ref` and `branch`, so it checks out, versions, and tags exactly the run's own branch (the schedule/pin - push's `main`, or a dispatch's branch). No branch matrix; the jobs are guarded to `main`/`develop`. + as `branch` and the triggering `github.sha` as `ref`, so it checks out, versions, and tags exactly the run's own branch (the schedule/pin + push's `main`, or a dispatch's branch). No branch matrix; the hub's `publish-plan-task.yml` publishes only + `main`/`develop`. *Prevents cross-branch ref mixing - `github.ref` is the branch being published.* -- **D0.3 One version, threaded.** Output: NBGV runs once (`get-version-task`); every consumer reads it via +- **D0.3 One version, threaded.** Output: NBGV runs once (the hub's `get-version-task.yml`, inside + `build-release-task.yml`); every consumer reads it via `needs:` outputs / the `semver2` input; no consumer recomputes it. *Allowed:* checking out a specific commit to compile it, and recording the built commit as the release `target_commitish`. *Prevents an image's embedded version diverging from its tag, and a second NBGV run colliding image tags.* @@ -362,15 +382,19 @@ Each is a **MUST**, stated as input -> output plus the failure it prevents. ### D1 - CI fast feedback - **D1.1 Every push validates; image changes smoke-build.** Output: on any push the `validate` job (the - reusable `validate-task`) runs with no paths filter; `smoke-build` (NxMeta + NxMeta-LSIO, amd64, no push) + hub's `validate-task.yml`) runs with no paths filter; `smoke-build` (NxMeta + NxMeta-LSIO, amd64, no push) runs when the inline change-gate detects an image-file change (`Docker/**`, `Make/Matrix.json`, - `Make/Version.json`). *Prevents a reusable-workflow or Dockerfile break shipping untested.* -- **D1.2 Unit tests always run.** Output: `validate-task` runs `dotnet test` (the codegen tool + its tests). -- **D1.3 Lint enforces the editor checks in CI.** Output: `validate-task` runs Husky (`dotnet husky run`: - CSharpier + `dotnet format style --verify-no-changes`) - the same checks the editor and the pre-commit - hook run. Workflow YAML is lint-checked by `actionlint` (run from the editor / locally). + `Make/Version.json`, `.github/actions/docker-*`, `.github/workflows/**`, the last covering a hub pin bump). + *Prevents a hook, Dockerfile, or hub-task break shipping untested.* +- **D1.2 Unit tests always run.** Output: the hub's `validate-task.yml` runs `dotnet test` (the codegen tool + + its tests). +- **D1.3 Lint enforces the editor checks in CI.** Output: the hub's `validate-task.yml` runs `dotnet + csharpier check` and `dotnet format style --verify-no-changes` - the same checks the editor and the Husky + pre-commit hook run - plus the document linters, `actionlint`, the composite-action schema check, + `shellcheck`, `editorconfig-checker`, and the hub's repo gates. - **D1.4 Smoke never publishes and never uploads a release asset.** Output: a smoke build compiles the image - subset but makes no GitHub release and no Docker push (`push: false`, `smoke: true`). The Docker login runs + subset but makes no GitHub release and no Docker push (`smoke: true`, which disables every push in the hub's + `build-release-task.yml`, and `github: false`). The Docker login runs on every build including smoke (for higher pull/cache rate limits against the registry buildcache), so a Dependabot-triggered push-CI smoke build needs the Docker Hub credentials in both secret stores; fork PRs do not run this push-CI. @@ -381,13 +405,13 @@ Each is a **MUST**, stated as input -> output plus the failure it prevents. ### D2 - Validation at entry -- **D2.1 Validate the cross-input invariant before publishing.** Output: the `github-release` job asserts each - cross-input invariant with `::error::` (the main-version backstop, D2.2) before the release is created; - downstream steps `needs:` the version job. The publish run also re-runs `validate-task` (the `validate` job - gates `build-docker`), so a publish can never ship a tree that would fail the same lint + `dotnet test` gate +- **D2.1 Validate the cross-input invariant before publishing.** Output: the hub's `validate-release` job + asserts each cross-input invariant with `::error::` (the version backstop, D2.2) before anything builds; + every build job `needs:` it. The publish run also re-runs the hub's `validate-task.yml` (the `validate` job + gates `build-base` and `publish`), so a publish can never ship a tree that would fail the same lint + `dotnet test` gate CI enforces on push - on the trigger branch CI already validated, re-checked at publish time. -- **D2.2 Main matches version classification.** Input: a real publish run for `main`. Output: the release - fails loudly if `main` carries a prerelease suffix. It strips `+buildmetadata` before testing for the +- **D2.2 Main matches version classification.** Input: a real publish run. Output: the run fails loudly if + `main` carries a prerelease suffix, or if any other branch carries none. It strips `+buildmetadata` before testing for the prerelease `-`. *Prevents a develop build published as the stable `latest`.* ### D3 - Versioning and classification @@ -400,15 +424,15 @@ Each is a **MUST**, stated as input -> output plus the failure it prevents. `X.Y.Z-g`. The release-version backstop names `main`; `publicReleaseRefSpec` is `^refs/heads/main$`. - **D3.3 Version floor + git height.** Output: `version.json` sets the major.minor floor, NBGV appends the git height as the patch, never bumped on a cadence. *(Who raises the floor and when is a human-process rule in - [`GOVERNANCE.md` "Release Model"](./GOVERNANCE.md#release-model).)* + [`GOVERNANCE.md` "Release Model"][governance-release-model].)* ### D4 - Release / publish - **D4.1 Publish only on schedule, the matrix-pin push, or dispatch - never on an ordinary merge.** Output: `publish-release` triggers are `schedule` (weekly), `workflow_dispatch`, and a `push` **branch-filtered to `main` and path-filtered to `Make/Matrix.json`**. There is no other `push` trigger and no `PUBLISH_ON_MERGE` - variable. The jobs are guarded to `github.ref_name` in (`main`, `develop`), so a stray dispatch from a - feature branch is a no-op. *Prevents per-merge release churn while still shipping a new product pin at once.* + variable. The hub's `publish-plan-task.yml` publishes only `main` and `develop`, and fails a dispatch from a + feature branch with an `::error::`, publishing nothing. *Prevents per-merge release churn while still shipping a new product pin at once.* - **D4.2 A publish builds the one trigger branch in full.** Output: the run builds the shared base (main run only) + the full product matrix and creates the GitHub release for `github.ref_name` - the schedule/pin push rebuilds `main` (stable / `latest`); a dispatch publishes its own branch (`main` stable / `latest`, @@ -418,24 +442,27 @@ Each is a **MUST**, stated as input -> output plus the failure it prevents. `main`), and main's images are built from that pinned commit, never the moving ref. *Prevents the tag / embedded version landing on the wrong commit.* - **D4.4 Release contents and gate.** Output: a release is a tag on the built commit plus the auto source zip, - README, and LICENSE - no binary release assets (`fail_on_unmatched_files` omitted; the images are the - artifact). The release is `main`-only (`github.ref_name == 'main'`); `prerelease: false`. *Prevents + README, and LICENSE - no binary release assets (`expect_release_assets: false`; the images are the + artifact). The release is `main`-only (`github: true` only when the plan says stable); `prerelease: false`. *Prevents publishing a develop build as a stable GitHub release.* - **D4.5 No-op republish.** Input: a weekly re-run whose version is unchanged. Output: the release-create step is skipped when the tag already exists (refreshed only on `workflow_dispatch`), while the Docker push still runs - re-pushing the same tags refreshes the shared base image. *Prevents duplicate releases while still refreshing the images.* - **D4.6 Publish is built from the tree CI validated.** Output: the run's ref **is** the published branch, so - the reusable-task definitions and the built tree resolve from that branch - the same tree CI validated on - push (the required check gates every merge to it) with the identical `validate-task` definition - and the publish run re-runs that `validate-task` (the - `validate` job gates `build-docker`). The main-version backstop (D2.2) is the additional in-publisher gate. -- **D4.7 Docker publishing authenticates with Docker Hub credentials.** Output: the base and product builds - log in via `docker/login-action` with `DOCKER_HUB_USERNAME` + `DOCKER_HUB_ACCESS_TOKEN` and push with - `docker/build-push-action`; the Docker Hub overview is pushed with the same token. There is **no NuGet/OIDC + the hooks and the built tree resolve from that branch - the same tree CI validated on push (the required + check gates every merge to it) - and the hub tasks resolve at the same pinned commit CI used. The publish + run re-runs the hub's `validate-task.yml` (the `validate` job gates `build-base` and `publish`). The version + backstop (D2.2) is the additional in-publisher gate. +- **D4.7 Docker publishing authenticates with Docker Hub credentials.** Output: the publisher's `build-base` + job and the hub's product build each log in via `docker/login-action` with `DOCKER_HUB_USERNAME` + + `DOCKER_HUB_ACCESS_TOKEN` and push with `docker/build-push-action`; the hub's readme task pushes the Docker + Hub overview with the same token, mapped by name to each hub task. There is **no NuGet/OIDC publishing** in this repo. *Prevents a missing-credential publish failure.* -- **D4.8 Branch-scoped Docker buildcache.** Output: the base and product builds read both branches' registry - caches (`buildcache-main`, `buildcache-develop`) and write only their own branch's cache, only when pushing, - so a `main` and a `develop` publish never overwrite each other's cache. *Prevents one branch's publish +- **D4.8 Branch-scoped Docker buildcache.** Output: the product build reads both branches' registry caches + (`buildcache-main`, `buildcache-develop`) and writes only its own branch's cache, only when pushing, so a + `main` and a `develop` publish never overwrite each other's cache. The shared base, pushed only by a `main` + publish, reads and writes `buildcache-main` alone. *Prevents one branch's publish destroying the other's cache hit-rate.* - **D4.9 Multi-arch, multi-product, shared-base fan-out.** Output: the publish builds every product image from `Make/Matrix.json` for `linux/amd64` + `linux/arm64`, on the shared base built once and reused; each product @@ -444,9 +471,10 @@ Each is a **MUST**, stated as input -> output plus the failure it prevents. ### D5 - Resource cleanup -- **D5.1 Terminal cleanup, best-effort.** The publisher and CI each run a terminal `cleanup-artifacts` job - (`always()`, `continue-on-error`) that deletes the run's transfer artifacts, independent of the required - aggregator so housekeeping never gates the merge. +- **D5.1 No dangling transfer artifacts.** Output: neither the publisher nor CI uploads a transfer + artifact - the images push straight to Docker Hub and the release carries no `release-asset-*` files - so a + run leaves none behind. The hub's `github-release` job owns the consume-then-delete cleanup + (`continue-on-error`) for a target that does upload one. - **D5.2 Never red the run on cleanup.** Cleanup failures warn, never fail. ### D6 - Self-testing workflows @@ -454,7 +482,7 @@ Each is a **MUST**, stated as input -> output plus the failure it prevents. - **D6.1 A change is testable on its own branch.** Output: a workflow or build change is exercised by CI on the branch that introduces it, no dependency on reaching `main` first. - **D6.2 Head-resolution, single producer, fork exception.** Output: CI runs on `push` to every branch so - reusable `./...` logic resolves from the head, and the aggregator's ruleset-bound `context:` is produced by + the `./.github/actions/` hooks resolve from the head (the hub tasks resolve at their pin), and the aggregator's ruleset-bound `context:` is produced by that push run as the sole producer of that name. Dependabot and codegen PRs are in-repo branches, validated the same way. A fork cannot push, so it has no run and is validated by maintainer action - the one exception. *Prevents a dual-producer context race and a false self-test claim for forks.* @@ -473,15 +501,16 @@ Each is a **MUST**, stated as input -> output plus the failure it prevents. ### D8 - Bots and automation -- **D8.1 Merge-bot.** Output: runs on `pull_request_target`, holds the App token, merges the PR by URL without - checking out its code. Enables auto-merge on `opened`/`reopened` with `--delete-branch`; squash on +- **D8.1 Merge-bot.** Output: a thin caller onto the hub's `merge-bot-task.yml`, run on + `pull_request_target`, holding the App token, merging the PR by URL without checking out its code. Enables + auto-merge on `opened`/`reopened` with `--delete-branch` (the `delete-branch: true` input); squash on `develop`, merge-commit on `main` by the PR's base ref; disables auto-merge when a maintainer pushes to a bot branch (no `--delete-branch` on the disable path). Concurrency keyed on PR number. -- **D8.2 Dependabot auto-merges on green, semver-major NuGet excepted.** Output: every Dependabot PR - auto-merges once the required checks pass, except a semver-major NuGet bump (human review). A failing check - blocks the merge. A merged dependency bump does **not** itself publish (it ships in the next scheduled run +- **D8.2 Dependabot auto-merges on green.** Output: every Dependabot PR, any ecosystem and any tier, + auto-merges once the required checks pass. A failing check blocks the merge. A merged dependency bump does **not** itself publish (it ships in the next scheduled run or the next matrix-pin push). -- **D8.3 Codegen dual-targets `main` AND `develop`.** Output: the daily codegen matrix opens a +- **D8.3 Codegen dual-targets `main` AND `develop`.** Output: the daily codegen matrix (the hub's + `run-codegen-pull-request-task.yml`, running the `codegen` hook) opens a `codegen-main` -> `main` and a `codegen-develop` -> `develop` PR (strict head/base pairing in the merge-bot), regenerating `Make/Version.json` + `Make/Matrix.json` **only** (not the Dockerfiles - those are a separate human-driven `Make/Create.sh` path). The merge-bot auto-merges each independently. develop's matrix update @@ -496,17 +525,17 @@ Each is a **MUST**, stated as input -> output plus the failure it prevents. - **D9.1** Every action SHA-pinned with a version comment (sole exception: `dotnet/nbgv@master`, whose tag stream lags master so Dependabot tag-tracking would only propose downgrades to stale tags - a deliberate - documented float, rationale inline in `get-version-task.yml`); an installed-tool version is left unpinned to + documented float, rationale inline in the hub's `get-version-task.yml`); an installed-tool version is left unpinned to track latest. - **D9.2** File/workflow/job/step names follow the suffix rules; a ruleset-bound `context:` name moves only in lockstep with the live ruleset and the hub payload it is applied from. - **D9.3** Bash `run:` blocks start `set -Eeuo pipefail`; multi-line `if:` uses `>-`. - **D9.4** Line endings follow `.editorconfig`. -- **D9.5 No decorative / dropped workflows.** No date-badge (`build-datebadge-*`), no standalone docker-readme - task (folded into the publisher), no `PUBLISH_ON_MERGE` variable, no `dorny/paths-filter` (replaced by the +- **D9.5 No decorative / dropped workflows.** No date-badge (`build-datebadge-*`), no carried `-task.yml` + (every task is hub-hosted), no `PUBLISH_ON_MERGE` variable, no `dorny/paths-filter` (replaced by the inline change-gate). Their presence is a defect to remove. -- **D9.6** Style is enforced in CI by `validate-task` (D1.3), from the same config files the editor and Husky - hook use. +- **D9.6** Style is enforced in CI by the hub's `validate-task.yml` (D1.3), from the same config files the + editor and Husky hook use. ### D10 - Repository configuration @@ -521,34 +550,35 @@ Each is a **MUST**, stated as input -> output plus the failure it prevents. Read the workflow files plus `version.json` and `Make/Matrix.json` and assert the fact behind each applicable guarantee with a `file:line` citation: -- **D0:** CI has no branch matrix; the publisher passes `github.ref_name` as `ref`/`branch` and is guarded to - `main`/`develop`; NBGV invoked once in `get-version-task`, `build-docker-task` has no nested `get-version` - and consumes the `semver2` input; the run builds the trigger ref so `GITHUB_REF` matches the versioned +- **D0:** CI has no branch matrix; the publisher passes `github.ref_name` as `branch` and `github.sha` as + `ref`, and the plan task publishes only `main`/`develop`; NBGV invoked once in the hub's + `build-release-task.yml`, whose Docker leg consumes the threaded `semver2`; the run builds the trigger ref so `GITHUB_REF` matches the versioned branch. - **D1:** CI runs on `push` with no paths filter on `validate`; the inline `changes` gate sets `image`/`base`; - `smoke-build` is `smoke: true`, `push: false`, amd64 NxMeta subset; the aggregator `needs:` `changes` + + `smoke-build` is `smoke: true`, `github: false`, `dockerhub: false`, the NxMeta subset named by + `docker_image`; the aggregator `needs:` `changes` + `validate` + `smoke-build`, blocks on non-success, treats `changes` failure as blocking. -- **D2:** the main release backstop checks the prerelease `-`, strips `+buildmetadata`; `validate-task` is the - shared gate. +- **D2:** the hub's `validate-release` checks the prerelease `-`, strips `+buildmetadata`; the hub's + `validate-task.yml` is the shared gate. - **D3:** `main` appears in the backstop and the release `prerelease: false`/guard; `publicReleaseRefSpec` is `^refs/heads/main$`; `semver2` threads to `LABEL_VERSION`. - **D4:** `publish-release` triggers are `schedule` + `workflow_dispatch` + `push` (branches `[main]`, paths - `[Make/Matrix.json]`) only (no other push, no `PUBLISH_ON_MERGE`); jobs guarded to `github.ref_name` in - (`main`, `develop`); `target_commitish` = `GitCommitId`; main pins `ref` to `GitCommitId`, develop reuses - the base (`build_base: false`); the build logs in with `DOCKER_HUB_*`; buildcache branch-scoped and - write-gated on push; release-create gated `exists == false || workflow_dispatch`; release + docker-readme - gated to `main`; the product matrix builds amd64+arm64 from `Make/Matrix.json`. -- **D5:** the publisher and CI each have a terminal `cleanup-artifacts` (`always()`, `continue-on-error`), - independent of the aggregator. -- **D6:** CI is `push` on every branch; the aggregator context has exactly one producer; no `pull_request` + `[Make/Matrix.json]`) only (no other push, no `PUBLISH_ON_MERGE`); every job gates on the plan outputs; + `ref` is `github.sha`, which the hub versions and tags as `GitCommitId`; `build-base` runs only when stable + and develop reuses the base (`docker_build_base: false`); the builds log in with `DOCKER_HUB_*`; buildcache + branch-scoped and write-gated on push; release-create gated `exists == false || workflow_dispatch`; + `github` and `publish-docker-readme` gated to stable; the `docker-prepare` hook emits the branch's + `Make/Matrix.json` rows, built amd64+arm64 on `main`. +- **D5:** no job uploads a transfer artifact; `expect_release_assets: false`. +- **D6:** CI is `push` on every branch; the hooks resolve from the checkout; the aggregator context has exactly one producer; no `pull_request` trigger; every CI job has the `!github.event.deleted` guard. - **D7:** the publisher group is ref-independent with `cancel-in-progress: false`; the merge-bot keys on PR - number; codegen keys on the workflow; CI uses the standard group + deletion guard; reusable jobs declare - permissions. -- **D8/D9:** the merge-bot runs on `pull_request_target` with the App token, keyed on PR number, both merge - jobs use `--delete-branch`; Dependabot auto-merge excepts semver-major NuGet only; codegen dual-targets - main+develop and regenerates only `Version.json`/`Matrix.json`; no date-badge, standalone docker-readme - task, `PUBLISH_ON_MERGE`, or `dorny/paths-filter`; actions SHA-pinned except `nbgv@master`; + number; codegen keys on the workflow; CI uses the standard group + deletion guard; every entry workflow + declares `permissions: {}` and each calling job grants its task's scope. +- **D8/D9:** the merge-bot calls the hub's `merge-bot-task.yml` on `pull_request_target`, keyed on PR number, + with `delete-branch: true`; Dependabot auto-merges every tier; codegen dual-targets main+develop and its + hook regenerates only `Version.json`/`Matrix.json`; no date-badge, carried `-task.yml`, + `PUBLISH_ON_MERGE`, or `dorny/paths-filter`; actions and hub tasks SHA-pinned except `nbgv@master`; names/shells/conditionals per section 2. ### 5B. End-to-end trace scenarios (deterministic from the YAML) @@ -557,17 +587,17 @@ guarantee with a `file:line` citation: | --- | --- | --- | --- | | S1 | push touching `Docker/**` | `validate` + `smoke-build` (NxMeta amd64) run, **no push, no release**; aggregator success; no dangling artifacts | D0.1, D1 | | S2 | push changing only docs | `validate` runs; the `changes` gate sets `image=false`; `smoke-build` skipped; aggregator success (skip allowed) | D1, D1.5 | -| S3 | push changing only `.github/workflows/**` | `validate` runs head-resolved; `smoke-build` skipped (no image files); aggregator success | D1.1, D6.1 | +| S3 | push changing only `.github/workflows/**` (a hub pin bump included) | `validate` + `smoke-build` run, **no push, no release**; aggregator success | D1.1, D6.1 | | S4 | weekly `schedule` | builds + publishes `main` only: shared base refresh + full product matrix (amd64+arm64) + stable release + `latest`; `target_commitish` = main's SHA; develop untouched; no dangling artifacts | D4.1, D4.2, D4.9 | | S5 | push to `main` changing `Make/Matrix.json` (codegen pin) | publishes `main` with the new product versions immediately | D4.1, D8.3 | -| S6 | `workflow_dispatch` from `develop` | builds + publishes `develop`: `:develop` images, prerelease classification, `build_base: false` (reuses main's base), **no GitHub release** | D4.1, D4.2, D3.2 | +| S6 | `workflow_dispatch` from `develop` | builds + publishes `develop`: `:develop` images, prerelease classification, `build-base` skipped (reuses main's base), **no GitHub release** | D4.1, D4.2, D3.2 | | S7 | `workflow_dispatch` re-run on `main`, no new commits | release-create refreshed on dispatch (skipped on schedule if the tag exists); Docker re-pushed (base refresh); no duplicate release | D4.5 | -| S8 | `workflow_dispatch` from a feature branch | the `github.ref_name in (main, develop)` guard skips every job -> no publish | D4.1 | +| S8 | `workflow_dispatch` from a feature branch | the plan job fails with an `::error::` and every later job skips -> no publish | D4.1 | | S9 | merged dependency bump (any) | not a matrix-pin change; merges don't publish -> **no release**; ships in the next scheduled run | D4.1, D8.2 | | S10 | merged develop codegen PR (`Matrix.json` change on develop) | sync-only; the pin push is main-only -> **no publish** | D8.3 | | S11 | PR with a CSharpier / format / unit-test failure | `validate` fails -> aggregator blocks the merge | D1.2, D1.3, D1.5 | | S12 | `version.json` floor bump merged | merges don't publish -> no immediate release; the new floor ships in the next publish | D3.3, D4.1 | -| S13 | Dependabot semver-major NuGet bump | gated on human review -> does not auto-merge; other majors auto-merge on green | D8.2 | +| S13 | Dependabot semver-major NuGet bump | auto-merges on green like every other tier; a red check blocks it | D8.2 | | S14 | branch-deletion push | every CI job + the aggregator skip (`!github.event.deleted`) -> no failing required check | D7.4 | | S15 | `develop` -> `main` promotion (merge commit) | the merge itself does not publish; if it changed `Matrix.json` the pin push publishes main, else the next schedule does | D4.1, D8.1 | @@ -631,9 +661,24 @@ the configuration is part of "operational" (D10; audit 5D). **Repository settings.** Auto-merge enabled; squash and merge-commit both allowed (each ruleset narrows its branch to one); rebase off; auto-delete-on-merge **off** (so `main`/`develop` survive a promotion; the -merge-bot deletes bot branches explicitly with `--delete-branch`). Dependabot version **and** security updates +merge-bot deletes bot branches explicitly through its `delete-branch: true` input). Dependabot version **and** security updates enabled. The GitHub App installed with the scopes above. **Validation.** This configuration is codified in the hub's repository-configuration payloads and applied/audited by the hub's `configure.sh`; `check` is the 5D audit. Secret values cannot be read back, so the audit asserts the names exist (failing if they cannot be queried); the App installation is a best-effort check. + + + +[actions]: ./.github/actions/ +[actions-codegen]: ./.github/actions/codegen/action.yml +[actions-docker-build-base]: ./.github/actions/docker-build-base/action.yml +[actions-docker-prepare]: ./.github/actions/docker-prepare/action.yml +[codestyle]: ./CODESTYLE.md +[docker-readme]: ./Docker/README.md +[editorconfig]: ./.editorconfig +[governance-release-model]: ./GOVERNANCE.md#release-model +[publish-release]: ./.github/workflows/publish-release.yml +[run-periodic-codegen-pull-request]: ./.github/workflows/run-periodic-codegen-pull-request.yml +[test-pull-request]: ./.github/workflows/test-pull-request.yml +[workflows]: ./.github/workflows/