diff --git a/.github/workflows/gitleaks.yml b/.github/workflows/gitleaks.yml index 3f23e7a..8c7c8dc 100644 --- a/.github/workflows/gitleaks.yml +++ b/.github/workflows/gitleaks.yml @@ -60,16 +60,67 @@ jobs: # chains through, so the built-in rules stay active either way. # # A repo with no config of its own still gets the org one, as before. - - name: Fetch the org allowlist + # + # The ruleset is fetched AT THE REF THIS WORKFLOW WAS CALLED AT, not from + # `main`. That is the whole point of the step, so do not "simplify" it + # back to a branch: + # + # A caller pins `gitleaks.yml@`. That digest already freezes the + # detector — `gitleaks-version` is an input default in THIS file — but + # while the config was fetched from `main` it did not freeze the rules the + # detector ran with. So the pin was half a pin, and the uncovered half + # failed OPEN: broadening the shared [allowlist] shrank detection in every + # calling repo on its next scan, with no commit, no PR and no review + # anywhere else, and every required check stayed green. A weakening of + # detection has to be visible somewhere, and the only place a consumer can + # see it is a change to a ref it controls. + # + # `job.workflow_sha` is the commit SHA of the file that defines the + # current job — this reusable workflow, at the caller's ref. For a caller + # on `@main` or a branch it resolves to that branch's current head, so an + # unpinned caller behaves exactly as before; only a pinned caller changes, + # and it changes to what its pin already claimed. + # + # The cost, stated plainly: a fix to the shared ruleset no longer reaches + # a pinned repo until that repo's pin moves (Renovate's digest PR). A + # repo sitting on a stale pin now visibly scans with a stale ruleset + # instead of a stale binary running current rules. + - name: Fetch the org ruleset at this workflow's ref + env: + CONFIG_SHA: ${{ job.workflow_sha }} + CONFIG_REF: ${{ job.workflow_ref }} run: | + set -euo pipefail + # Fail closed on an empty SHA rather than falling back to a branch: + # an unknown ruleset version is the one thing this step must not scan + # with. + : "${CONFIG_SHA:?job.workflow_sha was empty; refusing to resolve the org ruleset}" + if [ -f .gitleaks.toml ]; then local_config=yes; else local_config=no; fi # -f: fail on an HTTP error instead of writing the error body into the # config file. Unguarded, so a fetch failure fails the job rather than # silently scanning with a different (or empty) ruleset. - curl -fsSL "https://raw.githubusercontent.com/FlowMatrix-AI/.github/main/.gitleaks.toml" \ + curl -fsSL "https://raw.githubusercontent.com/FlowMatrix-AI/.github/${CONFIG_SHA}/.gitleaks.toml" \ -o .gitleaks.org.toml - if [ ! -f .gitleaks.toml ]; then + if [ "$local_config" = no ]; then cp .gitleaks.org.toml .gitleaks.toml fi + # Record which ruleset this run actually scanned with, so a change in + # detection behaviour is legible from the run itself and not only from + # a diff in another repository. Printed to the log as well as the job + # summary: the log is reachable over the API, so the ruleset a repo + # scanned with can be audited across the fleet without opening 69 + # run pages. The summary is for humans, the log line for scripts. + echo "gitleaks-ruleset commit=${CONFIG_SHA} sha256=$(sha256sum .gitleaks.org.toml | cut -d' ' -f1) local-config=${local_config}" + { + echo "### gitleaks ruleset" + echo + echo "| | |" + echo "| --- | --- |" + echo "| workflow ref | \`${CONFIG_REF}\` |" + echo "| org ruleset commit | \`${CONFIG_SHA}\` |" + echo "| org ruleset sha256 | \`$(sha256sum .gitleaks.org.toml | cut -d' ' -f1)\` |" + echo "| repo-local \`.gitleaks.toml\` | ${local_config} |" + } >> "$GITHUB_STEP_SUMMARY" - name: Scan run: | diff --git a/AGENTS.md b/AGENTS.md index c7cf4a8..1b8caee 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -28,9 +28,9 @@ shared secret-scanning plant. It holds two reusable workflows (`.gitleaks.toml`), the Actions workflow-template entry that seeds new repos, the shared `.pre-commit-config.yaml`, and the community health files (`SECURITY.md`, `PULL_REQUEST_TEMPLATE.md`, `ISSUE_TEMPLATE/`) plus -`profile/README.md`. **68 of the org's 70 active repos call `gitleaks.yml`** -from their own `.github/workflows/secret-scan.yml` — every repo but -`lead-intake` and this one, which calls it through a local ref — and the +`profile/README.md`. **Every active repo in the org but this one calls +`gitleaks.yml`** from its own `.github/workflows/secret-scan.yml` — 69 of 70, +this repo being the exception because it calls it through a local ref — and the resulting `gitleaks / gitleaks` status is a required check, so this repo gates merges fleet-wide. `commit-email.yml` has no caller in the org today. @@ -77,13 +77,21 @@ how the pin sat on 8.21.2 for a year. - **Do not make this repo private.** Three things break, and the third is silent: the org profile goes dark, the default community health files stop applying org-wide, and the unauthenticated fetch of `.gitleaks.toml` 404s so - the required `gitleaks / gitleaks` check fails in all 68 callers. -- **Do not read a caller's digest pin as freezing behaviour.** Callers pin - `gitleaks.yml` by commit digest, but the workflow fetches `.gitleaks.toml` - from `main` on **every** run. Merging a change to the ruleset changes what - every repo detects on its next scan, with no PR anywhere else — and a - broadened allowlist **fails open**: checks stay green while detection - shrinks. Treat an allowlist edit as a fleet-wide security change. + the required `gitleaks / gitleaks` check fails in every caller. +- **Do not fetch `.gitleaks.toml` from `main`.** It is fetched at + `job.workflow_sha`, the commit of the reusable workflow file being run, so a + caller's digest pin covers the ruleset as well as the workflow code. Fetching + from a branch instead made the pin half a pin, and the uncovered half failed + **open**: merging a broadened allowlist here shrank detection in every calling + repo on its next scan, with no commit or review anywhere else, while every + required check stayed green. A caller on `@main` is unaffected either way. + The cost of the pin is that a ruleset fix reaches a pinned repo only when its + pin moves; a repo on a stale pin was already running a stale **binary**, so + pinning makes the staleness visible in one place rather than half-hidden. +- **Still treat a ruleset edit as a fleet-wide security change.** The pin means + a weakening lands repo by repo as pins move, not instantly — it does not mean + it is reviewed. Code-owner review on `.gitleaks.toml` is not required by any + ruleset yet (eng-governance #111). - **Do not add a repo-local `.gitleaks.toml` without inheriting.** A local config *replaces* the org ruleset and the scan still shows green. Use `[extend] path = ".gitleaks.org.toml"`. Never set `useDefault` alongside diff --git a/README.md b/README.md index 3a2970a..3ea79ec 100644 --- a/README.md +++ b/README.md @@ -17,18 +17,37 @@ Making it private breaks three things, and the third is the least visible: | The default community health files (`ISSUE_TEMPLATE/`, `PULL_REQUEST_TEMPLATE.md`, `SECURITY.md`) | they stop applying org-wide ("a repository for default files cannot be private") | | The unauthenticated fetch of `.gitleaks.toml` in `.github/workflows/gitleaks.yml` | the required `gitleaks / gitleaks` check fails in every calling repo: `raw.githubusercontent.com` returns 404 for a private repo | -## `.gitleaks.toml` is live everywhere the moment it merges - -Callers pin `gitleaks.yml` by digest, but that pins the workflow code, not the -ruleset it runs with: the workflow fetches `.gitleaks.toml` from `main` on every -run. A change to it (an over-broad allowlist in particular) changes what every -repo detects on its next scan, and fails open: checks stay green. `CODEOWNERS` -names an owner for it and for the workflows. - -The fetch is deliberately fail-closed (`curl -f`, no fallback), so a scan never -runs with an empty or different ruleset. The accepted cost: if -`raw.githubusercontent.com` is unreachable, the required check fails and no repo -in the org can merge until it is back. +## `.gitleaks.toml` resolves at the ref the caller pinned + +A caller pins `gitleaks.yml@`, and that digest now covers the ruleset as +well as the workflow code: the workflow fetches `.gitleaks.toml` at +`job.workflow_sha` — the commit of the reusable workflow file being run — rather +than from `main`. + +It did not always. The pin used to freeze the detector (`gitleaks-version` is an +input default in the workflow file) while the rules it ran with were re-fetched +from `main` on every run, so merging an over-broad allowlist here shrank +detection in every calling repo on its next scan, with no commit, no PR and no +review anywhere else, and every required check stayed green. That failed open, +which is the one direction a scanner must not fail. A weakening of detection is +now a change to a ref the consuming repo controls. + +A caller on `@main`, or any branch, is unaffected: `job.workflow_sha` resolves to +that branch's current head. The cost falls on pinned callers — a fix to the +shared ruleset reaches them only when the pin moves, which is Renovate's digest +PR. A repo on a stale pin scans with a stale ruleset instead of with a stale +binary running current rules, and that is the more honest of the two. + +Each run records the ruleset commit and its `sha256` in the job summary, so what +a scan actually ran with is legible from the run. + +`CODEOWNERS` names an owner for `.gitleaks.toml` and for the workflows. + +The fetch is deliberately fail-closed (`curl -f`, no fallback, and an empty +`job.workflow_sha` aborts the step), so a scan never runs with an empty, +different or unknown ruleset. The accepted cost: if `raw.githubusercontent.com` +is unreachable, the required check fails and no repo in the org can merge until +it is back. A repo that needs its own `.gitleaks.toml` must `[extend] path = ".gitleaks.org.toml"`: a local config otherwise replaces the org ruleset