diff --git a/.changes/harness-ops/feat-release-intake-workflow-3e9b6d21.md b/.changes/harness-ops/feat-release-intake-workflow-3e9b6d21.md new file mode 100644 index 00000000000..477ee3ceede --- /dev/null +++ b/.changes/harness-ops/feat-release-intake-workflow-3e9b6d21.md @@ -0,0 +1,13 @@ +--- +bump: patch +--- + +### Changed + +- `changelog apply --ci` now says the calling workflow files a release's handoffs as one issue, with + one checklist item per handoff. A `cap-exceeded` plan's issue names its `stop_reason`, and the + release is marked done once that issue holds every handoff; a `range-not-computed` plan files + nothing and the next run takes the release again. +- `changelog apply --ci` now says a branch row may touch no hold path (the security review's + `hold-paths` list); the workflow stops a release whose branch changes one with `stop_reason` + `hold-path`, files its issue, and marks it done. diff --git a/.changes/pr-pipeline/feat-release-intake-workflow-7c41e9a2.md b/.changes/pr-pipeline/feat-release-intake-workflow-7c41e9a2.md new file mode 100644 index 00000000000..f34986c8461 --- /dev/null +++ b/.changes/pr-pipeline/feat-release-intake-workflow-7c41e9a2.md @@ -0,0 +1,14 @@ +--- +bump: minor +--- + +### Added + +- The `upstream-apply-changelog` lane takes in one Claude Code release a run: a model job runs + `/harness-ops:changelog apply --ci` with the API key and a read-only token, a scan job checks + and credential-scans its sealed plan and branches, `push-branches` (`apply-changelog-branches`, + mutate-branch) recreates each branch as signed lanes App commits, and `publish` + (`publish-changelog-intake`, publish-artifact) opens one pull request per branch and files each + handoff once. The token broker mints for it keyed on a new `release` request field. It polls + every ten minutes, checks the ten newest unread releases, and starts at most + `limits.upstream-apply-changelog-per-day` (3) runs in any 24 hours. diff --git a/.changes/source-control/feat-release-intake-workflow-38b891af.md b/.changes/source-control/feat-release-intake-workflow-38b891af.md new file mode 100644 index 00000000000..5c5ba85a76d --- /dev/null +++ b/.changes/source-control/feat-release-intake-workflow-38b891af.md @@ -0,0 +1,8 @@ +--- +bump: patch +--- + +### Changed + +- babysit-prs: a push touching `plugins/harness-ops/skills/changelog/scripts/` now stops an + approval carrying, since the gated upstream-apply-changelog workflow runs a script there. diff --git a/.github/actions/log-api-budget/log-api-budget.test.mjs b/.github/actions/log-api-budget/log-api-budget.test.mjs index fcc51c3e5b6..c6e808ad0a5 100644 --- a/.github/actions/log-api-budget/log-api-budget.test.mjs +++ b/.github/actions/log-api-budget/log-api-budget.test.mjs @@ -1037,6 +1037,12 @@ test("each heavy lane job names its API_BUDGET_TOKEN and ends with the log-api-b // own notice (a key, and no other action from a checkout); both pinned below. "pr-run-activity-read.yml": { "broker-check": "github-token", "issue-fields": "github-token", summary: "github-token", report: "github-token" }, "pr-run-activity-write.yml": { report: "github-token" }, + "upstream-apply-changelog.yml": { + gate: "github-token", + scan: "github-token", + "push-branches": ["lanes-app", "lanes-app github-token"], + publish: ["lanes-app", "lanes-app github-token"], + }, }; for (const [file, tokens] of Object.entries(jobs)) { const text = readFileSync(new URL(`../../workflows/${file}`, import.meta.url), "utf8"); diff --git a/.github/actions/request-lane-token/README.md b/.github/actions/request-lane-token/README.md index b8c3e54a326..b5c6babc869 100644 --- a/.github/actions/request-lane-token/README.md +++ b/.github/actions/request-lane-token/README.md @@ -21,6 +21,7 @@ run, and mints nothing ([Check mode](#check-mode)). | `lane`, `activity` | Sent to the broker; the lane must be the caller workflow's file stem | | `pr-number` | `token` mode: the gated PR; sent as `pr_number` when set, which the broker needs off `pull_request` | | `issue-number` | `token` mode: the issue an `issues` lane works; sent as `issue_number` when set, which the broker needs on an `issues` event. Never with `pr-number` or `dispatcher-run-id` | +| `release` | `token` mode: the Claude Code release an `upstream-apply-changelog.yml` job works, one `MAJOR.MINOR.PATCH`; sent as `release`, a string, when set, which the broker needs from that caller and refuses from any other. Anything else fails red. Never with `pr-number`, `issue-number` or `dispatcher-run-id` | | `dispatcher-run-id` | `token` mode: the `pr-unblock-dispatch.yml` run that dispatched this one; sent as `dispatcher_run_id` when set. The broker needs it to trust a run sent by `github-actions[bot]` | | `effect`, `contents`, `pull-requests`, `issues`, `actions` | `token` mode: this job's own `resolve-config` outputs, compared with the broker's answer. `actions` is `read` or empty; empty expects no `actions` permission | | `superseded-ok` | `true` ends green, with a `lane-token-superseded` warning and no token, on a 403 whose reason is `workflow-modified` (a newer copy of the caller workflow on the default branch, whose run does the work). Default `false` | @@ -44,9 +45,9 @@ run, and mints nothing ([Check mode](#check-mode)). each of `contents`, `pull-requests` and `issues`, or when the job lacks `id-token: write`. 2. Requests an OIDC token for `audience` and masks it. 3. Sends a `POST` to `broker-url` with `Authorization: Bearer ` and the body - `{"lane", "activity", "pr_number"?, "issue_number"?, "dispatcher_run_id"?}`. It sends at most - three requests, each with a fresh OIDC token, and retries only two 503s that come before the - broker reserves the job's mint, as GitHub's + `{"lane", "activity", "pr_number"?, "issue_number"?, "release"?, "dispatcher_run_id"?}`. It sends + at most three requests, each with a fresh OIDC token, and retries only two 503s that come before + the broker reserves the job's mint, as GitHub's [rate-limit guidance](https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api#exceeding-the-rate-limit) asks: - `github-rate-limited`: GitHub rate-limited the broker. The retry waits exactly the @@ -136,7 +137,7 @@ controls that job, so this adds nothing it could not already do. ## Check mode With `mode: check` the action takes only `broker-url`, `audience`, `lane` and `activity`; a -`pr-number`, `issue-number`, effect or grant fails it red, so the read file's check job never carries the write +`pr-number`, `issue-number`, `release`, effect or grant fails it red, so the read file's check job never carries the write path's inputs. It posts to `broker-url`, retrying a 503 `github-unavailable` or `github-rate-limited` as in step 3, with its final `/token` replaced by `/check` (a URL that does not match `^https://[^/]+/(.+/)?token$` fails red), with the body `{"lane", "activity"}`, and passes only on a 200 whose `decision` is `allow` and whose `lane` and `activity` are its own. Any other answer fails red diff --git a/.github/actions/request-lane-token/action.yml b/.github/actions/request-lane-token/action.yml index 515153da5d3..7e191cc683f 100644 --- a/.github/actions/request-lane-token/action.yml +++ b/.github/actions/request-lane-token/action.yml @@ -45,6 +45,14 @@ inputs: Never with `pr-number` or `dispatcher-run-id`. `token` mode only. required: false default: "" + release: + description: >- + The Claude Code release an upstream-apply-changelog.yml job works, one + MAJOR.MINOR.PATCH; sent as `release`, a string, which the broker needs + from that caller and refuses from any other. Never with `pr-number`, + `issue-number` or `dispatcher-run-id`. `token` mode only. + required: false + default: "" dispatcher-run-id: description: >- The pr-unblock-dispatch.yml run that dispatched this run; sent as @@ -159,6 +167,7 @@ runs: ACTIVITY: ${{ inputs.activity }} PR_NUMBER: ${{ inputs.pr-number }} ISSUE_NUMBER: ${{ inputs.issue-number }} + RELEASE: ${{ inputs.release }} DISPATCHER_RUN_ID: ${{ inputs.dispatcher-run-id }} EXPECTED_UPSTREAM_HEAD: ${{ inputs.expected-upstream-head }} EFFECT: ${{ inputs.effect }} @@ -218,6 +227,10 @@ runs: [[ -z ${ISSUE_NUMBER:-} || $ISSUE_NUMBER =~ ^[1-9][0-9]{0,9}$ ]] || fail "the issue number is malformed" [[ -z ${ISSUE_NUMBER:-} || -z $PR_NUMBER$DISPATCHER_RUN_ID ]] || fail "an issue number goes without a PR number or dispatcher run id" + [[ -z ${RELEASE:-} || $RELEASE =~ ^(0|[1-9][0-9]{0,3})\.(0|[1-9][0-9]{0,3})\.(0|[1-9][0-9]{0,5})$ ]] || + fail "the release must be one MAJOR.MINOR.PATCH" + [[ -z ${RELEASE:-} || -z $PR_NUMBER$ISSUE_NUMBER$DISPATCHER_RUN_ID ]] || + fail "a release goes without a PR number, issue number or dispatcher run id" # At most 15 digits keeps it a safe JSON integer for the broker. [[ -z ${DISPATCHER_RUN_ID:-} || $DISPATCHER_RUN_ID =~ ^[1-9][0-9]{0,14}$ ]] || fail "the dispatcher run id is malformed" @@ -227,8 +240,8 @@ runs: check) # The check job resolves no grant and gates no PR, and the broker's # check endpoint answers 400 to any body field but lane and activity. - [[ -z $EFFECT$CONTENTS$PULL_REQUESTS$ISSUES${ACTIONS:-}$PR_NUMBER$ISSUE_NUMBER$DISPATCHER_RUN_ID${EXPECTED_UPSTREAM_HEAD:-} ]] || - fail "check mode takes no effect, grant, PR or issue number, or dispatcher run id" + [[ -z $EFFECT$CONTENTS$PULL_REQUESTS$ISSUES${ACTIONS:-}$PR_NUMBER$ISSUE_NUMBER${RELEASE:-}$DISPATCHER_RUN_ID${EXPECTED_UPSTREAM_HEAD:-} ]] || + fail "check mode takes no effect, grant, PR or issue number, release, or dispatcher run id" [[ $BROKER_URL =~ ^https://[^/]+/(.+/)?token$ ]] || fail "lane-token-unconfigured: LANES_BROKER_URL does not end in a /token path segment, so it names no check endpoint" BROKER_URL="${BROKER_URL%/token}/check" @@ -260,10 +273,11 @@ runs: trap cleanup EXIT body=$(jq -cn --arg lane "$LANE" --arg activity "$ACTIVITY" --arg pr "$PR_NUMBER" \ - --arg issue "$ISSUE_NUMBER" --arg dispatcher "$DISPATCHER_RUN_ID" --arg upstream "${EXPECTED_UPSTREAM_HEAD:-}" \ + --arg issue "$ISSUE_NUMBER" --arg release "${RELEASE:-}" --arg dispatcher "$DISPATCHER_RUN_ID" --arg upstream "${EXPECTED_UPSTREAM_HEAD:-}" \ '{lane: $lane, activity: $activity} + (if $pr == "" then {} else {pr_number: ($pr | tonumber)} end) + (if $issue == "" then {} else {issue_number: ($issue | tonumber)} end) + + (if $release == "" then {} else {release: $release} end) + (if $dispatcher == "" then {} else {dispatcher_run_id: ($dispatcher | tonumber)} end) + (if $upstream == "" then {} else {expected_upstream_head: $upstream} end)') diff --git a/.github/actions/request-lane-token/request-lane-token.test.mjs b/.github/actions/request-lane-token/request-lane-token.test.mjs index f1cdb026346..0910974390e 100644 --- a/.github/actions/request-lane-token/request-lane-token.test.mjs +++ b/.github/actions/request-lane-token/request-lane-token.test.mjs @@ -696,7 +696,45 @@ for (const [name, inputs] of [ test("check mode refuses an issue number before the broker request", () => { const run = request({ MODE: "check", EFFECT: "", CONTENTS: "", PULL_REQUESTS: "", ISSUES: "", PR_NUMBER: "", ISSUE_NUMBER: "77" }); assert.equal(run.status, 1); - assert.match(run.stdout, /::error::check mode takes no effect, grant, PR or issue number, or dispatcher run id/); + assert.match(run.stdout, /::error::check mode takes no effect, grant, PR or issue number, release, or dispatcher run id/); + assert.equal(run.body, null); +}); + +const RELEASE_LANE = { LANE: "upstream-apply-changelog", ACTIVITY: "apply-changelog-branches", PR_NUMBER: "" }; + +test("a release is sent as release, a string, with no pr_number or issue_number", () => { + const run = request({ ...RELEASE_LANE, RELEASE: "2.1.312" }); + assert.equal(run.status, 0, run.stdout); + assert.deepEqual(JSON.parse(run.body), { lane: "upstream-apply-changelog", activity: "apply-changelog-branches", release: "2.1.312" }); + assert.match(run.output, /^token=ghs_lane$/m); +}); + +for (const value of ["2.1", "2.1.312.1", "v2.1.312", "02.1.312", "2.01.312", "2.1.0312", "2.1.3a", "2.1.312 ", "12345.1.1", "1.1.1234567"]) { + test(`a malformed release (${JSON.stringify(value)}) fails red before the broker request`, () => { + const run = request({ ...RELEASE_LANE, RELEASE: value }); + assert.equal(run.status, 1); + assert.match(run.stdout, /::error::the release must be one MAJOR\.MINOR\.PATCH/); + assert.equal(run.body, null); + }); +} + +for (const [name, inputs] of [ + ["a PR number", { PR_NUMBER: "42" }], + ["an issue number", { ISSUE_NUMBER: "77" }], + ["a dispatcher run id", { DISPATCHER_RUN_ID: "38155572048" }], +]) { + test(`a release with ${name} fails red before the broker request`, () => { + const run = request({ ...RELEASE_LANE, RELEASE: "2.1.312", ...inputs }); + assert.equal(run.status, 1); + assert.match(run.stdout, /::error::a release goes without a PR number, issue number or dispatcher run id/); + assert.equal(run.body, null); + }); +} + +test("check mode refuses a release before the broker request", () => { + const run = request({ MODE: "check", EFFECT: "", CONTENTS: "", PULL_REQUESTS: "", ISSUES: "", PR_NUMBER: "", RELEASE: "2.1.312" }); + assert.equal(run.status, 1); + assert.match(run.stdout, /::error::check mode takes no effect, grant, PR or issue number, release, or dispatcher run id/); assert.equal(run.body, null); }); diff --git a/.github/actions/resolve-config/callers.test.mjs b/.github/actions/resolve-config/callers.test.mjs index 56079641ad4..6e02b541a0f 100644 --- a/.github/actions/resolve-config/callers.test.mjs +++ b/.github/actions/resolve-config/callers.test.mjs @@ -4568,7 +4568,7 @@ test("pr-run-activity-read.yml: the scan-carrier key is passed by name by each r assert.equal(job.secrets["scan-carrier-private-key"], CARRIER_PASS, `${file} ${id}`); const rest = { ...job, secrets: { ...job.secrets, "scan-carrier-private-key": undefined } }; assert.doesNotMatch(JSON.stringify(rest), /SCAN_CARRIER_PRIVATE_KEY|scan-carrier-private-key/i, `${file} ${id}`); - } else if (file !== "pr-run-activity-read.yml") { + } else if (file !== "pr-run-activity-read.yml" && !(file === "upstream-apply-changelog.yml" && id === "scan")) { assert.doesNotMatch(JSON.stringify(job), /SCAN_CARRIER_PRIVATE_KEY|scan-carrier-private-key/i, `${file} ${id}`); } } @@ -4584,9 +4584,27 @@ test("pr-run-activity-read.yml: the scan-carrier key is passed by name by each r assert.equal(open.with["private-key"], CARRIER_SECRET); assert.equal(open.env, undefined); assert.equal(open.uses, "./.base/.github/actions/open-lane-output"); + // The second holder (owner decision, 2026-11-02): upstream-apply-changelog.yml's scan job, one + // open step, only while it runs from main with no id-token, no write token and a read-only cache. + const intake = workflow("upstream-apply-changelog.yml").jobs; + for (const [id, job] of Object.entries(intake)) { + const steps = (job.steps ?? []).filter((step) => /scan-carrier-private-key|SCAN_CARRIER_PRIVATE_KEY/i.test(JSON.stringify(step))); + assert.deepEqual(steps.map((step) => step.id), id === "scan" ? ["open"] : [], `upstream-apply-changelog.yml ${id}`); + } + const holder = intake.scan; + const intakeOpen = holder.steps.find((s) => s.id === "open"); + assert.deepEqual(stepSecrets(intakeOpen), ["SCAN_CARRIER_PRIVATE_KEY"]); + assert.equal(intakeOpen.with["private-key"], "${{ secrets.SCAN_CARRIER_PRIVATE_KEY }}"); + assert.equal(intakeOpen.env, undefined); + assert.equal(intakeOpen.uses, "./.base/.github/actions/open-lane-output"); + assert.deepEqual(holder.permissions, { contents: "read" }); + assert.equal(holder["cache-mode"], "read"); + // Main only: the scan job runs after the gate job, which runs only on refs/heads/main. + assert.ok([holder.needs].flat().includes("gate")); + assert.match(intake.gate.if ?? "", /github\.ref == 'refs\/heads\/main'/); for (const file of files) { for (const [id, job] of Object.entries(workflow(file).jobs ?? {})) { - if (file === "pr-run-activity-read.yml" && id === "scan") assert.equal(job.environment, "pr-pipeline-scan-carrier"); + if ((file === "pr-run-activity-read.yml" || file === "upstream-apply-changelog.yml") && id === "scan") assert.equal(job.environment, "pr-pipeline-scan-carrier"); else assert.doesNotMatch(JSON.stringify(job.environment ?? ""), /pr-pipeline-scan-carrier/, `${file} ${id}`); } } @@ -7609,6 +7627,8 @@ test("every kill-switch-ok caller runs no token step and no dependent job on a k "work-resolve-issue.yml:apply-issue-patch -> pr-run-activity-write.yml", "work-resolve-issue.yml:report-outcome -> pr-run-activity-write.yml", "work-resolve-issue.yml:report-skip -> pr-run-activity-write.yml", + "upstream-apply-changelog.yml:push-branches", + "upstream-apply-changelog.yml:publish", ].sort()); }); @@ -7776,6 +7796,7 @@ test("every job that runs or reads untrusted head content has a read-only Action "intake-triage.yml": ["refine-issue", "refine-issue-subscription"], "pr-review-main.yml": ["review", "review-subscription", "review-fallback"], "pr-verify.yml": ["check-intent", "check-intent-subscription", "check-intent-fallback"], + "upstream-apply-changelog.yml": ["model", "scan", "push-branches", "publish"], }; for (const [file, names] of Object.entries(readOnly)) { const { jobs } = workflow(file); diff --git a/.github/actions/resolve-config/lane-grants.json b/.github/actions/resolve-config/lane-grants.json index f7e092a05b1..05e0107c343 100644 --- a/.github/actions/resolve-config/lane-grants.json +++ b/.github/actions/resolve-config/lane-grants.json @@ -7,7 +7,8 @@ "pr-unblock": { "resolve-conflicts": "read", "apply-conflict-patch": "mutate-branch", "address-feedback": "read", "apply-feedback-patch": "mutate-branch", "publish-feedback": "publish-artifact", "review-dependency": "read", "apply-dependency-patch": "mutate-branch", "apply-dependency-verdict": "publish-artifact", "fix-ci": "read", "apply-patch": "mutate-branch", "sync-shared-copies": "read", "apply-shared-copies": "mutate-branch", "report-escalation": "publish-artifact" }, "pr-merge": { "merge-pr": "merge", "enable-auto-merge": "merge" }, "work-unblock": { "label-unblocked": "mutate-tracker" }, - "intake-triage": { "apply-verdict": "mutate-tracker", "apply-refinement": "mutate-tracker" } + "intake-triage": { "apply-verdict": "mutate-tracker", "apply-refinement": "mutate-tracker" }, + "upstream-apply-changelog": { "apply-changelog-branches": "mutate-branch", "publish-changelog-intake": "publish-artifact" } }, "trusted-actors": [], "ci-workflows-commits": [] diff --git a/.github/actions/resolve-config/lane-rules.json b/.github/actions/resolve-config/lane-rules.json index b906e02b3dc..cb593b0b9e3 100644 --- a/.github/actions/resolve-config/lane-rules.json +++ b/.github/actions/resolve-config/lane-rules.json @@ -74,5 +74,10 @@ "stage": "post-merge", "forbidden-effects": ["merge"], "never": "Reopens merged PRs" + }, + "upstream-apply-changelog": { + "stage": "upstream", + "forbidden-effects": ["mutate-tracker", "merge"], + "never": "Merges, or writes to main or a branch outside changelog//v" } } diff --git a/.github/actions/resolve-config/live-config.test.mjs b/.github/actions/resolve-config/live-config.test.mjs index b29f70dbfe8..575fddf1c65 100644 --- a/.github/actions/resolve-config/live-config.test.mjs +++ b/.github/actions/resolve-config/live-config.test.mjs @@ -651,6 +651,18 @@ test("live config: label-unblocked may only write issues", () => { }); }); +test("live config: upstream-apply-changelog pushes with contents write only and publishes without it", () => { + const grant = (activity) => resolveGrant(FILES, { lane: "upstream-apply-changelog", activity, configPath: CONFIG_PATH }); + assert.deepEqual(grant("apply-changelog-branches"), { + effect: "mutate-branch", + grant: { contents: "write", "pull-requests": "read", issues: "read" }, + }); + assert.deepEqual(grant("publish-changelog-intake"), { + effect: "publish-artifact", + grant: { contents: "read", "pull-requests": "write", issues: "write" }, + }); +}); + test("live config: fix-docs applies to a Markdown change and waits on do-not-merge", () => { assert.equal(selected("pr-refine", "fix-docs").applies, true); assert.equal( diff --git a/.github/artifact-retention.json b/.github/artifact-retention.json index df11111fe27..3a561d68047 100644 --- a/.github/artifact-retention.json +++ b/.github/artifact-retention.json @@ -443,6 +443,31 @@ ], "reason": "Manual inspection only; no workflow reads it after the run. A person reads a dispatched eval's results soon after starting it, and 3 days covers a weekend.", "scan": "upload" + }, + "upstream-apply-changelog-started": { + "retention-days": 2, + "readers": [ + ".github/scripts/upstream-apply-changelog/upstream-apply-changelog.mjs gate counts the start records of the last 24 hours against the per-day cap" + ], + "reason": "The cap counts runs started in the last 24 hours, so a record must outlive one day; 2 days keeps the newest day whole. It holds only the release number the gate chose, never model text, so it is not scanned.", + "scan": "none" + }, + "upstream-apply-changelog-plan-*": { + "retention-days": 1, + "readers": [ + ".github/workflows/upstream-apply-changelog.yml push-branches and publish download it by id, same run" + ], + "reason": "Read by its own run; 1 day is the minimum. The scan job uploads it only after its scans of the plan, the branch diffs and the publish text are clean.", + "scan": "plaintext" + }, + "upstream-apply-changelog-done-*": { + "retention-days": "max", + "readers": [ + ".github/scripts/upstream-apply-changelog/upstream-apply-changelog.mjs gate skips a release whose done marker exists" + ], + "max-reader": ".github/scripts/upstream-apply-changelog/upstream-apply-changelog.mjs", + "reason": "The marker is the only record that a release was processed: once it expires a schedule run takes the release again, so it keeps the longest retention. It holds the plan the scan job cleared and publish posted, downloaded by id and hash; the publish job runs no scan of its own.", + "scan": "none" } } } diff --git a/.github/lane-funding.json b/.github/lane-funding.json index 80152adaeab..4ee8b5530fb 100644 --- a/.github/lane-funding.json +++ b/.github/lane-funding.json @@ -28,7 +28,8 @@ { "id": "resolve-issue", "workflow": "work-resolve-issue.yml", "launcher": "run-claude-cli", "triggers": ["issues"], "gate": "author", "currentSecrets": ["ISSUE_LANE_CLAUDE_API_KEY"] }, { "id": "resolve-issue-subscription", "workflow": "work-resolve-issue.yml", "launcher": "run-claude-cli", "triggers": ["issues"], "gate": "author", "currentSecrets": ["ISSUE_LANE_CLAUDE_CODE_OAUTH_TOKEN"] }, { "id": "resolve-issue-handoff", "workflow": "work-resolve-issue.yml", "launcher": "run-claude-cli", "triggers": ["issues"], "gate": "author", "currentSecrets": ["FALLBACK_CLAUDE_CODE_OAUTH_TOKEN"] }, - { "id": "resolve-issue-fallback", "workflow": "work-resolve-issue.yml", "launcher": "run-claude-cli", "triggers": ["issues"], "gate": "author", "currentSecrets": ["FALLBACK_CLAUDE_CODE_OAUTH_TOKEN"] } + { "id": "resolve-issue-fallback", "workflow": "work-resolve-issue.yml", "launcher": "run-claude-cli", "triggers": ["issues"], "gate": "author", "currentSecrets": ["FALLBACK_CLAUDE_CODE_OAUTH_TOKEN"] }, + { "id": "model", "workflow": "upstream-apply-changelog.yml", "launcher": "run-claude-cli", "triggers": ["schedule", "workflow_dispatch"], "currentSecrets": ["ISSUE_LANE_CLAUDE_API_KEY"] } ] }, { diff --git a/.github/scripts/pr-merge/merge-holds.mjs b/.github/scripts/pr-merge/merge-holds.mjs index 3a9f45eb0e6..57e91216940 100755 --- a/.github/scripts/pr-merge/merge-holds.mjs +++ b/.github/scripts/pr-merge/merge-holds.mjs @@ -406,6 +406,7 @@ export const HOLD_PATH_CLASSES = Object.freeze([ [".github/workflows/pr-unblock.yml", "gate"], [".github/workflows/pr-verify.yml", "gate"], [".github/workflows/release-publish-plugins.yml", "gate"], + [".github/workflows/upstream-apply-changelog.yml", "gate"], [".github/workflows/work-resolve-issue.yml", "gate"], [".github/workflows/work-unblock.yml", "gate"], // END hold-floor HOLD_PATH_CLASSES @@ -990,7 +991,7 @@ export function approvalCarry(then, now, from, inBase, merges, based, dismissal // lanes still judge it. babysit_merge_policy.CARRY_REFUSES is the same list. // The trusted-actor list and the pipeline config are read too: a widening the // approver never saw is lifted by the same approval (actorsHold). -export const CARRY_REFUSES = Object.freeze([".github/workflows/**", ".github/actions/**", ".github/scripts/**", ".github/standards/**", "docs/conventions/pr-pipeline/**", "scripts/**", "lib/**", "plugins/source-control/skills/babysit-prs/scripts/**", "plugins/performance/scripts/**", "plugins/harness-ops/skills/inventory/scripts/**", "plugins/ai-briefing/skills/generate/scripts/**", "plugins/harness-config/skills/audit-permission-state/scripts/**", "plugins/knowledge/skills/course-digest/scripts/**", "plugins/knowledge/skills/video-digest/scripts/**", "plugins/skill-quality/scripts/**", "plugins/source-control/scripts/**", "plugins/testing/skills/check-visual-parity/scripts/**", ".github/recurring-schedule.json", "plugins/evals/skills/validate/scripts/**", "plugins/testing/evals/browser-probe/**", "plugins/work-items/scripts/**", "plugins/work-items/tools/work-item-tracker/**"]); +export const CARRY_REFUSES = Object.freeze([".github/workflows/**", ".github/actions/**", ".github/scripts/**", ".github/standards/**", "docs/conventions/pr-pipeline/**", "scripts/**", "lib/**", "plugins/source-control/skills/babysit-prs/scripts/**", "plugins/performance/scripts/**", "plugins/harness-ops/skills/inventory/scripts/**", "plugins/ai-briefing/skills/generate/scripts/**", "plugins/harness-config/skills/audit-permission-state/scripts/**", "plugins/knowledge/skills/course-digest/scripts/**", "plugins/knowledge/skills/video-digest/scripts/**", "plugins/skill-quality/scripts/**", "plugins/source-control/scripts/**", "plugins/testing/skills/check-visual-parity/scripts/**", ".github/recurring-schedule.json", "plugins/evals/skills/validate/scripts/**", "plugins/testing/evals/browser-probe/**", "plugins/work-items/scripts/**", "plugins/work-items/tools/work-item-tracker/**", "plugins/harness-ops/skills/changelog/scripts/**"]); // Whether the compare from...H' `merges` touches no hold path and no file a // hold path loads, old paths of renames included: pathHold with `holdPaths` diff --git a/.github/scripts/upstream-apply-changelog/package.json b/.github/scripts/upstream-apply-changelog/package.json new file mode 100644 index 00000000000..01b12199b82 --- /dev/null +++ b/.github/scripts/upstream-apply-changelog/package.json @@ -0,0 +1,7 @@ +{ + "name": "upstream-apply-changelog", + "private": true, + "scripts": { + "test": "node --test" + } +} diff --git a/.github/scripts/upstream-apply-changelog/upstream-apply-changelog.mjs b/.github/scripts/upstream-apply-changelog/upstream-apply-changelog.mjs new file mode 100755 index 00000000000..eebfef4e63f --- /dev/null +++ b/.github/scripts/upstream-apply-changelog/upstream-apply-changelog.mjs @@ -0,0 +1,601 @@ +#!/usr/bin/env node +// upstream-apply-changelog: the scripted halves of .github/workflows/upstream-apply-changelog.yml. +// No model runs here. Subcommands: +// gate picks the release, skips one already processed, applies the +// per-day cap (read-only GITHUB_TOKEN with actions: read) +// scan checks each planned head's net diff in the scan job and stops +// the release on a hold path (no write token) +// push checks each planned branch and recreates it through the Git Data +// API as one lanes App commit on the base (mutate-branch token) +// publish opens one PR per pushed branch and files the release's handoffs +// as one issue (publish-artifact token) +// The plan and each branch's tree come from the model job, so every field is +// data: names are held to fixed shapes and each branch's net diff to its +// plugin's paths, none of them a hold path. The bundle holds the scan job's +// synthetic commits only. +import { execFileSync } from "node:child_process"; +import { appendFileSync, existsSync, readFileSync, writeFileSync } from "node:fs"; +import { pathToFileURL } from "node:url"; + +import { rateLimitLatch } from "../../actions/log-api-budget/rate-guard.mjs"; +import { createGitHubApi } from "../apply-patch/apply-patch.mjs"; +import { LANES_APP_LOGIN, matchesHoldPath } from "../pr-merge/merge-holds.mjs"; + +export const RELEASE = /^(0|[1-9][0-9]{0,3})\.(0|[1-9][0-9]{0,3})\.(0|[1-9][0-9]{0,5})$/; +const PLUGIN = /^[a-z0-9][a-z0-9-]{0,63}$/; +const SHA = /^[0-9a-f]{40}$/; +const KEY = /^changelog:[0-9.]+\.\.[0-9.]+:[a-z-]+:[A-Za-z0-9_./-]{1,200}$/; +const DAY_MS = 24 * 60 * 60 * 1000; + +export const branchFor = (plugin, release) => `changelog/${plugin}/v${release}`; +// One name for every start record (an artifact is scoped to its run), so the +// cap lists them by name instead of paging the repository's artifacts. +export const STARTED_ARTIFACT = "upstream-apply-changelog-started"; +export const doneArtifact = (release) => `upstream-apply-changelog-done-${release}`; +export const alreadyProcessed = (artifacts) => (artifacts?.total_count ?? 0) > 0; +export const handoffMarker = (key) => ``; +export const releaseMarker = (release) => ``; +const ISSUE_BODY_MAX = 65536; // GitHub's issue body limit +const TRUNCATED = "Handoff bodies are truncated proportionally to fit GitHub's 65536-character issue body limit."; +const KIND = /^[a-z][a-z-]{0,39}$/; +// ci-mode.md: the two reasons a plan stops before deriving any row, and +// hold-path, which the scan job sets when a planned branch changes one. +const STOP_REASONS = new Set(["cap-exceeded", "range-not-computed", "hold-path"]); +// range-not-computed can pass on a later run (the changelog fetch failed, or +// the page lags the tag), so it files nothing and leaves the release undone; +// cap-exceeded and hold-path come from the release's fixed changelog text, so +// its issue is filed and the release marked done. +export const marksDone = (stop) => stop !== "range-not-computed"; + +const STOP_WHY = { + "hold-path": "a planned branch changed a hold path, so no branch was opened", +}; +const stopLine = (release, stop) => + `The plan for Claude Code v${release} stopped (stop_reason \`${stop}\`): ${STOP_WHY[stop] ?? "no changelog row was derived and no branch was opened"}. ` + + "The release is marked done once this issue holds its handoffs, so later runs do not take it again."; +const releaseTitle = (release, n) => `Claude Code v${release} changelog: ${n} handoff${n === 1 ? "" : "s"}`; +const keysIn = (body) => [...body.matchAll(//g)].map((m) => m[1]); + +// One checklist item per handoff, its body indented under it, within `budget` +// characters: when the bodies do not fit, each is cut in proportion to its +// length and a line says so. +export function renderItems(handoffs, budget) { + const indent = (text) => text.replace(/^(?=.)/gm, " "); + const item = (h, body) => { + const kind = KIND.test(h.kind ?? "") ? h.kind : "handoff"; + return `- [ ] ${kind}: ${h.title}\n\n${body ? `${body}\n\n` : ""} ${handoffMarker(h.key)}\n`; + }; + const bodies = handoffs.map((h) => indent(h.body)); + const full = handoffs.map((h, i) => item(h, bodies[i])).join("\n"); + if (full.length <= budget) return full; + const fixed = handoffs.map((h) => item(h, "")).join("\n").length + TRUNCATED.length + 2; + const room = budget - fixed - handoffs.length * 2; + if (room < 0) throw new Error("the handoff checklist does not fit in one issue body"); + const total = bodies.reduce((sum, b) => sum + b.length, 0); + const cut = bodies.map((b) => b.slice(0, Math.floor((b.length * room) / total)).trimEnd()); + return `${TRUNCATED}\n\n${handoffs.map((h, i) => item(h, cut[i])).join("\n")}`; +} + +// The new release issue's title and body, as publish posts them; a stopped +// plan's issue opens with its stop_reason. +export function releaseIssue(release, handoffs, stop = null) { + const head = `${releaseMarker(release)}\n\n${stop ? `${stopLine(release, stop)}\n\n` : ""}`; + return { title: releaseTitle(release, handoffs.length), body: head + renderItems(handoffs, ISSUE_BODY_MAX - head.length) }; +} + +// Every string publish can post from the plan, as plain text for the scan +// job: each pull request title and body, the new release issue, and the +// handoff items untruncated, as an append or a comment posts them. +export function publishText(plan, release) { + const parts = plan.branches.flatMap((b) => [b.pr.title, b.pr.body]); + if (marksDone(plan.stop_reason) && (plan.handoffs.length || plan.stop_reason)) { + const issue = releaseIssue(release, plan.handoffs, plan.stop_reason); + parts.push(issue.title, issue.body, renderItems(plan.handoffs, Number.POSITIVE_INFINITY)); + } + return `${parts.join("\n\n")}\n`; +} + +// Files a release's handoffs as one issue, found again by its release marker +// in any state. Only an issue the lanes App opened counts, and only key +// markers in its body and the App's comments: anyone can type a marker. An +// open issue missing some keys gets them appended in one update, or in one +// comment when the body would pass the limit; a closed one is left alone. +// A stopped plan publish files (`stop` is a stop_reason marksDone accepts) +// is marked done once this returns, so it files the issue even with no +// handoffs, and a closed issue missing a key throws instead. +export async function fileReleaseHandoffs({ repo, release, handoffs, stop = null, api }) { + if (!handoffs.length && !stop) return "no handoffs; nothing filed"; + const marker = releaseMarker(release); + const app = LANES_APP_LOGIN.replace(/\[bot\]$/, ""); + const q = encodeURIComponent(`repo:${repo} is:issue author:app/${app} in:body "upstream-apply-changelog release=${release}"`); + const found = await api(`/search/issues?q=${q}&per_page=20`); + const issue = (found?.items ?? []).find( + (i) => i?.user?.login === LANES_APP_LOGIN && typeof i.body === "string" && i.body.includes(marker), + ); + if (!issue) { + await api(`/repos/${repo}/issues`, { method: "POST", body: releaseIssue(release, handoffs, stop) }); + return `filed ${handoffs.length} handoffs as one issue`; + } + if (issue.state !== "open" && !stop) return `the release issue #${issue.number} is closed; nothing filed`; + const comments = await listComments(api, repo, issue.number); + const own = comments.filter((c) => c?.user?.login === LANES_APP_LOGIN && typeof c.body === "string"); + const have = new Set([issue.body, ...own.map((c) => c.body)].flatMap(keysIn)); + const missing = handoffs.filter((h) => !have.has(h.key)); + if (!missing.length) return `the release issue #${issue.number} already holds every handoff`; + if (issue.state !== "open") { + throw new Error(`the release issue #${issue.number} is closed and lacks ${missing.length} handoffs; the stopped release is not marked done`); + } + const base = `${issue.body.trimEnd()}\n\n`; + const items = renderItems(missing, Number.POSITIVE_INFINITY); + if (base.length + items.length > ISSUE_BODY_MAX) { + // The body is full: the missing items go in one comment instead. + const head = `${marker}\n\n`; + const body = head + renderItems(missing, ISSUE_BODY_MAX - head.length); + await api(`/repos/${repo}/issues/${issue.number}/comments`, { method: "POST", body: { body } }); + return `commented ${missing.length} handoffs on the full release issue #${issue.number}`; + } + await api(`/repos/${repo}/issues/${issue.number}`, { + method: "PATCH", + body: { title: releaseTitle(release, have.size + missing.length), body: base + items }, + }); + return `appended ${missing.length} handoffs to the release issue #${issue.number}`; +} + +const COMMENT_PAGES = 10; + +// Every comment the release issue holds, read a full page at a time up to +// COMMENT_PAGES pages; more than that, or an answer that is not a list, +// throws so no key is filed twice. +async function listComments(api, repo, number) { + const all = []; + for (let page = 1; page <= COMMENT_PAGES; page++) { + const batch = await api(`/repos/${repo}/issues/${number}/comments?per_page=100&page=${page}`); + if (!Array.isArray(batch)) throw new Error(`the comments on #${number} did not come back as a list`); + all.push(...batch); + if (batch.length < 100) return all; + } + throw new Error(`#${number} has more comments than ${COMMENT_PAGES} pages hold`); +} + +function output(name, value) { + if (process.env.GITHUB_OUTPUT) appendFileSync(process.env.GITHUB_OUTPUT, `${name}=${value}\n`); +} + +const guardedFetch = rateLimitLatch.guard(globalThis.fetch); + +async function api(path, { method = "GET", body, allow404 = false } = {}) { + const res = await guardedFetch(`${process.env.GITHUB_API_URL || "https://api.github.com"}${path}`, { + method, + headers: { + authorization: `Bearer ${process.env.GH_TOKEN}`, + accept: "application/vnd.github+json", + "x-github-api-version": "2022-11-28", + ...(body ? { "content-type": "application/json" } : {}), + }, + body: body ? JSON.stringify(body) : undefined, + }); + if (allow404 && res.status === 404) return null; + if (!res.ok) throw new Error(`${method} ${path.split("?")[0]} answered ${res.status}`); + return res.status === 204 ? null : res.json(); +} + +// The unread releases, oldest first, from changelog-status.sh's status file +// (the file scripts/claude-code-release-issues.sh reads): `range: up to date` +// is none; `cap: within budget` lists them on `releases:`; any other shape +// (cap exceeded, range not computed) throws, so a person handles it. +export function releasesFromStatus(text) { + const value = (key) => text.match(new RegExp(`^${key}: (.*)$`, "m"))?.[1]?.trim() ?? ""; + if (value("range") === "up to date") return []; + // changelog-status.sh:59 prints `cap: within budget ( releases / items)`; + // :57 prints `releases:` space-separated, oldest first. + if (!value("cap").startsWith("within budget")) throw new Error(`changelog-status.sh reports cap: ${value("cap") || "missing"}`); + const versions = value("releases").split(/ +/).filter(Boolean); + if (!versions.length || !versions.every((v) => RELEASE.test(v))) throw new Error("the status file's releases line is malformed"); + return versions; +} + +// A scheduled gate checks the done marker of at most this many unread +// releases, newest first: --ci never moves the read marker, so the unread list +// only grows, and each check is one REST call per tick. Older releases wait +// for a person to move the marker. Ten is the changelog skill's default +// replay cap (changelog-status.sh, --cap-releases). +export const SCHEDULE_WINDOW = 10; + +// The releases a scheduled gate checks, newest first. +export const scheduledCandidates = (text) => releasesFromStatus(text).reverse().slice(0, SCHEDULE_WINDOW); + +// The newest release by version, from the release list (newest first is not +// guaranteed, so it sorts). Tags are `v`; any other tag is skipped. +export function newestRelease(releases) { + const versions = releases + .filter((r) => r && !r.draft && !r.prerelease && typeof r.tag_name === "string") + .map((r) => r.tag_name.replace(/^v/, "")) + .filter((v) => RELEASE.test(v)); + versions.sort((a, b) => { + const [x, y] = [a, b].map((v) => v.split(".").map(Number)); + return x[0] - y[0] || x[1] - y[1] || x[2] - y[2]; + }); + return versions.at(-1) ?? ""; +} + +// Runs started in the last 24 hours, counted from the gate's started +// artifacts, as red-main counts reverts "in any 24 hours". +export function startedInWindow(artifacts, now) { + return artifacts.filter( + (a) => a?.name === STARTED_ARTIFACT && now - Date.parse(a.created_at) < DAY_MS, + ).length; +} + +// The plan file parsed and checked. A parse error names no input: Node +// quotes the text it could not parse, and the plan is model output. +export function readPlan(text, ctx) { + let plan; + try { + plan = JSON.parse(text); + } catch { + throw new Error("ci-plan.json: not valid JSON"); + } + return validatePlan(plan, ctx); +} + +// Every message names a field and its index, never a value: the plan is +// model output and the scan job reports these before it scans anything. +export function validatePlan(plan, { release, baseSha }) { + const fail = (why) => { + throw new Error(`ci-plan.json: ${why}`); + }; + if (plan?.schema !== "changelog-ci-plan/1") fail("schema is not changelog-ci-plan/1"); + const { status } = plan; + if (status !== "planned" && status !== "stopped") fail("status is neither planned nor stopped"); + if (plan.range !== `${release}..${release}`) fail("range is not this run's release"); + if (plan.base_sha !== baseSha) fail("base_sha is not the commit the run checked out"); + // A stopped plan derived no rows, so it publishes no branch; its handoffs + // (ci-mode.md: cap exceeded carries a docs-conformance one) are still + // filed, under an issue line naming its stop_reason. + const stopped = status === "stopped"; + const stop_reason = !stopped ? null : STOP_REASONS.has(plan.stop_reason) ? plan.stop_reason : fail("stop_reason is not cap-exceeded, range-not-computed or hold-path"); + const branches = stopped ? [] : Array.isArray(plan.branches) ? plan.branches : fail("branches is not a list"); + const handoffs = Array.isArray(plan.handoffs) ? plan.handoffs : fail("handoffs is not a list"); + const seen = new Set(); + branches.forEach((b, i) => { + const at = `branches[${i}]`; + if (!PLUGIN.test(b?.plugin ?? "")) fail(`${at}.plugin is malformed`); + if (seen.has(b.plugin)) fail(`${at}.plugin repeats an earlier branch's plugin`); + seen.add(b.plugin); + if (b.branch !== branchFor(b.plugin, release)) fail(`${at}.branch is not changelog//v`); + if (b.base_sha !== baseSha) fail(`${at}.base_sha is not the run's base`); + if (!SHA.test(b.head_sha ?? "") || b.head_sha === baseSha) fail(`${at}.head_sha is not a commit after the base`); + const title = b.pr?.title; + if (typeof title !== "string" || !title.startsWith(`chore(${b.plugin}): `) || title.length > 200 || /[\r\n]/.test(title)) { + fail(`${at}.pr.title is not chore(): ... on one line of at most 200 characters`); + } + if (typeof b.pr?.body !== "string" || b.pr.body.length > 60000) fail(`${at}.pr.body is missing or too long`); + }); + const keys = new Set(); + handoffs.forEach((h, i) => { + const at = `handoffs[${i}]`; + if (!KEY.test(h?.key ?? "") || !h.key.startsWith(`changelog:${release}..${release}:`)) fail(`${at}.key is malformed`); + if (keys.has(h.key)) fail(`${at}.key repeats an earlier handoff's key`); + keys.add(h.key); + if (typeof h.title !== "string" || !h.title || h.title.length > 256 || /[\r\n]/.test(h.title)) fail(`${at}.title is bad`); + if (typeof h.body !== "string" || h.body.length > 60000) fail(`${at}.body is bad`); + }); + return { status, stop_reason, branches, handoffs }; +} + +// Every path the raw diff changes, either side of a rename, must sit under +// plugins// or .changes//, as a regular file. Messages name +// the branch index and the raw line number, never a path. +export function checkRawLog(raw, plugin, index = 0) { + const allowed = [`plugins/${plugin}/`, `.changes/${plugin}/`]; + raw + .split("\n") + .filter((l) => l.startsWith(":")) + .forEach((line, n) => { + const at = `branches[${index}] raw line ${n + 1}`; + const [meta, ...paths] = line.split("\t"); + const [srcMode, dstMode] = meta.slice(1).split(" "); + for (const mode of [srcMode, dstMode]) { + if (mode !== "000000" && mode !== "100644" && mode !== "100755") { + throw new Error(`${at} writes a mode other than a regular file; only regular files may change`); + } + } + for (const p of paths) { + if (!allowed.some((prefix) => p.startsWith(prefix)) || p.split("/").includes("..")) { + throw new Error(`${at} changes a path outside plugins/${plugin}/ and .changes/${plugin}/`); + } + } + }); +} + +// The security review lane's hold-paths, which scripts/sync-hold-floor.py +// writes from the hold floor (HOLD_PATH_CLASSES in merge-holds.mjs), the +// trusted-actor list and the security review's own files. Read from the +// checkout this module runs from, never the model's tree; unreadable or empty +// throws, so nothing is published unchecked. +export const HOLD_PATHS_FILE = new URL("../../workflows/pr-review-security-hosted.yml", import.meta.url); +export function readHoldPaths(text) { + const lines = text.split("\n"); + const at = (word) => lines.findIndex((l) => l.trimStart().startsWith(`# ${word} hold-floor hold-paths`)); + const [begin, end] = [at("BEGIN"), at("END")]; + if (begin < 0 || end < begin || lines[begin + 1]?.trim() !== "hold-paths: |") throw new Error("the hold-paths block was not found"); + const globs = lines.slice(begin + 2, end).map((l) => l.trim()).filter(Boolean); + if (!globs.length) throw new Error("the hold-paths block lists no path"); + return globs; +} +let holdPaths; +export const loadHoldPaths = () => (holdPaths ??= readHoldPaths(readFileSync(HOLD_PATHS_FILE, "utf8"))); + +// Whether a raw diff changes a hold path, either side of a rename. +export const changesHoldPath = (raw, globs = loadHoldPaths()) => + raw + .split("\n") + .filter((l) => l.startsWith(":")) + .some((line) => line.split("\t").slice(1).some((p) => globs.some((g) => matchesHoldPath(p, g)))); + +// The first planned branch whose net diff changes a hold path, or -1, after +// checking each one stays in its plugin. Each head must be in repoDir. +export function holdPathBranch(repoDir, baseSha, branches, globs = loadHoldPaths()) { + return branches.findIndex((b, i) => { + const raw = netRawDiff(repoDir, baseSha, b.head_sha); + checkRawLog(raw, b.plugin, i); + return changesHoldPath(raw, globs); + }); +} + +// The plan text stopped on hold-path: no branch, the handoffs kept. +export const stoppedPlanText = (text) => JSON.stringify({ ...JSON.parse(text), status: "stopped", stop_reason: "hold-path", branches: [] }); + +const git = (repo, args, extra = {}) => execFileSync("git", ["-C", repo, ...args], { encoding: "utf8", ...extra }).trim(); +const MAX_BLOB = 64 * 1024 * 1024; + +// The net raw diff base..head: what the one published commit changes, and +// what the scan job scanned. Intermediate commits are never published. +export const netRawDiff = (repoDir, baseSha, headSha) => + git(repoDir, ["diff-tree", "-r", "--no-renames", "--raw", "--no-abbrev", baseSha, headSha]); + +// The files the net diff changes: a deleted file has content null. +export function readNetChanges(repoDir, baseSha, headSha) { + return netRawDiff(repoDir, baseSha, headSha) + .split("\n") + .filter((line) => line.startsWith(":")) + .map((line) => { + const [meta, path] = line.split("\t"); + const [srcMode, dstMode, , dstSha] = meta.slice(1).split(" "); + if (dstMode === "000000") return { path, mode: srcMode, content: null }; + return { path, mode: dstMode, content: execFileSync("git", ["-C", repoDir, "cat-file", "blob", dstSha], { maxBuffer: MAX_BLOB }) }; + }); +} + +// `git diff --numstat` marks a binary file `-\t-\t`; the scan reads text, +// so a branch with one is refused. The message names no path. +export function refuseBinary(numstat, index = 0) { + if (numstat.split("\n").some((line) => line.startsWith("-\t-\t"))) { + throw new Error(`branches[${index}] changes a binary file; only text is published`); + } +} + +// One commit on the base, with no author or committer, so GitHub authors and +// signs it as the lanes App, as apply-patch does (apply-patch.mjs +// createGitHubApi). Its message is the scanned pull request title and its +// tree must be the planned head's: what is pushed is exactly what the scan +// job scanned. Returns the new commit. +export async function commitNetTree(changes, { message, baseSha, baseTree, headTree, api }) { + if (!changes.length) throw new Error("a planned branch changes nothing against the base"); + const entries = []; + for (const { path, mode, content } of changes) { + entries.push({ path, mode, type: "blob", sha: content === null ? null : await api.createBlob(Buffer.from(content).toString("base64")) }); + } + const tree = await api.createTree(baseTree, entries); + if (tree !== headTree) throw new Error(`the built tree ${tree} is not the planned head's ${headTree}`); + return api.createCommit(message, tree, [baseSha]); +} + +const SYNTHETIC_ENV = { + GIT_AUTHOR_NAME: "upstream-apply-changelog", + GIT_AUTHOR_EMAIL: "upstream-apply-changelog@users.noreply.github.com", + GIT_AUTHOR_DATE: "1970-01-01T00:00:00Z", + GIT_COMMITTER_NAME: "upstream-apply-changelog", + GIT_COMMITTER_EMAIL: "upstream-apply-changelog@users.noreply.github.com", + GIT_COMMITTER_DATE: "1970-01-01T00:00:00Z", +}; +export const SYNTHETIC_MESSAGE = "upstream-apply-changelog scanned tree"; + +// The scan job's stand-in for each planned head: one commit on the base +// holding the head's tree, with a fixed identity, date and message, bundled +// alone. The bundle push gets then carries no model commit, header, merge or +// intermediate blob, only what the scan read as the net diff. Returns each +// branch's { commit, tree } in plan order. +export function syntheticBundle(repoDir, baseSha, heads, bundlePath) { + const env = { ...process.env, ...SYNTHETIC_ENV }; + const quiet = { stdio: ["ignore", "pipe", "pipe"] }; + const made = heads.map((head, i) => { + const tree = git(repoDir, ["rev-parse", "--verify", `${head}^{tree}`], quiet); + const commit = git(repoDir, ["commit-tree", "--no-gpg-sign", tree, "-p", baseSha, "-m", SYNTHETIC_MESSAGE], { env, ...quiet }); + if (git(repoDir, ["rev-parse", `${commit}^{tree}`], quiet) !== tree) throw new Error(`branches[${i}]'s synthetic commit does not hold its head's tree`); + git(repoDir, ["update-ref", `refs/synthetic/${i}`, commit], quiet); + return { commit, tree }; + }); + git(repoDir, ["bundle", "create", "--quiet", bundlePath, `^${baseSha}`, ...heads.map((_, i) => `refs/synthetic/${i}`)], quiet); + return made; +} + +// The scan job's synthetic output: one { commit, tree } per planned branch. +export function readSynthetic(text, count) { + let list; + try { + list = JSON.parse(text || "[]"); + } catch { + throw new Error("the scan job's synthetic commits are not JSON"); + } + if (!Array.isArray(list) || list.length !== count || !list.every((s) => SHA.test(s?.commit ?? "") && SHA.test(s?.tree ?? ""))) { + throw new Error("the scan job's synthetic commits do not match the plan's branches"); + } + return list; +} + +// Fetches a branch's synthetic commit from the scanned bundle and checks it +// is one commit on the base holding the tree the scan recorded. +export function fetchSynthetic(repoDir, bundleFile, baseSha, { commit, tree }, local, i) { + git(repoDir, ["fetch", "--quiet", "--no-tags", bundleFile, `${commit}:${local}`]); + if (git(repoDir, ["rev-parse", local]) !== commit) throw new Error(`branches[${i}] is not at its synthetic commit`); + if (git(repoDir, ["rev-list", "--parents", "-n", "1", commit]) !== `${commit} ${baseSha}`) throw new Error(`branches[${i}] is not one commit on the base`); + if (git(repoDir, ["rev-parse", `${commit}^{tree}`]) !== tree) throw new Error(`branches[${i}] does not hold the scanned tree`); + return commit; +} + +async function gate(env) { + const repo = env.GITHUB_REPOSITORY; + let candidates; + if (env.EVENT_NAME === "workflow_dispatch") { + if (!RELEASE.test(env.RELEASE_INPUT ?? "")) throw new Error("the release input is not one MAJOR.MINOR.PATCH"); + candidates = [env.RELEASE_INPUT]; + } else { + candidates = scheduledCandidates(readFileSync(env.STATUS_FILE, "utf8")); + } + let release = ""; + for (const candidate of candidates) { + const done = await api(`/repos/${repo}/actions/artifacts?per_page=1&name=${doneArtifact(candidate)}`); + // Only the done marker skips a release: push skips a branch that exists + // and publish opens a pull request a pushed branch lacks, so a run after a + // failed publish finishes the work instead of stranding the branch. + if (alreadyProcessed(done)) { + console.log(`Claude Code ${candidate} was already processed`); + continue; + } + release = candidate; + break; + } + if (!release) { + console.log("no unprocessed Claude Code release; nothing to do"); + output("proceed", "false"); + return; + } + output("release", release); + const cap = Number(env.DAILY_CAP); + if (!Number.isInteger(cap) || cap < 1) throw new Error("the per-day cap is not a positive whole number"); + const recent = await api(`/repos/${repo}/actions/artifacts?per_page=100&name=${STARTED_ARTIFACT}`); + const started = startedInWindow(recent?.artifacts ?? [], Date.now()); + if (started >= cap) { + console.log(`::notice::upstream-apply-changelog started ${started} runs in the last 24 hours, at its cap of ${cap}; Claude Code ${release} waits`); + output("proceed", "false"); + return; + } + output("proceed", "true"); +} + +async function push(env) { + const { RELEASE: release, BASE_SHA: baseSha, PLAN_FILE, BUNDLE_FILE, REPO_DIR: repoDir, GITHUB_REPOSITORY: repo } = env; + const plan = readPlan(readFileSync(PLAN_FILE, "utf8"), { release, baseSha }); + const writer = createGitHubApi({ token: env.GH_TOKEN, repository: repo, apiUrl: env.GITHUB_API_URL || "https://api.github.com" }); + const heads = {}; + if (plan.branches.length && !existsSync(BUNDLE_FILE)) throw new Error("the plan names branches and the run left no bundle"); + // The bundle holds the scan job's synthetic commits only, never the + // model's heads, which push never reads. + const synthetic = readSynthetic(env.SYNTHETIC, plan.branches.length); + for (const [i, b] of plan.branches.entries()) { + const local = `refs/upstream-apply-changelog/${b.plugin}`; + const commit = fetchSynthetic(repoDir, BUNDLE_FILE, baseSha, synthetic[i], local, i); + const raw = netRawDiff(repoDir, baseSha, commit); + checkRawLog(raw, b.plugin, i); + if (changesHoldPath(raw)) throw new Error(`branches[${i}] changes a hold path; the scan job should have stopped the release`); + refuseBinary(git(repoDir, ["diff", "--numstat", "--no-renames", baseSha, commit]), i); + const headTree = synthetic[i].tree; + const existing = await api(`/repos/${repo}/git/ref/heads/${b.branch}`, { allow404: true }); + if (existing) { + // A re-run's branch counts only when its tip holds the planned tree. + const tip = existing.object?.sha; + const commit = SHA.test(tip ?? "") ? await api(`/repos/${repo}/git/commits/${tip}`) : null; + if (commit?.tree?.sha === headTree) { + heads[b.branch] = tip; + console.log(`${b.branch} already exists at the planned tree; not pushed`); + continue; + } + // Another tree is an earlier run's work: done once it has a pull + // request, else stranded, so the run fails before the done marker. + const prs = await api(`/repos/${repo}/pulls?state=all&head=${repo.split("/")[0]}:${encodeURIComponent(b.branch)}`); + if (!prs?.length) throw new Error(`branches[${i}] already exists with another tree and no pull request; open one for it or delete it, then dispatch again`); + console.log(`${b.branch} already has pull request #${prs[0].number} with another tree; not pushed`); + continue; + } + const head = await commitNetTree(readNetChanges(repoDir, baseSha, commit), { + message: b.pr.title, + baseSha, + baseTree: git(repoDir, ["rev-parse", `${baseSha}^{tree}`]), + headTree, + api: writer, + }); + await writer.createRef(`refs/heads/${b.branch}`, head); + heads[b.branch] = head; + } + output("heads", JSON.stringify(heads)); + console.log(`${Object.keys(heads).length} of ${plan.branches.length} branches are at their planned tree`); +} + +// The scan job's path check, after it fetched each planned head into REPO_DIR. +// A branch outside its plugin throws; one changing a hold path rewrites the +// plan in IN to stopped (hold-path) with no branch, and empties heads.txt and +// rewrites publish.txt to match, so the scans read what push and publish get. +async function scan(env) { + const { IN, RELEASE: release, BASE_SHA: baseSha, REPO_DIR: repoDir } = env; + const text = readFileSync(`${IN}/plan.json`, "utf8"); + const plan = readPlan(text, { release, baseSha }); + // Only this check sets hold-path; the model's plan may not claim it. + if (plan.stop_reason === "hold-path") throw new Error("ci-plan.json: stop_reason hold-path is set by the scan job, not the plan"); + const held = holdPathBranch(repoDir, baseSha, plan.branches); + if (held < 0) { + console.log(`${plan.branches.length} branches stay in their plugins and change no hold path`); + return; + } + const stopped = stoppedPlanText(text); + writeFileSync(`${IN}/plan.json`, stopped); + writeFileSync(`${IN}/heads.txt`, ""); + writeFileSync(`${IN}/publish.txt`, publishText(readPlan(stopped, { release, baseSha }), release)); + console.log(`::warning::branches[${held}] changes a hold path; Claude Code v${release} stops with stop_reason hold-path and no branch is opened`); +} + +async function publish(env) { + const { RELEASE: release, BASE_SHA: baseSha, PLAN_FILE, GITHUB_REPOSITORY: repo } = env; + const plan = readPlan(readFileSync(PLAN_FILE, "utf8"), { release, baseSha }); + const owner = repo.split("/")[0]; + // push's heads output: each branch it pushed or found at the planned tree. + const heads = JSON.parse(env.HEADS || "{}"); + let opened = 0; + for (const b of plan.branches) { + const ref = await api(`/repos/${repo}/git/ref/heads/${b.branch}`, { allow404: true }); + if (!SHA.test(heads[b.branch] ?? "") || ref?.object?.sha !== heads[b.branch]) { + console.log(`${b.branch} is missing or not at the head push left; no pull request`); + continue; + } + const prs = await api(`/repos/${repo}/pulls?state=all&head=${owner}:${encodeURIComponent(b.branch)}`); + if (prs.length) { + console.log(`${b.branch} already has pull request #${prs[0].number}`); + continue; + } + await api(`/repos/${repo}/pulls`, { method: "POST", body: { title: b.pr.title, body: b.pr.body, head: b.branch, base: "main" } }); + opened++; + } + console.log(`opened ${opened} pull requests`); + // A deterministic stop's release is marked done once its issue holds every + // handoff; a filing that fails throws, and the next run takes it again. + if (!marksDone(plan.stop_reason)) { + console.log(`::notice::the plan for Claude Code v${release} stopped (${plan.stop_reason}); nothing filed, and the next run takes it again`); + output("done", "false"); + return; + } + console.log(await fileReleaseHandoffs({ repo, release, handoffs: plan.handoffs, stop: plan.stop_reason, api })); + output("done", "true"); +} + +if (import.meta.url === pathToFileURL(process.argv[1] ?? "").href) { + const commands = { gate, scan, push, publish }; + const run = commands[process.argv[2]]; + if (!run) { + console.error("usage: upstream-apply-changelog.mjs gate|scan|push|publish"); + process.exit(2); + } + run(process.env).catch((error) => { + console.log(`::error::${String(error.message).replace(/[\r\n]+/g, " ")}`); + process.exit(1); + }); +} diff --git a/.github/scripts/upstream-apply-changelog/upstream-apply-changelog.test.mjs b/.github/scripts/upstream-apply-changelog/upstream-apply-changelog.test.mjs new file mode 100644 index 00000000000..7d653e1daba --- /dev/null +++ b/.github/scripts/upstream-apply-changelog/upstream-apply-changelog.test.mjs @@ -0,0 +1,668 @@ +import assert from "node:assert/strict"; +import { execFileSync } from "node:child_process"; +import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { dirname, join } from "node:path"; +import { test } from "node:test"; + +import { + alreadyProcessed, + branchFor, + changesHoldPath, + checkRawLog, + commitNetTree, + fetchSynthetic, + fileReleaseHandoffs, + handoffMarker, + holdPathBranch, + loadHoldPaths, + marksDone, + netRawDiff, + newestRelease, + publishText, + readNetChanges, + readHoldPaths, + readPlan, + readSynthetic, + refuseBinary, + releaseIssue, + releaseMarker, + releasesFromStatus, + renderItems, + scheduledCandidates, + startedInWindow, + stoppedPlanText, + SYNTHETIC_MESSAGE, + syntheticBundle, + validatePlan, +} from "./upstream-apply-changelog.mjs"; + +const BASE = "a".repeat(40); +const HEAD = "b".repeat(40); +const plan = (over = {}) => ({ + schema: "changelog-ci-plan/1", + status: "planned", + range: "2.1.300..2.1.300", + base_sha: BASE, + branches: [ + { + plugin: "hooks-kit", + branch: "changelog/hooks-kit/v2.1.300", + base_sha: BASE, + head_sha: HEAD, + pr: { title: "chore(hooks-kit): address Claude Code v2.1.300 changelog", body: "Refs: #7407" }, + }, + ], + handoffs: [{ key: "changelog:2.1.300..2.1.300:adopt:plugins/x/a.sh", title: "t", body: "b" }], + ...over, +}); +const ctx = { release: "2.1.300", baseSha: BASE }; + +test("the newest release is the highest version, skipping drafts, prereleases and other tags", () => { + const releases = [ + { tag_name: "v2.1.99" }, + { tag_name: "v2.1.300" }, + { tag_name: "v2.1.301", prerelease: true }, + { tag_name: "v9.9.9", draft: true }, + { tag_name: "nightly" }, + ]; + assert.equal(newestRelease(releases), "2.1.300"); + assert.equal(newestRelease([]), ""); +}); + +test("the per-day cap counts only start records from the last 24 hours", () => { + const now = Date.parse("2026-11-02T12:00:00Z"); + const artifacts = [ + { name: "upstream-apply-changelog-started", created_at: "2026-11-02T01:00:00Z" }, + { name: "upstream-apply-changelog-started", created_at: "2026-11-01T11:00:00Z" }, + { name: "upstream-apply-changelog-done-2.1.300", created_at: "2026-11-02T02:00:00Z" }, + ]; + assert.equal(startedInWindow(artifacts, now), 1); +}); + +test("a valid plan passes and a stopped plan keeps its handoffs and publishes no branch", () => { + assert.equal(validatePlan(plan(), ctx).branches.length, 1); + // ci-mode.md: cap exceeded is status stopped with one docs-conformance handoff. + const conformance = { key: "changelog:2.1.300..2.1.300:docs-conformance:plugins/harness-ops/skills/changelog/SKILL.md", title: "t", body: "b" }; + assert.deepEqual(validatePlan(plan({ status: "stopped", stop_reason: "cap-exceeded", handoffs: [conformance] }), ctx), { + status: "stopped", + stop_reason: "cap-exceeded", + branches: [], + handoffs: [conformance], + }); + assert.equal(validatePlan(plan({ status: "stopped", stop_reason: "range-not-computed", handoffs: [] }), ctx).stop_reason, "range-not-computed"); + assert.equal(validatePlan(plan(), ctx).stop_reason, null); + assert.throws( + () => validatePlan(plan({ status: "stopped", stop_reason: "cap-exceeded", handoffs: [{ ...conformance, key: "changelog:2.1.1..2.1.1:docs-conformance:x" }] }), ctx), + /handoffs\[0\]\.key/, + ); + assert.throws(() => validatePlan(plan({ status: "stopped", stop_reason: "cap-exceeded", range: "2.1.299..2.1.299" }), ctx), /range/); +}); + +test("a stopped plan without a known stop_reason is refused, naming the field and not the value", () => { + const SECRET = "zzPLANTEDzz"; + for (const stop_reason of [undefined, null, "", SECRET]) { + assert.throws( + () => validatePlan(plan({ status: "stopped", stop_reason }), ctx), + (e) => /stop_reason/.test(e.message) && !e.message.includes(SECRET), + ); + } +}); + +test("a plan for another release, base or branch name is refused", () => { + assert.throws(() => validatePlan(plan({ range: "2.1.299..2.1.299" }), ctx), /range/); + assert.throws(() => validatePlan(plan({ base_sha: HEAD }), ctx), /base_sha/); + const renamed = plan(); + renamed.branches[0].branch = "main"; + assert.throws(() => validatePlan(renamed, ctx), /branches\[0\]\.branch is not changelog\/\/v/); + const titled = plan(); + titled.branches[0].pr.title = "feat(other): x"; + assert.throws(() => validatePlan(titled, ctx), /branches\[0\]\.pr\.title/); + assert.throws(() => validatePlan(plan({ handoffs: [{ key: "changelog:2.1.1..2.1.1:adopt:x", title: "t", body: "b" }] }), ctx), /handoffs\[0\]\.key/); + const empty = plan(); + empty.branches[0].head_sha = BASE; + assert.throws(() => validatePlan(empty, ctx), /branches\[0\]\.head_sha/); + assert.equal(branchFor("hooks-kit", "2.1.300"), "changelog/hooks-kit/v2.1.300"); +}); + +test("a refusal names the field and index, never the plan's value", () => { + const SECRET = "zzPLANTEDzz"; + const cases = []; + const renamed = plan(); + renamed.branches[0].branch = SECRET; + cases.push([renamed, /branches\[0\]\.branch/]); + const twice = plan(); + twice.branches.push({ ...twice.branches[0], branch: SECRET }); + cases.push([twice, /branches\[1\]\.plugin/]); + const titled = plan(); + titled.branches[0].pr.title = `chore(hooks-kit): ${SECRET}\n`; + cases.push([titled, /branches\[0\]\.pr\.title/]); + const keyed = plan({ handoffs: [{ key: `changelog:2.1.300..2.1.300:adopt:${SECRET}`, title: "t", body: "b" }, { key: `changelog:2.1.300..2.1.300:adopt:${SECRET}`, title: "t", body: "b" }] }); + cases.push([keyed, /handoffs\[1\]\.key/]); + const handTitled = plan({ handoffs: [{ key: "changelog:2.1.300..2.1.300:adopt:x", title: `${SECRET}\n`, body: "b" }] }); + cases.push([handTitled, /handoffs\[0\]\.title/]); + for (const [p, want] of cases) { + assert.throws( + () => validatePlan(p, ctx), + (e) => want.test(e.message) && !e.message.includes(SECRET), + ); + } + assert.throws(() => readPlan(`{"schema": ${SECRET}}`, ctx), (e) => /not valid JSON/.test(e.message) && !e.message.includes(SECRET)); + assert.equal(readPlan(JSON.stringify(plan()), ctx).branches.length, 1); + assert.throws( + () => checkRawLog(`:100644 100644 1 2 M\tplugins/hooks-kit/a\n:100644 100644 1 2 M\t${SECRET}`, "hooks-kit", 3), + (e) => /branches\[3\] raw line 2/.test(e.message) && !e.message.includes(SECRET), + ); +}); + +test("the publish text holds every pull request and issue string publish can post, decoded", async () => { + const p = plan({ + handoffs: [ + { key: "changelog:2.1.300..2.1.300:adopt:a", kind: "adopt", title: "first \"quoted\"", body: "line one\nline two" }, + { key: "changelog:2.1.300..2.1.300:adopt:b", kind: "adopt", title: "second", body: "z".repeat(50000) }, + { key: "changelog:2.1.300..2.1.300:adopt:c", kind: "adopt", title: "third", body: "y".repeat(50000) }, + ], + }); + p.branches[0].pr.body = "Refs: #7407\nwith \"quotes\" and a tab\t."; + const v = validatePlan(p, ctx); + const text = publishText(v, "2.1.300"); + assert.ok(text.includes(v.branches[0].pr.title)); + assert.ok(text.includes("with \"quotes\" and a tab\t.")); + const f = issueApi(); + await fileReleaseHandoffs({ repo: REPO, release: "2.1.300", handoffs: v.handoffs, api: f.api }); + const posted = f.writes()[0].body; + assert.deepEqual(posted, releaseIssue("2.1.300", v.handoffs)); + assert.ok(text.includes(posted.title) && text.includes(posted.body)); + assert.ok(posted.body.includes("truncated proportionally")); + assert.ok(text.includes(renderItems(v.handoffs, Number.POSITIVE_INFINITY))); + assert.equal(publishText(validatePlan(plan({ handoffs: [] }), ctx), "2.1.300").includes(releaseMarker("2.1.300")), false); +}); + +test("a stopped plan's publish text holds the issue it posts, stop line included, even with no handoffs", async () => { + const v = validatePlan(plan({ status: "stopped", stop_reason: "cap-exceeded", handoffs: [] }), ctx); + const f = issueApi(); + await fileReleaseHandoffs({ repo: REPO, release: "2.1.300", handoffs: v.handoffs, stop: v.stop_reason, api: f.api }); + const posted = f.writes()[0].body; + const text = publishText(v, "2.1.300"); + assert.ok(text.includes(posted.title) && text.includes(posted.body)); + assert.ok(posted.body.includes("stop_reason `cap-exceeded`")); +}); + +test("only a deterministic stop marks the release done; range-not-computed is retried and posts nothing", () => { + assert.equal(marksDone(null), true); + assert.equal(marksDone("cap-exceeded"), true); + assert.equal(marksDone("range-not-computed"), false); + const v = validatePlan(plan({ status: "stopped", stop_reason: "range-not-computed", handoffs: [] }), ctx); + assert.equal(publishText(v, "2.1.300"), "\n"); +}); + +test("the done marker uploads only when publish says the release is done", () => { + const wf = readFileSync(new URL("../../workflows/upstream-apply-changelog.yml", import.meta.url), "utf8"); + const step = wf.split(/\n(?= {6}- name: )/).filter((s) => s.includes("name: upstream-apply-changelog-done-")); + assert.equal(step.length, 1); + assert.match(step[0], /\n {8}if: steps\.publish\.outcome == 'success' && steps\.publish\.outputs\.done == 'true'\n/); +}); + +test("a commit outside its plugin, or writing a link or submodule, is refused", () => { + const ok = ":000000 100644 0000 1111 A\tplugins/hooks-kit/SKILL.md\n:000000 100644 0000 2222 A\t.changes/hooks-kit/x.md"; + assert.doesNotThrow(() => checkRawLog(ok, "hooks-kit")); + assert.throws(() => checkRawLog(":100644 100644 1 2 M\tplugins/other/SKILL.md", "hooks-kit"), /outside/); + assert.throws(() => checkRawLog(":100644 100644 1 2 M\t.github/workflows/x.yml", "hooks-kit"), /outside/); + assert.throws(() => checkRawLog(":000000 120000 0 2 A\tplugins/hooks-kit/link", "hooks-kit"), /mode other than a regular file/); + assert.throws(() => checkRawLog(":000000 160000 0 2 A\tplugins/hooks-kit/sub", "hooks-kit"), /mode other than a regular file/); +}); + +const SECURITY_SKILL = "plugins/review/skills/security-review/SKILL.md"; +const RISK_SCORER = "plugins/source-control/skills/babysit-prs/scripts/babysit_risk.py"; + +test("the hold paths come from the security review's generated hold-paths block, and a missing or empty block throws", () => { + const globs = loadHoldPaths(); + assert.ok(globs.includes("plugins/review/skills/security-review/**")); + assert.ok(globs.includes(RISK_SCORER)); + assert.throws(() => readHoldPaths("jobs: {}\n"), /hold-paths block was not found/); + assert.throws(() => readHoldPaths(" # BEGIN hold-floor hold-paths: x\n hold-paths: |\n # END hold-floor hold-paths\n"), /lists no path/); +}); + +test("a diff touching the security review skill or the risk scorer changes a hold path; an ordinary plugin path does not", () => { + const raw = (path) => `:100644 100644 1 2 M\t${path}`; + assert.equal(changesHoldPath(raw(SECURITY_SKILL)), true); + assert.equal(changesHoldPath(raw(RISK_SCORER)), true); + assert.equal(changesHoldPath(raw("plugins/Review/Skills/Security-Review/skill.md")), true); + // Either side of a rename counts. + assert.equal(changesHoldPath(`:100644 100644 1 2 R100\t${SECURITY_SKILL}\tplugins/review/skills/x.md`), true); + assert.equal(changesHoldPath(raw("plugins/discovery/skills/explore/SKILL.md")), false); + assert.equal(changesHoldPath(raw("plugins/review/skills/code-review/SKILL.md")), false); +}); + +test("the scan stops a release whose planned branch changes a hold path, naming no path, and passes an ordinary one", (t) => { + const dir = mkdtempSync(join(tmpdir(), "uac-")); + t.after(() => rmSync(dir, { recursive: true, force: true })); + const env = { PATH: process.env.PATH, HOME: dir, GIT_CONFIG_NOSYSTEM: "1", GIT_AUTHOR_NAME: "t", GIT_AUTHOR_EMAIL: "t@t", GIT_COMMITTER_NAME: "t", GIT_COMMITTER_EMAIL: "t@t" }; + const git = (...args) => execFileSync("git", ["-C", dir, ...args], { env, encoding: "utf8" }).trim(); + const commitFile = (path, from) => { + git("checkout", "-q", "--detach", from); + mkdirSync(join(dir, dirname(path)), { recursive: true }); + writeFileSync(join(dir, path), "edit"); + git("add", "."); + git("commit", "-qm", path); + return git("rev-parse", "HEAD"); + }; + git("init", "-q", "-b", "main"); + writeFileSync(join(dir, "keep.md"), "k"); + git("add", "."); + git("commit", "-qm", "base"); + const base = git("rev-parse", "HEAD"); + const branch = (plugin, head) => ({ plugin, head_sha: head }); + const ordinary = commitFile("plugins/discovery/skills/explore/SKILL.md", base); + assert.equal(holdPathBranch(dir, base, [branch("discovery", ordinary)]), -1); + const skill = commitFile(SECURITY_SKILL, base); + const risk = commitFile(RISK_SCORER, base); + assert.equal(holdPathBranch(dir, base, [branch("discovery", ordinary), branch("review", skill)]), 1); + assert.equal(holdPathBranch(dir, base, [branch("source-control", risk)]), 0); + // A branch outside its plugin still throws before any hold path is read. + assert.throws(() => holdPathBranch(dir, base, [branch("discovery", skill)]), (e) => /branches\[0\] raw line 1 changes a path outside/.test(e.message) && !e.message.includes("security-review")); + + // The scan subcommand rewrites the scan job's inputs to the stopped plan. + const scanIn = join(dir, ".scan-in"); + mkdirSync(scanIn); + const planned = plan({ base_sha: base }); + planned.branches[0] = { plugin: "review", branch: "changelog/review/v2.1.300", base_sha: base, head_sha: skill, pr: { title: "chore(review): x", body: "b" } }; + writeFileSync(join(scanIn, "plan.json"), JSON.stringify(planned)); + writeFileSync(join(scanIn, "heads.txt"), `${skill}\n`); + const runScan = () => + execFileSync(process.execPath, [new URL("./upstream-apply-changelog.mjs", import.meta.url).pathname, "scan"], { + env: { PATH: process.env.PATH, HOME: dir, GIT_CONFIG_NOSYSTEM: "1", GIT_CONFIG_GLOBAL: "/dev/null", IN: scanIn, RELEASE: "2.1.300", BASE_SHA: base, REPO_DIR: dir }, + encoding: "utf8", + stdio: ["ignore", "pipe", "pipe"], + }); + const out = runScan(); + assert.match(out, /::warning::branches\[0\] changes a hold path/); + assert.equal(out.includes("security-review"), false); + const rewritten = readPlan(readFileSync(join(scanIn, "plan.json"), "utf8"), { release: "2.1.300", baseSha: base }); + assert.deepEqual([rewritten.stop_reason, rewritten.branches.length], ["hold-path", 0]); + assert.equal(readFileSync(join(scanIn, "heads.txt"), "utf8"), ""); + assert.ok(readFileSync(join(scanIn, "publish.txt"), "utf8").includes("stop_reason `hold-path`")); + // A plan that claims hold-path itself is refused: only the scan sets it. + assert.throws(runScan, (e) => /stop_reason hold-path is set by the scan job/.test(String(e.stdout))); + + // The stopped plan keeps its handoffs, publishes no branch and marks done. + const stopped = readPlan(stoppedPlanText(JSON.stringify(plan())), ctx); + assert.deepEqual([stopped.status, stopped.stop_reason, stopped.branches], ["stopped", "hold-path", []]); + assert.equal(stopped.handoffs.length, 1); + assert.equal(marksDone(stopped.stop_reason), true); + const issue = releaseIssue("2.1.300", stopped.handoffs, stopped.stop_reason); + assert.ok(issue.body.includes("stop_reason `hold-path`): a planned branch changed a hold path")); + assert.ok(publishText(stopped, "2.1.300").includes(issue.body)); +}); + +test("only the done marker skips a release, so a run after a failed publish finishes the work", () => { + assert.equal(alreadyProcessed({ total_count: 1, artifacts: [{ name: "upstream-apply-changelog-done-2.1.300" }] }), true); + assert.equal(alreadyProcessed({ total_count: 0, artifacts: [] }), false); + assert.equal(alreadyProcessed(null), false); +}); + +test("the status file gives the unread releases, none when up to date, and stops on anything else", () => { + assert.deepEqual(releasesFromStatus("range: 2.1.299..2.1.300 (2 releases, 5 core items)\nreleases: 2.1.299 2.1.300\ncap: within budget (2 releases / 5 items)\n"), ["2.1.299", "2.1.300"]); + assert.deepEqual(releasesFromStatus("range: up to date\n"), []); + assert.throws(() => releasesFromStatus("range: 2.1.1..2.1.90 (90 releases, 400 core items)\ncap: exceeded (90 releases / 400 items)\n"), /cap: exceeded/); + assert.throws(() => releasesFromStatus("range: a..b\ncap: within budget\nreleases: v2\n"), /malformed/); +}); + +test("a scheduled gate checks only the newest unread releases, newest first", () => { + const versions = Array.from({ length: 14 }, (_, i) => `2.1.${300 + i}`); + const status = `range: 2.1.300..2.1.313 (14 releases, 90 core items)\nreleases: ${versions.join(" ")}\ncap: within budget (1000 releases / 1000000 items)\n`; + assert.deepEqual(scheduledCandidates(status), ["2.1.313", "2.1.312", "2.1.311", "2.1.310", "2.1.309", "2.1.308", "2.1.307", "2.1.306", "2.1.305", "2.1.304"]); + assert.deepEqual(scheduledCandidates("range: up to date\n"), []); +}); + +// A fake Git Data API: each object's sha is its index in the calls made. +function fakeApi() { + const calls = []; + const sha = (kind) => { + calls.push(kind); + return `${kind}-${calls.length}`; + }; + return { + calls, + createBlob: async (content) => sha(`blob:${content}`), + createTree: async (base, entries) => sha(`tree:${base}:${entries.map((e) => `${e.path}=${e.sha}`).join(",")}`), + createCommit: async (message, tree, parents) => sha(`commit:${message.trim()}:${tree}:${parents.join(",")}`), + }; +} + +test("a branch becomes one App commit on the base, titled with its pull request title, deleting a removed file", async () => { + const api = fakeApi(); + const changes = [ + { path: "plugins/p/a.md", mode: "100644", content: Buffer.from("a") }, + { path: "plugins/p/old.md", mode: "100644", content: null }, + ]; + const tree = "tree:t0:plugins/p/a.md=blob:YQ==-1,plugins/p/old.md=null-2"; + const head = await commitNetTree(changes, { message: "chore(p): title", baseSha: BASE, baseTree: "t0", headTree: tree, api }); + assert.equal(api.calls.length, 3); + assert.equal(api.calls[2], `commit:chore(p): title:${tree}:${BASE}`); + assert.equal(head, `${api.calls[2]}-3`); +}); + +test("a built tree that is not the planned head's, or an empty change, makes no commit", async () => { + const changes = [{ path: "plugins/p/a.md", mode: "100644", content: Buffer.from("a") }]; + const api = fakeApi(); + await assert.rejects(commitNetTree(changes, { message: "m", baseSha: BASE, baseTree: "t0", headTree: "other", api }), /not the planned head's/); + assert.ok(!api.calls.some((c) => c.startsWith("commit:"))); + await assert.rejects(commitNetTree([], { message: "m", baseSha: BASE, baseTree: "t0", headTree: "t0", api: fakeApi() }), /changes nothing/); +}); + +test("the net change reads each file once at its head content, whatever the commits in between", (t) => { + const dir = mkdtempSync(join(tmpdir(), "uac-")); + t.after(() => rmSync(dir, { recursive: true, force: true })); + const env = { PATH: process.env.PATH, HOME: dir, GIT_CONFIG_NOSYSTEM: "1", GIT_AUTHOR_NAME: "t", GIT_AUTHOR_EMAIL: "t@t", GIT_COMMITTER_NAME: "t", GIT_COMMITTER_EMAIL: "t@t" }; + const git = (...args) => execFileSync("git", ["-C", dir, ...args], { env, encoding: "utf8" }).trim(); + git("init", "-q", "-b", "main"); + writeFileSync(join(dir, "keep.md"), "k"); + git("add", "."); + git("commit", "-qm", "base"); + const base = git("rev-parse", "HEAD"); + writeFileSync(join(dir, "new.md"), "n1"); + git("add", "."); + git("commit", "-qm", "add new"); + writeFileSync(join(dir, "new.md"), "n2"); + git("add", "."); + git("commit", "-qm", "edit new"); + git("rm", "-q", "keep.md"); + git("commit", "-qm", "drop keep"); + const changes = readNetChanges(dir, base, git("rev-parse", "HEAD")); + assert.deepEqual( + changes.map((e) => [e.path, e.mode, e.content === null ? null : String(e.content)]), + [["keep.md", "100644", null], ["new.md", "100644", "n2"]], + ); + writeFileSync(join(dir, "img.bin"), Buffer.from([0, 1, 2, 0, 255])); + git("add", "."); + git("commit", "-qm", "binary"); + const numstat = git("diff", "--numstat", base, "HEAD"); + assert.throws(() => refuseBinary(numstat, 2), (e) => /branches\[2\] changes a binary file/.test(e.message) && !e.message.includes("img.bin")); + assert.doesNotThrow(() => refuseBinary(git("diff", "--numstat", base, "HEAD~1"), 2)); + // A merge commit's change shows in the net diff, so the path check sees it. + git("checkout", "-qb", "side", base); + git("checkout", "-q", "main", "--", "new.md"); + git("commit", "-qm", "side"); + git("merge", "-q", "--no-edit", "main"); + assert.match(netRawDiff(dir, base, git("rev-parse", "HEAD")), /\tnew\.md$/m); +}); + +test("the scanned bundle holds one fixed commit per head with its tree, and no blob a merge added then removed", (t) => { + const dir = mkdtempSync(join(tmpdir(), "uac-")); + t.after(() => rmSync(dir, { recursive: true, force: true })); + const env = { PATH: process.env.PATH, HOME: dir, GIT_CONFIG_NOSYSTEM: "1", GIT_AUTHOR_NAME: "model", GIT_AUTHOR_EMAIL: "m@m", GIT_COMMITTER_NAME: "model", GIT_COMMITTER_EMAIL: "m@m" }; + const repo = join(dir, "repo"); + const fresh = join(dir, "fresh"); + const git = (cwd, ...args) => execFileSync("git", ["-C", cwd, ...args], { env, encoding: "utf8" }).trim(); + git(dir, "init", "-q", "-b", "main", repo); + writeFileSync(join(repo, "keep.md"), "k"); + git(repo, "add", "."); + git(repo, "commit", "-qm", "base"); + const base = git(repo, "rev-parse", "HEAD"); + git(repo, "tag", "base"); + git(repo, "checkout", "-qb", "side"); + writeFileSync(join(repo, "secret.md"), "SECRET-VALUE-0123456789"); + git(repo, "add", "."); + git(repo, "commit", "-qm", "add secret"); + const secret = git(repo, "rev-parse", "side:secret.md"); + git(repo, "checkout", "-qb", "head", base); + writeFileSync(join(repo, "plugin.md"), "p"); + git(repo, "add", "."); + git(repo, "commit", "-qm", "plugin change"); + git(repo, "merge", "-q", "--no-edit", "side"); + git(repo, "rm", "-q", "secret.md"); + git(repo, "commit", "-qm", "drop secret"); + const head = git(repo, "rev-parse", "HEAD"); + assert.equal(git(repo, "cat-file", "-t", secret), "blob"); + + const bundle = join(dir, "branches.bundle"); + const [made] = syntheticBundle(repo, base, [head], bundle); + assert.equal(made.tree, git(repo, "rev-parse", `${head}^{tree}`)); + assert.deepEqual(syntheticBundle(repo, base, [head], join(dir, "again.bundle")), [made]); + assert.equal(git(repo, "log", "-1", "--format=%P|%an|%ae|%aI|%cn|%ce|%cI|%B", made.commit), + `${base}|upstream-apply-changelog|upstream-apply-changelog@users.noreply.github.com|1970-01-01T00:00:00Z|upstream-apply-changelog|upstream-apply-changelog@users.noreply.github.com|1970-01-01T00:00:00Z|${SYNTHETIC_MESSAGE}`); + + git(dir, "init", "-q", "-b", "main", fresh); + git(fresh, "fetch", "-q", "--no-tags", repo, "refs/tags/base:refs/base"); + assert.equal(fetchSynthetic(fresh, bundle, base, made, "refs/scanned", 0), made.commit); + const has = (sha) => { + try { + git(fresh, "cat-file", "-e", sha); + return true; + } catch { + return false; + } + }; + assert.equal(has(made.commit), true); + assert.equal(has(secret), false); + assert.equal(has(head), false); + assert.deepEqual(readNetChanges(fresh, base, made.commit).map((e) => e.path), ["plugin.md"]); + assert.throws(() => fetchSynthetic(fresh, bundle, base, { ...made, tree: base }, "refs/other", 3), /branches\[3\] does not hold the scanned tree/); +}); + +test("the scan job's synthetic list must name one commit and tree per planned branch", () => { + const one = [{ commit: HEAD, tree: BASE }]; + assert.deepEqual(readSynthetic(JSON.stringify(one), 1), one); + assert.deepEqual(readSynthetic("", 0), []); + assert.throws(() => readSynthetic(JSON.stringify(one), 2), /do not match/); + assert.throws(() => readSynthetic(JSON.stringify([{ commit: "x", tree: BASE }]), 1), /do not match/); + assert.throws(() => readSynthetic("{", 1), /not JSON/); +}); + +const REPO = "o/r"; +const REL = "2.1.300"; +const APP = { login: "melodic-automation-lanes[bot]", type: "Bot" }; +const OUTSIDER = { login: "someone", type: "User" }; +const handoff = (n, over = {}) => ({ key: `changelog:${REL}..${REL}:adopt:plugins/x/${n}.sh`, kind: "adopt", title: `adopt ${n}`, body: `body ${n}\nline two`, ...over }); +// A fake API: answers the search with `items`, a comment read with the page of +// `comments` it asks for (or `comments` itself when not a list), and records every call. +const issueApi = (items = [], comments = []) => { + const calls = []; + const api = async (path, opts = {}) => { + const method = opts.method ?? "GET"; + calls.push({ path, method, body: opts.body }); + if (path.startsWith("/search/issues")) return { items }; + if (method === "GET" && path.includes("/comments?")) { + if (!Array.isArray(comments)) return comments; + const page = Number(path.match(/page=(\d+)$/)[1]); + return comments.slice((page - 1) * 100, page * 100); + } + return {}; + }; + return { api, calls, writes: () => calls.filter((c) => c.method !== "GET") }; +}; + +test("no handoffs files nothing and makes no call", async () => { + const f = issueApi(); + await fileReleaseHandoffs({ repo: REPO, release: REL, handoffs: [], api: f.api }); + assert.equal(f.calls.length, 0); +}); + +test("one handoff files one release issue with the release marker and its keyed checklist item", async () => { + const f = issueApi(); + await fileReleaseHandoffs({ repo: REPO, release: REL, handoffs: [handoff(1)], api: f.api }); + const [w] = f.writes(); + assert.equal(f.writes().length, 1); + assert.equal(w.path, `/repos/${REPO}/issues`); + assert.equal(w.method, "POST"); + assert.equal(w.body.title, `Claude Code v${REL} changelog: 1 handoff`); + assert.ok(w.body.body.startsWith(`${releaseMarker(REL)}\n`)); + assert.ok(w.body.body.includes("- [ ] adopt: adopt 1\n\n body 1\n line two\n\n ")); + assert.ok(w.body.body.includes(handoffMarker(handoff(1).key))); +}); + +test("several handoffs go into one issue, one checklist item each, in plan order", async () => { + const f = issueApi(); + const hs = [handoff(1), handoff(2, { kind: "security" }), handoff(3)]; + await fileReleaseHandoffs({ repo: REPO, release: REL, handoffs: hs, api: f.api }); + assert.equal(f.writes().length, 1); + const { title, body } = f.writes()[0].body; + assert.equal(title, `Claude Code v${REL} changelog: 3 handoffs`); + assert.equal(body.match(/^- \[ \] /gm).length, 3); + assert.ok(body.includes("- [ ] security: adopt 2")); + const at = hs.map((h) => body.indexOf(handoffMarker(h.key))); + assert.ok(at.every((i) => i > 0) && at[0] < at[1] && at[1] < at[2]); +}); + +test("bodies over the issue limit are cut in proportion, every key kept, and a line says so", async () => { + const f = issueApi(); + const hs = [handoff(1, { body: "a".repeat(50000) }), handoff(2, { body: "b".repeat(25000) })]; + await fileReleaseHandoffs({ repo: REPO, release: REL, handoffs: hs, api: f.api }); + const { body } = f.writes()[0].body; + assert.ok(body.length <= 65536); + assert.match(body, /truncated proportionally/); + for (const h of hs) assert.ok(body.includes(handoffMarker(h.key))); + const a = body.match(/a+/g).reduce((m, s) => Math.max(m, s.length), 0); + const b = body.match(/b{100,}/g)[0].length; + assert.ok(Math.abs(a / b - 2) < 0.01); + assert.equal(renderItems(hs.slice(0, 1).map((h) => ({ ...h, body: "x" })), 1000).includes("truncated"), false); +}); + +test("a release issue holding every key gets no write", async () => { + const hs = [handoff(1), handoff(2)]; + const body = `${releaseMarker(REL)}\n\n${renderItems(hs, 65536)}`; + const f = issueApi([{ user: APP, number: 7, state:"open", body }]); + await fileReleaseHandoffs({ repo: REPO, release: REL, handoffs: hs, api: f.api }); + assert.equal(f.writes().length, 0); +}); + +test("an open release issue missing a key gets the missing item appended in one update", async () => { + const body = `${releaseMarker(REL)}\n\n${renderItems([handoff(1)], 65536)}`; + const f = issueApi([{ user: APP, number: 7, state:"open", body }]); + await fileReleaseHandoffs({ repo: REPO, release: REL, handoffs: [handoff(1), handoff(2)], api: f.api }); + const writes = f.writes(); + assert.equal(writes.length, 1); + assert.equal(writes[0].path, `/repos/${REPO}/issues/7`); + assert.equal(writes[0].method, "PATCH"); + assert.equal(writes[0].body.title, `Claude Code v${REL} changelog: 2 handoffs`); + assert.ok(writes[0].body.body.startsWith(body.trimEnd())); + assert.equal(writes[0].body.body.split(handoffMarker(handoff(1).key)).length, 2); + assert.ok(writes[0].body.body.includes(handoffMarker(handoff(2).key))); +}); + +test("a closed release issue gets no write, even when it lacks a key", async () => { + const body = `${releaseMarker(REL)}\n\n${renderItems([handoff(1)], 65536)}`; + const f = issueApi([{ user: APP, number: 7, state:"closed", body }]); + await fileReleaseHandoffs({ repo: REPO, release: REL, handoffs: [handoff(1), handoff(2)], api: f.api }); + assert.equal(f.writes().length, 0); +}); + +test("an issue for another release does not count as this release's issue", async () => { + const f = issueApi([{ user: APP, number: 7, state:"open", body: `${releaseMarker("2.1.30")}\n` }]); + await fileReleaseHandoffs({ repo: REPO, release: REL, handoffs: [handoff(1)], api: f.api }); + assert.equal(f.writes()[0].method, "POST"); +}); + +test("missing items that would push the body past the limit go in one comment instead", async () => { + const body = `${releaseMarker(REL)}\n\n${renderItems([handoff(1, { body: "a".repeat(60000) })], 65536)}`; + const f = issueApi([{ user: APP, number: 7, state:"open", body }]); + await fileReleaseHandoffs({ repo: REPO, release: REL, handoffs: [handoff(1), handoff(2, { body: "b".repeat(10000) })], api: f.api }); + const writes = f.writes(); + assert.equal(writes.length, 1); + assert.equal(writes[0].path, `/repos/${REPO}/issues/7/comments`); + assert.equal(writes[0].method, "POST"); + assert.ok(writes[0].body.body.includes(handoffMarker(handoff(2).key))); + assert.ok(!writes[0].body.body.includes(handoffMarker(handoff(1).key))); + assert.ok(writes[0].body.body.length <= 65536); +}); + +test("a key already in a comment, past the first page, is not added again", async () => { + const body = `${releaseMarker(REL)}\n\n${renderItems([handoff(1)], 65536)}`; + const comments = [...Array.from({ length: 100 }, () => ({ body: "noise" })), { user: APP, body: renderItems([handoff(2)], 65536) }]; + const f = issueApi([{ user: APP, number: 7, state:"open", body }], comments); + await fileReleaseHandoffs({ repo: REPO, release: REL, handoffs: [handoff(1), handoff(2), handoff(3)], api: f.api }); + const writes = f.writes(); + assert.equal(writes.length, 1); + assert.equal(writes[0].method, "PATCH"); + assert.ok(writes[0].body.body.includes(handoffMarker(handoff(3).key))); + assert.ok(!writes[0].body.body.includes(handoffMarker(handoff(2).key))); + assert.equal(f.calls.filter((c) => c.path.includes("/comments?")).length, 2); +}); + +test("a comment read that fails, or runs past the page cap, files nothing", async () => { + const body = `${releaseMarker(REL)}\n\n${renderItems([handoff(1)], 65536)}`; + const hs = [handoff(1), handoff(2)]; + const bad = issueApi([{ user: APP, number: 7, state:"open", body }], { message: "Server Error" }); + await assert.rejects(fileReleaseHandoffs({ repo: REPO, release: REL, handoffs: hs, api: bad.api }), /not come back as a list/); + assert.equal(bad.writes().length, 0); + const many = issueApi([{ user: APP, number: 7, state:"open", body }], Array.from({ length: 1000 }, () => ({ body: "" }))); + await assert.rejects(fileReleaseHandoffs({ repo: REPO, release: REL, handoffs: hs, api: many.api }), /more comments/); + assert.equal(many.writes().length, 0); +}); + +test("the search asks for issues the lanes App opened", async () => { + const f = issueApi(); + await fileReleaseHandoffs({ repo: REPO, release: REL, handoffs: [handoff(1)], api: f.api }); + const q = decodeURIComponent(f.calls[0].path.match(/q=([^&]+)/)[1]); + assert.match(q, /(^| )author:app\/melodic-automation-lanes( |$)/); +}); + +test("an outsider's issue carrying the release marker is ignored and a new issue is filed", async () => { + const body = `${releaseMarker(REL)}\n\n${renderItems([handoff(1)], 65536)}`; + const f = issueApi([{ user: OUTSIDER, number: 7, state: "open", body }]); + await fileReleaseHandoffs({ repo: REPO, release: REL, handoffs: [handoff(1)], api: f.api }); + const writes = f.writes(); + assert.equal(writes.length, 1); + assert.equal(writes[0].path, `/repos/${REPO}/issues`); + assert.equal(writes[0].method, "POST"); +}); + +test("a stopped plan with no handoffs still files one issue that names its stop_reason", async () => { + const f = issueApi(); + await fileReleaseHandoffs({ repo: REPO, release: REL, handoffs: [], stop: "cap-exceeded", api: f.api }); + const writes = f.writes(); + assert.equal(writes.length, 1); + assert.equal(writes[0].path, `/repos/${REPO}/issues`); + assert.equal(writes[0].body.title, `Claude Code v${REL} changelog: 0 handoffs`); + assert.ok(writes[0].body.body.startsWith(`${releaseMarker(REL)}\n\n`)); + assert.ok(writes[0].body.body.includes(`Claude Code v${REL} stopped (stop_reason \`cap-exceeded\`)`)); +}); + +test("a stopped plan's handoff issue opens with the stop line above its checklist", async () => { + const f = issueApi(); + await fileReleaseHandoffs({ repo: REPO, release: REL, handoffs: [handoff(1)], stop: "cap-exceeded", api: f.api }); + const { body } = f.writes()[0].body; + const stopAt = body.indexOf("stop_reason `cap-exceeded`"); + assert.ok(stopAt > 0 && stopAt < body.indexOf("- [ ] adopt: adopt 1")); + assert.ok(body.includes(handoffMarker(handoff(1).key))); +}); + +test("a stopped plan whose closed release issue lacks a key fails with no write", async () => { + const body = `${releaseMarker(REL)}\n\n${renderItems([handoff(1)], 65536)}`; + const f = issueApi([{ user: APP, number: 7, state: "closed", body }]); + await assert.rejects( + fileReleaseHandoffs({ repo: REPO, release: REL, handoffs: [handoff(1), handoff(2)], stop: "cap-exceeded", api: f.api }), + /#7 is closed and lacks 1 handoffs/, + ); + assert.equal(f.writes().length, 0); +}); + +test("a stopped plan whose release issue already holds every key succeeds with no write, open or closed", async () => { + const hs = [handoff(1), handoff(2)]; + const body = `${releaseMarker(REL)}\n\n${renderItems(hs, 65536)}`; + for (const state of ["open", "closed"]) { + const f = issueApi([{ user: APP, number: 7, state, body }]); + await fileReleaseHandoffs({ repo: REPO, release: REL, handoffs: hs, stop: "cap-exceeded", api: f.api }); + assert.equal(f.writes().length, 0); + } +}); + +test("a stopped plan whose issue filing fails rejects, so publish fails and the release is retried", async () => { + const f = issueApi(); + const failing = async (path, opts = {}) => { + if (opts.method === "POST") throw new Error("POST /repos/o/r/issues answered 502"); + return f.api(path, opts); + }; + await assert.rejects(fileReleaseHandoffs({ repo: REPO, release: REL, handoffs: [handoff(1)], stop: "cap-exceeded", api: failing }), /answered 502/); +}); + +test("a key marker in an outsider's comment does not count as filed", async () => { + const body = `${releaseMarker(REL)}\n\n${renderItems([handoff(1)], 65536)}`; + const f = issueApi([{ user: APP, number: 7, state: "open", body }], [{ user: OUTSIDER, body: renderItems([handoff(2)], 65536) }]); + await fileReleaseHandoffs({ repo: REPO, release: REL, handoffs: [handoff(1), handoff(2)], api: f.api }); + const writes = f.writes(); + assert.equal(writes.length, 1); + assert.equal(writes[0].method, "PATCH"); + assert.ok(writes[0].body.body.includes(handoffMarker(handoff(2).key))); +}); diff --git a/.github/workflows/pr-require-checks.yml b/.github/workflows/pr-require-checks.yml index 186bc611b7c..95a1990b69c 100644 --- a/.github/workflows/pr-require-checks.yml +++ b/.github/workflows/pr-require-checks.yml @@ -6830,7 +6830,7 @@ jobs: held_paths() { # The trust-gate paths, as bash patterns (`*` crosses `/`). # BEGIN hold-floor hold_patterns: generated by scripts/sync-hold-floor.py - hold_patterns=('.github/standards/trusted-actors/*' 'CODEOWNERS' '*/CODEOWNERS' 'docs/conventions/pr-pipeline.yaml' '.gitmodules' '*/.gitmodules' 'plugins/source-control/skills/babysit-prs/scripts/babysit_risk.py' 'plugins/source-control/skills/babysit-prs/scripts/babysit_check_inputs.py' 'plugins/source-control/skills/babysit-prs/scripts/babysit_checks.py' 'plugins/source-control/skills/babysit-prs/scripts/babysit_classify.py' 'plugins/source-control/skills/babysit-prs/scripts/babysit_dependabot.py' 'plugins/source-control/skills/babysit-prs/scripts/babysit_feedback.py' 'plugins/source-control/skills/babysit-prs/scripts/babysit_gh.py' 'plugins/source-control/skills/babysit-prs/scripts/babysit_merge.py' 'plugins/source-control/skills/babysit-prs/scripts/babysit_merge_policy.py' 'plugins/source-control/skills/babysit-prs/scripts/babysit_postmerge.py' 'plugins/source-control/skills/babysit-prs/scripts/babysit_repo_config.py' 'plugins/source-control/skills/babysit-prs/scripts/babysit_review_evidence.py' 'plugins/source-control/skills/babysit-prs/scripts/babysit_review_trigger.py' 'plugins/source-control/skills/babysit-prs/scripts/babysit_run_cache.py' 'plugins/source-control/skills/babysit-prs/scripts/babysit_state.py' 'plugins/source-control/skills/babysit-prs/scripts/babysit_tracker_binding.py' 'plugins/source-control/skills/babysit-prs/scripts/babysit_util.py' 'plugins/source-control/skills/babysit-prs/scripts/babysit_verify_evidence.py' 'plugins/source-control/skills/babysit-prs/scripts/babysit_whole_review.py' '.claude/*' '*/.claude/*' 'docs/conventions/source-control.yaml' '.claude-lane/*' '*/.claude-lane/*' '.mcp.json' '*/.mcp.json' 'CLAUDE.md' '*/CLAUDE.md' 'CLAUDE.local.md' '*/CLAUDE.local.md' 'AGENTS.md' '*/AGENTS.md' 'REVIEW.md' '.github/actions/check-trusted-trigger/*' '.github/actions/resolve-config/*' '.github/actions/check-kill-switch/*' '.github/actions/request-lane-token/*' '.github/actions/log-api-budget/rate-guard.mjs' '.github/actions/log-api-budget/rate-limits.json' '.github/scripts/pr-merge/merge-holds.mjs' '.github/scripts/pr-merge/enable-auto-merge.mjs' '.github/scripts/pr-merge/superseded.mjs' '.github/scripts/pr-merge/clear-escalations.mjs' '.github/scripts/pr-merge/carry-run-cache.mjs' '.github/scripts/pr-merge/gate-check.mjs' '.github/scripts/pr-merge/head-ref-map.mjs' '.github/scripts/pr-merge/lane-records.mjs' '.github/scripts/pr-merge/last-report.mjs' '.github/scripts/pr-merge/rate-stop.bash' '.github/scripts/pr-merge/retry-call.bash' '.github/scripts/pr-merge/refire-refused.mjs' '.github/scripts/pr-merge/rotation.mjs' '.github/scripts/pr-merge/sweep-summary.mjs' '.github/scripts/pr-merge/who.mjs' '.github/scripts/pr-merge/post-unarmed-status.mjs' '.github/scripts/pr-merge/verify-answers.mjs' '.github/scripts/pr-merge/listing-carry.mjs' '.github/actions/report-check-run/posting-run.mjs' 'scripts/sync-hold-floor.py' '.github/scripts/pr-unblock/address-feedback.mjs' '.github/scripts/pr-unblock/escalation-routes.mjs' '.github/scripts/pr-unblock/escalation-routes.json' '.github/scripts/pr-unblock/gate-source.mjs' '.github/scripts/pr-unblock/thread-scope.mjs' '.github/scripts/pr-unblock/retry-fetch.mjs' '.github/scripts/work-resolve-issue/report-outcome.mjs' '.github/scripts/work-resolve-issue/limits.mjs' '.github/scripts/work-resolve-issue/retry-fetch.mjs' '.github/scripts/pr-unblock/report-escalation.sh' '.github/scripts/pr-unblock/adjudicate-feedback.mjs' '.github/scripts/pr-unblock/publish-feedback.mjs' '.github/scripts/pr-unblock/request-budget.mjs' '.github/scripts/pr-verify/resolve-pr.mjs' '.github/scripts/pr-verify/write-verdict.mjs' '.github/scripts/intake-triage/admit.mjs' '.github/actions/select-trusted-text/select-trusted-text.mjs' 'lib/publish-gate.mjs' '.github/recurring-schedule.json' 'plugins/work-items/scripts/charge-words.json' '.github/scripts/pr-unblock/dependency-classify.mjs' '.github/scripts/apply-patch/apply-patch.mjs' '.github/scripts/apply-patch/predicates.mjs' '.github/scripts/apply-patch/protected-paths.mjs' '.github/scripts/apply-patch/retry-fetch.mjs' '.github/scripts/pr-unblock/fix-ci-context.mjs' '.github/scripts/pr-unblock/feedback-dispatch.mjs' '.github/scripts/pr-unblock/verify-notes.mjs' '.github/scripts/pr-unblock/limit-stop.mjs' '.github/scripts/lib/lane-outcome.mjs' '.github/scripts/pr-unblock/dispatch-record.mjs' '.github/scripts/pr-unblock/verify-cap.mjs' '.github/scripts/maintenance-reconcile-lost-runs/infra-failure.mjs' '.github/scripts/work-resolve-issue/stop-routes.json' '.github/scripts/work-resolve-issue/stop-routes.mjs' '.github/scripts/work-resolve-issue/known-names.json' '.github/scripts/lib/stop-reason.mjs' '.github/scripts/lib/admission-comment.mjs' '.github/scripts/lib/done-evidence.mjs' '.github/scripts/pr-review-main/*' 'docs/conventions/instruction-exception-register/*' 'docs/conventions/untrusted-content/*' 'docs/standards/*' 'docs/review/*' '.github/workflows/dependabot-fragments.yml' '.github/workflows/eval-browser-dispatch.yml' '.github/workflows/intake-reenter.yml' '.github/workflows/intake-triage.yml' '.github/workflows/maintenance-audit-lane-runs.yml' '.github/workflows/maintenance-reconcile-lost-runs.yml' '.github/workflows/post-merge-demotion-alert.yml' '.github/workflows/post-merge-test-mutation.yml' '.github/workflows/pr-feedback-dispatch.yml' '.github/workflows/pr-merge.yml' '.github/workflows/pr-refine.yml' '.github/workflows/pr-require-checks.yml' '.github/workflows/pr-review-approved.yml' '.github/workflows/pr-review-comment-relay.yml' '.github/workflows/pr-review-hosted.yml' '.github/workflows/pr-review-main.yml' '.github/workflows/pr-review-security-hosted.yml' '.github/workflows/pr-run-activity-read.yml' '.github/workflows/pr-run-activity-write.yml' '.github/workflows/pr-unblock-dispatch.yml' '.github/workflows/pr-unblock.yml' '.github/workflows/pr-verify.yml' '.github/workflows/release-publish-plugins.yml' '.github/workflows/work-resolve-issue.yml' '.github/workflows/work-unblock.yml') + hold_patterns=('.github/standards/trusted-actors/*' 'CODEOWNERS' '*/CODEOWNERS' 'docs/conventions/pr-pipeline.yaml' '.gitmodules' '*/.gitmodules' 'plugins/source-control/skills/babysit-prs/scripts/babysit_risk.py' 'plugins/source-control/skills/babysit-prs/scripts/babysit_check_inputs.py' 'plugins/source-control/skills/babysit-prs/scripts/babysit_checks.py' 'plugins/source-control/skills/babysit-prs/scripts/babysit_classify.py' 'plugins/source-control/skills/babysit-prs/scripts/babysit_dependabot.py' 'plugins/source-control/skills/babysit-prs/scripts/babysit_feedback.py' 'plugins/source-control/skills/babysit-prs/scripts/babysit_gh.py' 'plugins/source-control/skills/babysit-prs/scripts/babysit_merge.py' 'plugins/source-control/skills/babysit-prs/scripts/babysit_merge_policy.py' 'plugins/source-control/skills/babysit-prs/scripts/babysit_postmerge.py' 'plugins/source-control/skills/babysit-prs/scripts/babysit_repo_config.py' 'plugins/source-control/skills/babysit-prs/scripts/babysit_review_evidence.py' 'plugins/source-control/skills/babysit-prs/scripts/babysit_review_trigger.py' 'plugins/source-control/skills/babysit-prs/scripts/babysit_run_cache.py' 'plugins/source-control/skills/babysit-prs/scripts/babysit_state.py' 'plugins/source-control/skills/babysit-prs/scripts/babysit_tracker_binding.py' 'plugins/source-control/skills/babysit-prs/scripts/babysit_util.py' 'plugins/source-control/skills/babysit-prs/scripts/babysit_verify_evidence.py' 'plugins/source-control/skills/babysit-prs/scripts/babysit_whole_review.py' '.claude/*' '*/.claude/*' 'docs/conventions/source-control.yaml' '.claude-lane/*' '*/.claude-lane/*' '.mcp.json' '*/.mcp.json' 'CLAUDE.md' '*/CLAUDE.md' 'CLAUDE.local.md' '*/CLAUDE.local.md' 'AGENTS.md' '*/AGENTS.md' 'REVIEW.md' '.github/actions/check-trusted-trigger/*' '.github/actions/resolve-config/*' '.github/actions/check-kill-switch/*' '.github/actions/request-lane-token/*' '.github/actions/log-api-budget/rate-guard.mjs' '.github/actions/log-api-budget/rate-limits.json' '.github/scripts/pr-merge/merge-holds.mjs' '.github/scripts/pr-merge/enable-auto-merge.mjs' '.github/scripts/pr-merge/superseded.mjs' '.github/scripts/pr-merge/clear-escalations.mjs' '.github/scripts/pr-merge/carry-run-cache.mjs' '.github/scripts/pr-merge/gate-check.mjs' '.github/scripts/pr-merge/head-ref-map.mjs' '.github/scripts/pr-merge/lane-records.mjs' '.github/scripts/pr-merge/last-report.mjs' '.github/scripts/pr-merge/rate-stop.bash' '.github/scripts/pr-merge/retry-call.bash' '.github/scripts/pr-merge/refire-refused.mjs' '.github/scripts/pr-merge/rotation.mjs' '.github/scripts/pr-merge/sweep-summary.mjs' '.github/scripts/pr-merge/who.mjs' '.github/scripts/pr-merge/post-unarmed-status.mjs' '.github/scripts/pr-merge/verify-answers.mjs' '.github/scripts/pr-merge/listing-carry.mjs' '.github/actions/report-check-run/posting-run.mjs' 'scripts/sync-hold-floor.py' '.github/scripts/pr-unblock/address-feedback.mjs' '.github/scripts/pr-unblock/escalation-routes.mjs' '.github/scripts/pr-unblock/escalation-routes.json' '.github/scripts/pr-unblock/gate-source.mjs' '.github/scripts/pr-unblock/thread-scope.mjs' '.github/scripts/pr-unblock/retry-fetch.mjs' '.github/scripts/work-resolve-issue/report-outcome.mjs' '.github/scripts/work-resolve-issue/limits.mjs' '.github/scripts/work-resolve-issue/retry-fetch.mjs' '.github/scripts/pr-unblock/report-escalation.sh' '.github/scripts/pr-unblock/adjudicate-feedback.mjs' '.github/scripts/pr-unblock/publish-feedback.mjs' '.github/scripts/pr-unblock/request-budget.mjs' '.github/scripts/pr-verify/resolve-pr.mjs' '.github/scripts/pr-verify/write-verdict.mjs' '.github/scripts/intake-triage/admit.mjs' '.github/actions/select-trusted-text/select-trusted-text.mjs' 'lib/publish-gate.mjs' '.github/recurring-schedule.json' 'plugins/work-items/scripts/charge-words.json' '.github/scripts/pr-unblock/dependency-classify.mjs' '.github/scripts/apply-patch/apply-patch.mjs' '.github/scripts/apply-patch/predicates.mjs' '.github/scripts/apply-patch/protected-paths.mjs' '.github/scripts/apply-patch/retry-fetch.mjs' '.github/scripts/pr-unblock/fix-ci-context.mjs' '.github/scripts/pr-unblock/feedback-dispatch.mjs' '.github/scripts/pr-unblock/verify-notes.mjs' '.github/scripts/pr-unblock/limit-stop.mjs' '.github/scripts/lib/lane-outcome.mjs' '.github/scripts/pr-unblock/dispatch-record.mjs' '.github/scripts/pr-unblock/verify-cap.mjs' '.github/scripts/maintenance-reconcile-lost-runs/infra-failure.mjs' '.github/scripts/work-resolve-issue/stop-routes.json' '.github/scripts/work-resolve-issue/stop-routes.mjs' '.github/scripts/work-resolve-issue/known-names.json' '.github/scripts/lib/stop-reason.mjs' '.github/scripts/lib/admission-comment.mjs' '.github/scripts/lib/done-evidence.mjs' '.github/scripts/pr-review-main/*' 'docs/conventions/instruction-exception-register/*' 'docs/conventions/untrusted-content/*' 'docs/standards/*' 'docs/review/*' '.github/workflows/dependabot-fragments.yml' '.github/workflows/eval-browser-dispatch.yml' '.github/workflows/intake-reenter.yml' '.github/workflows/intake-triage.yml' '.github/workflows/maintenance-audit-lane-runs.yml' '.github/workflows/maintenance-reconcile-lost-runs.yml' '.github/workflows/post-merge-demotion-alert.yml' '.github/workflows/post-merge-test-mutation.yml' '.github/workflows/pr-feedback-dispatch.yml' '.github/workflows/pr-merge.yml' '.github/workflows/pr-refine.yml' '.github/workflows/pr-require-checks.yml' '.github/workflows/pr-review-approved.yml' '.github/workflows/pr-review-comment-relay.yml' '.github/workflows/pr-review-hosted.yml' '.github/workflows/pr-review-main.yml' '.github/workflows/pr-review-security-hosted.yml' '.github/workflows/pr-run-activity-read.yml' '.github/workflows/pr-run-activity-write.yml' '.github/workflows/pr-unblock-dispatch.yml' '.github/workflows/pr-unblock.yml' '.github/workflows/pr-verify.yml' '.github/workflows/release-publish-plugins.yml' '.github/workflows/upstream-apply-changelog.yml' '.github/workflows/work-resolve-issue.yml' '.github/workflows/work-unblock.yml') # merge.diff-check.path-holds is false: only Dependabot stays held. [ "$AUTHOR_ID" = 49699333 ] || hold_patterns=() # END hold-floor hold_patterns diff --git a/.github/workflows/pr-review-hosted.yml b/.github/workflows/pr-review-hosted.yml index eb060a54c05..83acbad4e08 100644 --- a/.github/workflows/pr-review-hosted.yml +++ b/.github/workflows/pr-review-hosted.yml @@ -377,6 +377,7 @@ jobs: .github/workflows/pr-unblock.yml .github/workflows/pr-verify.yml .github/workflows/release-publish-plugins.yml + .github/workflows/upstream-apply-changelog.yml .github/workflows/work-resolve-issue.yml .github/workflows/work-unblock.yml plugins/review/skills/security-review/** diff --git a/.github/workflows/pr-review-security-hosted.yml b/.github/workflows/pr-review-security-hosted.yml index d7e3d7a07c2..5076b35ef4b 100644 --- a/.github/workflows/pr-review-security-hosted.yml +++ b/.github/workflows/pr-review-security-hosted.yml @@ -377,6 +377,7 @@ jobs: .github/workflows/pr-unblock.yml .github/workflows/pr-verify.yml .github/workflows/release-publish-plugins.yml + .github/workflows/upstream-apply-changelog.yml .github/workflows/work-resolve-issue.yml .github/workflows/work-unblock.yml plugins/review/skills/security-review/** diff --git a/.github/workflows/upstream-apply-changelog.yml b/.github/workflows/upstream-apply-changelog.yml new file mode 100644 index 00000000000..5e14cef7eb9 --- /dev/null +++ b/.github/workflows/upstream-apply-changelog.yml @@ -0,0 +1,896 @@ +name: upstream-apply-changelog + +# The release intake lane (#7407): for one Claude Code release, a model job +# runs `/harness-ops:changelog apply --ci `, which commits one local +# branch per owner plugin and writes ci-plan.json +# (plugins/harness-ops/skills/changelog/context/ci-mode.md). Scripted jobs +# then act on that plan with lanes App tokens from the token broker, which +# mints for this file on schedule and workflow_dispatch from main only, keyed +# on the `release` request field: +# push-branches (mutate-branch: contents write) recreates each branch whose +# net diff changes only plugins// and .changes// as one +# commit on the base through the Git Data API, titled with the plan's pull +# request title, so GitHub authors and signs it as the lanes App; +# publish (publish-artifact: pull-requests and issues write, contents read) +# opens one pull request per pushed branch and files the release's +# handoffs as one issue. +# The broker never grants contents and pull-requests write in one mint, nor +# merge or actions, so no job here can merge. +# +# A schedule run takes the newest of the ten newest unread releases +# changelog-status.sh lists that is not done yet, so a later tick takes each +# release of a burst in turn; a dispatch names one. +# A release is skipped only when this workflow's +# upstream-apply-changelog-done- artifact exists, since --ci never +# moves the read marker; existing branches and PRs are skipped per item. At most limits.upstream-apply-changelog-per-day +# runs start in any 24 hours (docs/conventions/pr-pipeline.yaml). +# +# How to stop it: set the organization or repository lanes kill-switch +# variable to anything but "false"; every job checks it first and the broker +# reads it live. + +on: + # Every ten minutes, at minutes no other schedule here uses. A tick makes + # at most eleven GITHUB_TOKEN REST calls (ten done-marker reads and the cap + # read), at most 66 an hour of the token's 1,000. + schedule: + - cron: "5-59/10 * * * *" + workflow_dispatch: + inputs: + release: + description: The Claude Code release to take in, MAJOR.MINOR.PATCH. + required: true + type: string + +permissions: {} + +# One run at a time; a later run waits and is never cancelled. +concurrency: + group: upstream-apply-changelog + cancel-in-progress: false + +defaults: + run: + shell: bash + +jobs: + # Picks the release, skips one already processed, and applies the per-day + # cap. No model, no secret, a read-only token. + gate: + if: github.ref == 'refs/heads/main' + runs-on: ubuntu-24.04 + timeout-minutes: 10 + permissions: + actions: read # list this repository's artifacts for the done marker and the cap + contents: read # check out main and list the changelog branches + outputs: + proceed: ${{ steps.gate.outputs.proceed }} + release: ${{ steps.gate.outputs.release }} + base-sha: ${{ github.sha }} + env: + API_BUDGET_TOKEN: github-token + steps: + - name: Check out the default branch tip + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: ${{ github.sha }} + path: .base + persist-credentials: false + + - name: Set up Node + uses: ./.base/.github/actions/setup-pinned-node # zizmor: ignore[self-repository] + with: + node-version-file: .base/.node-version + + - name: Check the kill switch + id: kill + uses: ./.base/.github/actions/check-kill-switch # zizmor: ignore[self-repository] + with: + value: ${{ vars.CLAUDE_LANES_DISABLED }} + repo-value: ${{ vars.CLAUDE_LANES_REPO_DISABLED }} + + - name: Report the kill switch + if: steps.kill.outputs.proceed != 'true' + env: + REASON: ${{ steps.kill.outputs.reason }} + run: echo "::notice::the lanes kill switch stops this run (reason ${REASON:-unreadable}); nothing read or written" + + # A dispatch, and any re-run of one, must come from a listed human; the + # broker checks the same, so this only keeps the model from running for + # nothing. Actor-only mode takes no schedule event, so a re-run of a + # scheduled run stops here: dispatch the release instead. Any re-run, + # partial or full, also skips the model job (its run_attempt check), so + # a retry of the model is a new dispatch. + - name: Check who started the run + id: who + if: steps.kill.outputs.proceed == 'true' && (github.event_name == 'workflow_dispatch' || github.run_attempt != '1') + uses: ./.base/.github/actions/check-trusted-trigger # zizmor: ignore[self-repository] + with: + denied-actor-ids: ${{ vars.AUTOMATION_LANES_APP_SENDER_ID }} + require-pr: "false" + require-kind: human + + - name: Pick the release + id: gate + if: >- + steps.kill.outputs.proceed == 'true' + && (steps.who.outcome == 'skipped' || steps.who.outputs.proceed == 'true') + working-directory: .base + env: + GH_TOKEN: ${{ github.token }} + EVENT_NAME: ${{ github.event_name }} + RELEASE_INPUT: ${{ inputs.release }} + STATUS_FILE: ${{ runner.temp }}/changelog-status.txt + run: | + # The unread releases, as maintenance-file-claude-code-release-issues.yml + # reads them. The replay cap bounds a replayed range, and a run here + # takes one release, so the caps are raised to keep the list printed. + if [ "$EVENT_NAME" = schedule ]; then + bash plugins/harness-ops/skills/changelog/scripts/changelog-status.sh --ledger docs/upstream/claude-code.md \ + --cap-releases 1000 --cap-items 1000000 >"$STATUS_FILE" + fi + DAILY_CAP=$(yq '.limits."upstream-apply-changelog-per-day" // 3' docs/conventions/pr-pipeline.yaml) + export DAILY_CAP + node .github/scripts/upstream-apply-changelog/upstream-apply-changelog.mjs gate + + # The per-day cap counts these, so it counts runs that reached the model. + - name: Record the start + if: steps.gate.outputs.proceed == 'true' + env: + DIR: ${{ runner.temp }}/started + RELEASE: ${{ steps.gate.outputs.release }} + run: | + mkdir -p "$DIR" + printf '%s\n' "$RELEASE" >"$DIR/release.txt" + + - name: Upload the start record + if: steps.gate.outputs.proceed == 'true' + uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 + with: + name: upstream-apply-changelog-started + path: ${{ runner.temp }}/started/release.txt + retention-days: 2 + if-no-files-found: error + + - name: Log the API budget at the end + if: always() + continue-on-error: true + timeout-minutes: 1 + uses: ./.base/.github/actions/log-api-budget # zizmor: ignore[self-repository] + with: + tokens: github-token + + # The model, on ci-workflows' run-claude-cli, with the API key and a + # read-only GITHUB_TOKEN and nothing else: the skill needs Bash to commit, + # so this job holds no write token, and what it leaves is sealed to the + # scan-carrier key and opened only by the scan job. run-claude-cli runs the + # CLI with --restricted (no user, project or local settings); the plugins + # load from read-only copies staged from main before the root checkout, and + # every CLAUDE.md, AGENTS.md, .mcp.json and .claude/ in the checkout is + # removed from the working tree first (R13). + model: + needs: gate + # The first attempt only: a re-run never runs the model again. + if: needs.gate.outputs.proceed == 'true' && github.run_attempt == '1' + runs-on: ubuntu-24.04 + timeout-minutes: 90 + cache-mode: read + permissions: + contents: read # check out main with history, to branch from it + # No global or system git config applies to any step: one the model + # writes during its leg would otherwise reach the checkout after it + # (which copies the global config into its temporary HOME) and every + # later git call. The leg commits with the repository's own identity. + env: + GIT_CONFIG_GLOBAL: /dev/null + GIT_CONFIG_NOSYSTEM: "1" + outputs: + handoff: ${{ steps.run.outputs.handoff }} + lane-class: ${{ steps.run.outputs.lane-class }} + sealed-length: ${{ steps.seal.outputs.sealed-length }} + sealed-1: ${{ steps.seal.outputs.sealed-1 }} + sealed-2: ${{ steps.seal.outputs.sealed-2 }} + sealed-3: ${{ steps.seal.outputs.sealed-3 }} + sealed-4: ${{ steps.seal.outputs.sealed-4 }} + sealed-5: ${{ steps.seal.outputs.sealed-5 }} + sealed-6: ${{ steps.seal.outputs.sealed-6 }} + steps: + - name: Check out the default branch tip + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: ${{ needs.gate.outputs.base-sha }} + path: .base + persist-credentials: false + set-safe-directory: false # GIT_CONFIG_GLOBAL is /dev/null, so its --global write cannot land + + - name: Set up Node + uses: ./.base/.github/actions/setup-pinned-node # zizmor: ignore[self-repository] + with: + node-version-file: .base/.node-version + + - name: Check the kill switch + id: kill + uses: ./.base/.github/actions/check-kill-switch # zizmor: ignore[self-repository] + with: + value: ${{ vars.CLAUDE_LANES_DISABLED }} + repo-value: ${{ vars.CLAUDE_LANES_REPO_DISABLED }} + + # An existing route: the broker's pinned resolver refuses a config that + # names a route outside its fixed set. + - name: Resolve the model + id: models + if: steps.kill.outputs.proceed == 'true' + env: + API_BUDGET_TOKEN: github-token + uses: ./.base/.github/actions/resolve-config # zizmor: ignore[self-repository] + with: + model-route: resolve-issue + base-path: .base + output-path: ${{ runner.temp }}/upstream-apply-changelog-models.json + + # The plugins the skill and its stage skills load, copied out of main + # before the root checkout replaces .base. + - name: Stage the base plugins + if: steps.models.outcome == 'success' + env: + STAGE: ${{ runner.temp }}/upstream-apply-changelog-stage + run: | + mkdir -p "$STAGE/plugins" + for plugin in harness-ops planning implementation verification; do + cp -R ".base/plugins/$plugin" "$STAGE/plugins/" + printf '{"mcpServers":{}}\n' >"$STAGE/plugins/$plugin/.mcp.json" + done + [ -z "$(find "$STAGE" -type l -print -quit)" ] || { echo "::error::a staged plugin holds a symbolic link"; exit 1; } + + - name: Apply the Claude managed settings + if: steps.models.outcome == 'success' + uses: melodic-software/ci-workflows/.github/actions/apply-claude-managed-settings@2476d5dde8979eab34374150b7e4b0a2b99f4e88 # 2476d5d 2026-10-29 + + - name: Isolate the runner + if: steps.models.outcome == 'success' + uses: ./.base/.github/actions/setup-runner-isolation # zizmor: ignore[self-repository] + with: + install-subprocess-isolation: "true" + + # The skill branches from here. Nothing after this step runs from .base. + - name: Check out main with history + if: steps.models.outcome == 'success' + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: ${{ needs.gate.outputs.base-sha }} + fetch-depth: 0 + persist-credentials: false + set-safe-directory: false # GIT_CONFIG_GLOBAL is /dev/null, so its --global write cannot land + + - name: Prepare the working tree + id: prepare + if: steps.models.outcome == 'success' + env: + STAGE: ${{ runner.temp }}/upstream-apply-changelog-stage + RELEASE: ${{ needs.gate.outputs.release }} + MODEL: ${{ steps.models.outputs.model }} + run: | + [ -n "$MODEL" ] || { echo "::error::the resolve-issue route's model is default, which run-claude-cli cannot pass"; exit 1; } + git config user.name "upstream-apply-changelog" + git config user.email "upstream-apply-changelog@users.noreply.github.com" + printf '.lane/\n' >>.git/info/exclude + # Head instruction, settings and server files stay out of the + # session; skip-worktree keeps their removal out of every commit. + mapfile -t held < <(git ls-files -- ':(glob)**/CLAUDE.md' ':(glob)**/AGENTS.md' ':(glob)**/.mcp.json' ':(glob)**/.claude/**' CLAUDE.md AGENTS.md .mcp.json) + if ((${#held[@]})); then + git update-index --skip-worktree -- "${held[@]}" + rm -f -- "${held[@]}" + fi + mkdir .lane + cp -R "$STAGE/plugins" .lane/plugins + chmod -R a-w .lane/plugins + cat >"$RUNNER_TEMP/upstream-apply-changelog-prompt.md" <>"$GITHUB_OUTPUT" + + - name: Open the model job's sealed payloads + id: open + uses: ./.base/.github/actions/open-lane-output # zizmor: ignore[self-repository] + with: + private-key: ${{ secrets.SCAN_CARRIER_PRIVATE_KEY }} + expected-key-ids: ${{ steps.key.outputs.id }} + job-id: model + max-bytes: | + plan.json=262144 + branches.bundle=393216 + directory: ${{ runner.temp }}/scan-in + sealed-length: ${{ needs.model.outputs.sealed-length }} + sealed-1: ${{ needs.model.outputs.sealed-1 }} + sealed-2: ${{ needs.model.outputs.sealed-2 }} + sealed-3: ${{ needs.model.outputs.sealed-3 }} + sealed-4: ${{ needs.model.outputs.sealed-4 }} + sealed-5: ${{ needs.model.outputs.sealed-5 }} + sealed-6: ${{ needs.model.outputs.sealed-6 }} + + # The plan's checks, and each branch's diff as text for the scan: the + # bundle itself is compressed, so its bytes say nothing to a scanner. + - name: Check the plan and write the diffs + id: check + env: + IN: ${{ runner.temp }}/scan-in + OUT: ${{ runner.temp }}/scan-out + RELEASE: ${{ needs.gate.outputs.release }} + BASE_SHA: ${{ needs.gate.outputs.base-sha }} + REPO_DIR: .base + run: | + [ -f "$IN/plan.json" ] || { echo "::error::the model job left no ci-plan.json"; exit 1; } + # Nothing here prints model text before the scans below: the + # plan's checks name fields, and git runs quiet. + node --input-type=module -e ' + import { readFileSync, writeFileSync } from "node:fs"; + import { publishText, readPlan } from "./.base/.github/scripts/upstream-apply-changelog/upstream-apply-changelog.mjs"; + const plan = readPlan(readFileSync(process.env.IN + "/plan.json", "utf8"), + { release: process.env.RELEASE, baseSha: process.env.BASE_SHA }); + writeFileSync(process.env.IN + "/heads.txt", plan.branches.map((b) => b.head_sha + "\n").join("")); + // What publish posts, decoded from JSON, so the scan reads it as posted. + writeFileSync(process.env.IN + "/publish.txt", publishText(plan, process.env.RELEASE)); + console.log(plan.branches.length + " branches, " + plan.handoffs.length + " handoffs"); + ' + : >"$IN/diffs.txt" + count=0 + if [ -s "$IN/heads.txt" ] && [ ! -f "$IN/branches.bundle" ]; then + echo "::error::the plan names branches and the model job left no bundle" + exit 1 + fi + if [ -s "$IN/heads.txt" ]; then + git -C .base bundle verify --quiet "$IN/branches.bundle" >/dev/null 2>&1 || { echo "::error::the branches bundle does not verify against the base"; exit 1; } + # Every planned head must be a head of the bundle; its ref names + # are model text and are never printed. + mapfile -t have < <(git -C .base bundle list-heads "$IN/branches.bundle" 2>/dev/null | cut -d' ' -f1 | sort -u) + mapfile -t want < <(sort -u "$IN/heads.txt") + missing=$(comm -13 <(printf '%s\n' "${have[@]}") <(printf '%s\n' "${want[@]}") | grep -c . || true) + if [ "$missing" != 0 ]; then + echo "::error::$missing of ${#want[@]} planned heads are not heads of the bundle" + exit 1 + fi + while read -r head; do + [[ $head =~ ^[0-9a-f]{40}$ ]] || exit 1 + git -C .base fetch --quiet --no-tags "$IN/branches.bundle" "$head:refs/scan/$head" >/dev/null 2>&1 || { echo "::error::a planned head did not fetch from the bundle"; exit 1; } + # The scanners read text, so a binary file is refused, unnamed. + numstat=$(git -C .base diff --numstat --no-renames "$BASE_SHA" "$head") + if grep -q $'^-\t-\t' <<<"$numstat"; then + echo "::error::a planned branch changes a binary file; only text is published" + exit 1 + fi + done <"$IN/heads.txt" + fi + # Each net diff must stay in its plugin; one that changes a hold + # path (the security review's hold-paths) stops the release: the + # plan is rewritten stopped with no branch and heads.txt emptied. + node .base/.github/scripts/upstream-apply-changelog/upstream-apply-changelog.mjs scan + # The net diff is all that leaves this job: the bundle below + # carries one synthetic commit with the head's tree, and push + # publishes one commit with that tree and the plan's title. + while read -r head; do + git -C .base diff --no-renames --binary "$BASE_SHA" "$head" >>"$IN/diffs.txt" + count=$((count + 1)) + done <"$IN/heads.txt" + # What push and publish get: the plan, and a bundle of one + # synthetic commit per planned head (its tree on the base, a fixed + # identity, date and message), never the model's commits, whose + # headers, merges and intermediate blobs the scan never read. + rm -rf "$OUT" + mkdir -p "$OUT" + cp "$IN/plan.json" "$OUT/plan.json" + synthetic='[]' + if ((count)); then + synthetic=$(node --input-type=module -e ' + import { readFileSync } from "node:fs"; + import { syntheticBundle } from "./.base/.github/scripts/upstream-apply-changelog/upstream-apply-changelog.mjs"; + const heads = readFileSync(process.env.IN + "/heads.txt", "utf8").split("\n").filter(Boolean); + console.log(JSON.stringify(syntheticBundle(".base", process.env.BASE_SHA, heads, process.env.OUT + "/branches.bundle"))); + ') || { echo "::error::the planned heads did not bundle"; exit 1; } + fi + { echo "branches=$count"; echo "synthetic=$synthetic"; } >>"$GITHUB_OUTPUT" + + - name: Scan the plan for credentials + id: scan-plan + uses: ./.base/.github/actions/scan-upload # zizmor: ignore[self-repository] + with: + path: ${{ runner.temp }}/scan-in/plan.json + + - name: Scan the branch diffs for credentials + id: scan-diffs + uses: ./.base/.github/actions/scan-upload # zizmor: ignore[self-repository] + with: + path: ${{ runner.temp }}/scan-in/diffs.txt + + - name: Scan the pull request and issue text for credentials + id: scan-publish + uses: ./.base/.github/actions/scan-upload # zizmor: ignore[self-repository] + with: + path: ${{ runner.temp }}/scan-in/publish.txt + + # The SHA-256 of each file uploaded below; no bundle, no bundle hash. + - name: Hash the plan and the branches + id: hash + if: >- + steps.scan-plan.outputs.clean == 'true' && steps.scan-diffs.outputs.clean == 'true' + && steps.scan-publish.outputs.clean == 'true' + env: + OUT: ${{ runner.temp }}/scan-out + run: | + sum=$(sha256sum <"$OUT/plan.json") + echo "plan=${sum%% *}" >>"$GITHUB_OUTPUT" + if [ -f "$OUT/branches.bundle" ]; then + sum=$(sha256sum <"$OUT/branches.bundle") + echo "bundle=${sum%% *}" >>"$GITHUB_OUTPUT" + fi + + # The plan and the rebuilt bundle, scan-out's only files. + - name: Upload the plan and the branches + id: upload + if: steps.hash.outcome == 'success' + uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 + with: + name: upstream-apply-changelog-plan-${{ github.run_attempt }} + path: ${{ runner.temp }}/scan-out/ + retention-days: 1 + if-no-files-found: error + + - name: Log the API budget at the end + if: always() + continue-on-error: true + timeout-minutes: 1 + uses: ./.base/.github/actions/log-api-budget # zizmor: ignore[self-repository] + with: + tokens: github-token + + # mutate-branch: contents write only. Recreates each planned branch that does + # not exist yet as one lanes App commit holding the scanned head tree, as + # apply-patch does for work-resolve-issue, after checking its net diff stays + # inside its plugin. + push-branches: + needs: [gate, scan] + if: ${{ !cancelled() && needs.scan.outputs.clean == 'true' && needs.scan.outputs.branches != '0' }} + runs-on: ubuntu-24.04 + timeout-minutes: 15 + cache-mode: read + permissions: + contents: read # check out main + id-token: write # prove to the token broker which workflow this is + env: + GIT_CONFIG_GLOBAL: /dev/null + GIT_CONFIG_NOSYSTEM: "1" + outputs: + skip-reason: ${{ steps.lane-token.outputs.skip-reason }} + heads: ${{ steps.push.outputs.heads }} + steps: + - name: Check out the default branch tip + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: ${{ needs.gate.outputs.base-sha }} + path: .base + fetch-depth: 0 + persist-credentials: false + set-safe-directory: false # GIT_CONFIG_GLOBAL is /dev/null, so its --global write cannot land + + - name: Set up Node + uses: ./.base/.github/actions/setup-pinned-node # zizmor: ignore[self-repository] + with: + node-version-file: .base/.node-version + + - name: Check the kill switch + id: kill + uses: ./.base/.github/actions/check-kill-switch # zizmor: ignore[self-repository] + with: + value: ${{ vars.CLAUDE_LANES_DISABLED }} + repo-value: ${{ vars.CLAUDE_LANES_REPO_DISABLED }} + + - name: Resolve the config + id: config + if: steps.kill.outputs.proceed == 'true' + env: + API_BUDGET_TOKEN: github-token + uses: ./.base/.github/actions/resolve-config # zizmor: ignore[self-repository] + with: + lane: upstream-apply-changelog + activity: apply-changelog-branches + base-path: .base + output-path: ${{ runner.temp }}/resolved-config.json + + # Before the mint: the scan job's artifact, by id, and the SHA-256 it + # recorded for each file. A missing or altered file fails the job, so + # nothing is minted. + - name: Download the plan and the branches + if: steps.config.outcome == 'success' && steps.config.outputs.applies == 'true' + uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + with: + artifact-ids: ${{ needs.scan.outputs.artifact-id }} + path: ${{ runner.temp }}/plan + + - name: Check the plan and the branches + id: verify + if: steps.config.outcome == 'success' && steps.config.outputs.applies == 'true' + env: + DIR: ${{ runner.temp }}/plan + PLAN_SHA256: ${{ needs.scan.outputs.plan-sha256 }} + BUNDLE_SHA256: ${{ needs.scan.outputs.bundle-sha256 }} + run: | + for pair in "plan.json:$PLAN_SHA256" "branches.bundle:$BUNDLE_SHA256"; do + file="$DIR/${pair%%:*}" want="${pair#*:}" + if [ -L "$file" ] || [ ! -f "$file" ] || [ "$(stat -c %s "$file")" -gt 1048576 ]; then + echo "::error::the plan artifact holds no ${pair%%:*} of at most 1 MiB" + exit 1 + fi + sum=$(sha256sum <"$file") + if ! [[ $want =~ ^[0-9a-f]{64}$ ]] || [ "${sum%% *}" != "$want" ]; then + echo "::error::${pair%%:*} does not match the SHA-256 the scan job recorded" + exit 1 + fi + done + + # A kill switch turned on since the step above is a designed stop + # (pr-run-activity.md, The lane token broker). + - name: Request the lane token + id: lane-token + if: steps.verify.outcome == 'success' + uses: ./.base/.github/actions/request-lane-token # zizmor: ignore[self-repository] + with: + broker-url: ${{ vars.LANES_BROKER_URL }} + audience: ${{ vars.LANES_BROKER_AUDIENCE }} + lane: upstream-apply-changelog + activity: apply-changelog-branches + release: ${{ needs.gate.outputs.release }} + effect: ${{ steps.config.outputs.effect }} + contents: ${{ steps.config.outputs.contents }} + pull-requests: ${{ steps.config.outputs.pull-requests }} + issues: ${{ steps.config.outputs.issues }} + kill-switch-ok: true + + - name: Push the branches + id: push + if: steps.lane-token.outputs.token != '' + env: + GH_TOKEN: ${{ steps.lane-token.outputs.token }} + API_BUDGET_TOKEN: lanes-app + RELEASE: ${{ needs.gate.outputs.release }} + BASE_SHA: ${{ needs.gate.outputs.base-sha }} + PLAN_FILE: ${{ runner.temp }}/plan/plan.json + BUNDLE_FILE: ${{ runner.temp }}/plan/branches.bundle + SYNTHETIC: ${{ needs.scan.outputs.synthetic }} + REPO_DIR: ${{ github.workspace }}/.base + run: node .base/.github/scripts/upstream-apply-changelog/upstream-apply-changelog.mjs push + + - name: Revoke the lane token + if: always() && steps.lane-token.outputs.token != '' + continue-on-error: true + env: + GH_TOKEN: ${{ steps.lane-token.outputs.token }} + run: | + status=$(printf 'Authorization: Bearer %s\n' "$GH_TOKEN" | + curl -sS -o /dev/null -w '%{http_code}' --max-time 30 -X DELETE -H @- \ + -H 'Accept: application/vnd.github+json' "$GITHUB_API_URL/installation/token" || true) + [ "$status" = 204 ] || echo "::warning::the lane token was not revoked (DELETE $status); it expires within the hour" + + - name: Log the API budget at the end + if: always() + continue-on-error: true + timeout-minutes: 1 + uses: ./.base/.github/actions/log-api-budget # zizmor: ignore[self-repository] + with: + tokens: lanes-app github-token + + # publish-artifact: pull-requests and issues write, contents read. Opens a + # pull request for each branch at the head push-branches left that has none, and files + # the release's handoffs as one issue, appending keys an open one lacks; then + # records the release done. A cap-exceeded or hold-path plan files its issue, naming its + # stop_reason, and is recorded done once that issue holds every handoff key; + # a range-not-computed plan files nothing and is not recorded done, so the + # next run takes the release again. + publish: + needs: [gate, scan, push-branches] + if: >- + ${{ !cancelled() && needs.scan.outputs.clean == 'true' + && (needs.push-branches.result == 'success' || needs.push-branches.result == 'skipped') + && needs.push-branches.outputs.skip-reason == '' }} + runs-on: ubuntu-24.04 + timeout-minutes: 15 + cache-mode: read + permissions: + contents: read # check out main + id-token: write # prove to the token broker which workflow this is + env: + GIT_CONFIG_GLOBAL: /dev/null + GIT_CONFIG_NOSYSTEM: "1" + steps: + - name: Check out the default branch tip + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: ${{ needs.gate.outputs.base-sha }} + path: .base + persist-credentials: false + set-safe-directory: false # GIT_CONFIG_GLOBAL is /dev/null, so its --global write cannot land + + - name: Set up Node + uses: ./.base/.github/actions/setup-pinned-node # zizmor: ignore[self-repository] + with: + node-version-file: .base/.node-version + + - name: Check the kill switch + id: kill + uses: ./.base/.github/actions/check-kill-switch # zizmor: ignore[self-repository] + with: + value: ${{ vars.CLAUDE_LANES_DISABLED }} + repo-value: ${{ vars.CLAUDE_LANES_REPO_DISABLED }} + + - name: Resolve the config + id: config + if: steps.kill.outputs.proceed == 'true' + env: + API_BUDGET_TOKEN: github-token + uses: ./.base/.github/actions/resolve-config # zizmor: ignore[self-repository] + with: + lane: upstream-apply-changelog + activity: publish-changelog-intake + base-path: .base + output-path: ${{ runner.temp }}/resolved-config.json + + # Before the mint: the scan job's artifact, by id, and the SHA-256 it + # recorded for each file; with no bundle hash there must be no bundle. + # A missing or altered file fails the job, so nothing is minted. + - name: Download the plan + if: steps.config.outcome == 'success' && steps.config.outputs.applies == 'true' + uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + with: + artifact-ids: ${{ needs.scan.outputs.artifact-id }} + path: ${{ runner.temp }}/plan + + - name: Check the plan and the branches + id: verify + if: steps.config.outcome == 'success' && steps.config.outputs.applies == 'true' + env: + DIR: ${{ runner.temp }}/plan + PLAN_SHA256: ${{ needs.scan.outputs.plan-sha256 }} + BUNDLE_SHA256: ${{ needs.scan.outputs.bundle-sha256 }} + run: | + pairs=("plan.json:$PLAN_SHA256") + if [ -n "$BUNDLE_SHA256" ]; then + pairs+=("branches.bundle:$BUNDLE_SHA256") + elif [ -e "$DIR/branches.bundle" ] || [ -L "$DIR/branches.bundle" ]; then + echo "::error::the plan artifact holds a bundle the scan job recorded no SHA-256 for" + exit 1 + fi + for pair in "${pairs[@]}"; do + file="$DIR/${pair%%:*}" want="${pair#*:}" + if [ -L "$file" ] || [ ! -f "$file" ] || [ "$(stat -c %s "$file")" -gt 1048576 ]; then + echo "::error::the plan artifact holds no ${pair%%:*} of at most 1 MiB" + exit 1 + fi + sum=$(sha256sum <"$file") + if ! [[ $want =~ ^[0-9a-f]{64}$ ]] || [ "${sum%% *}" != "$want" ]; then + echo "::error::${pair%%:*} does not match the SHA-256 the scan job recorded" + exit 1 + fi + done + + - name: Request the lane token + id: lane-token + if: steps.verify.outcome == 'success' + uses: ./.base/.github/actions/request-lane-token # zizmor: ignore[self-repository] + with: + broker-url: ${{ vars.LANES_BROKER_URL }} + audience: ${{ vars.LANES_BROKER_AUDIENCE }} + lane: upstream-apply-changelog + activity: publish-changelog-intake + release: ${{ needs.gate.outputs.release }} + effect: ${{ steps.config.outputs.effect }} + contents: ${{ steps.config.outputs.contents }} + pull-requests: ${{ steps.config.outputs.pull-requests }} + issues: ${{ steps.config.outputs.issues }} + kill-switch-ok: true + + - name: Open the pull requests and file the handoffs + id: publish + if: steps.lane-token.outputs.token != '' + env: + GH_TOKEN: ${{ steps.lane-token.outputs.token }} + API_BUDGET_TOKEN: lanes-app + RELEASE: ${{ needs.gate.outputs.release }} + BASE_SHA: ${{ needs.gate.outputs.base-sha }} + PLAN_FILE: ${{ runner.temp }}/plan/plan.json + HEADS: ${{ needs.push-branches.outputs.heads }} + run: node .base/.github/scripts/upstream-apply-changelog/upstream-apply-changelog.mjs publish + + - name: Revoke the lane token + if: always() && steps.lane-token.outputs.token != '' + continue-on-error: true + env: + GH_TOKEN: ${{ steps.lane-token.outputs.token }} + run: | + status=$(printf 'Authorization: Bearer %s\n' "$GH_TOKEN" | + curl -sS -o /dev/null -w '%{http_code}' --max-time 30 -X DELETE -H @- \ + -H 'Accept: application/vnd.github+json' "$GITHUB_API_URL/installation/token" || true) + [ "$status" = 204 ] || echo "::warning::the lane token was not revoked (DELETE $status); it expires within the hour" + + # The gate's done marker: a later run skips this release even when its + # plan made no branch or stopped on cap-exceeded or hold-path. + - name: Upload the done marker + if: steps.publish.outcome == 'success' && steps.publish.outputs.done == 'true' + uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 + with: + name: upstream-apply-changelog-done-${{ needs.gate.outputs.release }} + path: ${{ runner.temp }}/plan/plan.json + retention-days: 90 + if-no-files-found: error + + - name: Log the API budget at the end + if: always() + continue-on-error: true + timeout-minutes: 1 + uses: ./.base/.github/actions/log-api-budget # zizmor: ignore[self-repository] + with: + tokens: lanes-app github-token diff --git a/docs/conventions/pr-pipeline.yaml b/docs/conventions/pr-pipeline.yaml index fa0e1b75cb7..6d5803af908 100644 --- a/docs/conventions/pr-pipeline.yaml +++ b/docs/conventions/pr-pipeline.yaml @@ -650,6 +650,29 @@ activities: # sub-issue never splits again, and the issue's body is never edited. # Admission waits on an issue while a sub-issue is open. inputs: [issue] + apply-changelog-branches: + # Recreates each changelog//v branch the + # upstream-apply-changelog model job's `/harness-ops:changelog apply --ci` + # committed, as lanes App commits through the Git Data API, after + # checking every commit changes only plugins// and + # .changes// as regular files and that the result holds the + # planned tree; skips a branch that exists. A script, no model. .github/workflows/upstream-apply-changelog.yml runs it. + script: .github/scripts/upstream-apply-changelog/upstream-apply-changelog.mjs + args: push + effect: mutate-branch + gating: advisory + # Reads the plan and the bundle the model wrote. + reads-untrusted: true + publish-changelog-intake: + # Opens one pull request per pushed branch at the head push left that + # has none, and files each plan handoff whose key no issue carries. A script, + # no model. .github/workflows/upstream-apply-changelog.yml runs it. + script: .github/scripts/upstream-apply-changelog/upstream-apply-changelog.mjs + args: publish + effect: publish-artifact + gating: advisory + # Posts the plan's titles and bodies, which the model wrote. + reads-untrusted: true lanes: # Started by the agent-ready label on an issue (.github/workflows/work-resolve-issue.yml). @@ -738,6 +761,16 @@ lanes: - activity: apply-verdict - activity: apply-refinement needs: [apply-verdict] + # Started daily and by a listed human's dispatch naming one Claude Code + # release (.github/workflows/upstream-apply-changelog.yml). Push and pull request + # open are separate jobs: the broker refuses contents and pull-requests + # write in one mint. + upstream-apply-changelog: + stage: upstream + slots: + - activity: apply-changelog-branches + - activity: publish-changelog-intake + needs: [apply-changelog-branches] # The model, effort and turn budget of every model run (README.md, Models). # Aliases only, never full model IDs. A profile with no effort keeps its @@ -972,6 +1005,13 @@ limits: # stranded issues wait for the next one; far above it, queued runs go stale # on a lane workflow change. restarts-per-run: 10 + # upstream-apply-changelog runs that reach the model in any 24 hours (counted from + # its gate's start records, as red-main.daily-cap counts reverts). Each run + # takes one release, so 3 is the most anthropics/claude-code releases + # published in one UTC day or any 24 hours from 2026-05-02 to 2026-11-01 + # (158 releases; GitHub releases API published_at, read 2026-11-02). + # Dispatches and retries count too; a release over the cap waits for the window. + upstream-apply-changelog-per-day: 3 # GitHub API margins per pool (`api:` with github-token, lanes-app, # graphql, owner-token, content-creation). limit-cap sets each pool's planned # budget, the header's x-ratelimit-limit at most the cap (github-token: @@ -1515,6 +1555,8 @@ merge: class: gate - glob: ".github/workflows/release-publish-plugins.yml" class: gate + - glob: ".github/workflows/upstream-apply-changelog.yml" + class: gate - glob: ".github/workflows/work-resolve-issue.yml" class: gate - glob: ".github/workflows/work-unblock.yml" diff --git a/docs/conventions/pr-pipeline/README.md b/docs/conventions/pr-pipeline/README.md index 33f11656e32..253531b42b4 100644 --- a/docs/conventions/pr-pipeline/README.md +++ b/docs/conventions/pr-pipeline/README.md @@ -3133,7 +3133,7 @@ Which GitHub identity each workflow in `.github/workflows/` writes as. GitHub's | Identity | Workflows | Why | |---|---|---| | `GITHUB_TOKEN` (`github-actions[bot]`) | every workflow not named below, and the reads and check runs of those that are | Its writes start no workflow run, a dispatch aside, which the lane sweep's digest and the monitoring lane rely on so their own issues start no lane; it has its own per-repository API quota ([ADR 0085](../../adr/0085-run-org-lanes-from-ci-workflows-under-one-lanes-app-with-central-grants.md)) | -| Lanes App `melodic-automation-lanes`, minted per job by the broker (`.github/actions/request-lane-token`) | `pr-merge.yml`, `pr-run-activity-write.yml` (which the write jobs of `pr-unblock`, `pr-refine` and `work-resolve-issue` call), `intake-triage.yml`, `work-unblock.yml` | Its pushes, labels and auto-merge arms must start the next lane's runs, which a `GITHUB_TOKEN` write does not; one App identity also lets the person tests tell a lane from a person ([lane write identity](../loop-lane/README.md#lane-write-identity)) | +| Lanes App `melodic-automation-lanes`, minted per job by the broker (`.github/actions/request-lane-token`) | `pr-merge.yml`, `pr-run-activity-write.yml` (which the write jobs of `pr-unblock`, `pr-refine` and `work-resolve-issue` call), `intake-triage.yml`, `work-unblock.yml`, `upstream-apply-changelog.yml` | Its pushes, labels and auto-merge arms must start the next lane's runs, which a `GITHUB_TOKEN` write does not; one App identity also lets the person tests tell a lane from a person ([lane write identity](../loop-lane/README.md#lane-write-identity)) | | Release App (`PLUGIN_RELEASE_APP_PRIVATE_KEY`, `actions/create-github-app-token`) | `release-publish-plugins.yml`, `dependabot-fragments.yml` | A pull request `GITHUB_TOKEN` opens or updates gets its `pull_request` runs only after approval, and these pull requests need theirs unattended | | Owner token | none | Only the owner's own sessions use it; the babysit scripts count its calls as the `owner-token` API pool | diff --git a/docs/conventions/pr-pipeline/pr-pipeline.schema.json b/docs/conventions/pr-pipeline/pr-pipeline.schema.json index 257b890b89b..b0483450278 100644 --- a/docs/conventions/pr-pipeline/pr-pipeline.schema.json +++ b/docs/conventions/pr-pipeline/pr-pipeline.schema.json @@ -157,6 +157,13 @@ "default": 1, "description": "Further fix-ci dispatches on one PR head after its first fix-ci run ended in failure." }, + "upstream-apply-changelog-per-day": { + "type": "integer", + "minimum": 1, + "maximum": 10, + "default": 3, + "description": "upstream-apply-changelog runs that reach the model in any 24 hours." + }, "fix-docs-per-run": { "type": "integer", "minimum": 1, diff --git a/docs/conventions/pr-pipeline/pr-run-activity.md b/docs/conventions/pr-pipeline/pr-run-activity.md index 5cae06d6a82..8ed8527d7fa 100644 --- a/docs/conventions/pr-pipeline/pr-run-activity.md +++ b/docs/conventions/pr-pipeline/pr-run-activity.md @@ -1937,7 +1937,10 @@ other than `${{ github.event.repository.default_branch }}` or a `with` key other `./.base/.github/actions/open-lane-output` with `private-key: ${{ secrets.scan-carrier-private-key }}` exactly; a job that calls the read file passes it by name only, `scan-carrier-private-key: ${{ secrets.SCAN_CARRIER_PRIVATE_KEY }}` in its `secrets` map; and only -the `scan` job may name an `environment`, `pr-pipeline-scan-carrier`. GitHub gives an environment +the `scan` job may name an `environment`, `pr-pipeline-scan-carrier`. The one second holder (owner +decision, 2026-11-02) is `upstream-apply-changelog.yml`'s `scan` job: one `open` step, the same +action and exact `private-key`, after the main-only gate job, with `contents: read` alone, no +id-token and a read-only cache; the environment admits main only. GitHub gives an environment secret to a reusable workflow only when the caller passes it by name, and the value it then holds is the environment's only in the job that names the environment ([Reuse workflows](https://docs.github.com/en/actions/how-tos/reuse-automations/reuse-workflows)). @@ -1977,7 +1980,13 @@ Both are organization variables declared in github-iac The request is `POST ` with `Authorization: Bearer ` and the JSON body `{"lane": "", "activity": "", "pr_number": , "dispatcher_run_id": }`; `pr_number` is the gate's PR and is required on `workflow_dispatch` and `workflow_run`, except from -`intake-triage.yml`. An `issues` request, allowed from `work-resolve-issue.yml`, `work-unblock.yml` +`intake-triage.yml` and `upstream-apply-changelog.yml`. A request from `upstream-apply-changelog.yml`, +on its ten-minute `schedule` or a `workflow_dispatch`, sends `release` (the `release` input), the +Claude Code release it works as a `MAJOR.MINOR.PATCH` string, and none of `pr_number`, +`issue_number` or `dispatcher_run_id`; the broker refuses `release` from any other caller. Its +dispatch, and a re-run of either, must be started by a listed human. Each of its jobs mints one +effect: the broker never grants `contents` and `pull_requests` write in one mint, nor `merge` or +`actions`. An `issues` request, allowed from `work-resolve-issue.yml`, `work-unblock.yml` and `intake-triage.yml`, sends `issue_number` (the `issue-number` input) instead, and neither of the other two. From `work-resolve-issue.yml` its actor, the labeler, must be a listed human, or the lanes App; from the other two a listed human. `work-unblock.yml` (also on its hourly `schedule`) @@ -2021,17 +2030,17 @@ reason; no response at all is `broker-unreachable`. Broker reasons: | Status | Reason | Meaning | |---|---|---| -| 400 | (none) | The body is not `{lane, activity, pr_number?, issue_number?, dispatcher_run_id?}` with valid names and positive integers, or `issue_number` is missing on `issues` or present off it | +| 400 | (none) | The body is not `{lane, activity, pr_number?, issue_number?, release?, dispatcher_run_id?}` with valid names, positive integers and a `MAJOR.MINOR.PATCH` release, `issue_number` is missing on `issues` or present off it, or `release` is missing from `upstream-apply-changelog.yml` or present from any other caller | | 403 | `invalid-token`, `wrong-audience` | The OIDC token does not verify, or is for another audience | | 403 | `wrong-owner`, `repo-not-allowed`, `self-hosted-runner` | Not a melodic-software repository on the broker's allowlist, or not a GitHub-hosted runner | -| 403 | `event-not-allowed` | Not `pull_request` on `refs/pull//merge`, or `workflow_dispatch` or `workflow_run` on `refs/heads/main`, or `issues` on `refs/heads/main` from `work-resolve-issue.yml`, `work-unblock.yml` or `intake-triage.yml`, or `schedule` on `refs/heads/main` from `pr-merge.yml` or `work-unblock.yml` only | +| 403 | `event-not-allowed` | Not `pull_request` on `refs/pull//merge`, or `workflow_dispatch` or `workflow_run` on `refs/heads/main`, or `issues` on `refs/heads/main` from `work-resolve-issue.yml`, `work-unblock.yml` or `intake-triage.yml`, or `schedule` on `refs/heads/main` from `pr-merge.yml`, `work-unblock.yml` or `upstream-apply-changelog.yml` only | | 403 | `workflow-not-allowed` | The job is not this repository's `pr-run-activity-write.yml`, or the caller is not a lane under `lanes:` in the tip's `pr-pipeline.yaml` | | 403 | `workflow-modified` | The caller or `pr-run-activity-write.yml` differs from the default-branch tip (a report job, `work-resolve-issue` `report-outcome` or `pr-unblock` `report-escalation`, still mints when every main commit that changed the file is signed, GitHub-made, on a linear range and landed after run start, at most 50 minutes ago: [azure-iac#39](https://github.com/melodic-software/azure-iac/pull/39)) | | 403 | `workflow-modified` | With `dispatcher_run_id`: the dispatcher workflow at the dispatcher run's head differs from the tip (same report-job exception) | | 403 | `lane-mismatch` | The requested lane is not the caller's file stem | | 403 | `activity-not-in-lane`, `slot-disabled`, `config-invalid`, `effect-forbidden-by-lane` | The tip config does not give this activity a write grant | | 403 | `effect-read` | The activity is `read`; it belongs in the read file | -| 403 | `untrusted-actor`, `untrusted-author`, `bot-actor` | The actor, the PR author or a re-runner is not on the tip's trusted-actor list (a scheduled `pr-merge.yml` run has no actor check, and its re-runner must be a listed human; a scheduled `work-unblock.yml` run has no actor check either and skips the workflow's own closer check, so the broker alone gates its re-runs; a `github-actions[bot]` dispatch of `intake-triage.yml` has no actor check and needs its issue open, and its re-runner must be a listed human), or is the lanes bot | +| 403 | `untrusted-actor`, `untrusted-author`, `bot-actor` | The actor, the PR author or a re-runner is not on the tip's trusted-actor list (a scheduled `pr-merge.yml` run has no actor check, and its re-runner must be a listed human; a scheduled `work-unblock.yml` run has no actor check either and skips the workflow's own closer check, so the broker alone gates its re-runs; a `github-actions[bot]` dispatch of `intake-triage.yml` has no actor check and needs its issue open, and its re-runner must be a listed human; a scheduled `upstream-apply-changelog.yml` run has no actor check, and its re-runner must be a listed human), or is the lanes bot | | 403 | `untrusted-actor` | With `dispatcher_run_id` from `github-actions[bot]`: the dispatcher run or this run is missing or fails a dispatcher check above | | 503 | `github-unavailable` | With `dispatcher_run_id`: a dispatcher, run, tag or compare read failed; nothing was minted | | 403 | `pr-not-eligible` | The PR is not open, not from a branch of this repository, or does not target `main` | diff --git a/plugins/harness-ops/skills/changelog/context/ci-mode.md b/plugins/harness-ops/skills/changelog/context/ci-mode.md index f5d5a8001fd..fcafb4dd437 100644 --- a/plugins/harness-ops/skills/changelog/context/ci-mode.md +++ b/plugins/harness-ops/skills/changelog/context/ci-mode.md @@ -36,7 +36,9 @@ A row goes on a branch only when every condition holds: disagreement is a `docs-lag` handoff. 3. The local verifier re-checked it against the current tree. 4. Its edits stay inside one owner plugin: `plugins//` and that plugin's changelog entry or - fragment location (here `.changes//`). + fragment location (here `.changes//`), and touch no hold path (the `hold-paths` list in + `.github/workflows/pr-review-security-hosted.yml`). The workflow's scan job stops the release + with `stop_reason` `hold-path` when a branch changes one. 5. It is not security-relevant: the item is not a security fix (the rubric's "never skip" class), and the owner surface does not enforce permissions, guard tool calls, or handle tokens or secrets. @@ -125,7 +127,12 @@ always false: the marker moves in the ledger pull request once every row is in a ``` `status` is `stopped` (no rows were derived; `stop_reason` is `cap-exceeded` or -`range-not-computed`) or `planned` (rows were derived; `branches` and `handoffs` may each be empty). +`range-not-computed`; the scan job alone sets `hold-path`) or `planned` (rows were derived; `branches` and `handoffs` may each be empty). Handoff bodies carry item ids and the required sentence, never changelog text, matching the ledger -rule in [decisions.md](decisions.md). PR bodies follow the consuming repository's pull request -template. +rule in [decisions.md](decisions.md). The workflow groups a release's handoffs into one issue, a +checklist item per handoff. A `cap-exceeded` or `hold-path` plan is the same on every run for its release, so the +workflow files its issue even with no handoffs, opens the issue with a line naming the +`stop_reason`, and marks the release done once that issue holds every handoff key; a filing that +fails leaves the release for the next run. A `range-not-computed` plan can pass on a later run (the +changelog fetch failed, or the page lags the tag), so the workflow files nothing and leaves the +release for the next run. PR bodies follow the consuming repository's pull request template. diff --git a/plugins/source-control/skills/babysit-prs/scripts/babysit_merge_policy.py b/plugins/source-control/skills/babysit-prs/scripts/babysit_merge_policy.py index b0d636217e5..48b3909b433 100755 --- a/plugins/source-control/skills/babysit-prs/scripts/babysit_merge_policy.py +++ b/plugins/source-control/skills/babysit-prs/scripts/babysit_merge_policy.py @@ -3197,6 +3197,7 @@ def _skill_scripts(skill: str) -> str: "plugins/testing/evals/browser-probe/**", "plugins/work-items/scripts/**", "plugins/work-items/tools/work-item-tracker/**", + _skill_scripts("harness-ops:changelog"), ) diff --git a/plugins/source-control/skills/babysit-prs/scripts/tests/test_babysit_risk.py b/plugins/source-control/skills/babysit-prs/scripts/tests/test_babysit_risk.py index 0ace8170d2f..0d6204f2ed2 100644 --- a/plugins/source-control/skills/babysit-prs/scripts/tests/test_babysit_risk.py +++ b/plugins/source-control/skills/babysit-prs/scripts/tests/test_babysit_risk.py @@ -294,7 +294,7 @@ def test_floor_paths_are_high_whatever_the_switch(self): # generated copies hold; here the count of each class. self.assertEqual( Counter(c for _, c in FLOOR.classes), - {"trust": 19, "gate": 53, "enforcement": 53}, + {"trust": 19, "gate": 54, "enforcement": 53}, ) self.assertEqual( FLOOR.high, diff --git a/plugins/source-control/skills/babysit-prs/scripts/tests/test_floor_paths.py b/plugins/source-control/skills/babysit-prs/scripts/tests/test_floor_paths.py index 4aa85924abb..6fef5895b47 100644 --- a/plugins/source-control/skills/babysit-prs/scripts/tests/test_floor_paths.py +++ b/plugins/source-control/skills/babysit-prs/scripts/tests/test_floor_paths.py @@ -90,21 +90,21 @@ class TheEffectiveFloorDidNotMove(unittest.TestCase): change updates these pins in the pull request that makes it.""" def test_each_glob_keeps_its_class(self): - self.assertEqual(len(FLOOR.classes), 125) + self.assertEqual(len(FLOOR.classes), 126) self.assertEqual( digest(f"{g}\t{c}" for g, c in FLOOR.classes), - "247c5cb31f4b4b0356f6b2cd635df9be0eab6b2b5195fea91c460d21bce0e1f6", + "ec9dffd684b2aeae787d25f86efb0bbfebfc5f947e4be280a52e3fc2b3696f8a", ) def test_each_clearing_route_keeps_its_globs(self): pins = { - FLOOR.high: "a90483acbad6edb33c4579fa4f1c26350449205ba4d32430bd4ec1a12cc32a62", + FLOOR.high: "89537e9030059a12ee6d85b9527dfda457383eaeaf12576241b32ed3338d95f0", FLOOR.review: "147fbbb746111bc40c801aa7787a35efdca7992f063bc150dd6115f771a2e38b", FLOOR.author_review: "10af3daeaaeaaa01c77eafd040b831be3f147fdf3fa850b636b87a8e64094399", } self.assertEqual( (len(FLOOR.high), len(FLOOR.review), len(FLOOR.author_review)), - (72, 53, 15), + (73, 53, 15), ) for globs, pin in pins.items(): self.assertEqual(digest(globs), pin) @@ -170,7 +170,7 @@ def test_it_holds_only_what_every_consumer_shares(self): self.assertEqual(len(risk.FLOOR_CLASSES), 30) def test_every_other_row_sits_in_the_config(self): - self.assertEqual(len(ROWS), 125 - 30) + self.assertEqual(len(ROWS), 126 - 30) self.assertFalse(table(ROWS) & set(risk.FLOOR_CLASSES)) def test_the_config_every_consumer_reads_is_held_by_a_built_in_row(self): diff --git a/plugins/source-control/skills/babysit-prs/scripts/tests/test_merge_holds_sync.py b/plugins/source-control/skills/babysit-prs/scripts/tests/test_merge_holds_sync.py index 8f40b874597..79e42032af1 100644 --- a/plugins/source-control/skills/babysit-prs/scripts/tests/test_merge_holds_sync.py +++ b/plugins/source-control/skills/babysit-prs/scripts/tests/test_merge_holds_sync.py @@ -442,7 +442,7 @@ def test_the_person_floor_is_the_same_list(self) -> None: check=True, ) self.assertEqual(tuple(json.loads(proc.stdout)), FLOOR.high) - self.assertEqual(len(FLOOR.high), 72) + self.assertEqual(len(FLOOR.high), 73) def test_the_floor_holds_what_its_actions_load(self) -> None: files = import_closure(FLOOR_ENTRIES) diff --git a/scripts/outside-node-packages.txt b/scripts/outside-node-packages.txt index c61ee9fc6c4..5cc9eeeacc9 100644 --- a/scripts/outside-node-packages.txt +++ b/scripts/outside-node-packages.txt @@ -35,6 +35,7 @@ .github/scripts/pr-unblock .github/scripts/pr-verify .github/scripts/security-outside-actor +.github/scripts/upstream-apply-changelog .github/scripts/work-unblock .github/scripts/work-resolve-issue plugins/knowledge/vendor/node-launcher diff --git a/scripts/read-lane-broker-check.test.sh b/scripts/read-lane-broker-check.test.sh index a37f5a58573..ffce109c83f 100755 --- a/scripts/read-lane-broker-check.test.sh +++ b/scripts/read-lane-broker-check.test.sh @@ -66,12 +66,12 @@ gate_case "an allow reused from an earlier attempt fails" 1 true 1 2 gate_case "no allow (the broker was skipped) fails" 1 false 2 2 gate_case "no broker-check outputs at all fails" 1 "" "" 1 -# url_case