Secure conflicted promotion preflight and bound evaluation cost

.github/workflows/promotion-pr.yml

@@ -1,6 +1,7 @@
 name: Promotion request verification
 on:
-  pull_request:
+  # Runs even for source-to-distribution merge conflicts. No PR code is executed.
+  pull_request_target:
     branches: [main]
     types: [opened, synchronize, reopened, ready_for_review]
 permissions:
@@ -17,18 +18,15 @@ jobs:
     steps:
       - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
         with:
-          ref: ${{ github.event.pull_request.head.sha }}
+          ref: ${{ github.event.pull_request.base.sha }}
           persist-credentials: false
       - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020
         with:
           node-version: '22.22.2'
-      - name: Check current request and reuse canonical develop proof
+      - name: Verify request with trusted base-tree code only
         env:
           GH_TOKEN: ${{ github.token }}
           PR_NUMBER: ${{ github.event.pull_request.number }}
           EXPECTED_SOURCE: ${{ github.event.pull_request.head.sha }}
           EXPECTED_MAIN: ${{ github.event.pull_request.base.sha }}
-        run: |
-          node scripts/production-request.mjs "$PR_NUMBER" "$EXPECTED_MAIN" "$EXPECTED_SOURCE" --identity-only
-          node scripts/production-verification.mjs "$EXPECTED_SOURCE" 1500
-          node scripts/production-request.mjs "$PR_NUMBER" "$EXPECTED_MAIN" "$EXPECTED_SOURCE" --identity-only
+        run: node .github/scripts/promotion-preflight.mjs

AGENTS.md

@@ -52,6 +52,17 @@ Docs:   feature -> PR -> develop -> bilingual build + browser checks
 - Workflow guards are not branch protection; verify actual repository/environment rules before claiming enforcement.
 - This checkout: `origin` = ServierHub fork; `upstream` = personal bioinfornatics repository; inspect remotes/account before publication.
 
+## Promotion PR conflicts and CI trust boundary
+
+- `develop` and `main` deliberately have different trees. A promotion PR may be `CONFLICTING` / `DIRTY`: source-only files removed from `main` can cause modify/delete conflicts, and contributor versus installation README/AGENTS content can differ. This is not, by itself, a failed publication.
+- Never resolve those conflicts to make the promotion PR mergeable. Never merge/rebase `develop` into `main`, restore authoring files there, or bypass provenance checks. Generate the allowlisted distribution and publish it with the previous `main` as sole parent; close the PR without merging.
+- GitHub does not run `pull_request` workflows on merge-conflicted PRs. An absent PR check is NOT success. Validate trigger compatibility before rollout, especially for source-to-distribution requests.
+- A conflict-compatible `pull_request_target` preflight must run trusted base-tree code only: checkout the exact base/main SHA, explicitly grant only read-only contents/actions/pull-request permissions, and reject foreign-repository heads. Never checkout/import/build PR-head code, execute its artifacts, interpolate PR text into shell code, or use publication secrets in this job.
+- Treat the PR head SHA only as validated API data. Check current request/source/base identities before and after canonical exact-source verification. Identity/proof success is informational, not approval or permission to publish; publication rechecks authorization and source/main races independently.
+- Main must contain the workflow and its complete trusted helper closure before the new trigger can run. Local YAML/CLI tests and a successful distribution install do not prove GitHub event execution. Verify an actual conflicted same-repository promotion request after rollout; disclose any bootstrap or missing-run limitation.
+- Report commit, push, develop integration, canonical CI, generated main publication and PR preflight separately. Verify remote main SHA, distribution sourceCommit and sole parent before reporting publication success. Do not reopen/republish merely because an informational check is absent; inspect provenance first and obtain authority for additional remote actions.
+- For timing, separate feature CI, post-merge CI, overlapping promotion wait, actual publishing and agent/API overhead. Do not add concurrent waits as sequential work or equate test-step critical path with total workflow wall time.
+
 ## Authoring and safety
 - Keep root `plugin.json` valid; keep every Skill independently installable.
 - Skill metadata/instructions: English; descriptions concise, third-person, stating behavior and activation conditions.

config/reproducibility/source-manifest.json

@@ -1566,6 +1566,7 @@
     "scripts/production-verification.mjs",
     "scripts/production.mjs",
     "scripts/promote-production.mjs",
+    "scripts/promotion-preflight.mjs",
     "scripts/publish-documentation.sh",
     "scripts/rebuild-production.mjs",
     "scripts/release-all.mjs",

docs/branch-migration.md

@@ -178,4 +178,13 @@ measured production latency target; those require a separately authorized PR run
 
 Diagnostic campaign matrices are replaced by a direct public classification/projection table and one split-secret receipt journey. The real Goose missing-provider journey and process lifecycle boundaries remain. config/ci/test-risk-decisions.json records the scope and the unverified 300-second CI target.
 
-Promotion PR verification is defined in .github/workflows/promotion-pr.yml. Informational --identity-only request mode checks identity but grants no publishing authority. The job checks identity before and after canonical verification so source or main changes invalidate the request. Fork requests never run this preflight. Full Development verification only accepts PRs targeting develop, develop pushes and explicit workflow calls.
+Promotion PR verification uses the trusted base preflight in .github/workflows/promotion-pr.yml. Standalone --identity-only remains an informational local check, not the PR entrypoint. Neither grants publication authority. Full Development verification accepts develop PRs/pushes and explicit workflow calls.
+
+The promotion PR preflight uses pull_request_target because source/distribution trees
+may have merge conflicts. It has read-only contents/actions/pull-request permissions,
+rejects fork requests, checks out the exact trusted base/main SHA, and executes only
+.github/scripts/promotion-preflight.mjs with its contained built-in-only helper closure.
+The PR head SHA is API data only: no checkout, import, build or artifact execution
+from the PR head. Source/base are checked before and after canonical proof. This
+informational check never substitutes for current publication approval. The trusted
+closure must first be published to main before this trigger can run live.

scripts/production.mjs

@@ -73,6 +73,8 @@ export function projectProduction(destination) {
   }
   // GitHub discovers manually dispatched workflows from the default branch.
   for(const file of ['development-ci.yml','promotion-pr.yml','promote-production.yml','reproducibility.yml'])copy(path.join(source,'.github','workflows',file),path.join(destination,'.github','workflows',file));
+  // Minimal built-in-only trusted base preflight closure, never PR source code.
+  for(const file of ['promotion-preflight.mjs','production-request.mjs','production-verification.mjs'])copy(path.join(source,'scripts',file),path.join(destination,'.github','scripts',file));
   const files=inventory(destination);const manifest={schemaVersion:1,runtimeMode:'node-bundled',sourceCommit:commit,sourceTree:tree,inventorySha256:hash(JSON.stringify(files)),files};
   writeFileSync(path.join(destination,'distribution-manifest.json'),JSON.stringify(manifest,null,2)+'\n');
   verifyProduction(destination);return manifest;

scripts/promotion-preflight.mjs

@@ -0,0 +1,20 @@
+#!/usr/bin/env node
+/** Trusted base-tree code only. PR source is API data, never code or artifacts to execute. */
+import {execFileSync} from 'node:child_process';
+import {pathToFileURL} from 'node:url';
+import {validateRequestIdentity} from './production-request.mjs';
+import {awaitCanonicalVerification} from './production-verification.mjs';
+export async function verifyPromotionPreflight({number,repository,source,main,api,verify=awaitCanonicalVerification,waitSeconds=0}) {
+ if(!/^[1-9][0-9]*$/.test(String(number))||!/^[A-Za-z0-9_.-]+\/[A-Za-z0-9_.-]+$/.test(repository??'')||![source,main].every(s=>/^[a-f0-9]{40}$/.test(s??'')))throw Error('invalid promotion preflight inputs');
+ const check=()=>validateRequestIdentity(api('pulls/'+number),repository,main,api('branches/develop').commit.sha,api('branches/main').commit.sha,source);
+ check();
+ const proof=await verify({source,repository,api,waitSeconds});
+ check();
+ return {sourceCommit:source,baseCommit:main,verification:proof,authority:'informational only; publication still requires approval'};
+}
+if(process.argv[1]&&import.meta.url===pathToFileURL(process.argv[1]).href){
+ const {PR_NUMBER:number,GITHUB_REPOSITORY:repository,EXPECTED_SOURCE:source,EXPECTED_MAIN:main}=process.env;
+ if(execFileSync('git',['rev-parse','HEAD'],{encoding:'utf8'}).trim()!==main)throw Error('preflight must execute from exact trusted base checkout');
+ const api=(endpoint,paginate=false)=>JSON.parse(execFileSync('gh',['api',...(paginate?['--paginate','--slurp']:[]),'repos/'+repository+(endpoint?'/'+endpoint:'')],{encoding:'utf8',timeout:30000,maxBuffer:16*1024*1024}));
+ console.log(JSON.stringify(await verifyPromotionPreflight({number,repository,source,main,api,waitSeconds:1500})));
+}

skills/agent-creator/SKILL.md

@@ -5,6 +5,14 @@ description: Creates and manages Goose custom agent definitions for reusable rol
 
 # Agent Creator
 
+## Evaluation value and cost boundary
+
+- Evaluate only when requested or required by an approved acceptance protocol; authoring or packaging alone does not authorize a model campaign. Use the smallest set covering distinct plausible failures, user journeys and safety boundaries, not a scenario-count target.
+- Before expensive execution, state the decision to inform, each scenario's distinct risk, observable expected outcome and cheapest valid oracle. An assertion must reject a plausible wrong result, not merely echo a mock, accept self-reported success, check file existence or reproduce the candidate's own calculation. Review conditional assertions that accept both success and failure; define the expected outcome for the declared preconditions.
+- Declare the total candidate and grader call bound, time/cost budget and stopping rule before launch. Account for scenarios, variants, repetitions, graders, criterion-level calls and repair iterations; label unknown prices as unknown, never free. Use existing supported options or compact planning notes, not invented schema fields. User authorization must cover paid execution and any budget expansion.
+- Add a scenario only for uncovered risk or a meaningful interaction. Add repetitions or graders only for a named uncertainty that can change the decision, with marginal calls/cost and a bound. A budget ceiling is not a spending target; no Cartesian campaign or per-component multiplication by default. Required samples and safety checks in an approved protocol remain required; a small set is not reliability proof.
+- Prefer deterministic artifact checks for deterministic properties and blinded semantic grading for contextual quality. Reuse compatible frozen evidence only when source, fixtures, protocol and required bindings still match. Report corrections re-render evidence; they do not rerun candidates. Preserve failures and missingness; never retry or rewrite criteria until green. At the bound, report inconclusive or blocked and the next decision, not automatic escalation.
+
 ## Offline runtime
 
 Use `node scripts/agent-creator.mjs` for a copied source Skill; it requires Node.js 22+ and resolves only this Skill’s generated runtime. Tagged native archives use the released executable and do not require Node.js. Report a missing runtime or mapping as a packaging defect.
@@ -123,7 +131,7 @@ Prefer the bundled **agent-creator** executable for automation. It provides init
 
 Evaluate the custom agent as a reusable role, not as a skill trigger. The key question is whether delegating the same realistic task to the specialized agent produces better, more consistent results than a controlled baseline.
 
-1. Create 2–3 realistic tasks covering the role's core responsibilities, boundaries, and likely failure modes. Ask the user to review them. Make each task autonomous: name the current and baseline agent files, declare whether it must explain or execute, list immutable fixture paths and preconditions, and give it enough turns and tools to finish. Freeze assertions before runs; graders must not add hidden deliverables afterward.
+1. Select a compact set of risk-distinct realistic tasks covering the role's core responsibilities, boundaries, and likely failure modes. Ask the user to review them. Make each task autonomous: name the current and baseline agent files, declare whether it must explain or execute, list immutable fixture paths and preconditions, and give it enough turns and tools to finish. Freeze assertions before runs; graders must not add hidden deliverables afterward.
 2. Save them in an eval JSON file:
 
    ```json
@@ -173,15 +181,15 @@ Evaluate the custom agent as a reusable role, not as a skill trigger. The key qu
 
    When improving an existing agent, pass `--baseline-agent <old-agent.md>` to compare against the old instructions instead.
 
-4. Classify and version every assertion before running. Prefer objects with `id`, positive `version`, `classification` (`deterministic` or `semantic`), and `criterion`; deterministic assertions also declare a reproducible `checker` (`contains`, `not-contains`, or `regex`). Legacy prefixed strings remain supported. Semantic criteria require multiple independently identified blinded graders, contained output quotations, and a bounded call budget. Graders receive the published criterion and an opaque variant alias, never hidden criteria, true variant identity, or other grades. Disagreement or invalid/missing evidence remains inconclusive for human review; never average it into pass/fail:
+4. Classify and version every assertion before running. Prefer objects with `id`, positive `version`, `classification` (`deterministic` or `semantic`), and `criterion`; deterministic assertions also declare a reproducible `checker` (`contains`, `not-contains`, or `regex`). Legacy prefixed strings remain supported. The current Agent Creator runtime requires at least two independently identified blinded graders per semantic assertion (unlike the output-batched Skill Creator path). Budget semantic assertions × declared variants/runs × graders before execution, plus any separately authorized retries; use the minimum supported grader count unless an additional grader answers a concrete uncertainty. Contained quotations support review but do not alone prove semantic correctness. Graders receive the published criterion and an opaque variant alias, never hidden criteria, true variant identity, or other grades. Disagreement or invalid/missing evidence remains inconclusive for human review; never average it into pass/fail:
 
    ```bash
    agent-creator grade \
      <workspace>/iteration-1 \
      --llm-grader \
      --grader reviewer-a=model-a \
      --grader reviewer-b=model-b \
-     --max-grader-calls 24
+     --max-grader-calls <authorized-call-bound>
    ```
 
 5. Aggregate results:

skills/hook-creator/SKILL.md

@@ -5,6 +5,14 @@ description: Creates, audits, and tests lifecycle hooks inside Open Plugins. Use
 
 # Hook Creator
 
+## Evaluation value and cost boundary
+
+- Evaluate only when requested or required by an approved acceptance protocol; authoring or packaging alone does not authorize a model campaign. Use the smallest set covering distinct plausible failures, user journeys and safety boundaries, not a scenario-count target.
+- Before expensive execution, state the decision to inform, each scenario's distinct risk, observable expected outcome and cheapest valid oracle. An assertion must reject a plausible wrong result, not merely echo a mock, accept self-reported success, check file existence or reproduce the candidate's own calculation. Review conditional assertions that accept both success and failure; define the expected outcome for the declared preconditions.
+- Declare the total candidate and grader call bound, time/cost budget and stopping rule before launch. Account for scenarios, variants, repetitions, graders, criterion-level calls and repair iterations; label unknown prices as unknown, never free. Use existing supported options or compact planning notes, not invented schema fields. User authorization must cover paid execution and any budget expansion.
+- Add a scenario only for uncovered risk or a meaningful interaction. Add repetitions or graders only for a named uncertainty that can change the decision, with marginal calls/cost and a bound. A budget ceiling is not a spending target; no Cartesian campaign or per-component multiplication by default. Required samples and safety checks in an approved protocol remain required; a small set is not reliability proof.
+- Prefer deterministic artifact checks for deterministic properties and blinded semantic grading for contextual quality. Reuse compatible frozen evidence only when source, fixtures, protocol and required bindings still match. Report corrections re-render evidence; they do not rerun candidates. Preserve failures and missingness; never retry or rewrite criteria until green. At the bound, report inconclusive or blocked and the next decision, not automatic escalation.
+
 ## Offline runtime
 
 Use `node scripts/hook-creator.mjs` for a copied source Skill; it requires Node.js 22+ and resolves only this Skill’s generated runtime. Tagged native archives use the released executable and do not require Node.js. Report a missing runtime or mapping as a packaging defect.