feat(fullsend): switch agents to claude-opus-5-5 at explicit medium effort

.fullsend/harness/triage-security.yaml

@@ -20,7 +20,7 @@
 # The credential mount stays required so a missing credential fails up front on
 # every pinned CLI. This is the sole sandbox credential (inference only); Jira
 # credentials stay runner-only in env.runner.
-base: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/942e605ce7564b0b64f75efe055813ed6ff6ca94/harness/triage-security.yaml#sha256=c9750b8feb846242420db12d1a20f93be08dc1221aac2ba722aac495b6a05b56
+base: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/dc6c74aef84d2297f6578ef864b3aa7685b6c344/harness/triage-security.yaml#sha256=10d7b3e478cedd2e2e9f2d7b5e249d0fa4accc6323989410106fae581980f4cb
 
 host_files:
   - src: ${GOOGLE_APPLICATION_CREDENTIALS}

.fullsend/harness/verify-pr.yaml

@@ -48,7 +48,7 @@
 # (base URLs are validated against that allowlist, not inherited).
 # Re-pin after any base edit: push harness/verify-pr.yaml, set the SHA to the new
 # commit + the sha256 to `shasum -a 256 harness/verify-pr.yaml`, `fullsend lock`.
-base: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/942e605ce7564b0b64f75efe055813ed6ff6ca94/harness/verify-pr.yaml#sha256=c57aadf6b26e8840b49edd8ec5e047a60e0b1f79e7f746eed000979551cf8854
+base: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/dc6c74aef84d2297f6578ef864b3aa7685b6c344/harness/verify-pr.yaml#sha256=fb171c9b4a13df8771d19de1809c9386e4638590365d65ee95a85ce191cc2572
 
 # CEL trigger (ADR 0061) — MUST be declared here, on the composing child, not
 # only on the base. `fullsend dispatch` registers this child (config.base.yaml

docs/testing/fullsend-gate-evals.md

@@ -121,11 +121,22 @@ After host services and those inputs are ready, the operator can run:
 python3.12 evals/fullsend/run.py run \
   --cache /tmp/tc-6677-eval-deps \
   --output /tmp/tc-6677-native-evals \
-  --model claude-opus-4-8 \
+  --model claude-opus-5-5 \
   --judge-model claude-opus-4-6 \
-  --effort high
+  --effort medium
 ```
 
+The native eval agent model and effort come from the `run.py` defaults
+(`--model claude-opus-5-5`, `--effort medium`): CI
+(`.github/scripts/run-native-fullsend-evals.sh`) passes only `--judge-model`, and
+`run.py` writes both values into the resolved suite config and hands them to
+`fullsend run` as `--model`/`--effort`, which override `harness.yaml`'s
+`model:`/`effort:` and `eval.yaml`'s `models.skill`/`runner.effort` (kept equal
+for readability; `test_model_pins.py` fails on drift). Because Eval PR Run is a
+`workflow_run` that executes `run.py` from `main`, a PR changing these defaults
+is not exercised by its own CI run — prove it with the local command above. See
+`fullsend.md` "Managing the agent model".
+
 This command **does perform paid inference**: four serial native Fullsend runs
 and 24 upstream Boolean LLM judgments. Select model IDs supported by your Vertex
 project/region. Model availability is not checked by preflight. The native

evals/fullsend/run.py

@@ -423,9 +423,9 @@ def main():
     parser.add_argument("command", choices=["setup", "preflight", "run"])
     parser.add_argument("--cache", type=Path, default=Path(tempfile.gettempdir()) / "tc-6677-eval-deps")
     parser.add_argument("--output", type=Path, default=Path(tempfile.gettempdir()) / "tc-6677-native-evals")
-    parser.add_argument("--model", default="claude-opus-4-8")
+    parser.add_argument("--model", default="claude-opus-5-5")
     parser.add_argument("--judge-model", default="claude-opus-4-6")
-    parser.add_argument("--effort", choices=["low", "medium", "high", "max"], default="high")
+    parser.add_argument("--effort", choices=["low", "medium", "high", "max"], default="medium")
     parser.add_argument("--plugin-root", type=Path, help="Sandbox-tested plugin; tooling always comes from this checkout")
     parser.add_argument("--report-dir", type=Path, help="Allowlisted CI result directory; never raw credentials/transcripts")
     args = parser.parse_args()

evals/fullsend/triage-security/eval.yaml

@@ -7,7 +7,7 @@ execution:
   parallelism: 1
 runner:
   type: cli
-  effort: high
+  effort: medium
   command:
     - '{python}'
     - '{runner}'
@@ -24,7 +24,7 @@ runner:
     - --effort
     - '{effort}'
 models:
-  skill: claude-opus-4-8
+  skill: claude-opus-5-5
   judge: claude-opus-4-6
 dataset:
   path: cases

evals/fullsend/triage-security/harness.yaml

@@ -1,8 +1,8 @@
 # SYNTHETIC TEST DATA — native test agent; production policy/provider/schema unchanged
 role: triage
 agent: evals/fullsend/triage-security/agent.md
-model: claude-opus-4-8
-effort: high
+model: claude-opus-5-5
+effort: medium
 image: ghcr.io/fullsend-ai/fullsend-code@sha256:3055d4f52136a7c89fb8aa1976b468a5f9dd929c6be55fd6cd958e222c6e18ca
 readonly_repo: true
 plugins:

fullsend.md

@@ -784,6 +784,111 @@ with the `base:` move that brings it the new harness `image:`. The rule applies
 The fullsend pins are independent of the harness `base:` re-pin (which tracks
 sdlc-plugins content) and of the plugin version below.
 
+### Managing the agent model
+
+The verify-pr and triage-security agents, and the native eval agent, run on
+**`claude-opus-5-5` at effort `medium`** (TC-7274). The model is pinned content
+like everything else: changing it is a harness edit that goes through the
+re-pin + re-lock loop.
+
+#### Where model and effort are set
+
+| Location | Sets | Consumed by |
+|---|---|---|
+| `harness/verify-pr.yaml`, `harness/triage-security.yaml` — `model:` / `effort:` | model + effort | every `fullsend run` (local + CI) through the pinned `base:` |
+| `plugins/sdlc-workflow/agents/{verify-pr,triage-security}.md` — frontmatter `model:` | model only | Claude Code when nothing else names a model (e.g. interactive `claude --agent`) |
+| `evals/fullsend/run.py` — `--model` / `--effort` defaults | model + effort | the native eval agent (CI passes only `--judge-model`) |
+| `evals/fullsend/triage-security/harness.yaml`, `eval.yaml` (`models.skill`, `runner.effort`) | model + effort | overridden at runtime by `run.py`; kept equal for readability |
+
+The `.fullsend/harness/*.yaml` composing children set **no** model or effort —
+they inherit both from the pinned base.
+
+**Which wins.** fullsend resolves the run model as `--model` flag >
+`FULLSEND_MODEL` > config `agents:` entry > harness `model:`, and only when that
+chain is empty falls back to the agent frontmatter `model:`
+(`internal/runtime/model.go` `EffectiveModel` at the supported commit). The
+Claude runtime passes the resolved value to Claude Code as `--model` and the
+harness `effort:` as `--effort` (`internal/runtime/claude.go`), and the CLI flag
+takes precedence over the agent definition's frontmatter. In production the
+**harness value wins**; the frontmatter is a fallback for runs that name no
+model. Both are pinned to the same ID anyway, and
+`plugins/sdlc-workflow/scripts/test_model_pins.py` fails if any pin listed above
+disagrees with the others, uses a floating alias, or if a composing child
+overrides the base. `metrics.json` records the outcome per run:
+`requested_model` (what fullsend asked for), `override_source` (`harness`,
+`--model flag`, …) and `model` (what the provider reported).
+
+#### Why a concrete model ID, not an alias
+
+Claude Code aliases (`opus`, `sonnet`) resolve to whatever generation the CLI in
+the image maps them to. Under CLI v0.43.0 `opus` resolved to `claude-opus-4-6`,
+which the Vertex deployment had not enabled (run 34951983374: *"model
+claude-opus-4-6 is not available on your vertex deployment"*). A concrete ID
+makes the model change only when this repo changes it — the same reason the
+frontmatter carries `claude-opus-5-5` instead of `opus`.
+
+#### Why an explicit effort
+
+Effort is set explicitly instead of inheriting the model's default so that a
+change of the default effort (per model generation or per CLI release) cannot
+silently alter run behaviour, cost, or latency.
+
+#### Changing the model
+
+1. **Enable the model on Vertex first.** The fullsend Vertex project must serve
+   the new ID in the configured region; an unknown or disabled model fails
+   inference (`not available on your vertex deployment` / HTTP 400/404), not
+   preflight — `run.py preflight` does not check model availability either.
+   **Check the sandbox image too:** the harness `image:` decides the Claude Code
+   version, and a model newer than that Claude Code fails every call with
+   `Claude Code <x> does not support this model; version <y> or newer is
+   required` (TC-7278 on the v0.37.0-era image). If the pinned image is too old,
+   move the CLI and `image:` together first through
+   [Updating fullsend (pin-and-bump)](#updating-fullsend-pin-and-bump) (TC-7300).
+2. **Edit every pin** in the table above in one change, and update
+   `test_model_pins.py`'s target-model assertion.
+3. **Commit and push to an upstream `RHEcosystemAppEng` branch** — a fork PR ref
+   fails the promisor fetch.
+4. **Re-pin both** `.fullsend/harness/verify-pr.yaml` and
+   `.fullsend/harness/triage-security.yaml` (`base:` commit SHA + `#sha256=` of
+   the base file), then **re-lock**:
+   `fullsend lock --all --fullsend-dir .fullsend` (see
+   [Releasing an update](#releasing-an-update-this-repo)). Commit `.fullsend/`
+   in the same PR.
+5. **Prove before merging** (below), then **merge with a merge commit**.
+   URL-registered adopters (`sdlc-fullsend`) pick the model up by moving their
+   own `base:` pin to the merge commit.
+
+#### Pre-merge proof
+
+CI cannot self-prove a model change: verify-pr dispatches from `main`
+(`pull_request_target`) and Eval PR Run is a `workflow_run` that executes
+`run.py` from `main`. Prove each consumer explicitly:
+
+- **verify-pr** — the temporary `pull_request` workflow copy from
+  [Updating fullsend](#updating-fullsend-pin-and-bump) step 5, on a same-repo
+  branch carrying the re-pinned `.fullsend/`. The run must complete, validate
+  against the result schema, and post its report.
+- **triage-security** — a local `fullsend run triage-security` of the
+  production harness from the re-pinned branch with `--no-post-script`
+  (report-only), validated against the result schema.
+- **Native eval agent** — the local paid run in
+  `docs/testing/fullsend-gate-evals.md` with the new defaults; all cases must
+  complete.
+
+For every run, confirm in `metrics.json` (or the run log) that the provider
+reported the new model. A run that does not reach it — model not enabled, HTTP
+400, an alias resolved elsewhere, or a different `override_source` than
+expected — is a **failed** proof, not a pass.
+
+#### Forced `tool_choice` and Opus 5.5
+
+`claude-opus-5-5` rejects a forced `tool_choice` with HTTP 400 (a model
+restriction, not a Vertex policy; verified 2026-10-29). The agents are
+unaffected — Claude Code does not force tool use. LLM judges that force a tool
+call (the agent-eval-harness judges used by the native eval) therefore cannot run
+on Opus 5.5; the judge model is managed separately (`--judge-model`, TC-7279).
+
 ### Keeping the two plugin-version files in sync
 
 The plugin/skill version is stored in **two** files that must always match:

harness/triage-security.yaml

@@ -22,8 +22,8 @@
 
 role: triage
 agent: plugins/sdlc-workflow/agents/triage-security.md
-model: claude-opus-4-8
-effort: high
+model: claude-opus-5-5
+effort: medium
 image: ghcr.io/fullsend-ai/fullsend-code@sha256:3055d4f52136a7c89fb8aa1976b468a5f9dd929c6be55fd6cd958e222c6e18ca
 readonly_repo: true
 

harness/verify-pr.yaml

@@ -62,13 +62,14 @@ agent: plugins/sdlc-workflow/agents/verify-pr.md
 # Pinned to a concrete generation, not the floating `opus` alias: under fullsend
 # CLI v0.43.0 that alias resolves to claude-opus-4-6, which the Red Hat Vertex
 # deployment has not enabled (run 34951983374: "model claude-opus-4-6 is not
-# available on your vertex deployment"). opus 4.8 is the fleet's target model.
-model: claude-opus-4-8
-# Reasoning effort pinned to the fleet's implicit default: Claude Code runs at
-# high effort on effort-capable models, and the fullsend pi runtime passes
-# --thinking high when effort is unset. Revisit if `model:` moves to a
-# generation with a different default effort.
-effort: high
+# available on your vertex deployment"). opus 5.5 is the fleet's target model
+# (TC-7274). This value reaches Claude Code as --model and wins over the agent
+# frontmatter `model:` (fullsend.md "Managing the agent model").
+model: claude-opus-5-5
+# Reasoning effort set explicitly rather than inherited from the model's
+# default, so a change of Opus 5.5's default effort cannot alter runs; passed
+# to Claude Code as --effort.
+effort: medium
 image: ghcr.io/fullsend-ai/fullsend-code@sha256:3055d4f52136a7c89fb8aa1976b468a5f9dd929c6be55fd6cd958e222c6e18ca
 readonly_repo: true
 

plugins/sdlc-workflow/agents/triage-security.md

@@ -3,7 +3,7 @@ name: triage-security
 description: >-
   Run triage-security from trusted runner evidence and emit a validated
   report or Jira action plan without direct external access.
-model: opus
+model: claude-opus-5-5
 ---
 
 # Triage Security Agent

plugins/sdlc-workflow/agents/verify-pr.md

@@ -3,7 +3,7 @@ name: verify-pr
 description: >-
   Verify a PR against its Jira task acceptance criteria using the
   sdlc-workflow verify-pr skill inside an OpenShell sandbox.
-model: opus
+model: claude-opus-5-5
 ---
 
 # Verify PR Agent

plugins/sdlc-workflow/scripts/test_fullsend_gate_eval.py

@@ -21,7 +21,7 @@
 SUITE = ROOT / "evals/fullsend/triage-security"
 FIXTURES = ROOT / "evals/triage-security/files"
 # Pin of the triage-security Step 0.6/0.7 gate commands that valid_case_tools diagnostics match against.
-GATE_COMMANDS_SHA256 = "5ba54375e463039e4a969e88a5c40af48e2a607b41b6c411b68b27a01f8711b1"
+GATE_COMMANDS_SHA256 = "330702b7cd14df099e7bdc7e9da5dac5b6e182854ad3dfe55273d8ad21bf75c8"
 
 
 def load_script(path):