Skip to content

PRD: model cost/performance by job (inherit parent, escalate on demand) - #181

Merged
cahenesy merged 1 commit into
masterfrom
docs/prd/model-cost-performance
Oct 5, 2026
Merged

cahenesy merged 1 commit into
masterfrom
docs/prd/model-cost-performance

Conversation

@cahenesy

@cahenesy cahenesy commented Oct 5, 2026

Copy link
Copy Markdown
Owner

The bar for model choice becomes best cost-to-performance per job, not
the most capable model. Judgment workers inherit the parent session's
model and effort; mechanical runtime-verify runs on a light tier at low
effort, never more expensive than the parent; /build-tdds can escalate a
TDD's judgment workers to the most capable model (operator request, or
automatically on a resumed gate FAIL with the TDD unrevised), falling
back to the parent model with a warning when the escalation target is
unusable. FR-86 narrows to a light-tier/unreadable parent check with no
web fetch. Rewrites NFR-3, FR-52, FR-86, FR-87; adds FR-88; updates
FR-10/FR-15 reviewer wording, non-goals, constraints, open questions,
and the evaluation rubric.

Open assumptions & waivers:

  • newer is not more capable — resolved: escalation targets the most capable model available, not the newest.
  • FR-86 conflicts with cost/perf — resolved: narrowed to light-tier or unreadable parent only.
  • auto-escalation pays for spec defects — resolved: escalate only when the TDD is unrevised since the halt.
  • no machine-readable capability ranking — resolved: maintained binding + env override, lag accepted.
  • expensive parent silently inherited — resolved: accepted as operator choice; queue confirmation names each worker's model before dispatch.
  • harness may not support per-worker model — resolved: treated as unavailable: warn, record fell-back, run on parent.
  • escalation no-op when parent is top — resolved: run record states escalation=already-top.

Cascade:

  • TDD 0064 (draft) — STALE, needs revision: binds unset build=review=fable
    (most capable by default) and a cheaper-pin warning against that default;
    NFR-3/FR-87 now require inheriting the parent and a light-pin warning.
    Its halted build branch (build/20260816-174909/0064-judgment-worker-defaults)
    was built to the old requirement and should be abandoned, not resumed.
  • TDD 0065 (draft) — STALE, needs revision: FR-86 no longer compares to the
    most capable model or fetches the models page; it is a light-tier /
    unreadable-parent check.
  • TDD 0062 (implemented) — pressured: introduced scripts/lib/models.sh with
    pinned judgment defaults; the new design changes that binding site's
    contract (supersede the model-binding part).
  • New design needed for FR-88 (escalation trigger, outcomes, fallback
    detection) and the effort inheritance/pins in FR-87/FR-52.
  • ADR 0014 (accepted) — needs supersession: its decision binds unset
    build/review to the latest top-tier id and FR-86 to a live models-page
    fetch. Its fresh-worker independence decision carries forward.
  • ADR 0008 / 0010 — unaffected.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Ki3TDnP2bNqqpEkFoZaEXq

The bar for model choice becomes best cost-to-performance per job, not
the most capable model. Judgment workers inherit the parent session's
model and effort; mechanical runtime-verify runs on a light tier at low
effort, never more expensive than the parent; /build-tdds can escalate a
TDD's judgment workers to the most capable model (operator request, or
automatically on a resumed gate FAIL with the TDD unrevised), falling
back to the parent model with a warning when the escalation target is
unusable. FR-86 narrows to a light-tier/unreadable parent check with no
web fetch. Rewrites NFR-3, FR-52, FR-86, FR-87; adds FR-88; updates
FR-10/FR-15 reviewer wording, non-goals, constraints, open questions,
and the evaluation rubric.

Open assumptions & waivers:
- newer is not more capable — resolved: escalation targets the most capable model available, not the newest.
- FR-86 conflicts with cost/perf — resolved: narrowed to light-tier or unreadable parent only.
- auto-escalation pays for spec defects — resolved: escalate only when the TDD is unrevised since the halt.
- no machine-readable capability ranking — resolved: maintained binding + env override, lag accepted.
- expensive parent silently inherited — resolved: accepted as operator choice; queue confirmation names each worker's model before dispatch.
- harness may not support per-worker model — resolved: treated as unavailable: warn, record fell-back, run on parent.
- escalation no-op when parent is top — resolved: run record states escalation=already-top.

Cascade:
- TDD 0064 (draft) — STALE, needs revision: binds unset build=review=fable
  (most capable by default) and a cheaper-pin warning against that default;
  NFR-3/FR-87 now require inheriting the parent and a light-pin warning.
  Its halted build branch (build/20260816-174909/0064-judgment-worker-defaults)
  was built to the old requirement and should be abandoned, not resumed.
- TDD 0065 (draft) — STALE, needs revision: FR-86 no longer compares to the
  most capable model or fetches the models page; it is a light-tier /
  unreadable-parent check.
- TDD 0062 (implemented) — pressured: introduced scripts/lib/models.sh with
  pinned judgment defaults; the new design changes that binding site's
  contract (supersede the model-binding part).
- New design needed for FR-88 (escalation trigger, outcomes, fallback
  detection) and the effort inheritance/pins in FR-87/FR-52.
- ADR 0014 (accepted) — needs supersession: its decision binds unset
  build/review to the latest top-tier id and FR-86 to a live models-page
  fetch. Its fresh-worker independence decision carries forward.
- ADR 0008 / 0010 — unaffected.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Ki3TDnP2bNqqpEkFoZaEXq
@cahenesy
cahenesy merged commit 6bb3c8e into master Oct 5, 2026
1 check passed
@cahenesy
cahenesy deleted the docs/prd/model-cost-performance branch October 5, 2026 16:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant