Repository navigation
PRD: model cost/performance by job (inherit parent, escalate on demand) - #181
Merged
Merged
Conversation
The bar for model choice becomes best cost-to-performance per job, not the most capable model. Judgment workers inherit the parent session's model and effort; mechanical runtime-verify runs on a light tier at low effort, never more expensive than the parent; /build-tdds can escalate a TDD's judgment workers to the most capable model (operator request, or automatically on a resumed gate FAIL with the TDD unrevised), falling back to the parent model with a warning when the escalation target is unusable. FR-86 narrows to a light-tier/unreadable parent check with no web fetch. Rewrites NFR-3, FR-52, FR-86, FR-87; adds FR-88; updates FR-10/FR-15 reviewer wording, non-goals, constraints, open questions, and the evaluation rubric. Open assumptions & waivers: - newer is not more capable — resolved: escalation targets the most capable model available, not the newest. - FR-86 conflicts with cost/perf — resolved: narrowed to light-tier or unreadable parent only. - auto-escalation pays for spec defects — resolved: escalate only when the TDD is unrevised since the halt. - no machine-readable capability ranking — resolved: maintained binding + env override, lag accepted. - expensive parent silently inherited — resolved: accepted as operator choice; queue confirmation names each worker's model before dispatch. - harness may not support per-worker model — resolved: treated as unavailable: warn, record fell-back, run on parent. - escalation no-op when parent is top — resolved: run record states escalation=already-top. Cascade: - TDD 0064 (draft) — STALE, needs revision: binds unset build=review=fable (most capable by default) and a cheaper-pin warning against that default; NFR-3/FR-87 now require inheriting the parent and a light-pin warning. Its halted build branch (build/20260816-174909/0064-judgment-worker-defaults) was built to the old requirement and should be abandoned, not resumed. - TDD 0065 (draft) — STALE, needs revision: FR-86 no longer compares to the most capable model or fetches the models page; it is a light-tier / unreadable-parent check. - TDD 0062 (implemented) — pressured: introduced scripts/lib/models.sh with pinned judgment defaults; the new design changes that binding site's contract (supersede the model-binding part). - New design needed for FR-88 (escalation trigger, outcomes, fallback detection) and the effort inheritance/pins in FR-87/FR-52. - ADR 0014 (accepted) — needs supersession: its decision binds unset build/review to the latest top-tier id and FR-86 to a live models-page fetch. Its fresh-worker independence decision carries forward. - ADR 0008 / 0010 — unaffected. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Ki3TDnP2bNqqpEkFoZaEXq
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bar for model choice becomes best cost-to-performance per job, not
the most capable model. Judgment workers inherit the parent session's
model and effort; mechanical runtime-verify runs on a light tier at low
effort, never more expensive than the parent; /build-tdds can escalate a
TDD's judgment workers to the most capable model (operator request, or
automatically on a resumed gate FAIL with the TDD unrevised), falling
back to the parent model with a warning when the escalation target is
unusable. FR-86 narrows to a light-tier/unreadable parent check with no
web fetch. Rewrites NFR-3, FR-52, FR-86, FR-87; adds FR-88; updates
FR-10/FR-15 reviewer wording, non-goals, constraints, open questions,
and the evaluation rubric.
Open assumptions & waivers:
Cascade:
(most capable by default) and a cheaper-pin warning against that default;
NFR-3/FR-87 now require inheriting the parent and a light-pin warning.
Its halted build branch (build/20260816-174909/0064-judgment-worker-defaults)
was built to the old requirement and should be abandoned, not resumed.
most capable model or fetches the models page; it is a light-tier /
unreadable-parent check.
pinned judgment defaults; the new design changes that binding site's
contract (supersede the model-binding part).
detection) and the effort inheritance/pins in FR-87/FR-52.
build/review to the latest top-tier id and FR-86 to a live models-page
fetch. Its fresh-worker independence decision carries forward.
🤖 Generated with Claude Code
https://claude.ai/code/session_01Ki3TDnP2bNqqpEkFoZaEXq