Skip to content

design: TDDs 0064–0067 + ADR 0015 (model cost/performance by job) - #182

Merged
cahenesy merged 1 commit into
masterfrom
docs/design/model-cost-performance
Oct 5, 2026
Merged

cahenesy merged 1 commit into
masterfrom
docs/design/model-cost-performance

Conversation

@cahenesy

@cahenesy cahenesy commented Oct 5, 2026

Copy link
Copy Markdown
Owner

Designs PRD f6ef178 (PR #181). Judgment workers inherit the parent
session's model; mechanical verify on a light tier capped at the parent;
FR-86 narrowed to a light-tier/unreadable parent check; /build-tdds
escalation (FR-88) with a new step-3 Retry path.

  • 0064 (revised in place): models.sh role resolution (inherit / pins /
    light / parent-cap), parent-model observation from the session
    transcript, tier placement, design-reviewer model: inherit.
    Supersedes 0057 and 0062's models.sh contract only (0062 stays
    implemented).
  • 0065 (revised in place): FR-86 light-tier/unreadable warn+ask via an
    executable, marker-tagged block in /prd-author, /tdd-author,
    /build-tdds; the vendor-page fetch is removed.
  • 0066 (new): /build-tdds dispatch (no model parameter = inherit),
    queue confirmation with model + effort per worker, light-pin and
    effort-pin warnings, per-TDD run-record sidecar .models.json.
  • 0067 (new): Retry for failed/gate-fail TDDs (implementer fix on the
    existing branch, then all gates), escalation requested/auto, outcomes
    escalated/already-top/fell-back, fall-back at dispatch, live probe.
  • ADR 0015 supersedes ADR 0014 (fresh-worker independence carried forward).

Every TDD carries the same co-created evaluation rubric; it adds an
"executable verification" criterion, because 0064-rev1 halted on a skill
snippet that its eval only grepped.

Design critique (design-reviewer, three fresh passes):

  • Pass 1: BLOCK. M1 auto-escalation was unreachable (step 3 resumed only
    non-terminal TDDs); M2 the confirmation could not show the escalated
    model; M3 skill blocks relied on cross-call shell state and "$@";
    M4 two 0064 observations contradicted 0064's own rules. All fixed. M1
    was resolved by a user-chosen step-3 Retry path.
  • Pass 2: BLOCK. The step-3 entry condition was not widened, and Retry
    left stale PASS verdicts. Fixed with tl_run_latest_run, a widened
    trigger, and tl_run_retry_begin (archives verdicts, so resume restarts
    at test-first).
  • Pass 3: DESIGN_REVIEW: PASS. Its one minor (no defined source for the
    Retry report path) and one nit were then fixed:
    tl_run_failed_report, plus an accepted-risk note.
  • Waiver: pass-1 minor m2 claimed CLAUDE_CODE_SUBAGENT_MODEL overrides an
    explicit dispatch model. Rejected: the documented precedence is
    per-dispatch model > agent frontmatter > CLAUDE_CODE_SUBAGENT_MODEL >
    main conversation (code.claude.com/docs/en/sub-agents). Pass 2 found
    no contrary evidence.

Open assumptions & waivers:

  • per-worker effort not controllable on Claude Code — resolved: workers run at the session effort, which is recorded with its source; mechanical verify records "low requested, session effort applies"; effort pins print an "unsupported on this harness; ignored" warning.
  • unusable-model dispatch may not error at start — resolved: a dispatch error, or no report and no work, counts as fell-back and the worker is re-dispatched on the parent; the live probe proves it, or the build halts BLOCKED.
  • CLAUDE_CODE_SUBAGENT_MODEL overrides inherit — resolved: treated as a pin for judgment slots and shown as pin:CLAUDE_CODE_SUBAGENT_MODEL; the light-pin warning applies.
  • transcript format undocumented — resolved: FR-86 warns and asks; on Continue, mechanical verify inherits, escalation is never already-top, and the run record shows parent=unknown.
  • escalated resume re-runs only the failed gate — resolved: Retry re-dispatches the implementer on the existing branch with the failed gate's report, then runs every gate.
  • no retry path for gate-FAIL halts — resolved: step 3 offers Retry for failed/gate-fail TDDs in the latest run.

Build order: 0064 → 0065 → 0066 → 0067 (sequential; stacked PRs).

🤖 Generated with Claude Code

https://claude.ai/code/session_01Ki3TDnP2bNqqpEkFoZaEXq

Designs PRD f6ef178 (PR #181). Judgment workers inherit the parent
session's model; mechanical verify on a light tier capped at the parent;
FR-86 narrowed to a light-tier/unreadable parent check; /build-tdds
escalation (FR-88) with a new step-3 Retry path.

- 0064 (revised in place): models.sh role resolution (inherit / pins /
  light / parent-cap), parent-model observation from the session
  transcript, tier placement, design-reviewer `model: inherit`.
  Supersedes 0057 and 0062's models.sh contract only (0062 stays
  implemented).
- 0065 (revised in place): FR-86 light-tier/unreadable warn+ask via an
  executable, marker-tagged block in /prd-author, /tdd-author,
  /build-tdds; the vendor-page fetch is removed.
- 0066 (new): /build-tdds dispatch (no model parameter = inherit),
  queue confirmation with model + effort per worker, light-pin and
  effort-pin warnings, per-TDD run-record sidecar <slug>.models.json.
- 0067 (new): Retry for failed/gate-fail TDDs (implementer fix on the
  existing branch, then all gates), escalation requested/auto, outcomes
  escalated/already-top/fell-back, fall-back at dispatch, live probe.
- ADR 0015 supersedes ADR 0014 (fresh-worker independence carried forward).

Every TDD carries the same co-created evaluation rubric; it adds an
"executable verification" criterion, because 0064-rev1 halted on a skill
snippet that its eval only grepped.

Design critique (design-reviewer, three fresh passes):
- Pass 1: BLOCK. M1 auto-escalation was unreachable (step 3 resumed only
  non-terminal TDDs); M2 the confirmation could not show the escalated
  model; M3 skill blocks relied on cross-call shell state and "$@";
  M4 two 0064 observations contradicted 0064's own rules. All fixed. M1
  was resolved by a user-chosen step-3 Retry path.
- Pass 2: BLOCK. The step-3 entry condition was not widened, and Retry
  left stale PASS verdicts. Fixed with tl_run_latest_run, a widened
  trigger, and tl_run_retry_begin (archives verdicts, so resume restarts
  at test-first).
- Pass 3: DESIGN_REVIEW: PASS. Its one minor (no defined source for the
  Retry report path) and one nit were then fixed:
  tl_run_failed_report, plus an accepted-risk note.
- Waiver: pass-1 minor m2 claimed CLAUDE_CODE_SUBAGENT_MODEL overrides an
  explicit dispatch model. Rejected: the documented precedence is
  per-dispatch model > agent frontmatter > CLAUDE_CODE_SUBAGENT_MODEL >
  main conversation (code.claude.com/docs/en/sub-agents). Pass 2 found
  no contrary evidence.

Open assumptions & waivers:
- per-worker effort not controllable on Claude Code — resolved: workers run at the session effort, which is recorded with its source; mechanical verify records "low requested, session effort applies"; effort pins print an "unsupported on this harness; ignored" warning.
- unusable-model dispatch may not error at start — resolved: a dispatch error, or no report and no work, counts as fell-back and the worker is re-dispatched on the parent; the live probe proves it, or the build halts BLOCKED.
- CLAUDE_CODE_SUBAGENT_MODEL overrides inherit — resolved: treated as a pin for judgment slots and shown as pin:CLAUDE_CODE_SUBAGENT_MODEL; the light-pin warning applies.
- transcript format undocumented — resolved: FR-86 warns and asks; on Continue, mechanical verify inherits, escalation is never already-top, and the run record shows parent=unknown.
- escalated resume re-runs only the failed gate — resolved: Retry re-dispatches the implementer on the existing branch with the failed gate's report, then runs every gate.
- no retry path for gate-FAIL halts — resolved: step 3 offers Retry for failed/gate-fail TDDs in the latest run.

Build order: 0064 → 0065 → 0066 → 0067 (sequential; stacked PRs).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Ki3TDnP2bNqqpEkFoZaEXq
@cahenesy
cahenesy merged commit 328d926 into master Oct 5, 2026
1 check passed
@cahenesy
cahenesy deleted the docs/design/model-cost-performance branch October 5, 2026 17:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant