Skip to content

design: TDDs 0064–0065 + ADR 0014 (model capability by job) - #179

Merged
cahenesy merged 1 commit into
masterfrom
docs/design/model-capability-by-job
Aug 16, 2026
Merged

cahenesy merged 1 commit into
masterfrom
docs/design/model-capability-by-job

Conversation

@cahenesy

Copy link
Copy Markdown
Owner

Design

PRD-rev 5dbf968 (PR #178). Two draft TDDs + one ADR.

  • TDD 0064 (supersedes 0057) — judgment-worker defaults and cheaper-pin warning (NFR-3, FR-87, FR-52, FR-10/15/50). review= equals build= when unset. Design-reviewer model: inherit. FR-87 parent warning uses “differs from,” not “below.”
  • TDD 0065 — FR-86 parent-session check. Observe the harness session artifact; fetch official models docs this turn; warn+ask if weaker or unreadable. No static rank map.
  • ADR 0014 — accepted; supersedes 0009. Revises ADR 0010’s “different-model artifact” consequence without editing 0010’s body.

Do not merge until you have reviewed the design. Merge is the design gate that makes these TDDs buildable (/build-tdds).

Design-critique verdict

Independent design-reviewer (fresh worker): DESIGN_REVIEW: PASS.

Rubric: co-created with the user (## Evaluation rubric identical in both TDDs). Gate graded against that table. No failing row. 0064 high-quality on traceability, interfaces, verification, scope, naming, no-rank-map; alternatives acceptable (the HQ live-fetch pair lives in 0065). 0065 high-quality on alternatives / no-rank-map / verification; interface concreteness acceptable (page-grounded judge is not mechanical).

Findings (none blocking; folded where cheap before this PR):

  • minor — Grok observe cwd: use $PWD first, git-root fallback (folded).
  • minor — skip Claude <synthetic> as message.model (folded).
  • minor — FR-87 verb “differs from” not “is below” (folded).
  • nit — Claude slug maps . → -; Grok encode example %2F… (folded).
  • nit — README different-model sweep deferred (named in 0064). Implement skill ADR cite retargeted 0009 → 0014 (in 0064 scope).

Open assumptions & waivers

  • session-model observation — resolved: session artifact (Grok summary.json current_model_id; Claude jsonl last non-synthetic message.model). Missing env/file/field = unreadable → warn+ask.
  • weaker/newer comparator — resolved: no static rank map. Compare is a this-turn official-docs fetch. Recency-only is forbidden. Fetch/judge fail is unreadable → warn+ask.
  • live catalog source and ranking — resolved: fetch Anthropic models overview + Claude Code model-config, or xAI models/flagship. Bar is live vendor top-tier and may be newer than the models.sh binding.
  • lookup failure / no key / offline — resolved: cannot fetch or cannot judge → unreadable → warn+ask. Evals do not hit the network.
  • design-reviewer frontmatter pin — resolved: inherit; no model: sonnet. Independence is a fresh worker.
  • FR-87 warning visibility — resolved: parent prints THROUGHLINE_*_MODEL=<id> differs from the most-capable default; continuing. Mechanical verify pin does not warn.
  • one binding site — resolved: models.sh is the only dispatch binding. FR-86 does not copy ids.
  • ADR 0009 conflict — resolved: ADR 0014 supersedes 0009.

PRD-rev 5dbf968. 0064 supersedes 0057: unset review= equals build=;
design-reviewer inherits; FR-87 parent warning. 0065 is FR-86:
session artifact + this-turn official-docs fetch; no static rank map.

Open assumptions & waivers:
- session-model observation — resolved: session artifact (Grok summary.json current_model_id; Claude jsonl last non-synthetic message.model). Missing env/file/field = unreadable → warn+ask.
- weaker/newer comparator — resolved: no static rank map. Compare is a this-turn official-docs fetch. Recency-only is forbidden. Fetch/judge fail is unreadable → warn+ask.
- live catalog source and ranking — resolved: fetch Anthropic models overview + Claude Code model-config, or xAI models/flagship. Bar is live vendor top-tier and may be newer than the models.sh binding.
- lookup failure / no key / offline — resolved: cannot fetch or cannot judge → unreadable → warn+ask. Evals do not hit the network.
- design-reviewer frontmatter pin — resolved: inherit; no model: sonnet. Independence is a fresh worker.
- FR-87 warning visibility — resolved: parent prints THROUGHLINE_*_MODEL=<id> differs from the most-capable default; continuing. Mechanical verify pin does not warn.
- one binding site — resolved: models.sh is the only dispatch binding. FR-86 does not copy ids.
- ADR 0009 conflict — resolved: ADR 0014 supersedes 0009.
@cahenesy
cahenesy merged commit a75c19e into master Aug 16, 2026
1 check passed
@cahenesy
cahenesy deleted the docs/design/model-capability-by-job branch August 16, 2026 17:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant