Repository navigation
registry: register the 3.8 Flash slug for the Google lane - #148
Closed
Zebra-rancher wants to merge 2 commits into
Closed
Zebra-rancher wants to merge 2 commits into
Zebra-rancher wants to merge 2 commits into
Conversation
* registry: add claude + gemini engines; MODEL-NOTES audition entries
Register two worker engines in the model-identity taxonomy:
- claude (Claude Code CLI): Anthropic sonnet/haiku/opus/fable via
`--model {model}`, default sonnet. Proven working 2026-09-03 (haiku
probe, PASS attempt 1). MODEL-NOTES documents the setup-token auth path
and its two footguns (silent expiry with no refresh token; paste
truncation -> 401 invalid).
- gemini (Gemini CLI): gemini-3.5-flash, verified 2026-09-02.
MODEL-NOTES also adds dated audition entries for codex, glm-5.2, and
gemini-3.5-flash from 2026-09-02/03 runs.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
* registry: key claude models by reported slug; sonnet/opus bakeoff
Fix the claude engine entries to key on the harness self-reported slug
(claude-sonnet-5, claude-opus-5, claude-haiku-4-5-20251001) instead of the
CLI aliases, per docs/TAXONOMY.md — alias keys left every claude model
showing as [unregistered] on the scoreboard. Note that `--model opus` now
self-reports claude-opus-5 (not 4.8). fable stays unregistered until a run
self-reports its slug.
Adds a code-feature bakeoff MODEL-NOTES entry: sonnet and opus both PASS
attempt 1 on roman_to_int; opus ~5x tokens and slower for an identical
correct result, so sonnet is the default lane.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
* registry: add claude-fable-5-1 from identity probe
fable probe PASS attempt 1, self-reports claude-fable-5-1 (1M context,
provider firstParty). Now registered, completing the claude engine's four
models (sonnet-5, opus-5, haiku-4-5, fable-5-1). MODEL-NOTES flags fable's
~$0.67 trivial-probe cost — orchestrator-class, not a worker lane.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
---------
Co-authored-by: Zebra-rancher <[email protected]>
Co-authored-by: Claude Opus 4.8 <[email protected]>
Gemini CLI 0.63.0 (upgraded 2026-10-06) self-reports gemini-3.8-flash in stats.models for a -m gemini-3.5-flash request, so lane runs were about to resolve as unregistered. Keep 3.5-flash for older logs. Co-Authored-By: Claude Fable 5.1 <[email protected]>
Author
|
Opened against the wrong repo by mistake; this belongs on the Zebra-rancher fork. Sorry for the noise. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Gemini CLI upgraded 0.58.0 -> 0.63.0 on 2026-10-06, prompted by Google's deprecation notice for
thinking_budget/temperature/top_p/top_k(the CLI was the only sender of those fields on Travis's Gemini key; the direct scripts and the shop-os gateway path were already clean).A headless smoke on the new CLI (
-m gemini-3.5-flash -o json) now self-reportsgemini-3.8-flashinstats.models, which the identity registry did not list, so lane runs would have been marked unregistered. This adds the entry and keeps 3.5-flash for older logs.tomllib).tests/test_identity_evidence.py,tests/test_model_field.py,tests/test_model_db.pypass.🤖 Generated with Claude Code