Repository navigation
feat(ui): score matrix cells over one case set, fold old configurations, and lay score cards on a tray - #33
Merged
Conversation
…ns, and lay score cards on a tray - Matrix coverage compares runs by their cases (`CaseNode.identity`: the `benchmark.case.id` metric, else `vieval.case.id`, else the case name). A settled run over other cases (smoke or limited run, another case set under the same entry) is `other`: kept in the history, never the headline or a baseline. - Cells of a configuration newer runs moved away from are `superseded`; their columns fold behind "Show N older configurations". - Scored cells show how the scores spread (`ScoreBar`) instead of an all-green case bar; shared axes move above the table, and columns name only the axes they differ in. - Project cards show the newest run's score instead of the best of all runs. - ScoreCard is a white panel on a gray tray, with the matrix values on the tray; the sequential ramp follows the theme hue. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Approval required for deploying to Cloudflare Workers (Preview) for docs.
|
The concurrency tests waited ~300 ms (waitForExpectation) or 200 ms for the first case to start, less than loading the config and transpiling the eval files takes on a busy CI runner. Wait with vi.waitFor and a shared 3 s timeout instead; it still returns as soon as the expectation holds. Co-Authored-By: Claude Opus 5.5 <[email protected]>
The previous commit attached RUN_STATE_WAIT to runVievalCli in the concurrency-cases-from-inputs test instead of to a waitFor, which broke the typecheck. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Reading AIRI
tts-speech-formulain the 0.8 viewer still misled:fullwhen it settled every case it planned. A 5-case smoke run replaced a 67-case result (apple-speech · official: 0.0% ↓65.7), and deltas compared runs over different cases. One entry (en) also ran a second 120-case set (mb-*), so "largest run = dataset" is wrong too.asraxis dropped, ajudgeaxis added) sat beside current ones asasr — · judge —.What changes
Matrix
CaseNode.identity: a case's identity across runs, resolved likevieval report compare(benchmark.case.id, thenvieval.case.id, then the case name).other: kept in the history as rings, never the headline or a baseline.MatrixCell.caseCountis the headline's case count.MatrixCell.superseded: a newer run ran the same entry and run row with other eval values. Columns with only superseded cells fold behind "Show N older configurations"; shown ones are dimmed.ScoreBar(fromlatest.caseScores) instead of the case-state bar, without apassedbadge. Cells show their case count.Cards and color
—without one).ScoreCardis a white panel on a gray tray (anatomy of airi'sIconStatusItem), matrix values on the tray;compactfor matrix cells.--seq-*follows the theme hue, light to mid tone (92% → 58% OKLCH lightness), staying in sRGB so a deep orange never clips to red.Known limits
benchmark.case.id/vieval.case.idfall back to the case name, which only aligns when names are stable.Testing
pnpm typecheck,pnpm lint(only existing warnings outside this change),pnpm exec vitest run packages/ui(187 passed).src/server/compare.test.ts: settled runs over some cases areother; a second case set under one entry; superseded cells.tts-speech-formulareports, light and dark.🤖 Generated with Claude Code