Redesign the report as a document, and add a demo walkthrough - #1
Merged
Merged
Conversation
The charts painted fixed hex greys and drew every legend entry as a square, whatever mark it was explaining; annotations sat on gridlines; the escalation line ran out of the plot and over the axis above it; and the segment chart's right-hand column printed "too few sam" in every report. - Okabe-Ito stays for series colour that carries meaning. Ink, muted, grid, diagonal and shade become constants the stylesheet re-maps for dark mode, so a chart grey can no longer be light-theme-only. - Legends draw the mark they explain: a line sample for a line, a dotted sample for points, a dashed sample for a dashed rule, a band for the flat region. - Annotations carry a halo instead of a background box, and the density strip is one shade of the page's own ink in either theme. - The reliability chart's "below the diagonal" region is the triangle it always was, not a full-width band, and the marker names the line it draws. - Component lines are clipped to the plot, and the "accept + escalate" line is omitted when it is the total line again. - Segments gain a track behind each bar, a bar scale that reads at a glance, and a right-anchored note column; a segment below the sample floor prints its reason instead of a stray "n/a". - The reliability chart's bottom budget is explicit: ticks, axis title, strip, then legend, so the axis title and the legend can no longer land on the same line.
The report looked assembled rather than drawn: shadowed cards, a run-on provenance line, headings the same size as body text, a tab row that read as text, and a live readout that disagreed with the table above it by half a point and three decimal places. - A real type scale, more air, and two semantic colours: the accent for the line the report recommends, the alert for the line in use. Light is a paper palette, dark a slate one, and print has its own; all three come from the same custom properties. - The header states its provenance one line at a time, because a `·`-separated run-on of paths and seed notes is read as boilerplate and skipped. - Sections gain headings, block subtitles, figure captions that name each encoding, a quiet disclosure for the data tables, and a tinted recommended column in the impact table. - The dark block's grey re-map is generated from the chart layer's constants, with a test that fails when a new grey is added to one and not the other. - The slider's readout uses the report's own formatting (whole percent, currency, grouped amounts), the monthly-cases field recomputes the monthly figures when it is typed into, and a question's line is labelled with the action it belongs to so two thresholds can no longer look like a contradiction.
The README opened with a 2x2 table of screenshots whose text rendered at half size, put the demo command 200 lines below the install, and buried what the tool answers under badge soup. - The screenshots are a single column of full-width crops, each with a caption that says what to look at; they are taken from the example report this repository commits, and every figure quoted in the README still checks against it. - The sections now run install, quick start, try it on your own log, hand it to an agent, instrument a service, what you get, what the numbers mean. - Regenerated docs/*.png and examples/report-example.html from the redesigned report, and updated the one size claim the README makes about it. - Recorded the design work in CHANGELOG.md, and ignored in-repo git worktrees.
The report is a document you read; a recording needs something you walk through. This is that, in one file, opened locally, driven with the arrow keys, with a narration line per step for the voice over: the verdict, the cost curve with a live threshold, four questions measured separately, the segments, and the drift gate failing on a degraded candidate model. It is not a product surface -- the product writes one report, and a dashboard is explicitly out of scope -- so it lives in `examples/` as a presentation of the same analysis: - `examples/make-demo-dashboard.py` calls the library the report calls (`evaluate`, `sweep_actions`, `segment_views`, `drift_view`, `build_verdict`, `build_impact`) and draws with the report's own chart modules and stylesheet, so nothing on screen is typed in, and the two artefacts cannot drift apart in look or in number. - The production half of the log is `jeval demo`'s own, at the same seed, so the first screen can be checked against `examples/report-example.html`. The candidate is the same four questions from a deliberately overconfident model, which is what makes the last step a real failure. - The live slider moves a marker along the curve and rewrites five figures, using the report's own formatting, and prints "n/a" where the auto-branch accuracy is undefined rather than a NaN. - `tests/test_demo_dashboard.py` holds it to the standards of a committed artefact: one file, nothing from the machine that built it, every control the walkthrough touches present, the printed numbers equal to a fresh computation -- and, because the build is deterministic, byte-equality with the committed file.
At 390px the report scrolled sideways: 538px of document in a 390px viewport, 737 elements past the right edge, starting with the impact table. A table of currency amounts has a minimum width wider than a phone -- "KRW 38,524,590.16" cannot wrap -- so at width:100% the table wins and takes the header, the paragraphs and every chart with it. The table now takes the overflow itself (display:block, max-width:100%, overflow-x:auto), and its type and padding come down at 640px so the impact table fits a 390px screen with nothing hidden. A narrow report now reads as a narrow report with one scrollable table, not as a wide page seen through a letterbox. The collapsible per-question detail tables are wider than the impact table and still need a swipe at 390px (26-127px). They are closed by default; the judgement was that a swipe inside one table beats a page that drifts sideways, and changing which columns they show is a product decision, not a layout fix.
The five-step walkthrough was a deck pretending to be a dashboard: a step rail to navigate, a narration bar to read from, a slider to drag, and every element set at the same 15px so nothing told the reader where to look. In dark mode it was worse than bland -- the greys were the same grey, the one semantic accent was a hairline, and the punchline (which threshold to move to) sat third in a row of three identically-styled statistics. This replaces it with one screen that makes one claim and then shows it: * "The line belongs at 0.75, not 0.60." in 29px, with the answer in the accent colour, repeated in the headline, the answer card, the table column and the chart line, so a reader who reads only the blue text still gets the whole argument. * The chart draws both thresholds and names them where they stand -- "in use 0.60" dashed amber, "where it belongs 0.75" solid accent -- and puts the confidence distribution underneath, so the line is something you can see rather than a number you have to trust. * The 95% interval on the recommendation is drawn as a band as well as printed: a point estimate quoted over a wide interval is the easiest thing on the page to over-read. * The small print now says what "cost" counts, from the cost matrix itself, and carries the drift caveat: a threshold measured on a model that is drifting is a number with an expiry date. Semantic colours are separated per theme instead of shared greys (accent, in-use, alert, plus a tinted answer panel), the type runs 29 / 46 / 15 / 13.5 / 12.5 instead of everything at one size, and there is no JavaScript left: one file, one screen, no scrolling on a 1440x900 window (measured: 900px of document in a 900px viewport, 945 in 945 at 1512x945). The guard test now defends the argument rather than the stepper: one h1, both thresholds drawn and named, the answer in the largest type, no walkthrough controls, plus the byte-for-byte rebuild and "nothing from this machine" checks it already had.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this is
A design overhaul of the two surfaces a reader actually looks at — the HTML report and the README —
plus a self-contained walkthrough built for recording a demo.
Five commits, one logical change each:
da7f6607d0bae2<title>/<desc>on every figure, dark mode as a palette swap with a single block of custom properties, print layout3891da8tests/test_readme_example.pygreen)2f2eb43examples/demo-dashboard.html: a five-step walkthrough — verdict, live threshold slider, per-question curves, segments, drift gate — one file, arrow keys, narration bar. Generated byexamples/make-demo-dashboard.py, guarded bytests/test_demo_dashboard.py(byte-equality with a fresh build, self-contained, no machine paths)8ab4db5main). The table now takes the overflow itself, and the impact table fits a 390px screen with nothing hiddenVerification
uv run pytest— 536 passed;ruff format/ruff check/mypy jevalcleandocument width equals viewport width at every size, no hidden columns in the impact table
/Count 7); pages 2–7 were not inspected pixel by pixelKnown limits
are closed by default, and choosing which columns to drop on a phone is a product decision
jevalships exactly one report, and a dashboard isexplicitly out of scope in
CLAUDE.md. It lives inexamples/as a generated artifact