Skip to content

docs: correct the load note, CONTRIBUTING's gate, PLAN, the env example, CLI help, and the issue templates - #85

Merged
TMHSDigital merged 1 commit into
mainfrom
docs/small-fixes
Sep 23, 2026
Merged

TMHSDigital merged 1 commit into
mainfrom
docs/small-fixes

Conversation

@TMHSDigital

Copy link
Copy Markdown
Owner

Fixes #62. Fixes #63. Fixes #64. Fixes #65. Fixes #66. Fixes #67.

#62: the load note and the "Asked" line.

  • The load summary said every JevBench row "is asked as a one-of-n choice". That's false for an adapter that asks yes/no rows as yes/no questions. It now says each row is asked as the question type it states.
  • "3 choice asked as failed" now reads "3 choice rows failed", and "67 choice asked as choice" reads "67 choice rows asked as choice".
  • The example report is regenerated from its recorded command. It differs in exactly those two lines and the date. This changes report text, not any number.

#63: CONTRIBUTING.

  • It said "All three must pass" above four commands, and named the four test jobs as the whole gate. It now lists the required checks.
  • A new section covers working on the site: the build and its four checks, the golden fixture, regenerating the example report, and the CSP.

#64: PLAN. It no longer lists pushing the tag, going public, the About box or the badge as still to do. It no longer calls datasets/private/.gitkeep untracked, and its repository description matches the live one.

#65: .env.example and the generative arm.

  • .env.example said to copy it to .env, which nothing reads, and listed an Ollama adapter that doesn't exist. It now says to export the variables, and lists ANTHROPIC_BASE_URL.
  • CONTRIBUTING and the adapter template called generative a home for "any chat model". It speaks Anthropic's Messages API only, and both now say so.

#66: CLI help.

  • The app help ("Choose and configure a decision model") contradicted the README's "no opinion about which model". It now says what the tool measures.
  • --adapter and --semantics list their valid values.
  • plumbline adapters prints local_logits (needs the local extra: uv sync --extra local) when the extra is missing.
  • The dataset README's command uses uv run python.

#67: issue templates. The adapter template applied adapter-config, a label that doesn't exist; it now applies adapter. A new site bug template asks for the page address (including its inputs), the browser, and console errors.

Checked

  • ruff, mypy --strict, and the full pytest suite pass.
  • The site builds, and the example report matches its command again after regenerating.
  • Parity, links (12 pages), search (98 entries) and the browser smoke test all pass, and the em-dash and -- checks are clean.

Separately, as #63 suggests, once this merges I'll make the quickstart, prose, wheel and site checks required in the main ruleset. They pass on every PR, and CONTRIBUTING now describes them as required.

🤖 Generated with Claude Code

…le, CLI help, and the issue templates

#62: the load summary said every JevBench row "is asked as a one-of-n
choice", which is false for an adapter that asks yes/no rows as yes/no
questions. It now says each row is asked as the question type it states.
The report's "Asked" line read "3 choice asked as failed"; it now reads
"3 choice rows failed", and "67 choice rows asked as choice". The example
report is regenerated from its command and differs in those two lines and
its date.

#63: CONTRIBUTING said "All three must pass" above four commands, and named
the four test jobs as the whole gate. It now lists the required checks and
adds a section on working on the site: the build and its four checks, the
golden fixture, regenerating the example report, and the CSP.

#64: PLAN listed pushing the tag, going public, the About box and the badge
as still to do, said datasets/private/.gitkeep was untracked, and carried a
repository description that no longer matched. All three are corrected.

#65: .env.example said to copy it to .env, which nothing reads, and listed
an Ollama adapter that does not exist. It now says to export the variables,
and lists ANTHROPIC_BASE_URL. CONTRIBUTING and the adapter template called
generative a home for "any chat model"; it speaks Anthropic's Messages API.

#66: the app help ("Choose and configure a decision model") contradicted the
README; it now says what the tool measures. --adapter and --semantics list
their choices. plumbline adapters marks local_logits when its extra is not
installed. The dataset README's command uses uv run python.

#67: the adapter template applied the label adapter-config, which does not
exist; it applies adapter. A site bug template asks for the page address
(inputs included), the browser, and console errors.

Fixes #62. Fixes #63. Fixes #64. Fixes #65. Fixes #66. Fixes #67.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
@TMHSDigital
TMHSDigital merged commit f19ccfc into main Sep 23, 2026
17 checks passed
@TMHSDigital
TMHSDigital deleted the docs/small-fixes branch September 23, 2026 19:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment