Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
78 changes: 66 additions & 12 deletions .github/workflows/site.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,10 @@ name: Site
# The site at tmhsdigital.github.io/plumbline carries a JavaScript
# reimplementation of the calibration floor. If it disagrees with the Python,
# the site misreports the tool, which is worse than having no site. This
# workflow holds the two together. The site is assembled inside this job, and
# the deploy job (not yet added) will need it, so a wrong calculator or a
# broken doc never ships. A stale site is the better failure.
# workflow holds the two together, checks the assembled site as a visitor would
# meet it, and only then deploys. Pull requests run both checks and never
# deploy; a push to main or a manual run deploys only if both pass. A stale
# site is the better failure.
#
# It is deliberately separate from ci.yml and release.yml: it shares no jobs,
# no concurrency group, and no permissions with either.
Expand All @@ -18,9 +19,11 @@ on:
permissions:
contents: read

# A newer push to a pull request makes its older run pointless. On main, a run
# is never cancelled part way, so a deploy is never cut off.
concurrency:
group: site-${{ github.ref }}
cancel-in-progress: true
cancel-in-progress: ${{ github.event_name == 'pull_request' }}

jobs:
parity:
Expand All @@ -44,15 +47,11 @@ jobs:
- name: The committed fixture is what the Python produces today
run: uv run python scripts/floor_golden.py --check

# The renderer's refusals (raw HTML off the allowlist, broken links and
# anchors, remote images, a modified vendored file) each have a case.
- name: The doc renderer refuses what it should
run: node scripts/render_docs.mjs --self-test

# Regenerates the worked example from the mock adapter, by the command
# docs/example-report.md records, and refuses if the report is stale.
# Renders the repository's docs from this commit, and refuses on a broken
# link or anchor, or raw HTML outside the allowlist.
# docs/example-report.md records, and refuses if any line of the report
# differs from what that command prints today. Renders the repository's
# docs from this commit, and refuses on a broken link or anchor, or raw
# HTML outside the allowlist.
- name: Assemble the site
run: uv run python scripts/build_site.py --out _site

Expand All @@ -63,3 +62,58 @@ jobs:
run: |
node --version
node scripts/check_floor_parity.mjs _site/example-run.json

# The exact bytes the deploy job publishes. On a pull request this is
# also a downloadable preview of the site.
- name: Keep the assembled site
uses: actions/upload-pages-artifact@v5
with:
path: _site

docs:
name: the assembled site's links, anchors, and meta tags resolve
needs: parity
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v5

# The renderer's refusals (raw HTML off the allowlist, broken links and
# anchors, remote images, a modified vendored file) each have a case.
- name: The doc renderer refuses what it should
run: node scripts/render_docs.mjs --self-test

- name: Fetch the assembled site
uses: actions/download-artifact@v8
with:
name: github-pages
path: artifact

# Checked as published, not as built: the same archive the deploy job
# serves, unpacked.
- name: Every link, anchor, and meta tag resolves, and nothing loads from another origin
run: |
mkdir site-as-published
tar -xf artifact/artifact.tar -C site-as-published
node scripts/check_site_links.mjs site-as-published

deploy:
name: deploy to GitHub Pages
needs: [parity, docs]
if: github.ref == 'refs/heads/main' && (github.event_name == 'push' || github.event_name == 'workflow_dispatch')
runs-on: ubuntu-latest
permissions:
pages: write
id-token: write
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
# One deploy at a time; a queued one waits rather than cancelling.
concurrency:
group: pages
cancel-in-progress: false

steps:
- name: Publish the checked site
id: deployment
uses: actions/deploy-pages@v5
8 changes: 7 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ different event from one that moved because it was wrong.
`synthetic_floor` that reproduces numpy's seeded random stream draw for draw.
`scripts/floor_golden.py` exports golden values from the Python and
`scripts/check_floor_parity.mjs` holds the port to them within 1e-9 in CI.
The page is not deployed yet.
- The site page: the argument, a worked example that derives the example
report's ECE line from its 105 rows in the browser, and a sample-size planner.
`scripts/build_site.py` regenerates the example from the mock adapter at build
Expand All @@ -29,6 +28,13 @@ different event from one that moved because it was wrong.
the runner's Node. Each page names the commit and build time it came from.
The build fails on a broken link or anchor, on raw HTML outside a short
allowlist, and on a modified vendored renderer.
- The site is live at <https://tmhsdigital.github.io/plumbline/>, deployed by
`site.yml` from `main` only after the parity check passes and
`scripts/check_site_links.mjs` finds every link, anchor, and meta tag in the
assembled site resolving and nothing loading from another origin. Every page
carries a canonical URL and Open Graph and Twitter card tags; the card image
(`site/og.png`) is rendered from `scripts/og_image.html`. Missing paths get a
404 page that links back.

### Changed

Expand Down
3 changes: 3 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@
Measure whether a decision model's probabilities are trustworthy on your own
labeled data, and decide what to do about it.

Check where your own ECE sits against its floor, in the browser, with nothing
installed: **[tmhsdigital.github.io/plumbline](https://tmhsdigital.github.io/plumbline/)**.

> **v0.1.0, one maintainer.** The measurement behaviour is settled; the Python
> API and the CLI flags are not, and will change in v0.2. Pin a version if you
> build on it.
Expand Down
104 changes: 81 additions & 23 deletions scripts/build_site.py
Original file line number Diff line number Diff line change
Expand Up @@ -351,11 +351,12 @@ def render_docs(prov: Provenance) -> dict[str, dict[str, str]]:
return rendered


def _nav(current: str | None, up: str) -> str:
def _nav(current: str | None, up: str, docs: str = "") -> str:
"""Links to the explainer (``up``) and every doc (``docs`` + slug)."""
items = [f'<li><a href="{up}">The ECE floor</a></li>']
for doc in DOCS:
mark = ' aria-current="page"' if doc.slug == current else ""
items.append(f'<li><a href="{doc.slug}.html"{mark}>{html.escape(doc.label)}</a></li>')
items.append(f'<li><a href="{docs}{doc.slug}.html"{mark}>{html.escape(doc.label)}</a></li>')
joined = "\n ".join(items)
return f'<nav aria-label="Documentation">\n <ul>\n {joined}\n </ul>\n</nav>'

Expand All @@ -367,22 +368,64 @@ def _nav(current: str | None, up: str) -> str:
"%3Cpath d='M5 10h6l-3 5z' fill='%23555'/%3E%3C/svg%3E"
)

#: Where Pages serves the site. Canonical and Open Graph URLs are absolute, and
#: the 404 page links by absolute path, because it is served at any depth.
SITE_URL = "https://tmhsdigital.github.io/plumbline/"
SITE_PATH = "/plumbline/"
OG_IMAGE_ALT = (
"A calibration claim has a floor: ECE 0.074 on 105 rows sits below the floor's "
"95th percentile of 0.111, so the result is inconclusive."
)


def social_meta(title: str, description: str, url: str) -> str:
"""Canonical, Open Graph, and Twitter card tags for one page.

``site/index.html`` carries the same tags written out by hand, and
``scripts/check_site_links.mjs`` checks every page has them and that its
canonical and og:url name the page itself.
"""
title, description = html.escape(title), html.escape(description)
image = f"{SITE_URL}og.png"
alt = html.escape(OG_IMAGE_ALT)
return "\n".join(
(
f'<link rel="canonical" href="{url}">',
'<meta property="og:type" content="website">',
'<meta property="og:site_name" content="plumbline">',
f'<meta property="og:title" content="{title}">',
f'<meta property="og:description" content="{description}">',
f'<meta property="og:url" content="{url}">',
f'<meta property="og:image" content="{image}">',
'<meta property="og:image:width" content="1200">',
'<meta property="og:image:height" content="630">',
f'<meta property="og:image:alt" content="{alt}">',
'<meta name="twitter:card" content="summary_large_image">',
f'<meta name="twitter:title" content="{title}">',
f'<meta name="twitter:description" content="{description}">',
f'<meta name="twitter:image" content="{image}">',
f'<meta name="twitter:image:alt" content="{alt}">',
)
)


PAGE = """<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>{title} | plumbline</title>
<meta name="description" content="{description}">
{meta}
<meta name="color-scheme" content="light dark">
<link rel="icon" href="{icon}">
<link rel="stylesheet" href="../base.css">
<link rel="stylesheet" href="../docs.css">
<link rel="stylesheet" href="{root}base.css">
<link rel="stylesheet" href="{root}docs.css">
</head>
<body>
<a class="skip" href="#doc">Skip to the document</a>
<header>
<p class="kicker"><a href="../">plumbline</a></p>
<p class="kicker"><a href="{root}">plumbline</a></p>
{nav}
</header>
<main id="doc">
Expand All @@ -398,6 +441,19 @@ def _nav(current: str | None, up: str) -> str:
"""


def _page(title: str, description: str, meta: str, root: str, nav: str, body: str) -> str:
return PAGE.format(
title=html.escape(title),
description=html.escape(description, quote=True),
meta=meta,
icon=ICON,
root=root,
nav=nav,
body=body,
repo=REPO,
)


def write_docs(out: Path, prov: Provenance, rendered: dict[str, dict[str, str]]) -> None:
docs = out / "docs"
docs.mkdir()
Expand All @@ -407,14 +463,8 @@ def write_docs(out: Path, prov: Provenance, rendered: dict[str, dict[str, str]])
f'<p class="provenance">{prov.line(doc)}</p>\n'
f'<article class="prose">\n{rendered[doc.slug]["html"]}</article>'
)
page = PAGE.format(
title=html.escape(doc.label),
description=html.escape(doc.blurb, quote=True),
nav=_nav(doc.slug, "../"),
body=body,
repo=REPO,
icon=ICON,
)
meta = social_meta(f"{doc.label} | plumbline", doc.blurb, f"{SITE_URL}docs/{doc.slug}.html")
page = _page(doc.label, doc.blurb, meta, "../", _nav(doc.slug, "../"), body)
(docs / f"{doc.slug}.html").write_text(page, encoding="utf-8")

listing = "\n".join(
Expand All @@ -431,17 +481,24 @@ def write_docs(out: Path, prov: Provenance, rendered: dict[str, dict[str, str]])
f"at {at}.</p>\n"
f'<ul class="doc-list">\n{listing}\n</ul>\n</article>'
)
(docs / "index.html").write_text(
PAGE.format(
title="Documentation",
description="plumbline's documentation, rendered from the repository.",
nav=_nav(None, "../"),
body=index,
repo=REPO,
icon=ICON,
),
encoding="utf-8",
description = "plumbline's documentation, rendered from the repository."
meta = social_meta("Documentation | plumbline", description, f"{SITE_URL}docs/")
page = _page("Documentation", description, meta, "../", _nav(None, "../"), index)
(docs / "index.html").write_text(page, encoding="utf-8")


def write_404(out: Path) -> None:
"""Pages serves this for any missing path, at any depth, so it links absolutely."""
body = (
'<article class="prose">\n<h1>Not found</h1>\n'
"<p>There is no page at this address.</p>\n"
f'<p><a href="{SITE_PATH}">Go to the explainer</a>, or '
f'<a href="{SITE_PATH}docs/">browse the documentation</a>.</p>\n</article>'
)
nav = _nav(None, SITE_PATH, f"{SITE_PATH}docs/")
meta = '<meta name="robots" content="noindex">'
page = _page("Not found", "No page at this address.", meta, SITE_PATH, nav, body)
(out / "404.html").write_text(page, encoding="utf-8")


def main() -> int:
Expand Down Expand Up @@ -470,6 +527,7 @@ def main() -> int:
shutil.copytree(SITE, out, ignore=shutil.ignore_patterns("vendor"))
(out / "example-run.json").write_text(json.dumps(example, indent=1) + "\n", encoding="utf-8")
write_docs(out, prov, rendered)
write_404(out)
print(f"site assembled in {out}")
return 0

Expand Down
Loading
Loading