Skip to content

[Bench] Group benchmark rows by workload label and keep manifest order #54

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
lcy-seso merged 4 commits into from
Sep 27, 2026
Merged

[Bench] Group benchmark rows by workload label and keep manifest order #54

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
e5eae86
[Bench] Group benchmark rows by workload label and keep manifest order
lcy-seso Sep 27, 2026
3339b6e
[Bench] Print each label's substituted shapes and wrap long key rows …
lcy-seso Sep 27, 2026
7df4e8a
[Docs] Bring the hand-written pages in line with TileOPs main
lcy-seso Sep 27, 2026
29b0429
[Docs] Describe the current backend example, and polish the pages bro…
lcy-seso Sep 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CLAUDE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,7 @@ implementation of the same op on that workload.
| Device time | The compared quantity is `device_busy_ms`, never wall-clock span. |
| Two questions | `Ratio` says whether someone else's kernel is faster; `SOL` how much faster the hardware allows anyone to go, with the binding resource (`mem`/`comp`/`lat`) in its own `Bound` column. The SOL arithmetic and thresholds are imported from the checkout's roofline tool (M5) — never re-derived here. |
| Order follows the API Reference | `DATA_PAGES` takes the API nav's order over the same families — a page per family except `Conv & Pool` (two) and `Other` (Top-k, FFT, mHC, Engram, the rest). Within a page, ops sit in the order `docs/api/` names them, read by `api_op_order()`; an op no API page names comes last, ranked by verdict. `_BENCH_ORDER` in `hooks.py` repeats the page order for the nav: change one, change the other. |
| Rows follow the manifest | One row group per manifest label, one row per dtype under it in a `dtype` column. Labels keep the order the snapshot lists them in, which is the manifest's; the key above the table lists them in the same order. A row no manifest describes takes its id, trailing dtype names split off, as its label. |
| Workload shapes | The snapshot names a workload but does not carry its shapes. `scripts/workload_shape.py` reads them from the TileOPs spec manifest at the commit the benchmark ran on, joined by the `<label>-<dtype>` the benchmark id is built from. A workload the manifest does not declare keeps its id and gets no shapes — never a guessed one. |

## Bilingual pages (en / zh)
Expand Down
3 changes: 2 additions & 1 deletion docs/api/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,8 @@ The pages are ordered by how much an op composes: the pointwise transforms first
the axis reductions and the normalizations built on them, then the matmul and the
expert routing over it, then the windowed and spectral transforms, then the
sequence-model kernels built on all of the above. It is the order `tileops` declares
its op families in.
its op families in. The exception is Top-k, whose one op is exported from
`tileops.attention`.

| Page | What it covers |
| --- | --- |
Expand Down
152 changes: 74 additions & 78 deletions docs/assets/extra.css
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -634,7 +634,11 @@ html[lang="zh"] .md-typeset .keystone {
text-align: center;
}

.md-typeset .datatable table:not([class]) td.colsep {
/* The label spans its dtype rows, and wraps at a hyphen rather than widening
* the column: an attention label runs past seventy characters.
*/
.md-typeset .datatable table:not([class]) td.wl-name {
white-space: normal;
text-align: left;
}

Expand Down Expand Up @@ -669,8 +673,8 @@ html[lang="zh"] .md-typeset .keystone {
}

/* Two header rows, still one middle rule: it belongs under the whole header
* block, not between its halves. The `Workload` cell spans both rows, so its
* own bottom edge already sits on that line.
* block, not between its halves. The `Workload` and `dtype` cells span both
* rows, so their own bottom edges already sit on that line.
*/
/* Each header word underlined thin and grey — under `Alternatives` that also
* says how far a header spanning two columns reaches. The heavy rule stays
Expand Down Expand Up @@ -698,9 +702,15 @@ html[lang="zh"] .md-typeset .keystone {

/* A cool cast on the names, so the teal rule divides a label column from ink
* measurements rather than sitting between them. The same in every row: a
* colour that never varies cannot be mistaken for a verdict.
*/
.md-typeset .datatable table:not([class]) td.colsep code {
* colour that never varies cannot be mistaken for a verdict. As wide as the
* label up to 18em and no narrower: a table squeezed to the page would
* otherwise break the label at every hyphen.
*/
.md-typeset .datatable table:not([class]) td.wl-name code {
display: inline-block;
width: max-content;
max-width: 18em;
white-space: normal;
color: #316c6f;
}

Expand Down Expand Up @@ -1191,12 +1201,10 @@ html[lang="zh"] .md-nav__title {
text-align: left;
}

/* The workload key carries the shapes at the width of the page, so the table
* holds only `W1`, `W2`, … and the measurements.
*/
/* The workload key: what an op's workloads have in common, stated once, and
* what varies, one line per workload. Three columns — code, what varies, the
* benchmark's own id — so a reader compares down a column.
* what varies, one line per label. Two columns — the label, what varies — so a
* reader compares down a column. It carries the shapes at the width of the
* page, so the table holds only the labels and the measurements.
*/
.md-typeset .wl-key {
margin: 0.35em 0 0.8em;
Expand Down Expand Up @@ -1224,7 +1232,7 @@ html[lang="zh"] .md-nav__title {
.md-typeset .wl-key .wl-shared {
display: flex;
flex-wrap: wrap;
gap: 0.15em 0.32em;
row-gap: 0.15em;
margin: 0 0 0.15em;
font-size: inherit;
line-height: inherit;
Expand All @@ -1235,17 +1243,26 @@ html[lang="zh"] .md-nav__title {
margin-bottom: 0.2em;
}

/* A middot between entries, not a second rule: punctuation separates without
* claiming to align. Each entry holds its middot in its left padding; the line
* is shifted left by that padding and clipped, so no line starts with one.
*/
.md-typeset .wl-key .wl-shared,
.md-typeset .wl-key .wl-flow {
margin-left: -1.1em;
clip-path: inset(0 0 0 1.1em);
}

.md-typeset .wl-key .wl-cell {
position: relative;
padding-left: 1.1em;
white-space: nowrap;
}

/* A middot between entries, not a second rule: the rule after the code falls in
* the same place on every row, a rule between entries would land wherever the
* previous name ended. Punctuation separates without claiming to align.
*/
.md-typeset .wl-key .wl-cell + .wl-cell::before {
.md-typeset .wl-key .wl-cell::before {
content: "·";
margin-right: 0.32em;
position: absolute;
left: 0.25em;
color: rgba(26, 23, 32, 0.5);
font-weight: 400;
}
Expand Down Expand Up @@ -1280,90 +1297,79 @@ html[lang="zh"] .md-nav__title {
}
}

/* Labels in one column, what varies in the next. A label past the column's
* width wraps at its hyphens, so a long one never pushes the values off.
*/
.md-typeset .wl-key ul.wl-rows {
display: grid;
grid-template-columns: fit-content(24em) minmax(0, 1fr);
gap: 0.1em 0.9em;
align-items: baseline;
margin: 0;
padding: 0;
list-style: none;
}

/* The id ranges right, off the same edge on every row, so the middle column is
* free to be as long as one workload needs without ragging the rest.
*/
.md-typeset .wl-key ul.wl-rows li {
display: flex;
flex-wrap: wrap;
align-items: baseline;
gap: 0 0.9em;
margin: 0;
display: contents;
}

.md-typeset .wl-key ul.wl-rows b {
flex: 0 0 2.1em;
color: var(--tf-violet);
font-weight: 700;
/* No width for two columns: each label on its own line, what varies under it. */
@media (max-width: 50rem) {
.md-typeset .wl-key ul.wl-rows {
grid-template-columns: minmax(0, 1fr);
}
}

/* A hairline between the code and what it ran on — the same rule the table
* draws between the workload column and its measurements.
/* A hairline between the label and what it ran on — the same rule the table
* draws between the workload columns and their measurements.
*/
.md-typeset .wl-key .wl-delta {
display: flex;
flex: 1 1 auto;
flex-wrap: wrap;
gap: 0.15em 0.32em;
padding-left: 0.9em;
border-left: 1px solid rgba(26, 23, 32, 0.2);
}

.md-typeset .wl-key .wl-delta:empty {
border-left: 0;
/* Tensors, then scalars: one line when both fit, else the scalars wrap whole. */
.md-typeset .wl-key .wl-flow,
.md-typeset .wl-key .wl-part {
display: flex;
flex-wrap: wrap;
row-gap: 0.15em;
}

/* A tensor list wider than the column: one tensor per line, and no middots —
* stacked entries are already separated by the line break.
*/
.md-typeset .wl-key .wl-shared.wl-stack {
flex-direction: column;
align-items: flex-start;
gap: 0;
.md-typeset .wl-key .wl-part {
min-width: 0;
}

.md-typeset .wl-key .wl-shared.wl-stack .wl-cell + .wl-cell::before {
content: none;
margin-right: 0;
/* Labels too long for the label column: each on its own line, values under it. */
.md-typeset .wl-key ul.wl-rows.wl-long {
grid-template-columns: minmax(0, 1fr);
row-gap: 0;
}

/* Scalars wider than the column: the id takes the first line beside the code,
* the scalars the second, with the rule running down both.
*/
.md-typeset .wl-key ul.wl-long li {
margin-bottom: 0.3em;
.md-typeset .wl-key ul.wl-rows.wl-long .wl-id {
margin-top: 0.4em;
}

.md-typeset .wl-key ul.wl-long li:last-child {
margin-bottom: 0;
.md-typeset .wl-key ul.wl-rows.wl-long li:first-child .wl-id {
margin-top: 0;
}

.md-typeset .wl-key ul.wl-long .wl-id {
order: 1;
flex: 1 1 auto;
margin-left: 0;
padding-left: 0.9em;
border-left: 1px solid rgba(26, 23, 32, 0.2);
.md-typeset .wl-key .wl-delta:empty {
border-left: 0;
}

.md-typeset .wl-key ul.wl-long .wl-delta {
order: 2;
flex: 0 0 calc(100% - 3em);
margin-left: 3em;
/* A tensor list wider than the column: one tensor per line. */
.md-typeset .wl-key .wl-shared.wl-stack {
flex-direction: column;
align-items: flex-start;
gap: 0;
}

.md-typeset .wl-key .wl-id {
flex: 0 0 auto;
margin-left: auto;
padding: 0;
background: transparent;
color: var(--tf-muted);
color: #316c6f;
}

/* A tensor's name is not its shape, and a symbol is not its value: teal
Expand All @@ -1380,16 +1386,6 @@ html[lang="zh"] .md-nav__title {
color: var(--tf-muted);
}

/* The workload column is now two characters wide, so it centres like the
* numbers rather than ranging left as a label did.
*/
.md-typeset .datatable table:not([class]) td.colsep b {
color: var(--tf-violet);
font-weight: 700;
}



/* Roofline figure (performance-guides/memory-bound). Log-log, so the two decades
between an elementwise kernel and the ridge stay legible. Teal is the roof,
violet the ridge, rose a kernel below it — the tables' rose for "behind". */
Expand Down
Loading
Loading