Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ renderdiff <left-origin> <right-origin> <route…> [options]
| `--rerun-failed` | Rerun only what the last run in `--out` didn't find clean or accepted, keeping the rest. |
| `--cache-right` | Replay the right side's network traffic from the last `--cache-right` run in `--out`, when it's under an hour old and covers every shot; otherwise load it live and record it. Keeps `<out>/.cache/` when the out dir is wiped. See **Accepted differences and reruns**. |
| `--no-recheck` | Don't reload a differing shot's right side to find content that varies per load. Off anyway under `--cache-right`. |
| `--computed` | Also diff the computed style of every rendered element and its `::before`/`::after`, and say per route whether the markup is identical apart from asset hashes and origin. Pixels still decide the score and exit code. See [docs/output.md](docs/output.md#computed-styles---computed). |
| `--computed` | Also diff the computed style of every rendered element and its `::before`/`::after`, and links' and controls' `:hover`/`:focus-visible`, and say per route whether the markup is identical apart from asset hashes and origin. Pixels still decide the score and exit code. See [docs/output.md](docs/output.md#computed-styles---computed). |
| `--aria` | Also diff each route's ARIA snapshot (Playwright's accessibility tree) per width: left lines `+`, right `-`, asset hashes and origin normalized, masked elements left out. Pixels still decide. See [docs/output.md](docs/output.md#aria-snapshots---aria). |
| `--sitemap[=<url>]` | Add the paths in the right side's sitemap. Default `<right>/sitemap.xml`; give another URL or path with `=`. |
| `--dry-run` | Print the URL pairs and exit. No browser. |
Expand Down
15 changes: 13 additions & 2 deletions docs/output.md
Original file line number Diff line number Diff line change
Expand Up @@ -166,7 +166,8 @@ DOM and iframes aren't searched.
## Computed styles (`--computed`)

With `--computed`, after the shot renderdiff reads the computed style of every rendered
element on both sides: the properties `dom` compares, plus `content`, and `::before` and
element on both sides: the properties `dom` compares, plus `outline`,
`text-decoration-line` and `content`, and `::before` and
`::after` wherever they generate content. Masked subtrees and `script`/`style`-like tags
are skipped; a `display: none` element is read, its subtree isn't. Script-driven
animations are finished first. The summary gets one block, after the rows:
Expand Down Expand Up @@ -232,7 +233,17 @@ computed: 2 findings; dom identical on 2 of 3 route(s), differs /pricing (1), so
50,000 elements is reported as failed, with why (`computed: null`, `computedError`), and
the pixel result stands. Pixels still decide the score and the exit code.

`:hover`/`:focus` states aren't snapshotted.
Links and controls (`a[href]`, `button`, `input`, `select`, `textarea`, `summary`,
`[tabindex]`) are also read in `:hover` and `:focus-visible`, with their subtrees:

- The key gets the state: `body > nav > a[href="#x"]:hover > span.icon`. `:focus-visible`
is keyboard focus, forced as `:focus` plus `:focus-visible`.
- Only what differs from resting is a finding: a resting change isn't repeated per state.
- The first 500 per page are read; `states:` in the summary says where there were more
(`computed.unchecked` in results.json). One with more than 50 descendants is skipped.
- All of them are forced at once, so a rule relating two of them
(`ul:has(a:hover) a:not(:hover)`) isn't seen, nor `:hover` on an ancestor (`li:hover`
menus), `:active`, `:focus-within` or script hover handlers.

## ARIA snapshots (`--aria`)

Expand Down
6 changes: 3 additions & 3 deletions skills/renderdiff/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ the § named at each step.

| Change | Run |
| --- | --- |
| CSS only, markup untouched (refactor, dead-CSS removal, token swap) | pixels + `--computed`. Expect `dom identical` on every route: then each `computed:` finding is a real style change, even on a clean route. `dom differs`: often a wrapper's class changed, re-keying its subtree (§ `--computed`). Hover, focus and open menus aren't shot without a `before` hook, and dead-CSS removal breaks exactly those. |
| CSS only, markup untouched (refactor, dead-CSS removal, token swap) | pixels + `--computed`. Expect `dom identical` on every route: then each `computed:` finding is a real style change, even on a clean route. `dom differs`: often a wrapper's class changed, re-keying its subtree (§ `--computed`). Hover menus and open menus aren't shot without a `before` hook, and dead-CSS removal breaks exactly those. |
| Markup changes (framework, template or component migration) | pixels, plus `--aria`: `aria: identical` means a screen reader gets the same page. `--computed` findings would only be hints. |
| Dependency or build upgrade | pixels, plus `--computed` if the markup should be identical. |
| One component | pixels with `--selector <sel>` or `'/route @sel'`. |
Expand Down Expand Up @@ -71,8 +71,8 @@ unaccepted difference. § Exit codes.
2. **What you saw**: per route and width, element and property ("/pricing @1440: `.cta`
margin-left 24px→16px").
3. **Not verified**: errors, unstable or failed-asset rows, skipped widths, masks, accepts,
hooks, unshot states (hover, open menus), unopened regions, unlisted pages, and the
reference (base build or production).
hooks, unshot states (hover on non-interactive elements, open menus, `:active`), unopened
regions, unlisted pages, and the reference (base build or production).
4. **Call**: parity, regression, or expected change, and why.

A score without looking is not a verdict: 99.6% can be a missing nav link.
Expand Down
5 changes: 3 additions & 2 deletions skills/renderdiff/reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,8 @@ results.json · Gotchas.
- **Baseline when the change is already made:** serve the base build twice, on two ports, and
diff it against itself; whatever differs varies per load.
- **States:** only the loaded state is shot. Hover, focus, open menus and dialogs need a
`before` hook (§ Scoping); dead-CSS removal often breaks exactly those.
`before` hook (§ Scoping); dead-CSS removal often breaks exactly those. `--computed` diffs
links' and controls' hover and focus styles without shooting them.

## Output

Expand Down Expand Up @@ -168,7 +169,7 @@ computed: 2 findings; dom identical on 2 of 3 route(s), differs /pricing (1), so
`none`. An `auto` margin reads as `auto`, not the px it resolved to, which varies with
layout.
- A finding on a clean route is a change you can't see in this state: a hidden element
(closed menu), another breakpoint, `:hover`/`:focus` (not snapshotted). Decide it; a
(closed menu), another breakpoint, hover on non-interactive elements, open menus. Decide it; a
`before` hook can open the state.
- Fails per shot (reported, pixels stand) past 15s or 50,000 elements. Not walked: SVG
internals, shadow DOM, iframes, masked and `display: none` subtrees.
Expand Down
3 changes: 2 additions & 1 deletion src/cli.ts
Original file line number Diff line number Diff line change
Expand Up @@ -250,7 +250,8 @@ const HELP = `
--no-recheck Don't reload a differing shot's right side to find content that
varies per load. Off anyway under --cache-right.
--computed Also diff every element's computed style and ::before/::after,
and say whether the markup is identical. Pixels still decide.
and links' and controls' :hover/:focus-visible, and say whether
the markup is identical. Pixels still decide.
--aria Also diff each page's accessibility tree (Playwright's ARIA
snapshot) line by line. Pixels still decide.
--sitemap[=<url>] Add the paths in the right side's sitemap (one index level, files
Expand Down
25 changes: 20 additions & 5 deletions src/computed.ts
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
import { textDiff } from "./compare.ts";
import { name } from "./config.ts";
import { PROPS } from "./locate.ts";
import { MAX_STATES, SNAP_PROPS } from "./locate.ts";
import type { ComparedResult, RouteResult } from "./report.ts";

/** `--computed`: every rendered element's computed style on both sides, diffed by a stable key. */

/**
* Selector path -> its opening tag with own text, `PROPS` then `content` computed, and its
* Selector path -> its opening tag with own text, `SNAP_PROPS` computed, and its
* specified width and height (`auto`, `100%`, `200px`) where the browser says.
*/
export type Snapshot = Record<string, { html: string; styles: string[]; size?: [string, string] }>;
Expand All @@ -28,6 +28,8 @@ export interface ComputedDiff {
truncated: number;
/** Elements only that side has, outermost only, at most `MAX_UNMATCHED` per side. */
unmatched: { left: string[]; right: string[] };
/** Links and controls past `MAX_STATES` on a side, not forced into `:hover`/`:focus-visible`. */
unchecked?: number;
}

/** One change, with every route and width it was seen at. */
Expand All @@ -47,7 +49,7 @@ const MAX_CHANGES = 500;
const MAX_ARIA_LINES = 200;
const ARIA_LINES_SHOWN = 10;
const MAX_UNMATCHED = 20;
const FIELDS = [...PROPS, "content"];
const FIELDS = SNAP_PROPS;

const INHERITED = new Set([
"font-family",
Expand All @@ -65,6 +67,8 @@ const DISPLAY = FIELDS.indexOf("display");

const PSEUDO = /::(before|after)$/;
const isPseudo = (key: string) => PSEUDO.test(key);
const STATE = /:(hover|focus-visible)(?= > |::|$)/;
const isState = (key: string) => STATE.test(key);

const parentOf = (key: string) => {
if (isPseudo(key)) return key.replace(PSEUDO, "");
Expand Down Expand Up @@ -121,6 +125,9 @@ export function collapseInherited<C extends { key: string; prop: string; values:
};
const tallKid = new Set([...tall].filter((k) => !follows(k)).map(parentOf));
return changes.filter((c) => {
// A state only reports what differs from resting.
if (isState(c.key) && raw.has(`${c.key.replace(STATE, "")}\0${c.prop}\0${c.values.join("\0")}`))
return false;
const p = parentOf(c.key);
if (INHERITED.has(c.prop)) return !raw.has(`${p}\0${c.prop}\0${c.values.join("\0")}`);
if (c.prop === "width") return !wide.has(p) || set(c.key, 0);
Expand Down Expand Up @@ -161,7 +168,7 @@ export function diffSnapshots(
const keys = new Set([...Object.keys(left), ...Object.keys(right)]);
let dom = 0;
for (const k of keys) {
if (isPseudo(k) || html(left, k) === html(right, k)) continue;
if (isPseudo(k) || isState(k) || html(left, k) === html(right, k)) continue;
if (!left[k] && hiddenIn(left, k)) continue;
if (!right[k] && hiddenIn(right, k)) continue;
dom++;
Expand Down Expand Up @@ -196,7 +203,9 @@ export function diffSnapshots(
}

const only = (a: Snapshot, b: Snapshot) => {
const ks = Object.keys(a).filter((k) => !isPseudo(k) && !b[k] && !hiddenIn(b, k));
const ks = Object.keys(a).filter(
(k) => !isPseudo(k) && !isState(k) && !b[k] && !hiddenIn(b, k),
);
const set = new Set(ks);
return ks.filter((k) => !set.has(parentOf(k))).slice(0, MAX_UNMATCHED);
};
Expand Down Expand Up @@ -280,6 +289,12 @@ export function fmtComputed(results: RouteResult[], routes: string[]): string[]
` truncated: only the first ${MAX_CHANGES} changes kept on ` +
cut.map((r) => `${name(r)}@${r.width} (+${r.computed!.truncated})`).join(" "),
);
const capped = computedOf(rows).filter((r) => r.computed!.unchecked);
if (capped.length)
out.push(
` states: only the first ${MAX_STATES} links and controls forced on ` +
capped.map((r) => `${name(r)}@${r.width} (+${r.computed!.unchecked})`).join(" "),
);
const unmatched = routes.flatMap((route) => {
const r = computedOf(rows).find((x) => name(x) === route && x.computed!.dom);
if (!r) return [];
Expand Down
150 changes: 131 additions & 19 deletions src/locate.ts
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,16 @@ export const PROPS = [
"height",
];

/** `--computed` reads these too; pixel mode's `changed`, and so accepted fingerprints, don't. */
export const SNAP_PROPS = [...PROPS, "outline", "text-decoration-line", "content"];

/** Links and controls forced into each state, per page per side. */
export const MAX_STATES = 500;
const STATES = [
[":hover", ["hover"]],
[":focus-visible", ["focus", "focus-visible"]],
] as const;

interface Seen {
sel: string;
tag: string;
Expand Down Expand Up @@ -110,10 +120,29 @@ interface ProbeArgs {
walk?: number;
/** Where the walk starts, keyed `:scope`; the body, keyed `body`, when null. */
root?: Node | null;
/** Of `walk`, already used by an earlier pass. */
spent?: number;
/** Keep up to this many links and controls on `globalThis.__renderdiffStates`, returning their keys. */
collect?: number;
/** Walk only the kept elements, keyed `states[i] + state`, instead of the page. */
state?: string;
states?: string[];
}

/** Runs in the page. At scroll 0 a client rect is page px, below the fold too. */
function probe({ boxes, find, masked, props, perSpot, walk, root }: ProbeArgs) {
function probe({
boxes,
find,
masked,
props,
perSpot,
walk,
root,
spent,
collect,
state,
states,
}: ProbeArgs) {
const MAX_SCAN = 50_000;
const HASHED = /\d{3,}|^(css|sc|jsx|svelte)-|__[\w-]{5,}$|^_/;
const TEST = ["data-testid", "data-test", "data-test-id", "data-cy", "data-qa"];
Expand Down Expand Up @@ -184,9 +213,18 @@ function probe({ boxes, find, masked, props, perSpot, walk, root }: ProbeArgs) {
};

// A resolved auto margin's px depends on the layout path: a grid item's reads 0px.
const styles = (e: Element, cs: CSSStyleDeclaration, m = e.computedStyleMap?.()) =>
const styles = (
e: Element,
cs: CSSStyleDeclaration,
m: StylePropertyMapReadOnly | null | undefined = e.computedStyleMap?.(),
) =>
props.map((p) =>
p.startsWith("margin-") && String(m?.get(p)) === "auto" ? "auto" : cs.getPropertyValue(p),
p.startsWith("margin-") && String(m?.get(p)) === "auto"
? "auto"
: // Unset, it reads `<currentColor> none 3px`, which would repeat every color change.
p === "outline" && cs.getPropertyValue("outline-style") === "none"
? "none"
: cs.getPropertyValue(p),
);

const seen = (e: Element) => {
Expand Down Expand Up @@ -261,7 +299,11 @@ function probe({ boxes, find, masked, props, perSpot, walk, root }: ProbeArgs) {
const snap: Record<string, { html: string; styles: string[]; size?: [string, string] }> = {};
const SKIP = new Set(["script", "style", "link", "meta", "template", "noscript"]);
let n = 0;
const end = performance.now() + walk;
const end = performance.now() + walk - (spent ?? 0);
const g = globalThis as { __renderdiffStates?: Element[] };
const kept: Element[] = [];
const keys: string[] = [];
let unchecked = 0;
// Generated class names change with the CSS, so they aren't markup changes.
const html = (e: Element) => {
const attrs = Array.from(e.attributes, (a) => {
Expand Down Expand Up @@ -293,9 +335,20 @@ function probe({ boxes, find, masked, props, perSpot, walk, root }: ProbeArgs) {
...(m ? { size: [String(m.get("width")), String(m.get("height"))] } : {}),
};
if (cs.display === "none") return;
// ponytail: skips containers like <main tabindex=-1>, which would re-walk the page per state
if (
collect &&
e.matches("a[href], button, input, select, textarea, summary, [tabindex]") &&
e.getElementsByTagName("*").length <= 50
) {
if (kept.length < collect) {
kept.push(e);
keys.push(key);
} else unchecked++;
}
for (const pseudo of ["::before", "::after"]) {
const ps = generated(e, pseudo);
if (ps) snap[key + pseudo] = { html: "", styles: props.map((p) => ps.getPropertyValue(p)) };
if (ps) snap[key + pseudo] = { html: "", styles: styles(e, ps, null) };
}
// Indexed among every child, as :nth-of-type counts, so the key works with querySelector.
const ofType = new Map<string, number>();
Expand All @@ -319,9 +372,13 @@ function probe({ boxes, find, masked, props, perSpot, walk, root }: ProbeArgs) {
visit(c, `${key} > ${count.get(g)! > 1 ? `${g}:nth-of-type(${k})` : g}`);
});
};
const start = (root as Element | null) ?? document.body;
if (start) visit(start, root ? ":scope" : "body");
return { picks: [], found: [], snap };
if (state) g.__renderdiffStates!.forEach((e, i) => visit(e, states![i]! + state));
else {
const start = (root as Element | null) ?? document.body;
if (start) visit(start, root ? ":scope" : "body");
}
if (collect) g.__renderdiffStates = kept;
return { picks: [], found: [], snap, states: keys, unchecked };
}
const els = document.body?.querySelectorAll("*") ?? [];
for (let i = 0; i < els.length && i < MAX_SCAN; i++) {
Expand Down Expand Up @@ -459,8 +516,22 @@ const moved = (a: Box, b: Box) => a.x !== b.x || a.y !== b.y || a.w !== b.w || a
const handles = async (ls: Locator[]) =>
(await Promise.all(ls.map((l) => l.elementHandles()))).flat();

const walker = async (page: Page, masks: Locator[], budget: number, root: Locator | null) => {
const args = {
root: root && (await root.elementHandle()),
boxes: [],
find: [],
masked: await handles(masks),
props: SNAP_PROPS,
perSpot: 0,
walk: budget,
};
return (more: Pick<ProbeArgs, "spent" | "collect" | "state" | "states"> = {}) =>
page.evaluate(probe, { ...args, ...more });
};

/**
* Every rendered element's computed `PROPS` and `content`, and its `::before`/`::after` when
* Every rendered element's computed `SNAP_PROPS`, and its `::before`/`::after` when
* generated, keyed by a selector path; masked and `display: none` subtrees are skipped.
* Finishes the page's animations, so it runs after `locate`.
*/
Expand All @@ -470,16 +541,57 @@ export async function snapshot(
budget: number,
root: Locator | null,
): Promise<Snapshot> {
const { snap } = await page.evaluate(probe, {
root: root && (await root.elementHandle()),
boxes: [],
find: [],
masked: await handles(masks),
props: [...PROPS, "content"],
perSpot: 0,
walk: budget,
});
return snap!;
return (await (await walker(page, masks, budget, root))()).snap!;
}

/**
* `snapshot`, plus the first `MAX_STATES` links and controls and their subtrees forced into
* each of `STATES` at once, keyed `…a[href="#x"]:hover > span`. Forcing all at once misses
* rules relating two of them, and never puts an ancestor in `:hover`. Leaves the page resting.
*/
export async function snapshotStates(
page: Page,
masks: Locator[],
budget: number,
root: Locator | null,
): Promise<{ snap: Snapshot; unchecked: number }> {
const t0 = Date.now();
const run = await walker(page, masks, budget, root);
const client = await page.context().newCDPSession(page);
try {
const { snap, states, unchecked } = await run({ collect: MAX_STATES });
await Promise.all([client.send("DOM.enable"), client.send("CSS.enable")]);
await client.send("DOM.getDocument", { depth: 0 });
const { result } = await client.send("Runtime.evaluate", {
expression: "globalThis.__renderdiffStates",
});
const { result: own } = await client.send("Runtime.getProperties", {
objectId: result.objectId!,
ownProperties: true,
});
const nodeIds = await Promise.all(
own
.filter((p) => /^\d+$/.test(p.name))
.map((p) =>
client.send("DOM.requestNode", { objectId: p.value!.objectId! }).then((r) => r.nodeId),
),
);
for (const [state, forced] of STATES) {
// A new force replaces the last; detaching clears them all.
await Promise.all(
nodeIds.map((nodeId) =>
client.send("CSS.forcePseudoState", { nodeId, forcedPseudoClasses: [...forced] }),
),
);
Object.assign(snap!, (await run({ spent: Date.now() - t0, state, states })).snap);
}
return { snap: snap!, unchecked: unchecked! };
} finally {
await client
.send("Runtime.evaluate", { expression: "delete globalThis.__renderdiffStates" })
.catch(() => {});
await client.detach().catch(() => {});
}
}

/**
Expand Down
Loading
Loading