Skip to content

Add --aria: diff ARIA snapshots per route - #93

Merged
tmashara merged 2 commits into
mainfrom
claude/issue-45-aria
Oct 29, 2026
Merged

tmashara merged 2 commits into
mainfrom
claude/issue-45-aria

Conversation

@tmashara

@tmashara tmashara commented Oct 29, 2026 •

Copy link
Copy Markdown
Owner

What

--aria (config aria: true): after each shot, renderdiff takes Playwright's ARIA snapshot (locator.ariaSnapshot()) of body or the --selector element on both sides and diffs it line by line, reusing textDiff from compare.ts.

Why

A markup migration can keep every pixel and still break the page for a screen reader. The new button-vs-div fixture shows it: a <div> styled like the <button> it replaced scores 100% at every width, and only the accessibility tree shows the lost role.

Output

aria: identical on 2 of 3 route(s), differs /button-vs-div
  /button-vs-div@1440,375
    +- text: Buy now
    -- button "Buy now"
  • + is left only (your change), - is right only, the same convention as text.
  • The same diff at several widths shows up once. A diff that only appears at mobile width gets its own group.
  • Each group shows 10 lines and the summary shows 20 groups. results.json keeps up to 200 lines per side, and truncated counts the rest.
  • Per shot in results.json: aria: { added, removed, truncated }, where both lists empty means identical. When the snapshot fails it is aria: null with ariaError. There is no summary.aria.
  • --progress rows show aria: N added, M removed. Report cards show aria identical or aria differs (+a −r), and the summary has a collapsed ARIA: block.
  • Asset hashes in /url: values are normalized the same way as --computed (normalizeAssets's rewrite, split out as normalizeUrl). The left origin is rewritten to the right's (rebase's regex, split out as rebaseText).
  • Pixels still decide the score and the exit code.

Masked elements are excluded

ariaSnapshot has no exclude option, so each mask's elements get aria-hidden="true" right before the snapshot. That happens after all shot work for the job: the shot, the recheck, locate and --computed. Nothing restores the attribute, and nothing needs to: every next job reloads the page with goto, and so do the retry and error paths.

Limits

  • Lines keep their indentation, so depth still carries information. The cost: when a wrapper role is added or removed (main swapped for a div), every line under it re-indents and shows up as changed. This is marked with a ponytail: comment. The upgrade path is to key lines by role path.
  • Lists are capped at 200 lines per side, with truncated counting what was cut.
  • It does not descend into iframes. The snapshot shares the --computed time budget (15s, raced at 20s).

How tested

  • New equivalence fixture tests/fixtures/pages/button-vs-div. It is pixel-identical at all six equivalence widths on macOS; Linux CI will confirm.
  • New visual test in tests/visual/controls.test.ts runs /button-vs-div, /origin and /margin with --aria. Exit code is 1 because /margin differs. /button-vs-div scores 1 and shows exactly the lost-button lines at both widths, once in stdout. /origin, which differs only in port, and /margin, which is CSS only, come out identical.
    • Without the src change it fails (unknown flag: --aria). With the change it passes.
  • A second visual test runs /button-vs-div with --mask .buy, and '/button-vs-div @.notes'. Both come out identical. It fails if the aria-hidden marking is removed, and it fails if the snapshot reads body instead of the route's selector.
  • New unit tests in tests/unit/computed.test.ts cover hash and origin normalization, the cap and truncated, and how fmtAria groups and splits diffs across widths.
  • The config hash is unchanged without --aria: ResolvedConfig.aria is true | undefined, so an unset or false value drops out of the hashed JSON.
  • npm run typecheck, lint, format:check, npm test (348 pass) and npm run test:visual (379 pass), rebased on main after Per-route selector in the route string: '/pricing @section.cta' #91 all pass locally.

Fixes #45

🤖 Generated with Claude Code

tmashara and others added 2 commits October 29, 2026 20:32
Playwright's ARIA snapshot of both sides, diffed line by line after the
shot, with asset hashes and the left origin normalized and masked elements
left out. Pixels still decide the score and exit code.

Fixes #45

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@tmashara
tmashara force-pushed the claude/issue-45-aria branch from 9175d1a to 5433ffb Compare October 29, 2026 19:39
@tmashara
tmashara merged commit dcd6401 into main Oct 29, 2026
3 checks passed
@tmashara
tmashara deleted the claude/issue-45-aria branch October 29, 2026 19:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

--aria: diff ARIA snapshots per route

1 participant