[ivogt]

devcorner: rango-t150 · session rango-s307

Pilot: content-negotiation HTTP-only e2e to dispatch / serveShellRequest unit suites (S22)

This is the one-area pilot the owner approved for S22 (rango-d195, section 7). Hard rule 3 is unchanged by this PR; the owner decides the remaining areas from this result.

Problem statement

The content-negotiation contract (Accept ranking, definition-order fallback, the HTML document versus the flight stream, variant middleware, Vary: Accept) was pinned by 79 e2e tests across two apps, every one of them an HTTP request with no page (no page. or goto( in either file): 39 in the router test-app (23 dev + 16 production) and 40 in cloudflare-basic (24 dev + 16 production). Since S14/S14b, dispatch() runs the production request stages, and serveShellRequest runs createRSCHandler itself, so the same contract can be pinned in unit suites through the public testing primitives, keeping e2e for what only a real build and server can show.

Before / after (test shape)

Before, one e2e test per Accept case, written twice (dev and production):

test("Accept: application/json,text/html prefers JSON (listed first)", async ({ request }) => {
  const res = await request.get(f.url("/negotiate-test"), {
    headers: { Accept: "application/json, text/html" },
  });
  expect(res.headers()["content-type"]).toContain("application/json");
});

After, the case is a unit row through the public primitive, with the exact content type, the body, and the Vary token checked on every negotiated response:

const res = await dispatch(router, {
  request: new Request("http://localhost/negotiate-test", {
    headers: { accept: "application/json, text/html" },
  }),
});
expect(res.headers.get("content-type")).toBe("application/json;charset=utf-8");
expectVaryAcceptOnce(res);

An Accept the page route wins cannot go through dispatch (it refuses component routes by design), so those rows go through serveShellRequest, which also decides the representation:

const result = await serveShellRequest(router, "/negotiate-test", { headers: { accept: "*/*" } });
expect(result.response.headers.get("content-type")).toBe("text/html;charset=utf-8");

Code-change summary

• packages/rangojs-router/src/testing/__tests__/content-negotiation.rsc-test.tsx (new, 22 tests, rsc project): a self-contained router with the test-app's negotiated URL shapes (/negotiate-test: RSC, JSON, Markdown; /negotiate-test-json-first: JSON, RSC), /negotiate-mw-test (a middleware per variant) and a plain route. The test-app has no unit harness, and importing e2e/test-app/src from the router's src/ would cross the package's typecheck boundary, so the router suite builds its own routes, as the router's other suites do.
• tests/cloudflare-basic/test/content-negotiation.rsc-test.tsx (new, 17 tests, rsc project): the app's own negotiated routes.
• tests/cloudflare-basic/src/pages/negotiation.tsx (new) + negotiation.gen.ts: the app's negotiated routes move out of src/urls.tsx into negotiationPatterns, mounted with include("/", negotiationPatterns, { name: "" }) (the holeyThrowsPatterns shape), so the unit suite imports the real routes. Route names are unchanged; router.named-routes.gen.ts did not change after a dev run and a build.
• packages/rangojs-router/e2e/content-negotiation.test.ts (539 to 64 lines) and tests/cloudflare-basic/e2e/content-negotiation.test.ts (457 to 63 lines): one smoke each, in a dev and a (production) describe, through one negotiationSmoke(f: Fixture) per file. The smoke retries a cold dev route's first request (toPass, 10 s), as the removed suite did for one case.
• packages/rangojs-router/e2e/test-app/src/urls.tsx: /negotiate-mw-test removed (only the removed e2e cases used it); urls.gen.ts and router.named-routes.gen.ts regenerated.
• packages/rangojs-router/skills/testing/response-routes.md: dispatch is scoped to negotiation won by a response route, and a new caveat says to serve a page-route win through serveShellRequest.

Lines: e2e -869 net (996 to 127), unit +446, cloudflare-basic app +21 net (negotiation.tsx 58 and negotiation.gen.ts 16 new; urls.tsx +3 -56), test-app -33.

What stays in e2e, and why

App	Kept smoke (dev + production)	Why a unit suite cannot pin it
router test-app	each route of a negotiated URL answers through the route trie: /negotiate-test with */* is the SSR document containing "Negotiate Test RSC"; /negotiate-test with text/markdown is the third route's Markdown; /negotiate-test-json-first with */* is { source: "json" }; accept once in Vary on all three	Production matches through the route trie the build emits: each leaf's negotiate variants (their route keys, by which loadManifest loads the winner) and its rf page-route-first flag. dispatch and serveShellRequest rebuild the trie from the routes at runtime (ensureRouterTrie), so a build that dropped or reordered variants is invisible to them. The SSR step is stubbed in serveShellRequest (the body is the Flight payload), so the real HTML document is e2e-only. Variant middleware is collected at runtime from the entry the variant's route key loads (collectRouteMiddleware in router/content-negotiation.ts), so the unit suite pins it (mutation M6).
cloudflare-basic	a negotiated URL serves each pipeline through workerd: /test/negotiate with */* is the bare JSON, with a browser Accept the SSR document containing "HTML version"; /test/negotiate-wild/deep/path with application/json keeps its wildcard param; accept once in Vary on all three	The same built trie (a wildcard leaf with a variant included), served by workerd (miniflare in dev, the built worker in production), with the real SSR document, and no second Accept added in the worker.

No removed test depended on NODE_ENV folding: each production test asserted the same contract as its dev twin. Both rsc unit projects run with NODE_ENV=production (vitest.rsc.config.ts).

Mapping: removed e2e test to the unit test that pins it

Router test-app (packages/rangojs-router/e2e/content-negotiation.test.ts). Unit tests are in the router's content-negotiation.rsc-test.tsx: describes "a response variant wins (dispatch)" (V), "the RSC route wins (serveShellRequest)" (R), "a plain RSC route (no variants)" (P).

Removed e2e test	Mode	Unit test
Accept: text/markdown,text/html,*/* prefers markdown (listed first)	dev + production	V client order: markdown listed first
Accept: text/html,text/markdown,*/* prefers HTML (listed first)	dev	R client order: HTML listed first
Accept: application/json,text/html prefers JSON (listed first)	dev + production	V client order: JSON listed first
text/markdown;q=0.5 loses to text/html;q=1.0	dev	R q-value: text/markdown;q=0.5 loses to text/html
text/html;q=0.9 loses to text/markdown;q=1.0	dev	V q-value: text/html;q=0.9 loses to text/markdown
application/json;q=0.8 loses to text/markdown;q=0.9	dev	V q-value: application/json;q=0.8 loses to text/markdown;q=0.9
browser Accept header returns HTML (text/html is q=1.0, first)	dev	R a browser Accept
Accept: text/markdown returns markdown	dev	V single type: text/markdown; also the kept smoke
Accept: application/json returns JSON	dev	V single type: application/json
Accept: */* serves the HTML document when RSC is defined first	dev + production	R */* picks the RSC route defined first; also the kept smoke
Accept: */* returns JSON when JSON is defined first	dev + production	V */* picks the JSON route defined first; also the kept smoke
no Accept header serves the HTML document when RSC is defined first	dev + production	R an empty Accept picks the RSC route defined first
no Accept header returns first defined (JSON)	dev + production	V an empty Accept picks the JSON route defined first, V no Accept picks the JSON route defined first
Accept: text/x-component returns the flight stream (RSC first)	dev + production	R text/x-component on an RSC-first route
Accept: text/x-component returns the flight stream (JSON first)	dev + production	R text/x-component on a JSON-first route
plain RSC route Accept: */* serves the HTML document	dev + production	P */*
plain RSC route no Accept header serves the HTML document	dev + production	P an empty Accept
plain RSC route Accept: application/json serves the HTML document (no JSON variant)	dev + production	P application/json (no JSON variant)
plain RSC route Accept: text/x-component serves the flight stream	dev + production	P text/x-component
all negotiated responses include Vary: Accept	dev + production	every V and R row (expectVaryAcceptOnce)
emits Accept in Vary exactly once (no duplicate token)	dev + production	every V and R row, the variant-middleware route included; also the kept smoke
Accept: application/json runs JSON variant middleware	dev + production	V runs the winning variant's own middleware
Accept: text/html runs RSC variant middleware	dev + production	R runs the RSC route's middleware when the document wins

cloudflare-basic (tests/cloudflare-basic/e2e/content-negotiation.test.ts). Unit tests are in cloudflare-basic's test/content-negotiation.rsc-test.tsx (V, R) unless marked "router". A production title is the dev title's case with "in production".

Removed e2e test (dev title)	Mode	Unit test
Accept: text/html returns HTML (RSC pipeline)	dev + production	R JSON-first: a browser Accept; also the kept smoke
Accept: application/json returns bare JSON	dev + production	V JSON-first: application/json
no Accept header returns JSON (negotiate handler)	dev	V JSON-first: an empty Accept picks JSON, V JSON-first: no Accept picks JSON
Accept: */* returns JSON (no explicit text/html)	dev	V JSON-first: */* picks JSON; also the kept smoke
Accept: text/x-component returns the flight stream on a JSON-first route	dev + production	R JSON-first: text/x-component
plain RSC route: Accept: */* serves the HTML document	dev + production	router P */* (framework behaviour with no app route involved)
plain RSC route: Accept: text/x-component serves the flight stream	dev + production	router P text/x-component (same)
Vary: Accept present on negotiated JSON response	dev + production	V JSON-first: application/json; also the kept smoke
Vary: Accept present on HTML response from negotiated URL	dev	R JSON-first: a browser Accept; also the kept smoke
text negotiate: Accept text/html returns HTML	dev + production	R text: a browser Accept
text negotiate: no text/html returns plain text	dev + production	V text: text/plain
text negotiate: Vary: Accept present	dev	V text: text/plain
xml negotiate: Accept text/html returns HTML	dev + production	R xml: a browser Accept
xml negotiate: no text/html returns XML	dev + production	V xml: application/xml
xml negotiate: Vary: Accept present	dev	V xml: application/xml
multi negotiate: text/html returns HTML	dev + production	R multi: a browser Accept
multi negotiate: application/json returns JSON	dev + production	V multi: application/json
multi negotiate: text/plain returns text	dev + production	V multi: text/plain
multi negotiate: application/xml returns XML	dev + production	V multi: application/xml
multi negotiate: */* returns first variant (json)	dev	V multi: */* picks the first variant (JSON)
wildcard negotiate: text/html returns HTML	dev + production	R wildcard: a browser Accept
wildcard negotiate: application/json returns JSON with wildcard param	dev + production	V wildcard: application/json carries the wildcard param; also the kept smoke
wildcard negotiate: Vary: Accept present	dev	V wildcard: application/json carries the wildcard param
multi negotiate: Vary: Accept on all negotiated responses	dev	V multi: application/json, multi: text/plain, multi: application/xml

The unit rows assert more than the e2e tests did: the exact content type (the cloudflare-basic HTML cases only asserted "not JSON"), the body, the document or flight representation of each page-route win, and accept exactly once in Vary on every negotiated response (cloudflare-basic only asserted "contains").

Red before green: mutation matrix

Each mutation was applied alone to the negotiation code, both new suites were run, and the mutation was reverted (the source files show no diff afterwards). All 39 unit tests pass on the unmodified code, and every one of them fails under at least one mutation:

Mutation	Code	Tests red
M1	pickNegotiateVariant returns the first candidate (ignores Accept)	16
M2	pickNegotiateVariant returns the last candidate	23
M3	acceptsFlightExplicitly back to the old rule (no text/html in Accept means flight)	5
M4	prefersFlightRepresentation returns false	4
M5	appendVaryAccept appends without the dedup (D6)	24
M6	a winning variant gets the primary route's middleware	1

Per test

Test	Red under
router V client order: markdown listed first	M1, M5
router V client order: JSON listed first	M1, M2, M5
router V q-value: text/html;q=0.9 loses to text/markdown	M1, M5
router V q-value: application/json;q=0.8 loses to text/markdown;q=0.9	M1, M5
router V single type: text/markdown	M1, M5
router V single type: application/json	M1, M2, M5
router V */* picks the JSON route defined first	M2, M5
router V an empty Accept picks the JSON route defined first	M2, M5
router V no Accept picks the JSON route defined first	M2, M5
router V runs the winning variant's own middleware	M1, M5, M6
router R client order: HTML listed first	M2
router R q-value: text/markdown;q=0.5 loses to text/html	M2
router R a browser Accept	M2
router R */* picks the RSC route defined first	M2, M3
router R an empty Accept picks the RSC route defined first	M2, M3
router R text/x-component on an RSC-first route	M2, M4, M5
router R text/x-component on a JSON-first route	M1, M4, M5
router R runs the RSC route's middleware when the document wins	M2
router P */*	M3
router P an empty Accept	M3
router P application/json (no JSON variant)	M3
router P text/x-component	M4
cloudflare-basic V JSON-first: application/json	M2, M5
cloudflare-basic V JSON-first: */* picks JSON	M2, M5
cloudflare-basic V JSON-first: an empty Accept picks JSON	M2, M5
cloudflare-basic V JSON-first: no Accept picks JSON	M2, M5
cloudflare-basic V text: text/plain	M1, M5
cloudflare-basic V xml: application/xml	M1, M5
cloudflare-basic V multi: application/json	M2, M5
cloudflare-basic V multi: text/plain	M1, M2, M5
cloudflare-basic V multi: application/xml	M1, M2, M5
cloudflare-basic V multi: */* picks the first variant (JSON)	M2, M5
cloudflare-basic V wildcard: application/json carries the wildcard param	M2, M5
cloudflare-basic R JSON-first: a browser Accept	M1
cloudflare-basic R JSON-first: text/x-component	M1, M4, M5
cloudflare-basic R text: a browser Accept	M2
cloudflare-basic R xml: a browser Accept	M2
cloudflare-basic R multi: a browser Accept	M1
cloudflare-basic R wildcard: a browser Accept	M1

Timing (local, this machine)

Suite	Tests	Run wall-clock	Sum of test durations
router e2e before (dev + production)	39	15.3 s	2.46 s
router e2e after, final smoke (dev + production)	2	23.3 s	1.57 s (dev 1.3 s, production 0.27 s)
cloudflare-basic e2e before (dev + production)	40	55.3 s	4.43 s
cloudflare-basic e2e after, final smoke (dev + production)	2	28.7 s	4.00 s (dev 3.8 s, production 0.20 s)
router unit (new)	22	0.9-2.9 s per file run	0.2-0.5 s
cloudflare-basic unit (new)	17	0.9-3.2 s per file run	0.2-0.6 s

Each e2e run's wall-clock is almost all server start and build (a dev server, a production build and a production server per app), which a full e2e run pays once for all suites; across the runs here it varied from 15 to 44 s (router) and 29 to 56 s (cloudflare-basic) for the same files, so the before/after wall-clock difference is noise. The removed tests' own cost was small: 4-50 ms per request once the servers were warm, 6.9 s summed across both apps. The kept smokes sum to 5.6 s in the final runs (router 1.3-2.3 s and cloudflare-basic 2.2-4.3 s across the runs with no other e2e run alongside), almost all of it the dev smoke's first request paying the cold route compile that the 47 dev tests used to share. So this area saves almost no local e2e time. CI e2e job time before and after: to be read from this PR's CI run against the base run; the results will be added here.

Test plan

• New unit suites: router content-negotiation.rsc-test.tsx (22 passed) and cloudflare-basic content-negotiation.rsc-test.tsx (17 passed); each test red under a mutation (matrix above).
• Kept smokes on the final commit, port offset 241: router content-negotiation and content-negotiation (production) 2/2; cloudflare-basic content negotiation (dev) and content negotiation (production) 2/2.
• Removed suites passed on the base before removal: router 39/39, cloudflare-basic 40/40.
• pnpm run typecheck: 16/16 tasks.
• pnpm run test:unit:all: router 9354 passed (5 skipped, 6 todo) and rsc 1089 passed (3 expected fail); cloudflare-basic 90 and rsc 56; vite-rsc-demo 27 and 1; mini 25 and 3; shared-e2e 20. No failures.
• oxlint --deny-warnings (0 warnings, 0 errors), oxfmt --check ..
• CI lint scripts: check-e2e-bucketing, check-e2e-parity --strict (it reports this file as a gap when the (production) describe is removed), check-e2e-console-baseline, check-suspense-contract, check-no-external-store, node --test tools/*.test.mjs (53 passed), check-doc-links, check-docs-api, check-skills, generate-skills-catalog --check, generate-route-types --check, check-internals.
• git status --porcelain | grep '\.gen\.ts' empty; negotiation.gen.ts, test-app urls.gen.ts and router.named-routes.gen.ts committed.

Practices

• Lint: oxlint --deny-warnings 0 findings and oxfmt --check . clean on the change; the pre-commit hooks (format, lint, e2e bucketing, route-gen, no-external-store) passed on each commit.
• Dead code: the test-app's /negotiate-mw-test routes were used only by the removed e2e cases, so they are removed and the generated route files regenerated; a repo-wide search (src, tests, e2e, docs, skills, tools) finds no reference left. Kept on purpose: the test-app's /negotiate-test JSON and Markdown routes and /negotiate-test-json-first (the smoke requests them), and cloudflare-basic's negotiated routes (its unit suite and smoke use them).
• Tests: the 39 new unit tests through dispatch and serveShellRequest, each shown red under a mutation; the kept smokes in dev and production in both apps; test:unit:all and typecheck. The full e2e suite runs on CI.
• Code review: the diff was reviewed for correctness and for reuse, simplification, efficiency and depth. Fixed: the smokes also cover a non-first variant's route key and a wildcard leaf, assert accept once in Vary on workerd, and retry a cold dev route's first request; the middleware rows check status and content type, and the page-route one is titled by what it checks; the smokes take the shared Fixture type; both unit suites use one row shape; cloudflare-basic's suite covers only the app's own routes; the skill note is scoped. Left, with reasons: production e2e for every Accept case (the pilot's premise; the build-only data and SSR are in the smokes, and the rest is runtime code the unit suites run); one serveShellRequest table for every row (dispatch is the HTTP-only primitive this pilot evaluates, and the split pins which pipeline answers); the router suite's own routes instead of importing the test-app (typecheck boundary, above); expectVaryAcceptOnce in four files (Playwright APIResponse and fetch Response differ, and a shared matcher would be new public testing API); the toPass retry also wrapping the production smoke (it only delays a real failure's report by up to 10 s); unit rows that overlap the internal unit tests of content-negotiation.ts, ssr-setup.ts and response-utils.ts (the userland rule asks for the public primitives on top of them).
• Docs: skills/testing/response-routes.md now scopes dispatch to negotiation won by a response route and says to serve a page-route win through serveShellRequest. No router source file was added or removed, so feature-map.md and feature-file-map.md are unchanged.

Notes

• Findings for the owner's decision on the remaining areas:
  • dispatch cannot answer a page-route win (it throws on component routes by design), so 18 of the 39 negotiation rows go through serveShellRequest and the rsc (react-server) unit project. Areas whose cases end in a component render will lean on serveShellRequest the same way; areas that end in a response route (response routes, redirects) fit dispatch alone.
  • The time saved is small per area: HTTP-only e2e requests are already cheap once the servers are up, and the servers stay for the smokes and the page-driving suites. The gains are 75 fewer e2e tests (79 to 4), 869 fewer e2e lines, sub-second unit feedback while editing, and stricter assertions.
  • What the unit suites cannot see is the build: the emitted route trie and the real SSR document (and workerd). Each area's smoke should target those.
• cloudflare-basic's two plain-route e2e cases map to the router suite's plain-route rows: they pin framework behaviour with no app route involved.

[pkg-pr-new]

Open in StackBlitz

npm i https://pkg.pr.new/@rangojs/router@1255

commit: cee1c01