Skip to content

test(e2e): run the Node suite against an astro build --devOutput build - #4518

Open
ostenbom wants to merge 4 commits into
emdash-cms:mainfrom
endformdev:e2e/dev-output-build
Open

ostenbom wants to merge 4 commits into
emdash-cms:mainfrom
endformdev:e2e/dev-output-build

Conversation

@ostenbom

@ostenbom ostenbom commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

What does this PR do?

Most of the Node E2E time goes to astro dev compiling pages on first request. This PR runs the Node shards against a built fixture instead, using astro build --devOutput. This is a simpler way of retaining existing methods of "dev only" functionality while providing a small speed boost.

It's opt-in: EMDASH_E2E_BUILD=production switches e2e/global-setup.ts to build and serve the fixture, and CI sets it only for the Node step. The Cloudflare step and the 8-shard layout are unchanged. The rest is what the fixture needs to work as a built site: manual i18n routing, a sharp dependency, the registry fixture installed from middleware, and an https: image URL in one test. There is also one small core fix. The build-time registry check rejected the suite's localhost mock marketplace for --devOutput builds, even though the built site accepts it at runtime.

Node E2E on CI (8 shards) Slowest shard Total across shards
astro dev (main) 7.1 min 35.4 min
astro build --devOutput (this PR) 5.5 min (−23%) 24.1 min (−32%)

A real production build is a lot faster (2.8 min), but it needs a new config option; that's #4519. For a full local run you also need the test fixes in #4517. I also tried this build on Endform, which runs the whole suite in one job against a single server (endformdev#8). Under that load the dev-output server times out, and the production build doesn't.

Honestly I think #7 is better - long term this will provide a more stable suite, if you accept the tradeoffs.

I used claude to help me create this PR, but I have run the suite and reviewed the code myself.

Type of change

  • Bug fix
  • Feature (requires an accepted feature plan or RFC)
  • Refactor (no behavior change)
  • Translation
  • Documentation
  • Performance improvement
  • Tests
  • Chore (dependencies, CI, tooling)

Your connection to this change

  • I use EmDash and ran into the problem this PR addresses
  • I maintain a tool, dependency, or service that EmDash uses, and this change concerns that integration
  • A Maintainer approved this specific change before I opened the PR

Checklist

  • I have read CONTRIBUTING.md
  • pnpm typecheck passes
  • pnpm lint passes
  • pnpm test passes (or targeted tests for my change): full Node E2E suite against the dev-output build (344 passed, 34 skipped)
  • pnpm format has been run
  • I have added/updated tests for my changes (if applicable)
  • User-visible strings in the admin UI are wrapped for translation (if applicable): not applicable, no admin UI change
  • I have added and reviewed the user-facing changeset (if this PR changes a published package)
  • New features link to their merged design PR: not applicable
  • I have included screenshots below if this PR changes the UI: not applicable

AI assistance disclosure

  • This PR includes AI-generated code or text — model/tool: Claude Opus 5.5 (Claude Code)

Screenshots / test output

Not applicable.

ostenbom and others added 4 commits October 27, 2026 13:45
EMDASH_E2E_BUILD=production now builds the Node fixture with
`astro build --devOutput` and serves dist/server/entry.mjs instead of
running `astro dev`. The build keeps `import.meta.env.DEV` true, so the
dev-bypass, dev-reset and captured-email endpoints the suite depends on
work without any test switch in EmDash.

Also fixes what a built fixture exposes: manual i18n routing so the
fixture's [locale] pages win over Astro's fallback routes, sharp for
the image endpoint, the registry fixture installed at server start, an
https image URL for the CSP, and an ignore rule for the migrations
journal that builds write into the fixture.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
The build-time registry check only allowed localhost for `astro dev`
and `astro sync`, while the runtime check follows import.meta.env.DEV.
A --devOutput build therefore failed at build time although the built
site would accept the URL. Astro sets NODE_ENV=development for
--devOutput before loading the config, so the build check uses that.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@changeset-bot

changeset-bot Bot commented Oct 29, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 837c119

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 12 packages
Name Type
emdash Patch
@emdash-cms/cloudflare Patch
@emdash-cms/plugin-test Patch
@emdash-cms/sandbox-workerd Patch
@emdash-cms/admin Patch
@emdash-cms/auth Patch
@emdash-cms/blocks Patch
create-emdash Patch
@emdash-cms/gutenberg-to-portable-text Patch
@emdash-cms/x402 Patch
@emdash-cms/auth-atproto Patch
@emdash-cms/plugin-embeds Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions github-actions Bot added the review/needs-review No maintainer or bot review yet label Oct 29, 2026
@github-actions

github-actions Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@emdashbot emdashbot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The opt-in built-fixture approach is a reasonable way to speed up the Node E2E shards. It keeps the dev-only auth bypass working through astro build --devOutput, and it leaves Cloudflare and the dev-server path unchanged. Both reviewers found the fixture, setup/teardown and changeset plumbing sound, and neither ran anything, so the timing numbers and suite results are unverified.

Both reviewers flagged the same line in core, packages/core/src/astro/integration/index.ts:638, as needing a fix. The change loosens the build-time registry URL policy so that HTTP and localhost aggregators are accepted when NODE_ENV === "development". That check has two problems:

  • It uses process.env.NODE_ENV, which the repository's AGENTS.md forbids (raised by one reviewer).
  • No unit test guards the new rule or the changeset's promise that production builds still reject these URLs (raised by both).

The E2E run exercises only the accepting side, and only indirectly.


Reviewed by anthropic/claude-opus-5-5 and openai/gpt-6.1-sol.

// `astro build --devOutput` sets NODE_ENV before loading the config and
// produces a build whose runtime registry check allows localhost.
allowLocalhost:
command === "dev" || command === "sync" || process.env.NODE_ENV === "development",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[needs fixing] This line changes the build-time registry URL policy. HTTP and localhost aggregators are now accepted when process.env.NODE_ENV === "development" during any command, not only dev and sync. Two problems:

  1. Convention (raised by one reviewer, not the other): AGENTS.md requires development/production decisions to use Astro/Vite environment semantics and says never to use process.env.NODE_ENV. The runtime validator in registry/config.ts uses import.meta.env.DEV, so this adds a second source of truth for the same policy. Determine development-output status through the appropriate Astro/Vite configuration boundary, or document an explicit config-time exception if the hook cannot access that signal. The other reviewer found the build-time and runtime checks agree in practice, since Astro defaults NODE_ENV only when it is unset and DEV follows it, but did not address the AGENTS.md rule.

  2. No test coverage (raised by both reviewers): tests/unit/astro/integration/ already calls the astro:config:setup hook directly with command: "build" (see image-remote-patterns.test.ts), so a test is cheap to add. The changeset promises "Production builds still reject them", and nothing guards that. Add cases with registry: "http://localhost:1234" and command: "build":

    • NODE_ENV unset or "production": expect REGISTRY_AGGREGATOR_URL_FORBIDDEN.
    • NODE_ENV = "development" (the --devOutput case): expect the hook to succeed.

    Restore process.env.NODE_ENV after each case.

@ostenbom

Copy link
Copy Markdown
Contributor Author

I have read the CLA Document and I hereby sign the CLA

@github-actions

Copy link
Copy Markdown
Contributor

Approval status

The current revision does not meet the approval requirements yet.

  • ❌ 2 human Maintainer approvals from someone other than the author (governance or privileged supply-chain change); found 0.
  • ✅ No outstanding change requests.

This comment updates as reviews arrive. See MAINTAINING.md and the proposal process for the rules.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant