diff --git a/.agents/skills/deco-v7-to-v8-migration/SKILL.md b/.agents/skills/deco-v7-to-v8-migration/SKILL.md index d085a487..6d3c34fb 100644 --- a/.agents/skills/deco-v7-to-v8-migration/SKILL.md +++ b/.agents/skills/deco-v7-to-v8-migration/SKILL.md @@ -5,7 +5,7 @@ description: Moves a v7 Deco site (@decocms/blocks 7.x with @decocms/tanstack or # Deco v7 → v8 Migration -Moves a v7 site onto the next major in two parts: a script that does the mechanical, content-safe part in one pass, and a list of manual steps it prints. Proven on `deco-sites/storefront-tanstack` (Shopify, TanStack Start), `deco-sites/blog-tanstack` (TanStack Start), a Next.js App Router storefront on VTEX, a TanStack Start storefront on VTEX and a non-ejected FastStore storefront on VTEX (hundreds of saved blocks, private). +Moves a v7 site onto the next major in two parts: a script that does the mechanical, content-safe part in one pass, and a list of manual steps it prints. Proven on `deco-sites/storefront-tanstack` (Shopify, TanStack Start), `deco-sites/blog-tanstack` (TanStack Start), two Next.js App Router storefronts on VTEX (ejected FastStore), a TanStack Start storefront on VTEX and a non-ejected FastStore storefront on VTEX (hundreds of saved blocks, private). The spec is the docs page **Migrating from v7** (`/next/renames-and-migrations`). When this skill and the docs disagree, the docs win. @@ -69,12 +69,12 @@ The script prints **Done** and **Left to do**, grouped by step. Every "Left to d ## Manual steps (what the report leaves) -1. **Dependencies.** Depend on `@decocms/blocks@^8.1` and the v8 `@decocms/apps-` client. Remove `@decocms/tanstack`/`@decocms/nextjs`, `@decocms/blocks-admin`, `@decocms/blocks-cli`, `@decocms/apps-commerce`, `@decocms/apps-website` and `@decocms/apps-blog` once nothing imports them, and drop the v7 codegen from `build`. On Next.js, add `@decocms/blocks` to `transpilePackages` in `next.config`: the `8.1.0-next.*` prereleases publish TypeScript source (`exports` → `src/*.ts`), v7's `withDeco` wrapper used to add it, and without it a clean install fails `next build` with `Module parse failed: Unexpected token`. +1. **Dependencies.** Depend on `@decocms/blocks@^8.1` and the v8 `@decocms/apps-` client. Remove `@decocms/tanstack`/`@decocms/nextjs`, `@decocms/blocks-admin`, `@decocms/blocks-cli`, `@decocms/apps-commerce`, `@decocms/apps-website` and `@decocms/apps-blog` once nothing imports them, and drop the v7 codegen from `build`. On Next.js, check what the installed `@decocms/blocks` publishes: when its `exports` default to `dist/*.js` (`8.1.0-next.7` does), it needs no `transpilePackages` entry; a prerelease whose `exports` point at `src/*.ts` needs `@decocms/blocks` in `transpilePackages` (v7's `withDeco` wrapper used to add it), or a clean install fails `next build` with `Module parse failed: Unexpected token`. 2. **Render pages with `createCMS`.** Follow the framework guide (`/next/tanstack-start-descriptors`, `/next/nextjs`): `createCMS` over the content, `matchRoute`, one promise per block, a view registry. Delete the v7 setup files, admin routes and `/deco/*` handlers. 3. **Move framework code into the site** (`reference/gotchas.md`): edge cache, image, SEO/head, device detection, cookies, cart/user/wishlist flows, commerce loaders and converters. 4. **Replace `/deco/invoke`** with server functions (TanStack `createServerFn`) or Next server actions/route handlers. One exported server function per loader or action the browser called, and each call site imports the one it calls: `invoke.site.loaders.spin(props)` becomes `$siteLoadersSpin({ data: props })`. Don't rebuild the `invoke.x.y` tree or a string-keyed dispatcher over the commerce-loader map (see DO NOT); the server function calls its handler directly. Details in `reference/gotchas.md` (`/deco/invoke` → server functions). 5. **Fix the content `deco check` rejects**: fields no type declares, `.tsx`-named preview blocks, v7 app blocks (apps are code now), `requestToParam` blocks in string fields. -6. **Telemetry, analytics and previews**: the `telemetry` option of `createCMS` (`/next/telemetry`), with switches and rates in the `telemetry` section of `CMS.json`; `AnalyticsScript` with `const { analytics } = await cms.settings()` and `track` from `@decocms/blocks/analytics`; drafts through `await cms.draftPointer(request)` / `cms.draftCookie(request)`, which serve the release on hosts outside `preview.hosts` (list the dev host, `localhost`, too: v8 ignores `DECO_ALLOWED_PREVIEW_HOSTS`) (with no `previewHosts`, v7 allowed previews only on a TanStack site's `.deco.site` and `.deco-cx.workers.dev`, or nowhere; here every host may preview unless you list some, `/next/releases-and-drafts#allow-previews-per-host`). The report carries GTM/GA4 IDs over). +6. **Telemetry, analytics and previews**: the `telemetry` option of `createCMS` (`/next/telemetry`), with switches and rates in the `telemetry` section of `CMS.json`; `AnalyticsScript` with `const { analytics } = await cms.settings()` and `track` from `@decocms/blocks/analytics`; drafts through `await cms.draftPointer(request)` / `cms.draftCookie(request)`, which serve the release on hosts outside `preview.hosts` (list the dev server's host with its port too, e.g. `localhost:3000` for `next dev`, in both the code cap and `CMS.json`, and name each host you add for the product owner: v8 ignores `DECO_ALLOWED_PREVIEW_HOSTS`) (with no `previewHosts`, v7 allowed previews only on a TanStack site's `.deco.site` and `.deco-cx.workers.dev`, or nowhere; here every host may preview unless you list some, `/next/releases-and-drafts#allow-previews-per-host`). The report carries GTM/GA4 IDs over). If v7 sent no CMS telemetry, pointing `telemetry` at the site's existing collector is new traffic and cost there: list it as a behaviour change. ## Verify @@ -91,6 +91,8 @@ Run it once more **without any local link** (a `decocms/blocks` checkout linked **Probe drafts by hand** on the dev server and on a production build: `?__draft=` on an allowed host renders the draft and sets the cookie, the cookie alone keeps the next page in the draft, a host outside `preview.hosts` and a malformed pointer get the release, `?__draft=off` clears the cookie. A parity harness can't catch a dead draft path when v7's was dead too: both sides render the release. +Grep the tree for v7 tooling in comments, docs and `.gitattributes` (`blocks-cli`, `meta.gen.json`, `sections.gen.ts`, `generate-schema`, `ts-morph`) and reword each to `deco schema` or the block map. Check that the claim still holds under `deco schema` (`/next/schema`) instead of renaming the tool: some v7 quirks are gone, and the comment should then say it was v7. + Then compare the migrated site with the v7 one page by page (a parity harness: SSR HTML, JSON-LD, analytics calls, cache headers, third-party requests). Compare the editor forms as well (v7 `meta.gen.json` vs v8 `schema.gen.json`): many differences are stale v7 files or v7 heuristics v8 drops on purpose, a few are CLI bugs to fix. Encode every difference the product owner approves as an explicit rule, never a blanket ignore. Keep explained-but-unapproved ones as `pending`, and have a strict compare fail on them (`reference/parity.md`). ## DO NOT diff --git a/.agents/skills/deco-v7-to-v8-migration/reference/faststore.md b/.agents/skills/deco-v7-to-v8-migration/reference/faststore.md index ac654283..b33fde78 100644 --- a/.agents/skills/deco-v7-to-v8-migration/reference/faststore.md +++ b/.agents/skills/deco-v7-to-v8-migration/reference/faststore.md @@ -22,7 +22,7 @@ FastStore's CMS pages are SSG and its CLI forbids `middleware`/`proxy` and `/api ## Preview hosts and draft sources - `preview.hosts` in `CMS.json` must list the dev host (`localhost`) as well as the store's hosts, within the code cap in `createCMS({ preview })`. v8 ignores v7's `DECO_ALLOWED_PREVIEW_HOSTS`: remove it from env files and harness env too. -- Drafts need no site loader: `cms.forDraft` fetches the pointer itself, only from v7's preview API domains (`*.decocms.com` and the loopback hosts by default; `DECO_PREVIEW_API_DOMAINS` replaces the list, as in v7). Delete any draft-fetching loader the site kept from v7. +- Drafts need no site loader: `cms.forDraft` fetches the pointer itself, only from v7's preview API domains (`*.decocms.com` and the loopback hosts by default; `createCMS({ preview: { draftHosts } })` replaces the list. The SDK reads no environment variable, so v7's `DECO_PREVIEW_API_DOMAINS` is read by the site, if at all, and passed in). Delete any draft-fetching loader the site kept from v7. - If the site keeps v7's policy that a draft that fails to load falls back to the release, write that down: the docs say a failed draft is an error. ## Content imported from FastStore's CMS diff --git a/.agents/skills/deco-v7-to-v8-migration/reference/gotchas.md b/.agents/skills/deco-v7-to-v8-migration/reference/gotchas.md index eea60af5..3f0f7851 100644 --- a/.agents/skills/deco-v7-to-v8-migration/reference/gotchas.md +++ b/.agents/skills/deco-v7-to-v8-migration/reference/gotchas.md @@ -1,6 +1,6 @@ # Gotchas from the site migrations -Learned on `deco-sites/storefront-tanstack` (Shopify), `deco-sites/blog-tanstack`, a Next.js storefront on VTEX, a TanStack Start storefront on VTEX and a non-ejected FastStore storefront on VTEX (the last three private, so no hashes; FastStore specifics are in `faststore.md`). Commit hashes refer to the two public repos. +Learned on `deco-sites/storefront-tanstack` (Shopify), `deco-sites/blog-tanstack`, two Next.js storefronts on VTEX, a TanStack Start storefront on VTEX and a non-ejected FastStore storefront on VTEX (the last five private, so no hashes; FastStore specifics are in `faststore.md`). Commit hashes refer to the two public repos. ## Before you run the script @@ -56,24 +56,28 @@ v8 has no invoke endpoint. Every call the browser made through `/deco/invoke` be ## Next.js App Router -- **`transpilePackages: ['@decocms/blocks']`** while the published package ships `.ts` source (see SKILL.md, step 1). A linked local checkout ships `dist/` and hides this; so does every parity run made against it. +- **`transpilePackages: ['@decocms/blocks']`** only while the installed package's `exports` point at `.ts` source (see SKILL.md, step 1); `8.1.0-next.7` ships `dist/` and needs none. A linked local checkout ships `dist/` and hides the difference; so does every parity run made against it. - **Drafts on Pages Router SSG pages** (no proxy available, as on FastStore): Next 16's compiled pages runtimes inline `tryGetPreviewData`, so a `require.cache` patch never runs. See `faststore.md`. - **Drafts on static pages.** `force-static`/ISR pages get stubbed `cookies()`, so they can't read the draft pointer. Divert drafted requests (`?__draft=` or the draft cookie) in `proxy.ts` onto a dynamic route group that binds the pointer, and 404 direct hits on that internal route. Send `Cache-Control: no-store, private` and `X-Robots-Tag: noindex` on both signals. +- **The proxy runs on every request: keep the CMS out of it for ordinary traffic.** Ask `cms.draftPointer`/`cms.draftCookie` only when the request has `?__draft=` or the `__deco_draft` cookie (the SDK doesn't export the names yet; pin them in a test). Give the proxy its own `createCMS` handle with `blocks: {}` and the same options as the app's (one shared options module, since a second call with other options keeps the first): the draft helpers read only the `CMS` settings block, so the block map, its section loaders and what they import (a GraphQL engine) leave the proxy bundle. Measured on one store: 2.2 MB to 1.5 MB, traced files 3,245 to 364. The content module still lands there. +- **A pointer that doesn't parse.** `cms.draftPointer` returns the raw `?__draft=`/cookie value and `cms.forDraft` throws on one that doesn't parse, so `?__draft=junk` on a preview host answers 500. Treat `parseDraftPointer(pointer) === null` as no draft, in the proxy and the page. +- **Views wrapped in `next/dynamic`.** The script types each section from its view (`PropsOf`). A lazy view that re-exports a `next/dynamic` wrapper loses its props type, and its form collapses. Type those sections from the component module (`section(…)`) and diff the forms against v7's. +- **`list('page')` order.** It sorts by block name; v7 listed saved pages in file order. Output that lists pages in order (a sitemap) has to keep v7's order explicitly. - **List pages once per revision.** `list('page')` expands every page's saved-block references; on a site with hundreds of pages that is tens of milliseconds of CPU. Keep the routable list per revision (a revision never changes, a draft has its own) and hand the same array to `matchRoute` so it reuses its lookup. Health and readiness probes go through the same cache, never a fresh `list`. -- **Head meta order.** If v7 emitted `theme-color`/`color-scheme` after the root layout's metas on some routes, that was a race with Next's viewport resolution. To keep the order deterministic, render those tags from the segment layout (React hoists in tree order) rather than approving a reorder. +- **Head meta order.** If v7 emitted `theme-color`/`color-scheme` after the root layout's metas on some routes, that was a race with Next's viewport resolution. To keep the order deterministic, render those tags from the segment layout (React hoists in tree order) rather than approving a reorder. The race can go the other way: where v7 had the viewport tags *before* the root layout's metas and v8's faster shell now wins, a segment layout can't move them earlier. Look for a deterministic fix (the tags rendered ahead of the root layout's metas on every route, checked against every route's head), and keep the reorder pending, with that reasoning, if none holds. - **Jest** is CJS-only: transform `@decocms/blocks` (ESM) with ts-jest and keep it out of `transformIgnorePatterns`. When mocking a site module in a test, spread `jest.requireActual` so sibling exports other code imports stay real. ## Hosted releases and bundled content -**v7 fast deploy becomes the hosted CMS.** A v7 TanStack site that published content without a deploy (`DECO_FAST_DEPLOY` plus a `DECO_KV` namespace) needs `site` and `token` on `createCMS` (`DECO_SITE` and `DECO_SITE_TOKEN` secrets) for the same feature; without them every content change needs a deploy. Put the two `wrangler secret put` commands in the deploy checklist. v7's fast deploy also kept the content out of the bundle; v8 bundles it, and hosted mode loads a second copy. On a site with ~300 saved blocks (11 MB of JSON), Node measured about 13 MB per parsed copy, plus the bundled module's source (about the size of the JSON) and a transient string up to twice the JSON while a snapshot parses: under the Workers 128 MB limit, but measure memory and cold start with `wrangler tail` on a preview before turning it on. +**v7 fast deploy becomes the hosted CMS.** A v7 TanStack site that published content without a deploy (`DECO_FAST_DEPLOY` plus a `DECO_KV` namespace) needs `site` on `createCMS` for the same feature (and `token` for hosted telemetry): the site reads its own `DECO_SITE` var and `DECO_SITE_TOKEN` secret and passes them, since the SDK reads no environment variable; without `site` every content change needs a deploy. Put the two `wrangler secret put` commands in the deploy checklist. v7's fast deploy also kept the content out of the bundle; v8 bundles it, and hosted mode loads a second copy. On a site with ~300 saved blocks (11 MB of JSON), Node measured about 13 MB per parsed copy, plus the bundled module's source (about the size of the JSON) and a transient string up to twice the JSON while a snapshot parses: under the Workers 128 MB limit, but measure memory and cold start with `wrangler tail` on a preview before turning it on. -With `site` and `token` set on `createCMS`, a published release goes live without a deploy, but only for what reads through the CMS client. Before turning it on, grep for direct imports of `.deco/blocks/*.json` (redirects, proxy tables, config blocks) and for build-time scripts that read `.deco/blocks`: those stay on the last deploy. v7 behaved the same, so it isn't a parity break; route them through the client or document the limitation and keep hosted releases off until you do. +With `site` set on `createCMS`, a published release goes live without a deploy, but only for what reads through the CMS client. Before turning it on, grep for direct imports of `.deco/blocks/*.json` (redirects, proxy tables, config blocks) and for build-time scripts that read `.deco/blocks`: those stay on the last deploy. v7 behaved the same, so it isn't a parity break; route them through the client or document the limitation and keep hosted releases off until you do. ## v7 leftovers to delete - Env vars and comments for the v7 admin protocol (`DANGEROUSLY_ALLOW_PUBLIC_ACCESS`, admin public keys) and for settings that lived in the v7 `site` app block. -- Middleware/proxy exclusions for `/deco`, `/live` and `/.decofile`: those routes are gone, and keeping them special only hides that they now 404. -- Add the new env vars (`DECO_SITE`, `DECO_SITE_TOKEN`) to the site's env example, marked secret, and blank them in parity runs. +- Middleware/proxy exclusions for `/deco`, `/live` and `/.decofile`: the routes are gone. Removing the exclusion sends these paths down the unknown-path route, which may ask upstream (a redirect lookup) on every stray hit from old tooling and scanners; keeping it may send them to a catch-all that does worse (a search fallback answering 200). Skip the upstream work for them explicitly, and check the status (404) and the upstream calls for each path in the parity run. +- Add the site's new env vars (`DECO_SITE`, `DECO_SITE_TOKEN`, which the site passes as `createCMS({ site, token })`) to its env example, the token marked secret, and blank them in parity runs. - **Editor settings that disappear with the `site` block.** Deleting the v7 `site` app block (apps are code now) also deletes any setting editors changed there, such as draft preview hosts. Moving it to code or env is fine, but it is an editor-visible change: list it for the product owner. ## Content @@ -81,6 +85,7 @@ With `site` and `token` set on `createCMS`, a published release goes live withou - **`deco check` in the build gates the deploy.** Once `build` runs it, a Studio save or a content importer that writes something check rejects fails the next deploy. Make importers emit check-clean content (and run `deco check` at their end), and say in the README that the gate exists. - **Don't let rendering depend on undeclared `__`-prefixed fields.** `deco check` accepts `_`/`$`/`@` keys, but the docs say the site editor keeps only declared fields, and whether those keys survive a save isn't documented. Routing data (a page type) belongs in a declared field. - **Case-only name pairs.** Two saved blocks whose names differ only by letter case can't coexist on macOS disks; v7 silently bundled one. Check which one content references, keep it, and fail the build on such a pair (reading the git index too). v8's write path refuses new ones; `deco content`/`deco check` don't flag an existing pair. +- **Required fields saved content omits.** `deco check` rejects a saved block missing a field its type requires, which v7 tolerated. If the component renders without it, make the field optional (an editor-visible change to list), and don't fill content with placeholders. - `deco check` rejects fields no type declares, `.tsx`-named preview blocks and dangling references: delete or type them (`9a097a1`, `ffa866a`). - v7 app blocks (`deco-shopify`, `deco-blog`, `site`) go: apps are code now, configured from env (e.g. `SHOPIFY_STORE_NAME`). Don't re-encrypt secrets runtime code never reads. - `website/functions/requestToParam.ts` in a string field becomes the page's route `param` (`/:slug`) the block reads itself. @@ -95,8 +100,9 @@ v8 ships no dev hook for content changes; two pieces of template code make a sav ## Telemetry and analytics -- Telemetry: set `OTEL_EXPORTER_OTLP_ENDPOINT` to the OTLP collector and the auth header as the `OTEL_EXPORTER_OTLP_HEADERS` secret in production; send nothing in dev or parity runs (`89eae7d`). v7's `DECO_OTEL_{METRICS,LOGS,TRACES}_ENDPOINT`, `DECO_OTEL_HEADERS` and `DECO_OTEL_AUTH_TOKEN` keep working as aliases (the standard name wins when both are set), so a site's existing variables don't need renaming. -- `service.version` comes from `DECO_COMMIT_SHA` (or another host's commit variable). Workers have none at runtime: set it from the build's commit (a Vite `define`) before `createCMS`. v7 read the deployment id from the `CF_VERSION_METADATA` binding. +- Telemetry is explicit params only: the SDK reads no environment variable (`OTEL_*` and v7's `DECO_OTEL_*` are ignored). A site that kept a collector reads its own variables and passes `telemetry: { endpoint, headers }`; with `createCMS({ site, token })` and no `endpoint`, telemetry goes to the hosted collector (a `token` without `site` is a configuration error). Pass `telemetry: false` in dev or parity runs (`89eae7d`). v7's per-signal `DECO_OTEL_{METRICS,LOGS,TRACES}_ENDPOINT` have no v8 equivalent: every signal goes to `/v1/`. +- **New telemetry is a behaviour change.** A site with its own OpenTelemetry export that v7 never fed CMS telemetry starts sending it once `telemetry` points at that collector. List the new traffic for the product owner. +- `service.version` (and `deployment.environment.name`, default `production`) come from `telemetry.resource`: pass the build's commit (a Vite `define` on Workers, which have no commit variable at runtime). v7 read the deployment id from the `CF_VERSION_METADATA` binding. - **v7 Workers bindings with no v8 writer.** v7's `instrumentWorker` wrote per-route request counts, edge cache hit/miss and request duration to an Analytics Engine dataset (`DECO_METRICS`) and read `CF_VERSION_METADATA`; v8 has no metric API for them. Don't keep the bindings as dead config, and don't drop them silently: list each by name (and the edge layer's `deco.cache.requests` counter) as a product decision in the PR, since the dashboards built on them go dark. - Analytics: `AnalyticsScript` with `(await cms.settings()).analytics` plus `track` replace OneDollarStats (the collector lives in the `analytics` section of `CMS.json`); it sends collector beacons directly instead of loading the SDK (`4a50b13`, blog `9e97455`). diff --git a/.agents/skills/deco-v7-to-v8-migration/reference/import-map.md b/.agents/skills/deco-v7-to-v8-migration/reference/import-map.md index e1517aed..19c7179c 100644 --- a/.agents/skills/deco-v7-to-v8-migration/reference/import-map.md +++ b/.agents/skills/deco-v7-to-v8-migration/reference/import-map.md @@ -40,7 +40,7 @@ From the storefront and blog migrations, the reported imports above were replace | v7 import | Site code | |---|---| -| `Image`, `Picture` (`@decocms/blocks/hooks`) | `src/vendor/blocks/Image.tsx`, a copy that keeps v7's exact CDN URLs | +| `Image`, `Picture` (`@decocms/blocks/hooks`) | `src/vendor/blocks/Image.tsx`, a copy that keeps v7's exact CDN URLs; a narrower copy inlined where the site already builds image URLs (an image loader) is fine when it yields v7's URL for every input the site passes | | `useDevice`, `detectDevice` (`@decocms/blocks/sdk/*`) | `src/sdk/device.ts` with v7's user-agent patterns | | `getCookies`/`setCookie`, `RequestContext` | read the request and write response headers explicitly, or a site-owned `AsyncLocalStorage` (`src/request-state.server.ts`) | | `useCart`/`useUser`/`useWishlist`, cart loaders (`@decocms/apps-`) | `src/vendor//…` copies, sending through the v8 client (`createShopifyClient`, …) | diff --git a/.agents/skills/deco-v7-to-v8-migration/reference/parity.md b/.agents/skills/deco-v7-to-v8-migration/reference/parity.md index b008d8cb..57d4a519 100644 --- a/.agents/skills/deco-v7-to-v8-migration/reference/parity.md +++ b/.agents/skills/deco-v7-to-v8-migration/reference/parity.md @@ -19,7 +19,7 @@ Never hide a pending entry: the compare summary and the PR list every one. - **Build against what ships.** Do the final compare on a build from the committed lockfile, not a linked local checkout: a published package can differ (source vs compiled output, older CLI). - **ISR and prerendered routes.** Pages prerendered at build time with `revalidate` were stale when the baseline recorded them. Compare a build older than the longest `revalidate`, or those cases differ only in `x-nextjs-cache`/`x-nextjs-prerender`. -- **Flakes.** Rerun a single failing case (`--only `) before treating it as a regression, and avoid running two harnesses on one machine at once: font rasterization under load produced one-off weight diffs. Record each full run's flake count and which cases flipped; a count that grows between runs is a determinism regression to document or fix, not noise. +- **Flakes.** Rerun a single failing case (`--only `) before treating it as a regression, and avoid running two harnesses on one machine at once: font rasterization under load produced one-off weight diffs. Record each full run's flake count and which cases flipped; a count that grows between runs is a determinism regression to document or fix, not noise. A case that fails again on rerun isn't a flake: look up its differing pixels in every earlier run, v7 self-compares included, with each run's value. If v7 against its own baseline flipped the same pixel, write that evidence and the rate on each side into a pending pixel rule (a rectangle that covers only those pixels), not a mask; if only v8 flips it, fix it or list it as pending. Never report N/N on a run where it failed. - **Pin the Workers `request.cf` object.** A local Workers runtime (Miniflare/wrangler) fills `request.cf` from a cache file fetched for the machine's network location (`node_modules/.mf/cf.json`), once per checkout. Two checkouts fetched at different times can disagree (region, city, colo), and anything keyed on the region (a cache segment, regional prices) differs. Copy one checkout's file into the other before recording, and record and compare on the same ports: URLs built from the origin can carry the port. - **A bad baseline is re-recorded, never masked.** If the v7 baseline itself captured a rendering flake (an icon laid out but not painted), re-record that case from the v7 site (`record --only `) instead of adding a pixel-ignore rect. The re-record also refreshes that case's upstream fixtures live, so its page can change (prices, stock, page height); run a full compare afterwards. - **Fix in code what code can fix, then prove it on every page type.** Before asking for approval, try removing a pending difference in site code; then run the full compare, not just the cases the rule named. A popup gated on the footer's position fixed the home page cases and broke the product page ones (`gotchas.md`, Rendering), so that rule stayed pending. @@ -34,7 +34,7 @@ Compare each section's form, not just the content. Sort each difference into one **2. v7 heuristics v8 drops on purpose.** Approve these, don't "fix" them: - **"Select from saved" on every list.** v7 offered it on every array. v8 offers it only where a saved block fits the field's type. Check that no content uses it, and whether the renderer could even resolve a saved block there. - **Every file under `sections/` as a section.** v7 listed helper files (shared `types.ts`) as sections. v8 lists only what the block map declares. -- **Dynamic option pickers.** v7's `@format dynamic-options` fields ran a site loader through `/deco/invoke` to suggest values as the editor typed. The v8 site editor never runs site code, so they become text fields (`/next/schema`). Make sure the field still accepts what an editor would type (a bare ID, a plain place name), update copy that promises suggestions, and list each field for approval. +- **Dynamic option pickers.** v7's `@format dynamic-options` fields ran a site loader through `/deco/invoke` to suggest values as the editor typed. The v8 site editor never runs site code, so they become text fields (`/next/schema`). Make sure the field still accepts what an editor would type (a bare ID, a plain place name), update copy that promises suggestions (grep each such field's `@description` for search wording in the site's language: search, type part of the name, pick from the list), and list each field, with its new copy, for approval. - **Page form order and labels.** A site page type that `extends Route` and adds `seo`/`sections` lists its own fields first and labels `seo` as "Seo". Redeclare `name` and `path` (with their `@title`) before them and give `seo` an explicit `@title SEO` to keep v7's form. - **The rich-text widget from a substring.** v7 gave the rich-text editor to any type whose name contained `RichText` (`PromoRichText`). v8 matches the alias name as a whole word. When editors rely on the toolbar, keep it with `/** @format rich-text */` on the alias rather than approving its loss. diff --git a/packages/blocks/conformance/observability/cms.ts b/packages/blocks/conformance/observability/cms.ts index 311d089f..3eef6a46 100644 --- a/packages/blocks/conformance/observability/cms.ts +++ b/packages/blocks/conformance/observability/cms.ts @@ -27,7 +27,20 @@ export const sampled = () => }, }); -// telemetry.mdx: `telemetry: { site, token }` and `telemetry: false`. -export const hostedForm = (site: string, token: string) => - createCMS({ blocks, content, telemetry: { site, token } }); +// telemetry.mdx › Resource attributes, verbatim inside createCMS: the app reads its own env. +export const described = () => + createCMS({ + blocks, + content, + telemetry: { + endpoint: process.env.OTLP_ENDPOINT!, + resource: { + "service.version": process.env.COMMIT_SHA ?? "unknown", // e.g. the deployed commit + "deployment.environment.name": "preview", + }, + }, + }); + +// telemetry.mdx: a top-level `token` (the hosted collector) and `telemetry: false`. +export const hostedForm = (site: string, token: string) => createCMS({ blocks, content, site, token }); export const off = () => createCMS({ blocks, content, telemetry: false }); diff --git a/packages/blocks/conformance/observability/hosted-cms.ts b/packages/blocks/conformance/observability/hosted-cms.ts index b6bc415d..5ac7e758 100644 --- a/packages/blocks/conformance/observability/hosted-cms.ts +++ b/packages/blocks/conformance/observability/hosted-cms.ts @@ -3,12 +3,10 @@ import { createCMS } from "@decocms/blocks"; import blocks from "./.deco"; import content from "./.deco/blocks.gen"; -const { DECO_SITE: site, DECO_SITE_TOKEN: token } = process.env; - +// Your app reads its own environment and passes the values; the SDK reads none. export const cms = createCMS({ blocks, content, - site, // loads releases and drafts - token, - telemetry: site && token ? { site, token } : false, // telemetry is separate: site/token above only load releases + site: process.env.DECO_SITE, // hosted releases + token: process.env.DECO_SITE_TOKEN, // hosted telemetry (server-only secret) }); diff --git a/packages/blocks/src/v8/__conformance__/cli.test.ts b/packages/blocks/src/v8/__conformance__/cli.test.ts index 1a3de633..82569fd3 100644 --- a/packages/blocks/src/v8/__conformance__/cli.test.ts +++ b/packages/blocks/src/v8/__conformance__/cli.test.ts @@ -428,6 +428,10 @@ describe("cli.mdx", () => { expect(r.code, r.out).toBe(0); const mod = f.read(".deco/blocks.gen.ts"); expect(mod).toContain('"Home"'); + // committedAt: the commit time of git HEAD (hosted releases: newer wins). The + // fixture isn't a git repository, so there is none, and deco content says so. + expect(mod).not.toContain("committedAt"); + expect(r.out).toMatch(/^no git commit found, so \.deco\/blocks\.gen\.ts has no committedAt/m); expect(mod).not.toContain("notes"); expect(mod).not.toContain("README"); expect(mod).not.toContain("index"); @@ -2213,22 +2217,23 @@ describe("studio-implementation.mdx", () => { const s = await serveFixture(f.root); expect((await call(s, "describe")).result.pollIntervalMs).toBe(2000); }); - describe("si-08 / si-09 / si-10: the SDK's release channel", () => { + describe("si-08 / si-09 / si-10: the SDK's release pointer (latest.json)", () => { const ORIGIN = "https://delivery.decocms.com"; - const MANIFEST = `${ORIGIN}/sites/acme/channels/production.json`; - let manifest: Json | undefined; + const LATEST = `${ORIGIN}/sites/acme/latest.json`; + const SCHEMA = "5".repeat(64); + let latest: Json | undefined; const assets = new Map(); const gates = new Map>(); beforeEach(() => { resetForTests(); - manifest = undefined; + latest = undefined; assets.clear(); gates.clear(); vi.stubGlobal("fetch", async (input: string | URL | Request) => { const url = String(input); - if (url === MANIFEST) - return manifest ? Response.json(manifest) : new Response("", { status: 404 }); + if (url === LATEST) + return latest ? Response.json(latest) : new Response("", { status: 404 }); const p = url.slice(ORIGIN.length).split("?")[0]; await gates.get(p); return assets.has(p) ? Response.json(assets.get(p)) : new Response("", { status: 404 }); @@ -2236,15 +2241,15 @@ describe("studio-implementation.mdx", () => { }); afterAll(() => vi.unstubAllGlobals()); - async function snap(title: string): Promise { - const { computeContentRevision } = await import("@decocms/blocks/protocol"); + function snap(title: string, revision: string): Snapshot { const blocks = { S: { __resolveType: "seo", title, description: "d" } }; - return { revision: await computeContentRevision(blocks), blocks }; + return { revision, blocks, schemaHash: SCHEMA }; } - function publish(generation: number, s: Snapshot) { + /** Studio's publish of commit `s.revision`: the revision object, then latest.json. */ + function publish(s: Snapshot) { const p = `/sites/acme/revisions/${s.revision}.json`; - assets.set(p, s); - manifest = { format: 1, generation, revision: s.revision, snapshot: p }; + assets.set(p, { revision: s.revision, schemaHash: SCHEMA, blocks: s.blocks }); + latest = { revision: s.revision, schemaHash: SCHEMA, publishedAt: new Date().toISOString() }; return p; } @@ -2254,62 +2259,51 @@ describe("studio-implementation.mdx", () => { expect(src).toMatch(/JITTER = 10_000/); }); - it("si-09: a stale fetch completion is discarded; a rollback by generation is accepted", async () => { - const fallback = await snap("bundled"); - const loader = remoteLoader(fallback, { site: "acme", token: "t" }) as Loader; - const a = await snap("A"); - const b = await snap("B"); - // Gen 5 (B) is slow; gen 6 (A) lands first. - let release!: () => void; - const pB = publish(5, b); - gates.set( - pB, - new Promise((r) => { - release = r; - }), - ); - const slow = loader.update!(); - await new Promise((r) => setTimeout(r, 10)); - publish(6, a); - await loader.update!(); - release(); - await slow; - expect((await loader.load()).revision).toBe(a.revision); - // Rollback: gen 7 selects the older revision B. - publish(7, b); + it("si-09: the SDK follows the revision latest.json names; a rollback (Make current) is accepted", async () => { + const fallback = snap("bundled", "rev-bundled"); + const loader = remoteLoader(fallback, { site: "acme" }) as Loader; + const a = snap("A", "a".repeat(40)); + const b = snap("B", "b".repeat(40)); + publish(a); + expect(await loader.update!()).toEqual({ updated: true }); + publish(b); expect(await loader.update!()).toEqual({ updated: true }); expect((await loader.load()).revision).toBe(b.revision); + // Make current: latest.json names the older revision again; nothing else changes. + latest = { revision: a.revision, schemaHash: SCHEMA, publishedAt: new Date().toISOString() }; + expect(await loader.update!()).toEqual({ updated: true }); + expect((await loader.load()).revision).toBe(a.revision); }); - it("si-09: a generation change with the same content hash still signals an update", async () => { - const fallback = await snap("bundled"); - const loader = remoteLoader(fallback, { site: "acme", token: "t" }) as Loader; - const a = await snap("A"); - publish(1, a); - expect(await loader.update!()).toEqual({ updated: true }); - publish(2, a); // a new generation (e.g. a re-promotion or rollback) of the same revision + it("si-09: the same revision as the one loaded is not an update", async () => { + const fallback = snap("bundled", "rev-bundled"); + const loader = remoteLoader(fallback, { site: "acme" }) as Loader; + const a = snap("A", "a".repeat(40)); + publish(a); expect(await loader.update!()).toEqual({ updated: true }); + publish(a); // republished: same commit, new publishedAt + expect(await loader.update!()).toEqual({ updated: false }); }); it("si-10: a draft's changes apply to the production this server has; no matching base is fetched", async () => { - const fallback = await snap("bundled"); - const published = await snap("published"); - publish(1, published); + const fallback = snap("bundled", "rev-bundled"); + const published = snap("published", "c".repeat(40)); + publish(published); gates.set(`/sites/acme/revisions/${published.revision}.json`, new Promise(() => {})); // never arrives - const studio = "https://studio.decocms.com/api/acme/decofile/store/b/changes"; + const drafts = `${ORIGIN}/sites/acme/drafts/b.json`; const fetch = globalThis.fetch; const urls: string[] = []; vi.stubGlobal("fetch", async (input: string | URL | Request) => { urls.push(String(input)); - return String(input).startsWith(studio) - ? Response.json({ format: 1, set: { New: { __resolveType: "seo" } }, delete: [] }) + return String(input).startsWith(drafts) + ? Response.json({ set: { New: { __resolveType: "seo" } }, delete: [] }) : fetch(input); }); - const cms = createCMS({ blocks: {}, content: fallback, site: "acme", token: "t" }); - const draft = cms.forDraft("studio.decocms.com/api/acme/decofile/store/b/changes?token=x@c1"); + const cms = createCMS({ blocks: {}, content: fallback, site: "acme" }); + const draft = cms.forDraft("delivery.decocms.com/sites/acme/drafts/b.json@c1"); const [entry] = await draft.resolve("S", { run: false }); expect(entry).toEqual(fallback.blocks.S); // the bundled release, never waiting for the new one - expect(urls.filter((url) => url.startsWith(studio))).toHaveLength(1); + expect(urls.filter((url) => url.startsWith(drafts))).toHaveLength(1); expect(urls.filter((url) => url.includes("/revisions/"))).toHaveLength(0); }); }); diff --git a/packages/blocks/src/v8/__conformance__/delivery.snippets.test.ts b/packages/blocks/src/v8/__conformance__/delivery.snippets.test.ts index 9c443482..37015bfd 100644 --- a/packages/blocks/src/v8/__conformance__/delivery.snippets.test.ts +++ b/packages/blocks/src/v8/__conformance__/delivery.snippets.test.ts @@ -23,9 +23,10 @@ export default { seo: (input: Seo) => input } satisfies Blocks; `, ".deco/blocks.gen.ts": `const content: { revision: string; + schemaHash: string; blocks: Record; aliases: Record; -} = { revision: "r", blocks: {}, aliases: {} }; +} = { revision: "r", schemaHash: "h", blocks: {}, aliases: {} }; export default content; `, "env.d.ts": `declare module "cloudflare:workers" { @@ -39,16 +40,12 @@ export default content; import blocks from "../.deco"; import content from "../.deco/blocks.gen"; -const site = process.env.DECO_SITE; // your site's ID in the hosted Deco CMS -const token = process.env.DECO_SITE_TOKEN; // your site token (secret) - +// Your app reads its own environment; the SDK reads none. export const cms = createCMS({ blocks, content, - site, - token, - // Optional: also send telemetry to the hosted collector - telemetry: site && token ? { site, token } : false, + site: process.env.DECO_SITE, // your site's ID: turns on hosted releases + token: process.env.DECO_SITE_TOKEN, // your site token (secret): turns on hosted telemetry }); `, @@ -62,8 +59,7 @@ export const cms = createCMS({ blocks, content, site: env.DECO_SITE, - token: env.DECO_SITE_TOKEN, - telemetry: { site: env.DECO_SITE, token: env.DECO_SITE_TOKEN }, // optional: the hosted collector + token: env.DECO_SITE_TOKEN, // optional: telemetry to the hosted collector }); `, @@ -73,7 +69,7 @@ import blocks from "../.deco"; import content from "../.deco/blocks.gen"; // Check for new releases every 2 minutes instead of every minute -createCMS({ blocks, content, site: process.env.DECO_SITE, token: process.env.DECO_SITE_TOKEN, interval: 120_000 }); +createCMS({ blocks, content, site: process.env.DECO_SITE, interval: 120_000 }); `, // HP-16: hosted-publishing.mdx, "Troubleshooting". @@ -81,7 +77,7 @@ createCMS({ blocks, content, site: process.env.DECO_SITE, token: process.env.DEC import blocks from "../.deco"; import content from "../.deco/blocks.gen"; -const loader = remoteLoader(content, { site: process.env.DECO_SITE, token: process.env.DECO_SITE_TOKEN }); +const loader = remoteLoader(content, { site: process.env.DECO_SITE }); export const cms = createCMS({ blocks, content: loader }); // in a request handler or a debug route: @@ -173,6 +169,18 @@ declare const request: Request; const pointer = isEmployee(request) ? await cms.draftPointer(request) : null; // isEmployee: your rule export { pointer }; +`, + + // DP-6: content-delivery.mdx, "Draft previews" (where draft pointers may point). + "drafts/draft-hosts.ts": `import { createCMS } from "@decocms/blocks"; +import blocks from "../.deco"; +import content from "../.deco/blocks.gen"; + +export const cms = createCMS({ + blocks, + content, + preview: { draftHosts: [".decocms.com", "drafts.example.com"] }, // replaces the defaults +}); `, // RD-11: releases-and-drafts.mdx, "Allow previews per host" (code caps the list). @@ -246,6 +254,7 @@ describe("hosted docs examples compile", () => { ["HD-13", "drafts/native.ts"], ["HD-18", "drafts/preview-host.ts"], ["RD-11", "drafts/preview-cap.ts"], + ["DP-6", "drafts/draft-hosts.ts"], ])("%s: %s", (_claim, file) => { expect(errorsIn(file)).toEqual([]); }); diff --git a/packages/blocks/src/v8/__conformance__/delivery.test.ts b/packages/blocks/src/v8/__conformance__/delivery.test.ts index 00d862d8..892013a8 100644 --- a/packages/blocks/src/v8/__conformance__/delivery.test.ts +++ b/packages/blocks/src/v8/__conformance__/delivery.test.ts @@ -3,15 +3,17 @@ * Docs conformance: content-delivery, draft-synchronization, * releases-and-deployment, hosted, hosted-publishing, hosted-drafts and * hosted-releases-internals (src/content/docs/en/storefront/blocks/next/*.mdx in - * deco-sites/docs-tanstack). Each test names the claim it checks (CD-*, RD-*, H-*, HP-*, HD-*, - * HRI-*, DP-*). A failing test is a claim the code doesn't meet yet. + * deco-sites/docs-tanstack), as the hosted contract (2026-10-06) defines them: + * `sites//latest.json` and `revisions/.json` on delivery.decocms.com, + * drafts on the CDN. Each test names the claim it checks (CD-*, RD-*, H-*, HP-*, + * HD-*, HRI-*, DP-*). A failing test is a claim the code doesn't meet yet. */ import fs from "node:fs"; import path from "node:path"; import { fileURLToPath } from "node:url"; import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; import * as root from "../../index"; -import { computeContentRevision } from "../../protocol/canonical"; +import { computeContentRevision, sha256Hex } from "../../protocol/canonical"; import { HOSTED_ANALYTICS_COLLECTOR } from "../builtins/data"; import { instanceOf } from "../cms"; import { @@ -23,7 +25,12 @@ import { resetForTests, } from "../index"; import { resolveDestination } from "../telemetry"; -import { docsBlocks, docsSnapshot, fakeStudio, STUDIO_HOST, STUDIO_TOKEN } from "../testFixtures"; +import { + DRAFT_HOST, + docsBlocks, + fakeStudio, + docsSnapshot as unhashedSnapshot, +} from "../testFixtures"; import type { Loader, RequestLike, Snapshot } from "../types"; /** cms.draftPointer and cms.draftCookie on a CMS with no settings: every host may preview. */ @@ -36,35 +43,36 @@ const HERE = path.dirname(fileURLToPath(import.meta.url)); const ORIGIN = "https://delivery.decocms.com"; const SITE = "acme"; const TOKEN = "site-token"; -const MANIFEST_URL = `${ORIGIN}/sites/acme/channels/production.json`; +const LATEST_URL = `${ORIGIN}/sites/acme/latest.json`; +/** The schemaHash of the schema the deployed code was built with (`deco content` writes it). */ +const SCHEMA = "5".repeat(64); /** A pointer string, for the cookie and parsing claims; drafts that load come from api.draft(). */ -const POINTER = `${STUDIO_HOST}/api/acme/decofile/store/summer-sale/changes?token=abc@9f3c1a`; -/** Where the fake Studio answers drafts. */ -const STUDIO = `https://${STUDIO_HOST}`; +const POINTER = `${DRAFT_HOST}/sites/acme/drafts/summer-sale.json@9f3c1a`; +/** Where the fake CDN answers drafts. */ +const DRAFTS = `https://${DRAFT_HOST}/sites/acme/drafts/`; + +/** The bundled content module: the docs' snapshot, built with the deployed schema. */ +const docsSnapshot = (revision?: string): Snapshot => ({ + ...unhashedSnapshot(revision), + schemaHash: SCHEMA, +}); -const remote = (fallback: Snapshot | Loader) => - remoteLoader(fallback, { site: SITE, token: TOKEN }) as Loader; +const remote = (fallback: Snapshot | Loader) => remoteLoader(fallback, { site: SITE }) as Loader; const flush = (ms = 10) => new Promise((resolve) => setTimeout(resolve, ms)); -let savedInterval: string | undefined; -beforeEach(() => { - resetForTests(); - savedInterval = process.env.DECO_CONTENT_INTERVAL; - delete process.env.DECO_CONTENT_INTERVAL; -}); +beforeEach(() => resetForTests()); afterEach(async () => { await flush(); // let background checks a test started finish against its own fetch stub - if (savedInterval === undefined) delete process.env.DECO_CONTENT_INTERVAL; - else process.env.DECO_CONTENT_INTERVAL = savedInterval; vi.unstubAllGlobals(); vi.unstubAllEnvs(); vi.restoreAllMocks(); resetForTests(); }); +/** A release as Studio publishes it for a commit: the blocks at that commit, keyed by its SHA. */ async function hashed(title: string, extra: Record = {}): Promise { const blocks = { SummerSEO: { __resolveType: "seo", title, description: "d" }, ...extra }; - return { revision: await computeContentRevision(blocks), blocks }; + return { revision: (await sha256Hex(title)).slice(0, 40), blocks }; } /** A content module whose revision is its real content hash, as `deco content` writes it. */ @@ -89,14 +97,13 @@ function deferred(): Deferred { const seoEntry = (title: string) => ({ __resolveType: "seo", title, description: "d" }); /** - * A fake delivery API (a channel manifest and immutable revision assets) and - * a fake Studio API answering draft pointers (see testFixtures' fakeStudio). + * A fake delivery CDN: `sites/acme/latest.json`, immutable + * `sites/acme/revisions/.json`, and drafts (see testFixtures' fakeStudio). */ function deliveryApi() { - const studio = fakeStudio(); - const assets = new Map(); + const drafts = fakeStudio(); + const objects = new Map(); const gates = new Map(); - let manifest: Record | undefined; let failAll = false; const requests: { url: string; headers: Record; init?: RequestInit }[] = []; const fetch = vi.fn(async (input: string | URL | Request, init?: RequestInit) => { @@ -105,49 +112,49 @@ function deliveryApi() { requests.push({ url, headers, init }); if (failAll) throw new TypeError("fetch failed"); await gates.get(new URL(url).pathname)?.promise; - if (url.startsWith(`${STUDIO}/`)) return studio.fetch(url, init); + if (url.startsWith(DRAFTS)) return drafts.fetch(url, init); if (!url.startsWith(ORIGIN)) return new Response("wrong host", { status: 599 }); const p = url.slice(ORIGIN.length).split("?")[0] ?? ""; - if (url === MANIFEST_URL) { - if (!manifest) return new Response("missing", { status: 404 }); - return Response.json(manifest); - } - if (assets.has(p)) { - const body = assets.get(p); + if (objects.has(p)) { + const body = objects.get(p); if (body instanceof Response) return body; return Response.json(body); } return new Response("not found", { status: 404 }); }); vi.stubGlobal("fetch", fetch); + const latest = (revision: string, schemaHash = SCHEMA) => ({ + revision, + schemaHash, + publishedAt: new Date().toISOString(), + }); return { fetch, requests, assetPath: (revision: string) => `/sites/acme/revisions/${revision}.json`, - publish(generation: number, snapshot: Snapshot, body: unknown = snapshot) { - const p = `/sites/acme/revisions/${snapshot.revision}.json`; - assets.set(p, body); - manifest = { format: 1, generation, revision: snapshot.revision, snapshot: p }; + /** Studio's publish: `revisions/.json`, then `latest.json` naming it. */ + publish(snapshot: Snapshot, body?: unknown, schemaHash = SCHEMA) { + objects.set( + `/sites/acme/revisions/${snapshot.revision}.json`, + body ?? { revision: snapshot.revision, schemaHash, blocks: snapshot.blocks }, + ); + objects.set("/sites/acme/latest.json", latest(snapshot.revision, schemaHash)); }, - point(generation: number, revision: string) { - manifest = { - format: 1, - generation, - revision, - snapshot: `/sites/acme/revisions/${revision}.json`, - }; + /** Studio's "Make current" (or a pointer to a revision that isn't there): `latest.json` only. */ + point(revision: string, schemaHash = SCHEMA) { + objects.set("/sites/acme/latest.json", latest(revision, schemaHash)); }, - setManifest(value: Record) { - manifest = value; + setLatest(value: unknown) { + objects.set("/sites/acme/latest.json", value); }, asset(p: string, body: unknown) { - assets.set(p, body); + objects.set(p, body); }, - /** Saves a draft branch's changes on the fake Studio; returns the pointer it would mint. */ - draft: studio.draft, - /** Answers a draft branch with this response instead. */ - respond: studio.respond, - draftFetches: () => requests.filter((r) => r.url.startsWith(`${STUDIO}/`)), + /** Saves a draft on the fake CDN; returns the pointer Studio would hand out. */ + draft: drafts.draft, + /** Answers a draft with this response instead. */ + respond: drafts.respond, + draftFetches: () => requests.filter((r) => r.url.startsWith(DRAFTS)), gate(p: string) { const d = deferred(); gates.set(p, d); @@ -157,7 +164,7 @@ function deliveryApi() { failAll = value; }, assetFetches: () => requests.filter((r) => r.url.includes("/revisions/")).length, - manifestFetches: () => requests.filter((r) => r.url === MANIFEST_URL).length, + latestFetches: () => requests.filter((r) => r.url === LATEST_URL).length, }; } @@ -172,75 +179,88 @@ const titleOf = async (client: ReturnType["forRelea // --------------------------------------------------------------------------- describe("content-delivery", () => { - it("CD-1: the SDK never calls GitHub; releases come from the delivery origin, drafts from the pointer's Studio host", async () => { - for (const file of ["../remoteLoader.ts", "../draftChanges.ts"]) { + it("CD-1: the SDK never calls GitHub; releases come from the delivery origin, drafts from the pointer's host", async () => { + for (const file of ["../remoteLoader.ts", "../draftChanges.ts", "../content.ts", "../cms.ts"]) { expect(fs.readFileSync(path.join(HERE, file), "utf8")).not.toMatch(/github/i); } const api = deliveryApi(); - api.publish(1, await hashed("Published")); + api.publish(await hashed("Published")); const pointer = api.draft({ set: { SummerSEO: seoEntry("Draft") } }); - const cms = createCMS({ - blocks: docsBlocks(), - content: docsSnapshot(), - site: SITE, - token: TOKEN, - }); + const cms = createCMS({ blocks: docsBlocks(), content: docsSnapshot(), site: SITE }); await cms.update(); expect(await titleOf(cms.forDraft(pointer))).toBe("Draft"); const urls = api.requests.map((r) => r.url); - expect(urls.filter((url) => url.startsWith(`${ORIGIN}/`))).toHaveLength(2); // manifest, release - expect(urls.filter((url) => url.startsWith(`${STUDIO}/`))).toHaveLength(1); + expect(urls.filter((url) => url.startsWith(DRAFTS))).toHaveLength(1); + expect(urls.filter((url) => url.startsWith(`${ORIGIN}/`) && !url.startsWith(DRAFTS))).toEqual([ + LATEST_URL, + expect.stringMatching(/\/sites\/acme\/revisions\/[0-9a-f]{40}\.json$/), + ]); expect(urls).toHaveLength(3); }); - it("CD-3: one canonical hash, insertion-order independent, shared by the CLI and remoteLoader", async () => { + it("CD-3: the bundled module's revision is the canonical content hash; a release's is its commit SHA", async () => { const a = { A: { __resolveType: "seo", x: 1, y: 2 }, B: { z: [1, 2] } }; const b = { B: { z: [1, 2] }, A: { y: 2, __resolveType: "seo", x: 1 } }; expect(await computeContentRevision(a)).toBe(await computeContentRevision(b)); const cli = fs.readFileSync(path.join(HERE, "../cli/content.ts"), "utf8"); + expect(cli).toMatch( + /import \{[^}]*\bcomputeContentRevision\b[^}]*\} from "[./]+(protocol\/)?canonical\.ts"/, + ); + // No content-hash verification of releases: the SDK never hashes what it downloads. const sdk = fs.readFileSync(path.join(HERE, "../remoteLoader.ts"), "utf8"); - const shared = - // remoteLoader imports the SDK's leaf module; the CLI may go through the protocol's re-export. - /import \{[^}]*\bcomputeContentRevision\b[^}]*\} from "[./]+(protocol\/)?canonical\.ts"/; - expect(cli).toMatch(shared); - expect(sdk).toMatch(shared); + expect(sdk).not.toMatch(/computeContentRevision|sha256/); }); - it("CD-4/CD-5/CD-6: reads /sites//channels/production.json and the { revision, blocks } asset it names", async () => { + it("CD-4/CD-5/CD-6: reads /sites//latest.json and the { revision, schemaHash, blocks } it names", async () => { const api = deliveryApi(); - const release = await hashed("Generation 184"); - // The docs' manifest example, with a real content hash as the revision. - api.setManifest({ - format: 1, - generation: 184, + const release = await hashed("Commit C"); + // The docs' examples: latest.json and the revision object Studio writes for commit C. + api.setLatest({ revision: release.revision, - snapshot: `/sites/acme/revisions/${release.revision}.json`, + schemaHash: SCHEMA, + publishedAt: "2026-10-06T12:00:00.000Z", }); - api.asset(`/sites/acme/revisions/${release.revision}.json`, release); - const loader = remote(docsSnapshot()); + api.asset(`/sites/acme/revisions/${release.revision}.json`, { + revision: release.revision, + schemaHash: SCHEMA, + blocks: release.blocks, + }); + const fallback = docsSnapshot(); + const loader = remote(fallback); expect(await loader.update?.()).toEqual({ updated: true }); expect(api.requests.map((r) => r.url)).toEqual([ - MANIFEST_URL, + LATEST_URL, `${ORIGIN}/sites/acme/revisions/${release.revision}.json`, ]); + expect(api.requests.every((r) => r.headers.authorization === undefined)).toBe(true); const loaded = await loader.load(); - expect(Object.keys(loaded).sort()).toEqual(["blocks", "revision"]); - expect(loaded.revision).toBe(release.revision); + expect(loaded).toEqual({ + revision: release.revision, + schemaHash: SCHEMA, + blocks: release.blocks, + aliases: fallback.aliases, + }); }); - it("CD-7: refuses unknown formats and snapshot paths outside the origin or site namespace", async () => { + it("CD-7: refuses a latest.json that isn't { revision: , schemaHash, publishedAt }", async () => { const release = await hashed("Evil"); const bad = [ - { format: 2, snapshot: `/sites/acme/revisions/${release.revision}.json` }, - { format: 1, snapshot: `/sites/other/revisions/${release.revision}.json` }, - { format: 1, snapshot: `https://evil.example/sites/acme/revisions/${release.revision}.json` }, - { format: 1, snapshot: `/sites/acme/revisions/../../other/revisions/x.json` }, + { revision: release.revision, schemaHash: SCHEMA }, // no publishedAt + { revision: "../../other/revisions/x", schemaHash: SCHEMA, publishedAt: "x" }, + { revision: `${release.revision}/../x`, schemaHash: SCHEMA, publishedAt: "x" }, + { revision: "f".repeat(64), schemaHash: SCHEMA, publishedAt: "x" }, // a content hash + { revision: release.revision, schemaHash: "x", publishedAt: "x" }, + { format: 1, generation: 1, revision: release.revision, snapshot: "/x.json" }, // v0 channels ]; - for (const fields of bad) { + for (const value of bad) { resetForTests(); const api = deliveryApi(); - api.asset(`/sites/acme/revisions/${release.revision}.json`, release); - api.setManifest({ generation: 1, revision: release.revision, ...fields }); + api.asset(api.assetPath(release.revision), { + revision: release.revision, + schemaHash: SCHEMA, + blocks: release.blocks, + }); + api.setLatest(value); const fallback = docsSnapshot(); const loader = remote(fallback); await expect(loader.update?.()).rejects.toThrow(); @@ -249,42 +269,74 @@ describe("content-delivery", () => { } }); + it("CD-8: swaps only when latest.json's schemaHash equals the bundled content's; otherwise keeps what it serves", async () => { + const api = deliveryApi(); + const one = await hashed("One"); + const two = await hashed("Two"); + const fallback = docsSnapshot(); + const cms = createCMS({ blocks: docsBlocks(), content: fallback, site: SITE }); + api.publish(one, undefined, "6".repeat(64)); + expect(await cms.update()).toEqual({ updated: false }); + expect(api.assetFetches()).toBe(0); + expect(await cms.forRelease().revision()).toBe(fallback.revision); + api.publish(one); + expect(await cms.update()).toEqual({ updated: true }); + api.publish(two, undefined, "6".repeat(64)); + expect(await cms.update()).toEqual({ updated: false }); + expect(await titleOf(cms.forRelease())).toBe("One"); + }); + + it("CD-9: bundled content without a schemaHash (no schema.gen.json at build) never swaps", async () => { + const api = deliveryApi(); + api.publish(await hashed("Published")); + const cms = createCMS({ blocks: docsBlocks(), content: unhashedSnapshot(), site: SITE }); + expect(await cms.update()).toEqual({ updated: false }); + expect(api.fetch).not.toHaveBeenCalled(); + }); + + it("CD-10: no comparison with the bundle: the first check downloads the pointer's revision even when it holds the bundled content", async () => { + const api = deliveryApi(); + const bundled = docsSnapshot(); + const sameContent = { revision: "c".repeat(40), blocks: bundled.blocks }; + api.publish(sameContent); + const cms = createCMS({ blocks: docsBlocks(), content: bundled, site: SITE }); + expect(await cms.update()).toEqual({ updated: true }); + expect(api.assetFetches()).toBe(1); + expect(await cms.forRelease().revision()).toBe(sameContent.revision); + expect(await cms.update()).toEqual({ updated: false }); // the revision it already loaded + expect(api.assetFetches()).toBe(1); + }); + it("CD-11: a missing or failing asset keeps the last good content", async () => { const api = deliveryApi(); const good = await hashed("Good"); - api.publish(1, good); + api.publish(good); const cms = createCMS({ blocks: docsBlocks(), content: docsSnapshot(), site: SITE, - token: TOKEN, }); await cms.update(); expect(await titleOf(cms.forRelease())).toBe("Good"); const missing = await hashed("Missing"); - api.point(2, missing.revision); // 404 + api.point(missing.revision); // 404 await expect(cms.update()).resolves.toEqual({ updated: false }); api.asset(api.assetPath(missing.revision), new Response("boom", { status: 500 })); await expect(cms.update()).resolves.toEqual({ updated: false }); expect(await titleOf(cms.forRelease())).toBe("Good"); }); - it("CD-12: a rollback (higher generation, older revision) is adopted", async () => { + it('CD-12: a rollback (Studio\'s "Make current" pointing latest.json at an older revision) is adopted', async () => { const api = deliveryApi(); const a = await hashed("A"); const b = await hashed("B"); - const cms = createCMS({ - blocks: docsBlocks(), - content: docsSnapshot(), - site: SITE, - token: TOKEN, - }); - api.publish(183, a); + const cms = createCMS({ blocks: docsBlocks(), content: docsSnapshot(), site: SITE }); + api.publish(a); await cms.update(); - api.publish(184, b); + api.publish(b); await cms.update(); expect(await titleOf(cms.forRelease())).toBe("B"); - api.publish(185, a); + api.point(a.revision); expect(await cms.update()).toEqual({ updated: true }); expect(await titleOf(cms.forRelease())).toBe("A"); }); @@ -302,8 +354,8 @@ describe("content-delivery", () => { controller.enqueue(CHUNK); }, }); - const revision = "f".repeat(64); - api.point(1, revision); + const revision = "f".repeat(40); + api.point(revision); api.asset(api.assetPath(revision), new Response(body)); const fallback = docsSnapshot(); const loader = remote(fallback); @@ -344,7 +396,7 @@ describe("draft previews", () => { }; const FAILED = { value: null, code: "LOADER_FAILED", list: null, listCode: "LOADER_FAILED" }; - it("DP-1: GET https://&v=: forced variants removed, no cookies or credentials, fetched once", async () => { + it("DP-1: GET https://?v=: forced variants removed, no cookies or credentials", async () => { const api = deliveryApi(); const pointer = api.draft({ set: { SummerSEO: seoEntry("Draft") } }); const withVariant = formatDraftPointer({ @@ -353,12 +405,9 @@ describe("draft previews", () => { }); const cms = cmsOf({ site: SITE, token: TOKEN }); expect(await titleOf(cms.forDraft(withVariant))).toBe("Draft"); - expect(await titleOf(cms.forDraft(pointer))).toBe("Draft"); const [request, ...more] = api.draftFetches(); expect(more).toEqual([]); - expect(request!.url).toBe( - `${STUDIO}/api/acme/decofile/store/summer-sale/changes?token=${STUDIO_TOKEN}&v=9f3c1a`, - ); + expect(request!.url).toBe(`${DRAFTS}summer-sale.json?v=9f3c1a`); expect(request!.headers).not.toHaveProperty("cookie"); expect(request!.headers).not.toHaveProperty("authorization"); // never the site token expect(request!.init?.credentials).not.toBe("include"); @@ -401,11 +450,10 @@ describe("draft previews", () => { it("DP-3: a client captures production once; the next client gets the new release and fetches the draft again", async () => { const api = deliveryApi(); const pointer = api.draft({ set: { SummerSEO: seoEntry("Draft") } }); - const cms = cmsOf({ site: SITE, token: TOKEN }); + const cms = cmsOf({ site: SITE }); const first = cms.forDraft(pointer); expect(await titleOf(first)).toBe("Draft"); api.publish( - 1, await hashed("Published", { HelloWorld: { __resolveType: "post", name: "Republished", path: "/x", date: "2026-10-02" }, }), @@ -427,14 +475,18 @@ describe("draft previews", () => { const draft = cmsOf().forDraft(pointer); const revision = await draft.revision(); expect(revision).toContain("rev-1"); - expect(revision).toContain("9f3c1a"); + expect(revision).toContain('"etag-1"'); // the draft body's ETag: the pointer outlives saves expect(revision).not.toBe("rev-1"); + api.draft({ set: { SummerSEO: seoEntry("Saved again") } }); + const saved = cmsOf().forDraft(pointer); + expect(await titleOf(saved)).toBe("Saved again"); + expect(await saved.revision()).not.toBe(revision); }); it("DP-5: only *.decocms.com and loopback hosts by default; any other host, or a look-alike, is refused before anything is fetched", async () => { const api = deliveryApi(); const cms = cmsOf(); - const changes = "/api/acme/decofile/store/summer-sale/changes?token=t"; + const changes = "/sites/acme/drafts/summer-sale.json"; for (const pointer of [ `evil.example${changes}@1`, `studio.decocms.com.evil.example${changes}@1`, @@ -451,33 +503,36 @@ describe("draft previews", () => { expect({ pointer, ...(await failed(cms.forDraft(pointer))) }).toEqual({ pointer, ...FAILED }); } expect(api.fetch).not.toHaveBeenCalled(); - // The host is compared lowercase, so an uppercase pointer still reaches Studio. + // The host is compared lowercase, so an uppercase pointer still reaches the CDN. const upper = api .draft({ set: { SummerSEO: seoEntry("Draft") } }) - .replace(STUDIO_HOST, STUDIO_HOST.toUpperCase()); + .replace(DRAFT_HOST, DRAFT_HOST.toUpperCase()); expect(await titleOf(cms.forDraft(upper))).toBe("Draft"); }); - it("DP-6: DECO_PREVIEW_API_DOMAINS replaces the default list; content can't widen it", async () => { + it("DP-6: createCMS({ preview: { draftHosts } }) replaces the default list; no environment variable does; content can't widen it", async () => { const api = deliveryApi(); const pointer = api.draft({ set: { SummerSEO: seoEntry("Draft") } }); - vi.stubEnv("DECO_PREVIEW_API_DOMAINS", "studio.example.com"); - expect(await failed(cmsOf().forDraft(pointer))).toEqual(FAILED); + expect( + await failed(cmsOf({ preview: { draftHosts: ["studio.example.com"] } }).forDraft(pointer)), + ).toEqual(FAILED); expect(api.fetch).not.toHaveBeenCalled(); resetForTests(); - vi.stubEnv("DECO_PREVIEW_API_DOMAINS", " .decocms.com , studio.example.com "); + const both = cmsOf({ preview: { draftHosts: [" .decocms.com ", "studio.example.com"] } }); + expect(await titleOf(both.forDraft(pointer))).toBe("Draft"); + // DECO_PREVIEW_API_DOMAINS (v7's variable) is read by the site, if at all, never by the SDK. + resetForTests(); + vi.stubEnv("DECO_PREVIEW_API_DOMAINS", "studio.example.com"); expect(await titleOf(cmsOf().forDraft(pointer))).toBe("Draft"); + vi.unstubAllEnvs(); // Content can't widen it: the CMS block has no say over where drafts come from. resetForTests(); - vi.unstubAllEnvs(); const release = docsSnapshot(); release.blocks.CMS = { __resolveType: "cms-settings", preview: { hosts: ["*"] } }; const fromContent = cmsOf({ content: release }); - expect( - await failed( - fromContent.forDraft("evil.example/api/acme/decofile/store/x/changes?token=t@1"), - ), - ).toEqual(FAILED); + expect(await failed(fromContent.forDraft("evil.example/sites/acme/drafts/x.json@1"))).toEqual( + FAILED, + ); expect(api.requests.filter((r) => r.url.includes("evil.example"))).toEqual([]); }); @@ -487,7 +542,7 @@ describe("draft previews", () => { "fetch", vi.fn(async (input: string | URL | Request) => { urls.push(String(input)); - return Response.json({ format: 1, set: {}, delete: [] }); + return Response.json({ set: {}, delete: [] }); }), ); const cms = cmsOf(); @@ -517,14 +572,18 @@ describe("draft previews", () => { expect(await failed(cms.forDraft("[::1]:4000/changes?token=t@v2"))).toEqual(FAILED); expect(urls).toEqual([]); // Configured, it is a loopback domain like the others: plain HTTP, any port. - vi.stubEnv("DECO_PREVIEW_API_DOMAINS", "[::1]"); + resetForTests(); expect( - (await cmsOf().forDraft("[::1]:4000/changes?token=t@v3").revision()).endsWith("~v3"), + ( + await cmsOf({ preview: { draftHosts: ["[::1]"] } }) + .forDraft("[::1]:4000/changes?token=t@v3") + .revision() + ).endsWith("~v3"), ).toBe(true); expect(urls).toEqual(["http://[::1]:4000/changes?token=t&v=v3"]); }); - it("DP-8: only a 200 is accepted: an error status, or a redirect, is a failed draft", async () => { + it("DP-8: only a 200 (or a 304 to the ETag it sent) is accepted: an error status, or a redirect, is a failed draft", async () => { for (const status of [201, 204, 301, 302, 400, 401, 403, 404, 413, 500, 502]) { resetForTests(); const api = deliveryApi(); @@ -532,7 +591,7 @@ describe("draft previews", () => { api.respond("summer-sale", () => status === 204 ? new Response(null, { status }) - : new Response(JSON.stringify({ format: 1, set: {}, delete: [] }), { + : new Response(JSON.stringify({ set: {}, delete: [] }), { status, headers: status >= 300 && status < 400 ? { location: "https://evil.example/" } : {}, }), @@ -593,19 +652,18 @@ describe("draft previews", () => { expect(await result).toEqual(FAILED); }); - it("DP-11: the shape is checked: exactly format 1, set and delete, disjoint", async () => { + it("DP-11: the shape is checked: exactly { set, delete }, disjoint", async () => { const bodies: unknown[] = [ - { format: 2, set: {}, delete: [] }, - { set: {}, delete: [] }, - { format: 1, set: {}, delete: [], baseRevision: "x" }, - { format: 1, set: [], delete: [] }, - { format: 1, set: null, delete: [] }, - { format: 1, set: {} }, - { format: 1, set: {}, delete: "HelloWorld" }, - { format: 1, set: {}, delete: [1] }, - { format: 1, set: {}, delete: ["A", "A"] }, - { format: 1, set: { A: seoEntry("x") }, delete: ["A"] }, - [{ format: 1, set: {}, delete: [] }], + { format: 1, set: {}, delete: [] }, // the old git-branch form + { set: {}, delete: [], baseRevision: "x" }, + { set: [], delete: [] }, + { set: null, delete: [] }, + { set: {} }, + { delete: [] }, + { set: {}, delete: "HelloWorld" }, + { set: {}, delete: [1] }, + { set: { A: seoEntry("x") }, delete: ["A"] }, + [{ set: {}, delete: [] }], "changes", null, ]; @@ -622,8 +680,8 @@ describe("draft previews", () => { it("DP-12: a failed draft is LOADER_FAILED on every call, never published content, and isn't reused", async () => { const api = deliveryApi(); - api.publish(1, await hashed("Release")); - const cms = cmsOf({ site: SITE, token: TOKEN }); + api.publish(await hashed("Release")); + const cms = cmsOf({ site: SITE }); await cms.update(); const pointer = api.draft({ set: { SummerSEO: seoEntry("Draft") } }); api.respond("summer-sale", () => new Response("bad gateway", { status: 502 })); @@ -633,46 +691,54 @@ describe("draft previews", () => { const [, error] = await client.resolve("SummerSEO"); expect(String((error?.cause as Error | undefined)?.message)).toContain("HTTP 502"); api.respond("summer-sale", () => - Response.json({ format: 1, set: { SummerSEO: seoEntry("Back") }, delete: [] }), + Response.json({ set: { SummerSEO: seoEntry("Back") }, delete: [] }), ); expect(await titleOf(cms.forDraft(pointer))).toBe("Back"); expect(api.draftFetches()).toHaveLength(2); }); - it("DP-13: an expired or wrong token gets no content", async () => { + it("DP-13: a draft that isn't there (a wrong or deleted slug) gets no content", async () => { const api = deliveryApi(); - const pointer = api.draft({ set: { SummerSEO: seoEntry("Draft") } }, { token: "expired" }); - expect(await failed(cmsOf().forDraft(pointer))).toEqual(FAILED); + api.draft({ set: { SummerSEO: seoEntry("Draft") } }); + const wrong = `${DRAFT_HOST}/sites/acme/drafts/guessed.json@9f3c1a`; + expect(await failed(cmsOf().forDraft(wrong))).toEqual(FAILED); expect(api.draftFetches()).toHaveLength(1); }); - it("DP-14: a fetched draft is reused for up to a minute per pointer, at most three per CMS", async () => { - let now = 1_000_000; - vi.spyOn(Date, "now").mockImplementation(() => now); + it("DP-14: every read revalidates with If-None-Match; a 304 reuses the body; at most three bodies per CMS", async () => { const api = deliveryApi(); - const pointer = (branch: string) => - api.draft({ set: { SummerSEO: seoEntry(branch) } }, { branch }); + const pointer = (slug: string) => api.draft({ set: { SummerSEO: seoEntry(slug) } }, { slug }); const cms = cmsOf(); - expect(await titleOf(cms.forDraft(pointer("a")))).toBe("a"); - now += 59_000; - expect(await titleOf(cms.forDraft(pointer("a")))).toBe("a"); - expect(api.draftFetches()).toHaveLength(1); - now += 2_000; - await titleOf(cms.forDraft(pointer("a"))); - expect(api.draftFetches()).toHaveLength(2); // a minute passed: fetched (and its token checked) again - for (const branch of ["b", "c", "d"]) await titleOf(cms.forDraft(pointer(branch))); - expect(api.draftFetches()).toHaveLength(5); - await titleOf(cms.forDraft(pointer("a"))); // the fourth evicted the oldest - expect(api.draftFetches()).toHaveLength(6); - await titleOf(cms.forDraft(pointer("d"))); - expect(api.draftFetches()).toHaveLength(6); + const a = pointer("a"); + expect(await titleOf(cms.forDraft(a))).toBe("a"); + expect(api.draftFetches()[0]?.headers["if-none-match"]).toBeUndefined(); + expect(await titleOf(cms.forDraft(a))).toBe("a"); + expect(api.draftFetches()).toHaveLength(2); + expect(api.draftFetches()[1]?.headers["if-none-match"]).toBe('"etag-1"'); + expect((await api.fetch.mock.results.at(-1)?.value)?.status).toBe(304); + // A save gives the object a new ETag: the next read gets the new body (a 200). + api.draft({ set: { SummerSEO: seoEntry("a, saved") } }, { slug: "a" }); + expect(await titleOf(cms.forDraft(a))).toBe("a, saved"); + expect((await api.fetch.mock.results.at(-1)?.value)?.status).toBe(200); + for (const slug of ["b", "c", "d"]) await titleOf(cms.forDraft(pointer(slug))); + await titleOf(cms.forDraft(a)); // the fourth evicted the oldest: no ETag to send + expect(api.draftFetches().at(-1)?.headers["if-none-match"]).toBeUndefined(); + await titleOf(cms.forDraft(`${DRAFT_HOST}/sites/acme/drafts/d.json@9f3c1a`)); + expect(api.draftFetches().at(-1)?.headers["if-none-match"]).toBeDefined(); + }); + + it("DP-14b: a 304 to a request that sent no ETag is a failed draft", async () => { + const api = deliveryApi(); + const pointer = api.draft({}); + api.respond("summer-sale", () => new Response(null, { status: 304 })); + expect(await failed(cmsOf().forDraft(pointer))).toEqual(FAILED); }); it("DP-15: drafts need no site or token, and never send them", async () => { const api = deliveryApi(); const pointer = api.draft({ set: { SummerSEO: seoEntry("Draft") } }); expect(await titleOf(cmsOf().forDraft(pointer))).toBe("Draft"); - expect(api.requests.map((r) => new URL(r.url).host)).toEqual([STUDIO_HOST]); + expect(api.requests.map((r) => new URL(r.url).host)).toEqual([DRAFT_HOST]); resetForTests(); const hosted = deliveryApi(); const hostedPointer = hosted.draft({ set: { SummerSEO: seoEntry("Draft") } }); @@ -705,7 +771,7 @@ describe("draft previews", () => { expect(api.fetch).not.toHaveBeenCalled(); }); - it("DP-17: forced variants apply on top of the draft's changes; every variant shares one fetch", async () => { + it("DP-17: forced variants apply on top of the draft's changes; every variant shares one body", async () => { const api = deliveryApi(); const banner = { __resolveType: "multivariate", @@ -722,12 +788,17 @@ describe("draft previews", () => { expect(await cms.forDraft(forced(1)).resolve("Banner")).toEqual(["draft summer", null]); expect(await cms.forDraft(forced(0)).resolve("Banner")).toEqual(["draft fallback", null]); expect(await cms.forDraft(pointer).resolve("Banner")).toEqual([undefined, null]); // the rules - expect(api.draftFetches()).toHaveLength(1); + // One body for every variant: the first read downloads it, the others revalidate it (304). + expect(api.draftFetches().map((r) => r.headers["if-none-match"])).toEqual([ + undefined, + '"etag-1"', + '"etag-1"', + ]); }); - it("DP-18: empty changes (a branch before its first save) show production", async () => { + it("DP-18: empty changes (a draft before its first change) show production", async () => { const api = deliveryApi(); - const pointer = api.draft({}, { branch: "not-saved-yet" }); + const pointer = api.draft({}, { slug: "not-saved-yet" }); const cms = cmsOf(); expect(await titleOf(cms.forDraft(pointer))).toBe("Sunny!"); const [pages] = await cms.forDraft(pointer).list("page"); @@ -737,20 +808,19 @@ describe("draft previews", () => { it("DP-19: release checks don't depend on drafts: a draft client schedules the same check, never in front of it", async () => { const api = deliveryApi(); api.publish( - 1, await hashed("Published", { HelloWorld: { __resolveType: "post", name: "Republished", path: "/x", date: "2026-10-02" }, }), ); - const channel = api.gate("/sites/acme/channels/production.json"); // the background check hangs + const channel = api.gate("/sites/acme/latest.json"); // the background check hangs const pointer = api.draft({ set: { SummerSEO: seoEntry("Draft") } }); - const cms = cmsOf({ site: SITE, token: TOKEN }); + const cms = cmsOf({ site: SITE }); const first = cms.forDraft(pointer); expect((await first.list<{ name: string }>("post"))[0]?.map((p) => p.name)).toEqual([ "Hello, world", ]); // cold: the fallback is the base expect(api.assetFetches()).toBe(0); - expect(api.manifestFetches()).toBeLessThanOrEqual(1); // scheduled beside the draft, still pending + expect(api.latestFetches()).toBeLessThanOrEqual(1); // scheduled beside the draft, still pending channel.resolve(); await flush(); const next = cms.forDraft(pointer); @@ -773,7 +843,7 @@ describe("draft previews", () => { }); it("DP-21: a response that was redirected anyway (a fetch polyfill ignoring redirect: manual) is refused", async () => { - const changes = { format: 1, set: { SummerSEO: seoEntry("Elsewhere") }, delete: [] }; + const changes = { set: { SummerSEO: seoEntry("Elsewhere") }, delete: [] }; const redirected = (url: string, flag: boolean) => { const response = Response.json(changes); Object.defineProperty(response, "redirected", { value: flag }); @@ -782,7 +852,7 @@ describe("draft previews", () => { }; for (const [url, flag] of [ ["https://evil.example/changes", false], - [`https://${STUDIO_HOST}/elsewhere`, true], + [`https://${DRAFT_HOST}/elsewhere`, true], ] as const) { resetForTests(); const api = deliveryApi(); @@ -832,35 +902,6 @@ describe("releases-and-deployment", () => { expect(r).toBe("rev-2"); expect(n).toBe(2); }); - - it("RD-6: client.revision() and cms.forRevision(revision) read the same content", async () => { - let current = docsSnapshot("rev-1"); - const loader: Loader = { load: async () => current, update: async () => ({ updated: true }) }; - const cms = createCMS({ blocks: docsBlocks(), content: loader }); - const r = await cms.forRelease().revision(); - current = structuredClone(docsSnapshot("rev-2")); - (current.blocks.SummerSEO as { title: string }).title = "Rev 2"; - await cms.update(); - expect(await titleOf(cms.forRelease())).toBe("Rev 2"); - const pinned = cms.forRevision(r); - expect(await pinned.revision()).toBe("rev-1"); - expect(await titleOf(pinned)).toBe("Sunny!"); - }); - - it("RD-7: a revision unlocks nothing: forRevision() never reaches the draft", async () => { - const api = deliveryApi(); - const pointer = api.draft({ set: { SummerSEO: seoEntry("Secret draft") } }); - const cms = createCMS({ - blocks: docsBlocks(), - content: docsSnapshot(), - site: SITE, - token: TOKEN, - }); - const draft = cms.forDraft(pointer); - expect(await titleOf(draft)).toBe("Secret draft"); - const byRevision = cms.forRevision(await draft.revision()); - expect(await titleOf(byRevision)).toBe("Sunny!"); - }); }); // --------------------------------------------------------------------------- @@ -880,65 +921,63 @@ describe("hosted", () => { expect(await cms.forRelease().revision()).toBe("rev-1"); }); - it("H-3: with site or token undefined the CMS reads content only, with no network", async () => { + it("H-3: without site the CMS reads content only, with no delivery request; a token without site is a configuration error", async () => { const api = deliveryApi(); - for (const [site, token] of [ - [SITE, undefined], - [undefined, TOKEN], - ] as const) { + for (const site of [undefined, ""]) { resetForTests(); const content = docsSnapshot(); - const cms = createCMS({ blocks: docsBlocks(), content, site, token, telemetry: false }); + const cms = createCMS({ blocks: docsBlocks(), content, site, telemetry: false }); expect(await cms.forRelease().revision()).toBe(content.revision); await cms.update(); await flush(); } + for (const site of [undefined, ""]) { + resetForTests(); + expect(() => + createCMS({ blocks: docsBlocks(), content: docsSnapshot(), site, token: TOKEN }), + ).toThrow("token needs site: pass both, or site alone"); + } expect(api.fetch).not.toHaveBeenCalled(); }); - it("H-4/HRI-1: with site and token, content is the fallback while the API is unreachable", async () => { + it("H-4/HRI-1: with site, content is the fallback while delivery is unreachable", async () => { const api = deliveryApi(); api.fail(true); const content = docsSnapshot(); - const cms = createCMS({ blocks: docsBlocks(), content, site: SITE, token: TOKEN }); + const cms = createCMS({ blocks: docsBlocks(), content, site: SITE }); const client = cms.forRelease(); const [entries, error] = await client.list("seo"); expect(error).toBeNull(); expect(entries).toHaveLength(1); expect(await titleOf(client)).toBe("Sunny!"); await expect(cms.update()).resolves.toEqual({ updated: false }); - expect(api.manifestFetches()).toBeGreaterThan(0); // remoteLoader wrapped it + expect(api.latestFetches()).toBeGreaterThan(0); // remoteLoader wrapped it }); it("H-5: in development, releases stay on local files but ?__draft= still loads", async () => { vi.stubEnv("NODE_ENV", "development"); const api = deliveryApi(); - api.publish(1, await hashed("Published")); + api.publish(await hashed("Published")); const pointer = api.draft({ set: { SummerSEO: seoEntry("Draft") } }); const cms = createCMS({ blocks: docsBlocks(), content: docsSnapshot(), site: SITE, - token: TOKEN, }); cms.forRelease(); await flush(); await cms.update(); - expect(api.manifestFetches()).toBe(0); + expect(api.latestFetches()).toBe(0); expect(await titleOf(cms.forRelease())).toBe("Sunny!"); expect(await titleOf(cms.forDraft(pointer))).toBe("Draft"); }); - it("H-6: site and token alone never send telemetry", async () => { - vi.stubEnv("OTEL_EXPORTER_OTLP_ENDPOINT", ""); + it("H-6: site alone never sends telemetry; site reads no environment variable", async () => { + vi.stubEnv("OTEL_EXPORTER_OTLP_ENDPOINT", "https://otel.example.com"); + vi.stubEnv("DECO_SITE_TOKEN", TOKEN); const api = deliveryApi(); - api.publish(1, await hashed("Published")); - const cms = createCMS({ - blocks: docsBlocks(), - content: docsSnapshot(), - site: SITE, - token: TOKEN, - }); + api.publish(await hashed("Published")); + const cms = createCMS({ blocks: docsBlocks(), content: docsSnapshot(), site: SITE }); await cms.update(); const client = cms.forRelease(); await client.resolve("SummerPage"); @@ -947,10 +986,11 @@ describe("hosted", () => { for (const { url } of api.requests) expect(url.startsWith(`${ORIGIN}/`)).toBe(true); }); - it("H-12: telemetry { site, token } goes to the hosted collector; analytics defaults to the hosted one", async () => { - const destination = resolveDestination({ site: SITE, token: TOKEN }); - expect(destination?.endpoint).toMatch(/^https:\/\/[^/]*decocms\.com/); + it("H-12: a token sends telemetry to the hosted collector as a Bearer token; analytics defaults to the hosted one", async () => { + const destination = resolveDestination(undefined, SITE, TOKEN); + expect(destination?.endpoint).toBe("https://otel.decocms.com"); expect(destination?.headers.authorization).toBe(`Bearer ${TOKEN}`); + expect(resolveDestination(undefined, SITE)).toBeNull(); const cms = createCMS({ blocks: docsBlocks(), content: docsSnapshot() }); const { analytics } = await cms.settings(); expect(analytics.collector).toBe(HOSTED_ANALYTICS_COLLECTOR); @@ -970,7 +1010,6 @@ describe("hosted-publishing", () => { blocks: docsBlocks(), content: docsSnapshot(), site: SITE, - token: TOKEN, }); void cms.update(); // a check that never finishes const client = cms.forRelease(); @@ -985,11 +1024,10 @@ describe("hosted-publishing", () => { blocks: docsBlocks(), content: docsSnapshot(), site: SITE, - token: TOKEN, }); const before = cms.forRelease(); expect(await titleOf(before)).toBe("Sunny!"); - api.publish(1, await hashed("New release")); + api.publish(await hashed("New release")); await cms.update(); expect(await titleOf(before)).toBe("Sunny!"); expect(await titleOf(cms.forRelease())).toBe("New release"); @@ -1017,11 +1055,10 @@ describe("hosted-publishing", () => { blocks: docsBlocks(), content: docsSnapshot(), site: SITE, - token: TOKEN, }); await expect(cms.update()).resolves.toEqual({ updated: false }); api.fail(false); - api.publish(1, await hashed("Now")); + api.publish(await hashed("Now")); await expect(cms.update()).resolves.toEqual({ updated: true }); expect(await titleOf(cms.forRelease())).toBe("Now"); }); @@ -1082,26 +1119,21 @@ describe("hosted-publishing", () => { expect(await measured({ interval: 120_000 })).toBe(120_000); }); - it("HP-6: interval defaults to DECO_CONTENT_INTERVAL, else 60 000 ms", async () => { + it("HP-6: interval defaults to 60 000 ms; DECO_CONTENT_INTERVAL isn't read", async () => { + expect(await measured()).toBe(60_000); + resetForTests(); + vi.stubEnv("DECO_CONTENT_INTERVAL", "90000"); expect(await measured()).toBe(60_000); resetForTests(); - process.env.DECO_CONTENT_INTERVAL = "90000"; - expect(await measured()).toBe(90_000); + expect(await measured({ interval: 90_000 })).toBe(90_000); }); - it("HP-6: values below 60 000 ms, from interval or DECO_CONTENT_INTERVAL, are raised with a warning", async () => { + it("HP-6: values below 60 000 ms are raised with a warning", async () => { const warn = vi.spyOn(console, "warn").mockImplementation(() => {}); expect(await measured({ interval: 1000 })).toBe(60_000); expect(warn).toHaveBeenCalledWith( expect.stringContaining("interval 1000 ms is below the minimum; raised to 60000 ms"), ); - resetForTests(); - warn.mockClear(); - process.env.DECO_CONTENT_INTERVAL = "1000"; - expect(await measured()).toBe(60_000); - expect(warn).toHaveBeenCalledWith( - expect.stringContaining("interval 1000 ms is below the minimum; raised to 60000 ms"), - ); }); it("HP-7/HRI-15: any loader with update() is checked on first use and then every interval", async () => { @@ -1128,36 +1160,30 @@ describe("hosted-publishing", () => { }); }); - it("HP-9: an unchanged manifest costs one small request, no snapshot download", async () => { + it("HP-9: an unchanged latest.json costs one small request, no revision download", async () => { const api = deliveryApi(); - api.publish(1, await hashed("Once")); - const cms = createCMS({ - blocks: docsBlocks(), - content: docsSnapshot(), - site: SITE, - token: TOKEN, - }); + api.publish(await hashed("Once")); + const cms = createCMS({ blocks: docsBlocks(), content: docsSnapshot(), site: SITE }); await cms.update(); const assets = api.assetFetches(); await cms.update(); await cms.update(); expect(api.assetFetches()).toBe(assets); - expect(api.manifestFetches()).toBe(3); + expect(api.latestFetches()).toBe(3); }); it("HP-10: a network error or an unparseable snapshot leaves memory as it was", async () => { const api = deliveryApi(); const good = await hashed("Good"); - api.publish(1, good); + api.publish(good); const cms = createCMS({ blocks: docsBlocks(), content: docsSnapshot(), site: SITE, - token: TOKEN, }); await cms.update(); const broken = await hashed("Broken"); - api.point(2, broken.revision); + api.point(broken.revision); api.asset(api.assetPath(broken.revision), new Response("{not json", { status: 200 })); await expect(cms.update()).resolves.toEqual({ updated: false }); api.fail(true); @@ -1171,12 +1197,10 @@ describe("hosted-publishing", () => { blocks: docsBlocks(), content: docsSnapshot(), site: SITE, - token: TOKEN, }); const client = cms.forRelease(); const [first] = await client.list("seo"); api.publish( - 1, await hashed("New", { Other: { __resolveType: "seo", title: "o", description: "o" } }), ); await cms.update(); @@ -1185,27 +1209,15 @@ describe("hosted-publishing", () => { expect(await client.revision()).toBe("rev-1"); }); - it("HP-12/HRI-7: a manifest naming the content module's revision is served from the bundle", async () => { - const api = deliveryApi(); - const content = await contentModule(); - api.point(7, content.revision); - const cms = createCMS({ blocks: docsBlocks(), content, site: SITE, token: TOKEN }); - await cms.update(); - expect(api.assetFetches()).toBe(0); - expect(await cms.forRelease().revision()).toBe(content.revision); - }); - it("HP-13: a type the deployed code lacks fails that block with UNKNOWN_BLOCK", async () => { const api = deliveryApi(); api.publish( - 1, await hashed("x", { NewBanner: { __resolveType: "brand-new-banner", title: "t" } }), ); const cms = createCMS({ blocks: docsBlocks(), content: docsSnapshot(), site: SITE, - token: TOKEN, }); await cms.update(); const client = cms.forRelease(); @@ -1216,41 +1228,22 @@ describe("hosted-publishing", () => { expect(seo).toMatchObject({ title: "x" }); }); - it("HP-15: the bundled content stays in memory beside the live release", async () => { - const api = deliveryApi(); - const content = await contentModule(); - const cms = createCMS({ blocks: docsBlocks(), content, site: SITE, token: TOKEN }); - api.publish(1, await hashed("Live")); - await cms.update(); - expect(await titleOf(cms.forRelease())).toBe("Live"); - const downloads = api.assetFetches(); - api.point(2, content.revision); // roll back to what the build shipped - await cms.update(); - expect(api.assetFetches()).toBe(downloads); - expect(await cms.forRelease().revision()).toBe(content.revision); - }); - - it("HP-16: remoteLoader(content, { site, token }) is a loader whose load() reports the served revision", async () => { + it("HP-16: remoteLoader(content, { site }) is a loader whose load() reports the served revision", async () => { deliveryApi(); const content = docsSnapshot(); - const connected = remoteLoader(content, { site: SITE, token: TOKEN }) as Loader; + const connected = remoteLoader(content, { site: SITE }) as Loader; expect((await connected.load()).revision).toBe(content.revision); - // The docs' troubleshooting snippet passes process.env values, which can be undefined. - const env = { DECO_SITE: undefined, DECO_SITE_TOKEN: undefined } as Record< - string, - string | undefined - >; - const loader = remoteLoader(content, { - site: env.DECO_SITE as string, - token: env.DECO_SITE_TOKEN as string, - }); + // The site reads its own env and passes it; an unset value is a plain loader. + const env = { DECO_SITE: undefined } as Record; + const loader = remoteLoader(content, { site: env.DECO_SITE as string }); expect(typeof (loader as Loader).load).toBe("function"); + expect((loader as Loader).update).toBeUndefined(); }); it("HP-17: a server that hasn't fetched a release reports its fallback's revision", async () => { deliveryApi().fail(true); const content = await contentModule(); - const cms = createCMS({ blocks: docsBlocks(), content, site: SITE, token: TOKEN }); + const cms = createCMS({ blocks: docsBlocks(), content, site: SITE }); expect(await cms.forRelease().revision()).toBe(content.revision); }); }); @@ -1273,18 +1266,14 @@ describe("hosted-drafts", () => { expect(await draftPointer(new Request(link))).toBe(raw); }); - it("HD-2: forDraft fetches the pointer once and serves it from memory afterwards", async () => { + it("HD-2: forDraft revalidates the draft on every read; an unchanged one is a 304 served from memory", async () => { const api = deliveryApi(); const pointer = api.draft({ set: { SummerSEO: seoEntry("Draft") } }); - const cms = createCMS({ - blocks: docsBlocks(), - content: docsSnapshot(), - site: SITE, - token: TOKEN, - }); + const cms = createCMS({ blocks: docsBlocks(), content: docsSnapshot(), site: SITE }); expect(await titleOf(cms.forDraft(pointer))).toBe("Draft"); expect(await titleOf(cms.forDraft(pointer))).toBe("Draft"); - expect(api.draftFetches()).toHaveLength(1); + expect(api.draftFetches()).toHaveLength(2); + expect(api.draftFetches()[1]?.headers["if-none-match"]).toBe('"etag-1"'); }); it("HD-3: a draft holds only changed blocks and deletions; every other block is inherited", async () => { @@ -1297,7 +1286,6 @@ describe("hosted-drafts", () => { blocks: docsBlocks(), content: docsSnapshot(), site: SITE, - token: TOKEN, }); const draft = cms.forDraft(pointer); expect(await titleOf(draft)).toBe("Draft"); @@ -1316,7 +1304,6 @@ describe("hosted-drafts", () => { blocks: docsBlocks(), content: docsSnapshot(), site: SITE, - token: TOKEN, }); const client = cms.forDraft(POINTER); const [value, error] = await client.resolve("SummerSEO"); @@ -1356,7 +1343,6 @@ describe("hosted-drafts", () => { blocks: docsBlocks(), content: docsSnapshot(), site: SITE, - token: TOKEN, }); const render = async (client: ReturnType, _request: Request) => new Response(await titleOf(client)); @@ -1382,12 +1368,12 @@ describe("hosted-drafts", () => { expect(await visitor.text()).toBe("Sunny!"); }); - it("HD-8: without site and token, forDraft layers the draft over the content module; only Studio is asked", async () => { + it("HD-8: without site and token, forDraft layers the draft over the content module; only the draft's host is asked", async () => { const api = deliveryApi(); const pointer = api.draft({ set: { SummerSEO: seoEntry("Draft") } }); const cms = createCMS({ blocks: docsBlocks(), content: docsSnapshot() }); expect(await titleOf(cms.forDraft(pointer))).toBe("Draft"); - expect(api.requests.map((r) => new URL(r.url).host)).toEqual([STUDIO_HOST]); + expect(api.requests.map((r) => new URL(r.url).host)).toEqual([DRAFT_HOST]); }); it("HD-10: __deco_draft is the cookie draftCookie writes and draftPointer reads", async () => { @@ -1422,7 +1408,6 @@ describe("hosted-drafts", () => { blocks: docsBlocks(), content: docsSnapshot(), site: SITE, - token: TOKEN, }); const url = `mystore://preview?__draft=${encodeURIComponent(POINTER)}`; const pointer = new URL(url).searchParams.get("__draft"); @@ -1442,7 +1427,6 @@ describe("hosted-drafts", () => { blocks: docsBlocks(), content: docsSnapshot(), site: SITE, - token: TOKEN, }); const [value, error] = await cms.forDraft("evil.example/x?token=t@1").resolve("SummerSEO"); expect(value).toBeNull(); @@ -1456,9 +1440,8 @@ describe("hosted-drafts", () => { blocks: docsBlocks(), content: docsSnapshot(), site: SITE, - token: TOKEN, }); - for (const pointer of ["garbage", "https://x/y@1", "@1", `${STUDIO_HOST}/x@`]) { + for (const pointer of ["garbage", "https://x/y@1", "@1", `${DRAFT_HOST}/x@`]) { const [value, error] = await cms.forDraft(pointer).list("seo"); expect(value).toBeNull(); expect(error?.code).toBe("LOADER_FAILED"); @@ -1480,7 +1463,7 @@ describe("hosted-drafts", () => { __resolveType: "cms-settings", preview: { hosts: ["staging.store.example.com"] }, }; - const cms = createCMS({ blocks: docsBlocks(), content: release, site: SITE, token: TOKEN }); + const cms = createCMS({ blocks: docsBlocks(), content: release, site: SITE }); const pick = async (request: Request) => { const pointer = await cms.draftPointer(request); const cookie = await cms.draftCookie(request); @@ -1523,7 +1506,7 @@ describe("CMS settings from hosted releases", () => { it("S-1: offline, at boot, settings come from the bundled content module, with no fetch", async () => { const api = deliveryApi(); api.fail(true); - const cms = createCMS({ blocks: docsBlocks(), content: bundled(), site: SITE, token: TOKEN }); + const cms = createCMS({ blocks: docsBlocks(), content: bundled(), site: SITE }); const settings = await cms.settings(); expect(settings.preview.hosts).toEqual(["staging.example.com"]); expect(settings.analytics.collector).toBe("https://bundled.example/events"); @@ -1543,8 +1526,8 @@ describe("CMS settings from hosted releases", () => { const next = await hashed("Published", { CMS: { __resolveType: "cms-settings", preview: { hosts: ["www.example.com"] } }, }); - api.publish(1, next); - const cms = createCMS({ blocks: docsBlocks(), content: bundled(), site: SITE, token: TOKEN }); + api.publish(next); + const cms = createCMS({ blocks: docsBlocks(), content: bundled(), site: SITE }); expect((await cms.settings()).preview.hosts).toEqual(["staging.example.com"]); expect(api.fetch).not.toHaveBeenCalled(); expect(await cms.update()).toEqual({ updated: true }); @@ -1557,7 +1540,7 @@ describe("CMS settings from hosted releases", () => { // --------------------------------------------------------------------------- describe("hosted-releases-internals", () => { - it("HRI-3: update() runs on first use, then a manifest check every interval", async () => { + it("HRI-3: update() runs on first use, then a latest.json check every interval", async () => { let now = 1_000_000; vi.spyOn(Date, "now").mockImplementation(() => now); vi.spyOn(Math, "random").mockReturnValue(0.5); @@ -1566,16 +1549,15 @@ describe("hosted-releases-internals", () => { blocks: docsBlocks(), content: docsSnapshot(), site: SITE, - token: TOKEN, }); - expect(api.manifestFetches()).toBe(0); // nothing at construction + expect(api.latestFetches()).toBe(0); // nothing at construction cms.forRelease(); await flush(); - expect(api.manifestFetches()).toBe(1); + expect(api.latestFetches()).toBe(1); now += 60_000; cms.forRelease(); await flush(); - expect(api.manifestFetches()).toBe(2); + expect(api.latestFetches()).toBe(2); }); describe("HRI-4: idle scheduling", () => { @@ -1640,65 +1622,41 @@ describe("hosted-releases-internals", () => { expect(sdk).not.toMatch(/webhook|addEventListener\(|createServer/i); }); - it("HRI-6: a manifest older than the newest observed generation is ignored", async () => { + it("HRI-6: no ordering among pointers: an older publishedAt is followed like any other (it's compared only with the bundle's committedAt)", async () => { const api = deliveryApi(); const loader = remote(docsSnapshot()); const five = await hashed("Five"); - api.publish(5, five); + const four = await hashed("Four"); + api.publish(five); await loader.update?.(); - api.publish(4, await hashed("Four")); - expect(await loader.update?.()).toEqual({ updated: false }); - expect((await loader.load()).revision).toBe(five.revision); - }); - - it("HRI-7: a same-revision manifest adopts its generation without downloading", async () => { - const api = deliveryApi(); - const loader = remote(docsSnapshot()); - const r = await hashed("R"); - api.publish(5, r); - await loader.update?.(); - const downloads = api.assetFetches(); - api.point(6, r.revision); - await loader.update?.(); - expect(api.assetFetches()).toBe(downloads); - api.publish(5, await hashed("Stale gen 5")); // older than the adopted 6 - await loader.update?.(); - expect((await loader.load()).revision).toBe(r.revision); + api.publish(four); + api.setLatest({ + revision: four.revision, + schemaHash: SCHEMA, + publishedAt: "1970-01-01T00:00:00Z", + }); + expect(await loader.update?.()).toEqual({ updated: true }); + expect((await loader.load()).revision).toBe(four.revision); + const sdk = fs.readFileSync(path.join(HERE, "../remoteLoader.ts"), "utf8"); + expect(sdk).not.toMatch(/generation|channels/); }); - it("HRI-8: a snapshot that doesn't hash to its revision is rejected", async () => { + it("HRI-8: a revision object that isn't the one latest.json names is rejected", async () => { const api = deliveryApi(); const real = await hashed("Real"); - const tampered = { revision: real.revision, blocks: { Other: { __resolveType: "seo" } } }; - api.publish(1, real, tampered); + const wrong = { revision: "e".repeat(40), schemaHash: SCHEMA, blocks: real.blocks }; + api.publish(real, wrong); const fallback = docsSnapshot(); const loader = remote(fallback); - await expect(loader.update?.()).rejects.toThrow(/doesn't match/); + await expect(loader.update?.()).rejects.toThrow(/not the revision latest\.json names/); expect(await loader.load()).toBe(fallback); }); - it("HRI-8: a slower earlier check can't undo a newer promotion", async () => { - const api = deliveryApi(); - const loader = remote(docsSnapshot()); - const one = await hashed("Gen 1"); - const two = await hashed("Gen 2"); - api.publish(1, one); - const gate = api.gate(api.assetPath(one.revision)); - const slow = loader.update?.(); - for (let i = 0; i < 50 && api.assetFetches() === 0; i++) await flush(2); - expect(api.assetFetches()).toBe(1); - api.publish(2, two); - await loader.update?.(); - gate.resolve(); - expect(await slow).toEqual({ updated: false }); - expect((await loader.load()).revision).toBe(two.revision); - }); - it("HRI-9: a loader with its own revision scheme is downloaded once", async () => { const api = deliveryApi(); const custom: Loader = { load: async () => docsSnapshot("my-own-scheme-1") }; const loader = remote(custom); - api.publish(1, await hashed("Hosted")); + api.publish(await hashed("Hosted")); await loader.update?.(); await loader.update?.(); await loader.update?.(); @@ -1707,12 +1665,11 @@ describe("hosted-releases-internals", () => { it("HRI-10/HD-4: a draft that can't be fetched is LOADER_FAILED on every call; published content is never substituted", async () => { const api = deliveryApi(); - api.publish(1, await hashed("Release")); + api.publish(await hashed("Release")); const cms = createCMS({ blocks: docsBlocks(), content: docsSnapshot(), site: SITE, - token: TOKEN, }); await cms.update(); const pointer = api.draft({ set: { SummerSEO: seoEntry("Draft") } }); @@ -1740,19 +1697,17 @@ describe("hosted-releases-internals", () => { expect(keys.join("\n")).not.toContain(TOKEN); }); - it("HRI-12: the key is site, token and content identity, never the revision", async () => { + it("HRI-12: the key is the site and the content identity, never the revision or the token", async () => { const blocks = docsBlocks(); const a = createCMS({ blocks, content: { ...docsSnapshot("rev-1"), root: ".deco" }, site: SITE, - token: TOKEN, }); const reloaded = createCMS({ blocks, content: { ...docsSnapshot("rev-2"), root: ".deco" }, site: SITE, - token: TOKEN, }); expect(reloaded).toBe(a); expect(await reloaded.forRelease().revision()).toBe("rev-2"); @@ -1760,14 +1715,12 @@ describe("hosted-releases-internals", () => { blocks, content: { ...docsSnapshot("rev-2"), root: ".deco" }, site: "other", - token: TOKEN, }); expect(other).not.toBe(a); const otherFolder = createCMS({ blocks, content: { ...docsSnapshot("rev-2"), root: "apps/b/.deco" }, site: SITE, - token: TOKEN, }); expect(otherFolder).not.toBe(a); }); @@ -1783,11 +1736,109 @@ describe("hosted-releases-internals", () => { it("HRI-14: resetForTests() clears every stored instance", () => { const content = { ...docsSnapshot(), root: ".deco" }; - const a = createCMS({ blocks: docsBlocks(), content, site: SITE, token: TOKEN }); - const loaderA = remoteLoader(content, { site: SITE, token: TOKEN }); + const a = createCMS({ blocks: docsBlocks(), content, site: SITE }); + const loaderA = remoteLoader(content, { site: SITE }); resetForTests(); - expect(createCMS({ blocks: docsBlocks(), content, site: SITE, token: TOKEN })).not.toBe(a); - expect(remoteLoader(content, { site: SITE, token: TOKEN })).not.toBe(loaderA); + expect(createCMS({ blocks: docsBlocks(), content, site: SITE })).not.toBe(a); + expect(remoteLoader(content, { site: SITE })).not.toBe(loaderA); expect(typeof (root as Record).resetForTests).toBe("function"); }); + + describe("HRI-16..HRI-21: whoever is newer wins (latest.json publishedAt vs the bundle's committedAt)", () => { + const at = (hour: number) => `2026-10-07T${String(hour).padStart(2, "0")}:00:00.000Z`; + /** The content module `deco content` wrote from a commit made at `committedAt`. */ + const built = (committedAt?: string): Snapshot => + committedAt === undefined ? docsSnapshot() : { ...docsSnapshot(), committedAt }; + /** Studio's Publish / Make current / Resync, at `publishedAt`. */ + async function release( + api: ReturnType, + title: string, + publishedAt: string, + schemaHash = SCHEMA, + ) { + const snapshot = await hashed(title); + api.publish(snapshot, undefined, schemaHash); + api.setLatest({ revision: snapshot.revision, schemaHash, publishedAt }); + return snapshot; + } + + it("HRI-16: publish, then deploy: the bundle wins", async () => { + const api = deliveryApi(); + await release(api, "Published", at(10)); + const cms = createCMS({ blocks: docsBlocks(), content: built(at(11)), site: SITE }); + expect(await cms.update()).toEqual({ updated: false }); + expect(api.assetFetches()).toBe(0); + expect(await titleOf(cms.forRelease())).toBe("Sunny!"); + }); + + it("HRI-17: deploy, then publish: the CDN wins", async () => { + const api = deliveryApi(); + await release(api, "Published", at(12)); + const cms = createCMS({ blocks: docsBlocks(), content: built(at(11)), site: SITE }); + expect(await cms.update()).toEqual({ updated: true }); + expect(await titleOf(cms.forRelease())).toBe("Published"); + }); + + it("HRI-18: rollback after deploy: Make current writes publishedAt = now, so the CDN wins", async () => { + const api = deliveryApi(); + const old = await release(api, "Old", at(9)); + const cms = createCMS({ blocks: docsBlocks(), content: built(at(11)), site: SITE }); + expect(await cms.update()).toEqual({ updated: false }); + api.setLatest({ revision: old.revision, schemaHash: SCHEMA, publishedAt: at(12) }); + expect(await cms.update()).toEqual({ updated: true }); + expect(await titleOf(cms.forRelease())).toBe("Old"); + }); + + it("HRI-19: deploy after rollback: the bundle wins", async () => { + const api = deliveryApi(); + const old = await release(api, "Old", at(9)); + api.setLatest({ revision: old.revision, schemaHash: SCHEMA, publishedAt: at(12) }); + const cms = createCMS({ blocks: docsBlocks(), content: built(at(13)), site: SITE }); + expect(await cms.update()).toEqual({ updated: false }); + expect(await titleOf(cms.forRelease())).toBe("Sunny!"); + }); + + it("HRI-20: a newer release built for another schema: the bundle wins", async () => { + const api = deliveryApi(); + await release(api, "Published", at(12), "6".repeat(64)); + const cms = createCMS({ blocks: docsBlocks(), content: built(at(11)), site: SITE }); + expect(await cms.update()).toEqual({ updated: false }); + expect(api.assetFetches()).toBe(0); + expect(await titleOf(cms.forRelease())).toBe("Sunny!"); + }); + + it("HRI-21: a bundle without committedAt (custom loader, older module, built outside git) is the oldest: the CDN wins", async () => { + const api = deliveryApi(); + await release(api, "Published", "1970-01-01T00:00:00.000Z"); + const cms = createCMS({ blocks: docsBlocks(), content: built(), site: SITE }); + expect(await cms.update()).toEqual({ updated: true }); + expect(await titleOf(cms.forRelease())).toBe("Published"); + resetForTests(); + const custom: Loader = { load: async () => built() }; + const viaLoader = createCMS({ blocks: docsBlocks(), content: remote(custom) }); + expect(await viaLoader.update()).toEqual({ updated: true }); + }); + + it("HRI-22: a build of an older commit that finishes after a publish loses to the CDN", async () => { + const api = deliveryApi(); + // Commit at 10:00, publish at 11:00, the slow build of that commit finishes at 12:00: + // the stamp is the commit time, so the build finishing last doesn't make it newer. + await release(api, "Published", at(11)); + const cms = createCMS({ blocks: docsBlocks(), content: built(at(10)), site: SITE }); + expect(await cms.update()).toEqual({ updated: true }); + expect(await titleOf(cms.forRelease())).toBe("Published"); + }); + + it("HRI-16..22: the SDK reads no git API and no environment besides NODE_ENV", () => { + const sdk = fs.readFileSync(path.join(HERE, "../remoteLoader.ts"), "utf8"); + expect(sdk).not.toMatch(/github|rev-parse|GIT_[A-Z]/i); + expect(new Set([...sdk.matchAll(/process\.env\.([A-Z_]+)/g)].map((m) => m[1]))).toEqual( + new Set(["NODE_ENV"]), + ); + const cli = fs.readFileSync(path.join(HERE, "../cli/content.ts"), "utf8"); + // The CLI stamps the commit time of git HEAD; it reads no environment. + expect(cli).toMatch(/execFileSync\("git", \["log", "-1", "--format=%cI"\]/); + expect(cli).not.toMatch(/process\.env/); + }); + }); }); diff --git a/packages/blocks/src/v8/__conformance__/extra.test.ts b/packages/blocks/src/v8/__conformance__/extra.test.ts index 4872a891..1b66bccf 100644 --- a/packages/blocks/src/v8/__conformance__/extra.test.ts +++ b/packages/blocks/src/v8/__conformance__/extra.test.ts @@ -15,7 +15,7 @@ import { createInstrumentedFetch } from "../fetch"; import { createCMS, matchRoute, parseDraftPointer, resetForTests } from "../index"; import { resolveDestination, setCurrentTelemetry } from "../telemetry"; import { docsBlocks, docsSnapshot } from "../testFixtures"; -import type { Blocks, Lazy, Loader, Snapshot } from "../types"; +import type { Blocks, Lazy, Snapshot } from "../types"; beforeEach(() => resetForTests()); afterEach(() => { @@ -407,27 +407,6 @@ describe("clients and content (api-reference.mdx, content.mdx)", () => { expect(viaAlias).toHaveLength(3); }); - it("X21 api-reference: forRevision pins to a served revision; an unknown revision (or a draft's) behaves like the release", async () => { - let current = docsSnapshot("rev-1"); - const loader: Loader = { - load: async () => current, - update: async () => ({ updated: true }), - }; - vi.stubGlobal("fetch", async () => Response.json({ format: 1, set: {}, delete: [] })); - const cms = createCMS({ blocks: docsBlocks(), content: loader }); - expect(await cms.forRelease().revision()).toBe("rev-1"); - const draftRevision = await cms - .forDraft("studio.decocms.com/api/acme/decofile/store/x/changes?token=t@v1") - .revision(); - expect(draftRevision).toBe("rev-1~v1"); - current = docsSnapshot("rev-2"); - await cms.update(); - expect(await cms.forRelease().revision()).toBe("rev-2"); - expect(await cms.forRevision("rev-1").revision()).toBe("rev-1"); - expect(await cms.forRevision("nope").revision()).toBe("rev-2"); - expect(await cms.forRevision(draftRevision).revision()).toBe("rev-2"); - }); - it("X22 api-reference › Loaders: a loader without update() is asked on every client; update() never throws", async () => { let n = 0; const cms = createCMS({ @@ -480,9 +459,9 @@ describe("clients and content (api-reference.mdx, content.mdx)", () => { }); }); - it("X25 api-reference: site and token never send telemetry by themselves", () => { - vi.stubEnv("OTEL_EXPORTER_OTLP_ENDPOINT", ""); + it("X25 api-reference: site never sends telemetry by itself; a token does, to the hosted collector", () => { expect(resolveDestination(undefined, "acme")).toBeNull(); + expect(resolveDestination(undefined, "acme", "t")?.endpoint).toBe("https://otel.decocms.com"); }); }); diff --git a/packages/blocks/src/v8/__conformance__/guides.test.ts b/packages/blocks/src/v8/__conformance__/guides.test.ts index e6715b4c..7a8ef677 100644 --- a/packages/blocks/src/v8/__conformance__/guides.test.ts +++ b/packages/blocks/src/v8/__conformance__/guides.test.ts @@ -594,27 +594,30 @@ describe("renames and migrations", () => { expect(a).not.toHaveBeenCalled(); }); - it("mig-08/tr-18: telemetry option wins; env OTEL_EXPORTER_OTLP_* when left out; false disables", () => { + it("mig-08/tr-18: telemetry is code's params only; v7's OTEL_*/DECO_OTEL_* variables aren't read; false disables", () => { vi.stubEnv("OTEL_EXPORTER_OTLP_ENDPOINT", "https://env.example.com"); - vi.stubEnv("OTEL_EXPORTER_OTLP_HEADERS", "x-api-key=abc%20d"); - expect(resolveDestination(undefined)).toMatchObject({ - endpoint: "https://env.example.com", + vi.stubEnv("DECO_OTEL_HEADERS", "x-api-key=abc%20d"); + expect(resolveDestination(undefined)).toBeNull(); + expect( + resolveDestination({ + endpoint: "https://code.example.com", + headers: { "x-api-key": "abc d" }, + resource: { "service.version": "abc123" }, + }), + ).toMatchObject({ + endpoint: "https://code.example.com", headers: { "x-api-key": "abc d" }, + resource: { "service.version": "abc123" }, }); - expect( - resolveDestination({ endpoint: "https://code.example.com", headers: { a: "b" } }), - ).toMatchObject({ endpoint: "https://code.example.com", headers: { a: "b" } }); - expect(resolveDestination(false)).toBeNull(); - vi.unstubAllEnvs(); - vi.stubEnv("OTEL_EXPORTER_OTLP_ENDPOINT", ""); - expect(resolveDestination(undefined)).toBeNull(); + expect(resolveDestination(false, "my-site", "t")).toBeNull(); }); - it("mig-09: telemetry: { site, token } is accepted and goes to the hosted collector", () => { - const destination = resolveDestination({ site: "my-site", token: "t" }); - expect(destination?.endpoint).toMatch(/^https:\/\//); + it("mig-09: createCMS({ site, token }) sends telemetry to the hosted collector; the telemetry: { site, token } form is gone", () => { + const destination = resolveDestination(undefined, "my-site", "t"); + expect(destination?.endpoint).toBe("https://otel.decocms.com"); + expect(destination?.headers.authorization).toBe("Bearer t"); expect(() => - createCMS({ blocks: {}, content: snap({}), telemetry: { site: "my-site", token: "t" } }), + createCMS({ blocks: {}, content: snap({}), site: "my-site", token: "t" }), ).not.toThrow(); }); @@ -789,11 +792,11 @@ describe("internals", () => { /* @vite-ignore */ `../cms?copy=${Date.now()}` )) as typeof import("../cms"); expect(instanceOf(copy.createCMS({ blocks: {}, content }))).toBe(instanceOf(first)); - const r1 = remoteLoader(content, { site: "s", token: "t" }); + const r1 = remoteLoader(content, { site: "s" }); const copyRemote = (await import( /* @vite-ignore */ `../remoteLoader?copy=${Date.now()}` )) as typeof import("../remoteLoader"); - expect(copyRemote.remoteLoader(content, { site: "s", token: "t" })).toBe(r1); + expect(copyRemote.remoteLoader(content, { site: "s" })).toBe(r1); }); it("in-13: remoteLoader is exported from the root", () => { @@ -974,7 +977,7 @@ describe("design decisions", () => { ...snap({ X: { __resolveType: "always" } }, "build"), root: ".deco-offline", }; - const cms = createCMS({ blocks: {}, content: fallback, site: "s", token: "t" }); + const cms = createCMS({ blocks: {}, content: fallback, site: "s" }); const client = cms.forRelease(); expect(await client.revision()).toBe("build"); expect(await client.resolve("X")).toEqual([true, null]); diff --git a/packages/blocks/src/v8/__conformance__/observability.test.ts b/packages/blocks/src/v8/__conformance__/observability.test.ts index 610bf092..161e6706 100644 --- a/packages/blocks/src/v8/__conformance__/observability.test.ts +++ b/packages/blocks/src/v8/__conformance__/observability.test.ts @@ -46,9 +46,6 @@ beforeEach(() => { vi.spyOn(Date, "now").mockImplementation(() => now() + clock); (globalThis as Record)[BACKGROUND_HOOK] = (task: () => Promise) => tasks.push(task); - vi.stubEnv("OTEL_EXPORTER_OTLP_ENDPOINT", ""); - vi.stubEnv("OTEL_EXPORTER_OTLP_HEADERS", ""); - vi.stubEnv("OTEL_RESOURCE_ATTRIBUTES", ""); }); afterEach(() => { delete (globalThis as Record)[BACKGROUND_HOOK]; @@ -175,9 +172,9 @@ describe("the docs' snippets typecheck (conformance/observability)", () => { it("tel-01/tel-02: cms.ts with telemetry { endpoint, headers } compiles, headers optional", () => { expect(errorsIn("cms.ts")).toBe(""); }); - it("tel-03: telemetry { site, token } and telemetry: false are accepted (cms.ts)", () => { + it("tel-03: a top-level site and token, and telemetry: false, are accepted (cms.ts)", () => { expect(fs.readFileSync(path.join(EXAMPLES, "cms.ts"), "utf8")).toContain( - "telemetry: { site, token }", + "createCMS({ blocks, content, site, token })", ); expect(errorsIn("cms.ts")).toBe(""); }); @@ -187,6 +184,10 @@ describe("the docs' snippets typecheck (conformance/observability)", () => { it("tel-29: the traced() block map compiles with `satisfies Blocks`", () => { expect(errorsIn(".deco/index.ts")).toBe(""); }); + it("tel-33: telemetry.resource (the app's own commit and environment) compiles (cms.ts)", () => { + expect(fs.readFileSync(path.join(EXAMPLES, "cms.ts"), "utf8")).toContain("resource: {"); + expect(errorsIn("cms.ts")).toBe(""); + }); it("htel-01: the hosted-telemetry cms.ts compiles with site/token possibly undefined", () => { expect(errorsIn("hosted-cms.ts")).toBe(""); }); @@ -234,13 +235,9 @@ describe("where telemetry goes (telemetry.mdx)", () => { expect(sent[0]?.headers.authorization).toBeUndefined(); }); - it("tel-03/htel-02: { site, token } sends to the hosted collector with token auth and deco.site", async () => { + it("tel-03/htel-02: a top-level token sends to the hosted collector with token auth, and site to deco.site", async () => { const { sent } = collector(); - createCMS({ - blocks: docsBlocks(), - content: docsSnapshot(), - telemetry: { site: "acme", token: "site-token" }, - }); + createCMS({ blocks: docsBlocks(), content: docsSnapshot(), site: "acme", token: "site-token" }); await upstreamWith().request("https://search.example/q", { operation: "search" }); await runBackground(); expect(sent[0]?.url).toBe(`${HOSTED_TELEMETRY_ENDPOINT}/v1/metrics`); @@ -248,31 +245,42 @@ describe("where telemetry goes (telemetry.mdx)", () => { expect(attrs(sent[0]?.body.resourceMetrics[0].resource.attributes)["deco.site"]).toBe("acme"); }); - it("tel-04: telemetry: false sends nothing, even with OTEL_EXPORTER_OTLP_ENDPOINT set", async () => { - vi.stubEnv("OTEL_EXPORTER_OTLP_ENDPOINT", ENDPOINT); + it("tel-04: telemetry: false sends nothing, even with a token", async () => { const { fetch } = collector(); - createCMS({ blocks: docsBlocks(), content: docsSnapshot(), telemetry: false }); + createCMS({ + blocks: docsBlocks(), + content: docsSnapshot(), + site: "acme", + token: "tok", + telemetry: false, + }); expect(currentTelemetry()).toBeUndefined(); await upstreamWith().request("https://search.example/q", { operation: "search" }); await runBackground(); expect(fetch).not.toHaveBeenCalled(); }); - it("tel-05: without telemetry, OTEL_EXPORTER_OTLP_ENDPOINT and OTEL_EXPORTER_OTLP_HEADERS are used", async () => { - vi.stubEnv("OTEL_EXPORTER_OTLP_ENDPOINT", ENDPOINT); - vi.stubEnv("OTEL_EXPORTER_OTLP_HEADERS", "x-team=store,authorization=Bearer%20env-token"); + it("tel-05: an endpoint wins over the token; no environment variable is read (OTEL_*, DECO_OTEL_*)", async () => { + vi.stubEnv("OTEL_EXPORTER_OTLP_ENDPOINT", "https://env.example.com"); + vi.stubEnv("OTEL_EXPORTER_OTLP_HEADERS", "authorization=Bearer%20env-token"); + vi.stubEnv("DECO_OTEL_AUTH_TOKEN", "Bearer v7"); const { sent } = collector(); - createCMS({ blocks: docsBlocks(), content: docsSnapshot() }); + createCMS({ + blocks: docsBlocks(), + content: docsSnapshot(), + site: "acme", + token: "tok", + telemetry: { endpoint: ENDPOINT, headers: { "x-team": "store" } }, + }); await upstreamWith().request("https://search.example/q", { operation: "search" }); await runBackground(); - expect(sent[0]?.url).toBe(`${ENDPOINT}/v1/metrics`); - expect(sent[0]?.headers).toMatchObject({ - "x-team": "store", - authorization: "Bearer env-token", - }); + expect(sent.map((s) => s.url)).toEqual([`${ENDPOINT}/v1/metrics`]); + expect(sent[0]?.headers["x-team"]).toBe("store"); + expect(sent[0]?.headers.authorization).toBeUndefined(); }); - it("tel-06: without telemetry and without the env, nothing is sent", async () => { + it("tel-06: without an endpoint or a token, nothing is sent (environment variables set or not)", async () => { + vi.stubEnv("OTEL_EXPORTER_OTLP_ENDPOINT", ENDPOINT); const { fetch } = collector(); const cms = createCMS({ blocks: { ...docsBlocks(), broken }, content: docsSnapshot() }); await cms.forRelease().resolve({ __resolveType: "broken" }); @@ -281,9 +289,9 @@ describe("where telemetry goes (telemetry.mdx)", () => { expect(fetch).not.toHaveBeenCalled(); }); - it("tel-07: top-level site and token never turn telemetry on", async () => { + it("tel-07: a top-level site alone never turns telemetry on", async () => { const { attempts } = collector(); - createCMS({ blocks: docsBlocks(), content: docsSnapshot(), site: "acme", token: "tok" }); + createCMS({ blocks: docsBlocks(), content: docsSnapshot(), site: "acme" }); expect(currentTelemetry()).toBeUndefined(); await upstreamWith().request("https://search.example/q", { operation: "search" }); await runBackground(); @@ -907,7 +915,9 @@ describe("how telemetry is sent (telemetry-internals.mdx)", () => { const cms = createCMS({ blocks: { ...docsBlocks(), broken }, content: withTelemetryBlock({ errorSampleRate: 1, traceSampleRate: 1 }, "rev-9"), - telemetry: { site: "acme", token: "tok", limits: { errorSampleRate: 1, traceSampleRate: 1 } }, + site: "acme", + token: "tok", + telemetry: { limits: { errorSampleRate: 1, traceSampleRate: 1 } }, }); await cms.forRelease().resolve({ __resolveType: "broken" }); await upstreamWith().request("https://search.example/q", { operation: "search" }); diff --git a/packages/blocks/src/v8/__conformance__/sdk.test.ts b/packages/blocks/src/v8/__conformance__/sdk.test.ts index 2d87f68f..6048c8f9 100644 --- a/packages/blocks/src/v8/__conformance__/sdk.test.ts +++ b/packages/blocks/src/v8/__conformance__/sdk.test.ts @@ -241,15 +241,17 @@ declare const content: Snapshot; declare const loader: Loader; createCMS({ blocks, content }); createCMS({ blocks, content: loader, interval: 60_000, telemetry: false, secrets: { key: "k" }, site: "s", token: "t" }); -createCMS({ blocks, content, telemetry: { site: "s", token: "t", limits: { errorSampleRate: 0.1, traceSampleRate: 0 } } }); -createCMS({ blocks, content, telemetry: { endpoint: "https://otel.example", headers: { a: "b" } } }); -createCMS({ blocks, content, preview: { hosts: ["*.example.com", "localhost:3000"] } }); -// @ts-expect-error draft hosts come from DECO_PREVIEW_API_DOMAINS, not code +createCMS({ blocks, content, site: "s", token: "t", telemetry: { limits: { errorSampleRate: 0.1, traceSampleRate: 0 } } }); +createCMS({ blocks, content, telemetry: { endpoint: "https://otel.example", headers: { a: "b" }, resource: { "service.version": "abc" } } }); +createCMS({ blocks, content, preview: { hosts: ["*.example.com", "localhost:3000"], draftHosts: [".decocms.com"] } }); +// @ts-expect-error draft hosts are preview.draftHosts createCMS({ blocks, content, preview: { sources: ["studio.example.com"] } }); +// @ts-expect-error the telemetry: { site, token } form is gone: site and token are top-level +createCMS({ blocks, content, telemetry: { site: "s", token: "t" } }); // @ts-expect-error not a documented option createCMS({ blocks, content, ignoreCase: true }); -assert>(); +assert>(); assert Promise>>(); assert Promise>>(); assert Promise>>(); @@ -257,7 +259,6 @@ assert>(); assert; analytics: Required }>>(); assert Client>>(); -assert Client>>(); assert Promise<{ updated: boolean }>>>(); assert>(); assert Promise>>(); @@ -283,27 +284,27 @@ assert>(); assert>(); assert>(); assert, [number, null] | [null, CMSError]>>(); -assert }) & { limits?: { errorSampleRate?: number; traceSampleRate?: number } }>>(); +assert; resource?: Record; limits?: { errorSampleRate?: number; traceSampleRate?: number } }>>(); export { r1, r2, r3 }; `, }); expect(errors).toEqual([]); }, 60_000); - it("AR-57 Snapshot is exactly { revision, blocks, aliases? }", () => { + it("AR-57 Snapshot is exactly { revision, blocks, aliases?, schemaHash?, committedAt? }", () => { const errors = typecheck({ "snapshot.ts": ` import type { Snapshot } from "@decocms/blocks"; type Equal = (() => T extends A ? 1 : 2) extends (() => T extends B ? 1 : 2) ? true : false; const assert = () => {}; -assert; aliases?: Record }>>(); +assert; aliases?: Record; schemaHash?: string; committedAt?: string }>>(); `, }); expect(errors).toEqual([]); }, 60_000); }); -describe("AR-05 interval: default DECO_CONTENT_INTERVAL or 60 000, minimum 60 000", () => { +describe("AR-05 interval: default 60 000, minimum 60 000; no environment variable", () => { const intervalOf = (cms: CMS) => (instanceOf(cms) as { fingerprint: { interval: number } }).fingerprint.interval; @@ -313,17 +314,15 @@ describe("AR-05 interval: default DECO_CONTENT_INTERVAL or 60 000, minimum 60 00 expect(intervalOf(cms)).toBe(60_000); }); - it("defaults to 60 000 and reads DECO_CONTENT_INTERVAL", () => { + it("defaults to 60 000 and never reads DECO_CONTENT_INTERVAL", () => { expect(intervalOf(createCMS({ blocks: {}, content: docsSnapshot() }))).toBe(60_000); resetForTests(); vi.stubEnv("DECO_CONTENT_INTERVAL", "120000"); - expect(intervalOf(createCMS({ blocks: {}, content: docsSnapshot() }))).toBe(120_000); - }); - - it("clamps DECO_CONTENT_INTERVAL below the minimum too", () => { - vi.spyOn(console, "warn").mockImplementation(() => {}); - vi.stubEnv("DECO_CONTENT_INTERVAL", "5000"); expect(intervalOf(createCMS({ blocks: {}, content: docsSnapshot() }))).toBe(60_000); + resetForTests(); + expect(intervalOf(createCMS({ blocks: {}, content: docsSnapshot(), interval: 120_000 }))).toBe( + 120_000, + ); }); }); @@ -333,42 +332,60 @@ describe("AR-06 / AR-07 telemetry destination and limits", () => { errorSampleRate: 0.1, traceSampleRate: 0, }); - expect(resolveDestination({ site: "s", token: "t" })?.limits).toEqual({ + expect(resolveDestination(undefined, "s", "t")?.limits).toEqual({ errorSampleRate: 0.1, traceSampleRate: 0, }); }); - it("false sends nothing, even with OTEL_EXPORTER_OTLP_ENDPOINT set", () => { - vi.stubEnv("OTEL_EXPORTER_OTLP_ENDPOINT", "https://env.example"); - expect(resolveDestination(false)).toBeNull(); + it("false sends nothing, even with a token", () => { + expect(resolveDestination(false, "s", "t")).toBeNull(); }); - it("an object sends there", () => { + it("an endpoint sends there, with its headers", () => { expect( - resolveDestination({ endpoint: "https://otel.example", headers: { a: "b" } }), + resolveDestination({ endpoint: "https://otel.example", headers: { a: "b" } }, "s", "t"), ).toMatchObject({ endpoint: "https://otel.example", headers: { a: "b" } }); }); - it("omitted: OTEL_EXPORTER_OTLP_ENDPOINT and _HEADERS when set, otherwise nothing", () => { - expect(resolveDestination(undefined)).toBeNull(); + it("omitted: the token's hosted collector, otherwise nothing; OTEL_* variables aren't read", () => { vi.stubEnv("OTEL_EXPORTER_OTLP_ENDPOINT", "https://env.example"); vi.stubEnv("OTEL_EXPORTER_OTLP_HEADERS", "x-key=abc"); - expect(resolveDestination(undefined)).toMatchObject({ - endpoint: "https://env.example", - headers: { "x-key": "abc" }, + expect(resolveDestination(undefined)).toBeNull(); + expect(resolveDestination(undefined, "s", "t")).toMatchObject({ + endpoint: "https://otel.decocms.com", + headers: { authorization: "Bearer t" }, }); }); }); -describe("AR-08 site and token load hosted content only, never telemetry", () => { - it("site without token: the content is read as is (no hosted loader, no fetch)", async () => { - const fetch = vi.fn(async () => new Response("{}")); +describe("AR-08 site loads hosted releases; token sends telemetry; neither is needed for drafts", () => { + it("site alone: releases from delivery, no telemetry", async () => { + const { currentTelemetry } = await import("../telemetry"); + const fetch = vi.fn( + async (_input: string | URL | Request) => new Response("{}", { status: 404 }), + ); vi.stubGlobal("fetch", fetch); - const cms = createCMS({ blocks: docsBlocks(), content: docsSnapshot(), site: "acme" }); + const cms = createCMS({ + blocks: docsBlocks(), + content: { ...docsSnapshot(), schemaHash: "5".repeat(64) }, + site: "acme", + }); const [seo] = await cms.forRelease().resolve("SummerSEO"); expect(seo).toEqual({ title: "Sunny!", description: "Light layers for long days." }); expect(await cms.update()).toEqual({ updated: false }); + expect(fetch.mock.calls.map(([url]) => String(url))).toEqual([ + "https://delivery.decocms.com/sites/acme/latest.json", + ]); + expect(currentTelemetry()).toBeUndefined(); + }); + + it("token alone is a configuration error: token needs site", () => { + const fetch = vi.fn(async () => new Response("{}")); + vi.stubGlobal("fetch", fetch); + expect(() => createCMS({ blocks: docsBlocks(), content: docsSnapshot(), token: "t" })).toThrow( + "token needs site: pass both, or site alone", + ); expect(fetch).not.toHaveBeenCalled(); }); @@ -378,12 +395,6 @@ describe("AR-08 site and token load hosted content only, never telemetry", () => const [seo] = await cms.forDraft(pointer).resolve<{ title: string }>("SummerSEO"); expect(seo?.title).toBe("Draft!"); }); - - it("site and token with telemetry omitted and no OTEL env: no telemetry", async () => { - const { currentTelemetry } = await import("../telemetry"); - createCMS({ blocks: docsBlocks(), content: docsSnapshot(), site: "acme", token: "t" }); - expect(currentTelemetry()).toBeUndefined(); - }); }); describe("AR-09 / AR-56 secrets", () => { @@ -412,7 +423,7 @@ describe("AR-11 / AR-20 / AR-26 / RD-03 / CT-09 drafts", () => { const draft = cms.forDraft(pointer); expect((await draft.resolve<{ title: string }>("SummerSEO"))[0]?.title).toBe("Draft!"); expect((await draft.resolve("HomePage"))[1]).toBeNull(); - expect(await draft.revision()).toBe("rev-1~9f3c1a"); + expect(await draft.revision()).toBe('rev-1~"etag-1"'); }); it("a draft whose changes can't be fetched makes every call return LOADER_FAILED with the cause", async () => { @@ -429,7 +440,7 @@ describe("AR-11 / AR-20 / AR-26 / RD-03 / CT-09 drafts", () => { expect(listError?.code).toBe("LOADER_FAILED"); }); - it("a pointer that doesn't parse, or names a host outside the preview API domains, is LOADER_FAILED with no fetch", async () => { + it("a pointer that doesn't parse, or names a host outside the draft hosts, is LOADER_FAILED with no fetch", async () => { const { fetch } = studioDraft(); const cms = createCMS({ blocks: docsBlocks(), content: docsSnapshot() }); for (const pointer of ["garbage", "api.deco.example/drafts/acme/main@9f3c1a"]) { @@ -516,7 +527,7 @@ describe("AR-66 a draft pointer's forced variants (releases-and-drafts#preview-a expect([value, error]).toEqual(["spring", null]); }); - it("a draft is fetched without them, once for every variant of one draft", async () => { + it("a draft is fetched without them, one body for every variant of one draft (later reads revalidate it)", async () => { const { fetch, pointer } = studioDraft(); const cms = createCMS({ blocks: docsBlocks(), content: content() }); const draft = (index: number) => @@ -526,24 +537,10 @@ describe("AR-66 a draft pointer's forced variants (releases-and-drafts#preview-a }); expect((await cms.forDraft(draft(1)).resolve("Banner"))[0]).toBe("summer"); expect((await cms.forDraft(draft(0)).resolve("Banner"))[0]).toBe("spring"); - expect(fetch).toHaveBeenCalledTimes(1); + expect(fetch).toHaveBeenCalledTimes(2); + expect(String(fetch.mock.calls[0]![0])).toBe(String(fetch.mock.calls[1]![0])); expect(String(fetch.mock.calls[0]![0])).not.toContain("__variant"); - }); -}); - -describe("AR-12 forRevision", () => { - it("pins a served revision; an unknown revision reads the release", async () => { - const { loader, publish } = swappableLoader(docsSnapshot("rev-1")); - const cms = createCMS({ blocks: docsBlocks(), content: loader }); - const old = await cms.forRelease().revision(); - const next = docsSnapshot("rev-2"); - (next.blocks.SummerSEO as { title: string }).title = "New"; - publish(next); - expect(await cms.update()).toEqual({ updated: true }); - expect(await cms.forRelease().revision()).toBe("rev-2"); - const [pinned] = await cms.forRevision(old).resolve<{ title: string }>("SummerSEO"); - expect(pinned?.title).toBe("Sunny!"); - expect(await cms.forRevision("nope").revision()).toBe("rev-2"); + expect((await fetch.mock.results[1]!.value).status).toBe(304); }); }); @@ -640,26 +637,25 @@ describe("AR-18 / AR-19 content: a snapshot or any loader", () => { }); describe("AR-22 remoteLoader", () => { - it("is exported from the root, and is a loader over the fallback without site or token", async () => { + it("is exported from the root, and is a loader over the fallback without site", async () => { expect(root.remoteLoader).toBe(remoteLoader); const fallback = docsSnapshot(); - expect(await remoteLoader(fallback, { site: "", token: "" }).load()).toBe(fallback); + expect(await remoteLoader(fallback, { site: "" }).load()).toBe(fallback); }); - it("createCMS builds it when site and token are set: update() reaches the Deco API", async () => { + it("createCMS builds it when site is set: update() reads latest.json from delivery.decocms.com", async () => { const fetch = vi.fn( async (_input: string | URL | Request) => new Response("nope", { status: 404 }), ); vi.stubGlobal("fetch", fetch); const cms = createCMS({ blocks: docsBlocks(), - content: docsSnapshot(), + content: { ...docsSnapshot(), schemaHash: "5".repeat(64) }, site: "acme", - token: "t", }); expect(await cms.update()).toEqual({ updated: false }); expect(String(fetch.mock.calls[0]?.[0])).toBe( - "https://delivery.decocms.com/sites/acme/channels/production.json", + "https://delivery.decocms.com/sites/acme/latest.json", ); }); }); diff --git a/packages/blocks/src/v8/browserBundle.test.ts b/packages/blocks/src/v8/browserBundle.test.ts index bfd97992..9e7b6d32 100644 --- a/packages/blocks/src/v8/browserBundle.test.ts +++ b/packages/blocks/src/v8/browserBundle.test.ts @@ -6,6 +6,8 @@ * and for a workerd-style target and check the output. */ import { execFileSync } from "node:child_process"; +import { mkdtempSync, readFileSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; import { dirname, join } from "node:path"; import { fileURLToPath } from "node:url"; import { describe, expect, it } from "vitest"; @@ -41,13 +43,22 @@ describe("v8 core bundle", () => { }); it("imports nothing at runtime but its own modules, the ciphertext format and the shared content hash (no React, no v7 code)", () => { - const output = bundle([ - "--platform=neutral", - "--metafile=/dev/stdout", - "--outfile=/dev/null", - "--log-level=error", - ]); - const inputs = Object.keys(JSON.parse(output.slice(output.indexOf("{"))).inputs); + // A real file, not /dev/stdout: esbuild leaves that empty on Linux. + const dir = mkdtempSync(join(tmpdir(), "blocks-bundle-")); + let meta: { inputs: Record }; + try { + const metafile = join(dir, "meta.json"); + bundle([ + "--platform=neutral", + `--metafile=${metafile}`, + `--outfile=${join(dir, "out.js")}`, + "--log-level=error", + ]); + meta = JSON.parse(readFileSync(metafile, "utf-8")); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + const inputs = Object.keys(meta.inputs); expect(inputs.length).toBeGreaterThan(0); for (const input of inputs) { expect(input, input).toMatch(/(^|\/)src\/(v8\/|protocol\/(canonical|ciphertext)\.ts$)/); diff --git a/packages/blocks/src/v8/cli/content.test.ts b/packages/blocks/src/v8/cli/content.test.ts index fd0116b6..960adea5 100644 --- a/packages/blocks/src/v8/cli/content.test.ts +++ b/packages/blocks/src/v8/cli/content.test.ts @@ -1,16 +1,35 @@ // @vitest-environment node + +import { spawnSync } from "node:child_process"; import fs from "node:fs"; import path from "node:path"; import { pathToFileURL } from "node:url"; -import { afterEach, describe, expect, it } from "vitest"; -import { computeContentRevision } from "../canonical"; +import { afterEach, describe, expect, it, vi } from "vitest"; +import { canonicalJson, computeContentRevision, sha256Hex } from "../canonical"; import { createFixture, type Fixture, recorder } from "./__tests__/fixture"; import { LEGACY_ALIASES } from "./builtins"; import { content, readSavedBlocks, renderContentModule, writeContent } from "./content"; import { decoPaths } from "./root"; let fixture: Fixture; -afterEach(() => fixture?.remove()); +afterEach(() => { + vi.useRealTimers(); + fixture?.remove(); +}); + +/** An (empty) commit in `root` at `date`, in a new repository if needed. */ +function commitAt(root: string, date: string): void { + const git = (...args: string[]) => { + const r = spawnSync("git", ["-c", "user.name=t", "-c", "user.email=t@t", ...args], { + cwd: root, + encoding: "utf8", + env: { ...process.env, GIT_AUTHOR_DATE: date, GIT_COMMITTER_DATE: date }, + }); + if (r.status !== 0) throw new Error(`git ${args.join(" ")}: ${r.stderr}`); + }; + if (!fs.existsSync(path.join(root, ".git"))) git("init", "-q"); + git("commit", "-q", "--allow-empty", "-m", date); +} const home = { __resolveType: "page", name: "Home", path: "/", sections: [] }; @@ -114,6 +133,69 @@ describe("reading .deco/blocks", () => { }); }); +/** + * The schemaHash test vector Studio's publish shares: the same file must give + * the same hash on both sides (sha256Hex(canonicalJson(JSON.parse(text)))). + */ +const SCHEMA_TEXT = + '{\n "version": "8.1.0-next.7",\n "blocksMajor": 8,\n "definitions": { "seo": { "type": "object", "title": "Seo" } },\n "root": {}\n}\n'; +const SCHEMA_HASH = "00b083655ee7af02aa92dbff85e402857bb1e50c253a579dbab23498a7842c98"; + +describe("schemaHash", () => { + it("is written into the module from .deco/schema.gen.json, as sha256Hex(canonicalJson(schema))", async () => { + fixture = createFixture({ + ".deco/blocks/HomePage.json": home, + ".deco/schema.gen.json": SCHEMA_TEXT, + }); + expect(await sha256Hex(canonicalJson(JSON.parse(SCHEMA_TEXT)))).toBe(SCHEMA_HASH); + const result = await writeContent(decoPaths(fixture.root)); + expect(result.schemaHash).toBe(SCHEMA_HASH); + const source = fixture.read(".deco/blocks.gen.ts"); + expect(source).toContain(` schemaHash: "${SCHEMA_HASH}",`); + expect(source).toContain(" schemaHash: string;"); + const mod = await import(/* @vite-ignore */ pathToFileURL(result.file).href); + expect(mod.default.schemaHash).toBe(SCHEMA_HASH); + }); + + it("ignores formatting and key order: the parsed JSON is hashed", async () => { + fixture = createFixture({ + ".deco/blocks/HomePage.json": home, + ".deco/schema.gen.json": JSON.stringify(JSON.parse(SCHEMA_TEXT)), + }); + expect((await writeContent(decoPaths(fixture.root))).schemaHash).toBe(SCHEMA_HASH); + }); + + it("is left out without a schema file, so hosted releases never swap", async () => { + fixture = createFixture({ ".deco/blocks/HomePage.json": home }); + const result = await writeContent(decoPaths(fixture.root)); + expect(result.schemaHash).toBeUndefined(); + expect(fixture.read(".deco/blocks.gen.ts")).not.toContain("schemaHash"); + }); + + it("a schema file that isn't JSON fails deco content", async () => { + fixture = createFixture({ + ".deco/blocks/HomePage.json": home, + ".deco/schema.gen.json": "{", + }); + await expect(writeContent(decoPaths(fixture.root))).rejects.toThrow( + /schema\.gen\.json: not valid JSON/, + ); + }); + + it("a schema change rewrites the module", async () => { + fixture = createFixture({ + ".deco/blocks/HomePage.json": home, + ".deco/schema.gen.json": SCHEMA_TEXT, + }); + const paths = decoPaths(fixture.root); + expect((await writeContent(paths)).changed).toBe(true); + fixture.write(".deco/schema.gen.json", { blocksMajor: 8, version: "other" }); + const next = await writeContent(paths); + expect(next.changed).toBe(true); + expect(next.schemaHash).not.toBe(SCHEMA_HASH); + }); +}); + describe("the content module", () => { it("imports each JSON file and exports { revision, blocks, aliases }", async () => { fixture = createFixture({ ".deco/blocks/HomePage.json": home }); @@ -172,18 +254,55 @@ describe("the content module", () => { expect(mod.default.aliases).toEqual(LEGACY_ALIASES); }); - it("only rewrites the file when it changes", async () => { + it("stamps committedAt with the commit time of git HEAD in the root (committer date)", async () => { fixture = createFixture({ ".deco/blocks/HomePage.json": home }); + commitAt(fixture.root, "2026-10-07T10:00:00-03:00"); const paths = decoPaths(fixture.root); - expect((await writeContent(paths)).changed).toBe(true); + const first = await writeContent(paths); + expect(first.committedAt).toBe("2026-10-07T10:00:00-03:00"); + expect(fixture.read(".deco/blocks.gen.ts")).toContain( + ' committedAt: "2026-10-07T10:00:00-03:00",', + ); + expect(fixture.read(".deco/blocks.gen.ts")).toContain(" committedAt: string;"); + const mod = await import(/* @vite-ignore */ pathToFileURL(first.file).href); + expect(mod.default.committedAt).toBe("2026-10-07T10:00:00-03:00"); + // Same commit, built again later: the same stamp, nothing to write. expect((await writeContent(paths)).changed).toBe(false); - fixture.write(".deco/blocks/Other.json", { - __resolveType: "page", - name: "O", - path: "/o", - sections: [], - }); - expect((await writeContent(paths)).changed).toBe(true); + // A later commit: a new stamp, so its deploy wins over earlier publishes. + commitAt(fixture.root, "2026-10-07T11:00:00-03:00"); + const later = await writeContent(paths); + expect(later.changed).toBe(true); + expect(later.revision).toBe(first.revision); + expect(fixture.read(".deco/blocks.gen.ts")).toContain( + ' committedAt: "2026-10-07T11:00:00-03:00",', + ); + }); + + it("writes no committedAt outside a git repository, and says so in one line", async () => { + fixture = createFixture({ ".deco/blocks/HomePage.json": home }); + const result = await writeContent(decoPaths(fixture.root)); + expect(result.committedAt).toBeUndefined(); + expect(fixture.read(".deco/blocks.gen.ts")).not.toContain("committedAt"); + const mod = await import(/* @vite-ignore */ pathToFileURL(result.file).href); + expect("committedAt" in mod.default).toBe(false); + const out = recorder(); + expect(await content({ cwd: fixture.root, reporter: out })).toBe(0); + expect(out.lines.filter((l) => /git/.test(l.message))).toEqual([ + { + level: "info", + message: + "no git commit found, so .deco/blocks.gen.ts has no committedAt: hosted releases with the same schema will replace it", + }, + ]); + }); + + it("renders the same module for the same content and committedAt", async () => { + fixture = createFixture({ ".deco/blocks/HomePage.json": home }); + const saved = readSavedBlocks(decoPaths(fixture.root).blocks); + const at = "2026-10-07T10:00:00.000Z"; + expect(await renderContentModule(saved, ".deco", undefined, at)).toBe( + await renderContentModule(saved, ".deco", undefined, at), + ); }); it("refuses to write a module from unreadable content", async () => { @@ -198,6 +317,7 @@ describe("the content module", () => { ".deco/blocks/HomePage.json": home, ".deco/index.ts": "this is not valid TypeScript (", }); + commitAt(fixture.root, "2026-10-07T10:00:00Z"); const out = recorder(); expect(await content({ cwd: path.join(fixture.root, "src"), reporter: out })).toBe(0); expect(out.text()).toMatch( diff --git a/packages/blocks/src/v8/cli/content.ts b/packages/blocks/src/v8/cli/content.ts index f5b875ef..e3c440a4 100644 --- a/packages/blocks/src/v8/cli/content.ts +++ b/packages/blocks/src/v8/cli/content.ts @@ -3,9 +3,10 @@ * module, `.deco/blocks.gen.ts` (spec: content › The content module). It never * reads the block map. */ +import { execFileSync } from "node:child_process"; import fs from "node:fs"; import path from "node:path"; -import { computeContentRevision } from "../../protocol/canonical.ts"; +import { canonicalJson, computeContentRevision, sha256Hex } from "../../protocol/canonical.ts"; import { blockNameFromFile, checkBlockName, @@ -154,7 +155,8 @@ function inlinedFiles(saved: SavedBlocks): string[] { /** * The `.deco` folder's identity for `createCMS` (one instance per folder, so * a hot-reloaded module keeps its instance): its path from the repository - * root, or `.deco` outside a repository. Relative, so builds are reproducible. + * root, or `.deco` outside a repository. Relative, so it doesn't depend on + * where the repository is checked out. */ export function contentRoot(deco: string): string { for (let dir = path.dirname(deco); ; dir = path.dirname(dir)) { @@ -165,8 +167,66 @@ export function contentRoot(deco: string): string { } } -/** The source of `.deco/blocks.gen.ts` for a set of saved blocks. */ -export async function renderContentModule(saved: SavedBlocks, root = ".deco"): Promise { +/** + * The `schemaHash` of a `.deco/schema.gen.json`: the SHA-256 of its parsed + * JSON in canonical form (`sha256Hex(canonicalJson(JSON.parse(text)))`, the + * `@decocms/blocks/protocol` helpers), so Studio computes the same value + * from the file at a commit. Hosted releases swap only when it matches. + */ +async function computeSchemaHash(text: string): Promise { + return sha256Hex(canonicalJson(JSON.parse(text))); +} + +/** `schema.gen.json`'s hash, or `undefined` when there's no such file. */ +async function readSchemaHash(schemaFile: string): Promise { + let text: string; + try { + text = fs.readFileSync(schemaFile, "utf8"); + } catch (error) { + if ((error as NodeJS.ErrnoException).code === "ENOENT") return undefined; + throw error; + } + try { + return await computeSchemaHash(text); + } catch (error) { + throw new CliError( + `.deco/schema.gen.json: not valid JSON (${(error as Error).message}); run deco schema`, + ); + } +} + +/** + * The commit time of git HEAD in `dir` (committer date, ISO 8601), or + * undefined when git is missing, `dir` isn't in a repository, or it has no + * commits. Commit time, not build time: a slow build of an older commit that + * finishes after a publish must not win over it. + */ +export function readCommittedAt(dir: string): string | undefined { + try { + const out = execFileSync("git", ["log", "-1", "--format=%cI"], { + cwd: dir, + encoding: "utf8", + stdio: ["ignore", "pipe", "ignore"], + }).trim(); + return out === "" || Number.isNaN(Date.parse(out)) ? undefined : out; + } catch { + return undefined; + } +} + +/** + * The source of `.deco/blocks.gen.ts` for a set of saved blocks, with the + * `schemaHash` of the schema it was built with when there is one, and + * `committedAt`: the commit time of git HEAD it was built from (ISO 8601), + * when there is one. A hosted release replaces the bundled content only when + * it was published later (see ../remoteLoader.ts). + */ +export async function renderContentModule( + saved: SavedBlocks, + root = ".deco", + schemaHash?: string, + committedAt?: string, +): Promise { const names = Object.keys(saved.blocks).sort(); const revision = await computeContentRevision(saved.blocks); const taken = new Set(); @@ -194,11 +254,15 @@ export async function renderContentModule(saved: SavedBlocks, root = ".deco"): P "", "const content: {", " revision: string;", + ...(schemaHash === undefined ? [] : [" schemaHash: string;"]), + ...(committedAt === undefined ? [] : [" committedAt: string;"]), " blocks: Record;", " aliases: Record;", " root: string;", "} = {", ` revision: ${JSON.stringify(revision)},`, + ...(schemaHash === undefined ? [] : [` schemaHash: ${JSON.stringify(schemaHash)},`]), + ...(committedAt === undefined ? [] : [` committedAt: ${JSON.stringify(committedAt)},`]), " blocks: {", ...entries, " },", @@ -217,7 +281,14 @@ export interface ContentResult { root: string; file: string; revision: string; + /** The hash of `.deco/schema.gen.json`, written into the module; absent without one. */ + schemaHash?: string; count: number; + /** + * The commit time of git HEAD in the root, written into the module as + * `committedAt`; absent when git isn't available there. + */ + committedAt?: string; /** False when the file already had this content. */ changed: boolean; /** Files written into the module instead of imported (see `inlinedFiles`). */ @@ -246,12 +317,16 @@ export async function writeContent(paths: DecoPaths): Promise { if (errors.length > 0) { throw new CliError(errors.map((d) => `.deco/blocks/${d.file}: ${d.message}`).join("\n")); } - const source = await renderContentModule(saved, contentRoot(paths.deco)); + const schemaHash = await readSchemaHash(paths.schema); + const committedAt = readCommittedAt(paths.root); + const source = await renderContentModule(saved, contentRoot(paths.deco), schemaHash, committedAt); const changed = writeIfChanged(paths.content, source); return { root: paths.root, file: paths.content, revision: await computeContentRevision(saved.blocks), + ...(schemaHash === undefined ? {} : { schemaHash }), + ...(committedAt === undefined ? {} : { committedAt }), count: Object.keys(saved.blocks).length, changed, inlined: inlinedFiles(saved), @@ -273,6 +348,11 @@ export async function content(options: ContentOptions = {}): Promise { const paths = decoPaths(findDecoRoot(options)); const result = await writeContent(paths); for (const d of result.diagnostics) reporter.warn(`.deco/blocks/${d.file}: ${d.message}`); + if (result.committedAt === undefined) { + reporter.info( + "no git commit found, so .deco/blocks.gen.ts has no committedAt: hosted releases with the same schema will replace it", + ); + } reporter.info( `${result.changed ? "wrote" : "unchanged"} .deco/blocks.gen.ts (${result.count} blocks, revision ${result.revision.slice(0, 12)})`, ); diff --git a/packages/blocks/src/v8/cli/run.ts b/packages/blocks/src/v8/cli/run.ts index 70ab15ec..3719e802 100644 --- a/packages/blocks/src/v8/cli/run.ts +++ b/packages/blocks/src/v8/cli/run.ts @@ -124,7 +124,7 @@ async function watchLoop( */ const PUBLISH_SIGNPOST = `There is no publish command: Git is the source of truth for content, so publishing is committing. Commit the changes in .deco/blocks (and push them). A deploy ships the commit; with the hosted -Deco CMS, a commit becomes a release without a deploy.`; +Deco CMS, Studio's Publish (or Resync) makes the commit a release without a deploy.`; /** Run one `deco` invocation; returns the exit code. */ export async function runCli(argv: string[], options: RunOptions = {}): Promise { diff --git a/packages/blocks/src/v8/cms.test.ts b/packages/blocks/src/v8/cms.test.ts index f03e091c..2fe04aca 100644 --- a/packages/blocks/src/v8/cms.test.ts +++ b/packages/blocks/src/v8/cms.test.ts @@ -87,7 +87,7 @@ describe("loaders", () => { title: "Draft title", description: "Draft", }); - expect(await client.revision()).toBe("rev-1~9f3c1a"); + expect(await client.revision()).toBe('rev-1~"etag-1"'); // the draft body's ETag names the view expect(loads()).toBe(2); // a loader without update() is asked per client }); @@ -208,21 +208,44 @@ describe("loaders", () => { expect(fetch).toHaveBeenCalledTimes(2); }); - it("an update that finds nothing new keeps fetched drafts", async () => { + it("an update that finds nothing new keeps the fetched draft body: the next read revalidates it", async () => { const { fetch, pointer } = studioDraft(); const { loader } = countingLoader({ update: async () => ({ updated: false }) }); const cms = createCMS({ blocks: docsBlocks(), content: loader }); await cms.forDraft(pointer).resolve("SummerSEO"); await cms.update(); await cms.forDraft(pointer).resolve("SummerSEO"); - expect(fetch).toHaveBeenCalledTimes(1); + expect(fetch).toHaveBeenCalledTimes(2); + expect(new Headers(fetch.mock.calls[1]?.[1]?.headers).get("if-none-match")).toBe('"etag-1"'); + expect((await fetch.mock.results[1]?.value)?.status).toBe(304); + }); + + it("every draft read revalidates: a 304 reuses the body, a new save is read again", async () => { + const { studio, fetch, pointer } = studioDraft(); + const cms = createCMS({ blocks: docsBlocks(), content: docsSnapshot() }); + const first = cms.forDraft(pointer); + await first.resolve("SummerSEO"); + const second = cms.forDraft(pointer); + expect((await second.resolve("SummerSEO"))[0]).toMatchObject({ title: "Draft title" }); + expect(await second.revision()).toBe(await first.revision()); + expect(fetch).toHaveBeenCalledTimes(2); + expect(new Headers(fetch.mock.calls[0]?.[1]?.headers).get("if-none-match")).toBeNull(); + expect((await fetch.mock.results[1]?.value)?.status).toBe(304); + + studio.draft({ set: { SummerSEO: { ...DRAFT_SEO, title: "Saved again" } } }); + const third = cms.forDraft(pointer); // the same pointer outlives the save + expect((await third.resolve("SummerSEO"))[0]).toMatchObject({ title: "Saved again" }); + expect(await third.revision()).toBe('rev-1~"etag-2"'); }); - it("drafts are fetched once per pointer within a minute", async () => { + it("concurrent reads of one draft share one request", async () => { const { fetch, pointer } = studioDraft(); const cms = createCMS({ blocks: docsBlocks(), content: docsSnapshot() }); - await cms.forDraft(pointer).resolve("SummerSEO"); - await cms.forDraft(pointer).resolve("SummerSEO"); + await Promise.all([ + cms.forDraft(pointer).resolve("SummerSEO"), + cms.forDraft(pointer).resolve("SummerSEO"), + cms.forDraft(pointer).resolve("HomePage"), + ]); expect(fetch).toHaveBeenCalledTimes(1); }); }); @@ -247,35 +270,6 @@ describe("one revision per client", () => { expect(await cms.forRelease().resolve("Name")).toEqual(["r2", null]); }); - it("forRevision pins a revision this CMS has served; an unknown one behaves like the release", async () => { - let revision = "r1"; - const cms = createCMS({ - blocks: docsBlocks(), - content: { - load: async () => ({ revision, blocks: { Name: revision } }), - update: async () => ({ updated: true }), - }, - }); - expect(await cms.forRelease().revision()).toBe("r1"); - revision = "r2"; - await cms.update(); - expect(await cms.forRelease().revision()).toBe("r2"); - expect(await cms.forRevision("r1").resolve("Name")).toEqual(["r1", null]); - expect(await cms.forRevision("never-served").revision()).toBe("r2"); - }); - - it("forRevision never reaches a draft: a draft revision behaves like the release", async () => { - const { pointer } = studioDraft(); - const cms = createCMS({ blocks: docsBlocks(), content: docsSnapshot() }); - expect(await cms.forDraft(pointer).revision()).toBe("rev-1~9f3c1a"); - const client = cms.forRevision("rev-1~9f3c1a"); - expect(await client.revision()).toBe("rev-1"); - expect(await client.resolve("SummerSEO")).toEqual([ - { title: "Sunny!", description: "Light layers for long days." }, - null, - ]); - }); - it("a client loads its content once, lazily, on first use", async () => { const load = vi.fn(async () => docsSnapshot()); const cms = createCMS({ blocks: docsBlocks(), content: { load } }); @@ -406,17 +400,13 @@ describe("update() checks on an interval", () => { expect(update).toHaveBeenCalledTimes(1); }); - it("reads DECO_CONTENT_INTERVAL when interval is left out", async () => { + it("reads no DECO_CONTENT_INTERVAL: left out, the interval is the 60 s minimum", async () => { vi.stubEnv("DECO_CONTENT_INTERVAL", "300000"); const { cms, update, at } = scheduled(); at("2026-10-03T00:00:00Z"); cms.forRelease(); await idle(); - at("2026-10-03T00:04:59Z"); - cms.forRelease(); - await idle(); - expect(update).toHaveBeenCalledTimes(1); - at("2026-10-03T00:05:00Z"); + at("2026-10-03T00:01:11Z"); cms.forRelease(); await idle(); expect(update).toHaveBeenCalledTimes(2); @@ -432,7 +422,6 @@ describe("one instance per process", () => { expect(instanceOf(proxy)).toBe(instanceOf(app)); expect(await app.forRelease().resolve("Hero")).toEqual(["app", null]); expect(await proxy.forRelease().resolve("Hero")).toEqual(["proxy", null]); - expect(await app.forRevision("r1").resolve("Hero")).toEqual(["app", null]); expect(await app.forDraft("localhost:4547/@local").resolve("Hero")).toEqual(["app", null]); // Still one store: an update through either is seen by both. createCMS({ @@ -502,7 +491,6 @@ describe("one instance per process", () => { content: { revision: "r1", root: ".deco", blocks: { Home: "after" } }, }); expect(await cms.forRelease().resolve("Home")).toEqual(["after", null]); - expect(await cms.forRevision("r1").resolve("Home")).toEqual(["after", null]); }); it("a loader you write is identified by the loader object", () => { @@ -516,7 +504,8 @@ describe("one instance per process", () => { ); }); - it("with the hosted Deco CMS, the site ID and token are part of the key", () => { + it("with hosted releases, the site ID is part of the key; the token isn't (another one warns)", () => { + const warn = vi.spyOn(console, "warn").mockImplementation(() => {}); const content = docsSnapshot(); const a = createCMS({ blocks: {}, content, site: "acme", token: "t1" }); expect(instanceOf(createCMS({ blocks: {}, content, site: "acme", token: "t1" }))).toBe( @@ -525,9 +514,11 @@ describe("one instance per process", () => { expect(instanceOf(createCMS({ blocks: {}, content, site: "other", token: "t1" }))).not.toBe( instanceOf(a), ); - expect(instanceOf(createCMS({ blocks: {}, content, site: "acme", token: "t2" }))).not.toBe( + expect(warn).not.toHaveBeenCalled(); + expect(instanceOf(createCMS({ blocks: {}, content, site: "acme", token: "t2" }))).toBe( instanceOf(a), ); + expect(warn).toHaveBeenCalledWith(expect.stringContaining("(token)")); expect(instanceOf(createCMS({ blocks: {}, content }))).not.toBe(instanceOf(a)); }); @@ -617,3 +608,68 @@ describe("one instance per process", () => { expect(load).toHaveBeenCalledTimes(1); }); }); + +describe("preview.draftHosts", () => { + it("replaces the default draft hosts a draft pointer's host must fall under", async () => { + const fetch = vi.fn(async (_input: string | URL | Request) => + Response.json({ set: {}, delete: [] }), + ); + vi.stubGlobal("fetch", fetch); + const pointer = "drafts.example.com/sites/acme/drafts/x.json@1"; + const defaults = createCMS({ blocks: docsBlocks(), content: docsSnapshot() }); + expect((await defaults.forDraft(pointer).resolve("SummerSEO"))[1]?.code).toBe("LOADER_FAILED"); + expect(fetch).not.toHaveBeenCalled(); + resetForTests(); + const own = createCMS({ + blocks: docsBlocks(), + content: docsSnapshot(), + preview: { draftHosts: [" Drafts.Example.com "] }, + }); + expect((await own.forDraft(pointer).resolve("SummerSEO"))[1]).toBeNull(); + expect(String(fetch.mock.calls[0]?.[0])).toBe( + "https://drafts.example.com/sites/acme/drafts/x.json?v=1", + ); + const studio = "delivery.decocms.com/sites/acme/drafts/x.json@1"; + expect((await own.forDraft(studio).resolve("SummerSEO"))[1]?.code).toBe("LOADER_FAILED"); + }); + + it("reads no DECO_PREVIEW_API_DOMAINS", async () => { + vi.stubEnv("DECO_PREVIEW_API_DOMAINS", "drafts.example.com"); + const fetch = vi.fn(async () => Response.json({ set: {}, delete: [] })); + vi.stubGlobal("fetch", fetch); + const cms = createCMS({ blocks: docsBlocks(), content: docsSnapshot() }); + const [, error] = await cms.forDraft("drafts.example.com/x.json@1").resolve("SummerSEO"); + expect(error?.code).toBe("LOADER_FAILED"); + expect(fetch).not.toHaveBeenCalled(); + }); + + it("throws a TypeError at createCMS on anything but a list of hosts", () => { + for (const draftHosts of ["a.example", [""], [1], null]) { + expect(() => + createCMS({ + blocks: {}, + content: docsSnapshot(), + preview: { draftHosts: draftHosts as never }, + }), + ).toThrow(TypeError); + } + }); +}); + +describe("site and token", () => { + it("token without site throws a configuration error", () => { + expect(() => createCMS({ blocks: {}, content: docsSnapshot(), token: "tok" })).toThrow( + new TypeError("createCMS: token needs site: pass both, or site alone"), + ); + }); + + it("accepts site with token, site alone, or neither", () => { + expect(() => + createCMS({ blocks: {}, content: docsSnapshot(), site: "acme", token: "tok" }), + ).not.toThrow(); + resetForTests(); + expect(() => createCMS({ blocks: {}, content: docsSnapshot(), site: "acme" })).not.toThrow(); + resetForTests(); + expect(() => createCMS({ blocks: {}, content: docsSnapshot() })).not.toThrow(); + }); +}); diff --git a/packages/blocks/src/v8/cms.ts b/packages/blocks/src/v8/cms.ts index 1dc381b1..d87b1717 100644 --- a/packages/blocks/src/v8/cms.ts +++ b/packages/blocks/src/v8/cms.ts @@ -13,8 +13,9 @@ import { secretBlock } from "./builtins/secret.ts"; import { CMSClient } from "./client.ts"; import { ContentStore, isLoader, isSnapshot } from "./content.ts"; import { draftCookieFor, endsPreview, parseDraftPointer, readDraftPointer } from "./draft.ts"; +import { parseDraftHosts } from "./draftChanges.ts"; import { allowsHost, type HostPattern, parseHostPattern } from "./hosts.ts"; -import { clearGlobals, contentIdentity, fnv1a, readEnv } from "./identity.ts"; +import { clearGlobals, contentIdentity, fnv1a } from "./identity.ts"; import { isPlainObject } from "./json.ts"; import { remoteLoader, resetRemoteLoaders } from "./remoteLoader.ts"; import { @@ -55,6 +56,7 @@ interface Fingerprint { telemetry: string; preview: string; secrets: string; + token: string; } /** @@ -87,13 +89,13 @@ class CMSInstance { constructor(config: CMSConfig, interval: number) { this.config = config; this.#interval = interval; - this.#store = new ContentStore(contentOf(config)); + this.#store = new ContentStore(contentOf(config), parseDraftHosts(config.preview)); this.fingerprint = fingerprintOf(config, interval); this.#caps = { hosts: parseCodeHosts(config.preview), limits: telemetryLimits(config.telemetry), }; - const destination = resolveDestination(config.telemetry, config.site); + const destination = resolveDestination(config.telemetry, config.site, config.token); if (destination !== null) { this.#telemetry = new TelemetryPipeline(destination); setCurrentTelemetry(this.#telemetry); @@ -153,10 +155,6 @@ class CMSInstance { return this.#client(blocks, () => load().then((snapshot) => forceVariants(snapshot, variants))); } - forRevision(blocks: Blocks, revision: string): Client { - return this.#client(blocks, () => this.#store.revision(revision)); - } - update(): Promise<{ updated: boolean }> { this.#nextCheck = this.#due(); return this.#store.update(); @@ -302,10 +300,6 @@ class CMSHandle implements CMS { return this.#instance.forDraft(this.#blocks, pointer); } - forRevision(revision: string): Client { - return this.#instance.forRevision(this.#blocks, revision); - } - update(): Promise<{ updated: boolean }> { return this.#instance.update(); } @@ -365,10 +359,10 @@ export function resetForTests(): void { setCurrentTelemetry(undefined); } -/** With `site` and `token`, the content is the fallback of hosted releases. */ +/** With `site`, the content is the fallback of hosted releases. */ function contentOf(config: CMSConfig): Snapshot | Loader { - if (!config.site || !config.token) return config.content; - return remoteLoader(config.content, { site: config.site, token: config.token }); + if (!config.site) return config.content; + return remoteLoader(config.content, { site: config.site }); } function validate(config: CMSConfig): void { @@ -383,12 +377,16 @@ function validate(config: CMSConfig): void { "createCMS: `content` must be the content module ({ revision, blocks }) or a loader with load()", ); } + // The token is the site's credential: it means nothing without the site it belongs to. + if (config.token && !config.site) { + throw new TypeError("createCMS: token needs site: pass both, or site alone"); + } parseCodeHosts(config.preview); + parseDraftHosts(config.preview); } function resolveInterval(configured: number | undefined): number { - const env = readEnv("DECO_CONTENT_INTERVAL"); - const raw = configured ?? (env ? Number(env) : MIN_INTERVAL); + const raw = configured ?? MIN_INTERVAL; if (!Number.isFinite(raw)) return MIN_INTERVAL; if (raw < MIN_INTERVAL) { console.warn( @@ -401,8 +399,7 @@ function resolveInterval(configured: number | undefined): number { /** * The instance key: the content's identity (never its revision, so a hot - * reload keeps the instance) and, with the hosted Deco CMS, the site and a - * hash of the token, which never appears in the global symbol registry. + * reload keeps the instance) and, with hosted releases, the site. * * A content module is identified by the `.deco` folder it was generated from * (its `root`). One without a `root` is identified by the object itself, so @@ -411,8 +408,7 @@ function resolveInterval(configured: number | undefined): number { */ function identityOf(config: CMSConfig): string { const content = contentIdentity(config.content); - if (!config.site || !config.token) return content; - return `${content}|site:${config.site}|token:${fnv1a(config.token)}`; + return config.site ? `${content}|site:${config.site}` : content; } function fingerprintOf(config: CMSConfig, interval: number): Fingerprint { @@ -421,6 +417,7 @@ function fingerprintOf(config: CMSConfig, interval: number): Fingerprint { telemetry: stableJson(config.telemetry ?? null), preview: stableJson(config.preview ?? null), secrets: config.secrets?.key ? fnv1a(config.secrets.key) : "", + token: config.token ? fnv1a(config.token) : "", }; } diff --git a/packages/blocks/src/v8/content.ts b/packages/blocks/src/v8/content.ts index 3f367c99..d778c34c 100644 --- a/packages/blocks/src/v8/content.ts +++ b/packages/blocks/src/v8/content.ts @@ -1,30 +1,30 @@ /** - * The content side of a CMS: one source (the content module or a `Loader`), - * the caches every client shares, and the release revisions this process has - * served. Clients only ever see whole `{ revision, blocks }` snapshots. + * The content side of a CMS: one source (the content module or a `Loader`) + * and the caches every client shares. Clients only ever see whole + * `{ revision, blocks }` snapshots. * * A draft is the release with a draft's changes layered over it (see - * ./draftChanges.ts). Drafts are never recorded as served: - * `forRevision(revision)` takes a revision string a client may hand back, so - * it must only ever reach published content, never a draft someone loaded - * with a pointer. + * ./draftChanges.ts). */ import { formatDraftPointer, parseDraftPointer } from "./draft.ts"; -import { type DraftChanges, fetchDraftChanges, LOCAL_VERSION, layerDraft } from "./draftChanges.ts"; +import { + DEFAULT_DRAFT_HOSTS, + type DraftChanges, + fetchDraftChanges, + LOCAL_VERSION, + layerDraft, +} from "./draftChanges.ts"; import { errors, isResolutionError } from "./errors.ts"; import { isPlainObject } from "./json.ts"; import type { Loader, Snapshot } from "./types.ts"; -const SERVED_REVISIONS = 16; /** Fetched drafts kept per CMS. */ const CACHED_DRAFTS = 3; -/** How long a fetched draft is reused before it's fetched again, so its token is checked again. */ -const DRAFT_TTL_MS = 60_000; -/** A fetched draft, and the views of it over each release it was layered on. */ +/** A fetched draft body, its ETag, and the views of it over each release it was layered on. */ interface CachedDraft { - changes: Promise; - at: number; + changes: DraftChanges; + etag: string | null; views: WeakMap; } @@ -55,7 +55,9 @@ export function isSnapshot(content: unknown): content is Snapshot { isPlainObject(content) && typeof content.revision === "string" && isPlainObject(content.blocks) && - (content.aliases === undefined || isPlainObject(content.aliases)) + (content.aliases === undefined || isPlainObject(content.aliases)) && + (content.schemaHash === undefined || typeof content.schemaHash === "string") && + (content.committedAt === undefined || typeof content.committedAt === "string") ); } @@ -64,12 +66,15 @@ export class ContentStore { #release: Promise | undefined; #updating: Promise<{ updated: boolean }> | undefined; readonly #drafts = new BoundedMap(CACHED_DRAFTS); - readonly #served = new BoundedMap(SERVED_REVISIONS); + /** Draft reads in flight, so concurrent renders of one draft share one request. */ + readonly #draftReads = new Map>(); + readonly #draftHosts: readonly string[]; /** The release this store last handed a client, for a loader that can't peek. */ #latest: Snapshot | undefined; - constructor(source: Snapshot | Loader) { + constructor(source: Snapshot | Loader, draftHosts: readonly string[] = DEFAULT_DRAFT_HOSTS) { this.#source = source; + this.#draftHosts = draftHosts; } /** Whether the source can change while the process runs (has `update()`). */ @@ -98,12 +103,13 @@ export class ContentStore { } /** - * The draft a pointer names: its changes, fetched from a preview API domain, + * The draft a pointer names: its changes, fetched from a draft host, * layered over the release. A pointer that doesn't parse, or any failure, is * `LOADER_FAILED`, never a silent fallback to the release. A pointer whose * version is `local` names no draft: the release, with nothing fetched. The * changes are keyed by the pointer without its `__variant` parameters (every - * variant of one draft shares one fetch) and reused for a minute. + * variant of one draft shares one body) and revalidated on every read: a + * `304` reuses the body held, a `200` replaces it. */ draft(pointer: string): Promise { const parsed = parseDraftPointer(pointer); @@ -113,28 +119,38 @@ export class ContentStore { if (parsed.version === LOCAL_VERSION) return this.release(); const { host, path, version } = parsed; const key = formatDraftPointer({ host, path, version }); - let entry = this.#drafts.get(key); - if (entry === undefined || Date.now() - entry.at >= DRAFT_TTL_MS) { - const fetched: CachedDraft = { - changes: fetchDraftChanges(parsed).catch((error: unknown) => { - throw errors.loaderFailed("the draft's changes couldn't be fetched", error); - }), - at: Date.now(), - views: new WeakMap(), + let read = this.#draftReads.get(key); + if (read === undefined) { + const held = this.#drafts.get(key); + read = fetchDraftChanges(parsed, { domains: this.#draftHosts, etag: held?.etag }).then( + (result) => { + if (result.status === 304 && held !== undefined) return held; + if (result.status === 304) throw new Error("draft changes: 304 without a held body"); + const entry: CachedDraft = { + changes: result.changes, + etag: result.etag, + views: new WeakMap(), + }; + this.#drafts.set(key, entry); + return entry; + }, + ); + const pending = read; + this.#draftReads.set(key, pending); + const settle = () => { + if (this.#draftReads.get(key) === pending) this.#draftReads.delete(key); }; - this.#drafts.set(key, fetched); - // A failure isn't reused: the next client fetches again. - fetched.changes.catch(() => { - if (this.#drafts.get(key) === fetched) this.#drafts.delete(key); - }); - entry = fetched; + pending.then(settle, settle); } - const { changes, views } = entry; - return Promise.all([this.release(), changes]).then(([base, draft]) => { - let view = views.get(base); + const changes = read.catch((error: unknown) => { + throw errors.loaderFailed("the draft's changes couldn't be fetched", error); + }); + return Promise.all([this.release(), changes]).then(([base, entry]) => { + let view = entry.views.get(base); if (view === undefined) { - view = layerDraft(base, draft, version); - views.set(base, view); + // OPEN: the view is keyed by the body's ETag (the pointer outlives later saves); the version without one. + view = layerDraft(base, entry.changes, entry.etag ?? version); + entry.views.set(base, view); } return view; }); @@ -149,12 +165,6 @@ export class ContentStore { return peekRelease(this.#source) ?? this.#latest; } - /** A release revision this store has served, or the release when it's unknown (drafts included). */ - revision(revision: string): Promise { - const served = this.#served.get(revision); - return served === undefined ? this.release() : Promise.resolve(served); - } - /** Asks the source for newer content; never throws, and concurrent calls share one check. */ update(): Promise<{ updated: boolean }> { const source = this.#source; @@ -197,7 +207,6 @@ export class ContentStore { this.#drafts.clear(); return; } - if (isSnapshot(previous)) this.#served.delete(previous.revision); this.#source = source; this.#release = undefined; this.#latest = undefined; @@ -220,7 +229,6 @@ export class ContentStore { #serve(snapshot: Snapshot): Snapshot { this.#latest = snapshot; - this.#served.set(snapshot.revision, snapshot); return snapshot; } } diff --git a/packages/blocks/src/v8/draftChanges.ts b/packages/blocks/src/v8/draftChanges.ts index 4b830544..599da3e2 100644 --- a/packages/blocks/src/v8/draftChanges.ts +++ b/packages/blocks/src/v8/draftChanges.ts @@ -2,38 +2,43 @@ * Draft changes (see /next/content-delivery#draft-previews): what a draft * pointer's address answers, and how the CMS layers it over production. * - * - The pointer's host must fall under one of the preview API domains (the - * same rule as v7): by default `*.decocms.com` (Studio, its PR previews, - * `local.studio.decocms.com`) and the loopback hosts; `DECO_PREVIEW_API_DOMAINS` - * replaces the list. Any other host is refused before anything is fetched. - * - `GET ://&v=`: the scheme comes from the + * - The pointer's host must fall under one of the draft hosts (the same rule + * as v7's preview API domains): by default `*.decocms.com` (Studio, the + * delivery CDN, `local.studio.decocms.com`) and the loopback hosts; + * `createCMS({ preview: { draftHosts } })` replaces the list. Any other host + * is refused before anything is fetched. + * - `GET ://?v=`: the scheme comes from the * domain that admitted the host, never from the pointer: plain `http` only * for loopback hosts. The pointer's `__variant` parameters are already gone (the parser * lifts them out), no cookies or credentials are sent, and redirects aren't * followed (a response that was redirected anyway is refused). - * - Only a 200 is accepted, within 10 s and up to 16 MiB, shaped exactly - * `{ format: 1, set: { : }, delete: […] }` with - * `set` and `delete` disjoint. + * - Every read revalidates: with a body this process already holds, it sends + * `If-None-Match` with that body's ETag, and a `304` reuses the body. Only + * a 200 or that 304 is accepted, within 10 s; a 200 is up to 16 MiB, shaped + * exactly `{ set: { : }, delete: […] }` with `set` + * and `delete` disjoint. */ import { readBoundedJson, timeoutSignal } from "./boundedJson.ts"; -import { readEnv } from "./identity.ts"; import { isPlainObject } from "./json.ts"; import type { DraftPointer, Snapshot } from "./types.ts"; /** The version a pointer has when it names no draft: `deco serve`'s working tree. */ export const LOCAL_VERSION = "local"; -const FORMAT = 1; const TIMEOUT_MS = 10_000; const MAX_BYTES = 16 * 1024 * 1024; /** What a draft changed compared with production: whole blocks, and deletions. */ export interface DraftChanges { - format: typeof FORMAT; set: Record; delete: string[]; } +/** A draft read: new changes (and their ETag), or `304`, the held body is current. */ +export type DraftRead = + | { status: 200; changes: DraftChanges; etag: string | null } + | { status: 304 }; + /** * The domains a draft pointer's host may fall under, the defaults v7 shipped. * Deco operates every one of them, so the defaults add no SSRF surface. An @@ -42,37 +47,48 @@ export interface DraftChanges { * matches that exact host. The first entry that matches decides whether a * port and plain `http` are allowed. */ -export const DEFAULT_PREVIEW_API_DOMAINS: readonly string[] = [ +export const DEFAULT_DRAFT_HOSTS: readonly string[] = [ "local.studio.decocms.com", // the Studio dev origin (https, with a port) "localhost", "127.0.0.1", ".localhost", - ".decocms.com", // Studio and its preview deployments + ".decocms.com", // Studio, its preview deployments and delivery.decocms.com ]; -/** `DECO_PREVIEW_API_DOMAINS` (a comma list) when set, else the defaults. */ -function previewApiDomains(): readonly string[] { - const configured = (readEnv("DECO_PREVIEW_API_DOMAINS") ?? "") - .split(",") - .map((domain) => domain.trim().toLowerCase()) - .filter(Boolean); - return configured.length > 0 ? configured : DEFAULT_PREVIEW_API_DOMAINS; +/** + * `createCMS`'s `preview.draftHosts`, trimmed and lowercased, or the defaults + * when it's left out. Throws a `TypeError` on anything but a list of + * non-empty strings. + */ +export function parseDraftHosts(preview: { draftHosts?: unknown } | undefined): readonly string[] { + const domains = preview?.draftHosts; + if (domains === undefined) return DEFAULT_DRAFT_HOSTS; + if ( + !Array.isArray(domains) || + domains.some((domain) => typeof domain !== "string" || domain.trim() === "") + ) { + throw new TypeError("createCMS: `preview.draftHosts` must be a list of hosts"); + } + return domains.map((domain: string) => domain.trim().toLowerCase()); } /** - * The origin a pointer's host is fetched from, or `null` when no preview API - * domain admits it. Loopback domains (and `local.studio.decocms.com`) may + * The origin a pointer's host is fetched from, or `null` when no draft host + * admits it. Loopback domains (and `local.studio.decocms.com`) may * carry a port; a public domain may not, so a pointer can't aim the fetch at * an odd port. Loopback hosts are `http`; everything else is `https`. */ -export function previewApiOrigin(authority: string): string | null { +export function previewApiOrigin( + authority: string, + domains: readonly string[] = DEFAULT_DRAFT_HOSTS, +): string | null { const lower = authority.toLowerCase(); const end = lower.startsWith("[") ? lower.indexOf("]") + 1 : -1; const host = end > 0 ? lower.slice(0, end) : lower.split(":")[0]!; const rest = end > 0 ? lower.slice(end) : lower.slice(host.length); const port = rest.startsWith(":") ? rest.slice(1) : undefined; if (!host) return null; - const domain = previewApiDomains().find((d) => + const domain = domains.find((d) => d.startsWith(".") ? host.length > d.length && host.endsWith(d) : host === d, ); if (domain === undefined) return null; @@ -92,26 +108,39 @@ function isLoopbackDomain(domain: string): boolean { /** * The URL the SDK fetches for a pointer: the origin its host is admitted - * under, the version as `v`. `null` when no preview API domain admits the host. + * under, the version as `v`. `null` when no draft host admits the host. */ -export function draftChangesUrl(pointer: DraftPointer): string | null { - const origin = previewApiOrigin(pointer.host); +export function draftChangesUrl( + pointer: DraftPointer, + domains: readonly string[] = DEFAULT_DRAFT_HOSTS, +): string | null { + const origin = previewApiOrigin(pointer.host, domains); if (origin === null) return null; const separator = pointer.path.includes("?") ? "&" : "?"; return `${origin}${pointer.path}${separator}v=${encodeURIComponent(pointer.version)}`; } -/** Fetches a draft's changes; rejects on a host no preview API domain admits (without fetching) or on any failure. */ -export async function fetchDraftChanges(pointer: DraftPointer): Promise { - const url = draftChangesUrl(pointer); +/** + * Reads a draft's changes, revalidating `etag` (the ETag of the body the + * caller holds) when given. Rejects on a host no draft host admits + * (without fetching), on a `304` to a request that sent no ETag, and on any + * other failure. + */ +export async function fetchDraftChanges( + pointer: DraftPointer, + options: { domains?: readonly string[]; etag?: string | null } = {}, +): Promise { + const url = draftChangesUrl(pointer, options.domains); if (url === null) { throw new Error( - `draft pointer names "${pointer.host}", which isn't under a preview API domain ` + - "(DECO_PREVIEW_API_DOMAINS); nothing was fetched", + `draft pointer names "${pointer.host}", which isn't under a draft host ` + + "(createCMS({ preview: { draftHosts } })); nothing was fetched", ); } + const headers: Record = { accept: "application/json" }; + if (options.etag) headers["if-none-match"] = options.etag; const response = await fetch(url, { - headers: { accept: "application/json" }, + headers, redirect: "manual", signal: timeoutSignal(TIMEOUT_MS), }); @@ -124,18 +153,24 @@ export async function fetchDraftChanges(pointer: DraftPointer): Promise new Error(`draft changes: ${why}`); - if (!isPlainObject(body) || body.format !== FORMAT) throw invalid("unknown format"); - if (Object.keys(body).some((key) => key !== "format" && key !== "set" && key !== "delete")) { + if (!isPlainObject(body)) throw invalid("expected an object"); + if (Object.keys(body).some((key) => key !== "set" && key !== "delete")) { throw invalid("unexpected field"); } const { set, delete: deleted } = body; @@ -143,7 +178,6 @@ export function parseDraftChanges(body: unknown): DraftChanges { if (!Array.isArray(deleted) || deleted.some((name) => typeof name !== "string")) { throw invalid("delete must be a list of names"); } - if (new Set(deleted).size !== deleted.length) throw invalid("delete repeats a name"); if (deleted.some((name) => Object.hasOwn(set, name))) throw invalid("set and delete overlap"); return body as unknown as DraftChanges; } @@ -152,9 +186,11 @@ export function parseDraftChanges(body: unknown): DraftChanges { * The draft as a snapshot: production's entries, with the changed ones * replaced whole and the deleted ones gone. A shallow copy: every unchanged * entry is production's own object, and production is never mutated. Its - * revision is an opaque identity of the pair, never a release's. + * revision, `~`, is an opaque identity of the pair, + * never a release's; `view` names the draft body (its ETag), so each save + * re-keys what's cached by revision. */ -export function layerDraft(base: Snapshot, changes: DraftChanges, version: string): Snapshot { +export function layerDraft(base: Snapshot, changes: DraftChanges, view: string): Snapshot { const blocks: Record = { ...base.blocks }; for (const name of changes.delete) delete blocks[name]; for (const [name, entry] of Object.entries(changes.set)) { @@ -166,7 +202,7 @@ export function layerDraft(base: Snapshot, changes: DraftChanges, version: strin configurable: true, }); } - const view: Snapshot = { revision: `${base.revision}~${version}`, blocks }; - if (base.aliases !== undefined) view.aliases = base.aliases; - return view; + const layered: Snapshot = { revision: `${base.revision}~${view}`, blocks }; + if (base.aliases !== undefined) layered.aliases = base.aliases; + return layered; } diff --git a/packages/blocks/src/v8/identity.ts b/packages/blocks/src/v8/identity.ts index 5d0c4580..47eb735e 100644 --- a/packages/blocks/src/v8/identity.ts +++ b/packages/blocks/src/v8/identity.ts @@ -46,16 +46,6 @@ export function fnv1a(input: string): string { return (hash >>> 0).toString(16).padStart(8, "0"); } -/** Reads an environment variable where there is one (Node, Bun, Workers with nodejs_compat). */ -export function readEnv(name: string): string | undefined { - try { - return (globalThis as { process?: { env?: Record } }).process - ?.env?.[name]; - } catch { - return undefined; - } -} - /** Clears every global instance whose registry key starts with `prefix`. */ export function clearGlobals(prefix: string): void { const store = globalThis as unknown as Record; diff --git a/packages/blocks/src/v8/noEnv.test.ts b/packages/blocks/src/v8/noEnv.test.ts new file mode 100644 index 00000000..15703262 --- /dev/null +++ b/packages/blocks/src/v8/noEnv.test.ts @@ -0,0 +1,69 @@ +// @vitest-environment node +/** + * The SDK reads no environment variable: configuration is explicit + * `createCMS` params (`site`, `token`, `interval`, `telemetry`, `preview`). + * The one exception is `NODE_ENV=development` in remoteLoader.ts, which keeps + * local development off hosted releases (local files win). + */ +import fs from "node:fs"; +import path from "node:path"; +import { fileURLToPath } from "node:url"; +import { describe, expect, it } from "vitest"; + +const SRC = path.resolve(path.dirname(fileURLToPath(import.meta.url)), ".."); +const ENV_READ = + /\bprocess\b\s*\??\.\s*env\b|\{\s*env\s*\}\s*=\s*(globalThis\s*\??\.\s*)?process\b|Bun\s*\??\.\s*env\b|Deno\s*\??\.\s*env\b|import\.meta\.env|\.env\s*\[|\.env\s*\?\./; +const ALLOWED = [ + { file: "v8/remoteLoader.ts", line: 'return process.env.NODE_ENV === "development";' }, +]; + +function sources(dir: string): string[] { + return fs.readdirSync(dir, { withFileTypes: true }).flatMap((entry) => { + const full = path.join(dir, entry.name); + if (entry.isDirectory()) { + return entry.name.startsWith("__") ? [] : sources(full); + } + if (!/\.tsx?$/.test(entry.name) || /\.test\.tsx?$/.test(entry.name)) return []; + return entry.name === "testFixtures.ts" ? [] : [full]; + }); +} + +describe("no environment variables", () => { + it("the guard catches every spelling of an env read", () => { + for (const read of [ + "process.env.X", + "process?.env.X", + "globalThis.process?.env.X", + "const { env } = process;", + "const { env } = globalThis.process;", + "Bun.env.X", + "Deno.env.get('X')", + "import.meta.env.X", + "globalThis.env?.X", + ]) { + expect(ENV_READ.test(read), read).toBe(true); + } + }); + + it("packages/blocks/src reads none, except NODE_ENV=development in remoteLoader.ts", () => { + const hits: string[] = []; + for (const file of sources(SRC)) { + const rel = path.relative(SRC, file).split(path.sep).join("/"); + fs.readFileSync(file, "utf8") + .split("\n") + .forEach((text, index) => { + if (!ENV_READ.test(text) || text.trim().startsWith("//") || text.trim().startsWith("*")) { + return; + } + if (ALLOWED.some((a) => a.file === rel && text.trim() === a.line)) return; + hits.push(`${rel}:${index + 1}: ${text.trim()}`); + }); + } + expect(hits).toEqual([]); + }); + + it("the NODE_ENV exception is still there, exactly once", () => { + const source = fs.readFileSync(path.join(SRC, "v8/remoteLoader.ts"), "utf8"); + expect(source.split(ALLOWED[0]!.line).length - 1).toBe(1); + }); +}); diff --git a/packages/blocks/src/v8/remoteLoader.test.ts b/packages/blocks/src/v8/remoteLoader.test.ts index 9e0070ce..3e80d889 100644 --- a/packages/blocks/src/v8/remoteLoader.test.ts +++ b/packages/blocks/src/v8/remoteLoader.test.ts @@ -2,23 +2,22 @@ /** * remoteLoader (hosted-releases-internals.mdx, hosted-publishing.mdx, * content-delivery.mdx): requests read memory, and the background check - * follows the channel manifest. Drafts aren't the loader's (see + * follows `sites//latest.json`. Drafts aren't the loader's (see * draftChanges.test.ts). */ import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; -import { computeContentRevision } from "./canonical"; import { createCMS, resetForTests } from "./cms"; import { remoteLoader } from "./remoteLoader"; import { docsBlocks, docsSnapshot } from "./testFixtures"; import type { Loader, Snapshot } from "./types"; -/** With `site` and `token` set, `remoteLoader` returns a loader. */ -const remote = (...args: Parameters) => remoteLoader(...args); - -const HOSTED_DELIVERY_ORIGIN = "https://delivery.decocms.com"; +const ORIGIN = "https://delivery.decocms.com"; const SITE = "acme"; -const TOKEN = "site-token"; -const MANIFEST_URL = `${HOSTED_DELIVERY_ORIGIN}/sites/acme/channels/production.json`; +const LATEST_URL = `${ORIGIN}/sites/acme/latest.json`; +const SCHEMA = "a".repeat(64); +const OTHER_SCHEMA = "b".repeat(64); +const SHA_1 = "1".repeat(40); +const SHA_2 = "2".repeat(40); beforeEach(() => resetForTests()); afterEach(() => { @@ -27,246 +26,420 @@ afterEach(() => { vi.restoreAllMocks(); }); -/** A snapshot whose revision is its real content hash, as the CLI and the Deco API compute it. */ -async function hashed(title: string): Promise { - const blocks = { - SummerSEO: { __resolveType: "seo", title, description: "d" }, +/** The bundled content module: a content-hash revision and the schema it was built with. */ +function bundled(schemaHash: string | null = SCHEMA, committedAt?: string): Snapshot { + const snapshot: Snapshot = { + ...docsSnapshot(), + aliases: { "website/sections/Seo.tsx": "seo" }, }; - return { revision: await computeContentRevision(blocks), blocks }; + if (schemaHash !== null) snapshot.schemaHash = schemaHash; + if (committedAt !== undefined) snapshot.committedAt = committedAt; + return snapshot; +} + +/** What Studio writes for a commit: the blocks at that commit. */ +function blocksAt(title: string): Record { + return { SummerSEO: { __resolveType: "seo", title, description: "d" } }; } -/** A fake delivery API: a channel manifest and immutable revision assets. */ -function deliveryApi() { - const assets = new Map(); - let manifest: Record | undefined; +/** A fake delivery CDN: `latest.json` and immutable `revisions/.json`. */ +function delivery() { + const objects = new Map(); const requests: { url: string; headers: Record }[] = []; const fetch = vi.fn(async (input: string | URL | Request, init?: RequestInit) => { const url = String(input); - const headers = Object.fromEntries(new Headers(init?.headers).entries()); - requests.push({ url, headers }); - if (url === MANIFEST_URL) { - if (!manifest) return new Response("missing", { status: 404 }); - return Response.json(manifest); - } - const path = url.slice(HOSTED_DELIVERY_ORIGIN.length).split("?")[0] ?? ""; - if (assets.has(path)) return Response.json(assets.get(path)); - return new Response("not found", { status: 404 }); + requests.push({ url, headers: Object.fromEntries(new Headers(init?.headers).entries()) }); + const key = url.slice(ORIGIN.length); + if (!objects.has(key)) return new Response("not found", { status: 404 }); + return Response.json(objects.get(key)); }); vi.stubGlobal("fetch", fetch); return { fetch, requests, - publish(generation: number, snapshot: Snapshot, body: unknown = snapshot) { - const path = `/sites/acme/revisions/${snapshot.revision}.json`; - assets.set(path, body); - manifest = { format: 1, generation, revision: snapshot.revision, snapshot: path }; + urls: () => requests.map((r) => r.url), + /** Studio's publish: the revision, then the pointer. */ + publish( + sha: string, + title: string, + schemaHash = SCHEMA, + publishedAt = "2026-10-06T12:00:00.000Z", + ) { + objects.set(`/sites/acme/revisions/${sha}.json`, { + revision: sha, + schemaHash, + blocks: blocksAt(title), + }); + objects.set("/sites/acme/latest.json", { + revision: sha, + schemaHash, + publishedAt, + }); }, - setManifest(value: Record) { - manifest = value; + /** Studio's "Make current": rewrites the pointer only. */ + point(value: unknown) { + objects.set("/sites/acme/latest.json", value); }, - asset(path: string, body: unknown) { - assets.set(path, body); + put(key: string, value: unknown) { + objects.set(key, value); }, - remove(path: string) { - assets.delete(path); + remove(key: string) { + objects.delete(key); }, }; } -describe("remoteLoader: releases", () => { - it("serves the fallback from memory until a release is fetched, with no network on load()", async () => { - const api = deliveryApi(); - const fallback = docsSnapshot(); - const loader = remote(fallback, { site: SITE, token: TOKEN }); +describe("remoteLoader: boot", () => { + it("load() serves the bundled content from memory and never touches the network", async () => { + const api = delivery(); + api.publish(SHA_1, "Published"); + const fallback = bundled(); + const loader = remoteLoader(fallback, { site: SITE }); expect(await loader.load()).toBe(fallback); expect(api.fetch).not.toHaveBeenCalled(); }); - it("asks for the manifest with the site token, fetches a new revision, verifies it and swaps it in whole", async () => { - const api = deliveryApi(); - const release = await hashed("Published"); - api.publish(1, release); - const loader = remote(docsSnapshot(), { site: SITE, token: TOKEN }); + it("createCMS with site makes no request before the first background check", async () => { + const api = delivery(); + const cms = createCMS({ blocks: docsBlocks(), content: bundled(), site: SITE }); + expect(api.fetch).not.toHaveBeenCalled(); + expect(await cms.forRelease().revision()).toBe("rev-1"); + }); +}); + +describe("remoteLoader: update()", () => { + it("reads latest.json with no credentials, downloads revisions/.json and swaps it in whole", async () => { + const api = delivery(); + api.publish(SHA_1, "Published"); + const fallback = bundled(); + const loader = remoteLoader(fallback, { site: SITE }); expect(await loader.update?.()).toEqual({ updated: true }); - expect(await loader.load()).toEqual(release); - expect(api.requests[0]).toMatchObject({ - url: MANIFEST_URL, - headers: { authorization: `Bearer ${TOKEN}` }, + expect(api.urls()).toEqual([LATEST_URL, `${ORIGIN}/sites/acme/revisions/${SHA_1}.json`]); + expect(api.requests.every((r) => r.headers.authorization === undefined)).toBe(true); + expect(await loader.load()).toEqual({ + revision: SHA_1, + schemaHash: SCHEMA, + blocks: blocksAt("Published"), + aliases: fallback.aliases, // the bundled alias table: aliases come from code + }); + }); + + it("the first check downloads even content equal to the bundle's: no comparison with the bundle", async () => { + const api = delivery(); + const fallback = bundled(); + api.put(`/sites/acme/revisions/${SHA_1}.json`, { + revision: SHA_1, + schemaHash: SCHEMA, + blocks: fallback.blocks, }); - expect(api.requests[1]?.url).toBe( - `${HOSTED_DELIVERY_ORIGIN}/sites/acme/revisions/${release.revision}.json`, - ); + api.point({ revision: SHA_1, schemaHash: SCHEMA, publishedAt: "2026-10-06T12:00:00.000Z" }); + const loader = remoteLoader(fallback, { site: SITE }); + expect(await loader.update?.()).toEqual({ updated: true }); + expect(api.urls()).toHaveLength(2); + expect((await loader.load()).revision).toBe(SHA_1); + }); + + it("the same revision as the one last swapped in downloads nothing", async () => { + const api = delivery(); + api.publish(SHA_1, "Published"); + const loader = remoteLoader(bundled(), { site: SITE }); + await loader.update?.(); + expect(await loader.update?.()).toEqual({ updated: false }); + expect(api.urls()).toEqual([ + LATEST_URL, + `${ORIGIN}/sites/acme/revisions/${SHA_1}.json`, + LATEST_URL, + ]); }); - it("uses the fallback without downloading when the release has the fallback's revision", async () => { - const api = deliveryApi(); - const fallback = await hashed("Bundled"); - api.publish(1, fallback); - const loader = remote(fallback, { site: SITE, token: TOKEN }); + it("follows the pointer it reads, older revisions included (a rollback), with no ordering among pointers", async () => { + const api = delivery(); + api.publish(SHA_1, "One"); + api.publish(SHA_2, "Two"); + const loader = remoteLoader(bundled(), { site: SITE }); + await loader.update?.(); + expect((await loader.load()).revision).toBe(SHA_2); + api.point({ revision: SHA_1, schemaHash: SCHEMA, publishedAt: "2026-01-01T00:00:00.000Z" }); + expect(await loader.update?.()).toEqual({ updated: true }); + const loaded = await loader.load(); + expect(loaded.revision).toBe(SHA_1); + expect(loaded.blocks).toEqual(blocksAt("One")); + }); + + it("swaps only when schemaHash equals the bundled content's; otherwise keeps the bundle", async () => { + const api = delivery(); + api.publish(SHA_1, "Published", OTHER_SCHEMA); + const fallback = bundled(); + const loader = remoteLoader(fallback, { site: SITE }); expect(await loader.update?.()).toEqual({ updated: false }); - expect(api.requests.map((r) => r.url)).toEqual([MANIFEST_URL]); + expect(api.urls()).toEqual([LATEST_URL]); expect(await loader.load()).toBe(fallback); }); - it("retries a release whose download failed on the next check", async () => { - const api = deliveryApi(); - const release = await hashed("Published"); - api.publish(1, release); - const path = `/sites/acme/revisions/${release.revision}.json`; - api.remove(path); - const fallback = docsSnapshot(); - const loader = remote(fallback, { site: SITE, token: TOKEN }); + it("a process that already swapped keeps its release when the pointer's schema stops matching", async () => { + const api = delivery(); + api.publish(SHA_1, "Published"); + const loader = remoteLoader(bundled(), { site: SITE }); + await loader.update?.(); + api.publish(SHA_2, "Next schema", OTHER_SCHEMA); + expect(await loader.update?.()).toEqual({ updated: false }); + expect((await loader.load()).revision).toBe(SHA_1); + }); + + it("bundled content without a schemaHash never swaps and never fetches", async () => { + const api = delivery(); + api.publish(SHA_1, "Published"); + const fallback = bundled(null); + const loader = remoteLoader(fallback, { site: SITE }); + expect(await loader.update?.()).toEqual({ updated: false }); + expect(api.fetch).not.toHaveBeenCalled(); + expect(await loader.load()).toBe(fallback); + }); + it("404 (nothing published), 500 and network errors keep memory; the next check retries", async () => { + const api = delivery(); + const fallback = bundled(); + const loader = remoteLoader(fallback, { site: SITE }); await expect(loader.update?.()).rejects.toThrow(/HTTP 404/); expect(await loader.load()).toBe(fallback); - api.asset(path, release); + + api.fetch.mockResolvedValueOnce(new Response("down", { status: 500 })); + await expect(loader.update?.()).rejects.toThrow(/HTTP 500/); + api.fetch.mockRejectedValueOnce(new TypeError("fetch failed")); + await expect(loader.update?.()).rejects.toThrow(/fetch failed/); + expect(await loader.load()).toBe(fallback); + + api.publish(SHA_1, "Published"); expect(await loader.update?.()).toEqual({ updated: true }); - expect(await loader.load()).toEqual(release); }); - it("a fallback that can't load (a KV key the deploy never wrote) is fixed by a release check", async () => { - const api = deliveryApi(); - const release = await hashed("Published"); - api.publish(1, release); - const failing: Loader = { load: () => Promise.reject(new Error("no key")) }; - const loader = remote(failing, { site: SITE, token: TOKEN }); - - await expect(loader.load()).rejects.toThrow("no key"); + it("a revision whose download failed is retried on the next check", async () => { + const api = delivery(); + api.publish(SHA_1, "Published"); + const key = `/sites/acme/revisions/${SHA_1}.json`; + const stored = { revision: SHA_1, schemaHash: SCHEMA, blocks: blocksAt("Published") }; + api.remove(key); + const fallback = bundled(); + const loader = remoteLoader(fallback, { site: SITE }); + await expect(loader.update?.()).rejects.toThrow(/HTTP 404/); + expect(await loader.load()).toBe(fallback); + api.put(key, stored); expect(await loader.update?.()).toEqual({ updated: true }); - expect(await loader.load()).toEqual(release); }); - it("refuses content that doesn't hash to its revision, keeping memory as it was", async () => { - const api = deliveryApi(); - const release = await hashed("Published"); - api.publish(1, release, { ...release, blocks: { Tampered: { __resolveType: "seo" } } }); - const fallback = docsSnapshot(); - const cms = createCMS({ blocks: docsBlocks(), content: fallback, site: SITE, token: TOKEN }); + it("refuses a malformed latest.json, downloading nothing", async () => { + const api = delivery(); + const loader = remoteLoader(bundled(), { site: SITE }); + for (const pointer of [ + null, + [], + { revision: SHA_1, schemaHash: SCHEMA }, // no publishedAt + { revision: "abc", schemaHash: SCHEMA, publishedAt: "x" }, // not a commit SHA + { revision: "../../other/revisions/x", schemaHash: SCHEMA, publishedAt: "x" }, + { revision: "A".repeat(40), schemaHash: SCHEMA, publishedAt: "x" }, // lowercase hex only + { revision: SHA_1, schemaHash: "short", publishedAt: "x" }, + { revision: SHA_1, schemaHash: SCHEMA, publishedAt: 1 }, + ]) { + api.point(pointer); + await expect(loader.update?.()).rejects.toThrow(/latest\.json: unexpected format/); + } + expect(api.urls().every((url) => url === LATEST_URL)).toBe(true); + }); - expect(await cms.update()).toEqual({ updated: false }); - expect(await cms.forRelease().revision()).toBe(fallback.revision); + it("refuses a revision body that isn't the one latest.json names, keeping memory", async () => { + const api = delivery(); + const fallback = bundled(); + const loader = remoteLoader(fallback, { site: SITE }); + api.point({ revision: SHA_1, schemaHash: SCHEMA, publishedAt: "x" }); + for (const body of [ + { revision: SHA_2, schemaHash: SCHEMA, blocks: {} }, + { revision: SHA_1, schemaHash: OTHER_SCHEMA, blocks: {} }, + { revision: SHA_1, schemaHash: SCHEMA, blocks: [] }, + { revision: SHA_1, schemaHash: SCHEMA }, + ]) { + api.put(`/sites/acme/revisions/${SHA_1}.json`, body); + await expect(loader.update?.()).rejects.toThrow(/unexpected format/); + expect(await loader.load()).toBe(fallback); + } }); - it("ignores a manifest with an unknown format or a snapshot path outside the site", async () => { - const api = deliveryApi(); - const release = await hashed("Published"); - api.asset(`/sites/other/revisions/${release.revision}.json`, release); - const loader = remote(docsSnapshot(), { site: SITE, token: TOKEN }); - const cms = createCMS({ blocks: docsBlocks(), content: loader }); + it("doesn't verify a content hash: the revision id is the commit SHA", async () => { + const api = delivery(); + api.publish(SHA_1, "Whatever the commit holds"); + const loader = remoteLoader(bundled(), { site: SITE }); + expect(await loader.update?.()).toEqual({ updated: true }); + }); - api.setManifest({ - format: 1, - generation: 1, - revision: release.revision, - snapshot: `/sites/other/revisions/${release.revision}.json`, - }); - expect(await cms.update()).toEqual({ updated: false }); - api.setManifest({ - format: 2, - generation: 1, - revision: release.revision, - snapshot: `/sites/acme/revisions/${release.revision}.json`, - }); - expect(await cms.update()).toEqual({ updated: false }); - expect(api.requests.every((r) => r.url === MANIFEST_URL)).toBe(true); + it("keeps local files in development: NODE_ENV=development never fetches", async () => { + vi.stubEnv("NODE_ENV", "development"); + const api = delivery(); + api.publish(SHA_1, "Published"); + const fallback = bundled(); + const loader = remoteLoader(fallback, { site: SITE }); + expect(await loader.update?.()).toEqual({ updated: false }); + expect(await loader.load()).toBe(fallback); + expect(api.fetch).not.toHaveBeenCalled(); + }); + + it("works over a fallback loader, reading its schemaHash from what it loads", async () => { + const api = delivery(); + api.publish(SHA_1, "Published"); + const fallback = bundled(); + const load = vi.fn(async () => fallback); + const loader = remoteLoader({ load }, { site: SITE }); + expect(await loader.load()).toBe(fallback); + expect(await loader.update?.()).toEqual({ updated: true }); + expect(load).toHaveBeenCalledTimes(1); }); - it("orders by generation: an older manifest is ignored, a newer one can roll back to an older revision", async () => { - const api = deliveryApi(); - const a = await hashed("A"); - const b = await hashed("B"); - const loader = remote(docsSnapshot(), { site: SITE, token: TOKEN }); + it("a fallback loader that can't load can't be compared: no swap", async () => { + const api = delivery(); + api.publish(SHA_1, "Published"); + const failing: Loader = { load: () => Promise.reject(new Error("no key")) }; + const loader = remoteLoader(failing, { site: SITE }); + await expect(loader.update?.()).rejects.toThrow("no key"); + expect(api.fetch).not.toHaveBeenCalled(); + }); +}); - api.publish(184, b); - await loader.update?.(); - expect((await loader.load()).revision).toBe(b.revision); +describe("remoteLoader: whoever is newer wins (publishedAt vs the bundle's committedAt)", () => { + const T = (hour: number) => `2026-10-07T${String(hour).padStart(2, "0")}:00:00.000Z`; - api.publish(183, a); // a delayed, older promotion + it("publish, then deploy: the bundle of the later commit wins; nothing is downloaded", async () => { + const api = delivery(); + api.publish(SHA_1, "Published", SCHEMA, T(10)); + const fallback = bundled(SCHEMA, T(11)); + const loader = remoteLoader(fallback, { site: SITE }); expect(await loader.update?.()).toEqual({ updated: false }); - expect((await loader.load()).revision).toBe(b.revision); + expect(api.urls()).toEqual([LATEST_URL]); + expect(await loader.load()).toBe(fallback); + }); - api.publish(185, a); // rollback: newer generation, older revision + it("deploy, then publish: the release published later wins", async () => { + const api = delivery(); + api.publish(SHA_1, "Published", SCHEMA, T(12)); + const loader = remoteLoader(bundled(SCHEMA, T(11)), { site: SITE }); expect(await loader.update?.()).toEqual({ updated: true }); - expect((await loader.load()).revision).toBe(a.revision); + expect((await loader.load()).blocks).toEqual(blocksAt("Published")); }); - it("rolling back to the fallback's revision serves the fallback again", async () => { - const api = deliveryApi(); - const fallback = await hashed("Bundled"); - const loader = remote(fallback, { site: SITE, token: TOKEN }); - api.publish(1, await hashed("Published")); - await loader.update?.(); - api.publish(2, fallback); + it("a publish at the bundle's exact committedAt keeps the bundle", async () => { + const api = delivery(); + api.publish(SHA_1, "Published", SCHEMA, T(11)); + const loader = remoteLoader(bundled(SCHEMA, T(11)), { site: SITE }); + expect(await loader.update?.()).toEqual({ updated: false }); + }); + + it("rollback after a deploy: Make current writes publishedAt = now, so the CDN wins", async () => { + const api = delivery(); + api.publish(SHA_1, "One", SCHEMA, T(9)); + const loader = remoteLoader(bundled(SCHEMA, T(11)), { site: SITE }); + expect(await loader.update?.()).toEqual({ updated: false }); + api.point({ revision: SHA_1, schemaHash: SCHEMA, publishedAt: T(12) }); expect(await loader.update?.()).toEqual({ updated: true }); - expect(await loader.load()).toBe(fallback); + expect((await loader.load()).revision).toBe(SHA_1); }); - it("keeps local files in development: no release checks, the fallback is served", async () => { - vi.stubEnv("NODE_ENV", "development"); - const api = deliveryApi(); - api.publish(1, await hashed("Published")); - const fallback = docsSnapshot(); - const loader = remote(fallback, { site: SITE, token: TOKEN }); + it("deploy after a rollback: the bundle of the later commit wins", async () => { + const api = delivery(); + api.publish(SHA_1, "One", SCHEMA, T(9)); + api.point({ revision: SHA_1, schemaHash: SCHEMA, publishedAt: T(12) }); // the rollback + const fallback = bundled(SCHEMA, T(13)); + const loader = remoteLoader(fallback, { site: SITE }); expect(await loader.update?.()).toEqual({ updated: false }); expect(await loader.load()).toBe(fallback); - expect(api.fetch).not.toHaveBeenCalled(); }); - it("works over a fallback loader, reading it only to serve or compare it", async () => { - deliveryApi(); - const fallback = await hashed("Bundled"); - const load = vi.fn(async () => fallback); - const loader = remote({ load }, { site: SITE, token: TOKEN }); + it("a build of an older commit that finishes after a publish loses to the CDN", async () => { + const api = delivery(); + // Commit at 10:00, publish at 11:00, the slow build of that commit finishes at 12:00. + api.publish(SHA_1, "Published", SCHEMA, T(11)); + const loader = remoteLoader(bundled(SCHEMA, T(10)), { site: SITE }); + expect(await loader.update?.()).toEqual({ updated: true }); + expect((await loader.load()).blocks).toEqual(blocksAt("Published")); + }); + + it("a newer release with another schema keeps the bundle", async () => { + const api = delivery(); + api.publish(SHA_1, "Published", OTHER_SCHEMA, T(12)); + const fallback = bundled(SCHEMA, T(11)); + const loader = remoteLoader(fallback, { site: SITE }); + expect(await loader.update?.()).toEqual({ updated: false }); expect(await loader.load()).toBe(fallback); - expect(load).toHaveBeenCalledTimes(1); + }); + + it("a bundle without committedAt is the oldest: a release with its schema wins", async () => { + const api = delivery(); + api.publish(SHA_1, "Published", SCHEMA, "1970-01-01T00:00:00.000Z"); + const loader = remoteLoader(bundled(SCHEMA), { site: SITE }); + expect(await loader.update?.()).toEqual({ updated: true }); + expect((await loader.load()).revision).toBe(SHA_1); + }); + + it("a fallback loader's committedAt is read from what it loads", async () => { + const api = delivery(); + api.publish(SHA_1, "Published", SCHEMA, T(10)); + const custom: Loader = { load: async () => bundled(SCHEMA, T(11)) }; + const loader = remoteLoader(custom, { site: SITE }); + expect(await loader.update?.()).toEqual({ updated: false }); + }); + + it("refuses a latest.json whose publishedAt isn't a date", async () => { + const api = delivery(); + api.publish(SHA_1, "Published"); + api.point({ revision: SHA_1, schemaHash: SCHEMA, publishedAt: "yesterday-ish" }); + const loader = remoteLoader(bundled(), { site: SITE }); + await expect(loader.update?.()).rejects.toThrow(/latest\.json: unexpected format/); }); }); describe("remoteLoader with createCMS", () => { - it("site and token wrap the content: a check swaps the release in for the next client", async () => { - const api = deliveryApi(); - const release = await hashed("Published"); - api.publish(1, release); - const cms = createCMS({ - blocks: docsBlocks(), - content: docsSnapshot(), - site: SITE, - token: TOKEN, - }); + it("site alone wraps the content: a check swaps the release in for the next client", async () => { + const api = delivery(); + api.publish(SHA_1, "Published"); + const cms = createCMS({ blocks: docsBlocks(), content: bundled(), site: SITE }); const before = cms.forRelease(); expect(await before.revision()).toBe("rev-1"); expect(await cms.update()).toEqual({ updated: true }); expect(await before.revision()).toBe("rev-1"); // a client keeps its revision - const [seo] = await cms.forRelease().resolve<{ title: string }>("SummerSEO"); + const client = cms.forRelease(); + expect(await client.revision()).toBe(SHA_1); + const [seo] = await client.resolve<{ title: string }>("SummerSEO"); expect(seo?.title).toBe("Published"); }); - it("without both site and token, the content is read as is", async () => { - const api = deliveryApi(); - const cms = createCMS({ blocks: docsBlocks(), content: docsSnapshot(), site: SITE }); - expect(await cms.update()).toEqual({ updated: false }); + it("a token without site is a configuration error, before anything is fetched", async () => { + const api = delivery(); + expect(() => + createCMS({ + blocks: docsBlocks(), + content: bundled(), + token: "t", + telemetry: false, + }), + ).toThrow("token needs site: pass both, or site alone"); expect(api.fetch).not.toHaveBeenCalled(); }); - it("is one instance per process: the same site, token and fallback share it; another interval warns", () => { + it("is one instance per process: the same site and fallback share it; another interval warns", () => { const warn = vi.spyOn(console, "warn").mockImplementation(() => {}); - const fallback = docsSnapshot(); - const first = remote(fallback, { site: SITE, token: TOKEN }); - expect(remote(fallback, { site: SITE, token: TOKEN })).toBe(first); - expect(remote(fallback, { site: SITE, token: TOKEN, interval: 120_000 })).toBe(first); + const fallback = bundled(); + const first = remoteLoader(fallback, { site: SITE }); + expect(remoteLoader(fallback, { site: SITE })).toBe(first); + expect(remoteLoader(fallback, { site: SITE, interval: 120_000 })).toBe(first); expect(warn).toHaveBeenCalledWith(expect.stringContaining("interval")); - expect(remote(fallback, { site: "other", token: TOKEN })).not.toBe(first); + expect(remoteLoader(fallback, { site: "other" })).not.toBe(first); }); it("createCMS paces a remoteLoader by its own interval when it has none", async () => { vi.useFakeTimers({ toFake: ["Date"] }); vi.spyOn(Math, "random").mockReturnValue(0.5); vi.setSystemTime(new Date("2026-10-03T00:00:00Z")); - deliveryApi(); - const loader = remote(docsSnapshot(), { site: SITE, token: TOKEN, interval: 300_000 }); + delivery(); + const loader = remoteLoader(bundled(), { site: SITE, interval: 300_000 }); const update = vi.spyOn(loader, "update" as never); const cms = createCMS({ blocks: docsBlocks(), content: loader }); cms.forRelease(); @@ -278,12 +451,9 @@ describe("remoteLoader with createCMS", () => { vi.useRealTimers(); }); - it("is a loader over the fallback alone when site or token is unset", async () => { - const fallback = docsSnapshot(); - for (const loader of [ - remoteLoader(fallback, { site: "", token: TOKEN }), - remoteLoader(fallback, { site: SITE }), - ]) { + it("is a loader over the fallback alone when site is unset or empty", async () => { + const fallback = bundled(); + for (const loader of [remoteLoader(fallback, { site: "" }), remoteLoader(fallback, {})]) { expect(await loader.load()).toBe(fallback); expect(loader.update).toBeUndefined(); } diff --git a/packages/blocks/src/v8/remoteLoader.ts b/packages/blocks/src/v8/remoteLoader.ts index 0ee18ebf..4736f0d2 100644 --- a/packages/blocks/src/v8/remoteLoader.ts +++ b/packages/blocks/src/v8/remoteLoader.ts @@ -1,73 +1,82 @@ /** - * `remoteLoader`: hosted releases from the Deco API, over a fallback (the - * content module, or any loader). See /next/hosted-releases-internals and - * /next/hosted-publishing. Drafts aren't a loader's job: the CMS layers a - * draft's changes over what `load()` returns (see ./draftChanges.ts). + * `remoteLoader`: hosted releases from `delivery.decocms.com`, over a fallback + * (the content module, or any loader). See /next/hosted-releases-internals. + * Drafts aren't a loader's job: the CMS layers a draft's changes over what + * `load()` returns (see ./draftChanges.ts). * - * - `load()` never touches the network: it returns the newest release this - * process fetched, or the fallback's content until one is fetched. - * - `update()` asks the delivery API for the small production channel - * manifest; the CMS calls it on first use and then every interval, at an - * idle moment. A manifest older than the newest observed generation is - * ignored; a revision equal to the fallback's is served from the fallback - * without a download; anything else is fetched, verified against its - * content hash and swapped in whole. Any error keeps memory as it was. - * - In development (`NODE_ENV=development`), releases stay on the fallback so - * local files win. + * - `load()` never touches the network: it returns the release this process + * last swapped in, or the fallback's content until then. Nothing persists + * across restarts: every boot starts from the fallback. + * - `update()` (the CMS calls it in the background, on first use and then + * every interval) reads `sites//latest.json`, + * `{ revision, schemaHash, publishedAt }`. Whoever is newer wins: when it + * names a revision other than the one this process last swapped in, its + * `schemaHash` equals the fallback's, and its `publishedAt` is later than + * the fallback's `committedAt` (the commit time of the git HEAD + * `deco content` built it from), it downloads + * `sites//revisions/.json`, `{ revision, schemaHash, blocks }`, + * and swaps it in whole. Otherwise, and on any error, memory stays as it is. + * A fallback without a `schemaHash` never swaps; one without a `committedAt` + * (a custom loader, an older content module, a build outside git) counts as + * the oldest. Studio writes `publishedAt` on Publish, on "Make current" (a + * rollback) and on Resync, so a rollback wins over the bundles of earlier + * commits and a deploy of a later commit wins over the rollback. Commit + * time, not build time: a slow build of an older commit that finishes after + * a publish still loses to it. + * Pointers aren't ordered among themselves, and the fallback's content is + * never compared: only the two timestamps are. + * - In development (`NODE_ENV=development`), it never swaps, so local files win. * - A release larger than `MAX_SNAPSHOT_BYTES` is refused while it downloads, * before it's buffered whole (a Worker isolate has 128 MB). - * - Without `site` or `token` it's a plain loader over the fallback. + * - Without `site` it's a plain loader over the fallback. * * Instances are process-wide singletons, like `createCMS`'s. */ import { readBoundedJson, timeoutSignal } from "./boundedJson.ts"; -import { computeContentRevision } from "./canonical.ts"; import { isSnapshot, PEEK_RELEASE, peekRelease } from "./content.ts"; -import { clearGlobals, contentIdentity, fnv1a } from "./identity.ts"; +import { clearGlobals, contentIdentity } from "./identity.ts"; import { isPlainObject } from "./json.ts"; import type { Loader, Snapshot } from "./types.ts"; -/** The hosted delivery origin: channel manifests and release assets. */ +/** The hosted delivery origin: release pointers and revisions. */ const HOSTED_DELIVERY_ORIGIN = "https://delivery.decocms.com"; +/** test-only: replaces the delivery origin in local end-to-end runs. Not documented, not exported. */ +const TEST_DELIVERY_ORIGIN = Symbol.for("decocms.blocks.test.deliveryOrigin"); const INSTANCE_PREFIX = "decocms.blocks.remote:"; -const MANIFEST_FORMAT = 1; const FETCH_TIMEOUT_MS = 10_000; /** The largest release accepted, in bytes of JSON. */ const MAX_SNAPSHOT_BYTES = 64 * 1024 * 1024; +// OPEN: a revision is the git commit SHA; only SHA-1 (40 hex) object names are accepted. +const REVISION = /^[0-9a-f]{40}$/; +const SCHEMA_HASH = /^[0-9a-f]{64}$/; interface RemoteLoaderOptions { site?: string; - token?: string; /** ms between release checks, used when `createCMS` has no `interval` of its own. */ interval?: number; } -interface Manifest { - generation: number; +/** `sites//latest.json`: the release a site serves. */ +interface Latest { revision: string; - snapshot: string; + schemaHash: string; + publishedAt: string; } class RemoteLoader implements Loader { readonly #site: string; - readonly #token: string; /** The `interval` this loader was created with; `createCMS` reads it. */ readonly interval: number | undefined; #fallback: Snapshot | Loader; - #fallbackRevision: string | undefined; /** The fallback's content as last loaded, for a fallback that is itself a loader. */ #fallbackSnapshot: Snapshot | undefined; + /** The release this process last swapped in; `undefined` serves the fallback. */ #current: Snapshot | undefined; - #generation = -1; - constructor( - fallback: Snapshot | Loader, - options: { site: string; token: string; interval?: number }, - ) { + constructor(fallback: Snapshot | Loader, options: { site: string; interval?: number }) { this.#fallback = fallback; this.#site = options.site; - this.#token = options.token; this.interval = options.interval; } @@ -75,7 +84,6 @@ class RemoteLoader implements Loader { adopt(fallback: Snapshot | Loader): void { if (fallback === this.#fallback) return; this.#fallback = fallback; - this.#fallbackRevision = undefined; this.#fallbackSnapshot = undefined; } @@ -91,27 +99,19 @@ class RemoteLoader implements Loader { async update(): Promise<{ updated: boolean }> { if (isDevelopment()) return { updated: false }; - const manifest = await this.#manifest(); - if (manifest.generation < this.#generation) return { updated: false }; - - // Best effort: a fallback that can't load (a KV key the deploy never wrote) is - // fixed by downloading the release, not by failing the check. - const fallbackRevision = await this.#fallbackRevisionNow().catch(() => undefined); - const served = this.#current !== undefined ? this.#current.revision : fallbackRevision; - if (served !== undefined && manifest.revision === served) { - // A new generation of the same content (a re-promotion, a rollback to - // it) still signals, so caches keyed on the release are invalidated. - const changed = this.#generation !== -1 && manifest.generation > this.#generation; - this.#generation = manifest.generation; - return { updated: changed }; - } - let next: Snapshot | undefined; - if (fallbackRevision === undefined || manifest.revision !== fallbackRevision) { - next = await this.#release(manifest); - } - // A slower, earlier check must not undo a newer publish or rollback. - if (manifest.generation < this.#generation) return { updated: false }; - this.#generation = manifest.generation; + const fallback = this.#fallbackSnapshot ?? (await this.#loadFallback()); + if (fallback.schemaHash === undefined) return { updated: false }; + const site = encodeURIComponent(this.#site); + const latest = await this.#latest(site); + // Another schema: keep what this process serves (the fallback, or the last swap). + if (latest.schemaHash !== fallback.schemaHash) return { updated: false }; + // The bundle is newer: keep what this process serves. + // OPEN: a process that already swapped a release in keeps it (memory stays as it is). + if (!isNewer(latest.publishedAt, fallback.committedAt)) return { updated: false }; + if (latest.revision === this.#current?.revision) return { updated: false }; + const blocks = await this.#revision(site, latest); + const next: Snapshot = { revision: latest.revision, blocks, schemaHash: latest.schemaHash }; + if (fallback.aliases !== undefined) next.aliases = fallback.aliases; this.#current = next; return { updated: true }; } @@ -119,68 +119,54 @@ class RemoteLoader implements Loader { async #loadFallback(): Promise { const fallback = this.#fallback; const snapshot = isSnapshot(fallback) ? fallback : await (fallback as Loader).load(); - if (fallback === this.#fallback) { - this.#fallbackRevision = snapshot.revision; - this.#fallbackSnapshot = snapshot; - } + if (fallback === this.#fallback) this.#fallbackSnapshot = snapshot; return snapshot; } - async #fallbackRevisionNow(): Promise { - return this.#fallbackRevision ?? (await this.#loadFallback()).revision; - } - - async #manifest(): Promise { - const site = encodeURIComponent(this.#site); - const response = await fetchWithTimeout( - `${HOSTED_DELIVERY_ORIGIN}/sites/${site}/channels/production.json`, - this.#auth(), - ); - if (!response.ok) throw new Error(`channel manifest: HTTP ${response.status}`); + async #latest(site: string): Promise { + const response = await fetchWithTimeout(`${deliveryOrigin()}/sites/${site}/latest.json`); + if (!response.ok) { + await response.body?.cancel(); + throw new Error(`latest.json: HTTP ${response.status}`); + } const body: unknown = await response.json(); - const prefix = `/sites/${site}/revisions/`; if ( !isPlainObject(body) || - body.format !== MANIFEST_FORMAT || - !Number.isSafeInteger(body.generation) || typeof body.revision !== "string" || - typeof body.snapshot !== "string" || - !body.snapshot.startsWith(prefix) || - !/^[\w.-]+\.json$/.test(body.snapshot.slice(prefix.length)) || - body.snapshot.includes("..") + !REVISION.test(body.revision) || + typeof body.schemaHash !== "string" || + !SCHEMA_HASH.test(body.schemaHash) || + typeof body.publishedAt !== "string" || + Number.isNaN(Date.parse(body.publishedAt)) ) { - throw new Error("channel manifest: unexpected format or snapshot path"); + throw new Error("latest.json: unexpected format"); } - return body as unknown as Manifest; + return body as unknown as Latest; } - async #release(manifest: Manifest): Promise { + async #revision(site: string, latest: Latest): Promise> { + const label = `revision ${latest.revision}`; const response = await fetchWithTimeout( - `${HOSTED_DELIVERY_ORIGIN}${manifest.snapshot}`, - this.#auth(), - ); - if (!response.ok) throw new Error(`release ${manifest.revision}: HTTP ${response.status}`); - const snapshot = await readBoundedJson( - response, - `release ${manifest.revision}`, - MAX_SNAPSHOT_BYTES, + `${deliveryOrigin()}/sites/${site}/revisions/${latest.revision}.json`, ); + if (!response.ok) { + await response.body?.cancel(); + throw new Error(`${label}: HTTP ${response.status}`); + } + const body = await readBoundedJson(response, label, MAX_SNAPSHOT_BYTES); if ( - !isSnapshot(snapshot) || - snapshot.revision !== manifest.revision || - (await computeContentRevision(snapshot.blocks)) !== manifest.revision + !isPlainObject(body) || + body.revision !== latest.revision || + body.schemaHash !== latest.schemaHash || + !isPlainObject(body.blocks) ) { - throw new Error(`release ${manifest.revision}: content doesn't match its revision`); + throw new Error(`${label}: unexpected format, or not the revision latest.json names`); } - return snapshot; - } - - #auth(): Record { - return { authorization: `Bearer ${this.#token}` }; + return body.blocks; } } -/** A loader over the fallback alone: `remoteLoader` without `site` or `token`. */ +/** A loader over the fallback alone: `remoteLoader` without `site`. */ class LocalLoader implements Loader { #fallback: Snapshot | Loader; @@ -203,16 +189,14 @@ class LocalLoader implements Loader { } /** - * Hosted releases over a fallback. `createCMS` builds it for you - * when `site` and `token` are set. With either unset (a dev or test - * environment without the variables), it's a loader over the fallback alone. + * Hosted releases over a fallback. `createCMS` builds it for you when `site` + * is set. Without `site`, it's a loader over the fallback alone. */ export function remoteLoader(fallback: Snapshot | Loader, options: RemoteLoaderOptions): Loader { - const { site, token } = options ?? {}; - const hosted = Boolean(site && token); + const { site } = options ?? {}; + const hosted = Boolean(site); const key = Symbol.for( - `${INSTANCE_PREFIX}${contentIdentity(fallback)}` + - (hosted ? `|site:${site}|token:${fnv1a(token!)}` : "|local"), + `${INSTANCE_PREFIX}${contentIdentity(fallback)}${hosted ? `|site:${site}` : "|local"}`, ); const store = globalThis as unknown as Record; const existing = store[key]; @@ -227,7 +211,7 @@ export function remoteLoader(fallback: Snapshot | Loader, options: RemoteLoaderO return existing; } const instance = hosted - ? new RemoteLoader(fallback, { site: site!, token: token!, interval: options.interval }) + ? new RemoteLoader(fallback, { site: site!, interval: options.interval }) : new LocalLoader(fallback); store[key] = instance; return instance; @@ -237,10 +221,31 @@ export function resetRemoteLoaders(): void { clearGlobals(INSTANCE_PREFIX); } -function fetchWithTimeout(url: string, headers: Record): Promise { - return fetch(url, { headers, signal: timeoutSignal(FETCH_TIMEOUT_MS) }); +/** + * Whether a release published at `publishedAt` is newer than content built + * from a commit made at `committedAt`. Content without a `committedAt` is the + * oldest. + * OPEN: a `committedAt` that doesn't parse as a date counts as missing. + */ +function isNewer(publishedAt: string, committedAt: string | undefined): boolean { + const committed = committedAt === undefined ? Number.NaN : Date.parse(committedAt); + return Number.isNaN(committed) || Date.parse(publishedAt) > committed; +} + +/** No credentials: delivery is public. */ +function fetchWithTimeout(url: string): Promise { + return fetch(url, { signal: timeoutSignal(FETCH_TIMEOUT_MS) }); +} + +function deliveryOrigin(): string { + const override = (globalThis as Record)[TEST_DELIVERY_ORIGIN]; + return typeof override === "string" ? override : HOSTED_DELIVERY_ORIGIN; } +/** + * The SDK's only environment read: local development never swaps in hosted + * releases, so local files win. + */ function isDevelopment(): boolean { try { // Written out so bundlers that define process.env.NODE_ENV replace it. diff --git a/packages/blocks/src/v8/settings.test.ts b/packages/blocks/src/v8/settings.test.ts index 1c09aea8..0934b5b7 100644 --- a/packages/blocks/src/v8/settings.test.ts +++ b/packages/blocks/src/v8/settings.test.ts @@ -366,7 +366,7 @@ describe("cms.settings(): always the release in memory", () => { vi.stubGlobal( "fetch", vi.fn(async (input: string | URL | Request, init?: RequestInit) => - String(input).startsWith("https://studio.decocms.com/") + String(input).startsWith("https://delivery.decocms.com/") ? studio.fetch(input, init) : new Response(null), ), diff --git a/packages/blocks/src/v8/telemetry.test.ts b/packages/blocks/src/v8/telemetry.test.ts index 19d750ae..f1e8dbd9 100644 --- a/packages/blocks/src/v8/telemetry.test.ts +++ b/packages/blocks/src/v8/telemetry.test.ts @@ -85,75 +85,71 @@ const upstream = createInstrumentedFetch({ }); describe("where telemetry goes", () => { - it("false sends nothing; an endpoint sends there; site and token go to the hosted collector", () => { - expect(resolveDestination(false)).toBeNull(); + it("false sends nothing; an endpoint sends there; otherwise a token goes to the hosted collector", () => { + expect(resolveDestination(false, "acme", "tok")).toBeNull(); expect( resolveDestination({ endpoint: ENDPOINT, headers: { authorization: "Bearer t" } }), ).toMatchObject({ endpoint: ENDPOINT, headers: { authorization: "Bearer t" } }); - expect(resolveDestination({ site: "acme", token: "tok" })).toMatchObject({ + expect(resolveDestination(undefined, "acme", "tok")).toMatchObject({ endpoint: HOSTED_TELEMETRY_ENDPOINT, headers: { authorization: "Bearer tok" }, site: "acme", }); + expect(resolveDestination({ limits: { traceSampleRate: 1 } }, "acme", "tok")).toMatchObject({ + endpoint: HOSTED_TELEMETRY_ENDPOINT, + limits: { traceSampleRate: 1 }, + }); }); - it("unset values (an environment variable that isn't set) turn it off instead of throwing", () => { - expect( - resolveDestination({ site: undefined, token: undefined } as unknown as TelemetryConfig), - ).toBeNull(); - expect(resolveDestination({ endpoint: "" })).toBeNull(); + it("an explicit endpoint wins over the token: the token isn't sent there", () => { + const destination = resolveDestination({ endpoint: ENDPOINT }, "acme", "tok"); + expect(destination).toMatchObject({ endpoint: ENDPOINT, headers: {} }); }); - it("left out, it reads OTEL_EXPORTER_OTLP_ENDPOINT and OTEL_EXPORTER_OTLP_HEADERS, or sends nothing", () => { + it("nothing set is off: no endpoint and no token, or empty values", () => { expect(resolveDestination(undefined)).toBeNull(); - vi.stubEnv("OTEL_EXPORTER_OTLP_ENDPOINT", ENDPOINT); - vi.stubEnv("OTEL_EXPORTER_OTLP_HEADERS", "x-team=store,authorization=Bearer%20abc"); - expect(resolveDestination(undefined)).toMatchObject({ - endpoint: ENDPOINT, - headers: { "x-team": "store", authorization: "Bearer abc" }, - }); + expect(resolveDestination({})).toBeNull(); + expect(resolveDestination({ endpoint: "" })).toBeNull(); + expect(resolveDestination(undefined, "acme", "")).toBeNull(); + expect(resolveDestination({ endpoint: "" } as TelemetryConfig, "acme")).toBeNull(); }); - it("reads v7's DECO_OTEL_* names as aliases; the standard name wins when both are set", () => { + it("reads no environment variable: OTEL_* and DECO_OTEL_* change nothing", async () => { + vi.stubEnv("OTEL_EXPORTER_OTLP_ENDPOINT", ENDPOINT); + vi.stubEnv("OTEL_EXPORTER_OTLP_HEADERS", "authorization=Bearer%20abc"); vi.stubEnv("DECO_OTEL_METRICS_ENDPOINT", `${ENDPOINT}/v1/metrics`); - vi.stubEnv("DECO_OTEL_LOGS_ENDPOINT", `${ENDPOINT}/v1/logs`); - vi.stubEnv("DECO_OTEL_TRACES_ENDPOINT", `${ENDPOINT}/v1/traces`); - vi.stubEnv("DECO_OTEL_HEADERS", "x-team=v7"); vi.stubEnv("DECO_OTEL_AUTH_TOKEN", "Bearer v7"); - expect(resolveDestination(undefined)).toMatchObject({ - endpoint: "", - signals: { - metrics: `${ENDPOINT}/v1/metrics`, - logs: `${ENDPOINT}/v1/logs`, - traces: `${ENDPOINT}/v1/traces`, - }, - headers: { "x-team": "v7", authorization: "Bearer v7" }, - }); - vi.stubEnv("OTEL_EXPORTER_OTLP_METRICS_ENDPOINT", "https://std.example/m"); - vi.stubEnv("OTEL_EXPORTER_OTLP_HEADERS", "x-team=std,authorization=Bearer%20std"); - expect(resolveDestination(undefined)).toMatchObject({ - signals: { metrics: "https://std.example/m", logs: `${ENDPOINT}/v1/logs` }, - headers: { "x-team": "std", authorization: "Bearer std" }, - }); - }); - - it("sends a signal to its own URL, and drops a signal with no destination", async () => { - const { sent } = collector(); - vi.stubEnv("DECO_OTEL_METRICS_ENDPOINT", "https://ingest.example/v1/metrics"); + expect(resolveDestination(undefined)).toBeNull(); + const { fetch } = collector(); createCMS({ blocks: docsBlocks(), content: docsSnapshot() }); + expect(currentTelemetry()).toBeUndefined(); await upstream("https://search.example/q"); await runBackground(); - expect(sent.map((s) => s.url)).toEqual(["https://ingest.example/v1/metrics"]); + expect(fetch).not.toHaveBeenCalled(); }); - it("top-level site and token never turn telemetry on", async () => { + it("top-level site alone never turns telemetry on", async () => { const { fetch } = collector(); - createCMS({ blocks: docsBlocks(), content: docsSnapshot(), site: "acme", token: "tok" }); + createCMS({ blocks: docsBlocks(), content: docsSnapshot(), site: "acme" }); expect(currentTelemetry()).toBeUndefined(); await upstream("https://search.example/q"); await runBackground(); expect(fetch).not.toHaveBeenCalled(); }); + + it("a top-level token turns it on to the hosted collector; telemetry: false opts out", async () => { + createCMS({ blocks: docsBlocks(), content: docsSnapshot(), site: "acme", token: "tok" }); + expect(currentTelemetry()).toBeDefined(); + resetForTests(); + createCMS({ + blocks: docsBlocks(), + content: docsSnapshot(), + site: "acme", + token: "tok", + telemetry: false, + }); + expect(currentTelemetry()).toBeUndefined(); + }); }); describe("metrics", () => { @@ -199,11 +195,7 @@ describe("metrics", () => { it("labels batches with the site and service name for the hosted collector", async () => { const { sent } = collector(); - createCMS({ - blocks: docsBlocks(), - content: docsSnapshot(), - telemetry: { site: "acme", token: "tok" }, - }); + createCMS({ blocks: docsBlocks(), content: docsSnapshot(), site: "acme", token: "tok" }); await upstream("https://search.example/q"); await runBackground(); expect(sent[0]?.url).toBe(`${HOSTED_TELEMETRY_ENDPOINT}/v1/metrics`); @@ -447,19 +439,44 @@ describe("sending", () => { expect(point.count).toBe("3"); }); - it("OTEL_RESOURCE_ATTRIBUTES sets service.version and the environment", async () => { - vi.stubEnv( - "OTEL_RESOURCE_ATTRIBUTES", - "service.version=abc123,deployment.environment.name=preview", - ); + it("defaults the resource to service.version unknown and the production environment", async () => { + vi.stubEnv("NODE_ENV", "development"); + vi.stubEnv("OTEL_RESOURCE_ATTRIBUTES", "service.version=abc123"); + vi.stubEnv("DECO_COMMIT_SHA", "abc123"); const { sent } = collector(); createCMS({ blocks: docsBlocks(), content: docsSnapshot(), telemetry: { endpoint: ENDPOINT } }); await upstream("https://search.example/q"); await runBackground(); expect(attrs(sent[0]?.body.resourceMetrics[0].resource.attributes)).toMatchObject({ "service.name": "decocms-site", + "service.version": "unknown", + "deployment.environment.name": "production", + }); + }); + + it("telemetry.resource sets service.version, the environment and the service name", async () => { + const { sent } = collector(); + createCMS({ + blocks: docsBlocks(), + content: docsSnapshot(), + site: "acme", + token: "tok", + telemetry: { + resource: { + "service.version": "abc123", + "deployment.environment.name": "preview", + "service.name": "acme-web", + }, + }, + }); + await upstream("https://search.example/q"); + await runBackground(); + expect(sent[0]?.url).toBe(`${HOSTED_TELEMETRY_ENDPOINT}/v1/metrics`); + expect(attrs(sent[0]?.body.resourceMetrics[0].resource.attributes)).toMatchObject({ + "service.name": "acme-web", "service.version": "abc123", "deployment.environment.name": "preview", + "deco.site": "acme", }); }); diff --git a/packages/blocks/src/v8/telemetry.ts b/packages/blocks/src/v8/telemetry.ts index 84c1057c..320af1c5 100644 --- a/packages/blocks/src/v8/telemetry.ts +++ b/packages/blocks/src/v8/telemetry.ts @@ -2,11 +2,10 @@ * Telemetry: OpenTelemetry over OTLP/HTTP (JSON, gzipped), with no SDK. See * /next/telemetry and /next/telemetry-internals. * - * - **Where** comes from code: `createCMS({ telemetry })`, or the standard - * `OTEL_EXPORTER_OTLP_ENDPOINT`/`OTEL_EXPORTER_OTLP_HEADERS` (and the - * per-signal `OTEL_EXPORTER_OTLP_{METRICS,LOGS,TRACES}_ENDPOINT`) when it's - * left out. v7's `DECO_OTEL_*` names are read as aliases; the standard name - * wins when both are set. `false` sends nothing. + * - **Where** comes from code only: `createCMS({ telemetry: { endpoint, + * headers } })`, else the hosted Deco CMS collector when `createCMS` has a + * `token` (sent as a Bearer token), else nowhere. `false` sends nothing. + * Nothing is read from the environment. * - **How much** comes from content: the `telemetry` section of the CMS * settings (`cms.settings()`, the release's `CMS` block), capped by * `telemetry.limits`. The CMS reads it outside any request, when the @@ -21,10 +20,9 @@ import { hasBackgroundHook, later, runInBackground } from "./background.ts"; import { rate, TELEMETRY_DEFAULTS } from "./builtins/data.ts"; import { isResolutionError } from "./errors.ts"; -import { readEnv } from "./identity.ts"; import type { CMSError, Snapshot, Telemetry, TelemetryConfig } from "./types.ts"; -/** The hosted Deco CMS collector, for `telemetry: { site, token }`. */ +/** The hosted Deco CMS collector, for `createCMS({ token })`. */ const HOSTED_TELEMETRY_ENDPOINT = "https://otel.decocms.com"; const SINK = Symbol.for("decocms.blocks.telemetry"); @@ -54,12 +52,12 @@ type Attributes = Record; type Signal = "metrics" | "logs" | "traces"; interface Destination { - /** The base URL each signal's `/v1/` is appended to; `""` with per-signal URLs only. */ + /** The base URL each signal's `/v1/` is appended to. */ endpoint: string; - /** Full per-signal URLs, used as they are (the `OTEL_EXPORTER_OTLP__ENDPOINT` form). */ - signals?: Partial>; headers: Record; site?: string; + /** `telemetry.resource`: merged over the default resource attributes. */ + resource: Record; limits: { errorSampleRate: number; traceSampleRate: number }; } @@ -98,69 +96,32 @@ interface LogRecord { } /** - * Where telemetry goes, or `null` for nowhere. `{ site, token }` or - * `{ endpoint }` with an empty value (an unset environment variable) is off. + * Where telemetry goes, or `null` for nowhere: `false` is off; a non-empty + * `endpoint` is that collector, with `headers`; otherwise a `token` is the + * hosted Deco CMS collector, with the token as a Bearer credential. */ export function resolveDestination( config: false | TelemetryConfig | undefined, site?: string, + token?: string, ): Destination | null { if (config === false) return null; - if (config === undefined || config === null) return destinationFromEnv(site); + // OPEN: no per-signal URLs (v7's OTEL_EXPORTER_OTLP__ENDPOINT): every signal goes to `/v1/`. const limits = telemetryLimits(config); - if ("endpoint" in config) { - if (typeof config.endpoint !== "string" || !config.endpoint) return null; - return { endpoint: config.endpoint, headers: { ...config.headers }, site, limits }; + const resource = { ...config?.resource }; + if (typeof config?.endpoint === "string" && config.endpoint) { + return { endpoint: config.endpoint, headers: { ...config.headers }, site, resource, limits }; } - if (!config.site || !config.token) return null; + if (!token) return null; return { endpoint: HOSTED_TELEMETRY_ENDPOINT, - headers: { authorization: `Bearer ${config.token}` }, - site: config.site, + headers: { authorization: `Bearer ${token}` }, + site, + resource, limits, }; } -/** v7's names for the standard OTLP variables, read when the standard one is unset. */ -const V7_ALIASES = { - OTEL_EXPORTER_OTLP_HEADERS: "DECO_OTEL_HEADERS", - OTEL_EXPORTER_OTLP_METRICS_ENDPOINT: "DECO_OTEL_METRICS_ENDPOINT", - OTEL_EXPORTER_OTLP_LOGS_ENDPOINT: "DECO_OTEL_LOGS_ENDPOINT", - OTEL_EXPORTER_OTLP_TRACES_ENDPOINT: "DECO_OTEL_TRACES_ENDPOINT", -} as const; - -/** The standard variable, else its v7 alias. */ -function readOtlpEnv(name: keyof typeof V7_ALIASES | "OTEL_EXPORTER_OTLP_ENDPOINT") { - return ( - readEnv(name) || - (name in V7_ALIASES ? readEnv(V7_ALIASES[name as keyof typeof V7_ALIASES]) : undefined) - ); -} - -/** - * The destination the environment names: the standard OTLP variables, with - * v7's `DECO_OTEL_*` names as aliases. `DECO_OTEL_AUTH_TOKEN` (v7's secret - * for the collector) becomes the `authorization` header unless the headers - * set one. `null` when no endpoint is set. - */ -function destinationFromEnv(site: string | undefined): Destination | null { - const endpoint = readOtlpEnv("OTEL_EXPORTER_OTLP_ENDPOINT") ?? ""; - const signals: Partial> = {}; - for (const signal of ["metrics", "logs", "traces"] as const) { - const url = readOtlpEnv( - `OTEL_EXPORTER_OTLP_${signal.toUpperCase() as Uppercase}_ENDPOINT`, - ); - if (url) signals[signal] = url; - } - if (!endpoint && Object.keys(signals).length === 0) return null; - const token = readEnv("DECO_OTEL_AUTH_TOKEN"); - const headers = { - ...(token ? { authorization: token } : {}), - ...parseKeyValues(readOtlpEnv("OTEL_EXPORTER_OTLP_HEADERS")), - }; - return { endpoint, signals, headers, site, limits: DEFAULT_LIMITS }; -} - export class TelemetryPipeline { readonly #destination: Destination; #settings: Required = { ...TELEMETRY_DEFAULTS }; @@ -430,24 +391,19 @@ export class TelemetryPipeline { #resource(): Attributes { const site = this.#destination.site; return { - "service.name": readEnv("OTEL_SERVICE_NAME") || site || "decocms-site", - "service.version": firstEnv(COMMIT_VARIABLES) ?? "unknown", - "deployment.environment.name": - readEnv("VERCEL_ENV") || - (readEnv("NODE_ENV") === "development" ? "development" : "production"), + "service.name": site || "decocms-site", + "service.version": "unknown", + // OPEN: the environment name defaults to "production"; `telemetry.resource` overrides it. + "deployment.environment.name": "production", ...(site ? { "deco.site": site } : {}), ...(this.#release ? { "deco.release": this.#release } : {}), - // The standard OTel override wins: service.version=, - // deployment.environment.name=preview, service.name=… - ...parseKeyValues(readEnv("OTEL_RESOURCE_ATTRIBUTES")), + // `telemetry.resource` wins: service.version=, deployment.environment.name=preview, … + ...this.#destination.resource, }; } async #send(signal: Signal, payload: unknown): Promise { - const { endpoint, signals } = this.#destination; - const url = - signals?.[signal] ?? (endpoint ? `${endpoint.replace(/\/+$/, "")}/v1/${signal}` : ""); - if (!url) return; // this signal has no destination + const url = `${this.#destination.endpoint.replace(/\/+$/, "")}/v1/${signal}`; const { body, gzipped } = await gzip(JSON.stringify(payload)); const headers: Record = { ...this.#destination.headers, @@ -485,26 +441,6 @@ export function setCurrentTelemetry(pipeline: TelemetryPipeline | undefined): vo // Helpers // --------------------------------------------------------------------------- -/** Where hosts put the deployed commit, tried in order (`service.version`). */ -const COMMIT_VARIABLES = [ - "DECO_COMMIT_SHA", - "WORKERS_CI_COMMIT_SHA", - "CF_PAGES_COMMIT_SHA", - "VERCEL_GIT_COMMIT_SHA", - "GITHUB_SHA", - "RENDER_GIT_COMMIT", - "SOURCE_VERSION", - "COMMIT_SHA", -]; - -function firstEnv(names: readonly string[]): string | undefined { - for (const name of names) { - const value = readEnv(name); - if (value) return value; - } - return undefined; -} - const QUERY = /(https?:\/\/[^\s?#"'<>]+)\?[^\s#"'<>]*/gi; const CREDENTIAL = /\b(Bearer|Basic)\s+[\w~+/.=-]+/gi; const SENSITIVE = @@ -537,23 +473,6 @@ function randomHex(bytes: number): string { return Array.from(values, (b) => b.toString(16).padStart(2, "0")).join(""); } -/** `k1=v1,k2=v2` with URL-encoded values, the format of the OTEL_* environment variables. */ -function parseKeyValues(raw: string | undefined): Record { - const out: Record = {}; - for (const pair of raw?.split(",") ?? []) { - const at = pair.indexOf("="); - if (at <= 0) continue; - try { - out[decodeURIComponent(pair.slice(0, at).trim())] = decodeURIComponent( - pair.slice(at + 1).trim(), - ); - } catch { - // A malformed pair is skipped. - } - } - return out; -} - function encodeAttributes(attributes: Attributes, scrub: (text: string) => string) { return Object.entries(attributes) .filter(([, value]) => value !== undefined) diff --git a/packages/blocks/src/v8/testFixtures.ts b/packages/blocks/src/v8/testFixtures.ts index 98d6cf19..a8135394 100644 --- a/packages/blocks/src/v8/testFixtures.ts +++ b/packages/blocks/src/v8/testFixtures.ts @@ -85,53 +85,62 @@ export function docsSnapshot(revision = "rev-1"): Snapshot { }; } -/** The host and token of the fake Studio API below. */ -export const STUDIO_HOST = "studio.decocms.com"; -export const STUDIO_TOKEN = "signed-token"; +/** The host of the fake delivery CDN below. */ +export const DRAFT_HOST = "delivery.decocms.com"; /** - * A fake Studio API answering draft pointers the way the docs describe - * (/next/content-delivery#draft-previews): per draft branch, the changes - * compared with production, behind the token Studio signed. Hand `fetch` to + * A fake delivery CDN answering draft pointers the way the docs describe + * (/next/content-delivery#draft-previews): `sites//drafts/.json` + * is `{ set, delete }`, served `no-cache` with an ETag that changes on every + * save, and a `304` to an `If-None-Match` that still matches. Hand `fetch` to * `vi.stubGlobal("fetch", …)`. */ export function fakeStudio() { - const branches = new Map; delete?: string[] }>(); + const drafts = new Map< + string, + { body: { set: Record; delete: string[] }; etag: string } + >(); const overrides = new Map Response>(); const requests: { url: string; init: RequestInit | undefined }[] = []; - const PATH = /^\/api\/acme\/decofile\/store\/([^/]+)\/changes$/; + const PATH = /^\/sites\/acme\/drafts\/([^/]+)\.json$/; + let saves = 0; const fetch = async (input: string | URL | Request, init?: RequestInit): Promise => { const url = new URL(String(input)); requests.push({ url: url.href, init }); - const branch = PATH.exec(url.pathname)?.[1]; - if (url.host !== STUDIO_HOST || branch === undefined) { + const slug = PATH.exec(url.pathname)?.[1]; + if (url.host !== DRAFT_HOST || slug === undefined) { return new Response("not found", { status: 404 }); } - const override = overrides.get(branch); + const override = overrides.get(slug); if (override) return override(); - if (url.searchParams.get("token") !== STUDIO_TOKEN) { - return Response.json({ error: "invalid token" }, { status: 401 }); + const draft = drafts.get(slug); + if (draft === undefined) return new Response("not found", { status: 404 }); + const headers = { + "cache-control": "no-cache, max-age=0, must-revalidate", + etag: draft.etag, + }; + if (new Headers(init?.headers).get("if-none-match") === draft.etag) { + return new Response(null, { status: 304, headers }); } - const changes = branches.get(branch) ?? {}; - return Response.json( - { format: 1, set: changes.set ?? {}, delete: changes.delete ?? [] }, - { headers: { "cache-control": "no-store", "access-control-allow-origin": "*" } }, - ); + return Response.json(draft.body, { headers }); }; return { fetch, requests, - /** Saves a draft branch's changes and returns the pointer Studio would mint for them. */ + /** Saves a draft (Studio's R2 write) and returns the pointer Studio would hand out. */ draft( changes: { set?: Record; delete?: string[] }, - { branch = "summer-sale", version = "9f3c1a", token = STUDIO_TOKEN } = {}, + { slug = "summer-sale", version = "9f3c1a" }: { slug?: string; version?: string } = {}, ): string { - branches.set(branch, changes); - return `${STUDIO_HOST}/api/acme/decofile/store/${branch}/changes?token=${token}@${version}`; + drafts.set(slug, { + body: { set: changes.set ?? {}, delete: changes.delete ?? [] }, + etag: `"etag-${++saves}"`, + }); + return `${DRAFT_HOST}/sites/acme/drafts/${slug}.json@${version}`; }, - /** Answers a branch with this response instead. */ - respond(branch: string, response: () => Response) { - overrides.set(branch, response); + /** Answers a draft with this response instead. */ + respond(slug: string, response: () => Response) { + overrides.set(slug, response); }, }; } diff --git a/packages/blocks/src/v8/types.ts b/packages/blocks/src/v8/types.ts index 5620c1a6..b206aa59 100644 --- a/packages/blocks/src/v8/types.ts +++ b/packages/blocks/src/v8/types.ts @@ -28,6 +28,19 @@ export type Snapshot = { blocks: Record; /** The alias table `deco content` writes: old type name → your type. */ aliases?: Record; + /** + * The hash of the `.deco/schema.gen.json` this content was built with, which + * `deco content` writes. A hosted release is swapped in only when its + * `schemaHash` equals the bundled content's; without one, never. + */ + schemaHash?: string; + /** + * The commit time of the git HEAD `deco content` built this content from + * (committer date, ISO 8601). A hosted release is swapped in only when its + * `publishedAt` is later; without one (a custom loader, an older content + * module, a build outside git), the bundled content counts as the oldest. + */ + committedAt?: string; }; /** @@ -48,11 +61,11 @@ export interface Loader { /** The parts of a draft pointer, `@`. */ export interface DraftPointer { - /** `host[:port]` of the Studio API that serves the draft's changes. */ + /** `host[:port]` that serves the draft's changes. */ host: string; /** Starts with `/`; opaque to the app. Never carries the `__variant` parameters. */ path: string; - /** Opaque: the commit of the editor's last save. `"local"` names no draft (`deco serve`). */ + /** Opaque: identifies the editor's last save. `"local"` names no draft (`deco serve`). */ version: string; /** The variants this preview forces (the query's `__variant` parameters); absent when none. */ variants?: ForcedVariant[]; @@ -212,12 +225,23 @@ export type Match = // The CMS and its clients // --------------------------------------------------------------------------- -export type TelemetryConfig = ( - | { site: string; token: string } - | { endpoint: string; headers?: Record } -) & { +/** + * Where telemetry goes. With `endpoint`, to that OTLP/HTTP collector, with + * `headers`; without one, to the hosted Deco CMS collector when `createCMS` + * has a `token`, and nowhere otherwise. + */ +export interface TelemetryConfig { + /** An OTLP/HTTP base URL; each signal is sent to `/v1/`. */ + endpoint?: string; + /** Headers sent with every batch to `endpoint`. */ + headers?: Record; + /** + * Resource attributes, merged over the defaults (`service.name`, + * `service.version`, `deployment.environment.name`, …). + */ + resource?: Record; limits?: { errorSampleRate?: number; traceSampleRate?: number }; -}; +} export interface CMSConfig { /** Your block functions; the CMS uses `{ ...builtIns, ...blocks }`. */ @@ -234,12 +258,24 @@ export interface CMSConfig { * (/next/api-reference#host-patterns). Without it, content may allow any host. */ hosts?: string[]; + /** + * The hosts a draft pointer may point at; replaces the defaults (v7's + * list: `.decocms.com` and the loopback hosts). An entry starting with a + * dot matches any subdomain; any other entry, that exact host. + */ + draftHosts?: string[]; }; /** The private key that decrypts `secret` blocks. */ secrets?: { key?: string }; - /** Your site's ID, for hosted releases. Drafts don't need it. */ + /** + * Your site ID (Studio's site slug): turns on hosted releases from + * `delivery.decocms.com`. Drafts don't need it. + */ site?: string; - /** Your site token (secret), for hosted releases. Drafts don't need it. */ + /** + * Your site token (server-only secret): sends telemetry to the hosted Deco + * CMS collector. Releases and drafts don't use it. Needs `site`. + */ token?: string; } @@ -280,13 +316,11 @@ export interface CMS { forRelease(): Client; /** * A client reading the draft a pointer names: its changes, fetched from a - * preview API domain (`*.decocms.com` and loopback by default; - * `DECO_PREVIEW_API_DOMAINS` replaces the list), over this server's - * production content. + * draft host (`*.decocms.com` and loopback by default; + * `createCMS({ preview: { draftHosts } })` replaces the list), over this + * server's production content. */ forDraft(pointer: string): Client; - /** A client pinned to a revision this CMS has served; an unknown revision behaves like the release. */ - forRevision(revision: string): Client; /** Ask the content source for newer content now; never throws. */ update(): Promise<{ updated: boolean }>; /**