Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/do-sqlite-keepalive.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@cloudhal/storage": patch
---

createMiniflareDoSqlite() sends its Durable Object calls over kept-alive connections instead of opening a new TCP connection per call through Miniflare 4's dispatchFetch, so long suites no longer exhaust local ports on macOS (#116).
18 changes: 16 additions & 2 deletions apps/studio/test/setup.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import "@testing-library/jest-dom/vitest";
import { configure } from "@testing-library/react";
import { beforeAll } from "vitest";
import { cleanup, configure } from "@testing-library/react";
import { afterAll, afterEach, beforeAll, vi } from "vitest";
import { LOCALE_LIST, loadLocale, loadNamespace, NAMESPACES } from "../src/i18n/index.tsx";

// Lazy message namespaces (src/i18n/namespaces.ts) arrive with their routes in the app. Screens
Expand All @@ -20,6 +20,20 @@ beforeAll(() =>
// take longer to settle. The test timeout (vitest.config.ts) stays the upper bound.
configure({ asyncUtilTimeout: 5000 });

// Vitest runs without globals, so Testing Library never registers its own cleanup: trees stayed
// mounted, Base UI / Floating UI work committed outside act after the file ended, and React's
// passive flush (setImmediate) read `window` after happy-dom's teardown ("window is not defined",
// CI run 39959905949). cleanup() is idempotent, so files that call it themselves are unaffected.
// Real timers come back first so the drain below is never faked; setImmediate is FIFO, so three
// rounds also cover a scheduler that yields and posts again.
afterEach(() => {
cleanup();
vi.useRealTimers();
});
afterAll(async () => {
for (let i = 0; i < 3; i++) await new Promise<void>((resolve) => setImmediate(resolve));
});

// Base UI / Kumo touch these at module level in some chunks.
globalThis.ResizeObserver ??= class {
observe() {}
Expand Down
2 changes: 1 addition & 1 deletion packages/storage/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ await sql.batch([

Migrations are TypeScript modules under `migrations/` (plain SQL strings) so they bundle for Workers without filesystem access. Each migration is applied as one atomic batch.

Every adapter must pass `describeSqlDriverConformance` / `describeBlobStoreConformance`; the D1, R2 and Durable Object suites run against Miniflare (workerd in-process), so no Cloudflare account is needed. `createMiniflareDoSqlite()` runs `DoSqliteDriver` inside a test-only Durable Object and hands Node a driver whose every call is one request to it, so any `SqlDriver` suite (core-ts's included) runs on Durable Object SQLite.
Every adapter must pass `describeSqlDriverConformance` / `describeBlobStoreConformance`; the D1, R2 and Durable Object suites run against Miniflare (workerd in-process), so no Cloudflare account is needed. `createMiniflareDoSqlite()` runs `DoSqliteDriver` inside a test-only Durable Object and hands Node a driver whose every call is one request to it, over kept-alive connections, so any `SqlDriver` suite (core-ts's included) runs on Durable Object SQLite.

A whole site in one Durable Object (the playground's shape):

Expand Down
71 changes: 56 additions & 15 deletions packages/storage/src/testing-do.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
/**
* Test harness for `DoSqliteDriver` (RFC-0036 §2 invariant 6). A Miniflare Worker holds a test-only,
* SQLite-backed Durable Object that runs the real driver on its own `ctx.storage`; from Node, every
* `SqlDriver` call is one request to that object. Suites written for any `SqlDriver` (the storage
* conformance suite, core-ts's behavioural suite) thereby run against Durable Object SQLite.
* `SqlDriver` call is one request to that object, sent over kept-alive connections. Suites written
* for any `SqlDriver` (the storage conformance suite, core-ts's behavioural suite) thereby run
* against Durable Object SQLite.
*/
import {
BlobIntegrityError,
Expand Down Expand Up @@ -54,10 +55,7 @@ const revive = (_k: string, v: unknown) =>
? new Uint8Array(Buffer.from((v as { $b: string }).$b, "base64"))
: v;

type Dispatch = (
url: string,
init: { method: string; body: string },
) => Promise<{ text(): Promise<string> }>;
type Dispatch = (path: string, body: string) => Promise<string>;

/** Node-side `SqlDriver` whose every call runs `DoSqliteDriver` inside the test Durable Object. */
class RemoteDoSqliteDriver implements SqlDriver {
Expand All @@ -71,11 +69,8 @@ class RemoteDoSqliteDriver implements SqlDriver {
}

async #call<T>(op: string, args: unknown[]): Promise<T> {
const res = await this.#dispatch(`http://do.test/?db=${encodeURIComponent(this.#db)}`, {
method: "POST",
body: encode({ op, args }),
});
const out = JSON.parse(await res.text(), revive) as {
const text = await this.#dispatch(`/?db=${encodeURIComponent(this.#db)}`, encode({ op, args }));
const out = JSON.parse(text, revive) as {
ok?: T;
error?: { name: string; message: string };
};
Expand Down Expand Up @@ -104,6 +99,50 @@ class RemoteDoSqliteDriver implements SqlDriver {
async close() {}
}

/**
* Miniflare 4's `dispatchFetch` sets undici's `reset` on every request, so each call closed its
* connection and left a local port in TIME_WAIT for 30 s; the Durable Object suites exhausted the
* ephemeral ports on macOS (#116). Calls go straight to the entry socket instead, over kept-alive
* connections; idle ones close after 1 s. The entry worker routes a plain request to the only
* Worker by URL, and the Worker reads only `?db=` and the body.
*/
async function keepAliveDispatch(entry: URL): Promise<{ dispatch: Dispatch; close(): void }> {
const { Agent, request } = await import("node:http");
const agent = new Agent({ keepAlive: true, maxSockets: 4, timeout: 1_000 });
const dispatch: Dispatch = (path, body) =>
new Promise((resolve, reject) => {
const headers = {
"content-length": Buffer.byteLength(body),
"MF-Disable-Pretty-Error": "true",
};
const url = new URL(path, entry).href;
const req = request(url, { method: "POST", agent, headers }, (res) => {
const chunks: Buffer[] = [];
res.on("data", (chunk: Buffer) => chunks.push(chunk));
// A connection that closes mid-body (workerd exiting, `close()` during a call) ends the
// response without 'end', and the request emits no 'error' once a response exists.
res.on("error", reject);
res.on("close", () => {
if (!res.complete)
reject(new Error("do-sqlite test harness: connection closed mid-response"));
});
res.on("end", () => {
const text = Buffer.concat(chunks).toString("utf8");
// The Worker answers 200 for everything the driver does, its errors included. Anything
// else is the harness failing: a plain Error, so it never passes `toThrow(StorageError)`.
if (res.statusCode === 200) resolve(text);
else
reject(
new Error(`do-sqlite test harness: HTTP ${res.statusCode}: ${text.slice(0, 200)}`),
);
});
});
req.on("error", reject);
req.end(body);
});
return { dispatch, close: () => agent.destroy() };
}

/**
* Spins up a Miniflare Worker with a SQLite-backed Durable Object running `DoSqliteDriver`.
* `driver(db)` returns a driver for the object named `db` (a fresh, empty database per name).
Expand Down Expand Up @@ -137,10 +176,12 @@ export async function createMiniflareDoSqlite(): Promise<{
],
durableObjects: { DB: { className: "TestDb", useSQLite: true } },
});
await mf.ready;
const dispatch: Dispatch = (url, init) => mf.dispatchFetch(url, init);
const http = await keepAliveDispatch(await mf.ready);
return {
driver: (db = "default") => new RemoteDoSqliteDriver(dispatch, db),
dispose: () => mf.dispose(),
driver: (db = "default") => new RemoteDoSqliteDriver(http.dispatch, db),
dispose: async () => {
http.close();
await mf.dispose();
},
};
}
Loading