You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
#69295 adds airflow-ts-pack, which embeds the bundle manifest as a single first-line comment in bundle.mjs (//# airflowMetadata=<base64 YAML>); the Node coordinator reads only that head line (bounded readline(), 1 MiB cap).
The Go SDK's AFBNDL01 executable bundle format provides two things the TS format does not (see the Trailer Layout section of task-sdk/docs/executable-bundle-spec.rst):
Integrity validation — a SHA-256 digest over the code region, verified by the scanner before the bundle is used.
Review on #69295 agreed to keep that PR to the packing flow and refine the packing spec in a follow-up. Since bundle.mjs must remain directly runnable by node, the structure cannot be a binary EOF trailer like AFBNDL01; instead the head comment can carry the layout — a format version, start/end offsets for each section, and the digest.
What needs to happen
Specify a versioned head-comment layout for .mjs bundles recording section offsets (bundled code, embedded source, metadata) and a SHA-256 digest of the code region, documented alongside AFBNDL01 in task-sdk/docs/executable-bundle-spec.rst.
Update airflow-ts-pack (ts-sdk/src/cli/pack.ts) to emit the new layout and embed the original entrypoint source.
Update the Node coordinator (task-sdk/src/airflow/sdk/coordinators/node/coordinator.py) to verify the digest during scan — log and skip on mismatch, with the same (path, inode, mtime, size) result caching as the AFBNDL01 reader algorithm — and expose the embedded source.
Acceptance criteria
A packed bundle.mjs remains directly runnable with node bundle.mjs.
The coordinator logs and skips a truncated or modified bundle on digest mismatch.
The embedded entrypoint source is retrievable for display.
node-bundle-spec.rst documents the TS / JS layout with its own format version.
Background
#69295 adds
airflow-ts-pack, which embeds the bundle manifest as a single first-line comment inbundle.mjs(//# airflowMetadata=<base64 YAML>); the Node coordinator reads only that head line (boundedreadline(), 1 MiB cap).The Go SDK's AFBNDL01 executable bundle format provides two things the TS format does not (see the Trailer Layout section of
task-sdk/docs/executable-bundle-spec.rst):.ts/.js), so the UI can display the code; needed for the native TypeScript Dag feature (TypeScript SDK: native TypeScript Dag declaration #69288).Review on #69295 agreed to keep that PR to the packing flow and refine the packing spec in a follow-up. Since
bundle.mjsmust remain directly runnable bynode, the structure cannot be a binary EOF trailer like AFBNDL01; instead the head comment can carry the layout — a format version, start/end offsets for each section, and the digest.What needs to happen
.mjsbundles recording section offsets (bundled code, embedded source, metadata) and a SHA-256 digest of the code region, documented alongside AFBNDL01 intask-sdk/docs/executable-bundle-spec.rst.airflow-ts-pack(ts-sdk/src/cli/pack.ts) to emit the new layout and embed the original entrypoint source.task-sdk/src/airflow/sdk/coordinators/node/coordinator.py) to verify the digest during scan — log and skip on mismatch, with the same(path, inode, mtime, size)result caching as the AFBNDL01 reader algorithm — and expose the embedded source.Acceptance criteria
bundle.mjsremains directly runnable withnode bundle.mjs.node-bundle-spec.rstdocuments the TS / JS layout with its own format version.Context
task-sdk/docs/executable-bundle-spec.rst