Skip to content

TypeScript SDK: embed source and integrity sections in packed bundles #69905

Description

@jason810496

Background

#69295 adds airflow-ts-pack, which embeds the bundle manifest as a single first-line comment in bundle.mjs (//# airflowMetadata=<base64 YAML>); the Node coordinator reads only that head line (bounded readline(), 1 MiB cap).

The Go SDK's AFBNDL01 executable bundle format provides two things the TS format does not (see the Trailer Layout section of task-sdk/docs/executable-bundle-spec.rst):

  • Integrity validation — a SHA-256 digest over the code region, verified by the scanner before the bundle is used.
  • An embedded source region — the original entrypoint (.ts / .js), so the UI can display the code; needed for the native TypeScript Dag feature (TypeScript SDK: native TypeScript Dag declaration #69288).

Review on #69295 agreed to keep that PR to the packing flow and refine the packing spec in a follow-up. Since bundle.mjs must remain directly runnable by node, the structure cannot be a binary EOF trailer like AFBNDL01; instead the head comment can carry the layout — a format version, start/end offsets for each section, and the digest.

What needs to happen

  1. Specify a versioned head-comment layout for .mjs bundles recording section offsets (bundled code, embedded source, metadata) and a SHA-256 digest of the code region, documented alongside AFBNDL01 in task-sdk/docs/executable-bundle-spec.rst.
  2. Update airflow-ts-pack (ts-sdk/src/cli/pack.ts) to emit the new layout and embed the original entrypoint source.
  3. Update the Node coordinator (task-sdk/src/airflow/sdk/coordinators/node/coordinator.py) to verify the digest during scan — log and skip on mismatch, with the same (path, inode, mtime, size) result caching as the AFBNDL01 reader algorithm — and expose the embedded source.

Acceptance criteria

  • A packed bundle.mjs remains directly runnable with node bundle.mjs.
  • The coordinator logs and skips a truncated or modified bundle on digest mismatch.
  • The embedded entrypoint source is retrievable for display.
  • node-bundle-spec.rst documents the TS / JS layout with its own format version.

Context

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    Status
    Backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions