diff --git a/docker-compose.published.yml b/docker-compose.published.yml index 427410a15..e105ba82e 100644 --- a/docker-compose.published.yml +++ b/docker-compose.published.yml @@ -16,7 +16,7 @@ name: durable-workflow-server # release whose default names another image). Set DW_SERVER_IMAGE to run # any other native image, by tag or digest; pin a digest for production # change control. -x-server-image: &server-image ${DW_SERVER_IMAGE:-durableworkflow/server:3.11.0} +x-server-image: &server-image ${DW_SERVER_IMAGE:-durableworkflow/server:3.12.0} services: server: diff --git a/docs/releases/3.12.0.md b/docs/releases/3.12.0.md new file mode 100644 index 000000000..dc0dbf966 --- /dev/null +++ b/docs/releases/3.12.0.md @@ -0,0 +1,175 @@ +# Durable Workflow Server 3.12.0 (native) + +Server 3.12.0 keeps native local external payloads readable and reclaimable +after a namespace's storage write policy is disabled or moved. It also adds a +read-only query that counts and pages open runs by their retained +version-marker decision. Its immutable images are +`durableworkflow/server:3.12.0` and `ghcr.io/durable-workflow/server:3.12.0`; +the published Compose default selects 3.12.0. Rolling aliases move separately +after published-image qualification. + +## Retained payloads across storage policy changes + +In 3.11.x and earlier, disabling a namespace's local external-payload policy +or moving its location left objects written under the earlier policy +unresolvable. Reads, retention pruning, abandoned-upload cleanup and namespace +deletion resolved every stored object through the current policy. + +The native local driver now records an immutable binding with each admitted +object: its namespace, storage generation and canonical root. Reads, run +pruning, abandoned-upload cleanup and namespace deletion use that original +binding. Disabling a policy stops new uploads only. Moving the location sends +new objects to the new root; existing objects stay where they were. Changing +the policy does not copy objects between roots. A disable request that omits +the location now keeps the configured location. An upload admitted before a +policy change can finish at its recorded root; later uploads must pass the +current enabled policy. Deletion journals keep their binding, so mixed +generations, backup holds and another namespace's ownership of the same +physical object are preserved. + +Keep every retained root mounted and accessible to the Server and its +maintenance processes until its objects and committed deletion records are +reclaimed. A missing or substituted root returns the nonretryable +`external_payload_storage_binding_unavailable` outcome (HTTP 409). The runtime +does not recreate an empty root or infer storage authority from an arbitrary +URI. Backups must include the registry and deletion journals together with the +bytes of every retained root, including roots no longer selected for uploads. +See [native local-storage policy changes](../contracts/external-payload-storage.md#native-local-storage-policy-changes). + +### Objects written before the upgrade + +The schema step adds the binding column without backfilling it. An existing +registration is bound on first use only when the namespace's admitted local +policy reproduces its recorded URI exactly. A policy change first pins every +unbound registration to the current policy or, when the request names an +explicit location, to that location where the recorded URI matches. If any +unbound registration matches neither, the policy change is refused with +`external_payload_storage_binding_unavailable` and the policy is not replaced. + +Objects whose namespace still resolves to their original root, including +through a disabled policy, become readable and are bound on first use. Under +3.11.x and earlier, moving the location, or disabling with a request that +omitted a custom location, replaced the root the Server resolves. Objects +stranded that way stay unavailable after the upgrade. To repair them, keep the +original root mounted and submit a policy change that names it explicitly +(enabled or disabled); that pins the stranded registrations to it, and an +enabled policy also sends new uploads there. A later policy change can move +new uploads again without unbinding them. Because one request can pin only to +the current and the requested location, stranded registrations under more than +one earlier root cause every policy change to be refused; this release +documents no repair for that case. This release does not relocate objects or +recover bytes that are no longer present at their recorded location. + +## Open-run version usage + +`GET /api/workflow-version-usage` reports, for one namespace, workflow type and +change ID, which open runs retain a version-marker decision below a proposed +`min_version` (`legacy`), at or above it (`current`), or have no authoritative +answer (`unknown`). Discover support by requiring +`capabilities.workflow_version_usage` to be exactly `true` on +`GET /api/cluster/info`; a missing capability or a failed request is not an +empty cohort. The endpoint requires an operator or admin token and the +existing control-plane version header. + +Missing markers, pruned or unsupported history, malformed ranges and conflicting +decisions remain `unknown` with a specific reason. Counts cover the whole +open-run cohort before the optional state filter, and each response is one +read-only snapshot. Signed, opaque cursors page by ascending run ID under the +first page's fixed upper bound; invalid or mismatched cursors return HTTP 422. +Pages are a live observation, not a deployment fence, and the response carries +no safe-to-remove flag. The query reads no workflow arguments, results, memos or +external payloads and changes no marker production, history or replay +behavior. The shared v1 contract and paging rules are in +[open-run version usage](../workflow-version-usage.md). + +## Portable native library builds + +Unix-specific storage-admission observation checks are now compiled only on +Unix targets, so the native library builds on other platforms, such as the +CLI's Windows development runtime. Admission stays disabled by default. A +configured observation file on a non-Unix target fails closed with +`storage_admission_unavailable`; no fallback reader is introduced. The Unix +descriptor, ownership and mode checks are unchanged, and the published Linux +images behave as in 3.11.0. This does not add native image platforms. See +[storage admission](../storage-admission.md#admission-observation-file). + +## Upgrade and rollback + +This release requires an explicit additive schema upgrade from 3.11.x on every +backend: + +| Backend | Schema step | +| --- | --- | +| SQLite | 14 → 15 | +| PostgreSQL | 15 → 16 | +| MariaDB | 15 → 16 | +| MySQL | 16 → 17 | + +Each step adds the nullable `runtime_external_payloads.storage_binding` column +and `workflow_namespaces.external_payload_storage_generation` (default `0`). +It rewrites no existing rows. Stop every Server replica, keep the database +running, and back up the database and every external-payload root. Select the +new immutable image and run that image's `schema-upgrade` once before +restarting every replica on the same version. Startup does not upgrade an older +schema automatically and refuses to serve it. MySQL and MariaDB resume an +interrupted upgrade when `schema-upgrade` is run again; SQLite and PostgreSQL +apply their step transactionally. From 3.9.x or earlier, the same command also +applies the intervening steps described in the +[3.10.0 upgrade notes](3.10.0.md#upgrade-and-rollback). + +For a single-node Compose deployment, stop Server while leaving MySQL running, +back up MySQL and `server_storage`, set `DW_SERVER_IMAGE` to the 3.12.0 image, +then pull and upgrade once: + +```bash +docker compose --env-file durable-workflow.prod.env \ + -f docker-compose.published.yml stop server +docker compose --env-file durable-workflow.prod.env \ + -f docker-compose.published.yml pull server +docker compose --env-file durable-workflow.prod.env \ + -f docker-compose.published.yml run --rm --no-deps server schema-upgrade +docker compose --env-file durable-workflow.prod.env \ + -f docker-compose.published.yml up -d --wait +``` + +From 3.11.x on MySQL, the upgrade reports `from_schema_version:16`, +`schema_version:17`, `status:upgraded`; a repeat reports `status:current`. +Then verify `/api/ready`, `/api/cluster/info` version `3.12.0`, and worker +registration. + +With Helm, run the 3.12.0 image's `schema-upgrade` through the opt-in +`native.schemaUpgrade.enabled` hook or the manual Job described in the chart's +native schema upgrade section, after scaling every Server replica to zero. The +chart that selects 3.12.0 by default is published separately; follow its +versioned upgrade notes, and make any explicit `image.tag` or `image.digest` +override select the intended image. + +Rolling back requires restoring the pre-upgrade database and payload-storage +backup. An older binary refuses the upgraded database with +`native_schema_newer_than_binary`; do not try to run one against it. Objects +written and policy changes made after the upgrade are not in that backup. + +Worker protocol `1.20`, control-plane version `2`, codec and command shapes, and +the public external-payload reference shape are unchanged. Existing +command-key, completion-byte, storage-admission and unsupported service-upsert +boundaries remain in force. The 3.11.0 fired-timer redrive rules are unchanged. + +## Qualification and retained limits + +The storage-binding fix is native only. Published PHP Server 2.11.8 remains +retired and unpatched and keeps its earlier behavior; see the +[retirement/security notice](../security/retired-php-server-2026-11-13.md). +The native parity contract records the retention and namespace-deletion +expectations that now differ from PHP. The PHP takeover runbook keeps its exact +historical version, backend and state limits. + +CLI and Waterline adoption of the version-usage endpoint is tracked on +[#1220](https://github.com/durable-workflow/server/issues/1220) and remains +separately attributable. The CLI's Windows development runtime is a source +build and does not qualify a published Server image. Installed SDK timer-clock +evidence ([#980](https://github.com/durable-workflow/server/issues/980)) and the +MariaDB registration-timeout investigation +([#1204](https://github.com/durable-workflow/server/issues/1204)) are unchanged +by this release. Publication, exact-image corpus, the published Compose cells, +scans, protected rolling aliases and the final Release must qualify this new +schema image.