Repository navigation
Description
Current delivery status
The embedded, sequential compensation-scope APIs shipped as experimental in 2.29.0, 2.29.1 and 2.30.0. The releases include retained versioned obligations, canonical recovery after origin closure, child-first verification, reserved children and the opt-in original-root reservation. 2.30.0 also refuses a schema rollback that would discard the ledger. The scope guide describes the contract.
Decision (revised after independent review): 2.31.0 ships #1195 and #1206 without the stability promotion. The promotion moves to a later minor release (2.32.0 or later, decided at that release) after the surface fixes in #1216 and a second independent review. The target scope is unchanged:
- Becomes stable (
@api):- authoring:
DurableScope,RegisteredObligation,Obligation,ScopeIdentity; - handler contract:
ScopedOperation,OperationContext,ForwardResult,Resolution,HandlerRegistry; - host:
ScopeAuthorizationandScopeAuthorizationRefused,ScopeRecovery::request()anddescribe()(with their result enums),ScopeRetention::release().
- authoring:
- Stays experimental:
reserveChild(), original root admission, parallel compensation and deeper nesting. Engine members stay@internal. - Guarantees and minimum limits: as listed in
api-stability.mdon Prepare the compensation-scope surface for promotion (#1063) #1216.
Remaining:
- Merge Prepare the compensation-scope surface for promotion (#1063) #1216 (surface fixes) and #1213 (markings), and ship them in a published tag whose upgrade journey is green.
- Second independent review of the surface.
- Merge #1214, the promotion (draft until then).
- Publish the docs-site guide, durable-workflow.github.io#568 (all eight languages; it marks the API "planned stable, 2.32.0 or later").
- After the guide is published, the reporter's consumer acceptance closes this issue.
Full business-state-machine delivery is tracked separately in the StateMachine feature.
Primary owning public component
Workflow engine
User problem
We use Durable Workflow to execute long-running business processes with nested workflows. Business cancellation may require undoing earlier operations after several transitions, while a child workflow is running, or after that child has already completed.
A forward activity may also produce an external side effect without returning a recorded result. Compensation must then identify and reconcile the original operation without relying on its successful return value.
Native Saga, replay, child workflows, and cooperative cancellation already provide important building blocks. We need a supported contract for retaining and addressing compensation obligations beyond the lifetime of their original workflow run.
Without such a contract, our integration must implement its own obligation registry, request routing, recovery ownership, and completion protocol around native Saga.
Example: a shipment workflow creates a carrier booking and completes. The parent order is cancelled later. The original shipment obligations must still be discoverable and compensated, even though its workflow run can no longer handle a cancellation request.
Proposed public contract
Introduce an optional compensation-scope abstraction built on native Saga.
The following describes proposed semantics, not existing API names:
-
Stable identity
A scope identifies the original business execution and its generation independently of a transient workflow run ID. Requests must never silently target a replacement execution. -
Registration before external effects
Allow a durable compensation obligation to be registered before the forward operation starts. Its handler receives a stable operation reference and can reconcile an absent, completed, partial, or unknown effect without requiring a successful forward result. -
Portable descriptors
Retained obligations contain a handler type/version, portable arguments, registration order, and ownership information. Persisted recovery must not depend on serializing arbitrary PHP closures. -
Idempotent requests
Request compensation using a stable request ID. Distinguish durable request acceptance from verified compensation completion. -
Nested scopes
A parent can register a child scope as a compensation participant. In sequential compensation, the parent waits for that child's obligations to be resolved before continuing with earlier parent obligations. -
Recovery after run closure
An explicit recovery execution may claim unresolved obligations from a closed run and execute them through native Saga. Preserve the original run's terminal history and prevent concurrent recovery owners. -
Explicit outcomes
Expose pending, resolved, and unresolved failure outcomes. Request delivery, child termination, timeout, or exhausted retries must not imply successful business compensation. -
Retention and compatibility
Define retention, release, handler-version compatibility, and unavailable-history behavior. Existing Saga and cancellation APIs remain compatible.
Business rules, external-effect verification, authorization, and idempotent handlers remain application responsibilities. This proposal does not promise exactly-once external effects.
Acceptance criteria
- Existing Saga behavior and public APIs remain backward compatible.
- Worker restart preserves obligation identity, ordering, and recorded compensation progress.
- A crash after an external effect but before recording the forward result can be recovered using the original operation reference.
- Cancellation during a child workflow waits for verified child compensation before earlier parent compensation in sequential mode.
- A completed child with retained obligations can be compensated through an explicit recovery execution.
- A request racing with run closure remains discoverable and receives an explicit outcome.
- Duplicate requests and competing recovery workers cannot create independent owners for the same obligation.
- Late forward completion is reconciled; compensation cannot report success merely because the effect was temporarily absent.
- Retry exhaustion, unknown outcomes, and failed compensation remain visibly unresolved.
- Generation changes cannot redirect an old request to a newer business execution.
- Retention expiry and incompatible handler versions produce explicit failures.
- Public examples and automated tests cover restart, lost acknowledgement, closed-child recovery, and compensation failure.
Initial scope may be limited to embedded Laravel and sequential nested compensation. Parallel dependency ordering can be a separate enhancement.
Supporting public evidence
-
Official “Sagas” documentation:
Native Saga executes registered compensations in reverse order and reconstructs registration through replay. Its example registers compensation after the forward activity succeeds. It also explains that workflow timeout and immediate cancellation can close the run before cleanup executes. -
Official “Child Workflows” documentation:
Child invocation identity is distinct from run identity. Parent-close policies act on open children, skip children that have already closed, and do not prevent parent closure when enforcement fails. -
Official “Cooperative cancellation” documentation:
Active runs support bounded cleanup, child cancellation policies, replay recovery, and cancellation diagnostics. These are existing capabilities and should be reused.
These documented boundaries motivate a separate contract for retained business compensation obligations, particularly after the originating run has closed. This is an enhancement proposal, not a claim that existing Saga replay or cooperative cancellation is broken.
Dependencies and affected repositories
Primary:
- durable-workflow/workflow:
Public compensation-scope contract, durable obligation storage, recovery ownership, native Saga integration, and regression tests.
Documentation:
- durable-workflow/durable-workflow.github.io:
Contract documentation, lifecycle guarantees, limitations, and runnable examples.
Optional follow-up:
- durable-workflow/waterline:
Display scope identity, originating and recovery runs, unresolved obligations, compensation requests, and verified outcomes.
Conditional follow-up:
- Service-mode runtime and SDK support, if maintainers decide to expose the same contract beyond embedded Laravel. Cross-SDK support is not required for the initial implementation.
No Symfony Workflow dependency is required. The contract should remain independent of application entities and state-machine libraries.
Public intake checks
- I searched open and closed GitHub issues for an existing request.
- This request contains only public product contracts and public-safe rationale.
Activity
rmcdaniel commentedon Nov 7, 2026
Security contract review — required invariants before implementation
I reviewed the current public proposal, exact body 6,936B/SHA256
8daa9ca0a6d13b56da2074c6f1197ea7fd05d485414010f202b78f8d19bb0544. The proposed stable generation, registration-before-effect, explicit outcomes, and fenced recovery direction is sound. The following must be part of the public contract and causal tests before any Source acceptance; they are not optional implementation details.Tenant and authority binding. Scope, obligation, operation, child-scope, and request identities must be bound to the originating namespace/application plus immutable business generation. Lookup by a globally guessable ID must not cross that boundary. Recovery must re-establish an authorized principal/capability for the exact scope; replay must not silently substitute the current user or a newer generation.
Safe portable descriptors. Persist only a closed, versioned handler registry and canonical bounded data. Never deserialize arbitrary PHP objects/classes, closures, callable names, service-container references, or code supplied by the stored record. Handler type/version, argument schema, and payload hash must be validated before claim/execution. Secrets and raw credentials must not be embedded in history, descriptors, errors, or audit output; use protected references with explicit expiry/rotation failure.
Atomic ownership and fencing. Claim, renew, abandon, and complete must use one monotonic fence/lease generation in the same transaction as visible state. A stale worker must be unable both to execute under lost authority and to record success/failure afterward. Duplicate request IDs bind the original caller/scope/content hash; reuse with different arguments, generation, or authorization must refuse rather than deduplicate.
Uncertain external-effect state. Registration-before-effect needs an explicit state machine separating registration, dispatch/attempt, observed effect, compensation attempt, verified compensation, and unresolved/unknown. A crash, timeout, lost acknowledgement, or temporarily absent effect must never become “resolved.” A handler may reconcile absence only from authoritative external evidence and must preserve the original operation reference and attempt identity.
Deterministic replay versus live authorization. Replay reconstructs recorded decisions only; it must not perform live I/O, resolve mutable service bindings, read current authorization, or execute compensation. Live recovery performs current authorization/handler-availability checks outside replay and records their exact decision/version. The boundary needs predecessor/candidate tests proving replay cannot trigger an effect.
Nested-scope graph safety. Child relationships must be acyclic, generation-bound, and capped by depth, fan-out, descriptor bytes, and total obligations. Parent sequential progress requires a verified terminal child-compensation outcome, not request delivery, child termination, lease expiry, or timeout. Missing/expired child history stays explicitly unresolved.
Compatibility and retirement. Handler upgrades require an explicit compatibility/migration table; an unknown or incompatible version refuses before effect. Retention expiry must be atomic with claims/requests and leave an auditable terminal reason, never redirect or silently drop a pending obligation. Released scope data must not leave reusable operation/request identifiers that can address a new generation.
Operational bounds and audit. Cap registrations, descriptor size, retries, lease duration, total recovery work, nested traversal, and error payloads. Audit records need immutable scope/generation, originating run, obligation/order, request/content hash, claimant/fence, handler version, attempt, authoritative outcome, and retirement reason—while redacting secret material. Metrics must distinguish accepted, executing, verified, failed, unknown, expired, incompatible, and fenced-stale states.
Required adversarial controls include cross-tenant ID substitution; same request ID/different payload; generation replacement; stale claimant after lease transfer; crash before/after external effect; lost request and completion acknowledgements; late forward completion racing compensation; cyclic/deep child graphs; descriptor type/version substitution; secret-bearing payload/error refusal; replay attempting I/O; retention racing claim; and parent progress after only child delivery/timeout. Each must demonstrate the unsafe predecessor or naive model and the exact fail-closed candidate behavior.
This review does not select a schema, storage backend, API name, branch, runtime execution, or cross-SDK expansion. Existing Saga/cancellation behavior remains authoritative until a separately reviewed implementation and documentation are accepted.
rmcdaniel commentedon Nov 7, 2026
Triaged as an additive compensation-scope feature, with General owning the contract-design follow-up. The initial design scope is optional embedded Laravel with sequential nested compensation. I have marked this P2/ready; it is not a release or API commitment.
The existing V2 Saga implementation accepts a callback before the forward call when the application already has a stable operation reference. Replay rebuilds that run-local callback list. That can cover a live-run compensation pattern without depending on a successful forward return value, but does not supply the separately addressed, retained obligation registry and closed-run recovery ownership requested here.
The first contract draft will separate:
Next action: map the proposed state transitions and transaction/retention boundaries onto existing activity, cooperative-cancellation and terminal-history code, then publish a concrete design and crash/race test matrix before selecting schema or API names. That matrix must cover pre-result effect loss, closure/request races, completed-child recovery, stale owners, duplicate IDs with changed content, generation replacement, handler incompatibility and retention expiry. The security contract envelope is part of that review; its external-fencing requirement will be expressed with the application responsibility above rather than an exactly-once promise.
Existing Saga/cancellation APIs and original terminal histories are the compatibility baseline. Parallel dependency ordering, service-runtime/cross-SDK exposure and Waterline presentation remain separate follow-ups. No implementation, migration, version, tag or delivery date is selected by this triage.
rmcdaniel commentedon Nov 7, 2026
Contract-design follow-up for the approved intake. General owns the initial optional embedded-Laravel/sequential implementation. This is the proposed state/transaction contract for review, with existing Saga, cancellation and terminal outcomes preserved. Service-mode, cross-SDK exposure, parallel compensation and UI work remain later slices.
Current source anchors are the published 2.21 Workflow Saga methods, workflow-task commit/dispatch boundary, attempt-outcome authority, and run retention cleanup. Existing
addCompensation()can register a callable before the forward call when its operation reference is already known; it does not itself create this retained obligation contract.Address and descriptor. Every operation uses an application/namespace-bound scope key plus an immutable business generation, originating run, obligation key, registration sequence, stable forward-operation reference and descriptor-content hash. A handler is selected from an application-configured closed type/version registry; stored data cannot supply a class, closure, service name or arbitrary callable. Canonical argument schemas and byte/count/depth limits are checked before storage and execution. Protected references replace embedded credentials.
Registration and dispatch. First execution commits the descriptor and its matching history decision before the associated forward task becomes dispatchable. Replay validates/reuses that recorded decision without database/provider I/O or resolving a mutable handler. Reusing an obligation key with different content is a determinism/identity failure. The compensation-request transaction closes that scope's new-forward-dispatch gate; registration racing it is either committed before the gate or explicitly refused. Already dispatched attempts remain tracked until authoritative reconciliation.
Separate state dimensions. Retain forward state (
registered,dispatch_pending,in_flight,observed,unknown) separately from compensation state (pending,executing,verified,failed_unresolved,unknown). Request acceptance is durable routing evidence only. Handler return, a timeout, cancellation, retry exhaustion, or momentary provider absence is insufficient forverified. Verification records the exact operation/attempt and authoritative external observation; a still-possible late forward effect keeps the obligation unresolved. These are conceptual states, not selected enum/API names.Request and owner transaction. Current authorization is required before accepting or disclosing a request. A request ID binds its original authorized caller, scope/generation and canonical content; same-ID/different-content requests refuse. Claim/renew/transfer/outcome updates atomically compare one monotonic fence, owner and lease with visible obligation progress. A stale claimant cannot schedule another permitted attempt or commit its result. External calls happen outside database locks. Crucially, a database fence cannot recall an already-issued provider request: external fencing/idempotency and reconciliation remain application/provider duties, and uncertainty cannot be converted into success merely after a lease expires.
Live versus closed recovery. An active run uses its ordinary replay/cooperative-cleanup path under the one scope owner. A closed originating run is never reopened or given a replacement terminal outcome. An explicit recovery execution records its own run identity, handler/authorization decisions and attempts, and claims only unresolved obligations for the original scope/generation. It executes through native activities/Saga ordering; it does not deserialize old callbacks. Recovery authorization or handler-version failure is visible and occurs before an effect.
Sequential children. Register the exact child scope/generation as a participant before relying on it for compensation. Traversal must be acyclic and bounded. Reverse registration order pauses at a child participant until that exact child's obligations are verified; delivery acknowledgement, a completed child workflow, child termination or cleanup timeout cannot advance earlier parent obligations. Missing/incompatible/expired child evidence produces an explicit unresolved outcome.
Retention and compatibility. Opt-in obligations and dedupe/generation tombstones have an explicit retention/release policy independent of ordinary run-detail pruning. Pruning must either preserve every required descriptor/authority/outcome reference or refuse atomically while it is needed. Claim/request/release races cannot erase work or permit an old identifier to target a new generation. No legacy Saga serialization, cancellation deadline, parent-close behavior or existing terminal result changes. Unknown handler versions and missing history fail explicitly rather than migrating silently.
The first causal test matrix is:
Next design checkpoint: specify the concrete storage constraints, per-path lock order, command/history projection and retention migration against these boundaries, then open the smallest Source PR with crash/race controls. In particular, existing activity lock order and run-retention holds must be extended coherently rather than adding an independent recovery scheduler. Security's contract envelope remains part of acceptance. No migration, public API spelling, release version or runtime execution is selected by this design note.
rmcdaniel commentedon Nov 7, 2026
DESIGN PASS — exact design record
6711670575(7,507B / SHA-25681a5ec220410ba8f7fa20683f419e9c85a161d64fd26aec57497a6fb81fe7503), with no Source/API selection.The proposed state, transaction, recovery, child-order, retention, and crash matrix faithfully carries the Security envelope in
6711381445. In particular it preserves application/namespace plus immutable-generation authority; content-bound request and obligation identity; a closed versioned handler registry with bounded canonical descriptors; registration before forward dispatch; replay without live I/O or mutable resolution; separate forward/compensation/unknown dimensions; monotonic owner/fence transitions; original-run terminal immutability; explicit closed-run recovery identity; verified child compensation before parent progress; and atomic retention/refusal rather than silent migration, redirection, or loss.The external-effect limitation is correctly explicit: a database fence can stop a stale owner from scheduling or committing, but cannot recall an already-issued provider request. Provider fencing/idempotency and authoritative reconciliation remain necessary, and lease expiry, timeout, cancellation, handler return, or temporary absence cannot manufacture a verified outcome. That distinction must remain visible in any public API and operator status.
The listed crash/race matrix is sufficient as the first design-level adversarial matrix. Before Source acceptance, the next checkpoint still must make these inherited requirements concrete rather than implicit:
Every unsafe predecessor/naive case must fail causally, not only assert the final row state. This design approval does not select storage DDL, enum/API spelling, migration, branch, version, cross-SDK exposure, execution, or release. Existing Saga, cancellation, terminal outcomes, and retention remain authoritative until separately reviewed Source and API contracts pass.
22 remaining items
rmcdaniel commentedon Nov 14, 2026
@chrom, main is sufficient for an experimental integration spike with embedded Laravel and sequential Order/Shipment compensation. I would not yet plan a production integration against a released package: the retained-scope and reserved-child APIs are not in published 2.28.1, and the complete StateMachine integration remains unfinished.
The foundation on main provides registration before effects, stable original operation/generation identities, retained obligations after run closure, explicit recovery and child-first verified compensation.
reserveChild()connects participation to genuine native child admission, or to a permanent never-admitted fence; a missing child lookup is not evidence of non-admission. The runnable Order/Shipment example and scope guide are now on main. The example also exposed and fixed a child-await replay issue caused by database JSON member ordering.The latest merged tests cover a forward worker dying after an effect commits but before its result is recorded (#1153), cancellation/termination during recovery routing, and a recovery worker dying after partial compensation (#1154). These passed the backend suites and checks on main. They do not establish your carrier's idempotency or late-outcome behavior.
Please account for these remaining boundaries:
A sanitized contribution would help most with a completed Shipment followed by late Order cancellation; a carrier booking succeeding before acknowledgement is recorded; or cancellation racing with Shipment admission/completion. Expected booking identities, business outcomes and compensation order would make useful regression cases. Please include whether Shipment has further nested children or uses continue-as-new, since those affect the limits above.
There is no committed release date or version yet. The remaining publication milestones are release-candidate/package qualification, installed database-upgrade, pinned-version recovery and rollback checks, then publication and an installed consumer with versioned guides. The shared PHP build-input repair currently blocks part of that qualification. A retained-scope release can precede completion of the full StateMachine layer; the two milestones are separate.
Merge pull request #1198 from durable-workflow/general/1063-compensat…
Merge pull request #1199 from durable-workflow/general/1063-compensat…
Merge pull request #1201 from durable-workflow/general/1063-example-r…
Merge pull request #1213 from durable-workflow/general/1063-compensat…
Rebase the compensation-scope promotion on the reviewed surface
Merge pull request #1216 from durable-workflow/general/1063-compensat…
Merge pull request #1214 from durable-workflow/general/1063-compensat…
Merge pull request #568 from durable-workflow/docs/1063-compensation-…
rmcdaniel commentedon Nov 19, 2026
@chrom, compensation scopes are now a stable API in Workflow 2.33.0. Could you try it against your nested order and shipment processes and tell us whether it covers your case? Your confirmation is the last step before we close this issue.
Install
docs/api-stability.md#compensation-scopesexamples/compensation-scopes(an Order and a Shipment child, with cancellation before and after the order closes)What is stable now
These follow the 2.x compatibility policy from 2.33.0:
DurableScope,RegisteredObligation,Obligation,ScopeIdentity.ScopedOperation,OperationContext,ForwardResult,Resolution,HandlerRegistry,HandlerContractViolation.ScopeAuthorization,ScopeAuthorizationRefused,ScopeRecovery::request()anddescribe()with their result enums, andScopeRetention::release().Recorded compensation histories, the ledger storage and the documented minimum limits are covered too.
Still experimental:
reserveChild(), original root admission, parallel compensation and deeper nesting.How it maps to your proposal
ScopeIdentity(application, namespace, key, generation)names the business execution, independent of run IDs. A request never targets a replacement generation.DurableScope::register()records the obligation before the forward operation runs. Handlers receiveoperationId.OperationContext::$reconcileBeforeForwardand$forwardOutcomeUnconfirmedtell them when an earlier forward attempt may have taken effect without a recorded result.HandlerRegistrykeys, never from serialized closures.ScopeRecovery::request($scope, $requestKey, ...)accepts each request key once. Its receipt is separate from verification.DurableScope::participate()registers a child scope. Sequential compensation waits for the child's verified ledger before compensating earlier parent obligations.ForwardResultisobserved,unknownorrejected.Resolutionisverified,unknownorfailed.describe()reports unresolved obligations and the next action. Delivery, child termination or exhausted retries never count as verified compensation.ScopeRetention::release()retires a verified scope explicitly. Run-history pruning never removes the ledger, and a schema rollback refuses while the ledger holds data.If you tried the earlier experimental releases (2.29 to 2.31)
The API changed before it was promoted:
Resolution::verified()takes only evidence. Passing the old second argument still works with a deprecation notice. Returnunknown()while a late forward effect is possible, orfailed()when compensation definitely cannot complete.ForwardResult::rejected()reports a definite no-effect outcome.ScopeAuthorizationRefusedto refuse an operation permanently. Any other exception from yourScopeAuthorizationis treated as transient and retried.HandlerContractViolationand fails the activity without retrying.describe()result enums may gain cases in a minor release, so handle unknown values with a default branch.What we'd like to hear
Please check what matters for your integration, especially the acceptance criteria above:
If anything is missing or awkward, comment here with the case, and we'll treat it as a follow-up issue. If it covers your needs, a short confirmation is enough, and we'll close this issue.