Skip to content

Extend Durable Workflow with composable business state machines, pluggable transition policies, and nested execution #1069

Open

Description

@chrom

Current delivery status

Updated 2026-11-19. The Workflow\V2\StateMachine API is experimental: the transition contract and TransitionRunner first shipped in 2.23.0, and definitions, rules, budgets, child bindings and Workflow::startChild() in 2.29.0. The compensation-scope API it uses is stable since 2.33.0 (#1063), except DurableScope::reserveChild(), which stays experimental. The feature is not declared complete. The original acceptance criteria below remain the completion authority. No in-flight migration or recursive-child support is implied.

Already on main before this update:

  • the experimental transition protocol, pinned rules and explanations, and original-generation child reservations;
  • retained child participation and protected business decisions;
  • shared machine definitions and the recorded Order/Shipment example;
  • profile versioning for new subjects (#1157);
  • configured definition, action and history budgets (Bound configured machines and preserve recovery after budget exhaustion #1158);
  • the provider-backed Order/Payment/Shipment example, covering:
    • the provider journal;
    • reservation-first and payment-first checkout;
    • refund outcomes;
    • authorization and capture after stock or after delivery;
    • bank, COD, advance-and-balance and subscription funding;
    • Cart leases;
    • unit money on the direct-capture checkout.

Merged examples, tests and engine fixes:

Release qualification, done:

Coverage: every PAY, STK, SHP and TECH family in the consumer matrix has native source coverage. Real process-kill coverage exists for checkout, subscriptions, PAY-02/04/05, returns, exchanges and transit incidents. These are synthetic-provider source controls, not real-provider or installed-package acceptance.

Documented open policies:

  • settling the remaining units after a partial return;
  • price-difference and multi-unit exchanges;
  • partial parcel loss and stock deduction for a replacement;
  • reactivation after a late subscription payment;
  • in-transit cancellation still needs a fresh authorized request after inspection, because automatic recovery is one-shot.

Stability decision:

Primary owning public component

Workflow engine

User problem

We need a supported business-state-machine layer built on Durable Workflow's existing execution capabilities.

Our application processes orders across multiple sales-channel and carrier integrations. The meaning of the business states is shared, but each integration can require different allowed transitions, conditions, actions, confirmation evidence, and compensation behavior.

For example, two orders can use the same Order state machine while selecting different integration-specific implementations of a transition. One integration may require an external operation before the transition becomes possible; another may already provide the required evidence and need no action.

A state can also host a nested machine. An Order may remain in DeliveryInProgress while a Shipment machine performs its own transitions. The parent needs the child's verified business outcome and must coordinate cancellation and compensation without losing the child's identity after retries, state changes, or worker restarts.

DW already provides the underlying primitives: workflow execution, activities, durable waits, child workflows, replay, retries, versioning, and Saga compensation. We currently have to assemble their relationship with business states and transition policies ourselves.

This request is to extend DW with a reusable public contract and execution layer for these scenarios, preserving the existing runtime and APIs.

Proposed public contract

Add an optional business-state-machine capability integrated with Workflow V2. Extend existing state-machine support and reuse DW execution, history, waits, retries, and native Saga.

The names below illustrate the requested contract; they are not claims about existing APIs.

  1. Shared machine definition and replaceable transition policies

A machine definition declares:

  • Stable machine identity and definition version.
  • State identifiers and their shared meaning.
  • Transition identifiers and machine-wide invariants.
  • Named slots for state-bound nested machines.

A versioned policy profile supplies:

  • Allowed edges within that machine's state catalogue.
  • Sets of target-state conditions.
  • Conditions permitting execution to start.
  • One or more actions, including nested-machine operations.
  • Required result evidence and compensation behavior.
  • Waiting, failure, and retry-policy references.

Select the profile from trusted application context, such as the instance's integration or configuration. Different instances of the same machine can select different profiles without duplicating the runner or redefining state meanings.

Profiles cannot bypass machine-wide invariants. Configuration references registered rule/action types with validated parameters, rather than arbitrary executable code.

Pin the selected definition, profile, rules, and compensation versions. Deployment or configuration changes must not silently replace them for an active execution. Any supported migration must be explicit and preserve existing compensation obligations.

  1. One transition-step contract for all machine types

Preserve four business operations:

canTransition(TransitionContext $context): bool
canExecute(TransitionContext $context): bool
execute(TransitionContext $context): ExecutionReceipt
compensate(
TransitionContext $context,
ExecutionReference $originalExecution
): void

The contract must not depend on Order, Shipment, a particular integration, or an ORM model.

execute() may perform ordinary actions, start/adopt a nested machine, or durably await its result. Suspension and recovery belong to DW.

  1. Defined execution sequence

Event
-> resolve the instance's pinned policy
-> select a structurally applicable transition
-> [1] evaluate target-state conditions
satisfied:
skip unnecessary actions and proceed to [4]
unsatisfied:
-> evaluate execution-start permission
-> if permitted, establish execution identity and compensation coverage
-> [2] execute prescribed work
-> obtain a fresh recorded fact snapshot
-> [3] evaluate target-state conditions again
-> [4] perform a protected, idempotent business-state commit

Unavailable transitions, prohibited work, missing evidence, and unknown external outcomes must remain distinguishable. A false check must not automatically authorize an action, retry, or compensation.

When execution is unnecessary, preserve existing obligations or explicitly establish adoption responsibility for existing effects.

  1. Pure, explainable rule evaluation

Conditions operate on immutable, recorded facts. Evaluation exposes:

  • Satisfied, unsatisfied, or unknown.
  • Stable reason codes.
  • Evidence references and missing facts.
  • Rule/profile version and fact-snapshot identity.
  • The evaluated condition structure.

The boolean methods bridge this typed evaluation; a mutable shared "last result" cache is not an acceptable result channel.

Fresh database/provider reads occur through recorded execution boundaries. Replay reuses recorded inputs. Guards do not perform I/O, dispatch actions, or wait.

If several transitions match an event, selection must follow a declared deterministic policy or report ambiguity.

  1. State-bound nested machines

Allow any declared parent state to host a nested machine using the same machine and transition contracts.

A durable binding records:

  • Authorized application/client scope.
  • Parent business instance and original owning execution.
  • Binding slot and immutable generation.
  • Hosting parent state.
  • Child machine and business-execution identity.
  • Pinned parent/child policies and required outcome.

Separate starting/adopting the child from awaiting its outcome.

For example, a transition into DeliveryInProgress can confirm the Shipment start/adoption milestone before committing the parent state. A later transition can await the same child's delivery result.

A queued request alone is not confirmed start. Retries reuse the original binding. Observation does not create another child or acquire a second compensation obligation.

Leaving the hosting state does not implicitly terminate the child or erase responsibility. The parent evaluates the child's concrete business outcome; a technically completed child run is not sufficient evidence.

  1. Compensation through native Saga

Compensation is a separate branch, invoked by an explicit business decision.

Identify the original execution and establish compensation coverage before external effects. Compensation must work even when execute() has not returned a receipt.

The original owning step resolves its effects; observer steps do not duplicate its undo. One step may own multiple effects, with explicit operation identity and ordering.

In sequential nested compensation:

  • Request compensation of the exact bound child.
  • Wait for verified resolution of its required obligations.
  • Then continue earlier parent compensations.

compensate(): void succeeds only after required effects are resolved or their absence is authoritatively established. Pending, unknown, refused, and technical-failure outcomes require distinct handling.

Integrate with #1063 for retained compensation scopes and recovery after originating runs close. Preserve terminal history and original business identity.

  1. Cancellation, retries, and external uncertainty

Accepting business cancellation prevents new ordinary forward work and competing completion commits. Already dispatched operations remain tracked and reconciled.

Use existing DW activities, timers, retry facilities, and Saga ordering. Retry the specific safe failed operation rather than restarting an entire transition or compensation sequence.

Provide integration points for operation budgets, provider backoff, and circuit-breaker decisions. Waiting for permission is not successful execution.

Technical cancellation or timeout must not imply business cancellation success. Multi-day business waits require explicit policies independent of bounded technical cleanup.

  1. Persistence and inspection boundaries

Provide a persistence integration contract for:

  • Expected source state and revision.
  • Transition/decision identity.
  • Authority and cancellation checks.
  • Atomic business state, decision result, and publication intent where they share a store.
  • Idempotent recovery after lost commit acknowledgement.

Cross-store execution requires an explicit durable handoff; no distributed atomicity is assumed.

Expose structured inspection data for the current business state, selected profile, blocked transition, reasons, pending work, child binding, and compensation outcome.

Business state remains distinct from workflow-run status. Application-specific states, authorization decisions, provider idempotency, and evidence validation remain application responsibilities.

Initial scope: embedded Laravel, single-state machines, and sequential nested compensation. Preserve existing workflows and avoid a separate scheduler, retry engine, or compensation executor.

Acceptance criteria

  • Two integrations use the same machine and state meanings with different conditions, action sequences, allowed edges, and compensation handlers.
  • Profile changes affect only explicitly selected new executions or an authorized migration; replay and outstanding compensation retain their pinned versions.
  • The same four-method transition contract supports parent and child machines without entity-specific dependencies.
  • Already-satisfied target conditions skip unnecessary actions.
  • Execution-start refusal prevents effects; post-action checks use a fresh recorded snapshot.
  • Undefined/source-ineligible transitions, business prohibition, missing evidence, and unknown outcomes produce distinguishable decisions.
  • Multiple matching transitions follow deterministic selection or return an explicit ambiguity.
  • A transition can execute several actions and compensate partial work when no final receipt exists.
  • Nested machines can be bound to different parent states through configuration without special-casing Order or Shipment.
  • Starting/adopting and awaiting a child are separate operations using the same immutable binding generation.
  • Worker restart, duplicate events, and lost start acknowledgements do not create duplicate logical children or compensation owners.
  • Cancellation while awaiting a child resolves required child compensation before earlier parent compensation.
  • Closed-child and closed-parent recovery use the retained-scope guarantees of Support durable Saga compensation scopes across nested workflow lifecycles #1063; technical termination alone never satisfies business completion.
  • Lost, delayed, or uncertain external results are reconciled before unsafe retries or successful compensation acknowledgement.
  • Failed compensation preserves completed evidence and unresolved obligations; exhausted retries cannot produce a false business Cancelled state.
  • Cancellation-versus-completion races and lost commit acknowledgements preserve one authoritative business decision.
  • Replay does not perform unrecorded provider calls, mutable configuration reads, or live guard I/O.
  • Cross-client binding substitution and stale-generation requests are rejected before protected access or effects.
  • Invalid definitions, incompatible handler versions, cyclic child bindings, and configured resource limits produce explicit outcomes.
  • Diagnostics explain blocked transitions and correlate business execution, child work, and compensation.
  • Automated crash/race tests and a runnable two-integration example demonstrate the contract using existing DW execution facilities.
  • Existing workflow, Saga, and cancellation APIs remain backward compatible.

Supporting public evidence

The requested capability builds on existing DW primitives:

  1. Child Workflows documents durable parent/child execution, stable invocation identity, child handles, and outcome propagation.

  2. Condition Waits documents replay-safe waiting on recorded workflow state and requires waits to execute in workflow context.

  3. Versioning documents compatibility mechanisms for long-running executions. The proposed machine/profile version contract applies these guarantees to configurable business behavior.

  4. Sagas documents native compensation registration and ordering. This feature should compose business transitions with that existing mechanism.

  5. Cooperative cancellation documents bounded cleanup and operation policies. Business-state cancellation must preserve the distinction between technical run control and verified business resolution.

  6. Issue Support durable Saga compensation scopes across nested workflow lifecycles #1063 covers retained compensation obligations and closed-run recovery. This feature should consume that contract rather than introduce a competing compensation subsystem.

These sources establish the existing execution foundation. The requested addition is a supported, composable business-state-machine API connecting those capabilities.

Dependencies and affected repositories

Primary repository:

  • durable-workflow/workflow
    Extend existing state-machine support with the public machine/profile/transition contracts, V2 execution integration, nested bindings, persistence boundaries, inspection data, and regression tests.

Related dependency:

Documentation:

  • durable-workflow/durable-workflow.github.io
    Document authoring contracts, profile selection/versioning, nested-machine behavior, compensation, persistence guarantees, and runnable examples.

Optional presentation follow-up:

  • durable-workflow/waterline
    Display business states, blocked-transition reasons, selected profiles, nested bindings, and compensation progress using the core inspection contract.

Initial compatibility boundary:

  • Embedded Laravel and Workflow V2.
  • Existing workflows remain usable without adopting the new state-machine layer.
  • No mandatory Symfony Workflow dependency. Existing graph libraries may integrate through a defined boundary.
  • Service-mode and cross-SDK support may follow separately.
  • No application-specific Order/Shipment models, integration names, or business status catalogue belong in DW core.

Public intake checks

  • I searched open and closed GitHub issues for an existing request.
  • This request contains only public product contracts and public-safe rationale.

Activity

  1. added
    authority:githubGitHub is the authoritative lifecycle record for this work
    kind:featureA public product capability or experience is requested
    priority:untriagedMaintainers have not assigned a priority
    status:triageAwaiting maintainer classification
    on Nov 8, 2026
  2. added
    intake:approvedCurrent issue title and body revision is approved for authority intake
    on Nov 8, 2026
  3. rmcdaniel commented on Nov 8, 2026

    @rmcdaniel
    Member

    Intake approved and reviewed. Maintainer triage: P2 feature, ready for contract design; owner General. This is separate from the current release work. The proposed API names remain proposals until the public contract is reviewed.

    The published 2.22.0 source provides graph-edge checking in StateMachine and model-state configuration in StateConfig. The existing state-machine feature test exercises WorkflowStub. Those surfaces do not establish the additional pinned-policy, business-evidence, nested-binding and protected-commit guarantees requested here. The closed state-machine work (#333/#334) therefore does not resolve this broader request.

    The first deliverable is a reviewable contract and execution protocol for embedded Laravel/V2, single-state machines and sequential nested compensation, with a two-integration worked example and an acceptance-to-test map. It will settle these boundaries before implementation:

    1. Definition/profile/handler version pinning, immutable recorded fact snapshots, typed satisfied/unsatisfied/unknown evaluations and deterministic transition selection. The four boolean/execution methods need an explicit connection to typed evaluation results without a mutable last-result channel.
    2. Stable business decision, operation and binding-generation identities; registration of compensation responsibility before effects; separate start/adopt and observe/await operations. A completed child run alone is not a verified business outcome.
    3. A persistence adapter for expected state/revision, authority/cancellation, idempotent commit acknowledgement and publication intent. Shared-store atomicity and cross-store durable handoff must be described separately.
    4. Cancellation, partial external outcomes and compensation recovery through existing activities, waits, retries and Saga ordering. Unknown or pending work must not become a successful business commit or cancellation.

    Dependency: #1063 remains open. Retained obligations, original ownership and closed-child/closed-parent recovery must consume that contract and cannot be accepted here before their integration is verified. Definition/rule/adapter design can proceed without claiming those recovery guarantees already exist.

    The implementation plan will preserve existing state-machine, workflow, Saga and cancellation APIs, avoid application-specific entities and a second scheduler/retry/compensation engine, and keep service-mode/cross-SDK and optional Waterline presentation outside the initial acceptance. Public documentation follows the reviewed core contract. No version, delivery date or new runtime behavior is committed by intake approval.

    Next action — General: produce that contract/sequence and crash-race test plan after the active release qualification work, then open a draft implementation PR for the agreed first slice. This issue remains the acceptance record; its checkboxes stay unchecked until corresponding evidence exists.

  4. added
    priority:P2Normal-priority product work
    status:readyReady for implementation
    and removed
    priority:untriagedMaintainers have not assigned a priority
    status:triageAwaiting maintainer classification
    on Nov 8, 2026
  5. chrom commented on Nov 9, 2026

    @chrom
    ContributorAuthor

    @rmcdaniel pls check MR #1070

  6. rmcdaniel commented on Nov 9, 2026

    @rmcdaniel
    Member

    Intake approved by the operator for the first slice, PR #1070. General reviewed it: the design is sound, and I pushed boundary/shape-validation/experimental-marker fixes so the full matrix can run. This slice doesn't complete the proposal: state-bound child bindings, durable compensation scopes (#1063) and the two-integration example remain, as the PR's design notes say.

  7. rmcdaniel commented on Nov 9, 2026

    @rmcdaniel
    Member

    Main after merge 0673e4dc: all 18 test jobs passed in push run 42670327804 (MySQL ×4, PostgreSQL ×4, MariaDB, coverage, coverage-unit, both Laravel embedded-upgrade cells, quality, regression corpus, tracing). The final aggregate build and Sonar jobs were never created, because GitHub has an ongoing major Actions/Checks incident (since 18:24 UTC). A fresh main run will be dispatched once Actions recovers.

  8. rmcdaniel commented on Nov 9, 2026

    @rmcdaniel
    Member

    Main 0673e4dc is green: run 42676776260 attempt 2 succeeded, including the aggregate build and the Sonar quality gate. Attempt 1 ran right after GitHub's Actions incident and had two time-limit stops (MySQL shard 0, PostgreSQL shard 2 at 96%) and one timing-sensitive watchdog test; all of them pass on rerun, and none touch the StateMachine code.

  9. 42 remaining items

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent:generalWork owned by Generalauthority:githubGitHub is the authoritative lifecycle record for this workintake:approvedCurrent issue title and body revision is approved for authority intakekind:featureA public product capability or experience is requestedpriority:P0Immediate release or data-integrity riskstatus:in-progressApproved work is actively being implemented or validated

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions