Skip to content

Let embedded workflow code signal other workflows durably (.github#223) - #1247

Merged
rmcdaniel merged 7 commits into
mainfrom
general/223-embedded-signal-external
Nov 20, 2026
Merged

rmcdaniel merged 7 commits into
mainfrom
general/223-embedded-signal-external

Conversation

@rmcdaniel

@rmcdaniel rmcdaniel commented Nov 19, 2026 •

Copy link
Copy Markdown
Member

Refs durable-workflow/.github#223 (the embedded PHP slice). Contract: server#1251 (docs/contracts/signal-external-workflow.md), matching sdk-php#438, sdk-python#346 and sdk-rust#294.

What

Embedded workflow code can signal another workflow in its namespace as a durable command:

use function Workflow\V2\signalExternalWorkflow;

$result = signalExternalWorkflow('order-42', 'approve', ['Ada']);   // ExternalSignalResult
$result->delivered();          // or outcome target_not_found / target_closed / rejected, with ->reason
$this->lastChild()?->signalDurably('approve', 'Ada');               // same command, child's current run

Also Workflow::signalExternalWorkflow(); a fourth $runId argument targets one run.

  • History, same shape as native. ExternalSignalRequested (sequence, target_workflow_instance_id, target_workflow_run_id, signal_name, arguments {codec, blob}, payload_codec, request_id) and exactly one ExternalSignalDelivered (target_command_id, target_signal_id, retained) or ExternalSignalFailed (outcome, reason, message, target_command_id) at the command's sequence. It consumes one durable sequence; the next step follows it.
  • Delivery is an outbox (redesigned after review; see the commit "Deliver durable signal-external through an outbox"):
    • The sending workflow task records ExternalSignalRequested and commits, locking only its own run; workflow code waits for the outcome as for an activity result.
    • After the commit, the target admits an ordinary signal (the WorkflowStub signal path: accepted command, signal row, SignalReceived, message task) in its own transaction, keyed workflow-signal:<sender run>:<sequence> and attributed through CommandContext::workflow() (source: workflow, parent_instance_id, parent_run_id, sequence). Then the sender's run is locked on its own to append exactly one outcome and get a workflow task. No transaction holds both workflows' locks.
    • Deadlock, lock wait timeout and lost connection are retried inline and never fail the sender. While a request is pending the sender keeps a delayed workflow task (doubling delay, max 60 s) that replays to the request and delivers it again; delivery is idempotent (the retained receipt answers a repeat with retained: true). A crash between commit and delivery is therefore recovered by ordinary task repair. Only definitive answers are recorded as failures.
    • A request with no outcome now waits on replay (executor, query replay, worker-protocol runner) instead of being malformed history.
  • Outcomes (Server's reasons). target_not_found: instance_not_found (missing, unstarted, collation-folded or other-namespace id), run_not_found. target_closed: run_not_active, historical_run. rejected: unknown_signal, invalid_signal_arguments, structural_limit_exceeded, runtime_reserved_signal, self_target, request_id_conflict. A failed outcome is a returned value.
  • Cancellation cleanup. As on Server (cancellation_cleanup_command_not_available: only activities, timers and side effects during cleanup), the command throws ExternalSignalRefusedException in workflow code and records nothing.
  • Name validation identical to Server's (external_signals.rs + operation_name): valid UTF-8, not empty, ≤255 bytes, no byte <0x20 or 0x7f, not blank (Unicode White_Space, as Rust trim), ≤191 characters; target and run ids non-blank and ≤191 characters. Refused in workflow code with InvalidExternalSignalException before anything is recorded.
  • Reserved prefix. A caller signal or signal-with-start keyed workflow-signal:… is refused before any read or write: embedded throws InvalidCommandRequestIdException (reason() request_id_reserved); the control plane and the embedded signal webhooks return the Server's 422 body (reason, outcome: rejected_request_id_reserved, accepted: false, Server's message). CHANGELOG carries an upgrade note. The reserved-key admission (WorkflowStub::attemptExternalWorkflowSignal) is private and reached only by the delivery.
  • Replay. A changed target, run, name or arguments (compared after a codec round trip) is a HistoryEventShapeMismatchException (external_signal_mismatch), which blocks replay without failing the run. Malformed history (outcome without request, two requests or outcomes, unknown outcome, renamed identity, failure without reason) is external_signal_history_invalid.

Embedded vs native

  • The worker-protocol runner (external PHP workers) replays recorded requests and outcomes but refuses a new command (UnsupportedWorkflowYieldException); embedded does not advertise signal_external_workflow.
  • ChildWorkflowHandle::signal() is unchanged (still direct, not replay-safe); signalDurably() is the durable path.
  • On MySQL, a signal racing a target's continue-as-new can make the target's closing task a deadlock victim (instance→run vs run→instance lock order, the same as for any HTTP signal); that task is retried. The sender is never the casualty.

API stability: experimental

signalExternalWorkflow(), the facade method, ChildWorkflowHandle::signalDurably(), ExternalSignalResult, ExternalSignalOutcome and InvalidExternalSignalException are @experimental (docs/api-stability.md#durable-signal-external). The wire contract is shipped (Server 3.15.0, three SDKs), but these PHP names have never been released, and the embedded runtime still differs from Server in the points above, which a minor release may align. The three history events are frozen now, like every recorded event (added to the frozen table and HistoryEventPayloadContract). No migration.

Tests

  • tests/Feature/V2/V2SignalExternalConcurrencyTest.php (the reviewer's scenarios, with assertions, forked processes on MySQL and PostgreSQL; skipped on SQLite): mutual signalling ×3 rounds, lock wait timeout on the target, target created mid-task, target task finishing mid-send, target completing / continuing as new mid-send, and a worker killed right after the sender's commit (recovered by the delayed task). Every sender completes with no failure and exactly one Requested + one outcome. Local: MySQL 8.4 6/6 (×2 runs), PostgreSQL 17 6/6.
  • tests/Feature/V2/V2SignalExternalWorkflowTest.php (9): delivery + sender/target history + attribution + cold and query replay sending nothing; rolled-back sender task sends nothing and its retry delivers once; seven failed outcomes as values; run-targeted delivery; child handle; invalid name; cleanup refusal; reserved prefix for signal, signal-with-start and control plane, and the Server-matching 422 body on the control plane and webhook. Local: PostgreSQL, MySQL and SQLite all pass.
  • tests/Unit/V2/SignalExternalWorkflowCallTest.php (23).
  • Replay fixtures signal-external-*: delivered cold replay, failed outcomes cold replay, pending request waits, target changed and outcome without request (both with all three replay consumers).
  • verify-feature-shards.php pass with weights; PHPStan (src, tests) and ECS clean.

Re-review follow-ups (273c64b)

  • Bounded retries. Only deadlock / lock wait timeout / lost connection are transient (retried without limit). Other delivery errors count toward workflows.v2.external_signal.max_delivery_failures (DW_V2_EXTERNAL_SIGNAL_MAX_DELIVERY_FAILURES, default 10); at the limit one ExternalSignalFailed (rejected, embedded-only reason delivery_failed, message with the count and last error) is recorded and the sender resumes.
  • One task row. A single delayed task per pending request carries every re-attempt (external_signal_delivery_attempt, _failures, _last_error in its payload); the executor re-readies the same row instead of creating one per try. Reviewer's poison scenario: 2 task rows total, ends delivery_failed after 10 failures.
  • Visible while retrying. Run summary: wait_kind: external_signal, liveness waiting_for_external_signal_delivery or external_signal_delivery_retrying ("after N failed attempts (last error: …); next attempt at …").
  • Terminate race. The outcome append re-checks the sender's terminal state under the sender lock; a terminated sender gets no outcome and keeps no open task. The target may already have the signal (at least once, as with an HTTP signal at that moment). Documented in api-stability and CHANGELOG; tested by terminating between the target admission and the append.
  • Mixed fleet. External PHP workers now refuse a pending request (UnsupportedWorkflowYieldException, "needs an embedded worker") instead of waiting; documented. The pending-request fixture became a feature test since the corpus has no failure type for that refusal.
  • Tests: permanent path (MySQL, PostgreSQL, SQLite), transient path via a 40001 trigger (MySQL, PostgreSQL), terminate race, worker refusal. Local: both signal-external feature files pass on MySQL 8.4 and PostgreSQL 17, the workflow file on SQLite. The reviewer's v2 file on MySQL: poison bounded, concurrent redelivery admits one target command, missing target one instance_not_found, terminate/cancel leave a consistent sender.

Not for merge without an independent review.

🤖 Generated with Claude Code

Durable signal-external for the embedded engine (.github#223), following
the native signal_external_workflow contract (server#1251).

signalExternalWorkflow() (also Workflow::signalExternalWorkflow() and
ChildWorkflowHandle::signalDurably()) records ExternalSignalRequested and
exactly one ExternalSignalDelivered or ExternalSignalFailed at one workflow
sequence, in the same transaction that admits an ordinary signal on the
target. The target command is keyed workflow-signal:<sender run>:<sequence>
and attributed through CommandContext::workflow(). Replay, query replay and
cold replay return the recorded ExternalSignalResult and never send again;
a changed target, run, name or arguments, or malformed resolution history,
is a history shape mismatch.

Missing, closed and refusing targets are recorded outcomes with Server's
reasons (instance_not_found, run_not_found, run_not_active, historical_run,
unknown_signal, self_target, runtime_reserved_signal, ...), not task
failures. Signal names follow Server's rules and are refused in workflow
code before anything is recorded. Callers can no longer use the
workflow-signal: request-id prefix (request_id_reserved).

The authoring API is experimental; the history events are frozen.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@rmcdaniel rmcdaniel added the agent:general Work owned by General label Nov 19, 2026
@rmcdaniel

Copy link
Copy Markdown
Member Author

CI is green at a1e1278. regression-corpus (the official validator against base) passes with the four new fixtures. One run of Literal 1,000-member fan-out (children, mysql) failed its 50 s timing bound (73.9 s scheduling); the rerun passed at 44.1 s. The deterministic width-100 reference made identical query counts in both runs (6270 / 12561 / 912-915), as on other branches, so this change adds no queries there; the 1,000-wide timing varies between runs on other branches too (43.5 s to 66.5 s).

🤖 Generated with Claude Code

rmcdaniel and others added 2 commits November 19, 2026 20:29
Review of the in-transaction delivery found that mutual signals deadlocked
and failed a sender, that a lock timeout or a target continuing as new
failed the sender permanently, and that MySQL's repeatable-read snapshot
inside the sender's task read a stale target. Delivery is now an outbox, as
on the Server and in Temporal:

- The sending task records ExternalSignalRequested and commits, locking
  only its own run, and workflow code waits for the outcome.
- After that commit, the target admits an ordinary signal in its own
  transaction under the workflow-signal:<run>:<sequence> key; the sender's
  run is then locked on its own to append exactly one
  ExternalSignalDelivered or ExternalSignalFailed and get a workflow task.
- Deadlocks, lock timeouts and lost connections are retried and never fail
  the sender. A delayed sender task re-attempts a pending delivery, so a
  crash between the commit and the delivery is recovered by task repair.
- A request without an outcome now waits on replay instead of being
  malformed history.

Also: the command is refused during cleanup after a cancellation request
(cancellation_cleanup_command_not_available, as on Server); reserved
request-id refusals carry the Server's outcome/accepted body; the CHANGELOG
has an upgrade note for the reserved prefix; and the reserved-key admission
is private to the delivery.

The reviewer's concurrent scenarios are tests on MySQL and PostgreSQL:
mutual signalling, lock wait timeout, a target created or finishing its
task mid-send, a target closing or continuing as new, and a crash after the
sender's commit.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
rmcdaniel and others added 3 commits November 20, 2026 01:08
Re-review follow-ups for the delivery outbox:

- Only deadlocks, lock wait timeouts and lost connections are transient
  and retried without limit. Other delivery errors count toward
  workflows.v2.external_signal.max_delivery_failures (default 10); at the
  limit the request is recorded as ExternalSignalFailed (rejected,
  delivery_failed) and the sender resumes.
- One delayed task per pending request carries every re-attempt with its
  attempt and failure counts, instead of a new row per try.
- The run summary reports wait_kind external_signal with liveness
  waiting_for_external_signal_delivery or external_signal_delivery_retrying,
  the attempt count, the last error and the next attempt time.
- A sender terminated while a delivery is admitting its signal records no
  outcome and stays consistent; the target may still receive that signal,
  as with an HTTP signal at that moment. Documented and tested.
- External PHP workers refuse a pending request instead of waiting on a
  delivery they cannot make, so a mixed fleet fails the task visibly.

Tests cover the permanent path (MySQL, PostgreSQL, SQLite), the transient
path (MySQL, PostgreSQL), the terminate race and the worker refusal.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@sonarqubecloud

Copy link
Copy Markdown

@rmcdaniel
rmcdaniel merged commit d9f4574 into main Nov 20, 2026
48 of 49 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent:general Work owned by General

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant