Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/feature-test-timings.json
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
"tests/Feature/ExceptionLoggingReplayTest.php": 7.228751,
"tests/Feature/ExceptionWorkflowTest.php": 8.416919,
"tests/Feature/ExplicitQueueWorkerTest.php": 4.186096,
"tests/Feature/FailedActivityRedispatchTest.php": 13.0,
"tests/Feature/FailingWorkflowTest.php": 6.256735,
"tests/Feature/HeartbeatWorkflowTest.php": 13.477668,
"tests/Feature/MigrationTest.php": 55.40979,
Expand Down Expand Up @@ -272,6 +273,7 @@
"tests/Feature/ExceptionLoggingReplayTest.php": 8.584815,
"tests/Feature/ExceptionWorkflowTest.php": 4.583109,
"tests/Feature/ExplicitQueueWorkerTest.php": 0.87937,
"tests/Feature/FailedActivityRedispatchTest.php": 12.0,
"tests/Feature/FailingWorkflowTest.php": 3.523026,
"tests/Feature/HeartbeatWorkflowTest.php": 13.476006,
"tests/Feature/MigrationTest.php": 12.426278,
Expand Down
19 changes: 19 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,25 @@
published package. `docs/business-state-machines.md` describes what it
checks. The compensation-scope and state-machine journeys now share
`scripts/ci/upgrade-journey-lib.sh`.
- A v1 activity whose job failed no longer runs a second time when the
workflow replays before the failure is recorded (#1241). The failure is
recorded by a separately queued `Workflow\Exception` job after the activity's
unique lock is released, so a wake in between (a signal, timer or sibling
result) dispatched the activity again and the copy executed it, adding an
exception row and repeating its side effects. The failed job now marks the
activity in the cache before its lock is released. A copy that finds the mark
completes without executing, recording a result or firing activity events, and
the Exception job removes the mark once it has recorded the failure or let it
go. Recorded history, job payloads and replay are unchanged.
- The mark expires after `workflows.activity_failure_mark_seconds`
(`DW_V1_ACTIVITY_FAILURE_MARK_SECONDS`, default 3600). If the Exception job
is lost (a flushed queue, an undecryptable payload), a later wake then runs
the activity again, as before this release, instead of the workflow waiting
forever. Keep it well above the time an Exception job can wait behind
earlier pending work.
- A job that times out, or exhausts its attempts after a worker crash, fails
without running the activity's catch block, so it writes no mark and the
redispatch race stays open on those paths, as before.

## 2.33.0 - 2026-11-19

Expand Down
82 changes: 81 additions & 1 deletion src/Activity.php
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,15 @@

public $key = '';

/**
* @internal Whether another job of this activity failed and its outcome
* is not yet recorded, once read for this handling. Unset typed
* properties are not serialized, so payloads are unchanged.
*/
public bool $awaitingRecordedFailure;

private bool $ownsFailureMark;

private Container $container;

public function __construct(
Expand Down Expand Up @@ -103,7 +112,7 @@
]);
}

public function handle()

Check warning on line 115 in src/Activity.php

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

This method has 4 returns, which is more than the 3 allowed.

See more on https://sonarcloud.io/project/issues?id=durable-workflow_workflow&issues=AaH0qgtnEhGp2gOc0mNx&open=AaH0qgtnEhGp2gOc0mNx&pullRequest=1245
{
if (! method_exists($this, 'execute')) {
throw new BadMethodCallException('Execute method not implemented.');
Expand All @@ -111,19 +120,50 @@

$this->container = App::make(Container::class);

if (self::awaitsRecordedFailure($this)) {
return;
}

if ($this->storedWorkflow->hasLogByIndex($this->index)) {
return;
}

// The Exception job forgets the failure only after recording it, so
// logs loaded with this job's payload may predate that record.
if ($this->storedWorkflow->relationLoaded('logs') && $this->storedWorkflow->logs()
->whereIndex($this->index)
->exists()) {
return;
}

try {
return $this->{'execute'}(...$this->resolveClassMethodDependencies($this->arguments, $this, 'execute'));
$result = $this->{'execute'}(
...$this->resolveClassMethodDependencies($this->arguments, $this, 'execute')
);

if ($this->ownsFailureMark ?? false) {
Cache::forget(self::failureKey($this->storedWorkflow->id, $this->index));
}

return $result;
} catch (\Throwable $throwable) {
$this->storedWorkflow->exceptions()
->create([
'class' => $this::class,
'exception' => Serializer::serialize($throwable),
]);

// Recorded before the job can fail and release its unique lock.
// It expires so that a lost Exception job cannot strand the run.
$uuid = $this->queueJobUuid();
if ($uuid !== null) {
Cache::put(
self::failureKey($this->storedWorkflow->id, $this->index),
$uuid,
self::failureMarkSeconds()
);
}

if ($throwable instanceof NonRetryableExceptionContract) {
$this->fail($throwable);
}
Expand All @@ -132,6 +172,39 @@
}
}

/**
* @internal A failed job's outcome is recorded later by a queued Exception
* job, after this activity's unique lock is gone. A replay in between
* dispatches the activity again; that copy must not run it again. Read
* once per handling, before the activity is reported as started.
*/
public static function awaitsRecordedFailure(self $activity): bool
{
if (! isset($activity->awaitingRecordedFailure)) {
$failedJob = Cache::get(self::failureKey($activity->storedWorkflow->id, $activity->index));
$activity->ownsFailureMark = $failedJob !== null && $failedJob === $activity->queueJobUuid();
$activity->awaitingRecordedFailure = $failedJob !== null && ! $activity->ownsFailureMark;
}

return $activity->awaitingRecordedFailure;
}

/**
* @internal
*/
public static function failureMarkSeconds(): int
{
return max(1, (int) config('workflows.activity_failure_mark_seconds', 3600));
}

/**
* @internal
*/
public static function failureKey(int|string $workflowId, int $index): string
{
return 'laravel-workflow-activity-failure:' . $workflowId . ':' . $index;
}

public function middleware()
{
return [
Expand Down Expand Up @@ -183,4 +256,11 @@
Cache::put($this->key, 1, $this->timeout);
}
}

private function queueJobUuid(): ?string
{
$uuid = $this->job?->uuid();

return is_string($uuid) && $uuid !== '' ? $uuid : null;
}
}
10 changes: 10 additions & 0 deletions src/Exception.php
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,8 @@ public function handle()
return;
}

$released = false;

try {
$workflow = $this->storedWorkflow->toWorkflow();

Expand All @@ -84,13 +86,21 @@ public function handle()
$workflow->next($this->index, $this->now, self::class, $this->exceptionPayload());
} elseif ($probeDecision === self::PROBE_RETRY) {
$this->release(self::LOCK_RETRY_DELAY);
$released = true;
}
}
} catch (TransitionNotFound) {
if ($workflow->running()) {
$this->release();
$released = true;
}
}

// Until here a redispatched copy of the failed activity waits for
// this outcome instead of running again (Activity::handle).
if (! $released) {
Cache::forget(Activity::failureKey($this->storedWorkflow->id, $this->index));
}
} finally {
$lock->release();
}
Expand Down
7 changes: 7 additions & 0 deletions src/Middleware/ActivityMiddleware.php
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
use Illuminate\Support\Str;
use LimitIterator;
use SplFileObject;
use Workflow\Activity;
use Workflow\Events\ActivityCompleted;
use Workflow\Events\ActivityFailed;
use Workflow\Events\ActivityStarted;
Expand All @@ -22,6 +23,12 @@ final class ActivityMiddleware

public function handle($job, $next): void
{
// A copy of an activity whose failed job is not yet recorded neither
// runs nor reports anything; the Exception job records the outcome.
if ($job instanceof Activity && Activity::awaitsRecordedFailure($job)) {
return;
}

$this->job = $job;
$this->uuid = (string) Str::uuid();

Expand Down
9 changes: 9 additions & 0 deletions src/config/workflows.php
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,15 @@
// below this release requires draining the runs that recorded v2 first.
'definition_fingerprint_version' => env('DW_DEFINITION_FINGERPRINT_VERSION', 'v1'),

// Seconds a failed v1 activity stays marked while its queued
// Workflow\Exception job records the failure. A copy of the activity
// dispatched by a replay in that window does not run it again. The mark
// expires so that a lost Exception job (a flushed queue, an undecryptable
// payload) cannot leave the workflow waiting forever; a later wake then
// runs the activity again. A top-level key, so an application whose
// published workflows.php predates it still receives the default.
'activity_failure_mark_seconds' => (int) env('DW_V1_ACTIVITY_FAILURE_MARK_SECONDS', 3600),

'v1' => [
// Existing installations keep their v1 drain. Disable only for a
// fresh v2-only application or after all legacy work is retired.
Expand Down
Loading