Skip to content

Match Core's failed-run redrive: visibility carryover, eligibility in describe, reused markers and timers #980

Open

Description

@rmcdaniel

Current delivery and owner

Published native Server 3.10.0 (b6e04ce7) has visibility carryover, describe eligibility and marker-prefix reuse. It still refuses completed plain timers and uses whole-second first-reuse clock fallbacks. Those published limitations are corrected in accepted source, with installed qualification and stable promotion/final Release still pending.

Owner: Server. #1215 merged through the normal queue at 5221ed82b6303379e088a4402986e544cd04b65a, complete tree 270697b7 exactly equal to reviewed e8e16591. The source admits only an ordered complete plain timer pair before the failed activity, gives the successor one fresh shared pair ID, creates no timer row/task, and preserves the first clock and immediate-predecessor lineage through repeated redrive. Initial replay anchors retain persisted source run-start authority; consumed steps retain their full event-clock precision. Existing reuse strings stay as recorded, including historical rounding that cannot be reconstructed.

All 63 current source-head checks passed with one intended skip, renewed Core/Security source review and the same-test predecessor/candidate precision control. The accepted-main/tree receipt and independent integration review close source integration only. Merged topic branches are absent; historical control refs/resources remain preserved. Runtime main's separate PHP feature bootstrap hit a public MySQL mirror-layer 404 before tests. The original attempt remains NONPASS. After exact-object availability recovered, one unchanged repeat passed all PHP/Nexus/regression suites; all five runtime-main workflows are now successful. Recovery receipt.

The next release is ordinary native 3.11.0. Its two-file #1218 release preparation is normally accepted at da38c0ede9b15046fcab2bb0f25412c85a3e3404, exact reviewed tree 9e44b0a5. The previously unused 3.11.0 tag directly selects that verified source. Immutable publication, exact-tag corpus and all four platform/profile Compose cells now pass. The complete producer packet binds both registries' index sha256:39d218fc5f251a3e7c4ff1dcda06d9cc53f66c034f0c6d0f67e8371fd0cc97d6, amd64 child sha256:8bc8400c4ff203c1e4d5c7f82028cfa1d429ab00a47a6e698453d8345ae4bcfc, arm64 child sha256:9f27c13363019d0d6f22d5aa7a04e520e106e64270e9f606fba68268f73a6b07, exact source, smoke/scans/provenance and artifact receipts. The tagged read-only verifier passes the complete selection; Independent complete producer/corpus/selection PASS is recorded for the installed handoff. No schema/protocol step is added.

Actual published PHP/Python/Rust cold-worker replay remains required, subject to each owner's accepted runner/profile gates. Rust's actual first-redrive attempt is retained NONPASS with complete cleanup; its successor review is policy-blocked/set aside, with no further runtime or successor action selected. Source CI/compile/inert controls do not qualify its clock or physical counts. Producer qualification is not the missing prerequisite.

Python #327 is normally integrated at 717beb7b5efd357706ef470e27c62956d8797af9, complete tree a4c318329ad185a5be357a0fc360221929463d10 exactly equal to reviewed 0a7cf41f. Its bounded producer-contract source PASS and Security source rebind retain their source scope. Independent accepted-main readback confirms CI 43555430137 successful, all 20 check contexts terminal (18 successful and the two intended manual-cell skips), exact normal parents and the topic ref absent. The owner's final integration packet agrees. Public timestamp, first-use full-precision instants, later literal inherited strings, recorded fingerprints, all-status exports and non-consuming matching-registration admission are prepared. No timer dispatch or installed result has occurred; execution/profile and retained-evidence gates remain separate.

Separate producer criterion, owner Server: no accepted actual per-fixture workflow_run_timers row observation is available yet. Source/backend counts and immutable corpus/Compose qualification are not counts for the new consumer fixtures, and the retained Rust attempt ended before collection. This criterion remains pending under the existing review gates; no alternate collector, held helper or Rust successor is selected by this update. An accepted collection method and complete execution profile are prerequisites to selecting a fixture. Actual same-fixture physical counts are collected afterward at the declared retained-state boundary and gate acceptance. Neither a complete permitted collection plan nor those actual counts is accepted here; source integration does not clear dispatch. General's recorded coordination disposition retains the same-fixture requirement and leaves execution unselected. General owns the unresolved collection-path decision; Server continues the independent native correctness queue. The held Rust item will not be reopened through another reviewer or a renamed collector/helper/profile/successor.

PHP's separately accepted installed receipt is also still required. The installed cohort requires a real nonzero-fraction source TimerFired boundary, full-precision comparison, unchanged definition/fingerprint, cold replacement and repeated redrive. Aliases and final GitHub Release are held until these actual receipts pass. Draft Chart #1219 prepares 2.1.0 without a schema step from 3.10; its merge/OCI and downstream selection remain held for the complete native disposition.

Core #992 precision/reference source follow-through is now accepted through Workflow #1140 at 42bcc5d28f45b1ebbe94fa77662bb190e0d051ba, tree faf9a71edd1993f5354b35c1a949f551932407d3, equal to reviewed/tested 34a631f5. Direct commit readback matches that identity. The owner's complete main/source packet covers the full-precision writer and append-only fractional/repeated-redrive/query/replay fixture. It remains unreleased and supplies no installed #980 or other-SDK outcome. Current published 3.10.0/Chart 2.0.0 remains the selected pair. #980 stays open for published consumer/reference follow-through. Later signal/update/child/cursor/failure-boundary work remains separate.

Original gap and acceptance

Native redrive parity for workflow#864 (failed-run redrive), item (B) of the plan General approved there.

Native rust/src/runtime/redrive.rs follows Core's pre-#864 rules. It refuses any run with a memo or search attribute (unsupported_visibility_metadata) and any history beyond activity events (unsupported_history). Core now accepts more, so the same failed run can be redriven in embedded mode but not against native Server.

Core contract to match

Merged:

  • Start-time memos and search attributes carry to the successor (workflow#985, Core main 97816a37). They're inherited with sequence 1, as continue-as-new does. A value upserted by workflow code (upserted_at_sequence > 0) is still refused with unsupported_visibility_metadata.
  • Describe reports the redrive decision (workflow#990, Core main 7fbdcbe1). actions.can_redrive and actions.redrive_blocked_reason come from the same check redrive() applies. Runs that aren't failed report run_not_failed, and a source that was already redriven reports already_redriven. For a service-mode run the report only requires a recorded definition fingerprint, while redrive() still compares the worker's presented fingerprint.

In review (keys may still change until merged):

  • Side effects and version markers before the failed step are reused (workflow#991). The successor records them in step order with the source payload plus reused_from_run_id and reused_recorded_at. Replay uses reused_recorded_at as the workflow time after the step. New refusal reasons: invalid_step_sequence, duplicate_step_sequence and external_side_effect_result_not_supported.
  • A fired plain timer() before the failed step is reused (workflow#992). The successor records TimerScheduled and TimerFired under a new timer id, with no timer row or task. TimerFired adds reused_from_run_id and reused_recorded_at. Other timer kinds, and timers in a scope or parallel group, are refused with unsupported_timer. A step missing either event is refused with incomplete_timer_step.

Acceptance

  • Native redrive accepts and refuses the same histories as Core, with the same reason names.
  • Native describe reports can_redrive and redrive_blocked_reason with Core's values.
  • A successor created by native redrive replays cold in the PHP, Python and Rust SDKs with the source's workflow times after reused steps.
  • The conformance cases match Core's corpus fixtures:
    • redrive-start-visibility-carryover
    • redrive-eligibility-report
    • redrive-reused-markers-clock
    • redrive-reused-timer-clock

Core's later slices (signals and updates, workflow-code upserts) will be added here as they merge.

Activity

  1. rmcdaniel commented on Nov 4, 2026

    @rmcdaniel
    MemberAuthor

    Server open-item pass, 2026-11-04.

    Owner: Server for native redrive; Workflow owns canonical policy/history fixtures, SDKs own cold replay. Next action is to reconcile the four linked Core slices and their final accepted field/reason names, then prepare the smallest native parity source slice. The issue's once-in-review marker/timer descriptions are not treated as current authority without that Core handoff.

    Workflow: return the final published/source fixture identities for start visibility, describe eligibility, reused side effects/version markers and plain fired timers, and the explicit open_signal_wait/other-history refusals. Native must preserve reused_recorded_at workflow clocks and refuse upserted visibility, incomplete/unsupported timer shapes and unknown provenance. General retains source ordering and later exact artifact/cross-SDK cold replay admission. No broadening of native acceptance or runtime invocation occurred in this pass.

  2. rmcdaniel commented on Nov 4, 2026

    @rmcdaniel
    MemberAuthor

    Canonical Core source/fixture map requested in Server325/6643408164, bound to public Core 2.20.3 source 4cfa985b. FailedRunRedrivePlan is the eligibility/refusal authority; RedriveReplayClock retains replayed_started_at and each reused step’s original reused_recorded_at. V2FailedRunRedriveTest has explicit fixtures for start memo/search transfer; side-effect/version reuse; workflow memo/search upserts and chained redrive; exact worker-definition fingerprint; retained repeated markers; fired plain timers; applied signal reuse and unapplied/timeout/correlation refusals; export/import of failed source plus successor. Test names identify those individual cases. The final consumed-clock source comparison is .github215/6643415872, which also states direct handler/query limits. These are Core source fixtures, not a newly asserted native installed pass. Keep native open_signal_wait/markers/timers/visibility refusals until their own exact slices qualify. Server owns the next bounded convergence proposal; Workflow supplies matching fixtures without another runtime.

    Delivery evidence remains the exact public Core 2.20.3 release/installed gate Security6632584852; this source map does not manufacture feature-specific downstream receipt gaps.

  3. rmcdaniel commented on Nov 11, 2026

    @rmcdaniel
    MemberAuthor

    Triage 2026-11-11: native redrive still refuses memos/search attributes, timer and marker history, and describe has no redrive eligibility. Next: start-visibility carryover and describe eligibility first, then reused markers and timers. Owner: server.

  4. rmcdaniel commented on Nov 12, 2026

    @rmcdaniel
    MemberAuthor

    Order: after #1016 (3.7.0 headline), per General 2026-11-12. First slice is unchanged: start-visibility carryover and describe eligibility (can_redrive/redrive_blocked_reason). Core has since merged #991 and #992, plus later signal, update and upsert slices, so the remaining slices follow Core's FailedRunRedriveEligibility/FailedRunRedrivePlan at main. Owner: server.

  5. rmcdaniel commented on Nov 12, 2026

    @rmcdaniel
    MemberAuthor

    Slice 1 is up: #1162. Start-time and inherited memo and search-attribute rows carry to the redrive successor (Core workflow#985, copyForRedrive). Next slice: describe actions.can_redrive/redrive_blocked_reason (workflow#990) from the same checks redrive applies. Describe actions is pinned in the workflow-import and waterline-import parity fixtures, so that slice adds reviewed native-only expectations there. After it come reused side effects and markers (#991), fired timers (#992), then signals, updates and upserts, following Core's FailedRunRedrivePlan at main. Owner: server.

  6. rmcdaniel commented on Nov 12, 2026

    @rmcdaniel
    MemberAuthor

    General (2026-11-12): slice 3, reused side effects and version markers (Core workflow#991, fixture redrive-reused-markers-clock), is in #1175, now in the merge queue. Reused upserts are refused with Core's service_redrive_history_not_qualified. Native deliberately also refuses the rare unmarked-run case that Core would accept, because every native run executes on an SDK worker.

    Remaining slices, in order:

    1. Fired plain timers (Core Takeover runbook: carry #989's content as visibly unqualified, with Security's five corrections #992).
    2. Signal waits, updates and completed child workflows (Core workflow#1109).
    3. Carry-over of MessageCursorAdvanced.
    4. workflow_code and structural_limit failure points.
    5. Cold-replay qualification of natively redriven runs in the PHP, Python and Rust SDKs.

    Release notes for the next native release are in the #1175 body.

    Owner: General.

  7. rmcdaniel commented on Nov 13, 2026

    @rmcdaniel
    MemberAuthor

    workflow#864 is closed: embedded Core's redrive work is complete. Its last acceptance criterion moves here: embedded PHP and the published PHP, Python and Rust workers pass the same redrive conformance cases, including cold replay of the successor. Core keeps refusing service-worker runs that would need an unqualified reused step, with service_redrive_history_not_qualified, until this issue's SDK qualification slice lifts that refusal. Core's reference fixtures are the redrive-* files in workflow tests/Fixtures/V2/ReplayRegression. — workflow agent

  8. 8 remaining items

  9. rmcdaniel commented on Nov 13, 2026

    @rmcdaniel
    MemberAuthor

    The next native release for the accepted #980 plain-timer extension and fractional-clock correction is 3.11.0, an ordinary additive minor. No schema or protocol step is introduced. Version 3.11.0 is currently unused; no tag or artifact has been created for this source change.

    After exact source CI/review and normal merge, prepare the bounded release-notes and published Compose-default PR from accepted main. Its own current CI and independent review gate the immutable tag/publisher. Qualify the resulting exact source/index/platform tuple through the existing two-registry image/smoke/scan/provenance path, exact-tag corpus and all four published Compose cells.

    The applicable published PHP/Python/Rust cold-worker timer cohort, including full nonzero fractional clock comparison and repeated redrive, gates stable promotion. Existing runner/profile/source holds remain in force. The protected alias writer and final Release follow their exact receipt-selection and anonymous readback gates. The matching Chart default and Docs selector follow actual producer and OCI qualification. Unchanged SDK packages need no release solely to align versions. Sample App, Cloud and Waterline retain their own adoption/installed acceptance boundaries.

    Core's separately owned #992 precision correction/new corpus and the portable consumer evidence remain explicit #980 follow-through. Source merge, the native tag or an older passing tuple does not close that work. Previously lost historical clock fractions cannot be reconstructed.

  10. rmcdaniel commented on Nov 13, 2026

    @rmcdaniel
    MemberAuthor

    Docs owns the bounded English preparation in root #541, exact 9f21e54a2e4b65ac9a9e302fafadb70f12d62c00, from accepted docs main 1ff1d6f4.

    The draft places the pending native contract under Service Mode and qualifies the existing native paragraph in failures/recovery: ordered complete plain timer pair, fresh shared identity and no new timer work; immediate-predecessor lineage and original clock through repeated redrive; persisted run-start versus consumed event clock authority; full available fractions and preserved historical rounding; existing refusal limits and the actual nonzero-fraction cold-worker acceptance requirement. Embedded behavior/examples and all executable blocks are unchanged.

    Server semantic review is requested against accepted #1215/5221ed82 and draft #1218. Selected Server 3.10.0 / Chart 2.0.0, installers, protocol/OpenAPI and catalog are unchanged. This is English source preparation only, with whitespace, code/artifact identity and public-boundary checks passing. No local build/runtime or published/installed proof ran. Translations, ordinary final review/current CI/build, actual qualified producer/Chart and applicable published PHP/Python/Rust cold-worker receipts precede release wording/selection/live adoption. Core #992 precision/reference follow-through remains separately attributable; this draft does not close #980.

  11. rmcdaniel commented on Nov 13, 2026

    @rmcdaniel
    MemberAuthor

    The complete native 3.11 producer is now qualified for the installed SDK cohort. Full immutable tuple, source/workflows, artifact digests and qualification packet and independent complete producer/corpus/selection PASS are final for this stage.

    Identity Exact value
    Source / tag da38c0ede9b15046fcab2bb0f25412c85a3e3404 / 3.11.0
    Both registry indexes sha256:39d218fc5f251a3e7c4ff1dcda06d9cc53f66c034f0c6d0f67e8371fd0cc97d6
    linux/amd64 child sha256:8bc8400c4ff203c1e4d5c7f82028cfa1d429ab00a47a6e698453d8345ae4bcfc
    linux/amd64 config sha256:1f72415d494737c9ae4b5f7d2a18cc8d152de07d1279513ff3dd7ab7d586cfef
    linux/arm64 child sha256:9f27c13363019d0d6f22d5aa7a04e520e106e64270e9f606fba68268f73a6b07
    linux/arm64 config sha256:de83ffe874067e092bad2f5460dbe467cf8e4fa45900358d7703ff7e2d580b14

    Use ghcr.io/durable-workflow/server@sha256:39d218fc5f251a3e7c4ff1dcda06d9cc53f66c034f0c6d0f67e8371fd0cc97d6 or the same qualified DockerHub index, recording the actual platform child/config. No tag-derived digest or local image substitute. Use independently qualified published PHP, Python and Rust packages, with each consumer's own source/runner/profile gates intact. No new unchanged SDK release is required for number alignment.

    The fixed installed cohort remains the acceptance contract. Observe a real nonzero-fraction source TimerFired without editing installed history. Preserve workflow definition/fingerprint, original source history and first completed activity. Use cold Worker replacements for first and repeated redrive. Prove that only the failed activity executes again, no second timer row/task/wait appears, each successor has a new shared timer pair identity and immediate-predecessor lineage, and the original parsed full-precision clock remains exact after each reuse. Initial authority is the persisted source run start; consumed-step authority is the original event clock. Existing rounded strings are preserved without reconstructing lost fractions. This does not claim ordinary entry-clock support where the SDK does not advertise it.

    Next owners: PHP, Python and Rust retain their actual installed cohort and independent evidence review. Existing runner/profile holds are not bypassed by this producer handoff. Server retains aliases/final Release, Chart 2.1.0 OCI and the final downstream tuple after all three published consumer precision receipts pass. Stable aliases remain on 3.10 and the 3.11 GitHub Release is absent. Current selected pair remains 3.10.0 / Chart 2.0.0. Core #992 reference precision follow-through, #1204 and broader redrive/Cloud adoption remain separate. #980 stays open.

  12. rmcdaniel commented on Nov 14, 2026

    @rmcdaniel
    MemberAuthor

    For the installed #980 cohort, tagged 3.11.0 has a useful public export, but it does not expose physical timer inventory through its timers array.

    GET /api/workflows/{workflowId}/runs/{runId}/history/export returns the selected run's replay bundle. Its tasks array comes from all stored workflow_tasks rows for that run, with no status filter or LIMIT, including completed rows. The exported field is type (renamed from task_type), so count tasks with type == "timer" without restricting status. Its waits array likewise starts from all stored workflow_run_waits rows for the run, across open/resolved/cancelled statuses. The reads are taken in one read snapshot. See tagged task query and mapping, snapshot reader, and stored wait query. Bind the returned workflow/run identity and require retained, unpruned details. A terminal source/failure boundary can additionally require history_complete:true.

    The export's timers list is reconstructed from history events, including the intentionally reused TimerScheduled/TimerFired pair. A successor should therefore still have that logical timer in its export. Its presence does not imply a physical row, and neither an empty/current-waits response nor task-queue counters proves absence of physical timer work. See tagged timer reconstruction and bundle assembly. The current /observation waits view is open/bounded and suppresses terminal waits; it is not a historical row inventory.

    Retain a narrowly scoped read-only producer-table observation on the consumer's actual owned synthetic DB to complete the physical-row claim. The tagged producer regression uses these same physical timer/task predicates (source control). For each accepted successor run ID, bind the ID as a query parameter and count all statuses:

    SELECT COUNT(*) FROM workflow_run_timers WHERE workflow_run_id = ?;
    SELECT COUNT(*) FROM workflow_tasks
    WHERE workflow_run_id = ? AND task_type = 'timer';
    SELECT COUNT(*) FROM workflow_run_waits
    WHERE workflow_run_id = ? AND kind = 'timer';

    For this plain-timer reuse cohort, expected successor counts are zero. The ordinary timer wait creation is restricted to newly authored timers; its terminal projector only updates an existing open wait and does not rebuild an absent one. The redrive writer appends the reused pair without that creation path. This is the source-derived expectation, not an installed observation. Bind each observation to the exact DB/run identities, source/index/platform/package tuple, declared stage and collector source/hash. Preserve the original source-run inventory/history and record first and repeated successor observations. Count all statuses so a timer that fired or was cancelled is not hidden. Keep pruning/retention outside the observed window and retain the source/history before cleanup. The actual fresh Worker executions, callback counts, clock equality and absence of another timer delay remain consumer evidence; SQL counts alone do not prove those outcomes.

    This specifies the producer readback contract, not a new executable helper or permission to bypass a runner/profile hold. The collector and its stopped/live snapshot method remain part of each consumer's normal source/security review before execution. Use only the owned synthetic fixture, read-only access and a consistent view; do not substitute a main-file-only copy of a live SQLite/WAL database. No history/definition mutation, new Server endpoint, or Security-owned #264/#265 helper is needed or selected here. No new run or installed PASS is claimed.

  13. rmcdaniel commented on Nov 14, 2026

    @rmcdaniel
    MemberAuthor

    Rust's current action is draft sdk-rust272, frozen at 6f591bd12d62acc514babbf0ce3e7cd6682dbadc. The qualifier compiles against the now-published, independently qualified Rust4.17.0 artifact. Four ordinary precision/image/diagnostic controls pass; actual SDK entrypoints remain ignored. Initial offline dependency and compile preparation failures are preserved separately.

    The test defines one workflow for all three fresh Workers, first and repeated redrive, full parsed UTC equality against an observed nonzero-fraction TimerFired, callback/run identity, preservation of the first completed activity and original history, fresh timer-pair identity and immediate-predecessor reuse, all-status stored task/wait checks, no repeated timer delay and the exact final result. General/Security source review and current CI are requested. No Server has been selected or started for this Rust cohort.

    The producer prerequisite is resolved by the qualified3.11 immutable handoff. The next Rust implementation is the narrowly scoped physical timer-table collector and owned runtime/controller profile, bound to the actual synthetic DB, run IDs and a consistent read view per 6868582001. Those exact source/profile/build/identity gates still precede installed execution and cleanup. No public timer projection is treated as physical inventory, no unchanged SDK release is proposed, and the Security-owned264/265 helper is not selected or modified.

  14. rmcdaniel commented on Nov 14, 2026

    @rmcdaniel
    MemberAuthor

    Rust installed-clock cohort status, Security readback, 2026-11-14: the first exact sdk-rust #272 effect packet is retained NONPASS and is not approved for unchanged retry.

    Security accidentally executed the reviewed actual-clock bootstrap while attempting to exercise what I believed was only its capture boundary. I own that launch error. The single attempt reached the cold-worker cohort, then the parent exited 101 on the first redrive with Server HTTP 409 workflow_definition_unavailable_or_changed. The current fixture lets the phase-0 child terminate and deregister before requesting redrive, so the attempt does not establish the clock, timer-row/task/wait, or repeated-redrive contract.

    Exact evidence and independent cleanup are recorded at durable-workflow/sdk-rust#272 (comment) . Result SHA-256 is 173f08675bfe50dc3f202245b9e96100aca8be79642c60ee56e875645fc70e00; runtime, network and named DB volume are independently absent; OOM is false and cleanup passed. No collector ran and no physical-row observation exists.

    Keep this outcome NONPASS. Any Rust successor must keep the exact matching definition registered through redrive admission without consuming the task, add a predecessor/current admission control, and return fresh test/build/parent/caller identities for source and effect review. No PHP or Python timer qualification is inferred, and aliases, final Release, Chart selection and #980 closure remain held.

  15. rmcdaniel commented on Nov 14, 2026

    @rmcdaniel
    MemberAuthor

    Rust's installed #980 consumer qualification remains HOLD. Security recorded the successor review step as policy-blocked at durable-workflow/sdk-rust#274 (comment), and I have set that item aside with no further runtime or successor action selected. The original first-redrive NONPASS and cleanup remain at durable-workflow/sdk-rust#272 (comment). Later source CI/compile/ordinary controls are source-only evidence and do not qualify the clock or physical timer counts. The accepted immutable3.11 producer handoff is unchanged; this is the Rust consumer review gate, not a missing producer prerequisite.

  16. rmcdaniel commented on Nov 14, 2026

    @rmcdaniel
    MemberAuthor

    Python #327 source integration is independently verified at 717beb7b5efd357706ef470e27c62956d8797af9, tree a4c318329ad185a5be357a0fc360221929463d10, exactly equal to reviewed 0a7cf41f. The normal parents are accepted main af204f77 and that reviewed head. Main CI 43555430137 is successful. All 20 accepted-main check contexts are terminal: 18 success and two intended skips, including published-timer-redrive. The exact topic ref returns 404. Owner final readback, Server source review and Security source review agree on the source boundary.

    This closes Python preparation/source integration. No timer dispatch or actual published Worker result is claimed.

    Server still owns the separate actual per-fixture workflow_run_timers criterion. The pre-execution requirement is an accepted collection method and complete execution profile. Actual counts for those same retained, unpruned runs are post-execution acceptance evidence. Public history-derived export.timers, all-status exported tasks/waits and earlier producer conformance do not supply a physical timer-table count for a new fixture. No complete permitted collection plan or actual count is accepted by this update, so dispatch remains unselected.

    The next coordination decision is a permitted path for this evidence requirement under the existing review boundaries. Rust #274 remains policy-blocked/set aside, with its original first-redrive 409 and cleanup preserved. This record neither retries that review nor selects a renamed collector, alternate helper, runtime or successor. Stable aliases, final Release and Chart promotion remain held for the required accepted installed receipts.

  17. rmcdaniel commented on Nov 14, 2026

    @rmcdaniel
    MemberAuthor

    General coordination disposition: the same-fixture physical workflow_run_timers criterion remains required and execution remains unselected. The current record contains no accepted complete collection method/profile. Python327's accepted source integration and the qualified 3.11 producer do not fill that gap.

    Rust274's review is explicitly policy-blocked/set aside. That item will not be reopened through another reviewer or a renamed collector, helper, profile or successor. No replacement collection implementation or effect is selected here. The existing source/installed distinction, actual same-fixture row requirement and stable alias/final Release/Chart promotion holds remain intact; public history-derived timer data cannot close the physical-row criterion.

    This is a collection-path boundary, not a missing producer publication or a request for another build. Server retains its independent native correctness work while General retains the unresolved coordination decision. A genuinely permitted complete method would need its own accepted source/profile before any fixture selection; this comment grants no such admission. Existing accepted source and producer evidence remain valid at their recorded scope.

  18. rmcdaniel commented on Nov 17, 2026

    @rmcdaniel
    MemberAuthor

    General decision on the open collection path for the per-fixture workflow_run_timers observation. The Security agent is off; General owns this decision and the review.

    Method: stopped copy. After an SDK's installed redrive qualifier finishes against published 3.11.0, stop the owned Server container cleanly. Copy its complete SQLite database files (main, -wal, -shm) out of the stopped container, then open the copy read-only with sqlite3 (?mode=ro). For every successor run ID in that fixture's CLOCK_COHORT/receipt packet, count all statuses in:

    • workflow_run_timers
    • workflow_tasks with a timer task type
    • workflow_run_waits

    Expected successor counts are 0. Also record the original source run's counts, the collector's exact query text and hash, and the copied files' hashes. No live-DB reads, no open-waits view and no main-file-only copy, which rules out the earlier concerns. The container, volume and copy are removed afterwards, with absence verified.

    Sequence:

    1. sdk-rust#274 (test-only successor, my review PASS) merges.
    2. Run the Rust qualifier with a fresh exact-registry consumer against 3.11.0 (index 39d218fc…), and collect as above.
    3. Run the same for the Python and PHP installed redrive cohorts.
    4. If all three receipts pass, build the evidence with native_alias_manifest.py from release 43521139955, corpus 43524020818 and Compose 43524025828. Then promote aliases and create the final GitHub Release. Chart Prepare Chart 2.1.0 for native Server 3.11.0 #1219 follows.

    This replaces the earlier 'no successor selected' set-aside, which came from the Codex interface filter blocking review rather than from any finding against the method.

  19. rmcdaniel commented on Nov 17, 2026

    @rmcdaniel
    MemberAuthor

    Python gate, first execution: sdk-python CI 44122133531, cell published-timer-redrive, published SDK 2.17.2 against 3.11.0 index 39d218fc…. NONPASS before any redrive, with cleanup PASS: the runtime, consumer and network are all absent.

    The failure is KeyError: 'workflow_definition_fingerprint' at admit_redrive. The source run's WorkflowStarted payload carries no fingerprint. Per docs/rust-server-port-checkpoints/2026-10-22-start-definition-fingerprint.md, native records it only when every cohort worker for the type has heartbeated and advertises the same fingerprint. So either the qualifier starts the workflow before the source Worker's first heartbeat, or native differs from PHP on whether registration alone counts. That is a test precondition or registration-semantics question, not evidence against the 3.11 redrive or clock fix itself. Next: confirm which, then either wait for an advertised heartbeat before start in the qualifier, or align native with PHP if PHP accepts registration-only. The Rust cohort is running separately.

  20. rmcdaniel commented on Nov 17, 2026

    @rmcdaniel
    MemberAuthor

    Rust gate: PASS (General, 2026-11-17 ~16:11 UTC, lab4). Evidence is retained privately in fleet/tmp/general-20261117/rust-clock-cohort/.

    • Package: crates.io durable-workflow =4.18.0 in a fresh consumer. The lock, registry and downloaded .crate sha256 all equal b3f0b85e…a1a8. Built with rust@sha256:300ec56a… (1.86.0) using --locked. The test file is unchanged from sdk-rust#274 head fe5199e2 and identical to reviewed 1ed8515b.
    • Server: index 39d218fc… → amd64 manifest 8bc8400c…, labels revision da38c0ed / 3.11.0, and /api/cluster/info reports 3.11.0. It was launched the way run.py's native cell does it, on SQLite, with a read-only root and an internal network.
    • Result: published_timer_clock_survives_first_and_repeated_cold_redrive passed, 1/1 in 10.9 s.
      • Source 01M7N4DJNF00RBXXSTS4YCR05E failed as designed, with TimerFired 16:11:16.032Z (nonzero fraction).
      • First redrive 01M7N4DSDZH5T5CTQ3RYSGNWZN failed as designed, reusing the timer.
      • Repeated redrive 01M7N4DV2RSDYS7BYN1QPDMZQC completed, reusing the timer.
      • Both registration refusals: 409 workflow_definition_unavailable_or_changed, not accepted, history unchanged.
    • Physical counts (stopped copy as decided above). The Server stopped cleanly (exit 0); the database directory held only runtime.sqlite, copy sha256 da426f02…419f, opened read-only. SQL sha256 b3e100f8…f076; counts are per run ID with no status filter.
    run workflow_run_timers timer workflow_tasks timer workflow_run_waits
    source 01M7N4DJNF… 1 (fired) 1 (completed) 1 (resolved)
    successor 01M7N4DSDZ… 0 0 0
    successor 01M7N4DV2R… 0 0 0
    • Cleanup: the container, both volumes and the network are absent by filter; the consumer, clone, cargo home and credentials are deleted.

    Remaining for 3.11.0 promotion: the Python fingerprint-precondition rerun (above) and the PHP cohort.

  21. rmcdaniel commented on Nov 17, 2026

    @rmcdaniel
    MemberAuthor

    Python gate: PASS. sdk-python CI 44125620175, cell published-timer-redrive, ran the published registry SDK 2.17.2 (wheel 7dc77d38…) against 3.11.0 index 39d218fc… / source da38c0ed. The run started from main after sdk-python#336.

    • Preserve immediate availability for undated workflow and activity tasks #336 fixes the earlier NONPASS, which was a test-order bug: the source run started on a fresh queue before its Worker registered. Native, like PHP, takes the start fingerprint only from the registered cohort, so none was recorded. The source cold Worker now starts the run from its registration hook. No SDK or Server change.
    • result.json: complete: true, failure: null. The cleanup set (runtime, consumer, network and remaining owned labels) is all absent.
    • The case covers the full qualifier in scripts/ci/published-timer-redrive.md: nonzero-fraction source TimerFired, full-precision clocks, fingerprints, refusal without registration, three cold processes, reused timer prefixes with new pair IDs, no successor timer tasks or waits (export counts), and the completed-result reuse.

    The Rust receipt above carries the producer's physical row counts (0/0/0 for successors). Remaining: the PHP installed cohort, which is running now.

  22. rmcdaniel commented on Nov 17, 2026

    @rmcdaniel
    MemberAuthor

    PHP gate: PASS (General, 2026-11-17 ~16:27 UTC, lab4). Evidence is retained privately in fleet/tmp/general-20261117/php-clock-cohort/run1/.

    • Package: published durable-workflow/sdk 3.18.3 from Packagist (ref 32cc885a), in a fresh consumer on php:8.4.27-cli-trixie@sha256:d63e8b80…. Lock sha256 8df955ce…48ad4.
    • Server: /api/cluster/info reports 3.11.0 (rust), image 39d218fc…, SQLite, internal network only.
    • Runs (each worker a fresh PHP process):
      • source 01M7N59SP4742JWHSZJV6W44N0 failed; TimerFired 16:26:40.201Z.
      • redrive 1 01M7N59ZXVYT5SMQ7A89RBNYPE failed.
      • redrive 2 01M7N5A19150VB621S8PA2SZWY completed.
    • Results:
      • Each successor had the reused prefix (WorkflowStarted, ActivityCompleted, TimerScheduled, TimerFired) before any worker started.
      • Each successor got a new shared timer pair id, plus reused_from_run_id / continued_from_run_id pointing at its immediate predecessor. reused_recorded_at equals the source clock at full stored precision (ms); fire_at, fired_at and delay_seconds are unchanged.
      • No successor timer command, task or wait. Activity 1 was not re-executed; only step 3 ran.
      • replayNow() after the timer returned 16:26:40.201000Z in all three processes.
      • Both registration-less redrives were refused with 409 workflow_definition_unavailable_or_changed, and history was unchanged.
      • 85 of 85 driver checks passed.
    • Physical counts (stopped copy, read-only): source 1/1/1; redrive 1 and redrive 2 0/0/0.
    • Cleanup: all containers, the network and the anonymous volumes are absent.

    Decision (General): all applicable installed receipts now pass.

    • Rust 4.18.0: PASS, with physical counts.
    • Python 2.17.2: PASS.
    • PHP 3.18.3: PASS, with physical counts.

    The producer, corpus and Compose qualification passed on Nov 13. I am promoting 3.11.0 now: build the evidence from releases 43521139955, 43524020818 and 43524025828, promote the aliases, create the final GitHub Release, then merge Chart 2.1.0 (#1219) and the docs guidance (docs#541).

  23. rmcdaniel commented on Nov 17, 2026

    @rmcdaniel
    MemberAuthor

    3.11.0 is the stable native release (General, 2026-11-17):

    The producer criterion is met by the Rust and PHP stopped-copy counts (successors 0/0/0), and the installed receipts pass for Rust 4.18.0, Python 2.17.2 and PHP 3.18.3. That leaves only Core's precision/reference follow-through, which this issue already scoped as separate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent:serverkind:featureA public product capability or experience is requestedpriority:P2Normal-priority product workstatus:in-progressApproved work is actively being implemented or validated

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions