feat(topology)!: a failed property lens no longer skips the rest of the campaign

[aviggiano]

Refs: #1120

The owner approved partly reversing #1120: a continuing producer's output is now optional to consumers in other groups, the properties group continues on failure, and the property fan-in moves to a halting property-catalog group.

Problem

The shipped topologies behind the default, low-cost, exhaustive and invariant-only audit profiles run eight property lenses feeding property-specification-fanin. The lenses and the fan-in share the properties group, which has no failure_policy, so it halts.

When one lens ends in a terminal failure (attempts exhausted, or a post-agent contract failure), the Smithers engine fails the workflow. The fan-in is skipped, and no strategy, stateful stage or review node runs, so the campaign ends without a report. This happens under the default best-effort completion policy too.

Observed on origin/main (a46a496) with the real engine, using the harness described under Verification: the run ended failed. verify:property-specification-fanin was skipped. verify:boundary-tests-0, verify:stateful-invariant-setup and verify:final-report stayed pending. Of the 65 tasks, 1 failed, 14 finished, 1 was skipped and 49 stayed pending.

Root cause

Four things combine:

1. The lenses halt. They are in a halting group, so their tasks are compiled without continueOnFail. Smithers treats a failed task without continueOnFail as a failed workflow.
2. Only review could use partial results. Even with failure_policy: continue on the lenses, the fan-in would still have required all eight. reconcilesPartialResults (packages/runtime/src/dynamic-runtime.ts) returned task.metadata.node.group === "review". Since feat(runtime): default to best-effort runs with agent-written PARTIAL reports #1120, the group literally named review is the only consumer that may treat a continuing producer's output as optional. That is true in both the compiler (compileSmithersWorkflow) and dynamic lowering.
3. Required lenses also blocked admission downstream. Each task's dependencyArtifactDirs is its whole transitive ancestor closure. So a failed lens would also fail input admission for every strategy and specialist behind the fan-in, because assertVerifiedDependency requires a verification marker for each required agentic ancestor. This is from code reading of admitTaskDependencyInputs.
4. The host rejected a partial fan-in. The host-side fan-in gates (semanticPropertyLenses, verifyLensReferenceExpectationPreservation, both through sealedDirectArtifactDependencies) loaded every direct lens dependency. That included one the in-engine verifier had left out. A fan-in that correctly used only the verified lenses was then rejected with PROPERTY_LENS_AUTHORITY_INVALID.

Change

Ten commits, +803/−73 over 27 files, stacked on #1183 (claude/w27-final-report-selection-record, c9b278a7), which sits on #1201, #1197 and main 538b6188. Of that, +86/−30 is in packages/*/src, much of it comments; the rest is tests, topology YAML, the prompt, docs and the changelog. Commits 7 to 10 address the reviews of the promoted PR.

1. feat(runtime): continuing results are optional to every other group.
   • reconcilesPartialResults(consumer, producerGroup) now returns consumer.group !== producerGroup. It is still the single helper, used by the compiler and by dynamic lowering.
   • A continuing producer's output is optional to a consumer in any other group and stays required inside its own group.
   • The launch task-manifest gate, a subset check since fix: failed launches are recorded and reported, and topology validation cannot reject compiler output #1185, needs no change: every optional input still names a continuing producer.
2. fix(runtime): host fan-in gates skip a lens the verifier did not admit.
   • sealedDirectArtifactDependencies drops an optional producer that is absent from the verifier-persisted admission. It uses the same optionalDeclaredProducerWasNotAdmitted check that finalizedDeclaredContractProducers and the in-engine gate already apply.
   • Required lenses, and optional lenses the verifier admitted, are still loaded and authenticated.
3. feat(topology)!: the topology switch. In .ultrafuzz/topology.yml and packages/config/topologies/{default,exhaustive,invariant-only}.yml:
   • properties gets failure_policy: continue.
   • The fan-in moves to a new halting property-catalog group. The fan-in itself still halts, so no strategy runs without a property catalog.
   • The fan-in prompt says its lens authority lists only lenses that passed verification, and that a missing lens's properties must not be recreated.
   • The docs (topology-yaml.md, artifacts-reports.md, campaigns.md, prompt catalog) state the general rule instead of implying review is the only consumer of partial results.
4. fix(runtime): a task keeps its result when an optional input it ran without is retried successfully.
   • With the lenses continuing, a lens failure no longer ends the run. So when an interrupted run is resumed with --retry-failed (Modal's durable resume always passes it), the failed lens reruns alongside the strategies, and every strategy admitted before it verifies runs without it.
   • Once the retried lens finalized as succeeded, host finalization failed each of those strategies with "finalized optional dependency is missing from verifier admission". dedupe-findings had admitted those strategies in the engine, so it then failed the inverse check ("unfinalized optional dependency is present in verifier admission"). That halts review and fails the run over bookkeeping. The dedupe-findings step is from code reading; the test below exercises the first check.
   • assertOptionalDependencyAuthoritiesCurrent (workflow-sync.ts) now accepts that omission when the producer's verifier finished after the consumer's preparation started. It orders them by the Smithers event timestamps the sync pass already reads. The consumer's admission begins when its prepare: task starts.
   • It still rejects the omission of a producer that verified before the admission began. That is the deleted-marker case the check exists for.
   • The artifact gates already treat the consumer's recorded admission as the authority. optionalDeclaredProducerWasNotAdmitted says: "A marker that appears or disappears later cannot enlarge or erase that immutable ancestor set".
   • main has the same race for a retried strategy and the review tasks admitted without it, and this fixes it too.
   • docs/how-to/restart-continue.md says what a retry of a continuing node does to work that already ran without it.
5. docs(topology): review follow-ups.
   • topology-yaml.md no longer says a same-group dependent is always skipped. A same-group ancestor reached only through another group makes the dependent fail its input admission instead; see Risk, "Custom topologies".
   • The field table no longer calls continue a policy "for optional branches".
   • The reconcilesPartialResults comment is scoped to chains within one group.
   • The dependency-policy test names its reconciling node join, so the old review name does not look significant.
6. docs(changelog): the CHANGELOG.md entries. Under ## Unreleased, a "Breaking changes" entry says which packaged topologies change, that an existing project keeps the old behaviour until it makes three named edits to .ultrafuzz/topology.yml and refreshes the fan-in prompt, and the new rule for custom topologies. An "Other changes" entry covers commit 4. Commit 10 revises both.
7. fix(runtime): a task synchronized while its optional input is still retrying keeps its result.
   • Commit 4 covered a consumer synchronized after the retried producer succeeded. A sync pass while the rerun was still running failed every consumer admitted without it with "optional dependency is not terminal for verifier admission". Modal's waitForTerminalRun runs ultrafuzz inspect, which syncs, every 60 s, so such passes are the norm on that path.
   • That consumer recovered on a later pass, because its failure had no terminal disposition. But a dependent whose host gate reads its output, such as triage after dedupe, failed the gate in the same pass ("finalized ultrafuzz/findings@2 authority is invalid for dedupe"). That records the immutable task-output-validation-failure disposition, so the run ended failed, and --retry-failed never reset the dependent because the engine shows it finished.
   • assertOptionalDependencyAuthoritiesCurrent now accepts the omission of an optional producer that is not terminal, and still throws when the producer has no recorded state. Tasks synchronize in dependency order and start only after their producers settle, so a producer that is not terminal at that point is being rerun and was unverified when the consumer was admitted. If the rerun fails, the omission matches; if it succeeds, it verified after the admission, which commit 4 accepts.
   • This is the reviewer's one-line fix. I did not take the stricter variant it mentions, which accepts only a rerun that started after the admission: in the common order the resume starts the rerun first and the consumers are admitted while it runs, and that variant would still fail them.
8. fix(runtime): dynamic group templates follow the optional-input rule.
   • compileSmithersWorkflow applied the rule only to static tasks, so a dynamic group's task templates, and the children cloned from them, kept every inherited ancestor required. With a continuing producer in another group as an ancestor of a dynamic group, a static sibling ran without the failed producer while every generated child failed its input admission, an artifact-contract failure that leaves the report unverified. main had the same gap for a dynamic group in review.
   • The compiler now applies the same filter to the templates. A template without optional inputs keeps its bytes, so no packaged profile's compiled plan changes (counted below). Generated children remap inherited optional directories the way they already remap dependencyArtifactDirs.
   • A run launched before the upgrade keeps its sealed templates, so its dynamic lowering re-derives unchanged.
9. fix(dashboard): the property fan-in still renders as a property node. flowNodeType keyed the property card on group === "properties", so /api/flow reported the moved fan-in as agentAttempt. It now maps property-catalog to propertySpecification too. None of refactor!: remove per-node cloud execution (execution.mode = "cloud") #1197, feat(runtime)!: run lifecycle commands from the pnpm-patched install instead of per-command npm installs (#921 step 1) #1201 or fix(runtime): record final-report producer selections in the run instead of querying smithers #1183 touches this file.
10. docs(changelog): review follow-ups to the entries and reference docs.
    • The breaking entry is split in two: the packaged topologies and the migration, and the optional-input rule for custom topologies.
    • The first names the packaged default topology; qualifies "PARTIAL" with "unless a later resume --retry-failed reruns the lens successfully"; says what that resume reruns; records the lost failure provenance (output_contracts.missing, the terminal disposition, the public eval failure_code); offers ultrafuzz topology copy default .ultrafuzz/topology.yml --force for an uncustomized project; and applies the fan-in prompt refresh to every existing project, because a project prompt overrides the built-in one under every profile.
    • The second names nodes with no group and the nodes a dynamic group generates, and says a node that reaches a failed ancestor of its own group only through another group fails its admission as an artifact-contract failure, which leaves the report unverified.
    • The retry entry drops "before the task's own result is synchronized", covers commit 7's case, and says the retried node then counts as succeeded, so completion can read COMPLETE. restart-continue.md says the same, and topology-yaml.md states the report consequence of the cross-group chain.
    • A stale comment in smithers-task-manifest.test.ts no longer attributes the emitted shape to "only the review task".

Why the fan-in moves group: if it stayed inside a continuing properties group, the new rule would keep the lenses required for it (same group). Its own output would also become optional to the strategies, so they could run without a catalog. A separate halting group avoids both.

Measured effect on the compiled task plan, re-measured on the stack: I planned and compiled every packaged audit profile on this PR's base (c9b278a7) and on this branch (3e73bf1a), and compared each task's optionalDependencyArtifactDirs. The numbers are the same as they were against main a46a4960.

Profile	Tasks	Lens attempts made non-blocking	Tasks gaining optional inputs	Where the new optional inputs come from
default	65	8	50: fan-in 1, strategies 39, specialists 5, review 5	lens dirs only
low-cost	46	8	31	lens dirs only
invariant-only	25	8	13	lens dirs only
exhaustive	91	8	76	lens dirs, plus dynamic-strategy-generator's 58 strategy attempts
smoke	8	0	0 (unchanged)	none

In every profile, no optional input was removed, and no task has an optional input from its own group.

On 3e73bf1a, too, the number of tasks with optional inputs is the same in every profile (default 50, low-cost 31, invariant-only 13, exhaustive 76; smoke has 2, none of them new), and no packaged dynamic template has optional inputs, so commit 8 changes no packaged plan.

Lens directories are optional to every later task, not only to the fan-in. They sit in each later task's ancestor closure (root cause 3), so without this a failed lens would fail every strategy's input admission. Outside review, lens directories are the only inputs that became optional in default; the real-engine test asserts exactly that.

Why this reverses part of #1120

#1120 (82879ed) changed optional inputs from "every consumer of a continuing producer" to "only group review". Its reason: "Continuation lets independent tasks settle. It does not make a strategy's required inputs optional". In other words, a step in a chain must not run without its predecessor, for example stateful-invariant-handlers after stateful-invariant-setup.

Kept: inputs from the same group stay required. Every stateful and differential chain in the shipped topologies is inside one group, and the compiled plans above contain zero same-group optional inputs.

Reversed: tying the "reconciler" role to the group name review. The property fan-in reconciles by design: the property-source-join gate already checks the catalog against exactly the lenses it was given. The name rule turned one lens failure into the loss of the whole campaign. It also meant that a custom topology whose reconciling group has any other name silently got fail-closed skipping, because on main reconcilesPartialResults returns task.metadata.node.group === "review".

Also changed, as a consequence: in exhaustive, dynamic-strategy-generator (group specialists) now runs with the strategy attempts that succeeded. Since #1120 made the strategies continue on failure, the generator was skipped whenever any of its 58 strategy inputs failed. Before #1120 the strategies halted, so this is new behaviour, not a restoration. The generator reads those inputs only through verifier-admitted ancestor authorities, which omit failed producers.

For custom topologies the rule is now: a consumer in another group runs without a failed continuing input. To keep a step strict, put the chain in one group. This is the rule the owner approved.

Deliberately not built

• No node-level failure_policy. That would be a schema change; a separate group is enough.
• No change to the host recheck of a failed verifier. workflow-sync.ts skips it for any task with optional inputs; see Risk.
• No change to what --retry-failed resets. A retry still reruns a failed lens that the fan-in already consolidated without, and a successful rerun then counts as succeeded, so the report can read COMPLETE although the fan-in never read the lens. The changelog and restart-continue.md now say so. Risk explains the gap, and resume --retry-failed can report COMPLETE after rerunning a continuing node its consumers already ran without #1231 tracks the fix.
• No removal of the optional-authority tamper check. If fix(runtime): a renderer or projection change no longer strands runs whose dynamic groups expanded #1216's rethink drops tamper detection, assertOptionalDependencyAuthoritiesCurrent, both of its exceptions (commits 4 and 7) and the three marker-tamper sync tests can be removed together.
• No cleanup of dependencyGateForNode (artifact-gates.ts). It is exported but only its tests call it, and it carries a third copy of the optional-input rule that matches neither main's rule nor this PR's. Deleting it and its tests is a follow-up, kept out of this PR's conflict-prone hunks.
• No migration for existing projects. Projects keep their scaffolded .ultrafuzz/topology.yml and fan-in prompt; the changelog entry and Risk list the edits.
• No edit to the dynamic strategy generator prompt. It already says to treat missing evidence as lower confidence.
• No handling for "all eight lenses fail". No new code or test for that case.

Verification

Discriminating tests

Each of the first six tests fails on origin/main (a46a496) and passes on this branch, except the sixth's control case, which is meant to pass on both. I checked this after this rebase by copying the branch's six changed test files into a clean origin/main worktree and running them against its source. The last three cover commits 7 to 9; each fails without its commit's source change (checked on b3808c24, or by reverse-applying that commit's source hunks) and passes on eb783f17. These checks ran before the stacked rebase and were not repeated on it. Every test below passes on 3e73bf1a.

• packages/runtime/test/smithers-dependency-skip.integration.test.ts, "a failed property lens leaves the packaged default fan-in, strategies and review to finish" (about 10 s).
  • It plans and compiles the packaged default profile. It asserts that no attempt requires a lens output, and that outside review only lens directories became optional.
  • It then runs the real Smithers engine (smithers up) on a workflow with one task per compiled attempt. That workflow uses the compiled dependencies, verifier IDs, continueOnFail and optional flags, plus the template's own dependencyVerificationProducersFromCompiledTask and skip helpers. property-specification-a16z is made to fail.
  • It asserts every other attempt finishes.
  • It copies two regions of the workflow template: from type WorkflowTaskStateContext = to const agentPromptTemplate =, and from function dependencyVerificationProducersFromCompiledTask to function dynamicExecutionMetadata. If a marker is gone, the test fails with "… is missing from the workflow template".
  • On main it fails because the fan-in requires all eight lens dirs. With that assertion removed, the main engine run ends failed as described under Problem.
• packages/runtime/test/artifact-gates.test.ts, "property fan-in consumes the admitted lenses when an optional lens failed". This is the host gate.
  • On main it fails with PROPERTY_LENS_AUTHORITY_INVALID for the failed lens.
  • On this branch the gate accepts the partial catalog. It still rejects a catalog that cites the failed lens's property (property-source-join at $.properties[0].sources[1]). It also still rejects the failed lens if the admission claims it was admitted.
• packages/runtime/test/workflow-dependency-policy.test.ts. The reconciling consumer's group is renamed from review to catalog. On main it gets no optional inputs.
• packages/runtime/test/dynamic-expansion.test.ts. Adds a catalog join over a continuing strategies fan-out. On main the join gets no optional inputs.
• packages/topology/test/packaged-topologies.test.ts. Pins properties: continue and the fan-in in the halting property-catalog group. This is a configuration pin, not a behavioural test. On main it fails for default, exhaustive and invariant-only.
• packages/runtime/test/runtime.test.ts, "syncRun keeps a consumer admitted without an optional prerequisite whose retry verified after the admission", with the control "syncRun rejects … whose retry verified before the admission" (about 17 s each on this shared machine).
  • The optional specialist fails. The report's verifier then records an admission made while the specialist had no marker, and the specialist's retry verifies. The two cases differ only in whether that verification comes after or before the report's prepare: task started.
  • On main the first case fails: the report is failed with ARTIFACT_VERIFICATION_AUTHORITY_INVALID, "finalized optional dependency is missing from verifier admission optional-specialist".
  • The control passes on both, so the deleted-marker guard stays.
• packages/runtime/test/runtime.test.ts, "syncRun finalizes consumers admitted without an optional prerequisite whose retry is still running" (commit 7, about 23 s). This is the review's reproduction.
  • Topology: lens (continuing properties) → catalog (halting property-catalog) → dedupe (findings@2 and a lifecycle ledger) → triage (triaged-findings@1 and a ledger), all empty.
  • The first sync records the lens failed. The second runs while the lens reruns (in-progress) and dedupe and triage have finished without it. The third runs after the lens verified.
  • On b3808c24 the second sync fails dedupe ("optional dependency is not terminal for verifier admission lens") and triage's gate ("finalized ultrafuzz/findings@2 authority is invalid for dedupe"). With the intermediate assertion removed, the end state has triage failed, with no diagnostics in the final pass.
  • On this branch all four nodes and the run succeed.
• packages/runtime/test/dynamic-workflow.test.ts, "dynamic templates and their children treat a continuing producer in another group as optional" (commit 8). It plans and compiles producer (continuing strategies) → planner (halting) → a dynamic fanout and a static hunter (both goals), then materializes two children. Without the compiler hunk the template's optional inputs are undefined; with it, the template and both children carry the producer's directory. The remap hunk in instantiateDynamicTasks runs here only as an identity remap; relocation itself is not tested.
• packages/dashboard/test/dashboard.test.ts, "serves logical topology flow with expanded attempt details" (commit 9) now asserts the fan-in's flow type is propertySpecification. Without the fix it is agentAttempt.

Existing suites that pass on this branch

Run on 3e73bf1a:

• dynamic-expansion, dynamic-workflow, dynamic-lifecycle, workflow-dependency-policy, task-workflow-identity, generated-workflow-footprint, source-revision and pinned-submodules, full files: 57/57. refactor!: remove per-node cloud execution (execution.mode = "cloud") #1197 deleted cloud-worker-handoff, which the earlier count included. smithers-dependency-skip.integration: 3/3.
• artifact-gates, prompt-artifact-authority, verified-output, invariant-suite-ancestor-order, topology-transform, generated-workflow-verifier, invariant-suite-handoff-durability and terminal-report-completion, full files: 399/399. refactor!: remove per-node cloud execution (execution.mode = "cloud") #1197 and fix(runtime): record final-report producer selections in the run instead of querying smithers #1183 deleted tests from these files.
• Dashboard (all dist-test/test/*.test.js): 39/39.
• Artifacts: smithers-task-manifest.test.ts, full file, 14/14. It includes "optional task inputs must come from producers that continue on failure".
• 27 targeted runtime.test.ts tests, 27/27:
  • commit 4's two retry sync tests and commit 7's;
  • "every packaged audit profile compiles into a task plan the launch manifest gate accepts", which confirms the launch manifest gate accepts the new topology in every profile;
  • the best-effort and require-complete "nonblocking specialist fails and reporting finishes" sync tests;
  • the four existing optional-prerequisite sync tests: "rejects a finalized optional prerequisite behind a" malformed leaf, dangling leaf and symlinked root, and "binds an optional prerequisite digest before a final-boundary manifest swap";
  • the compile and plan tests "compileSmithersWorkflow marks specialist attempts and their artifact handoffs nonblocking", "current-controller rendering preserves prompts idempotently and continue policy for a leaf task", "the shipped default topology expands against the shipped reference catalog", "a clean scaffold plans the threat-model, goal-plan, and dynamic fanout nodes", "plan renders prompt variables against attempt artifact directories for model fan-out" and "startRun warns for optional specialist commands but still rejects blocking command gaps";
  • the tests of the merged PRs that changed the code this PR touches: fix(runtime): resume --retry-failed can recover a run whose verifier rejected output #1205's three verifier-rejection sync tests ("judges a replacement that started after a sealed output-validation failure on its own output", "resume --retry-failed recovers a node whose verifier rejected its output", "keeps a verifier occurrence whose cancellation is stamped before its start"), fix: model fan-out runs can be synchronized repeatedly, recovered aggregates leave skipped, and private evals survive dynamic expansion #1192's three model fan-out sync tests, fix(runtime): attempts abandoned by a crash are recorded, so stats and status agree #1200's three abandoned-attempt sync tests, fix: symlinked project roots, locale-independent ordering, clock-skew-tolerant audit journals and other small verified bugs #1195's "compileSmithersWorkflow seals the canonical selector union from rendered prompt provenance", and "resume --retry-failed after a pre-agent failure keeps synchronizing the reused attempt".
• Vitest: topology 73/73, prompts 58/58, config 90/90 (refactor!: remove per-node cloud execution (execution.mode = "cloud") #1197 changed the config tests). The prompts run includes prompt-structure.test.ts, whose required-variable table still matches the fan-in prompt; the prompt keeps {{ancestor_contract_artifact_authority:ultrafuzz/property-lens@2}}.
• Tests in other packages that read the packaged topologies: modal workspace-config 8/8, evals benchmark-manifest and history 42/42, and CLI audit-profile-commands 3/3.

Gates (all pass)

Run on 3e73bf1a, with exit codes checked directly:

• pnpm install --frozen-lockfile (this PR changes no lockfile, manifest or patch) and pnpm -w build
• pnpm -w format:check
• pnpm -w lint, with the complexity ceiling of 83
• CI=1 ESLINT_PLUGIN_DIFF_COMMIT=origin/claude/w27-final-report-selection-record pnpm -w lint:strict:ci
• pnpm -w knip, three times: after the build, with every dist and dist-test removed, and after the rebuild
• pnpm --filter typecheck for runtime, dashboard, artifacts, config and topology, and tsc -p tsconfig.test.json for the runtime, dashboard, artifacts and CLI tests
• node scripts/docs-check.mjs and pnpm -w docs:check (audit-profile docs, prompt catalog, docs-check.mjs)

Complexity, measured with ESLint's complexity rule on this PR's base (c9b278a7) and on 3e73bf1a: the most complex functions this PR changes are synchronizeTasks (66) and finalizeTerminalTask (65) in workflow-sync.ts. Both have the same complexity on the base. No other changed function goes up by more than 2: assertOptionalDependencyAuthoritiesCurrent 14 → 16 (17 before commit 7), sealedDirectArtifactDependencies 12 → 13, flowNodeType 12 → 13, compileSmithersWorkflow unchanged at 15, and lowerTaskDynamicDependencies goes down, 12 → 11. The only function at 80 or above in the changed runtime source files is verifyCoverageProductionInventory (artifact-gates.ts, 83, at the ceiling). This PR does not change it.

Not run

• The discriminating checks, on the stack (see above).
• pnpm -w size, which passed on eb783f17 before the stacked rebase.
• The full runtime.test.ts, CLI, evals and modal suites.
• Bun adapter contracts.
• A live campaign.
• An end-to-end run of the generated workflow.tsx with a failed lens. The real-engine test drives the compiled scheduling inputs, not the generated workflow. The generated workflow's admission of optional ancestors is the existing path review tasks already use.
• A real-engine resume --retry-failed after a lens failure. Commits 4 and 7 are tested at the sync layer with fixture events.
• A relocated project for commit 8's remap of a generated child's optional directories.

Risk / compatibility

• Breaking semantic change. In the shipped topologies a failed lens no longer fails the campaign:

  • Its properties are missing from the catalog.
  • It is reported as a failed node.
  • Under best-effort completion the run can still succeed. The report then depends on how the lens failed (code reading):
    • If the lens exhausted its attempts, the report is verified and PARTIAL, because summarizeOutcomes counts the lens as incomplete.
    • If the lens failed its post-agent contract check, its failure category is artifact-contract, which assertTerminalState refuses to downgrade. The best-effort report is then published unverified, and report --require-verified fails. Strategy contract failures already behave this way.
    • Both hold only until a later resume --retry-failed reruns the lens successfully; see the next item, the known gap.
  • --require-complete runs still end unsuccessful, with the same exception.

• Known gap, deferred to resume --retry-failed can report COMPLETE after rerunning a continuing node its consumers already ran without #1231: a successful lens retry can make an incomplete catalog read COMPLETE. Greptile raised this as P1.

  • When resume --retry-failed reruns a failed lens and the rerun succeeds, the lens counts as succeeded.
  • The report's completion can then read COMPLETE, and a --require-complete run can end succeeded. Yet the fan-in and every task admitted before the rerun verified never read that lens's properties.
  • I checked this at the sync layer on 3e73bf1a. A require-complete variant of commit 4's retry test ends succeeded, while the report's recorded admission lists only direct-strategy. I did not commit that probe.
  • Commits 4 and 7 reach this outcome for consumers synchronized after the rerun. Without them, those consumers fail the run over bookkeeping instead.
  • main has the same gap for a failed strategy and the review tasks that finalized without it, because immutableTerminalFinalization never re-finalizes a succeeded task (code reading).
  • The changelog and restart-continue.md state the gap.
  • The fix belongs in --retry-failed and in Modal's durable resume, so it is not in this PR. resume --retry-failed can report COMPLETE after rerunning a continuing node its consumers already ran without #1231 proposes that --retry-failed skip a failed continuing attempt that a started task already treated as optional. That keeps the report PARTIAL and avoids the wasted rerun. Modal's durable resume would then have to stop resuming a succeeded run for such failures, because waitForTerminalRun waits until the resume changes the run.

• Resume with --retry-failed. Modal's durable resume always passes this flag (packages/modal/src/resume.ts), and modalDurableRunNeedsResume also resumes a succeeded run that has a failed node. Commits 4 and 7 stop a retried lens from failing the tasks that ran without it, whether a sync pass sees them during the rerun or after it. These costs remain, all by code reading, and the changelog now states them:

  • A lens whose agent failed is reset alone and reruns. Tasks admitted before it verifies keep running without it, so its new output reaches only tasks admitted afterwards.
  • A lens whose verifier failed is reopened with its dependents. Smithers timetravel then resets every node with an attempt that started at or after the lens attempt (@smthrs/time-travel 0.35.0, resolveResetNodes), so the rest of the campaign reruns. Since fix(runtime): resume --retry-failed can recover a run whose verifier rejected output #1205 that rerun is judged on its own output.
  • A successful rerun can make the report read COMPLETE; see the previous item.

• All eight lenses fail. By code reading, the fan-in still runs with only the discovery ledger as a known source. It may produce a ledger-only catalog or fail, and a fan-in failure halts as today. Not tested.

• Less failure detail for strategies, specialists and the fan-in. These tasks now have optional inputs, so when their verifier fails, workflow-sync.ts no longer re-runs the output gate on the host. Review tasks already worked this way.

  • Such a node still fails, with the verifier's error and category artifact-contract.
  • It no longer carries terminal_disposition: task-output-validation-failure or output_contracts.missing.
  • Public eval diagnostics lose the optional failure_code for it.
  • The failure is not an immutable finalization.
  • Modal's terminal-disposition classifier also reads it, but its result does not change. isGenuineTaskFailure additionally requires the verifier's recorded workflow state to be finished, and a failed verifier's is failed. So these failures were already classified as operational on main.
  • The breaking changelog entry now says this.

• Exhaustive behaviour change. dynamic-strategy-generator runs with partial strategy results.

• Existing projects. The default and low-cost profiles use the project's .ultrafuzz/topology.yml. Already-initialized projects keep the old behaviour until they edit it in three places, which the changelog entry spells out:

  • add defaults: {failure_policy: continue} to properties;
  • add a property-catalog group with no failure_policy, so it halts (the scaffold gives it label: Property catalog and color: "#854d0e");
  • set property-specification-fanin's group to property-catalog.

  An uncustomized project can instead run ultrafuzz topology copy default .ultrafuzz/topology.yml --force, the path docs/how-to/edit-prompts-topology.md already documents; the changelog names both.

  Every existing project, under every profile including exhaustive and invariant-only, should also refresh .ultrafuzz/prompts/properties/property-specification-fanin.md: delete it and rerun ultrafuzz init, which is the path ultrafuzz validate already gives for a project prompt that differs from the built-in one (PROMPT_DIFFERS_FROM_BUILT_IN, a warning). A project prompt overrides the built-in one under every profile, and the scaffolded copy still asks the fan-in to cover every topology-required lens. The gate checks the catalog against the admitted lenses only, so a fan-in that covers what it can read still passes, but the agent is told to cover a lens it cannot read; a catalog that cites the failed lens fails the gate (tested). The packaged exhaustive and invariant-only topologies change on upgrade.

• In-flight runs. The static task plan is sealed at launch. A run started before the upgrade keeps its optional inputs, and commits 2 and 4 only affect tasks that list optional inputs. Dynamic lowering is different:

  • Each resume re-derives it with the installed rule and compares it byte for byte with the persisted plan (verifyDynamicRuntimeMaterialization).
  • So in a custom topology where a non-review node consumes a continuing dynamic group, an in-flight run stops after the upgrade with "published dynamic runtime controls no longer re-derive from their sealed base".
  • This only happens once the group has expanded, and the existing Unreleased breaking change already says a run whose dynamic groups have expanded cannot be synchronized or resumed after upgrading (fix(runtime): stop host artifact gates rejecting valid campaign output #1176, fix: symlinked project roots, locale-independent ordering, clock-skew-tolerant audit journals and other small verified bugs #1195). So the changelog entry does not repeat it.
  • The shipped topologies are unaffected. Their only consumer of a dynamic group is dedupe-findings, in review, which gets the same optional inputs under both rules.
  • Commit 8 changes only newly compiled templates. A run launched before the upgrade keeps its sealed templates, and a template without optional inputs keeps its bytes, so its generated children re-derive unchanged.

• Custom topologies. A consumer in another group now runs without a failed continuing producer; see "Why this reverses part of feat(runtime): default to best-effort runs with agent-written PARTIAL reports #1120". That includes a node with no group (reconcilesPartialResults compares undefined with the producer's group), which on main required the input and was skipped, and, since commit 8, the nodes a dynamic group generates. A same-group ancestor reached only through another group behaves differently:

  • Example: strategy s1 → specialist x → strategy s2.
  • When s1 fails, x runs and s2 is not skipped. s2 fails its input admission instead, as an artifact-contract preparation failure. assertTerminalState then refuses to derive the report's completion, so the best-effort report is published unverified. On main both were dependency-cascade skips and the report could verify as PARTIAL. The changelog and topology-yaml.md now say this and advise keeping a strict chain in one group.
  • None of the packaged profiles has this shape.

• Stacked on refactor!: remove per-node cloud execution (execution.mode = "cloud") #1197, feat(runtime)!: run lifecycle commands from the pnpm-patched install instead of per-command npm installs (#921 step 1) #1201 and fix(runtime): record final-report producer selections in the run instead of querying smithers #1183. Until they merge, this branch carries their commits, and the diff against main shows them. Merge in the approved order: refactor!: remove per-node cloud execution (execution.mode = "cloud") #1197, feat(runtime)!: run lifecycle commands from the pnpm-patched install instead of per-command npm installs (#921 step 1) #1201, fix(runtime): record final-report producer selections in the run instead of querying smithers #1183, then this PR. After fix(runtime): record final-report producer selections in the run instead of querying smithers #1183 merges, the diff is this PR's own ten commits.

Rebase notes

Stacked rebase onto #1183 (c9b278a7). This PR's ten commits moved from main a46a4960 onto origin/claude/w27-final-report-selection-record c9b278a7. That is #1183's six commits on #1201's five, on #1197's fourteen, on main 538b6188, which adds #1228 and #1229. The pre-rebase head was eb783f17, and the new head is 3e73bf1a. This PR changes no lockfile, manifest or patch, so pnpm install --frozen-lockfile was enough. Commits 1, 2, 5, 7, 8 and 9 applied unchanged (git range-diff shows =).

The conflicts, and how each was resolved:

• CHANGELOG.md (commits 6 and 10). fix(runtime): record final-report producer selections in the run instead of querying smithers #1183 and refactor!: remove per-node cloud execution (execution.mode = "cloud") #1197 added their breaking entries at the top of ### Breaking changes, and fix(security): merge per-range registry advisories and patch brace-expansion and undici #1229 added its entry at the top of ### Other changes.
  • This PR's breaking entry goes above fix(runtime): record final-report producer selections in the run instead of querying smithers #1183's, and its retry entry above fix(security): merge per-range registry advisories and patch brace-expansion and undici #1229's. The other entries are unchanged.
  • Commit 10 then replaces its two commit-6 entries in place: it splits the breaking entry in two and rewrites the retry entry, as before.

Changes made during the rebase that no conflict forced. Both are the manual edits this description's Risk section had flagged:

• Commit 3, the integration test's template markers. refactor!: remove per-node cloud execution (execution.mode = "cloud") #1197 removed both markers that compiledSchedulingWorkflowSource sliced on: type DependencyVerificationProducer = and function dynamicExecutionPath.
  • The first slice now ends at const agentPromptTemplate =, like the file's two other slices.
  • The second slice now ends at function dynamicExecutionMetadata, the function that follows dependencyVerificationProducersFromCompiledTask on refactor!: remove per-node cloud execution (execution.mode = "cloud") #1197's template.
  • Without this edit, the test fails with "… is missing from the workflow template". With it, the test passes and still runs the real engine.
• Commit 4, the dead fallback in verifiedAfterAdmissionOf (workflow-sync.ts). The consumer's admission time used to fall back to the agent task's start, commented "a cloud attempt has none".
  • refactor!: remove per-node cloud execution (execution.mode = "cloud") #1197 removed per-node cloud execution, so every attempt now renders a prepare: task that its agent task depends on. The fallback could not be reached, and I deleted it and its comment.
  • If a sync pass ever lacks the preparation's start, the check now fails closed and keeps rejecting the omission.
  • Commit 4's and commit 7's sync tests order the omission by prepare: events and pass.

Also re-checked on the stack: the task-plan counts under Change, and the complexity figures under Verification. Both are unchanged.

Greptile (review of eb783f17):

• P1, packages/runtime/src/workflow-sync.ts:3737 (now line 3736), "Incomplete catalog can appear complete": deferred to resume --retry-failed can report COMPLETE after rerunning a continuing node its consumers already ran without #1231.
  • The observation is right, and a sync-layer probe on 3e73bf1a confirms it (Risk, "Known gap").
  • main has the same gap for strategies, the changelog and restart-continue.md already state it, and the fix belongs in --retry-failed and Modal's durable resume, not in this PR's sync exception.
  • Dropping the exception instead would bring back the failure that commits 4 and 7 fixed: the consumers and the run fail over bookkeeping.
• P2, packages/runtime/src/dynamic-runtime.ts:278, "Relocation remapping lacks coverage": declined.
  • Commit 8 passes the optional directories through the same remapProjectPath(directory, group.promptContext.projectRoot, projectRoot) call as the required dependencyArtifactDirs beside them. The required remap has no relocation test either.
  • After refactor!: remove per-node cloud execution (execution.mode = "cloud") #1197, no shipped flow plans a run under one project root and runs its engine under another. Per-node cloud workers are gone, and the Modal eval runner keeps its target at a fixed path. Only an operator who moves a project directory in the middle of a run would relocate one.
  • A missed remap would fail closed. The child's optional directories would then match none of its dependency directories, so it would require the failed producer and be skipped, as before this PR.
  • "Not run" still lists the relocated case.

Earlier rebase. Rebased from 2cacf4ca onto origin/main a46a4960 (19 new commits on main). There was one textual conflict, resolved by keeping both sides. The other overlaps merged cleanly, and I checked them by hand:

• docs/reference/artifacts-reports.md (conflict in commit 3). fix: a reworded finding no longer discards the final report #1204 added the "Each report row carries a source finding …" paragraph directly above the best-effort paragraph that this PR rewrites. I kept fix: a reworded finding no longer discards the final report #1204's paragraph unchanged and put this PR's rewritten paragraph after it.
• packages/runtime/src/workflow-sync.ts (merged without conflict, checked by hand). fix(runtime): resume --retry-failed can recover a run whose verifier rejected output #1205 changed immutableTerminalFinalization and workflowEvidenceSupersedesPrevious, so that a rerun of a verifier-rejected node is judged on its own output. Commit 4 changes synchronizeTasks' setup, finalizeTerminalTask's input and assertOptionalDependencyAuthoritiesCurrent. These are separate functions, and the two changes combine: after fix(runtime): resume --retry-failed can recover a run whose verifier rejected output #1205 a retried verifier-rejected producer can finalize as succeeded, and in that case commit 4's exception covers the tasks that ran without it. fix(runtime): resume --retry-failed can recover a run whose verifier rejected output #1205's three sync tests pass on this branch.
• packages/runtime/src/dynamic-runtime.ts and packages/runtime/test/dynamic-expansion.test.ts (merged without conflict). fix(runtime): a source retry withdraws the prompts rendered from the withdrawn expansion #1220 changed dynamic-expansion-retry.ts, not dynamic-runtime.ts, and added a test in another part of dynamic-expansion.test.ts. Its prompt withdrawal does not use reconcilesPartialResults. The whole dynamic-expansion file passes.
• Topology YAML (no conflict). fix(topology): restore the review group time budget and remove prompt/gate contradictions #1164's timeout_seconds: 7200 pin on the review group was already in this branch's base. This PR's hunks add only the properties defaults, the property-catalog group and the fan-in's group.
• Prompts (no conflict). test(prompts): fail when the goal hunter prompt loses its node_id lookup key #1212 changed only prompt-structure.test.ts. The prompts suite passes against the edited fan-in prompt.
• docs/reference/topology-yaml.md (merged without conflict). fix(runtime): a source retry withdraws the prompts rendered from the withdrawn expansion #1220's sentence is in another section; docs:check passes.
• New commit 6 adds the CHANGELOG.md entries. They replace the "Changelog entry" section that this description used to carry.
• Changes from earlier rebases that still hold: refactor(runtime): delete dead controller-generation, sealed-refresh, re-finalization and recovery-authority code #1193 deleted refinalizeControllerFailures, so synchronizeTasks is the only caller of finalizeTerminalTask. fix: model fan-out runs can be synchronized repeatedly, recovered aggregates leave skipped, and private evals survive dynamic expansion #1192's model fan-out aggregate change is independent of commit 4, and its tests pass. test: delete vacuous and prompt-prose tests outside the runtime package #1196's topology cache in packaged-topologies.test.ts is used by commit 3's pin.

🤖 Generated with Claude Code

Confidence Score: 2/5

The PR does not yet appear safe to merge because three previously reported correctness and cleanup issues remain outstanding.

Findings

1.  Incomplete catalog can appear complete ▶
2.  Old attempts enter report history ▶
3.  Cloud cleanup leaves resources behind ▶
4.  Relocation remapping lacks coverage ▶

Fix with agent prompt

### Issue 1
packages/runtime/src/workflow-sync.ts:undefined-3736
When `resume --retry-failed` successfully reruns a property lens after the fan-in ran without it, this exception keeps the fan-in’s earlier result. The lens then counts as succeeded, so the report can say COMPLETE and a `--require-complete` run can succeed even though the catalog and downstream analysis never used that lens’s properties. The retry needs to preserve the partial outcome or rerun the work that used the incomplete catalog.

### Issue 2
packages/runtime/src/templates/smithers/workflows/workflow.tsx:2235-2236
After a reset reuses an attempt number, a report attempt that fails before reaching `generate` leaves its old selection in this file. When a later attempt runs, this filter includes that selection as if it belonged to the current round. The report and a restarted verifier then agree on `failed_attempts` that include a model attempt that did not execute in this round, allowing incorrect history in a verified report.

### Issue 3
packages/runtime/src/clean.ts:undefined-85
When cleaning a run created with the former per-node Modal execution, this path now deletes the local run directory without terminating its sandboxes or deleting its remote volume. If that run left cloud resources behind, `clean` can report success while they remain billable and remove the run evidence needed to identify them for manual cleanup.

### Issue 4
packages/runtime/src/dynamic-runtime.ts:276-278
The new test materializes generated children under the same project root used to compile their templates. It therefore cannot catch an optional dependency path that still points to the old root after relocation, even when the required dependency path points to the new one. Test with distinct roots and assert that both paths identify the same producer.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

The PR lets property-lens failures leave the campaign running, moves property consolidation into a halting catalog group, and makes continuing producers optional to consumers in other groups. It also adjusts artifact admission, retry synchronization, dynamic task templates, dashboard rendering, tests, and migration documentation. There are no changes since the previous review.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  L["Property lenses<br/>continue on failure"] -->|"verified outputs only"| C["Property catalog<br/>halts on failure"]
  C --> S["Strategies and specialists"]
  S --> R["Review and report"]

Loading

_{Reviews (3) · Last reviewed commit: "docs(changelog): state what a failed or ..."}

[greptile-apps]

Incomplete catalog can appear complete When resume --retry-failed successfully reruns a property lens after the fan-in ran without it, this exception keeps the fan-in’s earlier result. The lens then counts as succeeded, so the report can say COMPLETE and a --require-complete run can succeed even though the catalog and downstream analysis never used that lens’s properties. The retry needs to preserve the partial outcome or rerun the work that used the incomplete catalog.

Prompt To Fix With AI

This is a comment left during a code review.
Path: packages/runtime/src/workflow-sync.ts
Line: 3737

Comment:
**Incomplete catalog can appear complete** When `resume --retry-failed` successfully reruns a property lens after the fan-in ran without it, this exception keeps the fan-in’s earlier result. The lens then counts as succeeded, so the report can say COMPLETE and a `--require-complete` run can succeed even though the catalog and downstream analysis never used that lens’s properties. The retry needs to preserve the partial outcome or rerun the work that used the incomplete catalog.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

[greptile-apps]

Relocation remapping lacks coverage The new test materializes generated children under the same project root used to compile their templates. It therefore cannot catch an optional dependency path that still points to the old root after relocation, even when the required dependency path points to the new one. Test with distinct roots and assert that both paths identify the same producer.

Prompt To Fix With AI

This is a comment left during a code review.
Path: packages/runtime/src/dynamic-runtime.ts
Line: 276-278

Comment:
**Relocation remapping lacks coverage** The new test materializes generated children under the same project root used to compile their templates. It therefore cannot catch an optional dependency path that still points to the old root after relocation, even when the required dependency path points to the new one. Test with distinct roots and assert that both paths identify the same producer.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[greptile-apps]

Old attempts enter report history After a reset reuses an attempt number, a report attempt that fails before reaching generate leaves its old selection in this file. When a later attempt runs, this filter includes that selection as if it belonged to the current round. The report and a restarted verifier then agree on failed_attempts that include a model attempt that did not execute in this round, allowing incorrect history in a verified report.

Prompt To Fix With AI

This is a comment left during a code review.
Path: packages/runtime/src/templates/smithers/workflows/workflow.tsx
Line: 2235-2236

Comment:
**Old attempts enter report history** After a reset reuses an attempt number, a report attempt that fails before reaching `generate` leaves its old selection in this file. When a later attempt runs, this filter includes that selection as if it belonged to the current round. The report and a restarted verifier then agree on `failed_attempts` that include a model attempt that did not execute in this round, allowing incorrect history in a verified report.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

[greptile-apps]

Cloud cleanup leaves resources behind When cleaning a run created with the former per-node Modal execution, this path now deletes the local run directory without terminating its sandboxes or deleting its remote volume. If that run left cloud resources behind, clean can report success while they remain billable and remove the run evidence needed to identify them for manual cleanup.

Prompt To Fix With AI

This is a comment left during a code review.
Path: packages/runtime/src/clean.ts
Line: 85

Comment:
**Cloud cleanup leaves resources behind** When cleaning a run created with the former per-node Modal execution, this path now deletes the local run directory without terminating its sandboxes or deleting its remote volume. If that run left cloud resources behind, `clean` can report success while they remain billable and remove the run evidence needed to identify them for manual cleanup.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.