fix(runtime)!: a schema change in an upgrade no longer fails every resumed task - #1238
Merged
Merged
Conversation
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| // A compile past the validator's 30 s cap fails the host gate. This also shows that the simulated compile | ||
| // is the gate's wait for its own worker, after the validator set its deadline: had an earlier wait on the | ||
| // gate's path moved the clock first, the deadline would count from the moved clock and the gate would pass. | ||
| assert.equal(afterSlowSchemaCompile(40_000, hostGate)[0]?.code, "JSON_VALIDATION_TIMEOUT"); |
There was a problem hiding this comment.
When the simulated compile exceeds 30 seconds, the validator returns a timeout result with a null schema. The host gate checks schema identity before validation status, so it returns ARTIFACT_VALIDATOR_IDENTITY_MISMATCH rather than the asserted JSON_VALIDATION_TIMEOUT. This makes the new regression test fail and masks the timeout diagnostic.
Prompt To Fix With AI
This is a comment left during a code review.
Path: packages/runtime/test/artifact-gates.test.ts
Line: 5288
Comment:
**Timeout assertion expects wrong diagnostic**
When the simulated compile exceeds 30 seconds, the validator returns a timeout result with a null schema. The host gate checks schema identity before validation status, so it returns `ARTIFACT_VALIDATOR_IDENTITY_MISMATCH` rather than the asserted `JSON_VALIDATION_TIMEOUT`. This makes the new regression test fail and masks the timeout diagnostic.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.
aviggiano
force-pushed
the
claude/schema-bundle-records-not-gates
branch
3 times, most recently
from
September 30, 2026 22:59
e6fa614 to
d03b29f
Compare
…sumed task A native resume runs the launch-rendered workflow against the installed @ultrafuzz/artifacts, and three task-preparation steps required the installed schema bundle to equal the one the run was launched with. One `$comment` added to an installed schema failed every resumed task, and neither --retry-failed nor --refresh-controller recovered it (#921, the prompt-sealing investigation's map 4): - materialize-prompt-schemas refused a workspace copy that differed from the installed bundle ("prompt schema destination differs from checked-in source"); - assert-task-output-schema-bindings compared this build's binding with the plan ("planned schema binding changed for <path>"); - preflight-json-validator parsed the run's own validator envelope against this build's findings binding by default, while that validator, the launch closure, reports the launch bundle ("mismatched identity"). The run's plan already records which schema each output must satisfy, and the host gates validate against it since #1188. The workflow now does the same: - plannedArtifactSchemaBundle(runRoot, sha256) resolves the bundle a plan names: this build's schemas when their digest is the planned one, otherwise the copy sealed in the run's execution snapshot. The host gate uses it too, so the host and the workflow share one selection rule and one snapshot lookup. - Task preparation copies that bundle into .ultrafuzz/schemas and rewrites a copy that differs instead of failing: the copy is only the agent's view. The bindings step checks that the planned bundle holds each planned schema, and the preflight expects the planned bundle's identity. The verifier, dependency admission and the goal-search count validate against the planned schema (validateArtifactContractBytes takes it as an optional argument, and no longer compares the reported validator build). - --refresh-controller no longer rebinds declared outputs to the installed bundle (#982), so the refreshed verifier's markers keep matching the plan. The replacePromptSchemas flag and its __ULTRAFUZZ_REPLACE_PROMPT_SCHEMAS__ placeholder are deleted. Breaking: parseJsonValidatorPreflightSuccessEnvelope requires the expected identity instead of defaulting to this build's, and materializePromptSchemas takes the bundle and drops replaceExisting. A workflow rendered before this change calls both the old way, so a run launched before it must be continued with --refresh-controller. A run no longer adopts an upgrade's schemas. The new runtime test launches a run, loads its project workflow the way a native resume does against an upgraded copy of @ultrafuzz/artifacts, and runs task preparation and verification in process. On the base branch it fails at preflight-json-validator (and at materialize-prompt-schemas or assert-task-output-schema-bindings when those run first). Co-Authored-By: Claude Opus 5.5 <[email protected]>
The specification said task preparation and the validator preflight must still require the installed schema bundle to be the planned one; the CLI reference said `--refresh-controller` rebinds each output to the installed bundle; the artifacts reference said the preflight checks the build identity; the security overview said the preflight checks the validator identity. They now say what the code does: - the host and the workflow validate each artifact against the run's planned bundle, this build's when its digest is the planned one and otherwise the copy sealed in the execution snapshot; - task preparation copies that bundle into the task workspace, replacing a copy that differs, and the preflight requires the run's own validator to report it; - `--refresh-controller` keeps every recorded binding, and after a schema change keeps the run's own validator launcher; - an upgrade's schemas apply only to runs launched after it (also in the restart how-to). The CHANGELOG entry lists the failures this removes, the refresh change, the API breaks, the migration for runs launched earlier, and what is given up. Co-Authored-By: Claude Opus 5.5 <[email protected]>
…emas
Review follow-ups for keeping a run on the schema bundle it was planned
with.
- A plain `resume` of a run whose launch workflow predates planned schema
bundles now fails with WORKFLOW_CONTROLLER_REFRESH_REQUIRED before
Smithers starts. That workflow still calls
`materializePromptSchemas(schemaDirectory)` or passes it
`{ replaceExisting }`, which this release rejects in every task
preparation, so every remaining prepare, agent and verifier failed with
a TypeError about the `path` argument and used up its retries.
`materializePromptSchemas` now names the refresh instead, for an engine
the supervisor relaunches from an install upgraded in place.
- The validator preflight runs only for a task with a schema-backed
output. A plan with none names no bundle, so its preflight compared the
run's own validator with this build's bundle and failed every task after
a schema change.
- Synchronization counts a findings output validated against its planned
schema, as the artifact gate and the engine's verifier do. After an
upgrade that tightened the findings schema it validated with this
build's, recorded a verified attempt as FINDINGS_VALIDATION_FAILED and
wrote no manifest.
- A controller refresh no longer tries to rotate the run's trusted
launcher to an installed CLI that validates with another schema bundle,
a rotation its preflight always refuses. It keeps the run's launcher,
which the resume preflight still verifies, without a
WORKFLOW_TRUSTED_CLI_UNVERIFIED warning that predicted failing tasks.
- The workflow resolves one planned bundle per run, and
`plannedOutputSchema` refuses a schema-backed output that does not name
its schema in that bundle instead of returning undefined, which fell
back to this build's schema. The assert-task-output-schema-bindings step
is deleted: the task manifest schema requires a complete binding for
every JSON output, and the planned bundle's digest commits to every
schema it holds, so the step could not fail for a plan the planner
wrote.
- A validation against a sealed bundle, which starts a worker that
compiles the whole bundle, gets the validator's 30 s cap instead of 5 s,
and bytes it accepted are remembered for that registry, so dependency
admission's re-checks before and after each attempt start no worker. Its
"no longer valid" error now names the issues, such as a timeout.
- The sealed bundle is found through state.json's `executionSnapshot`
only; nothing has written `controllerExecutionSnapshot` since #1193.
`schema-bundle.ts` uses `sha256Bytes` instead of its own copy.
None of the five validator-build modules and no schema file changes, so
VALIDATOR_BUILD_IDENTITY and every schema digest are unchanged.
Tests: the resume-after-upgrade test gains a dependent task whose
admission must accept `[]` while the upgraded findings schema requires an
entry, and a text-only variant; its in-process workflow loader is now
shared with the render test. New tests cover the refusal, the named
error, sync's findings count in a child process that installs the
upgraded schemas, the refresh keeping the launcher, the planned-output
binding, the goal-search finding count, and the remembered acceptance.
Each fails with its fix reverted.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
…ue an earlier run The docs claimed more than the code does. Each artifact's contract validation (host artifact gates, verified reads, synchronization's findings count, the workflow's verifier and dependency admission) uses the run's planned bundle. Code that parses an already validated artifact into this build's types (the properties JSON/Markdown parity, invariant suite and coverage checks, and the workflow's and host's semantic contexts for properties, implemented-properties, property-campaign and campaign findings), and the final report's canonical projection, still use the installed schemas, so an upgrade that tightens one of them within a contract version can still fail a resumed task. SPECS, schemas.md, restart-continue.md and the CHANGELOG now say that, and the SPECS sentence that compared the installed bundle with "the reading build's own" says what it meant. security.md and the CHANGELOG state the guarantee given up: a task's `.ultrafuzz/schemas` copy is no longer tamper-evident, because preparation rewrites a copy that differs without refusing or recording it. The CHANGELOG entry and cli.md name WORKFLOW_CONTROLLER_REFRESH_REQUIRED for a plain resume of a run launched before this change, which must be continued with `resume --refresh-controller` every time, replacing quote the error a supervisor relaunch from an install upgraded in place reports, which `--refresh-controller --retry-failed` recovers. They also drop the documented WORKFLOW_TRUSTED_CLI_UNVERIFIED warning after a schema upgrade, and say the validator preflight runs only for a task with a schema-backed output. Co-Authored-By: Claude Opus 5.5 <[email protected]>
… as long as the workflow does After a schema upgrade, the workflow validates a run's artifacts against the bundle sealed in its execution snapshot with a 30 s deadline, because each such validation starts a worker that compiles the whole bundle. The host artifact gate, which synchronization and verified reads run, validated against the same bundle with the validator's 5 s default, so a first compile slower than 5 s under load failed at sync an attempt that the run's verifier had accepted. Both now validate through validateJsonBytesAgainstBundleSync (@ultrafuzz/artifacts), which gives any bundle other than this build's the sealed-bundle deadline. No validator-build module changes, so VALIDATOR_BUILD_IDENTITY and every schema digest are unchanged. Co-Authored-By: Claude Opus 5.5 <[email protected]>
… it waits on The sealed-bundle deadline test simulates a slow compile by moving the clock at the first Atomics.wait. Its self-check proved that only for a direct validator call: had a later change added a wait on the host gate's path before the validator sets its deadline (a lock acquisition, say), the clock would move first and the host gate would pass with any deadline. The host gate now also has to time out on a 40 s compile, past the validator's 30 s cap, which it does only when the simulated compile is its own wait for its worker. With a wait added on that path before the validator's deadline, the previous test passed with or without the host gate's deadline; this one fails at that assertion. Co-Authored-By: Claude Opus 5.5 <[email protected]>
aviggiano
force-pushed
the
claude/schema-bundle-records-not-gates
branch
from
September 30, 2026 23:42
d03b29f to
ecb2e9a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #1237 (
claude/prompts-refresh-on-resume,4bd75ada) for the merge train #1235 → #1237 → #1238: merge #1235 and #1237 first; until then this PR also lists their twelve commits, and this PR's own are the top six. #1230, named below, has merged (e45076a4).Stacked on #1230 (
claude/prompt-records-not-seals, 9b42ae0, which sits onmainafter the #1197/#1201/#1183/#1198 train merged). Review the top four commits only.Problem
A native
resumeruns the run's launch-rendered workflow against the installed@ultrafuzz/artifacts, and three task-preparation steps required the installed schema bundle to equal the one the run was launched with. Map 4 of the prompt-sealing investigation measured that one"$comment"added to an installed schema file (no rebuild is needed; the registry reads schema files at runtime) fails every resumed task, including a text-only one, and that neither--retry-failednor--refresh-controller --retry-failedrecovers it. The #1230 implementer hit the same two failures.materializePromptSchemasrefused a workspace copy that differed from the installed bundle (only a refresh could replace it)prepare:<attempt> failed at step materialize-prompt-schemas: prompt schema destination differs from checked-in source: …assertTaskOutputSchemaBindingscompared this build's binding with the plan…step assert-task-output-schema-bindings: artifact-contract failure: planned schema binding changed for <path>parseJsonValidatorPreflightSuccessEnvelopedefaulted its expected identity to this build's findings binding, while the run's own validator (the launch closure) reports the launch bundle…step preflight-json-validator: … invalid success envelope(mismatched identity)--refresh-controllerrebound every declared output to the installed bundle (#982)graph.jsonafter a schema changeBehind those gates, the workflow's verifier and dependency admission also validated with the installed schemas, and so did synchronization's findings count, so after a substantive schema change the engine and the host would check artifacts against a schema the run was never planned with; C1–C3 stopped every task before that could be reached. The host artifact gates had already moved to the planned schema in #1188.
Change
The run's plan already records the schema each output must satisfy (file,
$id, SHA-256, bundle digest; the validator build is provenance since #1188). Every contract validation of a run's artifact now follows that record, and nothing a task runs compares the installed bundle with it.plannedArtifactSchemaBundle(runRoot, bundleSha256)(packages/artifacts/src/schema-bundle.ts) returns this build's schemas when their digest is the planned one, otherwise the copy sealed in the run's execution snapshot (state.json→provenance.workflow.executionSnapshot→modules/@ultrafuzz/artifacts/schema), and refuses a sealed copy whose digest is not the planned one.verifyRequiredArtifactSchemaBinding(host gates, verified reads) calls it instead of its own copy of that rule and lookup. When the bundle is unchanged, which is the common case, the snapshot is never opened. The workflow resolves it once per engine process..ultrafuzz/schemasand rewrites a copy that differs. That copy is only the agent's view; the host and the run's validator check the bundle itself. Path safety (symlinks, hard links, regular files, the digest check after copying) is unchanged.plannedOutputSchemarefuses a schema-backed output that does not name its schema in the run's planned bundle (<path> does not name its schema in the run's planned schema bundle), where it used to returnundefinedand let the validator fall back to this build's schema.parseJsonValidatorPreflightSuccessEnveloperequires the caller to name the expected identity. The preflight passes the planned bundle's findings identity, and runs only for a task with a schema-backed output: a plan with none names no bundle, and its preflight compared the run's own validator with this build's bundle.validateArtifactContractBytestakes it as an optional fourth argument and no longer compares the validator build the worker reports. Synchronization counts the array it validated instead of re-parsing it withvalidateFindingsSchema.verified dependency artifact is no longer validerror now carries the issues, such as a timeout.renderCurrentSmithersControllerno longer rebinds outputs (taskWithCurrentArtifactSchemas), and thereplacePromptSchemasflag and its__ULTRAFUZZ_REPLACE_PROMPT_SCHEMAS__placeholder are gone. After a schema change the refresh keeps the run's own validator launcher:prepareTrustedCliEnvironmentno longer tries to rotate a sound launcher to an installed CLI whose schema bundle differs from the recorded one, a rotation the candidate preflight always refused, so resume no longer reportsWORKFLOW_TRUSTED_CLI_UNVERIFIEDfor it.materializePromptSchemas(schemaDirectory)or passes it{ replaceExisting }. A plainresumeof such a run now fails withWORKFLOW_CONTROLLER_REFRESH_REQUIREDbefore Smithers starts, namingultrafuzz resume <run-id> --refresh-controller, instead of failing every remaining task with aTypeErrorabout thepathargument.materializePromptSchemasnames the same refresh (with--retry-failed) for an engine that the supervisor relaunches from an install upgraded in place.sealedArtifactSchemaDirectoryreads onlyexecutionSnapshot; nothing has writtencontrollerExecutionSnapshotsince refactor(runtime): delete dead controller-generation, sealed-refresh, re-finalization and recovery-authority code #1193.Files:
@ultrafuzz/artifactsschema-bundle.ts,artifact-contracts.ts,json-validator-preflight.ts;@ultrafuzz/runtimeworkflow.tsx,smithers.ts,artifact-gates.ts,start-run.ts,trusted-cli.ts,workflow-sync.ts;eslint.config.js(the removed placeholder). None of the five validator-build modules (json-file-validator,json-schema-validator,json-validation-worker,schema-registry,strict-json) and no schema file is edited, soVALIDATOR_BUILD_IDENTITYand every schema digest are unchanged. Theworkflow-sync.tshunks are the findings count infinalizeTerminalTaskand one helper, away from #1198's optional-dependency code;smithers.tshunks are one import, the threereplacePromptSchemaslines andrenderWorkflowSource.Deliberately not built
verifyCanonicalPropertiesMarkdownPair,verifiedCanonicalPropertyCatalog), the invariant-suite checks (materializeInvariantSuiteCompanions,assertInvariantSuiteDependencyExpectations) and the property-coverage check (authoritativeFinalReportCoverage); on the host, the artifact-gate semantic contexts for properties, implemented-properties, property-campaign and campaign findings (finalizedCanonicalPropertyPair,semanticImplementedPropertiesArtifact,readDeclaredSiblingImplementedProperties,semanticCampaignArtifacts,readCampaignFuzzerBackends,readImplementedProperties). The final report's canonical projection validates a document this build derives, so a resumed run's report must satisfy both its planned schema and this build's projection. A$commentor a loosening does not affect any of them; an upgrade that tightens one of those schemas within the same contract version can still fail a resumed task: a producer in its verifier, after its agent ran, and a consumer at preparation, generation or verification (the base failed every task before its agent instead). Deleting only the workflow's copies would move a consumer's rejection to the host's semantic contexts at sync (finalizedCanonicalPropertyPair, for example), where the engine would have verified an attempt thatstate.jsonrecords as failed; each reader needs the planned binding threaded to it, host and workflow together. CHANGELOG, SPECS,schemas.mdandrestart-continue.mdsay this.validateArtifactContractBytes's optional fourth argument still defaults to this build's schema. Every caller that validates a run's planned artifact now passes the planned schema, and the workflow'splannedOutputSchemathrows instead of returningundefinedfor a JSON output, so the default is unreachable from the workflow. The remaining callers validate text contracts, this build's own documents (reference and vulnerability-database manifests, the report projection, contract examples,ultrafuzz artifact validate) or, inreadDeclaredSiblingImplementedProperties, an artifact in the typed-reader class above. Making the argument required touches eight callers in four packages without changing behaviour; it belongs with threading the planned bundle into those readers.registeredSchemaForPath(schema-registry.ts) lost its only caller. Deleting it edits a validator-build module and changesVALIDATOR_BUILD_IDENTITY(provenance only since fix: a validator rebuild no longer strands in-flight runs (validator build becomes provenance) #1188), which this PR avoids; it is a follow-up.resumeruns a pre-change run's launch workflow; this PR's entry says it replaces that note, instead of editing a line that refactor(runtime)!: a run's prompt files are records, not seals #1230's own follow-ups rewrite (the rebase onto 9b42ae0 conflicted there twice).Verification
Discriminating tests. Each fails on
origin/claude/prompt-records-not-seals(94123ad, before #1230's two newest commits, which touch neither schema bundles nor preparation) or with its fix reverted on this branch, and passes here:runtime.test.ts› "a run resumed after an upgrade changed its schemas finishes its tasks against the schemas it was planned with". It launches a three-task run throughstartRun:summarizeis text-only;project-discoveryhas markdown andfindings.json, and keeps the launch schema copy in its worktree;reviewdepends onproject-discovery. It then renders the run's project workflow in process the way a native resume does (the sharedtest/in-process-workflow.tsloader, which the render test now also uses), against a copy of@ultrafuzz/artifactsin whichreport.schema.jsongained a$commentand the findings schema accepts any non-empty array, with the run's owntrusted-binlauncher first onPATH.summarizeandproject-discoveryverify, and both workspaces end with the planned schema bytes.[{"title": …}], which the upgraded findings schema accepts, still fails verification against the planned one.[], which the upgraded schema rejects, verifies with the planned binding in its marker, andreview's admission accepts it.prepare:summarize failed at step preflight-json-validator: … mismatched identity. With the verifier reverted to this build's schema it accepts[{"title": …}]; with admission reverted,prepare:reviewfails withverified dependency artifact is no longer valid findings.json: must NOT have fewer than 1 items.runtime.test.ts› "a run whose plan has no schema-backed output still prepares its tasks after an upgrade changed a schema": a one-nodeultrafuzz/text@1plan after a$commentupgrade. Without the per-task preflight skip it fails withmismatched identity, the review's probe X1.runtime.test.ts› "sync counts an attempt's findings against the schema the run planned them with after an upgrade changed it":syncRunruns in a child process whose@ultrafuzz/artifacts, for the runtime and every package, is a copy withmaxItems: 0on the findings schema. Here the node succeeds withfindings_count: 1; with sync's old validation it fails withFINDINGS_VALIDATION_FAILED … must NOT have more than 0 items.runtime.test.ts› "a plain resume refuses a workflow rendered before task preparation took the planned schema bundle": the persisted workflow is given each earliermaterializePromptSchemascall; resume fails withWORKFLOW_CONTROLLER_REFRESH_REQUIREDnaming--refresh-controller, no Smithers command runs, andresume --refresh-controllerthen submits.runtime.test.ts› "current-controller rendering preserves prompts idempotently and continue policy for a leaf task". The refreshed controller keeps a historical output's whole binding (C4). On the base it is rebound (schemaBundleSha256: '6a05a7a6…').trusted-cli.test.ts› "controller refresh keeps a sealed launcher that validates with the run's schemas after this build's changed". With the rotation attempted it fails withmismatched identity, which resume reported asWORKFLOW_TRUSTED_CLI_UNVERIFIED.schema.test.ts› "an artifact that a sealed bundle accepted is not validated again against the same registry" (fails with the memo disabled), "schema materialization names the refresh that a workflow rendered by an earlier release needs", "a run's planned schema bundle is the one its plan names, not the one this build installs" and "schema materialization replaces a workspace copy that differs from the bundle".generated-workflow-verifier.test.ts› "the generated workflow validates a declared output against its schema in the run's planned bundle, never this build's", "the generated goal-search census counts a lane's findings validated against the run's planned schema" and "generated validator preflight expects the run's planned schema identity, not this build's".json-validator-preflight.test.ts› "validator preflight parser compares the reported identity with the caller's, not this build's" passes on the base too, which already honoured an explicit identity; it pins the caller-named comparison, not the removed default, which the required parameter now enforces at compile time.End to end on the real engine (Bun, the pnpm-patched Smithers runner, the run's own trusted CLI, and a stub
codex):packages/cli/test/e2e/campaign-resume.test.ts(shipped, unchanged) passes on this head in 4m42s."$comment"to the installedpackages/artifacts/schema/report.schema.jsonafter the kill, continues withresume --refresh-controller, and restores the file afterwards. Resume returned no diagnostics, the run endedRunFinished, and the report wasavailable/complete/verified(4m39s). The previous round ran the same probe on 221a6fc: with--refresh-controllerit ended the same way but resume reportedWORKFLOW_TRUSTED_CLI_UNVERIFIED … mismatched identity; with a plainresumeit endedRunFinishedtoo, while on 94123ad a plainresumeendedRunFailedwith map 4'smaterialize-prompt-schemaserror verbatim.Other suites on this head:
artifact-gates,trusted-cli,verified-output,workflow-dependency-policy,workspace-preparation-lifecycle,generated-workflow-verifier,generated-workflow-rendertogether: 358/358 (the lifecycle file passes since refactor(runtime)!: a run's prompt files are records, not seals #1230's harness fix, b970047)dynamic-lifecycle: 20/20runtime.test.tstests matching schema, preflight, trusted, validator, bundle, refresh, current-controller, native continuation, planned schema, removed per-node cloud or sync counts: 35 pass; the 6 skips are Bun-only adapter contractsjson validate: 16/16Gates on this head, each exit 0:
npx prettier --checkon the changed filespnpm -w lintCI=1 ESLINT_PLUGIN_DIFF_COMMIT=origin/claude/prompt-records-not-seals pnpm -w lint:strict:cipnpm -w knip, on a fresh unbuilt checkout of this head (pnpm install --frozen-lockfile --ignore-scripts, nodist) and afterpnpm -w buildpnpm --filter @ultrafuzz/artifacts --filter @ultrafuzz/runtime --filter @ultrafuzz/cli typechecknode scripts/docs-check.mjspnpm -w format:checkRisk / compatibility
resumeof one fails withWORKFLOW_CONTROLLER_REFRESH_REQUIREDbefore Smithers starts, whether or not a schema changed; continue it withultrafuzz resume <run-id> --refresh-controller, on every resume, because a plainresumeruns the launch workflow even after a refresh. No task fails, so no--retry-failedis needed. An engine that the supervisor relaunches from an install upgraded in place (whichdocs/reference/cli.mdalready advises against) fails each task atmaterialize-prompt-schemaswiththis run's workflow was rendered by an earlier Ultrafuzz release …; that run needsresume --refresh-controller --retry-failed, which reruns a failed verifier from its agent.parseJsonValidatorPreflightSuccessEnvelope(bytes, expected)requiresexpected.materializePromptSchemas(destination, bundle)requires the bundle and dropsreplaceExisting.CompiledSmithersWorkflow.replacePromptSchemasis gone.--refresh-controller, which reverses refresh-controller cannot retry tasks across schema bundle upgrades #982's intent; start a new run to use them. A task's.ultrafuzz/schemascopy is no longer tamper-evident: preparation rewrites a copy that differs without refusing or recording it (the run's own validator still rejects a schema file whose bytes are not the planned ones).docs/security.mdand the CHANGELOG say both.state.jsonand the sealed snapshot's schema directory, as the host gates already do. A run whose snapshot is gone then fails atmaterialize-prompt-schemaswith the lookup error instead of the equality error. A run whose bundle is unchanged never reads them.schemaRegistryBundleDigestequals the planned digest. That digest also covers each entry'smaxInstanceBytes, which the loader takes from this build. A future build that changes that limit would make every sealed bundle look foreign, for the host gates as well as for the workflow.workspace-preparation-lifecycle.test.tspass again; this PR removes one stale stub from that harness, in a different hunk.Changelog
One entry under
## Unreleased› Breaking changes, after #1230's. It covers the removed failures, which checks use the planned bundle and which still use this build's, the refresh change, the sealed-bundle deadline, the API breaks, what is given up, and how to continue a run launched earlier.Greptile follow-up
artifact-gates.ts:3348): valid, fixed in 679cd72, a fifth commit on top of the four above. The host gateverifyRequiredArtifactSchemaBinding, which synchronization's artifact gates and verified reads run, validated against the sealed bundle with the validator's 5 s default for ordinary-size contracts, while the workflow's checks gave that bundle 30 s. A first compile between 5 and 30 s therefore failed at sync (JSON_VALIDATION_TIMEOUT) an attempt that the verifier had accepted. Both now call one new export,validateJsonBytesAgainstBundleSync(artifact-contracts.ts), which gives any bundle other than this build's the 30 sSEALED_BUNDLE_VALIDATION_DEADLINE_MS. The installed bundle keeps the default, and no validator-build module changes, soVALIDATOR_BUILD_IDENTITYand every digest are unchanged. The CHANGELOG sentence on the deadline now names the host's checks. Test:artifact-gates.test.ts› "host artifact validation gives a run's sealed schema bundle the deadline its workflow gives it" movesDate.now10 s ahead at the validator's first wait for its worker, a deterministic 10 s compile. The validator's default times out, the workflow's check passes, and the host gate returns no diagnostics. On 72a65c4 the gate returnsJSON_VALIDATION_TIMEOUT; here the test passed 5 of 5 runs. Not changed: the host gate still compiles the sealed bundle for every validation, because it has no acceptance memo and loads the sealed registry on each call. Under load, a host command can now block for up to 30 s per artifact where it used to fail at 5 s. Re-run on this head, each exit 0:pnpm -w format:check,pnpm -w lint,CI=1 ESLINT_PLUGIN_DIFF_COMMIT=origin/main pnpm -w lint:strict:ci,pnpm -w knip, artifacts and runtimetypecheck, andnode scripts/docs-check.mjs. Tests: artifactsschema,contract-fixturesandartifact-validation, 88/88;artifact-gatesmatching schema, sealed, binding or contract, 26/26;verified-output3/3;generated-workflow-verifiermatching planned, 5/5; andruntime.test.ts's historical-bundle, upgraded-resume, text-only-plan and sync-count tests, 4/4.Atomics.wait, and its self-check, the validator's default timing out, covered only a direct validator call. Had a later change put a wait on the host gate's path before the validator sets its deadline (a lock acquisition, say), the clock would move first and the host-gate assertion would pass with any deadline. The host gate must now also time out on a simulated 40 s compile, past the validator's 30 s cap, which it does only when the simulated compile is its own wait for its worker. Mutants: the host gate without the sealed deadline fails the old and the new test at the 10 s assertion; anAtomics.waitadded before its validation, with or without that deadline, passes the old test and fails the new one at the 40 s assertion. Moving the clock only after aWorkeris constructed, as the review suggested, would neednode:worker_threadspatched andmodule.syncBuiltinESMExports()to reach the validator's import; the 40 s check needs no patching and also bounds the host gate's deadline from above.mainthe host gate already maps every result that is notvalid, including asetup-error(JSON_VALIDATION_TIMEOUT; a worker death also surfaces as that timeout), to a severity-errorartifact-schemadiagnostic, and synchronization then recordstask-output-validation-failure, which seals that occurrence untilresume --retry-failedor--reset-nodereruns it. The installed bundle, which every run uses, validates with the 5 s default, and on the first validation in each process that budget also covers the served worker's start and compile: a probe on this head (not committed) with a simulated 10 s compile gotJSON_VALIDATION_TIMEOUT(severityerror, sourceartifact-schema) for the installed bundle, and the same bytes passed on the next call, on the real clock. This PR leaves that path unchanged and gives sealed bundles 30 s instead of 5 s. Fixing it means choosing, for synchronization's gates, its failed-verifier recheck and verified reads, between failing the pass and recording the failure without sealing it so that a later sync retries, for every run and not only resumed ones; that belongs in its own change with its own tests.pnpm -w format:check,pnpm -w lint,CI=1 ESLINT_PLUGIN_DIFF_COMMIT=origin/main pnpm -w lint:strict:ci,pnpm -w knip, runtimetypecheck, andnode scripts/docs-check.mjs. Tests: the deadline test, 5 of 5 runs;artifact-gatesmatching schema, sealed, binding or contract, 26/26.🤖 Generated with Claude Code
The PR does not appear ready to merge because the host-gate deadline regression test expects a diagnostic the gate does not return.
Fix with agent prompt
Summary
The PR makes resumed runs validate artifacts against their planned schema bundle rather than an upgraded installation’s schemas. It also preserves planned bindings on controller refresh and extends sealed-bundle validation deadlines for the workflow and host. The follow-up test still expects the wrong diagnostic when a host validation times out.
Diagram
%%{init: {'theme': 'neutral'}}%% flowchart LR Plan[Planned output binding] --> Bundle{Installed bundle digest matches?} Bundle -->|Yes| Installed[Installed schemas] Bundle -->|No| Snapshot[Sealed execution-snapshot schemas] Installed --> Validation[Workflow and host validation] Snapshot --> ValidationReviews (7) · Last reviewed commit: "test(runtime): pin the host gate's seale..."