refactor: delete dead prompt rename, config helpers, and CI scripts orphaned by #1131

[aviggiano]

Problem

Several features and helpers have no production caller but are still built, exported, documented and tested:

• prompts: the prompt rename feature (rename.ts, renamePromptArtifactReferences, diffPromptIdentity, serializePromptDocument). Output-contract templates also had a second asset resolver that still probed dist/prompts, a layout the build stopped producing in fix(prompts): build bundled assets into the directory the resolver reads #796.
• config: the prompt-metadata layer, restoreRedactedConfig, assertNoRedactionPlaceholders, applyDefaultProfileOverrides, assertResolvedConfigZod and resolvedConfigSchemaEntry. The prompt-metadata layer did run: resolveConfig applied it on every resolution. But createDefaultPromptMetadataLayer() always returned {}, and only one test passed promptMetadata, so in production it only ever applied an empty layer. docs/config.md described a restore step and a placeholder launch guard, and no code runs either one.
• ci: outside tests, the only callers of scripts/ci/validate-threat-model-benchmark-gate.mjs and scripts/ci/modal-benchmark-control-window.mjs were eval-benchmarks.yml and the watch-modal-benchmark.sh it ran. Remove paid benchmark CI and repair CI gates #1131 deleted both. The gate's 333-line test still ran in every PR's test:ci-scripts step. The control-window script stayed alive only through a knip entry and a ci-config.test.ts block.
• modal: exported constants and helpers that nothing references, or that only tests reference.
• dashboard: commandCapabilities() returned seven flags as the constant false (restartWholeRun, doctor, triage, merge, restartFromNode, rerunSelectedNode, arbitraryShell). The frontend only declares them in a type, and capabilityForCommand never maps a command to any of them.
• runtime: scripts/run-tests.mjs defines three selectors (materialize, clean, smithers) that nothing passes.

Root cause

These features were removed, or never wired up, but their library code and tests were left behind. CI cannot see this: pnpm -w knip runs with --include files,dependencies,unlisted,unresolved, so it does not report unused exports. And any file that a test imports still counts as used.

Change

One commit per area. Only deletions, apart from the resolver change.

1. refactor(prompts):
   • Deletes the rename feature and its tests, along with the frontmatter round-trip support only rename needed: the allowUnknownFields option, unknownFrontmatter, rawFrontmatter, the invalid-rename code and TemplateOccurrence.rawName.
   • Output-contract templates now resolve through builtInPromptRoot(), the packaged-first lookup the agent-preamble templates already use.
   • Removes the render test that rebuilt the old dist/prompts layout. Its content assertions move into the packaged-layout test, which copies the real dist under a directory with no repository above it (the sealed-snapshot case).
2. refactor(config):
   • Deletes the prompt-metadata layer, including PromptMetadataLayer, ResolveConfigInput.promptMetadata and the prompt-metadata diagnostic source.
   • Deletes restoreRedactedConfig and assertNoRedactionPlaceholders, plus their path helpers and the redaction diagnostic source.
   • Deletes applyDefaultProfileOverrides; its test now drives applyModelProfileOverrides, which validate.ts and plan-run.ts call. Also deletes assertResolvedConfigZod and resolvedConfigSchemaEntry.
   • resolvedConfigValidatorsAgree moves into its only test.
   • docs/config.md now states that no command restores values from the redaction manifest.
3. ci: deletes both scripts, the gate's test, the knip.jsonc entry, and the control-window block (~lines 206-297) of the EVMBench full-mode test in packages/modal/test/ci-config.test.ts. The manifest assertions around that block stay.
4. refactor(modal):
   • Deleted, no reference anywhere: MAX_MODAL_DOCUMENT_BYTES, MODAL_SMOKE_STOP_PATH, DEFAULT_NODE_TIMEOUT_SECONDS.
   • Deleted together with the tests that only exercised them: createTrackedSourceArchive, persistentWorkspaceRoot, and locateModalResumeWorkspace. The last one's "locates one exact linked durable run" test duplicated the existing tolerant-lookup test.
   • Moved into tests:
     • DEFAULT_BENCHMARK_MODELS becomes test/model-spec-fixtures.ts, typed as a const tuple.
     • modalBenchmarkConfigValidatorsAgree moves into its test.
     • WORKER_RESULT_ALLOWED_KEYS: the test now states the expected key set.
     • publicEvalFailureDiagnosticLogPayload: the same two calls runCommand already makes inline.
     • The two PUBLIC_*_MAX_PARALLEL_EVAL_ROWS aliases.
   • Un-exported five public-worker.ts constants and WORKER_STDERR_TAIL_BYTES; they are used only inside their own modules.
5. refactor(dashboard): drops the seven flags from the server, dashboard-http.schema.json and the frontend type. The schema edit is required: commandCapabilities is closed (required plus additionalProperties: false).
6. chore(runtime): drops the unused run-tests.mjs selectors.
7. CHANGELOG.md entry.
8. Review follow-ups:
   • refactor(prompts): leave getPrompt to the catalog work item restores packages/prompts/src/catalog.ts to origin/main. The earlier revision deleted getPrompt from it.
   • docs(changelog) tightens the entry's wording (see Verification).

Net: 47 files, +170 / −2164.

Deliberately not built (and why)

• getPrompt (packages/prompts/src/catalog.ts) has no caller either. But fix(topology): restore the review group time budget and remove prompt/gate contradictions #1164 adds projectPromptsDifferingFromBuiltIns directly above it, and catalog.ts is outside this item's files. Deleting it here made git merge-tree report CONFLICT (content) in catalog.ts against fix(topology): restore the review group time budget and remove prompt/gate contradictions #1164, so it is left for a follow-up after fix(topology): restore the review group time budget and remove prompt/gate contradictions #1164 lands.
• The retry-failure agent-preamble template is still referenced by smithers.ts (the __ULTRAFUZZ_RETRY_FAILURE_TEMPLATE__ injection) and by workflow.tsx:1666-1667, a few lines from the renderAgentPrompt region another work item is editing. Deleting only the prompts side would break workflow generation, so it stays.
• createPublicEvalDiagnostics is test-only, but it is the pure seam that 23 calls in public-eval-diagnostics.test.ts use to reach the private createPublicEvalDiagnosticsFromRecords. Moving it into tests would mean either exporting the private function or rewriting those tests over file fixtures. Kept.
• assertCurrentPersistentWorkerLineage stays exported. Un-exporting it touches a line that the strict diff lint rejects for an unrelated reason: it is async with no await.
• Modal's triple 24 h sandbox-timeout aliases. Production uses two of the three names, so collapsing them is a rename, not a deletion.
• hasRedactionPlaceholder (security): its only callers were the config helpers deleted here, so it now has none. packages/security belongs to another work item, so it is left in place.
• No new knip exports gate. CI gating belongs to another work item. For reference, pnpm dlx [email protected] --include exports,types,duplicates reports 40 unused exports on origin/main and 33 on this branch, and nothing new. The seven that leave the report are all modal:
  • six are un-exported here: the five public-worker.ts constants and WORKER_STDERR_TAIL_BYTES;
  • the seventh, publicEvalFailureEnvelopeDiagnostics, is still exported. It leaves the report only because the rewritten public-worker.test.ts now imports it. Production uses it inside public-worker.ts.
• Out of this item's scope:
  • the snake_case aliases on PromptConcreteNode/PromptGraphNode
  • config keys that are parsed but never read
  • RuntimeConfigOverrides.triageQuorum/triagePanelSize
  • dead exports in topology and references
  • the automatic-publication path in scripts/ci/prepare-eval-history-publication.mjs, also orphaned by Remove paid benchmark CI and repair CI gates #1131

Verification

Deletions. For every deleted or un-exported symbol, a grep over packages/, scripts/, docs/ and the runtime templates (excluding dist, dist-test and node_modules) finds no remaining reference, apart from the test-local helpers this PR adds.

At a49a626f^, the parent of #1131, the scripts' only invocations outside tests were:

• .github/workflows/eval-benchmarks.yml lines 335, 408, 518 and 600 (control-window) and 735 (gate);
• scripts/ci/watch-modal-benchmark.sh:24 (control-window), which eval-benchmarks.yml ran at lines 426 and 620.

#1131 deleted both files. The other two workflows #1131 deleted never invoked either script.

Discriminating evidence. This PR deletes code, and only two behaviours change.

• Output-contract template lookup. The supported layouts (packaged dist/assets/prompts and the source tree) resolve as before. The packaged-layout test covers the sealed-snapshot case and passes. The deleted installed-layout test was the only thing that built dist/prompts.
• Dashboard capabilities. Every dashboard test response is validated against the schema. As a mutation check, I re-added doctor: false to commandCapabilities() in the test build of the server, packages/dashboard/dist-test/src/index.js, which is the file the dashboard test imports.
  • With the mutation, serves logical topology flow with expanded attempt details fails with /capabilities additionalProperties: must NOT have additional properties. It passes before the mutation and again after the file is restored. I re-ran this on this revision.
  • The same edit to dist/index.js does not fail the test, because the test does not import that build.

Tests run on e6c395c1 (targeted). The follow-up commits only restore catalog.ts to origin/main and edit CHANGELOG.md, so they cannot affect any suite except prompts, which is re-run below.

• prompts: full vitest suite, 98/98.
• config: full vitest suite, 92/92.
• modal: vitest over config, layout, resume, runner, worker-result, worker-lineage, workspace-config, public-worker, ci-config, modal-documents, smoke and worker-diagnostics.
  • Everything passes except three public-worker tests, which exceed their explicit 30 s budget: recognizes and cleans the legacy persistent public workspace…, rejects a present dangling public bundle… and accepts the bounded full lane….
  • The same three time out the same way on an origin/main worktree on this machine (load average 30-39), so the timeouts are not caused by this PR.
  • After the fixture-tuple change I re-ran config, worker-lineage, workspace-config, worker-result, layout and resume: 115/115.
• dashboard: contracts 4/4; frontend-wire-contracts + dashboard 34/34.
• pnpm -w test:ci-scripts: 94 pass, 1 fail.
  • The failure is safe-archive-bun.test.ts > repeatedly extracts a synthetic archive without stalling Bun callbacks, a 15 s timeout. It also times out on origin/main here, and this PR does not touch safe-archive.ts.
  • prepare-eval-history-publication.test.ts passes 17/17. It includes trustedCandidateRuntimePolicyDimensions(process.cwd(), "smoke"), which parses the real public-worker.ts after the un-exports.
• runtime: dynamic-expansion 15/15. It materializes dynamic children, which calls renderPrompt and so loads the output-contract templates through the new lookup. prompt-artifact-authority + model-override-validation 10/10.

Re-run on this revision (d7704ae7):

• prompts: full vitest suite 98/98, and tsc --noEmit.
• npx prettier --check and npx eslint --max-warnings 0 over all 47 changed files.
• CI=1 ESLINT_PLUGIN_DIFF_COMMIT=origin/main pnpm -w lint:strict:ci, pnpm -w knip, node scripts/docs-check.mjs and node scripts/prompt-catalog-docs.mjs --check.
• the knip exports comparison above, run on both trees;
• the dashboard mutation check above.
• git merge-tree --write-tree against the 25 other open work-item branches. The only conflict is CHANGELOG.md, which every entry touches. The catalog.ts conflict with fix(topology): restore the review group time budget and remove prompt/gate contradictions #1164 is gone.

Static checks on e6c395c1:

• pnpm --filter @ultrafuzz/{prompts,config,dashboard,modal,runtime} typecheck.
• tsc --noEmit for topology, references, runtime, evals, evmbench and cli.

Not run: the full runtime and CLI suites, CI itself, and any real campaign.

Risk / compatibility

• Removed library exports. @ultrafuzz/prompts, @ultrafuzz/config and @ultrafuzz/modal are private workspace packages, and no in-repo caller of a removed export remains. parsePromptFrontmatter no longer takes an options argument; unknown frontmatter keys already failed by default, and no caller passed allowUnknownFields.
• Resolver error wording. builtInPromptRoot() rethrows non-ENOENT stat errors, where the old output-contract resolver moved on to its next candidate. Rendering fails either way; only the error message differs.
• In-flight runs. None of the five validator modules changed, so the validator build identity is unchanged. No artifact schema changed. The dashboard HTTP schema feeds neither the validator build identity nor the artifact schema bundle digest that planned outputs and the trusted CLI identity bind to.
• Execution snapshots. Snapshots copy each package's whole dist. Since fix(prompts): build bundled assets into the directory the resolver reads #796 that contains dist/assets/prompts and no dist/prompts, so dropping that probe matches every current build.
• Parallel work in shared files.
  • packages/modal/test/ci-config.test.ts: this PR's hunk is ~206-297; the ci.yml string assertions (~625-808) are untouched.
  • docs/config.md: only the three-line "Redaction" paragraph changes.
  • CHANGELOG.md: one entry at the top of "Other changes".
  • Merge order with ci: validate every package on PRs, stop cancelling main runs, and add a global complexity ceiling #1184's lint baseline. ci: validate every package on PRs, stop cancelling main runs, and add a global complexity ceiling #1184 adds eslint-suppressions.json, and its pnpm -w lint fails when a suppression no longer matches a violation. Violations this PR deletes therefore make that file stale.
    • I overlaid ci: validate every package on PRs, stop cancelling main runs, and add a global complexity ceiling #1184's eslint.config.js, eslint-suppressions.json, package.json and the merged ci-config.test.ts on this branch. ESLint over this PR's 38 changed lint targets exited 2 with There are suppressions left that do not occur anymore. Linting only the merged ci-config.test.ts also exits 2, while ci: validate every package on PRs, stop cancelling main runs, and add a global complexity ceiling #1184's own copy of that file lints clean against its own baseline.
    • --prune-suppressions then removes the three scripts/ci/validate-threat-model-benchmark-gate.mjs entries (complexity, max-lines-per-function, max-statements) and lowers packages/modal/test/ci-config.test.ts max-lines-per-function from 2 to 1.
    • Whichever of the two merges second needs pnpm -w lint:prune and the smaller baseline committed.

Refs #462

🤖 Generated with Claude Code

Confidence Score: 5/5

The PR appears safe to merge; no actionable new issue was established.

Summary

This PR removes unused prompt, configuration, Modal, CI, dashboard, and runtime code. Output-contract templates now use the shared packaged-first prompt resolver, and the dashboard contract drops seven always-false capability flags. A follow-up removes this PR’s changelog entry for consolidation elsewhere.

_{Reviews (3) · Last reviewed commit: "chore: move the changelog entry to the c..."}