refactor(artifacts): delete never-dispatched gates, production-dead code, and the unread event index

[aviggiano]

Problem

packages/artifacts ships a lot of code that production never runs, and part of it slows down or can stop a campaign:

• Semantic gates. 48 of the 173 registered gates are never dispatched. The 12 planned-graph gates duplicate assertPlannedGraphSemantics, which the runtime does call, and at least one copy had drifted from it: planned-graph-artifact-dir-identity required artifacts/<id> for every node, while assertPlannedGraphSemantics accepts the storage/attempt directory of dynamically generated nodes.

• Dead code. Several modules and functions have no production caller, and some have large test suites: finding-provenance.ts, the generated-test manifest writer/reader, appendLineDurable, truncateDurable, buildRunArtifactIndex, loadOrCreateRunState, validateWorkflowContract, resolvePathInside and about 60 declaration-only exports.

• Event journal. appendEvent validated the whole journal several times per append:

  • It pre-read events.jsonl plus up to five events.index/* fan-out files.
  • Each append then read its file again.
  • appendStrictJsonlRecords read the file a third time after writing.

  So append cost grew with journal size, and a run's total append cost grew quadratically. Separately, the event codec inherited the generic 100,000-record cap. Once a journal reached it, every later append and replay threw, and the sync, cancelRun and other lifecycle paths all append events.

• Parity throws. The findings and property validators ran the registered JSON Schema, which is documented as authoritative. They then parsed the document again with Zod and threw internal schema parity invariant violated whenever Zod disagreed. These validators back workflow-sync and the artifact gates.

Root cause

• Gates are registered per schema file, including runtime-state schemas. Production only dispatches:

  • artifact-contract schemas (workflow.tsx, artifact-gates.ts, artifact validate);
  • report.schema.json (modal);
  • the validator preflight, the invariant source proof and the attempt/usage ledgers;
  • a fixed set of gates called by name.

  No caller ever passed the planned-graph, task-manifest, artifact-verification/manifest, agent-source-proof, run-metadata/plan/source-run, config-redactions, invariant-suite-manifest, single-finding, run-state-fingerprint or analysis-bundle-digest schemas.

• The events.index/ fan-out and its query-inputs.json facade have no reader. queryEvents filters events.jsonl, and the only thing that touches the directory is the report bundle, which copies it.

• The 100k cap is the strict-JSONL default. The operator journal never overrode it, and neither did the report bundle's private copy of the journal parser.

Change

1. Delete the 48 never-dispatched gates (4c64f583). This removes:
   • their registrations, exclusive handlers and helpers;
   • their names in ARTIFACT_SCHEMA_METADATA;
   • the context fields only they read (plannedGraph, runtimeState, git.refs) and the one runtime line that built the unused plannedGraph context;
   • the tests and fixtures that only exercised these gates. The single-finding provenance cases now run through the live findings-campaign-provenance-coherence gate.
2. Delete production-dead code in artifacts and security (6c3c69e0). Each symbol was checked with git grep -w over the whole repository (every package's src and tests, scripts, the workflow templates) and with git log -S over the workflow template's history; none has a production caller. Their tests go with them. The two usage-ledger tests that used appendLineDurable to inject a bad line now use fs.appendFileSync. In security this removes resolvePathInside and its helpers, the UnsafeModeAudit* types, and PolicyResult.audit, which no caller ever set.
3. Event journal (5ae10beb):
   • appendEvent writes only events.jsonl. It checks the new record against the final record plus any trailing records with the same timestamp, using appendStrictJsonlRecordsAfterTail. The event ID hashes the timestamp and timestamps never decrease, so a record outside that window can repeat the ID only through a 96-bit hash collision.
   • replayEvents, queryEvents and the new parseEventJournalBytes still validate the whole journal. The byte length read before the write still fences it.
   • appendStrictJsonlRecords returns the snapshot it validated plus the fenced append instead of re-reading the file.
   • The event codec has no record cap. Its 64 MiB byte limit still applies.
   • createRunLayout no longer creates events.index/ or query-inputs.json, and RunLayout.eventsIndexDir is gone.
   • report bundle validates its journal snapshot with parseEventJournalBytes. This deletes its ~80-line copy of the parser, which also enforced the 100k cap.
   • Docs (artifacts-reports.md, SPECS.md) are updated.
4. Return the Ajv verdict (eaa1ed59). The findings and property validators drop the second Zod parse and its throws. The Zod schemas still generate the JSON Schema and the TS types, and contract-fixtures.test.ts still checks that Ajv and Zod agree on every fixture, so drift fails CI instead of a campaign.
5. CHANGELOG entry (15944d03).
6. <!-- markdownlint-disable MD013 --> at the top of CHANGELOG.md (638ff30f). Every entry there is one line past markdownlint's 400-character limit, so any PR that touches the file fails External static analysis, and that failure skips the release-validation lanes. fix(runtime): resume --retry-failed reopens descendants skipped behind a recovered producer #1169 adds the same line.

LOC delta: +285 / −4,660 (net −4,375). Source is −2,530, tests −1,851, docs/CHANGELOG +6.

Deliberately not built (and why)

• writeArtifact stays. It has no production caller, but about 350 test call sites across three suites use it as a fixture helper. Moving it into each suite would add code, and one of those suites (runtime/test/artifact-gates.test.ts) is being rewritten in parallel.
• The event-query-facade.schema.json schema, its Zod twin and its JSON export stay registered. Deleting a schema file changes the schema-bundle digest, and that strands in-flight runs.
• The CLI bundle manifest still lists events.index in included_roots. The CLI manifest schema pins that constant list, and changing it is a schema change. The bundle still copies the directory when an older run has one.
• The 64 MiB journal byte limit is unchanged. Readers load the whole journal into memory, so raising it is a separate decision. At about 454 B per workflow-synced record it holds roughly 148k records.
• No positioned tail read. An append still reads the journal's bytes. Only the parse is windowed, and the byte read is what remains of the per-append growth in the numbers below. A positioned read would duplicate the hardened reader in schema-registry.ts, which must not be edited without rotating the validator identity.
• Other readers not changed.
  • evals/src/node-telemetry.ts keeps its own 100k cap on run journals. It is eval-only, and the evals dead-code work item covers that file.
  • analysis-bundle.ts has its own Ajv/Zod disagreement path, which returns issues rather than throwing.
  • Security's validateSafeId(label, id) still shares a name with the artifacts version. It is not dead code (materialize-policy uses it).
• One stale test title. The planned graph v4 validates whole documents and executes every registered document semantic gate test keeps its title and gate-style rule labels, although planned-graph now has no registered gates; the test itself exercises assertPlannedGraphSemantics. Retitling touches the first line of a 176-line test function, so the diff-limited strict lint reports max-lines-per-function on it, and the validator-build work item is editing that test.
• Ledgers unchanged. Attempt and usage ledgers still validate their whole history on append. They need it for idempotent identity replay, and that code belongs to the ledger work item.

Verification

Discriminating tests

• event appends and replays keep working past 100,000 records (artifacts.test.ts): fails on main with event journal exceeds the record limit, and passes here. I ran it by copying the test file into a main worktree.
• an event append refuses a repeated or out-of-order event without changing the journal passes on both main and this branch. It fails with Missing expected exception when I temporarily shrank the window to the final record only (() => false). So it guards the same-timestamp window, not the old behaviour.
• Part 4 has no discriminating test. I know of no document on which the two validators disagree: the contract fixtures pin their agreement, and I probed timestamp, integer and string edge cases without finding one. On such documents the change only removes the second parse.

Identity unchanged. I compared artifactSchemaBundleDigest(), VALIDATOR_BUILD_IDENTITY, all 44 contract digests and all 70 schema SHA-256s between a main build and this branch: they are byte-identical. pnpm --filter @ultrafuzz/artifacts schema:check passes, and no schema/*.json, validator-identity module or contract description changed.

Dispatch analysis. I computed the gate set from the built registry: registered names minus the gates of every schema file a dispatch site can pass, minus the gates called by name. The result is exactly 48 gates, and none of the 48 names appears outside the artifacts gate module, metadata and tests.

Benchmark. CPU per appendEvent, measured with fsync disabled (eatmydata) on a 32-core host at load average about 38 (noisy):

existing events	main	branch
250	17.8 ms	4.8 ms
10,000	478 ms	12.5 ms
50,000	2,183 ms	39 ms

Suites and checks run

• artifacts full suite: 322/322.
• security full suite: 21/21.
• runtime:
  • getRunStatus counts only a canonical event-v2 journal and fails closed on invalid presence: pass. This is the corrupt-journal status test.
  • aggregation-semantic-context, audit-contracts, verified-output, workflow-dependency-policy, semantic-artifact-context, forge-guard, artifact-gates and generated-workflow-verifier: 404/404 pass.
• cli: every report bundle and stats test (--test-name-pattern='report bundle|^stats', --test-concurrency=1): 54 of 55 passed. report bundle --require-verified rejects manifest bytes injected only into the recursive archive read failed in its setup. There, ultrafuzz run returned WORKFLOW_SUBMISSION_FAILED: workflow execution file dependencies/packages/000006/LICENSE changed while reading while I was rebuilding the workspace in parallel. Rerun alone, it passes on this branch and on main. report bundle --require-verified fails closed on every present invalid event journal passes; it exercises the parser the bundle now shares.
• CI=1 ESLINT_PLUGIN_DIFF_COMMIT=origin/main pnpm -w lint:strict:ci, eslint and prettier --check on the changed files: pass.
• pnpm --filter @ultrafuzz/{security,artifacts,runtime,cli} typecheck: pass. tsc --noEmit for every other package against the new builds: pass.
• pnpm -w knip and node scripts/docs-check.mjs: pass.

PR CI on 638ff30f: every check passes. That includes the four runtime integration shards and the runtime support/Bun adapter-contract lane, which together run the whole runtime suite.

Not run: the whole cli.test.ts suite (only the patterns above; the CLI lane is push-only), and the modal suite. The evals and dashboard suites ran only partially: evals recovery-equivalence.test.ts 42/42, and the dashboard event|journal|audit tests 4/4. None of those packages imported any removed export, and all of them typecheck.

Risk / compatibility

• Removed public exports. @ultrafuzz/artifacts and @ultrafuzz/security lose exports: the gate names in SemanticGateName, the functions listed above, appendEventRecord, the facade reader/validators, and RunLayout.eventsIndexDir. No package in the repo uses them, and none has ever appeared in the workflow template (checked with git log -S). Generated workflows load @ultrafuzz/artifacts from their sealed execution snapshot, and none of their imports changed.
• Existing runs. Stale events.index/ directories stay on disk. Nothing reads them, and report bundles still copy them.
• Append validation is narrower. A journal already corrupted before the window, for example by a hand edit, is no longer rejected at append time. It is still rejected by the next read, and every sync replays the journal. The append applies the readers' rules to the new record and its window.
• Merge overlap.
  • semantic-gates.ts: the open Reconcile invocation usage and attempt gaps #1155 and the parallel timeout-evidence change edit it. The hunks here are the deleted handlers and registrations, away from propertyCampaignTimeoutEvidenceIssues.
  • artifacts.test.ts: the parallel ledger and secret-gate changes edit the attempt-ledger tests and the secret-gate test. This PR deletes other blocks and touches the import list.

Refs #462

🤖 Generated with Claude Code

Confidence Score: 5/5

The PR appears safe to merge based on the changes reviewed since the previous review.

Summary

The PR removes unused artifact gates and production-dead code, drops the unread event index, changes event appends to validate a trailing window, and uses the registered JSON Schema verdict for findings and property validation.

• Event replay and report bundling continue to validate complete journal snapshots.
• No new actionable issue was established in the changes since the previous review.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[appendEvent] --> B[Validate trailing timestamp window]
  B --> C[Append to events.jsonl]
  C --> D[replayEvents and queryEvents]
  C --> E[Report bundle]
  D --> F[Validate complete journal]
  E --> F

Loading

_{Reviews (3) · Last reviewed commit: "chore: move the changelog entry to the c..."}