Skip to content

Implement ProjectData terminal archive-sharding bridge - #1983

Draft
raphaeltm wants to merge 1 commit into
sam/continue-zero-loss-projectdata-j6dat0from
sam/execute-task-using-skill-87btkd
Draft

Implement ProjectData terminal archive-sharding bridge#1983
raphaeltm wants to merge 1 commit into
sam/continue-zero-loss-projectdata-j6dat0from
sam/execute-task-using-skill-87btkd

Conversation

@raphaeltm

Copy link
Copy Markdown
Owner

Summary

Implements the Fable-approved ProjectData terminal archive-sharding bridge from tasks/active/2026-08-31-projectdata-terminal-archive-sharding.md.

  • adds the disabled-by-default external Worker coordinator with D1 CAS journal/lease/fence state and source-local intent fencing
  • adds exact owner/generation/location routing that fails closed on ambiguity or migration state
  • migrates terminal transcript payloads in bounded idempotent chunks with canonical per-column SHA-256 verification and immutable paged R2 recovery evidence
  • preserves root anchors, summaries (including last_message_at), comments, lifecycle/control state, and rejects unsafe late writes/wakes
  • adds capacity-gated rehome/root copyback, exact explicit fallback placement, recovery/finalize reconciliation, storage measurement via sql.databaseSize, and the unsafe-deploy routing guard
  • creates follow-up SAM Idea 01M1CZT4PEWX62RS4V95DT48YS for active-session SessionData direct ownership

Migrations

  • D1: 0132_project_data_terminal_archive_sharding.sql
  • D1: 0133_project_data_archive_recovery_cursors.sql
  • ProjectData DO: 043-projectdata-terminal-archive-sharding
  • ProjectData DO: 044-projectdata-archive-paged-manifests

All migrations are append-only. Archive sharding remains production-disabled by default and fails closed.

Validation

  • ProjectData archive Workers suite: 6/6 passed
  • focused archive/routing unit and integration suites: passed
  • superadmin route security/contracts: 11/11 passed
  • wake/recovery suites: passed
  • TaskRunner exact-root Workers: passed
  • deploy guard: 4/4 passed
  • API typecheck, build, lint: passed
  • root typecheck: 19/19 workspace tasks passed
  • migration safety/order, Wrangler bindings, format ratchet, and AST checks: passed
  • Cloudflare/constitution, security/env/docs, and test/task-completion specialist reviews: PASS on the reviewed tree

Safety / review state

DRAFT — DO NOT MARK READY OR MERGE.

No staging or production deployment occurred. No production configuration was changed. No production migration, cleanup, enablement, or raw transcript/message deletion was run. The coordinator remains explicitly disabled unless PROJECT_DATA_ARCHIVE_SHARDING_ENABLED=true.

The remote commit is a connector-published exact tree snapshot of reviewed local head 3341e54616f0d746204d6eba18a323025ff50843; its tree SHA is ff3fd6823ba96dd6d43a7eaa53269c34e44951e6.

@coderabbitai

coderabbitai Bot commented Sep 1, 2026

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 36 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Team

Run ID: 0e736c2b-cfbc-4454-9570-5de4fb3e4dd7

📥 Commits

Reviewing files that changed from the base of the PR and between 15fb724 and 4ac3e6a.

📒 Files selected for processing (68)
  • .claude/skills/api-reference/SKILL.md
  • .claude/skills/env-reference/SKILL.md
  • .env.example
  • .github/workflows/deploy-reusable.yml
  • AGENTS.md
  • CLAUDE.md
  • apps/api/.env.example
  • apps/api/src/db/migrations/0132_project_data_terminal_archive_sharding.sql
  • apps/api/src/db/migrations/0133_project_data_archive_recovery_cursors.sql
  • apps/api/src/db/schema.ts
  • apps/api/src/durable-objects/migrations.ts
  • apps/api/src/durable-objects/project-data/archive-sharding-canonical.ts
  • apps/api/src/durable-objects/project-data/archive-sharding.ts
  • apps/api/src/durable-objects/project-data/comments.ts
  • apps/api/src/durable-objects/project-data/grouped-fts-cleanup.ts
  • apps/api/src/durable-objects/project-data/index.ts
  • apps/api/src/durable-objects/project-data/materialization.ts
  • apps/api/src/durable-objects/project-data/message-persistence.ts
  • apps/api/src/durable-objects/project-data/messages.ts
  • apps/api/src/durable-objects/project-data/row-schemas/sessions.ts
  • apps/api/src/durable-objects/project-data/session-summary-sync.ts
  • apps/api/src/durable-objects/project-data/sessions.ts
  • apps/api/src/durable-objects/project-data/tool-payload-archive.ts
  • apps/api/src/durable-objects/project-data/types.ts
  • apps/api/src/durable-objects/sam-session/tools/search-task-messages.ts
  • apps/api/src/durable-objects/task-runner/index.ts
  • apps/api/src/durable-objects/task-runner/state-machine.ts
  • apps/api/src/env.ts
  • apps/api/src/routes/admin.ts
  • apps/api/src/routes/admin/project-data-archive.ts
  • apps/api/src/routes/mcp/session-tools.ts
  • apps/api/src/scheduled/handler.ts
  • apps/api/src/scheduled/project-data-archive-sharding.ts
  • apps/api/src/schemas/admin.ts
  • apps/api/src/schemas/index.ts
  • apps/api/src/services/diagnostic-incident-config.ts
  • apps/api/src/services/project-data-archive-routing.ts
  • apps/api/src/services/project-data-archive-types.ts
  • apps/api/src/services/project-data.ts
  • apps/api/src/services/session-recovery-authority.ts
  • apps/api/src/services/session-snapshot-recovery-lifecycle.ts
  • apps/api/tests/integration/session-recovery-handoff.test.ts
  • apps/api/tests/unit/db/project-data-archive-migration.test.ts
  • apps/api/tests/unit/durable-objects/migrations.test.ts
  • apps/api/tests/unit/durable-objects/task-runner-state-machine.test.ts
  • apps/api/tests/unit/routes/admin-security.test.ts
  • apps/api/tests/unit/services/project-data-archive-inventory.test.ts
  • apps/api/tests/unit/services/project-data-archive-sharding.test.ts
  • apps/api/tests/unit/services/project-data-snapshot-recovery-wake.test.ts
  • apps/api/tests/unit/session-snapshots.test.ts
  • apps/api/tests/workers/project-data-archive-sharding.test.ts
  • apps/api/tests/workers/project-data-do.test.ts
  • apps/api/tests/workers/project-data-service.test.ts
  • apps/api/tests/workers/project-data-storage-safety.test.ts
  • apps/api/tests/workers/session-summary-index-sync.test.ts
  • apps/api/tests/workers/support/expected-error-doubles.ts
  • apps/api/tests/workers/task-runner-do.test.ts
  • apps/api/wrangler.toml
  • apps/www/src/content/docs/docs/architecture/overview.md
  • apps/www/src/content/docs/docs/guides/self-hosting.mdx
  • apps/www/src/content/docs/docs/reference/api.md
  • apps/www/src/content/docs/docs/reference/configuration.md
  • infra/__tests__/config.test.ts
  • infra/resources/config.ts
  • scripts/deploy/project-data-routing-guard.ts
  • scripts/deploy/sync-wrangler-config.ts
  • scripts/quality/project-data-routing-guard.test.ts
  • tasks/active/2026-08-31-projectdata-terminal-archive-sharding.md

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@raphaeltm raphaeltm added the coderabbit-review Trigger CodeRabbit review for opt-in PRs label Sep 1, 2026 — with ChatGPT Codex Connector
@codspeed-hq

codspeed-hq Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 6 untouched benchmarks


Comparing sam/execute-task-using-skill-87btkd (4ac3e6a) with main (f724f76)1

Open in CodSpeed

Footnotes

  1. No successful run was found on sam/continue-zero-loss-projectdata-j6dat0 (15fb724) during the generation of this report, so main (f724f76) was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

@sonarqubecloud

sonarqubecloud Bot commented Sep 1, 2026

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
B Security Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

@raphaeltm

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Sep 1, 2026

Copy link
Copy Markdown
⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

coderabbit-review Trigger CodeRabbit review for opt-in PRs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant