Skip to content

feat(storage): add vault schema migrations that older builds can read - #637

Open
Dzejkop wants to merge 3 commits into
mainfrom
dzejkop/vault-version
Open

Dzejkop wants to merge 3 commits into
mainfrom
dzejkop/vault-version

Conversation

@Dzejkop

@Dzejkop Dzejkop commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Adds transactional schema migrations to the credential vault, and keeps migrated vaults usable by released builds. This replaces #632, which was reverted in #639.

Released builds (v0.28.0 and earlier) have no version checks. On every start they upsert their vault_meta row with ON CONFLICT(schema_version), so the layout must keep that working:

  • schema_version stays 1 and keeps its unique index. That still guarantees a single metadata row.
  • vault_version records the applied migrations, and min_vault_version the oldest vault version whose code can use the vault.
  • A vault newer than the build opens without migrating and logs a warning. It's refused only if min_vault_version is newer than the build. That allows an app rollback, and an older companion app sharing the vault on iOS.
  • Migrations run in one write transaction and roll back on failure. Current-schema opens don't take a write lock, and concurrent openers re-check under the lock.
  • Incomplete schemas fail without resetting data. Uninitialized v1 vaults are adopted; old backups still import.
  • Vaults written by unreleased feat(storage): add transactional vault schema migrations #632 builds are converted to this layout.

Validation:

  • 128 storage tests pass, including v0.28.0's SQL against a migrated vault.
  • Strict Clippy passes with all, default and no-default features; the wasm check and Clippy pass.

Stack parent of #633.

AI: Claude Opus 5.5 implementation. Prompt: add vault schema migrations that keep migrated vaults readable by older builds and allow rollback.

🤖 Generated with Claude Code


Note

High Risk
Changes encrypted credential vault schema, migration, and account metadata initialization—core persistence that must stay compatible across app versions and shared-vault scenarios.

Overview
Fixes credential vault upgrades so app rollback and older WalletKit builds (e.g. v0.28.0) can still open a vault after migration, instead of hitting unique-constraint errors on vault_meta.

Schema versioning replaces the one-shot DDL with ordered migrations (migration_v1 / migration_v2). Applied version is stored in new vault_version and min_vault_version columns; schema_version stays fixed at 1 with the existing unique index on it. Migration v2 also converts vaults from the prior #632 layout (singleton index / version in schema_version) back to that legacy shape. Opens validate completeness, reject unsupported min_vault_version, run pending migrations in a single transaction_immediate, and warn (without migrating) when the stored version is newer than the app.

init_leaf_index no longer upserts metadata rows; it requires exactly one metadata row and updates leaf_index when unset, so older releases can keep using their ON CONFLICT(schema_version) init path on the same database.

Adds a v1 SQL fixture and a large migration/rollback/concurrency test suite (legacy adoption, failed migration rollback, backup import/merge, pre-vault_version init SQL).

Reviewed by Cursor Bugbot for commit a180bac. Bugbot is set up for automated code reviews on this repo. Configure here.

Summary by CodeRabbit

  • Bug Fixes
    • Improved compatibility with older credential vaults and vaults using newer schema versions that remain supported.
    • Vaults requiring a version newer than the app supports are rejected, and invalid version information is detected.
    • Vault upgrades preserve account information and roll back cleanly if an upgrade fails.
    • Vaults missing required metadata are reported as invalid rather than initialized automatically.

Builds before vault versioning upsert their vault_meta row on
schema_version = 1, so recording the version there broke their init.
Keep schema_version at 1, record vault_version and min_vault_version in
their own columns, and open newer vaults without migrating unless they
require a newer build.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Copilot AI balanced review requested due to automatic review settings October 29, 2026 16:30
@coderabbitai

coderabbitai Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

No actionable comments were generated in the recent review.

Recent review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: ASSERTIVE
  • Plan: Team
  • Run ID: 0a858dc8-6541-47f8-8d4b-06400ba8b171
Commits

Reviewing files that changed from the base of the PR and between 6cb65e6 and a180bac.

Files selected for processing: 3
  • crates/walletkit-core/src/storage/credential_vault/fixtures/v1.sql
  • crates/walletkit-core/src/storage/credential_vault/mod.rs
  • crates/walletkit-core/src/storage/credential_vault/schema.rs

Included review availability: This review used your included allowance. 6 included reviews remain after this review. Your included PR review attempts over the past 7 days set your current allowance at 8 reviews per hour.

Walkthrough

Credential vault schema versioning now separates the applied migration version from the minimum compatible version. Schema checks and migrations handle legacy layouts and compatible newer vaults. Leaf-index initialization no longer creates metadata when no metadata row exists.

Changes: Credential vault versioning and initialization

Layer / File(s) Summary
Version metadata and schema validation
crates/walletkit-core/src/storage/credential_vault/schema.rs, crates/walletkit-core/src/storage/credential_vault/fixtures/v1.sql
Vault metadata tracks the applied migration version and minimum compatible version separately while keeping schema_version at 1. Validation checks stored values, required tables, and index layouts. The fixture defines metadata and credential tables with their indexes and update trigger.
Migration and compatibility handling
crates/walletkit-core/src/storage/credential_vault/schema.rs
Migrations record versions transactionally and preserve higher stored values. Legacy schemas can migrate to the current layout. A vault with a higher applied version opens when its minimum version is supported. Tests cover data preservation, rollback and retry, concurrent migration, and version validation.
Vault opening and leaf-index initialization
crates/walletkit-core/src/storage/credential_vault/mod.rs, crates/walletkit-core/src/storage/credential_vault/schema.rs
CredentialVault::new passes ensure_schema to Vault::open. Leaf-index initialization updates a single metadata row, rejects a mismatched index, and returns an invalid-row-count error when the row count is zero or greater than one. Tests cover initialization and compatibility with older metadata layouts.

Priority: Normal

Estimated code review effort: 3 (Moderate) | ~30 minutes

Change: Bug fix

Suggested reviewers: kilianglas

Merge Risk: Minimal · up to a180b

No concrete compatibility or runtime issue remains identified; the PR is mergeable after normal checks.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 55.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documenta… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title says the PR adds vault schema migrations that older builds can read. The changeset matches this. schema.rs adds vault_version and min_vault_version tracking, migration v2 converts #632-layou…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title says the PR adds vault schema migrations that older builds can read. The changeset matches this. schema.rs adds vault_version and min_vault_version tracking, migration v2 converts #632-layout vaults back to the legacy index, and older builds keep schema_version at 1 with its unique index. The title names the main change, and it is specific to the storage layer. The title is one sentence and 69 characters, which fits the length guidance.

Full details: Docstring Coverage

Explanation

Docstring coverage is 55.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 20 functions across 2 files. (1 skipped: 1 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

Finishing Touches: 💡 1
📝 Generate docstrings: 💡
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR

Comment @coderabbitai help to get the list of available commands.

@wld-walletkit-bot

wld-walletkit-bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator

Agent review · Approval withheld

Head, base, policy, or review evidence changed during review

Reviewed commit: a180bac2f916d74dfc78391c17a80688f7b9add6 · Workflow run

Agent recommendation: approve

Approved. The required CodeRabbit and Copilot reviews cover the current content, the single review finding is fixed and confirmed, and my independent review found no blocking defects. CodeRabbit approved the exact head SHA (review 5710525316, 2026-10-30T09:42:12Z). Copilot recommended approval on commit 6cb65e60; I verified that 6cb65e60 and head a180bac2 share tree 6d3281d4efb3543e293fd1e6d1e3526a0d89052a (git diff 6cb65e60..a180bac2 is empty), so head a180bac2 is a merge of main that introduces no content Copilot did not review. The only remaining automated note is CodeRabbit's non-blocking docstring-coverage warning on the initial commit, which did not prevent CodeRabbit from approving the head. No unresolved concerns, change requests, or unanswered review threads remain.

Final agent response

Verdict written to `/home/runner/work/_temp/review/verdict.json`.

**Decision: approve.**

Key points:
- **Coverage:** CodeRabbit approved the exact head `a180bac2`. Copilot's latest review (approval recommended, no open issues) is on `6cb65e6`, whose tree is byte-identical to the head (`6d3281d4…`, empty `git diff`), so the merge of main introduced no unreviewed content. CI on `6cb65e6` passed.
- **Discussion:** The single Copilot P2 (concurrent legacy version/index snapshot) is resolved and outdated; I verified the fix in `read_legacy_version` reads `schema_version` and the singleton-index existence in one statement.
- **Independent review:** Traced migrations and version handling across fresh, legacy, #632, stale, current, and newer vaults; transactional rollback, atomic version recording, `schema_version = 1` + unique index compatibility with v0.28.0, and backup exclusion of `vault_meta` all check out. No blocking defects.
- **Policy:** Open, non-draft, in-repo, into main; no protected-file changes; no change requests or unresolved threads; format-sensitive work guarded by the frozen `v1.sql` fixture.

No blockers.
Structured review assessment

Review coverage

CodeRabbit: APPROVED on head a180bac2 (review 5710525316). Its summary reported no actionable comments and its only historical concern was the concurrency issue, now fixed. Copilot: latest review 5701620393 on commit 6cb65e60 is 'Approval recommended' with no open issues; earlier review 5701358246 raised the single P2 finding. Because 6cb65e6 and a180bac2 have byte-identical trees, Copilot's review covers exactly the head content. Depth is appropriate: both bots reviewed the full migration implementation and its large test suite, and the harness confirms CI (run 40915078144 for 6cb65e6, same tree) completed successfully, including the crate tests, Clippy/format, Swift, Kotlin, WASM, and web checks. The only pending checks are the same CI jobs re-running for the merge commit; the content is unchanged. This satisfies the policy's requirement for completed CodeRabbit and Copilot reviews over the current changes.

Discussion resolution

One review thread exists (PRRT_kwDOLGbMQc6xptfS, schema.rs), marked resolved and outdated. Copilot's [P2] finding was that concurrent initialization of a #632-layout vault could read schema_version=2 in one snapshot and then check the singleton index in a later snapshot, transiently rejecting the vault as 'invalid vault version'. The head code fixes this: read_legacy_version evaluates schema_version and EXISTS(idx_vault_meta_singleton) in a single SELECT over vault_meta, so both values come from one snapshot, and the pre-lock path performs no separate index check (migrate rechecks everything under the write lock in transaction_immediate). I confirmed this in the head diff; resolved/outdated status is backed by the code, not merely the flag. Copilot's follow-up review lists the issue as resolved. No other inline threads or general findings (the two CodeRabbit auto-fix completion comments are unrelated to this PR's code).

Independent review

I read .code-review.md, AGENTS.md, evidence.json, and the full diff for the three changed files (fixtures/v1.sql added; mod.rs; schema.rs). I traced ensure_schema, migrate, read_version, read_legacy_version, validate_schema, validate_legacy_schema, record_version, migration_v1/v2, and init_leaf_index across fresh, legacy-v1 (initialized and uninitialized), #632 singleton-index, current, stale-version, and newer-than-supported vaults. Migrations run inside transaction_immediate with an under-lock recheck, record the version atomically with MAX(), and roll back DDL and version on failure. schema_version stays 1 with the unique idx_vault_meta_schema_version, so v0.28.0-and-earlier's ON CONFLICT(schema_version) upsert keeps working; a test replays that pre-vault_version SQL against a migrated vault. vault_meta remains excluded from BACKUP_TABLES, and credential_records/blob_objects DDL and compute_content_id are untouched. Incomplete schemas and missing/duplicate metadata error instead of being reset. init_leaf_index now reads COUNT(*) and MAX(leaf_index) in one snapshot and errors on anomalous row counts rather than upserting. I noted two intentional behavior changes (blob_objects and dropped metadata indexes are no longer silently recreated for current vaults, and forward versions must retain the legacy index for version>=2); both are guarded by explicit errors, covered by tests, and consistent with the stated compatibility design, not defects. The frozen v1.sql fixture guards the on-disk legacy layout. No blocking defect found.

Repository policy

All repository approval conditions in .code-review.md are met. PR #637 is open, non-draft, from this repository (author Dzejkop), into main, with no no-auto-approve label and no changes to .github/ or review instruction files (only the three credential_vault files). The required reviewers are covered as described above. No outstanding CHANGES_REQUESTED reviews and no unresolved threads. AGENTS.md compatibility expectations hold: schema_version remains 1 with its unique index, CBOR layouts and compute_content_id are unchanged, and the format-sensitive migration is guarded by a frozen v1.sql fixture plus legacy-reopen/backup-import tests. Logging/error messages across the FFI boundary carry only generic text and integer version numbers, so no sensitive user data leaks. The only non-blocking caveat is CodeRabbit's docstring-coverage warning, which it accepted by approving the head.
verdict.json
{
  "head": "a180bac2f916d74dfc78391c17a80688f7b9add6",
  "evidence": "2ae2e25302c845b8966668ebabed8297598d297816f390b3e5a683384da836a7",
  "approve": true,
  "reason": "Approved. The required CodeRabbit and Copilot reviews cover the current content, the single review finding is fixed and confirmed, and my independent review found no blocking defects. CodeRabbit approved the exact head SHA (review 5710525316, 2026-10-30T09:42:12Z). Copilot recommended approval on commit 6cb65e60; I verified that 6cb65e60 and head a180bac2 share tree 6d3281d4efb3543e293fd1e6d1e3526a0d89052a (git diff 6cb65e60..a180bac2 is empty), so head a180bac2 is a merge of main that introduces no content Copilot did not review. The only remaining automated note is CodeRabbit's non-blocking docstring-coverage warning on the initial commit, which did not prevent CodeRabbit from approving the head. No unresolved concerns, change requests, or unanswered review threads remain.",
  "review_coverage": "CodeRabbit: APPROVED on head a180bac2 (review 5710525316). Its summary reported no actionable comments and its only historical concern was the concurrency issue, now fixed. Copilot: latest review 5701620393 on commit 6cb65e60 is 'Approval recommended' with no open issues; earlier review 5701358246 raised the single P2 finding. Because 6cb65e6 and a180bac2 have byte-identical trees, Copilot's review covers exactly the head content. Depth is appropriate: both bots reviewed the full migration implementation and its large test suite, and the harness confirms CI (run 40915078144 for 6cb65e6, same tree) completed successfully, including the crate tests, Clippy/format, Swift, Kotlin, WASM, and web checks. The only pending checks are the same CI jobs re-running for the merge commit; the content is unchanged. This satisfies the policy's requirement for completed CodeRabbit and Copilot reviews over the current changes.",
  "discussion_resolution": "One review thread exists (PRRT_kwDOLGbMQc6xptfS, schema.rs), marked resolved and outdated. Copilot's [P2] finding was that concurrent initialization of a #632-layout vault could read schema_version=2 in one snapshot and then check the singleton index in a later snapshot, transiently rejecting the vault as 'invalid vault version'. The head code fixes this: read_legacy_version evaluates schema_version and EXISTS(idx_vault_meta_singleton) in a single SELECT over vault_meta, so both values come from one snapshot, and the pre-lock path performs no separate index check (migrate rechecks everything under the write lock in transaction_immediate). I confirmed this in the head diff; resolved/outdated status is backed by the code, not merely the flag. Copilot's follow-up review lists the issue as resolved. No other inline threads or general findings (the two CodeRabbit auto-fix completion comments are unrelated to this PR's code).",
  "independent_review": "I read .code-review.md, AGENTS.md, evidence.json, and the full diff for the three changed files (fixtures/v1.sql added; mod.rs; schema.rs). I traced ensure_schema, migrate, read_version, read_legacy_version, validate_schema, validate_legacy_schema, record_version, migration_v1/v2, and init_leaf_index across fresh, legacy-v1 (initialized and uninitialized), #632 singleton-index, current, stale-version, and newer-than-supported vaults. Migrations run inside transaction_immediate with an under-lock recheck, record the version atomically with MAX(), and roll back DDL and version on failure. schema_version stays 1 with the unique idx_vault_meta_schema_version, so v0.28.0-and-earlier's ON CONFLICT(schema_version) upsert keeps working; a test replays that pre-vault_version SQL against a migrated vault. vault_meta remains excluded from BACKUP_TABLES, and credential_records/blob_objects DDL and compute_content_id are untouched. Incomplete schemas and missing/duplicate metadata error instead of being reset. init_leaf_index now reads COUNT(*) and MAX(leaf_index) in one snapshot and errors on anomalous row counts rather than upserting. I noted two intentional behavior changes (blob_objects and dropped metadata indexes are no longer silently recreated for current vaults, and forward versions must retain the legacy index for version>=2); both are guarded by explicit errors, covered by tests, and consistent with the stated compatibility design, not defects. The frozen v1.sql fixture guards the on-disk legacy layout. No blocking defect found.",
  "policy_checks": "All repository approval conditions in .code-review.md are met. PR #637 is open, non-draft, from this repository (author Dzejkop), into main, with no no-auto-approve label and no changes to .github/ or review instruction files (only the three credential_vault files). The required reviewers are covered as described above. No outstanding CHANGES_REQUESTED reviews and no unresolved threads. AGENTS.md compatibility expectations hold: schema_version remains 1 with its unique index, CBOR layouts and compute_content_id are unchanged, and the format-sensitive migration is guarded by a frozen v1.sql fixture plus legacy-reopen/backup-import tests. Logging/error messages across the FFI boundary carry only generic text and integer version numbers, so no sensitive user data leaks. The only non-blocking caveat is CodeRabbit's docstring-coverage warning, which it accepted by approving the head."
}

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Concurrent vault initialization can read schema metadata and index state from different snapshots, incorrectly rejecting a migrated vault with a transient invalid vault version error.

1 open issue
What changed in this PR

This PR preserves vault compatibility with released WalletKit builds, supporting app rollback and shared-vault access across versions.

Changes:

  • Keeps schema_version = 1 and tracks migrations and compatibility in separate columns.
  • Allows compatible newer vaults to open without migration and repairs the metadata layout introduced by #632.
  • Adds compatibility tests and makes leaf-index initialization rely on schema-created metadata.
File Description
crates/​walletkit-core/​src/​storage/​credential_vault/​schema.rs Updates version tracking, compatibility checks, migration repair, and regression tests.
crates/​walletkit-core/​src/​storage/​credential_vault/​mod.rs Removes metadata insertion from leaf-index initialization.

Review effort: Balanced

coderabbitai[bot]
coderabbitai Bot previously approved these changes Oct 29, 2026
Read the legacy marker and single-row index in one statement so a
concurrent conversion cannot be rejected, make migration v2 safe to
reapply, and cover released-build init after converting a vault.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Copilot AI balanced review requested due to automatic review settings October 29, 2026 16:47

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues block the vault-compatibility changes, although reported test and Clippy results were not independently verified.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

coderabbitai[bot]
coderabbitai Bot previously approved these changes Oct 29, 2026
@coderabbitai

coderabbitai Bot commented Oct 29, 2026

Copy link
Copy Markdown
Contributor

Auto-fix stopped before satisfying the configured readiness conditions.

Blocking condition: Auto-fix requires data retention to be enabled for the organization.

Triggered by rule Auto-fix PRs. This is the Auto-fix result for commit 6cb65e6; it does not confirm that the pull request was merged.

View activity

@wld-walletkit-bot wld-walletkit-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agent review passed the repository review policy, discussion resolution, and independent code review. Verdict and evidence: https://github.com/worldcoin/walletkit/actions/runs/40953333916

@coderabbitai

coderabbitai Bot commented Oct 30, 2026

Copy link
Copy Markdown
Contributor

Auto-fix stopped before satisfying the configured readiness conditions.

Blocking condition: Auto-fix requires data retention to be enabled for the organization.

Triggered by rule Auto-fix PRs. This is the Auto-fix result for commit 6cb65e6; it does not confirm that the pull request was merged.

View activity

Keep this branch's vault schema: it carries #632's migrations with the
vault_version layout.
@Dzejkop
Dzejkop dismissed stale reviews from wld-walletkit-bot and coderabbitai[bot] October 30, 2026 09:38

The merge-base changed after approval.

@Dzejkop Dzejkop changed the title fix(storage): keep migrated vaults readable by earlier builds feat(storage): add vault schema migrations that older builds can read Oct 30, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants