Migrate persistence to embedded Turso vectors

[EyJunge1]

Summary

• replace the SQLite/USearch backend stack with embedded Turso/libSQL and native vector indexes
• migrate existing memory shards safely with backups, locking, verification, resume support, and dimension-aware re-embedding
• harden lifecycle, shard scoping, web API authentication, CI platform checks, and upgrade coverage

Test plan

• bun run typecheck
• bun test (222 tests)
• bun run build
• bun run format:check
• verify the packaged libSQL vector extension in the platform smoke workflow

Made with Cursor

[tickernelz]

I tried to merge this PR after integrating the newer PRs in this batch, but it now has merge conflicts in:

• src/config.ts
• src/index.ts
• src/services/web-server.ts

Please rebase or merge the latest main and resolve the conflicts. This is a large persistence migration, so please also rerun the full build and test suite after resolving them; we can merge it afterward.

[EyJunge1]

Release note

This is planned as a major 3.0.0 release because it replaces the persistence format and vector backend. Existing data is migrated with backups and verification, but downgrading to v2 is not supported after migration.

All automated validation is green:

• bun run typecheck
• bun test (249 passing)
• bun run build
• bun run format:check

Please independently test the migration from existing SQLite/USearch data to embedded Turso/libSQL, including backup creation, verification, recovery, and real-world data.

[tickernelz]

Thanks for rebasing this. I retried it against the latest main after merging the newer PRs in this batch, but it still has merge conflicts in:

• src/services/api-handlers.ts
• src/services/migration-service.ts
• src/services/web-server-worker.ts (modify/delete conflict)
• src/services/web-server.ts
• src/web/app.js

Please rebase or merge the latest main again and resolve these conflicts. Because this PR replaces the persistence layer, please rerun the frozen install, typecheck, build, and full test suite after resolving them; we can merge it afterward.

[EyJunge1]

The branch has been rebased onto the latest main, all conflicts have been resolved, and the full validation suite passes: frozen install, typecheck, build, and all 250 tests.

[EyJunge1]

Hi, I’ve also tested the SQLite/USearch → embedded Turso migration locally on my machine (backup, migrate, verification, and basic recall afterwards). Branch is rebased, conflicts resolved, and the full suite is green. Happy to address anything else if you want another look before merging.