Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion KNOWN_LIMITATIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ ZigBase v0.13.0 is an early release. The gaps below are known and tracked for fu

## Schema / migrations
- **Consumer migration coordination is cooperative.** PostgreSQL apply/rollback batches require a direct or session-pooled connection, not transaction/statement pooling. SQLite batches use a permanent `<canonical-main-database-path>.migrations.lock` sidecar and fail fast with `MigrationBusy` on contention. Use a writable, trusted local directory with working file locks; never replace/unlink the sidecar or database during use, or access the database through hard-link aliases. Individual commit boundaries and non-transactional callbacks are preserved. Old binaries, external SQL writers, automatic collection provisioning and distributed filesystems still need a single leader. Coordination does not make external callback effects atomic or incompatible rolling schema upgrades safe.
- **Comptime auto-migration is additive-only.** Startup provisioning of a comptime `.collections` schema creates missing collections and adds new fields, preserving data. **Non-additive changes (rename, drop, or type-change a field) are detected, logged, and skipped** — they require an explicit `.migrations` entry. (Access **rules** are the exception: a changed rule *is* re-applied on every startup, as a metadata-only write.)
- **Comptime auto-migration applies only safe additive changes.** Startup provisioning creates missing collections, adds fields, and widens select values or selection limits only when existing values remain valid, flags stay unchanged, and scalar/array storage stays the same. Narrowing, other option/flag changes, and rename/drop/type changes are detected, logged, and skipped; they require an explicit `.migrations` entry. Access rules are reapplied without rebuilding records. Accepted changes, declared indexes, search objects and the cache-generation bump commit atomically; a conflicting concurrent definition change returns `Conflict` before overwriting it, while an already-applied identical plan succeeds without another rewrite. Reordering select values alone is a no-op.
- **Schema rule preflight is structural, not a proof of authorization behavior.** `schema apply` (including dry runs) and REST collection mutations reject invalid syntax and unresolved field/relation references against the prospective schema. Request-dependent values and application policy still require runtime tests. Trusted low-level collection writers can stage incomplete graphs; run `zigbase schema check-rules --data-dir PATH` against their final state before cutover.
- **A live server picks up another process's schema change within about five seconds, not instantly.** `schema apply`, `migrate`, and `import` run as separate CLI processes; the server notices via a background observer that polls the `_schema_state` generation marker (~5s cadence) and then drops its in-process collection cache. Requests in that window still see the pre-change view. A cutover that must be seen immediately should still restart the server. See [Migrating an existing backend to ZigBase](docs/migration-tools.md).
- **Collection rename requires an explicit offline migration.** `m.renameCollection(from, to, .{ .offline = true })` preserves identities, database/auth/search references, and immutable file namespaces; document diffs do not infer renames. Stop all processes and update compiled name references; old URLs/topics are not aliased. Namespace reservations survive deletion, so reserved physical prefixes cannot be claimed by a new collection. See [offline collection rename](docs/framework.md#offline-collection-rename).
Expand Down
3 changes: 3 additions & 0 deletions changelog.d/provision-field-options.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
### Fixes
- Startup provisioning now applies safe select-option widening to existing databases without rebuilding tables. Select narrowing, single/multi-value changes, and other field option or flag changes emit an explicit-migration warning instead of being silently ignored.
- Concurrent startup plans for the same definition converge without conflicts or repeated cache invalidation; select-value reordering alone leaves the stored definition unchanged.
25 changes: 24 additions & 1 deletion docs/framework.md
Original file line number Diff line number Diff line change
Expand Up @@ -4163,19 +4163,42 @@ and applies the **minimal safe change set** (running it twice is a clean no-op):
- A field present in the spec but missing from the live collection is **added**,
rebuilding the table while **preserving existing data** (the new column is null
for old rows).
- An existing **select** field's options are updated when the declared `.values`
contain every live value and `.maxSelect` does not decrease. This is a
metadata-only write, preserving records, indexes, and triggers. The field's
other flags must be unchanged, and it must keep the same single/multi-value
representation: increasing `.maxSelect` from 1 to 2 or more needs an explicit
data migration from scalar strings to JSON arrays. A value of 0 is a zero limit,
not an unlimited sentinel. If another process changes any stored collection definition (including rules or options) between
the startup comparison and schema write, provisioning fails with `Conflict`
rather than overwriting that change, including when widening accompanies a
new field or TTL change; restart against the current schema. Concurrent boots
requesting the same resulting definition succeed without another rewrite.
Reordering the same select values keeps the stored order and does not count as
widening or invalidate caches. Accepted fields, rules,
declared indexes, search objects and one cache-generation bump commit together;
any failure rolls the whole collection update back.
- A changed **access rule** (`.list_rule` / `.view_rule` / `.create_rule` /
`.update_rule` / `.delete_rule`) is **re-applied to the live collection**. This is a
metadata-only write — no table rebuild, no data copy — so tightening a rule in code
and redeploying really does tighten it in the running server. A rule left **unset**
(`null`) in the literal means "unspecified: leave whatever is live alone"; to lock a
rule explicitly, spell it `""` (blank and unset are the same rule at evaluation time
— superusers only — so switching between them is not treated as a change).
- A **non-additive** change — a field rename, drop, or type/storage-class change —
- A **non-additive** change — a field rename, drop, type/storage-class change,
select narrowing, or any other existing field option or flag change —
is **detected, logged, and SKIPPED** (never applied, so no data loss). Relation
targets must reference a known collection (a comptime collection or a pre-existing
live one such as `_superusers`); an unknown target is a startup error.

For the changes auto-migration won't do, use the `.migrations` escape hatch.
Changing a text length/pattern, number scale/bounds, relation target, or required
flag in the declaration alone does not update an existing database; startup logs
a warning. Test schema upgrades against an existing database as well as a fresh
one: a fresh `zigbase.testing.start` harness already has the new options.
Search indexes follow the effective persisted flags: a skipped `.searchable`
change neither enables nor disables an existing field's search index. A newly
added searchable field is indexed after its column is created.

Declared `.indexes` are reconciled on every startup, including when no fields change.
See [Indexes](#indexes) for ownership, adoption, and failure behavior.
Expand Down
9 changes: 1 addition & 8 deletions src/backend/postgres/conn.zig
Original file line number Diff line number Diff line change
Expand Up @@ -16,14 +16,7 @@ const scram = @import("scram.zig");
const connstr = @import("connstr.zig");
const tls_trust = @import("tls_trust.zig");

/// A TLS/auth misconfiguration is a hard `.err` in production, but Zig's test runner
/// counts every `std.log.err` as a test failure — so the tests that deliberately
/// exercise these refuse-to-connect paths would fail on the log alone even though
/// their assertions (the returned error) are correct. Under the test runner we emit
/// the identical message at `.warn`; production still logs `.err`.
fn logMisconfig(comptime fmt: []const u8, args: anytype) void {
if (builtin.is_test) std.log.warn(fmt, args) else std.log.err(fmt, args);
}
const logMisconfig = @import("../../startup_log.zig").refusal;

pub const ConnError = error{
ConnectFailed,
Expand Down
Loading
Loading