Skip to content

fix: Calvin predicate writes in transactions, fusion primary keys, bitemporal predicate writes - #429

Merged
farhan-syah merged 1 commit into
mainfrom
fix/calvin-predicate-fusion
Oct 20, 2026
Merged

farhan-syah merged 1 commit into
mainfrom
fix/calvin-predicate-fusion

Conversation

@farhan-syah

Copy link
Copy Markdown
Member

Summary

Predicate writes commit through Calvin from every transaction path, GRAPH RAG FUSION returns primary keys, ARRAY_AGG output is a relation, and predicate writes on bitemporal collections work.

Closes #424. Closes #425. Settles #428: every search response names rows through one Control-Plane resolver, not the Data Plane.

Calvin predicate writes (#424 part A)

  • A predicate UPDATE or DELETE inside BEGIN … COMMIT, a trigger body or a procedure body reads its rows at its own statement and carries them in its plan. Before this, the buffered plan carried no rows and the Calvin stage refused it with XX000.
  • Every in-transaction write goes through one staging point, so the client block, triggers and procedures share it.
  • The session stage and the Calvin bulk stage both check the carried rows against the rows the predicate matches at the plan's turn. A change votes PredictionDrift: autocommit retries, and COMMIT aborts with SQLSTATE 40001.
  • The static Calvin stage checks the prediction again. Before this, it applied a predicted row set without checking it.
  • A predicate write on a materialized-sum source checks its sum targets at the stage.
  • An in-block UPDATE that moves an edge's _from or _to writes the edge update.
  • One retry policy (budget and backoff) serves autocommit and in-block statements.

Calvin stage refusals (#425)

  • calvin_stage_refusal logs every refusal: debug for prediction drift, warn for any other.
  • An internal refusal is recorded with a faultbox capture on the refusing node.

GRAPH RAG FUSION naming (#424 part B)

  • Fusion keys its legs with a typed RagKey (graph node, row surrogate, unbound entry), so a key "7" never merges with surrogate 7. A text hit on a graph node fuses with that node.
  • A hit in no edge returns its primary key. Rows are named after fusion, so scores and ranks do not change.
  • Vector, full-text, hybrid and fusion responses name rows through one resolver. It reads this node's catalog in one read transaction, then asks the collection home leader for the misses in one request. In a cluster, every node answers with the same keys.
  • A row whose key is its own surrogate bytes resolves to the decimal identity.
  • translate_search_response and shape_response_materialized are async through every caller.
  • A stored vector hit body that does not decode is an error, not a silently empty row.

ARRAY_AGG and ARRAY_ELEMENTWISE as relations

  • WHERE, computed columns, GROUP BY, ORDER BY, DISTINCT and LIMIT apply to their rows directly, through the same path ARRAY_SLICE uses.
  • A temporal call declares its truncated_before_horizon column.

Bitemporal collections

On a bitemporal collection, predicate UPDATE and DELETE matched no rows: they read only the plain table. This PR adds one shared current-version read, and every caller uses it.

  • Bulk UPDATE and DELETE write versions through the same per-row path as point writes: tombstones, versioned index entries, and versioned redo entries.
  • TRUNCATE, in autocommit and in a transaction, closes every current version. History stays readable with AS OF.
  • Facet counts, aggregates and joins, recursive CTEs, index backfill, vector rebuild and period locks read current versions.
  • DROP INDEX removes the versioned index entries in the same transaction as the plain ones.
  • CREATE INDEX backfills each version at its own system time.
  • An AS OF SYSTEM TIME indexed equality reads the versioned index at the cutoff. Before this, it answered from the current index. The same holds for an IN (SELECT …) side read AS OF.
  • CONVERT to a document engine rewrites every version. CONVERT to kv is refused before any row moves.

Behaviour changes

  • In-block predicate writes whose rows change before COMMIT abort with 40001. The client retries the transaction.
  • TRUNCATE on a bitemporal collection keeps the history.
  • CONVERT of a bitemporal collection to kv is refused.
  • A malformed vector hit payload or body fails the response.

Lite

Lite compiles against this branch. Lite answers an AS OF indexed equality from the current index, which it already did before this change. The fix needs the new IndexedFetch.system_as_of_ms field at:

  • nodedb-lite/src/query/visitor/queries.rs lower_document_index_lookup
  • nodedb-lite/src/query/physical_visitor/adapter/document/dispatch.rs, the DocumentOp::IndexedFetch arm

Testing

  • Full suite: stage 1 22,759/22,759. Stage 2 514/515. The one failure, sync_peer_id_collision::a_second_replica_claiming_an_owned_peer_id_is_refused_not_absorbed, sits in CRDT sync code this PR does not touch. It passed 25/25 when run alone.
  • Workspace clippy (all features), the nodedb-client feature sets, cargo deny, every static gate, the wasm32 check, and the replay of all fuzz targets.
  • New wire tests cover in-block, trigger and procedure predicate writes, 40001 on a concurrent change, materialized sums, edge moves, fusion naming on one node and across three nodes, array relations, and bitemporal writes, TRUNCATE, CONVERT, DROP INDEX and AS OF index reads.
  • Four timing-dependent tests now force their schedule: two trigger_e2e publish waits, audit group commit, and event-consumer catch-up.

…state

Predicate UPDATE and DELETE inside a session transaction now read their
rows at the statement as the transaction sees them. The plan carries the
matched surrogates and implicit edges, the stage refuses a mismatched
prediction, and a Calvin COMMIT stages exactly those rows and checks
them again at its turn. Redo records and admission keys name every
collection a transaction writes.

Bitemporal collections gain system-time index lookups, version-aware
index backfill, point delete and truncate, and CONVERT COLLECTION that
rewrites live versions in place. Index lookups fall back to a version
scan for valid-time and all-versions reads.

Array table functions form relations: ARRAY_AGG and ARRAY_ELEMENTWISE
rows are selectable, and temporal reads append a horizon summary row.

GraphRAG fusion keys legs by graph node name, then surrogate, so a node
name never collides with a surrogate. Response translation resolves
primary keys and shapes fusion results accordingly.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

run-ci Opt this PR into the full test suite; re-add to force a re-run

Projects

None yet

1 participant