Repository navigation
H0: close cvc::state data() object replication (§52) — integer codec + value type_name + IPC/echo tests + roadmap - #767
Conversation
…s type spellings register_builtin_codecs keyed its integer codecs on the C++ spellings "int32_t"/"int64_t"/"uint32_t"/"uint64_t", but state::dataTypeName() returns the names app::registerDefaultTypes gives those types — "int", "unsigned int", "unsigned short", "int64", "uint64". So for an integer data() the sync adapter's _codecs->encode(dataTypeName(), ...) found no codec, threw, and the exception was swallowed: the integer object silently failed to replicate while value() and child structure still flowed. (float/double/std::string/bool already match the app spelling.) Add codec aliases keyed by the app's spellings, each any_cast-ing the exact stored type (boost::any_cast is exact-typeid, not converting): int / unsigned int / unsigned short / boost::int64_t / boost::uint64_t. The "_t"-keyed codecs are kept as-is. No wire/proto change. Test: TwoNodeInprocIntegerDataReplication round-trips int, int64 and uint64 data() across two inproc nodes and asserts the peer decodes each (and pins the app spellings, so a rename re-surfaces the coupling). The ASSERT_FALSE(empty) guards the drop: without a matching-key codec the object never crosses.
…napshot The valueChanged emit already carried the value's declared type (m.type_name = n.valueTypeName()), but the receive side dropped it: apply_remote::set_value only called node.value(string), which stamps "std::string", and shard.snapshot() never captured the value type at all. So a typed value (written via value<T>()) arrived on a peer — and on a late joiner's snapshot resync — as a std::string, losing the type a typed read depends on, even though the carriers (proto Mutation.type_name f8, SnapshotEntry.type_name f6, the struct field) were already on the wire. Close it: - state::valueTypeName(const std::string&): a signal-free setter for the declared type (the value is set separately, under the sync adapter's suppression guard, so this must not re-enter the journal). - apply_remote set_value: after node.value(string), apply m.type_name when present. - shard.snapshot(): capture e.type_name = node.valueTypeName(). - ingest_snapshot_entry: carry e.type_name on the value mutation so the resync path preserves it too. Test: TwoNodeValueTypeNamePreserved writes a typed int value and asserts the peer keeps valueTypeName "int" after BOTH a live apply and a snapshot resync. Full distributed_state_session (19), snapshot-protocol (4), ipc-transport (18) and sync-adapter (8) suites stay green.
§52's testing bullet asks for a typed data() round-trip over both IPC and gRPC. gRPC has it (SnapshotCarriesData / BlobPayloadRoundTrip) but compiles out unless CVC_ENABLE_GRPC=ON; the IPC suite only ever asserted value()/metadata, so the IPC wire serialization of a Mutation Payload (put_payload/read_payload) was carried in code but untested on the default build. Add StateTransportIpcTest.DataPayloadReplicatesOverIpc: two in-process IPC endpoints round-trip (1) a custom inline object, (2) a builtin int data() — exercising the S1 alias over a real transport — and (3) a 4 KiB payload that offloads to a chunked blob_ref and is reconstructed on the peer via the hydrator from a shared in-process blob store. Asserts byte-identity for all three. 8x repeat clean (async receive gated by the suite's wait_until helper).
…e-emit) §52 requires that applying a remote set_data not re-emit a set_data back toward the origin; only value() echo was pinned (the gRPC RoundTripSuppressedByDedup). The mechanism is apply_remote's thread_suppression_guard, which makes the dataChanged an apply triggers early-return rather than journal. Add DistributedStateSession.SetDataApplyDoesNotEcho: via the adapter's on_local_mutation hook, node2 — which only APPLIES node1's data() — emits zero local mutations (none at all, and specifically no set_data), local_mutation_count() stays 0, and a quiescent second pump moves nothing. An echo would surface as an outbound mutation on that pump.
…LANDED The §52 / Phase 10 section (dated 2026-10-01) said data() replication was "NOT STARTED", the apply side "an explicit no-op", the carriers "unused end to end", and the adapter "never hooks dataChanged" — all false since the cross-node streaming critical-path work. A planner starting from this doc would have rebuilt what already exists. Rewrite it to the landed state: the codec registry (per-shard), the dataChanged→set_data emit, the inline-decode / blob-hydrate apply, the snapshot capture+ingest, the four-transport wire serialization, the session wiring, and the three-layer echo suppression. Record this change's closing items (integer-key fix, value type_name, IPC coverage, set_data echo test) as LANDED, and the genuinely-remaining work as clearly-scoped deferrals: the format_desc codec has no production registration because no production shard replicates /streams descriptors today (the relay scopes to conf.* and media rides stream_bridge, not the state tree); plus WS late-joiner blob hydration, the async hydrator install guard, and the oversized-value() resolver. Also corrects the two stale "easy to mistake for finished" bullets near the top.
|
Ran a 3-lens adversarial review of this branch (S1 codec type-exactness / S2 replication correctness / S4+S5 test validity + hidden regressions), each finding to be independently verified before surfacing. Zero confirmed findings — all three reviewers returned clean (journal verified: genuine Specifically checked: the integer codec aliases |
* feat(physics): body_state / pose replication codec (H1 slice 1)
The first H1 (single-holon) slice and the dependency H0 flagged: a holon
replicates a participant's avatar pose as a node's data(), but no
cvc::physics type had a registered data-type name or a codec, so none
could cross the cluster.
Add cvc::physics::physics_codec (inc/cvc/physics/physics_codec.h,
src/cvc/physics/physics_codec.cpp), mirroring the
cvc::ariadne::stream::format_desc codec: a little-endian, fixed-width,
FIELD-BY-FIELD wire (never a struct memcpy, so trailing-bool padding and
ABI never leak) with the double serializer stream_codec lacked — put_f64
writes the IEEE-754 binary64 bit pattern as 8 LE bytes (memcpy, C++17-
clean, matching the built-in "cvc.f64.v1" double codec). body_state is 13
doubles (position, rotation, linear_velocity, angular_velocity) + an
active byte; pose is 7 doubles. The decoder bounds-checks via reader{ok}
and throws on truncation. register_physics_data_types(app&) and
register_physics_codecs(state_codec_registry&) key both the
registerDataType name and the registry codec on the identical strings
"cvc::physics::body_state" / "cvc::physics::pose" (codec_ids
"cvc.physics.body_state.v1" / ".pose.v1"), so dataTypeName() always
matches a registered codec — the #767 S1 mismatch class.
Mass is intentionally NOT on the wire (body_state has none; it lives in
body_settings and stays server-owned). No floating-point math, so the TU
is not in the -ffp-contract=off exact-sources set.
physics_codec_test (8 cases): bit-for-bit round-trip of body_state and
pose including NaN / Inf / -0.0 / denormals; registry has() both;
registry key == dataTypeName<T>(); golden fixed-width LE wire (105 bytes
/ 56 bytes); truncated payloads throw. All green.
* refactor(physics): type names as inline accessors, not static-global consts
Replace the `extern const char* const kBodyStateTypeName / kPoseTypeName`
namespace-scope constants with inline accessors body_state_type_name() /
pose_type_name() returning the literal.
Two reasons:
- No static globals (project convention): the names are now a plain
value returned by a function, not a global variable.
- It also fixes a Windows-only CI failure. CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS
exports functions but NOT data symbols, so cvc.dll did not export the
`const char* const` consts; physics_codec_test (a separate exe linking
the DLL) failed to link them (LNK2019). An inline accessor is compiled
into each consumer, so it links across a shared-library boundary on
every platform with no exported symbol at all.
register_physics_data_types / register_physics_codecs and the test now
call the accessors. No wire or behavior change; 8 physics_codec_test
cases still green.
* feat(ariadne): headless avatar predict/reconcile/interpolate (H1 slice 2) The one genuinely new mechanism H1 needs on top of the clock_driver + data() replication it already has: a participant's avatar is server-authoritative, so a client must move its own avatar without a round-trip (predict), correct when the authoritative state arrives (reconcile), and show peers lag-free without guessing ahead (interpolate). New cvc::ariadne (headless, engine-free, no GL / state tree / transport -- that plumbing is the holon, slice 3): - avatar_sim: a deterministic linear kinematic integrator over a fixed dt (position += velocity*dt, pure double arithmetic, no trig), so the same intents reproduce bit-for-bit -- what makes replay sound. A stub for the physics::world save/restore stepping slice 3 swaps in behind the same shape. - predicted_avatar: client prediction + server reconciliation. Applies local intents immediately, logs the unacknowledged ones; on an authoritative state for a past tick, keeps the prediction when it already matches within tolerance (no correction), else snaps to the authoritative state and REPLAYS the inputs issued after that tick, so in-flight player input is corrected-on-top, not dropped. - interpolate_peer: a non-local peer between its last two authoritative samples, alpha clamped to [0,1) so it never extrapolates past newest. avatar_sim_test (6 cases, all green): bit-reproducible stepping; a perfect prediction reconciles with no correction; prediction ahead of the server keeps inputs pending; a server override snaps + replays in-flight ticks (and really moves the avatar); a matching ack after a correction does not re-snap or jump backward; peer interpolation never overshoots (and clamps negative/NaN alpha to the previous sample). All comparisons are lib-step vs lib-step, so bit-equality holds without -ffp-contract=off. Branches off master (has #766 clock_driver + #767 H0). Independent of #768 (the physics codec); the MVP pose rides scalar keys, so this does not need the typed-object codec. Next: S3 wires this into a cvc::ariadne::holon on its own clock_driver. * fix(ariadne): guard reconcile against stale/duplicate acks; pin tol + integrator tests Adversarial review of the predict/reconcile mechanism surfaced three items, all addressed here: - [med] predicted_avatar::reconcile had no guard against a non-monotonic server_tick. A reordered or duplicate ack (tick <= one already reconciled) failed the lookup, fell into the diverged branch, and snapped the avatar back to a stale authoritative state while dropping the inputs between the two acks (a visible backward rubber-band). Real datagram transports reorder/duplicate, so this was a live latent hazard for S3's caller. Fix: track the highest reconciled server_tick and no-op (return false, touch nothing) on any ack at or below it. Test: StaleOrDuplicateAckIsIgnored. - [med] the held-vs-snap tolerance was never exercised at a nonzero distance — every "no correction" test had a bit-identical prediction (d==0), so a tol/tol2 or threshold-direction regression would ship green. Test: ReconcileHoldsWithinToleranceAndSnapsBeyond (0.5*tol held, 2*tol snaps). - [low] the determinism test was tautological (two in-process runs of a pure function). Replaced with IntegratesConstantVelocityToClosedForm: a closed-form n*v*dt check (EXPECT_NEAR, not a bit-exact golden, so FMA contraction across toolchains can't flip it) that catches a formula regression, plus a retained purity assertion. Noted for S3 (dismissed as out-of-scope for the headless stub): the tolerance is position-only (fine while the stub integrates no rotation; revisit when physics::world adds orientation), and _log is unbounded if a caller never acks (a real server acks every tick). avatar_sim_test now 8 cases, all green.
…lar keys (#773) * feat(ariadne): headless avatar predict/reconcile/interpolate (H1 slice 2) The one genuinely new mechanism H1 needs on top of the clock_driver + data() replication it already has: a participant's avatar is server-authoritative, so a client must move its own avatar without a round-trip (predict), correct when the authoritative state arrives (reconcile), and show peers lag-free without guessing ahead (interpolate). New cvc::ariadne (headless, engine-free, no GL / state tree / transport -- that plumbing is the holon, slice 3): - avatar_sim: a deterministic linear kinematic integrator over a fixed dt (position += velocity*dt, pure double arithmetic, no trig), so the same intents reproduce bit-for-bit -- what makes replay sound. A stub for the physics::world save/restore stepping slice 3 swaps in behind the same shape. - predicted_avatar: client prediction + server reconciliation. Applies local intents immediately, logs the unacknowledged ones; on an authoritative state for a past tick, keeps the prediction when it already matches within tolerance (no correction), else snaps to the authoritative state and REPLAYS the inputs issued after that tick, so in-flight player input is corrected-on-top, not dropped. - interpolate_peer: a non-local peer between its last two authoritative samples, alpha clamped to [0,1) so it never extrapolates past newest. avatar_sim_test (6 cases, all green): bit-reproducible stepping; a perfect prediction reconciles with no correction; prediction ahead of the server keeps inputs pending; a server override snaps + replays in-flight ticks (and really moves the avatar); a matching ack after a correction does not re-snap or jump backward; peer interpolation never overshoots (and clamps negative/NaN alpha to the previous sample). All comparisons are lib-step vs lib-step, so bit-equality holds without -ffp-contract=off. Branches off master (has #766 clock_driver + #767 H0). Independent of #768 (the physics codec); the MVP pose rides scalar keys, so this does not need the typed-object codec. Next: S3 wires this into a cvc::ariadne::holon on its own clock_driver. * fix(ariadne): guard reconcile against stale/duplicate acks; pin tol + integrator tests Adversarial review of the predict/reconcile mechanism surfaced three items, all addressed here: - [med] predicted_avatar::reconcile had no guard against a non-monotonic server_tick. A reordered or duplicate ack (tick <= one already reconciled) failed the lookup, fell into the diverged branch, and snapped the avatar back to a stale authoritative state while dropping the inputs between the two acks (a visible backward rubber-band). Real datagram transports reorder/duplicate, so this was a live latent hazard for S3's caller. Fix: track the highest reconciled server_tick and no-op (return false, touch nothing) on any ack at or below it. Test: StaleOrDuplicateAckIsIgnored. - [med] the held-vs-snap tolerance was never exercised at a nonzero distance — every "no correction" test had a bit-identical prediction (d==0), so a tol/tol2 or threshold-direction regression would ship green. Test: ReconcileHoldsWithinToleranceAndSnapsBeyond (0.5*tol held, 2*tol snaps). - [low] the determinism test was tautological (two in-process runs of a pure function). Replaced with IntegratesConstantVelocityToClosedForm: a closed-form n*v*dt check (EXPECT_NEAR, not a bit-exact golden, so FMA contraction across toolchains can't flip it) that catches a formula regression, plus a retained purity assertion. Noted for S3 (dismissed as out-of-scope for the headless stub): the tolerance is position-only (fine while the stub integrates no rotation; revisit when physics::world adds orientation), and _log is unbounded if a caller never acks (a real server acks every tick). avatar_sim_test now 8 cases, all green. * feat(ariadne): cvc::ariadne::holon — clock + avatar bundle on its own clock_driver (H1 slice 3) The holon object the roadmap §5h names: a server-authoritative, client-replicable bundle of ONE world_clock and the participant avatars on it. The server drives the clock and owns every avatar's authoritative pose; a client runs a replica that predicts its own avatar (slice 2) and reconciles to the authoritative state, and interpolates peers. cvc::ariadne::holon (inc/cvc/ariadne/holon.h, src/cvc/ariadne/holon.cpp): - Owns its world_clock AND cvc::clock_driver BY VALUE (not the app-global clock), so two holons in one process never fight over time. - Drives avatar integration as a clock_driver STEP observer and records the presentation alpha as a FRAME observer — the same step-then-present order tick_scene uses, so a frame observer never reads a pose the step has not yet produced. - Per participant: server integrates the latest intent into the authoritative pose over the quanta that fell due; a client-local avatar predicts + reconciles (slice 2's predicted_avatar); a peer is driven by authoritative samples and interpolated at the frame alpha. A client may set an intent only on its OWN avatar, never a peer — the proposed/ authoritative split (the verb path that enforces it server-side is slice 4). Deliberately the headless clock + avatar-pose core: the Jolt cvc::physics::world (replacing the linear avatar_sim behind the same step shape) and the cvcGL scene_realize / physics_scene_binding rendering are later slices, so this stays deterministically testable with no GL / state tree / transport. holon_test (5 cases, all green): each holon owns a distinct clock (advancing one never ticks another); a server + client holon reproduce slice 2's convergence through the holon API (perfect prediction holds bit-for-bit; a server +y override snaps the client off its wrong +x prediction); a client cannot drive a peer; a peer interpolates through the frame alpha and never overshoots. Stacks on #770 (slice 2). The scalar-key state wiring that connects this to replication lands with slice 4 (the intent/pose verb path). * fix(ariadne): holon review fixes — peer sample guard, explicit non-movable, replay/stale tests Adversarial review of the holon surfaced six items; addressed here: - [med] the peer interpolation path ignored server_tick while the local path guards against stale/duplicate/reordered acks. A reordered peer sample would rewind the interpolation window (prev <- newer, newest <- older) and jerk the peer backward; a duplicate would collapse it. apply_authoritative's peer branch now applies the same monotonic guard (drop a sample whose server_tick <= the newest applied). Test: PeerIgnoresStaleOrDuplicateSample. - [med] the convergence tests always acked the client's latest tick, so reconcile drained the whole log and the replay-suffix path (and on_step's tick numbering) was never exercised — an off-by-one would pass. Added ClientReplaysInFlightInputsOnServerCorrection: a divergent authoritative state for a PAST tick snaps the client and replays its inputs after that tick; the result is checked against a reference, so a wrong tick/replay-suffix fails. - [low/med] ClientCannotDriveAPeer passed vacuously (a peer is never intent-driven). Reframed as PeerIsSampleDrivenNotIntentDriven, stating what it actually proves; the set_intent guard stays (defensive). - [low] the holon was only implicitly non-movable (via world_clock's mutex). Made it explicit (deleted copy/move) so a future movable-clock refactor can't silently re-enable a move and dangle the observers' this. - [low] dropped the dead `have_prev` field; the peer window now tracks `have_sample` + `last_sample_tick`. - [low] strengthened the peer no-overshoot assertion with a lower bound (advanced most of the way, not frozen). No memory-safety/crash bugs were found. holon_test now 7 cases, green. * feat(ariadne): holon <-> state-tree sync over scalar keys (H1 slice 4 core) Wires cvc::ariadne::holon to the replicated conference room tree over plain SCALAR leaf keys, under the participant subtree the conference already uses (conf.rooms.<room>.participants.<pid>, via room_participant_path): <participant>.pose.pos/orient/vel.{...} + .pose.server_tick server-written <participant>.intent.move.{x,y,z} client-written cvc::ariadne holon_sync.{h,cpp}: write_pose/read_pose and write_intent/read_intent. Scalar leaves (not a typed body_state data() object) are deliberate — they already journal + replicate through the existing "conf" interest filter with zero transport changes, and sidestep the data()-object work (§52). Each double is written round-trip-safe (%.17g, portable across toolchains, no std::to_chars FP dependency) so a reader recovers it exactly. Writes use the room tree's touch-before-value idiom so a fresh node's first write is not lost to the distributed-state attach race. Authority (the proposed/authoritative path pair): a client writes only its own .intent.*; the server reads every participant's .intent, integrates it into the holon, and is the SOLE writer of .pose + .server_tick; the client reads .pose read-only and reconciles. Writes go through a C++ host (never a DSL state-set, since ui.docs.* is excluded from replication). H1 is loopback-trusted sole-writer-by-convention; the connection<->pid admission binding is the §5e/mesh increment. holon_sync_test (4 cases, green): pose + intent scalar round-trip recover binary64 exactly; read returns false when absent; and an END-TO-END over a real two-shard state_transport_inproc — the client proposes intent, it replicates to the server, the server holon integrates it and writes the authoritative pose, that replicates back, and the client reconciles and tracks the authoritative avatar bit-for-bit across 8 frames. Stacks on #772 (slice 3). The remaining live-integration piece (a C++ room-set-intent verb + arisrv hosting the server holon + a per-room client pump) is the follow-on that the live two-client cyprty test exercises. * feat(arisrv): server-authoritative holon loop + room-set-intent verb (H1 slice 4b) Wire the holon into the live relay so a participant's avatar becomes server-authoritative across the wire. - holon_sync: server_pump(holon, app, room, wall_dt=-1) — one authoritative frame against the replicated room tree: for every present participant (room_roster) add it to the holon if new, read its proposed .intent, advance the holon, and write each authoritative .pose + .server_tick. wall_dt defaults to measured real time (the relay); a test injects a fixed dt for a deterministic step count. - arisrv: construct one server holon for the room and call server_pump each relay tick, right after room_relay_tick, so the authoritative pose writes are local shard mutations the existing drain_local block fans to every client on both transports. (~5 Hz relay cadence -> the holon catches up several quanta per tick and clients interpolate; a smoother authoritative rate is a later tuning.) - room_intrinsics: a (room-set-intent HANDLE MOVE-X MOVE-Y MOVE-Z) C++ verb so a client proposes only its OWN avatar's intent (pid from the room-join handle, so it cannot propose for a peer). Writes via the holon_sync intent keys; a C++ verb because the DSL chroot ui.docs.* is excluded from replication (same reason room-join is C++). holon_sync_test gains ServerPumpIntegratesRosterFromIntents: two participants join + propose intent on a client app, replicate, and server_pump integrates each from its own intent and writes authoritative poses that replicate back (A moves +x, B moves +y). 5 cases green; arisrv builds + links. Extends #773 (slice 4). The remaining client-side piece (arigl driving its local avatar's intent from input + rendering peer avatars from the replicated poses) is S5/S6 + the live two-client cyprty test. * ci: run the master-targeted matrix (base retargeted from feat/h1-pose-reconcile)
What — H0 (holon prereq) closure
H0 of the holon plan is "close
data()object replication (DISTRIBUTED_STATE §52)". A code audit found the core path is already landed and live (the roadmap's "NOT STARTED" was stale): per-shard codec registry →dataChanged→set_dataemit (inline / content-addressedblob_ref) →apply_remoteinline-decode / blob-hydrate →node.data(), plus snapshot capture+ingest, across all four transports. The existing inproc tests already prove the round-trip, blob-ref, snapshot, and stream-descriptor cases.This PR closes the small gaps that remained to meet §52's own design + testing bullets:
data()key fix (state_codec_registry.cpp).register_builtin_codecskeyed integer codecs on the C++ spellings (int32_t…), butstate::dataTypeName()returns the app's registered names (int,unsigned int,unsigned short,int64,uint64), so an integerdata()silently failed to replicate (encode threw "no codec", swallowed). Added codec aliases under the app spellings. Real bug.type_nameacross apply + snapshot (state.{h,cpp},state_sync_adapter.cpp,state_cluster_shard.cpp). The emit carriedvalueTypeName()butapply_remote::set_valuestamped"std::string"andsnapshot()never captured it. Added a signal-freestate::valueTypeName(const std::string&)setter and wired apply + snapshot capture/ingest.state_transport_ipc_test.cpp): inline object + builtin-int (exercises S1 over a real transport) + blob-ref, closing §52's "both IPC and gRPC" testing bullet (gRPC coverage exists but compiles out by default).distributed_state_session_test.cpp): an appliedset_dataemits no local mutation; a quiescent re-pump moves nothing.DISTRIBUTED_STATE_ROADMAP.md §52): rewritten from "NOT STARTED" to the landed state + accurately-scoped deferrals, so a future planner doesn't rebuild what exists.Deferred (documented in §52, not done here)
format_desccodec on a production shard. The codec exists but no production shard registers it — because none currently replicates/streamsdescriptors: the conference relay scopes replication toconf.*(enforce_interest), descriptors are rooted at the bare stream token, and cross-node media ridesstream_bridge/media_frame, not the state tree. Registering it on the arisrv shard today would be dormant. It belongs with whatever first replicates descriptors (a holon / a conference feature). Covered in isolation byTwoNodeStreamDescriptorDiscovery.value()blob resolver.Testing
All affected suites green:
distributed_state_session(20, +3 new),state_transport_ipc(19, +1 new),state_snapshot_protocol(4),state_sync_adapter(8),state_codec_registry(9). New tests:TwoNodeInprocIntegerDataReplication,TwoNodeValueTypeNamePreserved,SetDataApplyDoesNotEcho,StateTransportIpcTest.DataPayloadReplicatesOverIpc(8× repeat clean). Formatted with clang-format 18.1.8.Context
First step of the holon plan after the clock_driver time-slice framework (#766). Next: H1 — the single holon (shared scene + avatars). Note for H1: replication is TRUST-ALL and
origin_node_idis forgeable, so the first physics payloads (pose/body_state) must go through the server-mediated-write / propose→veto→override path, not direct replicated writes — and no codec exists for any physics type yet (authoring one is an H1 dependency).