Skip to content

test(ariadne): the room relay tests count what happened, not what fit in N seconds - #867

Merged
transfix merged 2 commits into
masterfrom
fix/room-relay-loaded-runner
Oct 29, 2026
Merged

transfix merged 2 commits into
masterfrom
fix/room-relay-loaded-runner

Conversation

@transfix

Copy link
Copy Markdown
Owner

Five RoomRelayLoopbackTest cases fail on loaded runners in package-linux / libcvc-debug-grpc and turn PRs' lanes red: OuterPeerGetsPushesNotLiveFrames, MiddleGetsThumbnailsAndAudio, LateJoinerBootstrapIsFiltered, ChurnSmoke and AViewersVideoRungShapesWhatItGets.

Root cause

run_room runs a fixed stretch of wall time, and a starved loop skips ahead instead of catching up. A slow runner gets fewer loop passes, so fewer room steps, client frames and camera and mic sends (a client sends one of each a pass at most). The checks counted those per wall second.

The failing jobs' numbers:

job what it shows
#854's d24c7e6 run, job 122650960389 OuterPeer: C sent 8 camera frames of the 10 checked (3/3 attempts). Middle: 21 of 31. ChurnSmoke: 7 of 10 tier generations, and 8 frames a second where more than 10 was checked. LateJoiner: 2.15 to 2.28 s against < 1.5
master 83b6020's push run, job 122277740072 the same four: 8 to 9, 22 to 24, 6 generations, 1.79 to 2.19 s
perf/locks-l1 a56cb12 (122651470430), fix/native-stack-real-frame e5ca4fe (122924848460) all five, alongside a GeometryTest.QualityImproveTetrahedralMesh timeout: very slow runners. AViewersVideoRung: 34 to 40 camera frames and 36 to 37 mic chunks of the 41 checked. LateJoiner up to 3.25 s

The same lane passes when the runner keeps up: master 6744512 and 45dab15, #847's a4179b5, #861's 70ae2be.

Fix

Each window keeps its minimum length and then runs on until what its checks count has happened:

  • OuterPeer: until C has sent 10 camera frames. The push bounds stay "1.5 to 12 a second", now over the window's measured wall time. At two seconds that is 3 to 24, as before.
  • Middle: until C has sent 31. The thumbnail bounds stay "2 ± 1 a second plus the first frame", over the window's wall time. At two seconds that is 2 to 7, as before. The gate's timer runs on wall time, so the rate is the right measure.
  • ChurnSmoke: C and D swap every two room steps instead of every 100 ms of wall time, so a slow runner still swaps on every other step. This runs at least 3 s and until the tiers have flipped ten times. Back together, each publisher sends more than ten frames, and every receiver must get every frame sent since. That replaces "more than ten in one second", which a slow runner can't send.
  • AViewersVideoRung: "off" and "full" run until A has sent more than 40 frames of the stream they count, and B must get every one of them (mic in "off", camera in "full"). "Thumb" is unchanged: it is a wall-time rate and passes under load.
  • LateJoiner: the wall-clock bound (on.seconds < 1.5) is removed, not loosened. Its comment says it only bounds the bootstrap "well inside the 2 s hold", standing in for "the filter, not the timeout, released it". held == 1 and timeouts == 0 check that exactly. On loaded runners the bootstrap took 1.8 to 3.3 s, past the hold, and the timeout count stayed 0 every time. The rest of that time is the join's trip to the relay and the bootstrap's trip back. The test now prints the time and the room steps it took.

Two checks are stricter than before: ChurnSmoke's and AViewersVideoRung's "full" now require every frame sent, not a count. That is what "full rate" means here; DefaultRoomUnchangedUnderConferencePreset already asserts inner media is lossless.

Tested

  • The edited test compiles (MSVC 14.44 Release, CVC_ENABLE_WS=ON, no warnings), and clang-format 18.1.8 leaves it unchanged.
  • Not run locally. This box's cvcpkg protobuf.lib and grpc.lib were built with a newer MSVC than its 14.44.35207: the link of cvc.dll with either transport fails on __std_rotate and __std_find_first_not_of_trivial_pos_1. CI runs it in package-linux / libcvc-debug-grpc and the other gRPC lanes.
  • A PR run only shows the loaded case if its runner is loaded. The timings above are the evidence for the mechanism; the new windows are bounded by deadlines (8 to 10 s) that every failing count would have met at the rate its job measured.

… in N seconds

Five RoomRelayLoopbackTest cases fail on a loaded runner in package-linux / libcvc-debug-grpc:
OuterPeerGetsPushesNotLiveFrames, MiddleGetsThumbnailsAndAudio, LateJoinerBootstrapIsFiltered,
ChurnSmoke and AViewersVideoRungShapesWhatItGets. run_room runs a fixed stretch of wall time, and a
starved loop skips ahead instead of catching up, so a slow runner gets fewer loop passes: fewer
room steps, client frames and camera and mic sends (one a pass at most). The checks counted those
per wall second. In the failing jobs the loop managed 4 to 17 passes a second where 30 is due:
C sent 8 of 10 camera frames, 21 of 31, 7 of 10 tier generations, 5 to 8 frames a second.

Each window now keeps its minimum and runs on until what its checks count has happened:
- OuterPeer: until C has sent 10 camera frames. The push bounds stay 1.5 to 12 a second, now over
  the window's measured wall time (3 to 24 at two seconds, as before).
- Middle: until C has sent 31. The thumbnail bounds stay 2 +/- 1 a second plus the first frame,
  over the window's wall time (2 to 7 at two seconds, as before).
- ChurnSmoke: C and D swap every two room steps, not every 100 ms of wall time, until the tiers
  have flipped ten times. Back together, each publisher sends more than ten frames, and each
  receiver must get every frame sent since; that replaces "more than ten in a second".
- AViewersVideoRung: off and full run until A has sent more than 40 frames of the stream they
  count, and B must get every one of them.
- LateJoiner: the wall-clock bound (under 1.5 s) is dropped. It stood in for "the filter, not the
  timeout, released it", which held == 1 and timeouts == 0 check exactly. On loaded runners the
  bootstrap took 1.8 to 3.3 s, past the 2 s hold, with no timeout. The time and the room steps are
  printed.
@transfix
transfix merged commit efdda85 into master Oct 29, 2026
24 checks passed
@transfix
transfix deleted the fix/room-relay-loaded-runner branch October 29, 2026 20:54
transfix added a commit that referenced this pull request Oct 29, 2026
The project version and the cvcpkg recipe family move to 3.7.4. Every recipe is at cvc_revision 1
already, so none restarts.

3.7.4 carries the whole plant ledger on every plant (#876): plant_sample's vehicle
contact and what each plant measures, the dynamic plants' building and vehicle contacts and
attitude, nav_stats' vehicle_hits, cvc/nav/plant_ledger.h, and publish_nav_stats writing every
item, n/a where a plant does not measure it. demo3's Ledger screen (CVC-DBG/cvcdbg#256) reads it.
It also carries SDL3 owning cvcGL's onscreen window (#829), state_exec's state-incr, state-cas and
state-swap (#860), and Ariadne's window fitting (#828) and room relay test counting (#867).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant