Keep the camera's sender when the camera turns off

[sivert-io]

The engine half of GRYT-1329. The task id is left out of the branch and the title so this merge doesn't close the task. The client PR, Gryt-chat/client#625, closes it.

What was wrong

Turn your camera on, off and on again, then share your screen. Everyone else in the call gets "'s screen isn't coming through" for as long as the share lasts. Share first and then turn the camera off and on, and the camera doesn't come back.

The SFU is always the offerer. It opens each connection with four receive transceivers: microphone, camera, screen video and screen audio (CreatePeerConnection in the SFU's internal/webrtc/peer.go). A client can only send on an m-line the SFU offered. removeVideoTrack took the camera off with pc.removeTrack. That leaves the transceiver behind as one that has sent, and addTrack never reuses those. So the camera coming back took the next free video m-line, which was the screen's. The share then got a transceiver with no mid, and the SFU never received it.

Alice's transceivers, read in Chrome after each step, against a local server and SFU from main. Each entry is mid:kind:currentDirection:track:

main, voice 0.5.8
camera on        1:video:sendonly:camera   2:video:inactive:-
camera off       1:video:recvonly:-        2:video:inactive:-
camera on again  1:video:recvonly:-        2:video:sendonly:camera
share            1:video:recvonly:-        2:video:sendonly:camera   null:video:null:screen

this branch
camera on        1:video:sendonly:camera   2:video:inactive:-
camera off       1:video:sendonly:-        2:video:inactive:-
camera on again  1:video:sendonly:camera   2:video:inactive:-
share            1:video:sendonly:camera   2:video:sendonly:screen

On main a second round is worse. The camera lands on the screen's transceiver, which never got a mid, so the camera doesn't get out either. On this branch a second round stays on mids 1 and 2.

What changed

removeVideoTrack pauses the camera's sender with replaceTrack(null) and keeps it, like removeScreenVideoTrack does. The next addVideoTrack takes the replace path and puts the new track on the same sender. Nothing in the SDP changes, so it no longer asks the SFU to renegotiate.

check-camera-sender runs useSFU through the engine harness: camera on, off and on, then a share, twice. It checks that the camera keeps one sender, that removeTrack never runs, and that pausing and resuming don't renegotiate. Against main it fails with "turning the camera off removed its sender".

Where to look

• The camera keeps its first stream id for the whole connection. replaceTrack doesn't change the msid, so a camera that comes back is still published under the first camera stream's id. Apps have to announce that id in voice:camera:state. The client does that for the screen, and Turning the camera off and on left no room for a screen share (GRYT-1329) client#625 does it for the camera. I ran this branch with the client's old id handling, and the camera never got a frame for the other guest after it came back. So the client's pin and its id change go in together.
• The phone announces a new id each time. Mobile's useCamera.ts sends next.id on every camera-on, so it has to change when it takes this release. That's GRYT-1336. Its second screen share should have the same mismatch already.
• A race this widens. If a sender is paused before its first renegotiation lands, addTrack for the other role can reuse it, and the engine holds one sender under two roles. On main that already happens screen first: a share started and stopped during an SFU signalling outage hands its sender to the camera. With this change the camera-first order does it too. I reproduced both by holding the renegotiate message in the page. In practice it takes a signalling outage, so it's filed as GRYT-1337 with a likely fix.
• The SFU keeps the paused camera's track until the person leaves. It did after removeTrack too, since OnTrack waits for the connection to close. With this, a connection has one camera track, where main added another each time the camera came back.

Release order

1. Merge this.
2. Dispatch Release for @gryt/voice as 0.5.9.
3. Turning the camera off and on left no room for a screen share (GRYT-1329) client#625 pins 0.5.9. It needs yarn install for yarn.lock after the release, then CI can go green.
4. Mobile takes 0.5.9 in the same PR as GRYT-1336.

What fixed slots would need to change for several shares (GRYT-1335)

This fix keeps the camera and the screen on one m-line each, and neither cares which one it gets. Several windows shared at once, each as its own stream, won't fit in four fixed slots, though. For that:

• The SFU adds receive transceivers on demand. CreatePeerConnection makes exactly four, and the SFU is always the offerer, so a client can't add an m-line itself. Its renegotiate message would have to say how many video and audio senders it has, and the SFU would add receive transceivers to match before CreateOffer.
• The microphone stays at index 0. isMicrophone in handler_join.go finds the mic by position: transceiver 0, audio. New m-lines added after it keep that true.
• A share that's gone for good releases its track. The SFU holds every incoming track until the connection closes, because OnTrack waits on it. That's one idle track today. With windows added and removed through a long call, every viewer would collect dead tracks.
• The engine keeps a sender per window. It holds one per role today: camera, screen video, screen audio. A stopped window's paused sender should be reused before asking the SFU for another m-line, since addTrack won't reuse one that has sent.
• Screen state becomes a list. voice:screen:state carries one video and one audio stream id per person, and the client's senderStreamIds.ts keys ids by role.

🤖 Generated with Claude Code