Skip to content

Add H.264 WebRTC signaling and snapshots, moved from the Home Assistant integration - #65

Merged
chiconws merged 4 commits into
devfrom
feature/webrtc-signaling
Oct 28, 2026
Merged

chiconws merged 4 commits into
devfrom
feature/webrtc-signaling

Conversation

@chiconws

Copy link
Copy Markdown
Collaborator

Summary

Moves the H.264 WebRTC signaling and the camera snapshot from the Home Assistant integration into the client, so any caller can open a WebRTC session or grab a still image without knowing how the two hardware families differ. Adds a streaming guide to docs/.

Changes

  • open_webrtc_signaling() picks the signaling endpoint for the detected hardware and returns a nanokvm.webrtc.WebRTCSignaling session on its own WebSocket. One offer goes in, and one answer and trickled candidates come out, on either family.
  • Pro offer split and answer merge. The Pro negotiates video and audio as two peer connections. The session splits the caller's offer per media kind, merges the answers back into one, holds early candidates until the merged answer, and maps candidates both ways. It also sends the Pro's application heartbeat, because the Pro ignores WebSocket pings. split_pro_webrtc_offer() and merge_pro_webrtc_answers() are public for callers that run their own signaling.
  • NanoKVMVideoModeError is raised from the iteration when the Pro reports video status -4, which means another video mode took the encoder.
  • get_snapshot(timeout=20.0) (non-Pro) returns the first raw JPEG of the MJPEG stream. The Pro refuses it with NanoKVMNotSupportedError before any request, because opening MJPEG there stops WebRTC (Reading one frame from /api/stream/mjpeg freezes active H.264/H.265 streams sipeed/NanoKVM-Pro#157).
  • Sync client. open_webrtc_signaling() joins open_ws() as async-only, because its WebSocket belongs to the client's event loop. get_snapshot() works as a blocking call.
  • Docs. docs/streaming.md covers snapshots, MJPEG and the WebRTC session step by step, including the Pro's single encoder. docs/api.md keeps the method table and links to it.

Notes

  • The non-Pro firmware sends an ice-servers event (its configured STUN/TURN) on connect, before the offer, since 2.4.2. The session doesn't expose it yet. It only matters when TURN is configured on the device.
  • On the non-Pro, MJPEG and H.264 don't stop each other, but H.264 fell to about 55% of its frame rate while MJPEG was open (PCIe 2.5.2, H.264 direct).

Behaviour changes

None. All methods are additions.

Testing

504 unit tests pass, 29 of them new. Against devices:

  • NanoKVM PCIe 2.5.2: get_snapshot() returned a complete 48 KB JPEG in 0.28 s.
  • NanoKVM Pro 1.2.15: get_snapshot() was refused without a request.
  • The integration's 653 tests pass with this branch in place of the released client, with its camera on open_webrtc_signaling().

open_webrtc_signaling() picks the signaling endpoint for the detected
hardware and returns a WebRTCSignaling session: one offer in, one answer
and trickled candidates out, on either family. For the Pro it splits the
browser offer into its separate video and audio peers, merges their
answers, maps candidates both ways and keeps the session alive with the
application heartbeat. A Pro video-status of -4 raises
NanoKVMVideoModeError.

Moved from the Home Assistant integration.
open_webrtc_signaling() joins open_ws() in the sync client's async-only
set: its session belongs to the client's event loop. Rename
home_assistant_candidate_for_kind to client_candidate_for_kind, the
counterpart of upstream_candidate_for_kind.
get_snapshot() returns the first raw JPEG of the MJPEG stream within a
timeout. The Pro refuses it: its MJPEG and H.264 streams share one
encoder mode, so opening MJPEG would interrupt WebRTC (NanoKVM-Pro#157).

Moved from the Home Assistant integration's camera snapshot.
docs/streaming.md covers snapshots, the MJPEG streams and the WebRTC
signaling session step by step, with the Pro's single encoder and its
offer split and answer merge. The API reference keeps the method table
and points to it.
@chiconws
chiconws merged commit 22593c8 into dev Oct 28, 2026
2 checks passed
@chiconws
chiconws deleted the feature/webrtc-signaling branch October 28, 2026 22:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant