feat(meta_quest_camera): streaming MJPEG → MP4 (no on-device disk) - #18
Merged
Conversation
…e disk)
The previous design recorded MJPEG-AVI on the Quest itself and only
pulled the resulting ~90 MB-per-eye file *after* /recording/stop
returned. On a real WiFi link that meant a 30 s+ blocking wait that
looked like a hang, fragile resume logic, and four serialised file
downloads. Stop becomes instant with this change and the file lands
on disk in real time.
The same /preview/{eye} MJPEG channel that powers the viewer panel
now also feeds a passthrough MP4 writer when recording is active.
One TCP connection per eye carries both responsibilities; recording
is a cheap sink-attach toggle, so the viewer feed never blanks across
a record/stop cycle.
What's new on the Python side
-----------------------------
* mp4_writer.py — StreamingVideoRecorder muxes raw JPEG bytes into
MP4 (mjpeg codec, PyAV) without re-encoding, plus a sidecar
timestamps JSONL that now includes BOTH the host-projected and
Quest-native nanosecond timestamps for post-hoc clock-drift
correction.
* preview.py — MjpegPreviewConsumer gains a set_frame_sink() hook
that delivers the raw MjpegFrame to a per-recording callback. Sink
exceptions are caught so a misbehaving recorder cannot kill the
viewer feed. MjpegFrame.quest_native_ns is parsed from the new
X-Quest-Native-Ns header (None on older Quest builds).
* stream.py — MetaQuestCameraStream's start_recording attaches sinks
to the existing consumers and stop_recording detaches and closes.
No more Quest-side /recording/start, file pull, or DELETE.
* file_puller.py + timestamps.py removed (and their tests).
Quest-side changes ship in the paired opengraph-studio commit:
PreviewFrameSlot now carries both timestamps, /preview/{eye} emits
X-Quest-Native-Ns, defaults bump to 1280x720 / decimate=1 / Q=75.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Refactor the Quest passthrough camera adapter to stream-record rather than pull files off device after stop. The same
/preview/{eye}MJPEG channel that powers the viewer panel now also feeds a passthrough MP4 writer when recording is active. Stop is instant; files land on disk in real time.Pairs with opengraph-studio#74 (TBD) which delivers the Quest-side companion app changes.
Why
The old design recorded MJPEG-AVI on the Quest itself and pulled ~90 MB-per-eye files over HTTP after
/recording/stop. On a real WiFi link that meant 30 s+ blocking after stop, fragile resume logic, and four serialised file downloads. Streaming eliminates all of that.Bandwidth measurement on the live AP showed:
Changes
New:
mp4_writer.py—StreamingVideoRecorder*.timestamps.jsonlwith both host-projected and Quest-native ns per frame (post-hoc clock-drift correction)start/stop, thread-safewrite_frameModified:
preview.py—MjpegPreviewConsumerset_frame_sink(sink)hook delivers rawMjpegFrameto a per-recording callbackMjpegFrame.quest_native_nsparsed from newX-Quest-Native-Nsheader (None on older Quest builds)Refactored:
stream.py—MetaQuestCameraStreamstart_recordingattaches sinks to existing consumers (no Quest-side/recording/startcall)stop_recordingdetaches sinks, closes recorders, returns report from in-memory state — no file pullcompleted/partial(one eye missing or mux errors) /failed(no frames at all)Removed
file_puller.py+test_file_puller.pytimestamps.py+test_timestamps.pyTest Plan
tests/unit/adapters/meta_quest_camera/tests pass (5 new tests added for streaming flow)mjpeg/720x720/30fps🤖 Generated with Claude Code