feat(meta_quest): propagate clock_domain + connection health events - #7
Merged
Merged
Conversation
Quest 3 samples arrive via WiFi UDP from a remote device, so they don't share the host monotonic clock domain. Tag them with "remote_quest3" and a 10 ms WiFi uncertainty so downstream sync tooling can account for the wireless jitter. Also revive the dead _last_packet_mono field: a 3-state watchdog (waiting/connected/lost) now emits HEARTBEAT on first packet, DROP after CONNECTION_TIMEOUT_S silence, and RECONNECT when the stream resumes, mirroring the recorder's is_connected semantics. SampleEvent gains an optional clock_domain field; orchestrator respects it when set and falls back to host_id otherwise, so other adapters are unaffected. Co-Authored-By: Claude Opus 4.6 <[email protected]>
styu12
added a commit
that referenced
this pull request
Apr 15, 2026
…ample New example `examples/meta_quest/` pairs with the newly-landed MetaQuestCameraStream (PR #10) and the MetaQuestHandStream that's been on main since PR #7. Captures, in one synchronised session: - ``quest_tracking`` (UDP 14043, 72 Hz) — 26-joint hand skeleton × 2 hands + head pose, all timestamped in the ``remote_quest3`` clock domain. - ``quest_cam`` (HTTP 14045) — 720p stereo passthrough MJPEG recorded on the Quest and pulled to output_dir after stop. Viewer extensions to render the hand skeleton in 3-D: 1. State / poller / SSE backend now passes vector-valued channels (e.g. ``hand_joints``: 156 floats) through to the frontend under a new ``pose`` field on each SensorEvent. ``StreamStatsBuffer`` retains only the latest sample per list channel — 3-D pose panels render instantaneous state, buffering 300 × 500-float poses per adapter would blow memory. 2. New ``Quest3PosePanel`` component (React Three Fiber), ported from opengraph-studio/showcase with egocentric framing and the viewer's warm-paper palette. Draws the 26-joint skeleton per hand (cyan / orange) plus an RGB-axis head rig inside the same 0.84-m workspace box used by the showcase. 3. ``SensorPanel`` dispatcher routes streams with ``hand_joints`` payloads to the new 3-D panel; roll/pitch/yaw IMUs still go to the existing cube panel; everything else falls back to the multi-channel line chart. All existing adapters keep their rendering unchanged. 4. Frontend deps: ``three``, ``@react-three/fiber``, ``@react-three/drei``, ``@types/three``. Bundle grows ~1 MB gz which is acceptable for a desktop-launched viewer. Backward compatibility: adapters that emit only scalar channels never populate ``latest_pose``, ``pose`` on the SSE event stays ``null``, and the hook / dispatcher fall through to the existing chart path — zero behaviour change for non-Quest streams. Co-Authored-By: Claude Opus 4.6 <[email protected]>
styu12
added a commit
that referenced
this pull request
Apr 15, 2026
…ample (#12) New example `examples/meta_quest/` pairs with the newly-landed MetaQuestCameraStream (PR #10) and the MetaQuestHandStream that's been on main since PR #7. Captures, in one synchronised session: - ``quest_tracking`` (UDP 14043, 72 Hz) — 26-joint hand skeleton × 2 hands + head pose, all timestamped in the ``remote_quest3`` clock domain. - ``quest_cam`` (HTTP 14045) — 720p stereo passthrough MJPEG recorded on the Quest and pulled to output_dir after stop. Viewer extensions to render the hand skeleton in 3-D: 1. State / poller / SSE backend now passes vector-valued channels (e.g. ``hand_joints``: 156 floats) through to the frontend under a new ``pose`` field on each SensorEvent. ``StreamStatsBuffer`` retains only the latest sample per list channel — 3-D pose panels render instantaneous state, buffering 300 × 500-float poses per adapter would blow memory. 2. New ``Quest3PosePanel`` component (React Three Fiber), ported from opengraph-studio/showcase with egocentric framing and the viewer's warm-paper palette. Draws the 26-joint skeleton per hand (cyan / orange) plus an RGB-axis head rig inside the same 0.84-m workspace box used by the showcase. 3. ``SensorPanel`` dispatcher routes streams with ``hand_joints`` payloads to the new 3-D panel; roll/pitch/yaw IMUs still go to the existing cube panel; everything else falls back to the multi-channel line chart. All existing adapters keep their rendering unchanged. 4. Frontend deps: ``three``, ``@react-three/fiber``, ``@react-three/drei``, ``@types/three``. Bundle grows ~1 MB gz which is acceptable for a desktop-launched viewer. Backward compatibility: adapters that emit only scalar channels never populate ``latest_pose``, ``pose`` on the SSE event stays ``null``, and the hook / dispatcher fall through to the existing chart path — zero behaviour change for non-Quest streams. Co-authored-by: Claude Opus 4.6 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
clock_domain=\"remote_quest3\"+uncertainty_ns=10ms메타데이터를 주입해 downstream 동기화 툴이 WiFi 지터를 반영할 수 있게 함._last_packet_mono를 실제 사용하는 3-state watchdog(waiting/connected/lost)으로 연결 상태를 추적하고HEARTBEAT/DROP/RECONNECThealth event 발행.is_connected시맨틱을 동일하게 복원 (2초 타임아웃 기반 프로퍼티).Changes
src/syncfield/types.py:SampleEvent에 optionalclock_domain: str | None = None필드 추가 (기본 None, 기존 어댑터 영향 없음).src/syncfield/orchestrator.py:_make_sample_handler가event.clock_domain or host_id로 어댑터 override 존중.src/syncfield/adapters/meta_quest.py:CLOCK_DOMAIN/UNCERTAINTY_NS클래스 상수 추가 후SampleEvent에 전달.is_connected프로퍼티 부활 (recorder와 동일 시맨틱)._receive_loop의 1초 recv 타임아웃을 watchdog tick으로 활용하여 DROP 감지.tests/unit/adapters/test_meta_quest.py: 6개 신규 테스트 추가.TestClockMetadata: SampleEvent에 remote_quest3 도메인과 10ms 불확실성 전파 검증.TestConnectionHealth:is_connected초기/수신/재개 상태와 HEARTBEAT/DROP/RECONNECT 발행 순서 검증.Test Plan
uv run pytest tests/unit/adapters/test_meta_quest.py— 26 passed (기존 20 + 신규 6)clock_domain필드가\"remote_quest3\"로 찍혔는지 확인Context
opengraph-studio/recorder의 quest3_tracker.py 대비 이식 검토에서 발견한 2개 누락사항 보강. 나머지 차이점(Manus 호환 채널 누락, JSON decode 에러 로깅 빈약)은 우선순위 낮아 추후 처리.
🤖 Generated with Claude Code