Skip to content

feat(teleop): route operator audio to Go2 - #3268

Draft
ruthwikdasyam wants to merge 4 commits into
mainfrom
ruthwik/hostedteleop/6
Draft

feat(teleop): route operator audio to Go2#3268
ruthwikdasyam wants to merge 4 commits into
mainfrom
ruthwik/hostedteleop/6

Conversation

@ruthwikdasyam

@ruthwikdasyam ruthwikdasyam commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Contribution path

  • Small, safe change that does not need a tracking issue

Problem

Hosted Go2 teleoperation receives video and control data from the operator, but it does not route the operator's microphone audio to the robot speaker. The Go2 Air also lacks speaker hardware, so the integration must avoid repeatedly calling unsupported firmware APIs.

Solution

  • Add a subscribe-only WebRTC audio transport that shares the existing hosted broker provider.
  • Decode operator audio to PCM and bridge it to the Go2 audio-hub megaphone API.
  • Resample and mix incoming audio, normalize quiet speech, batch uploads, bound queued latency, and exit megaphone mode after idle or failed uploads.
  • Support auto, enabled, and disabled speaker modes so deployments can explicitly handle Go2 Pro and Go2 Air hardware.
  • Wire the bridge into the hosted Go2 Cloudflare blueprint without changing the existing video or command session.

How to Test

dimos run teleop-hosted-go2-transport -o go2audiobridgemodule.speaker=enabled

AI assistance

OpenCode with GPT-5.6 was used to review the implementation, address lifecycle cleanup, and verify the change. The author reviewed and understands the resulting code.

Checklist

  • I have read and approved the CLA.

# Conflicts:
#	dimos/teleop/hosted/README.md
#	dimos/teleop/hosted/blueprints/cloudflare.py
@codecov

codecov Bot commented Jul 29, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 91.04116% with 37 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
dimos/teleop/hosted/go2_audio_bridge.py 87.43% 12 Missing and 14 partials ⚠️
dimos/core/transport.py 73.52% 7 Missing and 2 partials ⚠️
...os/protocol/pubsub/impl/webrtc/providers/broker.py 0.00% 2 Missing ⚠️
@@            Coverage Diff             @@
##             main    #3268      +/-   ##
==========================================
+ Coverage   74.96%   75.21%   +0.24%     
==========================================
  Files        1117     1130      +13     
  Lines      106875   108585    +1710     
  Branches     9699     9998     +299     
==========================================
+ Hits        80123    81670    +1547     
- Misses      23945    24067     +122     
- Partials     2807     2848      +41     
Flag Coverage Δ
OS-ubuntu-24.04-arm 68.89% <91.04%> (+0.35%) ⬆️
OS-ubuntu-latest 70.95% <91.04%> (+0.32%) ⬆️
Py-3.10 70.95% <91.04%> (+0.31%) ⬆️
Py-3.11 70.95% <91.04%> (+0.32%) ⬆️
Py-3.12 70.95% <91.04%> (+0.31%) ⬆️
Py-3.13 70.95% <91.04%> (+0.31%) ⬆️
Py-3.14 70.95% <91.04%> (+0.32%) ⬆️
Py-3.14t 70.94% <91.04%> (+0.31%) ⬆️
SelfHosted-Large 29.17% <27.84%> (-0.07%) ⬇️
SelfHosted-Linux 35.60% <27.84%> (-0.22%) ⬇️
SelfHosted-macOS 34.66% <27.84%> (-0.24%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
dimos/core/test_webrtc_audio_transport.py 100.00% <100.00%> (ø)
dimos/robot/all_blueprints.py 100.00% <ø> (ø)
dimos/teleop/hosted/blueprints/cloudflare.py 100.00% <100.00%> (ø)
dimos/teleop/hosted/test_go2_audio_bridge.py 100.00% <100.00%> (ø)
...os/protocol/pubsub/impl/webrtc/providers/broker.py 34.57% <0.00%> (-0.09%) ⬇️
dimos/core/transport.py 68.50% <73.52%> (+0.49%) ⬆️
dimos/teleop/hosted/go2_audio_bridge.py 87.43% <87.43%> (ø)

... and 108 files with indirect coverage changes

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@ruthwikdasyam ruthwikdasyam left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hermes Agent Review

Found 1 issue worth fixing before marking this ready. CI is green, and the audio transport/blueprint wiring otherwise looks coherent.

⚠️ Warning

  • dimos/teleop/hosted/go2_audio_bridge.py:170 — failed uploads clear _megaphone_active without sending EXIT_MEGAPHONE, so the firmware can be left in megaphone mode after a partial upload failure.

✅ Looks good

  • Broker provider singleton config no longer forks on audio-only options.
  • Operator-audio transport is subscribe-only and starts the shared provider lazily.
  • Tests cover config override, speaker auto-disable, resampling/mixing, queue drop policy, batching, and response-code handling.

Note: I reviewed via GitHub API/diff because the local clone checkout hit filesystem timeout on this large repo; all GitHub checks are passing.

Comment thread dimos/teleop/hosted/go2_audio_bridge.py Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant