Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 44 additions & 6 deletions examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,22 +9,60 @@ Start with the simplest example that matches hardware you actually have, then sc
| Example | Hardware | What it shows |
|---|---|---|
| [`iphone_mac_webcam/`](./iphone_mac_webcam/) | Mac built-in webcam + iPhone (Continuity Camera) | Shortest end-to-end recipe: two OpenCV video streams through `UVCWebcamStream`, live preview in the desktop viewer, MP4 + timestamps written to disk |
| [`mac_iphone_dual_oak/`](./mac_iphone_dual_oak/) | Mac webcam + iPhone + OAK-D-Lite + OAK-D-S2 | Four video streams on one host: two `UVCWebcamStream`s and two `OakCameraStream`s, each OAK pinned to its DepthAI serial. Optional stereo depth flags on either OAK. |
| [`full_rig/`](./full_rig/) | Mac webcam + iPhone + OAK-D-Lite + OAK-D-S2 + OGLO glove (BLE) | Mixes four video streams with a 100 Hz tactile sensor stream: same four cameras as above plus an `OgloTactileStream` that renders a 5-finger FSR plot card in the viewer. Shows how video + sensor streams share one atomic session. |

More recipes will be added as the rigs they target come online. Expected next:

- **`oak_plus_webcam/`** — OAK-D Pro depth camera + Mac webcam (add depth to the dual-camera setup)
- **`iphone_imu/`** — iPhone + BLE IMU (`BLEImuGenericStream`) showing mixed video + sensor streams
- **`tactile_rig/`** — webcam + tactile sensor via `OgloTactileStream` showing custom-adapter integration
- **`multi_host_pair/`** — two Macs on the same WiFi recording together with `LeaderRole` / `FollowerRole`

## How to run any example
## Run any example (`uv run`)

Every example follows the same shape:
Every example is a plain Python script inside this repo, so the easiest way to run one is with `uv run` from the repo root — no virtualenv setup, no `pip install` step. `uv` resolves the extras you pass with `--extra` against the root `pyproject.toml` and executes the script in a temporary env:

```bash
cd examples/<name>
pip install "syncfield[uvc,audio,viewer]" # extras vary — see the example's README
python record.py # blocking, opens the viewer
# iphone_mac_webcam — Mac webcam + iPhone Continuity Camera
uv run --extra uvc --extra audio --extra viewer \
python examples/iphone_mac_webcam/record.py

# mac_iphone_dual_oak — Mac webcam + iPhone + OAK-D-Lite + OAK-D-S2
uv run --extra uvc --extra oak --extra audio --extra viewer \
python examples/mac_iphone_dual_oak/record.py

# full_rig — dual_oak + OGLO tactile glove over BLE
uv run --extra uvc --extra oak --extra ble --extra audio --extra viewer \
python examples/full_rig/record.py
```

The first run for each extra set downloads the wheels into the uv cache (~10–30 s); every subsequent run is instant.

> **Why `audio` is always there.** SyncField plays a 3/2/1 countdown tick and start/stop sync chirps through `sounddevice`. Without the `audio` extra installed the session runs in total silence and the console prints a WARNING telling you to add it. Every example in this directory includes `audio` in its recommended extras for that reason.

### Common flags

Every `record.py` accepts at least:

```bash
--output-dir ./my_recording # where to write session artifacts (default ./output)
```

Individual examples have their own extra flags — see the per-example README.

### Alternative: install once, then `python`

If you'd rather install the package into a persistent environment and run `python record.py` directly (closer to how end-users would ship it), use either of:

```bash
# Plain pip + venv
python -m venv .venv && source .venv/bin/activate
pip install "syncfield[uvc,oak,audio,viewer]"
python examples/mac_iphone_dual_oak/record.py

# uv sync
uv sync --extra uvc --extra oak --extra audio --extra viewer
uv run python examples/mac_iphone_dual_oak/record.py
```

Inside the viewer, click **Record** to start the session, **Stop** to finish, and close the window to exit. Output files land in `./output/` by default; every example accepts `--output-dir` if you want a different location.
Expand Down
95 changes: 95 additions & 0 deletions examples/full_rig/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
# Full Rig — Mac + iPhone + Dual OAK + OGLO Glove

**Five streams on one host.** Mac built-in webcam, iPhone over Continuity Camera, OAK-D-Lite, OAK-D-S2, and an OGLO tactile glove over BLE — all recorded as one atomic SyncField session.

This is the next step up from [`mac_iphone_dual_oak/`](../mac_iphone_dual_oak/): same four video streams, plus a 100 Hz tactile sensor card that renders a live 5-finger FSR plot in the viewer card row.

## Hardware checklist

- [x] Mac with a working webcam
- [x] iPhone with Continuity Camera enabled
- [x] OAK-D-Lite connected via USB
- [x] OAK-D-S2 connected via USB (different bus if possible)
- [x] OGLO tactile glove powered on and advertising over BLE within ~5 m of the Mac
- [x] Bluetooth enabled on the Mac

## Install

```bash
pip install "syncfield[uvc,oak,ble,audio,viewer]"
```

| Extra | What it's for |
|---|---|
| `uvc` | OpenCV — Mac webcam + iPhone Continuity Camera |
| `oak` | DepthAI v3 — both OAK cameras |
| `ble` | `bleak` — BLE scan + notify subscription for the OGLO glove |
| `audio` | `sounddevice` — 3/2/1 countdown ticks + start/stop sync chirps |
| `viewer` | DearPyGui + NumPy — the bundled desktop viewer |

## Run

From the repo root:

```bash
# Default serials + the currently paired OGLO address on the maintainer's rig
uv run --extra uvc --extra oak --extra ble --extra audio --extra viewer \
python examples/full_rig/record.py

# Override any identifier
uv run --extra uvc --extra oak --extra ble --extra audio --extra viewer \
python examples/full_rig/record.py \
--oak-lite 19443010813AF02C00 \
--oak-d 1944301071781C1300 \
--oglo-address C1718989-5A77-F3EB-B00A-01A758D99D54 \
--oglo-hand right \
--output-dir ./my_recording
```

Drop `--oglo-address` to fall back to a BLE name-substring scan for `"oglo"` — slower (scans for ~10 s at session start) but works on any Mac even if the CoreBluetooth address changes after re-pairing.

## Finding the OGLO BLE address on a new Mac

On macOS the BLE "address" returned by `bleak` is a per-host CoreBluetooth UUID — stable across reboots but different on every Mac. Discover yours with:

```bash
uv run --extra ble python -c "
import asyncio, bleak
SERVICE = '4652535f-424c-4500-0000-000000000001'
async def main():
results = await bleak.BleakScanner.discover(timeout=10, return_adv=True)
for addr, (d, ad) in results.items():
if SERVICE.lower() in [s.lower() for s in (ad.service_uuids or [])]:
print(f'OGLO → address={addr} local_name={ad.local_name!r}')
asyncio.run(main())
"
```

Copy the printed address into the `--oglo-address` flag (or into `DEFAULT_OGLO_ADDRESS` at the top of `record.py`).

## Output

```
output/
├── mac_webcam.mp4 mac_webcam.timestamps.jsonl
├── iphone.mp4 iphone.timestamps.jsonl
├── oak_lite.mp4 oak_lite.timestamps.jsonl
├── oak_d.mp4 oak_d.timestamps.jsonl
├── oglo.timestamps.jsonl (no .mp4 — OGLO is a sensor stream)
├── sync_point.json
├── manifest.json
└── session_log.jsonl
```

The OGLO stream is tagged `kind="sensor"` with `produces_file=False`, so you get one JSONL per tactile sample (thumb/index/middle/ring/pinky + device timestamp in nanoseconds) instead of a video file. In the viewer it renders as a multi-series line plot card alongside the four camera cards.

## Troubleshooting

| Symptom | Likely cause | Fix |
|---|---|---|
| `oglo` card stays blank | Glove is off, out of range, or claimed by another app | Power-cycle the glove; make sure the iOS egonaut app isn't connected to it at the same time |
| Session hangs for 10 s at connect | You dropped `--oglo-address` and it's running the fallback name scan | Normal — the scan runs once at connect and the session proceeds after it |
| `OGLO connection failed: no peripheral matched` | Name scan couldn't find the glove | Use the one-liner above to discover the address, then pass `--oglo-address` |
| `RuntimeError: No OAK devices found` | Old viewer still holds the USB handles | Close the previous viewer window, wait ~5 s, retry |

See [`mac_iphone_dual_oak/README.md`](../mac_iphone_dual_oak/README.md) for OAK-specific troubleshooting and [`iphone_mac_webcam/README.md`](../iphone_mac_webcam/README.md) for webcam troubleshooting.
50 changes: 50 additions & 0 deletions examples/full_rig/record.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
"""Record Mac webcam + iPhone + OAK-D-Lite + OAK-D-S2 + OGLO glove.

pip install "syncfield[uvc,oak,ble,audio,viewer]"
python record.py
"""

import argparse
from pathlib import Path

import syncfield as sf
import syncfield.viewer
from syncfield.adapters import (
OakCameraStream,
OgloTactileStream,
UVCWebcamStream,
)

DEFAULT_OAK_LITE_SERIAL = "19443010813AF02C00"
DEFAULT_OAK_D_SERIAL = "1944301071781C1300"
DEFAULT_OGLO_ADDRESS = "C1718989-5A77-F3EB-B00A-01A758D99D54"


def main() -> None:
parser = argparse.ArgumentParser()
parser.add_argument("--webcam-index", type=int, default=0)
parser.add_argument("--iphone-index", type=int, default=1)
parser.add_argument("--oak-lite", default=DEFAULT_OAK_LITE_SERIAL)
parser.add_argument("--oak-d", default=DEFAULT_OAK_D_SERIAL)
parser.add_argument("--oglo-address", default=DEFAULT_OGLO_ADDRESS)
parser.add_argument("--oglo-hand", default="right", choices=("left", "right"))
parser.add_argument("--output-dir", type=Path, default=Path("./output"))
args = parser.parse_args()

args.output_dir.mkdir(parents=True, exist_ok=True)

session = sf.SessionOrchestrator(
host_id="mac_studio",
output_dir=args.output_dir,
)
session.add(UVCWebcamStream("mac_webcam", args.webcam_index, args.output_dir))
session.add(UVCWebcamStream("iphone", args.iphone_index, args.output_dir))
session.add(OakCameraStream("oak_lite", args.output_dir, device_id=args.oak_lite))
session.add(OakCameraStream("oak_d", args.output_dir, device_id=args.oak_d))
session.add(OgloTactileStream("oglo", address=args.oglo_address, hand=args.oglo_hand))

syncfield.viewer.launch(session)


if __name__ == "__main__":
main()
33 changes: 25 additions & 8 deletions examples/iphone_mac_webcam/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,21 +33,38 @@ pip install "syncfield[uvc,audio,viewer]"
| Extra | What it's for |
|---|---|
| `uvc` | OpenCV — the `UVCWebcamStream` adapter that drives both cameras |
| `audio` | `sounddevice` — needed by the sync tone / chirp path, even though chirps are skipped in single-host mode |
| `audio` | `sounddevice` — plays the 3/2/1 countdown ticks and start/stop sync chirps through the MacBook speakers |
| `viewer` | `dearpygui` + `numpy` — the bundled desktop viewer |

## Run

The shortest way to run this example is `uv run` from the **repo root** — no virtualenv, no install step, `uv` resolves the extras against the root `pyproject.toml`:

```bash
# Default: webcam at index 0, iPhone at index 1
python record.py
# Default: Mac webcam at index 0, iPhone at index 1
uv run --extra uvc --extra audio --extra viewer \
python examples/iphone_mac_webcam/record.py

# Custom indices, output dir, geometry
python record.py \
--webcam-index 0 \
--iphone-index 1 \
--output-dir ./my_recording \
--width 1920 --height 1080 --fps 30
uv run --extra uvc --extra audio --extra viewer \
python examples/iphone_mac_webcam/record.py \
--webcam-index 0 \
--iphone-index 1 \
--output-dir ./my_recording \
--width 1920 --height 1080 --fps 30
```

> If you forget `--extra audio`, the recording still works but the countdown and chirps are silent and the console prints a WARNING telling you to add it.

### Alternative: plain `python` after install

If you'd rather `pip install` into a venv and run the script directly:

```bash
python -m venv .venv && source .venv/bin/activate
pip install "syncfield[uvc,audio,viewer]"
cd examples/iphone_mac_webcam
python record.py
```

### Not sure which index is which?
Expand Down
Loading
Loading