Skip to content

Add agent-started idb simulator Portals - #349

Open
jfrolich wants to merge 1 commit into
mainfrom
feat/idb-simulator-portals
Open

jfrolich wants to merge 1 commit into
mainfrom
feat/idb-simulator-portals

Conversation

@jfrolich

Copy link
Copy Markdown
Contributor

Summary

  • Add opensession-portals.start_simulator_portal for a prebuilt workspace-local simulator app.
  • Stream idb MJPEG frames and forward validated taps, swipes, text, Home and common keys through an authenticated Portal viewer.
  • Own a private device set and companion per session, with two-device capacity, bounded input/video buffers, idle shutdown and conservative crash recovery.
  • Let desktop users pin any Portal beside the conversation; retain the phone full-width viewer.
  • Fix Mac Portal process groups and listener discovery without Linux-only setsid/ss.

Verification

  • OPENSESSION_TEST_JOBS=4 bun run check passed, including strict transcript snapshots.
  • Module import side-effect check passed.
  • Real browser fixture tests at 1440×900 DPR 2 and 390×844 DPR 3 exercised screen display, tap/swipe/text/Home, reconnect, side-panel pin/expand, phone behavior, and missing-dependency errors.
  • Native command tests replace the executable boundary. No real iOS Simulator was started: this Mac has no full Xcode/idb setup.
  • A live Portal launch was attempted but the deployed Mac launcher still requires setsid; authenticated Portal routing also needs host configuration. No live URL or deployment is claimed.
  • No native iOS/Chrome protocol changes: the viewer has its own Portal-local WebSocket protocol.

Limits

Local macOS source installations only. Requires Xcode, an iOS runtime, idb/idb_companion and configured Portal HTTPS routing. No auto-build, signing, release or hot reload. Restart creates a fresh simulator. Real idb/Xcode qualification remains required before rollout.

Started by Local User in this Assistant session

Expose private iOS simulator viewers through the Portal MCP, with bounded
idb capture/input, session-owned device sets, cleanup and capacity recovery.
Let desktop users pin Portals beside the conversation, retain phone previews,
and support macOS process groups and listener discovery.

Co-authored-by: Jaap Frolich <[email protected]>
@vercel

vercel Bot commented Sep 10, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
opensession Ready Ready Preview Sep 10, 2026 7:46pm UTC

@jfrolich jfrolich self-assigned this Sep 10, 2026
@open-session-os-tella-dev

open-session-os-tella-dev Bot commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

🤖 OS review · approve · quality 5/5 · risk medium

Safe to merge. Current CI failures originate in pre-existing host-memory admission tests and secret-scan data outside this diff.

🟠 Risk medium · recovery in hours · DNS or infra, public API contract, wide blast radius, large diff
The 3,107-line diff adds an MCP/WebSocket contract, changes shared portal process/network supervision, and creates durable CoreSimulator leases requiring operator cleanup. Canary on one Mac and verify cleanup before broader rollout.

Reviewed 28fcc74 · GPT-5.6 Sol · open session · labels: os-auto-fix fix and push · os-adversarial deeper pass · os-simplify cleanup

@jfrolich

Copy link
Copy Markdown
Contributor Author

CI note: the history secret scan is failing on an existing Slack-session test identifier in packages/core/opensession-server/src/server/session-cache-catalog.test.ts:208, introduced by main commit 7bb0ed55dc3640b53454ef20e4ff6f19d58854a8. This PR does not change that file or add the flagged value. The complete local checkout check passed again before publishing. Failing scan log.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant