Conversation
QianCyrus
force-pushed
the
feat/macos-desktop-actions
branch
from
September 27, 2026 12:01
4d9994e to
49ad3ac
Compare
QianCyrus
force-pushed
the
feat/macos-desktop-actions
branch
2 times, most recently
from
September 27, 2026 16:31
fd089c2 to
9e1b350
Compare
QianCyrus
marked this pull request as ready for review
September 27, 2026 16:35
QianCyrus
force-pushed
the
feat/macos-desktop-actions
branch
from
September 27, 2026 16:51
9e1b350 to
520c0f9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this enables
The desktop agent can now enter supplied text into a macOS application, click Save, and check that the file contains the expected text.
It can insert text at the current selection or replace a field's entire contents, including Chinese and multiple lines. After input, it reads the field again to confirm the result. An optional file check requires a fresh write and matching content before the task succeeds.
The implementation extends the existing desktop CLI, window environment, and Cua Driver adapter. Part of #15; RFC #26.
macOS results
Rerun on this branch with macOS 26.2 arm64 and Cua Driver 0.30.1:
Each successful trial checks the saved file independently. The Laya run uses no fixed plan, Jev, or chat-model calls. Episode time excludes model loading.
Tests
Added
test_desktop_actions.pyfor text entry, replacement, readback, and saved-file verification.Extended
test_desktop_driver.py,test_agents_desktop.py, andtest_desktop_env.pyfor window binding, native field identifiers, CLI configuration, dry runs, and stale or refused actions.Core + dev suite: 515 passed, 41 skipped. Ruff, ty, lock check, package build, shell syntax, and CLI/MCP smoke passed.
How to run
Requires macOS,
uv, the Swift command-line tools, and Cua Driver with Accessibility and Screen Recording permissions. These commands use the signed driver at/Applications/CuaDriver.app/Contents/MacOS/cua-driver. Run from the current project checkout. Close any older S1A document fixture before starting.Run a fixed input-and-save sequence to check execution:
To let local Laya select the actions instead, run: