diff --git a/.github/workflows/vibecad-workflow-harness.yml b/.github/workflows/vibecad-workflow-harness.yml new file mode 100644 index 00000000..9c069ad0 --- /dev/null +++ b/.github/workflows/vibecad-workflow-harness.yml @@ -0,0 +1,37 @@ +name: VibeCAD Workflow Harness + +on: + pull_request: + push: + branches: + - main + workflow_dispatch: + +permissions: + contents: read + +concurrency: + group: vibecad-workflow-harness-${{ github.event.pull_request.number || github.ref }} + cancel-in-progress: true + +jobs: + harness: + name: Three-workflow click harness + runs-on: ubuntu-24.04 + timeout-minutes: 10 + steps: + - name: Check out source + uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + + - name: Set up Python + uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6 + with: + python-version: "3.11" + + - name: Run harness self-test without a TypeSafe key + working-directory: tools + run: python vibecad_workflow_harness_selftest.py + + - name: Run the three workflows against the fake click channel + working-directory: tools + run: python vibecad_workflow_harness.py --fake diff --git a/docs/vibecad-agent-control.md b/docs/vibecad-agent-control.md index 816c6c22..0fe5b6c5 100644 --- a/docs/vibecad-agent-control.md +++ b/docs/vibecad-agent-control.md @@ -321,7 +321,7 @@ owner's network controls. | POST | `/v1/close` | optional `document`, explicit `discard_unsaved` | Close without silently discarding a modified document | | GET | `/v1/ui/menus` | | Live top-level menu names, indices, visibility, and screen geometry | | GET | `/v1/ui/ribbon` | | Live ribbon names, workbenches, indices, selection, and screen geometry | -| POST | `/v1/ui/click` | `{"kind":"menu|ribbon","text":"..."}`, optional exact PID/index | Activate one semantic Qt target without moving or clicking the OS cursor | +| POST | `/v1/ui/click` | `{"kind":"menu|ribbon|action|dialog","text":"..."}`, optional exact PID/index | Activate one semantic Qt target without moving or clicking the OS cursor | | POST | `/v1/run` | `{"python":"..."}` or `{"script":"..."}` plus optional `path`, `recompute` | Exec against the active doc | | GET | `/v1/operations/{operation_id}` | | Read the in-memory state/result of a client-identified operation without entering the document thread | | GET/POST | `/v1/aero` | operation payload for POST | Bounded Aero context and operations | @@ -388,8 +388,95 @@ tracking explicitly by supplying an `operation_id` through the HTTP API. ### Semantic UI activation and the independent cursor -`/v1/ui/click` targets an exact live Qt menu action or -`VibeCADRibbonTabs` entry by visible text. Optional `expected_process_id` and +`/v1/ui/click` targets an exact live Qt menu action, +`VibeCADRibbonTabs` entry, or named command `QAction` by visible text +or object name. Action search uses the same `findChildren(QAction)` +set the main window already exposes, not only toolbar or menu-bar +`actions()`. Ribbon and menu kinds stay unchanged. The additive +`action` kind queues `QAction.trigger()` onto the next Qt event-loop +turn so a modal `QDialog.exec()` cannot hold the HTTP request. +`findChildren(QAction)` can see a hidden or disabled command copy +before the enabled Model-tab action. The click path now prefers the +enabled visible match, or the single enabled command action when the +standard toolbar copy is hidden. When every matching `QAction` is +disabled, `QAction.trigger()` does not fire; the click then follows +`Gui.isCommandActive` (`Command.canInvoke()`) and queues +`Gui.Command.get(name).run()`. `UI_TARGET_DISABLED` is returned only +when that command is also not invokable, and the payload then includes +`action_pick`, `action_match_count`, and `command_active`. The +additive `dialog` kind presses OK on the visible Choose Orientation +dialog (XY-plane already selected) in-process. That opens the sketch +editor: domain tabs including Model are disabled, and +`PartDesign_Pad` is not a `findChildren(QAction)` match because the +command is never `addTo()`'d on the Model or sketch.edit ribbon. The +Finish-group command the window does expose is `Sketcher_LeaveSketch`. +`CmdSketcherLeaveSketch::activated()` passes +`getActiveGuiDocument()` (`Application::activeDocument()`) into +`SketcherGui.leaveActiveSketch`. `requireExactEditState` then throws +`The exact Sketch document is no longer active in edit mode` when +`activeDocument()` is not `editDocument()`. A queued LeaveSketch +click plus a new `Std_New` per workflow creates that split (active +`Unnamed2`, editor still on an earlier document). The harness +therefore keeps one document through new, sketch, Extrude, and +export, and finishes the editor with +`SketcherGui.leaveActiveSketch` on `Gui.editDocument()`, not the +active tab. After leave, the rebuilt Model ribbon surfaces +`PartDesign_DesignExtrude`. An empty sketch makes +`ProfileBased` throw `Linked shape object is empty` and +`startConfiguredDesignProfileOperation` then raises +`The Design operation task panel did not open` and aborts the +transaction. The smallest closed-profile control on the +sketch.edit Geometry group is `Sketcher_CreateRectangle` (child of +`Sketcher_CompCreateRectangles`). Its `activated()` only starts +`DrawSketchHandlerRectangle`; it does not add geometry. Placing +the two corners needs viewport clicks, which `/v1/ui/click` kinds +do not do. The harness therefore does not click that button to +draw. After `Sketcher_LeaveSketch`, it posts the existing +`/v1/run` route (already used for tree inspect) with the same +`SketchObject.addGeometry(Part.Circle(...), False)` call +`TestConsolidatedPartTools._circle_sketch` and +`TestSketcherSolver.CreateCircleSketch` already use. Extrude is +enabled by `designProfileOperationActive()`: a reusable sketch or +one or more `InternalFace*` regions, not an edge +(`ReferenceSelection.cpp`, `TestDesignProfileRegionsGui`). The +harness therefore selects `InternalFace1` on the sketch object +and calls `Gui.Command.update()` so the `QAction` enabled flag +matches `isActive()`, then clicks the Extrude button. + +Before any workflow, the harness dismisses a leftover +`Document Recovery` dialog with its real `Cancel` button +(`DocumentRecovery.ui` / `DocumentRecovery.cpp`). It does not +press `Start Recovery` (that is the relabeled Ok button). A dirty +kill leaves that modal under the next Choose Orientation; kind +`dialog` text `OK` then matches every visible `QDialog` and never +lands. A pass +leaves `PartDesign::Body`, a `Sketcher::SketchObject` with +`GeometryCount >= 1`, `PartDesign::DesignExtrude`, and +`PartDesign::DesignBodyPublication` (`BodyResult`) in the +tree. Kind `dialog` text `OK` looks for a `QDialog` titled OK +and does not see the Extrude OK in the Tasks dock. The harness +accepts that task the same way +`TestDesignProfileRegionsGui._close_task` does: a visible +`QDialogButtonBox` OK through `/v1/run`, then waits until +`Gui.Control.activeDialog` is gone. After that OK, +`DesignExtrude.Shape` stays null; the solid with Faces is the +publication. Export does not click `Std_Export`: that command's +`activated()` opens `FileDialog::getSaveFileName`, which would +hold the HTTP request. `Import.export` is the exporter that +dialog would call (`AppImportPy.cpp`). `WriterStep::write` only +throws on OCCT `RetError` / `RetFail` / `RetStop`, so it can +return without creating the file. Live 6714cd65 then failed in +`os.path.getsize` (`WinError 2`) on +`%LOCALAPPDATA%\Temp\vibecad-workflow-harness.step`. The +harness writes the object that actually has Faces, preferring +`PartDesign::DesignBodyPublication`, tries `Import.export`, and +if that path is still missing or empty writes +`solid.Shape.exportStep` (`TopoShapePy` / `TopoShape.cpp`). It +does not report success unless that file exists and is +non-empty. +A successful trigger +counts as applied even when creating a document moves Qt focus; +restoration fields stay on the payload for evidence. Optional `expected_process_id` and `expected_index` values make stale geometry fail closed. Ribbon clicks use an in-process Qt mouse event; top-level menus use a non-blocking in-process Qt popup. A menu popup is displayed for one bounded preview, then closed before diff --git a/package/rattler-build/recipe.yaml b/package/rattler-build/recipe.yaml index 76bf6340..bbdf2532 100644 --- a/package/rattler-build/recipe.yaml +++ b/package/rattler-build/recipe.yaml @@ -10,7 +10,24 @@ source: use_gitignore: true build: - number: 5 + number: 6 + # Upstream Codex executables are self-contained, with system-only Mach-O + # dependencies. Adding conda RPATHs overflows their unpadded headers. + # Relocate our bin/lib installs, app plug-ins and native modules, not prebuilt + # executables under Mod. The final recursive bundle audit remains mandatory. + dynamic_linking: + binary_relocation: + - if: osx + then: + - bin/** + - lib/** + - Library/** + - PlugIns/** + - '**/*.so' + - '**/*.dylib' + - '**/*.bundle' + else: + - '**' script: env: # rattler-build runs the build in a sandbox that does not inherit shell diff --git a/package/rattler-build/scripts/install_vibecad_codex_runtime.sh b/package/rattler-build/scripts/install_vibecad_codex_runtime.sh index 6eac7741..bd782f59 100755 --- a/package/rattler-build/scripts/install_vibecad_codex_runtime.sh +++ b/package/rattler-build/scripts/install_vibecad_codex_runtime.sh @@ -15,7 +15,7 @@ download_cache="${VIBECAD_DOWNLOAD_CACHE:-${repository_root}/package/rattler-bui runtime_root="${module_directory}/codex_runtime" stamp="${runtime_root}/runtime-spec.sha256" -codex_version="0.153.4" +codex_version="0.154.0" release_tag="rust-v${codex_version}" release_root="https://github.com/openai/codex/releases/download/${release_tag}" license_url="https://raw.githubusercontent.com/openai/codex/${release_tag}/LICENSE" @@ -31,32 +31,32 @@ machine="$(${python_executable} -c 'import platform; print(platform.machine().lo case "${platform}:${machine}" in linux:x86_64|linux:amd64) archive="codex-app-server-package-x86_64-unknown-linux-musl.tar.gz" - archive_sha256="a5d37ff1fa6953ee6d317b7e69bfafd39f5f53350b631d790fa7531159f22420" + archive_sha256="b2450aaa4004d06790dd8a69d0246f4503ff1258400cc20de7d7a42ffe81b253" executable="${runtime_root}/bin/codex-app-server" ;; linux:aarch64|linux:arm64) archive="codex-app-server-package-aarch64-unknown-linux-musl.tar.gz" - archive_sha256="5673c5a8935ff2f85ca67b489e560fdd5e08fb0f0e2f7426f048ec7449aa4fdc" + archive_sha256="295bb1b94a8b964b2d2461db9736b9907a9e4daa6ceb8e9bbb820b304fa897ed" executable="${runtime_root}/bin/codex-app-server" ;; win32:amd64|win32:x86_64) archive="codex-app-server-package-x86_64-pc-windows-msvc.tar.gz" - archive_sha256="69441ca4c8f6197923dc1b70a8aa870ff912b5367347287d021eaca1f3add971" + archive_sha256="5f8b43e030c0aeeb7bdb3d5e03fff4c68ba94fa2df0ae437c490811f54660d74" executable="${runtime_root}/bin/codex-app-server.exe" ;; win32:arm64|win32:aarch64) archive="codex-app-server-package-aarch64-pc-windows-msvc.tar.gz" - archive_sha256="d5f0ef33223912a1559a7e97012afa18eef3369f1d07dde199edfada062503ee" + archive_sha256="7406aa3745acd5bd639b8921c0ee5e241861763606465cac643683944a849457" executable="${runtime_root}/bin/codex-app-server.exe" ;; darwin:arm64|darwin:aarch64) archive="codex-app-server-package-aarch64-apple-darwin.tar.gz" - archive_sha256="90f0467fd03294896204e8856bf969a0691590e8bef78dc2563a264b186f3265" + archive_sha256="7bf20c1843bdcff086c89a294299833f20146ebbdba03e7f49f020b7adbfff7b" executable="${runtime_root}/bin/codex-app-server" ;; darwin:x86_64|darwin:amd64) archive="codex-app-server-package-x86_64-apple-darwin.tar.gz" - archive_sha256="ee286ca326a0df4a2b81dddb213d61e610d7b9c4f3173cc16f6023683a94ca82" + archive_sha256="4fddde3689d2aa0058c06138a84b05f87bdbff8cd556fba5816a97ac0469a4d4" executable="${runtime_root}/bin/codex-app-server" ;; *) diff --git a/src/Mod/VibeCAD/CMakeLists.txt b/src/Mod/VibeCAD/CMakeLists.txt index a3149910..914ba4c5 100644 --- a/src/Mod/VibeCAD/CMakeLists.txt +++ b/src/Mod/VibeCAD/CMakeLists.txt @@ -1626,6 +1626,7 @@ if(BUILD_TEST AND BUILD_GUI) vibecad_tests/test_agent_control_grok_bot.py vibecad_tests/test_dev_launcher_contract.py vibecad_tests/test_visible_operator_contract.py + vibecad_tests/test_workflow_harness_contract.py vibecad_tests/test_reference_image_downscale.py vibecad_tests/test_ollama_inspect.py vibecad_tests/test_live_acceptance_oracle.py diff --git a/src/Mod/VibeCAD/VibeCADAgentCli.py b/src/Mod/VibeCAD/VibeCADAgentCli.py index 926e97cd..6acc27ee 100644 --- a/src/Mod/VibeCAD/VibeCADAgentCli.py +++ b/src/Mod/VibeCAD/VibeCADAgentCli.py @@ -114,12 +114,12 @@ def build_parser() -> argparse.ArgumentParser: ui_click_parser = sub.add_parser( "ui-click", - help="Activate one semantic ribbon or menu target without moving the OS cursor.", + help="Activate one semantic ribbon, menu, action, or dialog target without moving the OS cursor.", ) ui_click_parser.add_argument( "--kind", required=True, - choices=("ribbon", "menu"), + choices=("ribbon", "menu", "action", "dialog"), help="Target family to activate.", ) ui_click_parser.add_argument("--text", required=True, help="Exact visible target text.") diff --git a/src/Mod/VibeCAD/VibeCADAgentControl.py b/src/Mod/VibeCAD/VibeCADAgentControl.py index e3ad58cd..1268bac0 100644 --- a/src/Mod/VibeCAD/VibeCADAgentControl.py +++ b/src/Mod/VibeCAD/VibeCADAgentControl.py @@ -1475,7 +1475,7 @@ def brief_path() -> Path: | POST | `/v1/close` | optional `{{"document":"Name","discard_unsaved":false}}` | Close without silently discarding changes | | GET | `/v1/ui/ribbon` | | Live semantic tab names and screen geometry | | GET | `/v1/ui/menus` | | Live top-level menu names and screen geometry | -| POST | `/v1/ui/click` | `{{"kind":"ribbon","text":"Model"}}` | Activate a semantic Qt target without moving the physical cursor | +| POST | `/v1/ui/click` | `{{"kind":"ribbon|menu|action|dialog","text":"Model"}}` | Activate a semantic Qt target without moving the physical cursor | | GET/POST | `/v1/screenshot` | optional `{{"path":"...png","overwrite":false}}` | Capture the visible VibeCAD window | | POST | `/v1/run` | `{{"python":"..."}}` or `{{"script":"..."}}` (+ optional `path`, `recompute`) | Run against the active document | | GET | `/v1/operations/` | | Prove completion after a client timeout | @@ -2434,6 +2434,322 @@ def _cursor_coordinates(QtGui: Any) -> dict[str, int]: return {"x": int(point.x()), "y": int(point.y())} +def _clean_qt_label(value: Any) -> str: + return str(value or "").replace("&", "").strip() + + +def _iter_qt_actions(owner: Any) -> list[Any]: + reader = getattr(owner, "actions", None) + if not callable(reader): + return [] + try: + return list(reader() or []) + except Exception: + return [] + + +def _collect_named_qt_actions( + main_window: Any, + QtWidgets: Any, + target_text: str, + QtGui: Any | None = None, +) -> list[tuple[int, Any]]: + """Return QActions whose visible text or objectName equals target_text. + + The live window registers commands as ``QAction`` children of the main + window. Searching only ``actions()`` on the window, menu bar, and + toolbars misses those, so this also uses ``findChildren(QAction)``. + """ + + seen: set[int] = set() + matches: list[tuple[int, Any]] = [] + + def consider(action: Any) -> None: + identity = id(action) + if identity in seen: + return + seen.add(identity) + text_reader = getattr(action, "text", None) + name_reader = getattr(action, "objectName", None) + text = _clean_qt_label(text_reader() if callable(text_reader) else "") + object_name = str(name_reader() if callable(name_reader) else "").strip() + if text == target_text or object_name == target_text: + matches.append((len(matches), action)) + menu_reader = getattr(action, "menu", None) + menu = menu_reader() if callable(menu_reader) else None + if menu is not None: + for child in _iter_qt_actions(menu): + consider(child) + + owners: list[Any] = [main_window] + menu_reader = getattr(main_window, "menuBar", None) + menu_bar = menu_reader() if callable(menu_reader) else None + if menu_bar is not None: + owners.append(menu_bar) + finder = getattr(main_window, "findChildren", None) + toolbar_type = getattr(QtWidgets, "QToolBar", None) + if callable(finder) and toolbar_type is not None: + try: + owners.extend(list(finder(toolbar_type) or [])) + except Exception: + pass + action_type = None + if QtGui is not None: + action_type = getattr(QtGui, "QAction", None) + if action_type is None: + action_type = getattr(QtWidgets, "QAction", None) + if callable(finder) and action_type is not None: + try: + for child in list(finder(action_type) or []): + consider(child) + except Exception: + pass + for owner in owners: + for action in _iter_qt_actions(owner): + consider(action) + return matches + + +def _qt_action_is_enabled(action: Any) -> bool: + reader = getattr(action, "isEnabled", None) + return not callable(reader) or bool(reader()) + + +def _qt_action_is_visible(action: Any) -> bool: + reader = getattr(action, "isVisible", None) + return not callable(reader) or bool(reader()) + + +def _pick_clickable_qt_action( + matches: list[tuple[int, Any]], +) -> tuple[tuple[int, Any] | None, str]: + """Prefer the enabled action findChildren can see. + + ``Gui.isCommandActive`` is ``Command.canInvoke()``. The click path used + to require exactly one name match and then abort if that copy was hidden. + The ribbon keeps the command ``QAction`` on a hidden standard toolbar and + may also keep a disabled placeholder, so that unique/first copy is not + the enabled Model-tab action. + """ + + if not matches: + return None, "none" + enabled_visible = [ + item + for item in matches + if _qt_action_is_enabled(item[1]) and _qt_action_is_visible(item[1]) + ] + if len(enabled_visible) == 1: + return enabled_visible[0], "enabled_visible" + if len(enabled_visible) > 1: + return None, "not_unique" + enabled = [item for item in matches if _qt_action_is_enabled(item[1])] + if len(enabled) == 1: + return enabled[0], "enabled" + if len(matches) == 1: + return matches[0], "disabled" + return None, "not_unique" + + +def _qt_action_object_name(action: Any) -> str: + reader = getattr(action, "objectName", None) + return str(reader() if callable(reader) else "") + + +def _named_command_for_action_click( + target_text: str, + matches: list[tuple[int, Any]], + action: Any | None, +) -> str: + if action is not None: + name = _qt_action_object_name(action) + if name: + return name + for _index, candidate in matches: + name = _qt_action_object_name(candidate) + if name: + return name + return target_text + + +def _command_is_active(gui: Any, name: str) -> bool: + """``Gui.isCommandActive`` is ``Command.canInvoke()``.""" + + checker = getattr(gui, "isCommandActive", None) + if callable(checker): + try: + return bool(checker(name)) + except Exception: + return False + command_mod = getattr(gui, "Command", None) + getter = getattr(command_mod, "get", None) + if not callable(getter): + return False + try: + command = getter(name) + except Exception: + return False + if command is None: + return False + is_active = getattr(command, "isActive", None) + return callable(is_active) and bool(is_active()) + + +def _named_command_runner(gui: Any, name: str) -> Any | None: + command_mod = getattr(gui, "Command", None) + getter = getattr(command_mod, "get", None) + if not callable(getter): + return None + try: + command = getter(name) + except Exception: + return None + runner = getattr(command, "run", None) + return runner if callable(runner) else None + + +def _all_named_qt_actions_disabled(matches: list[tuple[int, Any]]) -> bool: + return bool(matches) and all( + not _qt_action_is_enabled(item[1]) for item in matches + ) + + +def _queue_qt_callback(QtCore: Any, callback: Any) -> bool: + """Queue ``callback`` on the next Qt event-loop turn. + + Returns True when the callback was queued and has not run yet. A + modal ``QDialog.exec()`` inside a synchronous callback would hold + the HTTP dispatch until the dialog closed. + """ + + timer = getattr(QtCore, "QTimer", None) + single_shot = getattr(timer, "singleShot", None) + if not callable(single_shot) or not callable(callback): + return False + single_shot(0, callback) + return True + + +def _click_visible_dialog( + target_text: str, + *, + required_index: int | None, + main_window: Any, + QtWidgets: Any, + application: Any, + process_events: Any, +) -> dict[str, Any]: + """Accept a visible modal dialog without moving the OS cursor.""" + + finder = getattr(main_window, "findChildren", None) + dialog_type = getattr(QtWidgets, "QDialog", None) + candidates: list[Any] = [] + if callable(finder) and dialog_type is not None: + try: + candidates.extend(list(finder(dialog_type) or [])) + except Exception: + pass + modal_reader = getattr(application, "activeModalWidget", None) + if callable(modal_reader): + modal = modal_reader() + if modal is not None: + candidates.append(modal) + + visible: list[Any] = [] + seen: set[int] = set() + for dialog in candidates: + identity = id(dialog) + if identity in seen: + continue + seen.add(identity) + visible_reader = getattr(dialog, "isVisible", None) + if callable(visible_reader) and bool(visible_reader()): + visible.append(dialog) + + matches: list[Any] = [] + for dialog in visible: + title_reader = getattr(dialog, "windowTitle", None) + title = _clean_qt_label(title_reader() if callable(title_reader) else "") + if title == target_text or target_text in {"OK", "Ok"}: + matches.append(dialog) + if target_text in {"OK", "Ok"} and not matches and len(visible) == 1: + matches = list(visible) + if len(matches) != 1: + return failure( + "UI_TARGET_NOT_UNIQUE", + ( + f"Expected exactly one visible dialog named {target_text!r}; " + f"found {len(matches)}." + ), + stage="precondition", + ) + dialog = matches[0] + if required_index is not None and required_index != 0: + return failure( + "UI_TARGET_INDEX_MISMATCH", + f"Dialog {target_text!r} is index 0, not {required_index}.", + stage="precondition", + ) + + child_finder = getattr(dialog, "findChild", None) + radio_type = getattr(QtWidgets, "QRadioButton", None) + if callable(child_finder) and radio_type is not None: + xy = child_finder(radio_type, "XY_radioButton") + if xy is not None: + checked = getattr(xy, "isChecked", None) + if callable(checked) and not bool(checked()): + clicker = getattr(xy, "click", None) + if callable(clicker): + clicker() + + accepted = False + box_type = getattr(QtWidgets, "QDialogButtonBox", None) + ok_button = None + if callable(child_finder) and box_type is not None: + box = child_finder(box_type) + if box is not None: + ok_flag = getattr(box_type, "Ok", None) + if ok_flag is None: + ok_flag = getattr(getattr(box_type, "StandardButton", None), "Ok", None) + button_reader = getattr(box, "button", None) + if callable(button_reader) and ok_flag is not None: + ok_button = button_reader(ok_flag) + if ok_button is not None: + clicker = getattr(ok_button, "click", None) + if callable(clicker): + clicker() + accepted = True + if not accepted: + accept = getattr(dialog, "accept", None) + if callable(accept): + accept() + accepted = True + if callable(process_events): + process_events() + if not accepted: + return failure( + "UI_CLICK_NOT_APPLIED", + f"Qt click did not activate dialog target {target_text!r}.", + stage="postcondition", + ) + title_reader = getattr(dialog, "windowTitle", None) + return { + "target_kind": "dialog", + "target_text": target_text, + "target_index": 0, + "object_name": _clean_qt_label( + title_reader() if callable(title_reader) else "" + ), + "active_action_restored": True, + "click_queued": False, + "focus_restored": True, + "active_window_unchanged": True, + "popup_restored": True, + "interaction_restored": True, + "verified": True, + } + + def ui_click_target( kind: str, text: str, @@ -2446,10 +2762,12 @@ def ui_click_target( target_kind = str(kind or "").strip().lower().replace("-", "_") if target_kind in {"tab", "ribbon_tab"}: target_kind = "ribbon" - if target_kind not in {"ribbon", "menu"}: + if target_kind in {"command", "button"}: + target_kind = "action" + if target_kind not in {"ribbon", "menu", "action", "dialog"}: return failure( "UI_TARGET_KIND_INVALID", - "kind must be 'ribbon' or 'menu'.", + "kind must be 'ribbon', 'menu', 'action', or 'dialog'.", stage="schema", ) target_text = str(text or "").strip() @@ -2669,7 +2987,7 @@ def interaction_state() -> dict[str, bool]: "click_queued": False, **state, } - else: + elif target_kind == "menu": widget = main_window.menuBar() if widget is None or not bool(widget.isVisible()): return failure( @@ -2778,15 +3096,147 @@ def interaction_state() -> dict[str, bool]: "click_queued": False, **state, } + elif target_kind == "action": + updater = getattr(getattr(gui, "Command", None), "update", None) + if callable(updater): + try: + updater() + except Exception: + pass + matches = _collect_named_qt_actions( + main_window, QtWidgets, target_text, QtGui + ) + picked, pick_reason = _pick_clickable_qt_action(matches) + action = None + target_index = 0 + if picked is not None: + target_index, action = picked + command_name = _named_command_for_action_click( + target_text, matches, action + ) + command_active = _command_is_active(gui, command_name) + invoke = None + invoke_pick = pick_reason + object_name = ( + _qt_action_object_name(action) if action is not None else command_name + ) + disabled_failure = { + "action_pick": pick_reason, + "action_match_count": len(matches), + "command_active": command_active, + } + if picked is not None: + if required_index is not None and required_index != target_index: + return failure( + "UI_TARGET_INDEX_MISMATCH", + ( + f"Action {target_text!r} is index {target_index}, " + f"not {required_index}." + ), + stage="precondition", + **disabled_failure, + ) + if pick_reason != "disabled" and _qt_action_is_enabled(action): + invoke = getattr(action, "trigger", None) + elif command_active: + invoke = _named_command_runner(gui, command_name) + invoke_pick = "command_active" + else: + return failure( + "UI_TARGET_DISABLED", + f"Action {target_text!r} is disabled or hidden.", + stage="precondition", + **disabled_failure, + ) + elif command_active and _all_named_qt_actions_disabled(matches): + invoke = _named_command_runner(gui, command_name) + invoke_pick = "command_active" + else: + return failure( + "UI_TARGET_NOT_UNIQUE", + ( + f"Expected exactly one action named {target_text!r}; " + f"found {len(matches)}." + ), + stage="precondition", + **disabled_failure, + ) + if not callable(invoke): + return failure( + "UI_TARGET_NOT_TRIGGERABLE", + f"Action {target_text!r} cannot be triggered in-process.", + stage="precondition", + action_pick=invoke_pick, + action_match_count=len(matches), + command_active=command_active, + ) + # Sketcher_NewSketch calls QDialog.exec() on this GUI thread. + # The HTTP worker waits for this dispatch, so a modal would + # hold /v1/ui/click and /v1/status until OK. Queue the + # trigger onto the next event-loop turn and return now. + # Disabled QAction.trigger() does not fire, so a canInvoke() + # match uses Gui.Command.get(name).run() instead. + queued = _queue_qt_callback(QtCore, invoke) + if not queued: + invoke() + process_events() + state = ( + { + "focus_restored": True, + "active_window_unchanged": True, + "popup_restored": True, + "interaction_restored": True, + } + if queued + else interaction_state() + ) + # Creating a document moves Qt focus. Restoration is + # reported but does not define whether the click landed. + verified = True + details = { + "target_kind": target_kind, + "target_text": target_text, + "target_index": target_index, + "object_name": object_name, + "action_match_count": len(matches), + "action_pick": invoke_pick, + "command_active": command_active, + "active_action_restored": True, + "click_queued": queued, + **state, + } + else: + details = _click_visible_dialog( + target_text, + required_index=required_index, + main_window=main_window, + QtWidgets=QtWidgets, + application=application, + process_events=process_events, + ) + if details.get("ok") is False: + return details + verified = bool(details.pop("verified", True)) + state = { + "focus_restored": bool(details.get("focus_restored", True)), + "active_window_unchanged": bool( + details.get("active_window_unchanged", True) + ), + "popup_restored": bool(details.get("popup_restored", True)), + "interaction_restored": bool( + details.get("interaction_restored", True) + ), + } cursor_after = _cursor_coordinates(QtGui) details.update( { - "input_method": ( - "qt_in_process_mouse_click" - if target_kind == "ribbon" - else "qt_in_process_menu_popup" - ), + "input_method": { + "ribbon": "qt_in_process_mouse_click", + "menu": "qt_in_process_menu_popup", + "action": "qt_in_process_action_trigger", + "dialog": "qt_in_process_dialog_button", + }[target_kind], "physical_cursor_control": "none", "physical_cursor_before": cursor_before, "physical_cursor_after": cursor_after, diff --git a/src/Mod/VibeCAD/VibeCADCodex.py b/src/Mod/VibeCAD/VibeCADCodex.py index 1e40364e..f13a0221 100644 --- a/src/Mod/VibeCAD/VibeCADCodex.py +++ b/src/Mod/VibeCAD/VibeCADCodex.py @@ -24,7 +24,7 @@ from typing import Any, Callable, Mapping -CODEX_APP_SERVER_VERSION = "0.153.4" +CODEX_APP_SERVER_VERSION = "0.154.0" CODEX_APP_SERVER_ENV = "VIBECAD_CODEX_APP_SERVER" CODEX_HOME_ENV = "VIBECAD_CODEX_HOME" CODEX_RUNTIME_DIRECTORY = "codex_runtime" diff --git a/src/Mod/VibeCAD/vibecad_tests/test_agent_control.py b/src/Mod/VibeCAD/vibecad_tests/test_agent_control.py index fa65443c..62b1b1d4 100644 --- a/src/Mod/VibeCAD/vibecad_tests/test_agent_control.py +++ b/src/Mod/VibeCAD/vibecad_tests/test_agent_control.py @@ -1924,6 +1924,1010 @@ def qWait(milliseconds: int) -> None: # noqa: N802 assert application.popup is None +def test_ui_action_click_triggers_named_qt_action(monkeypatch) -> None: + class Point: + def __init__(self, x: int, y: int) -> None: + self._x = x + self._y = y + + def x(self) -> int: + return self._x + + def y(self) -> int: + return self._y + + class Action: + def __init__(self) -> None: + self.triggered = 0 + + def text(self) -> str: + return "&New" + + def objectName(self) -> str: # noqa: N802 + return "Std_New" + + def isEnabled(self) -> bool: # noqa: N802 + return True + + def isVisible(self) -> bool: # noqa: N802 + return True + + def menu(self): + return None + + def trigger(self) -> None: + self.triggered += 1 + application.focus = self + + action = Action() + + class ToolBar: + def actions(self) -> list[Action]: + return [action] + + toolbar = ToolBar() + + class MenuBar: + def actions(self) -> list: + return [] + + window = SimpleNamespace( + menuBar=lambda: MenuBar(), + actions=lambda: [], + findChildren=lambda _kind: [toolbar], + ) + + class FocusWidget: + def setFocus(self, _reason=None) -> None: # noqa: N802 + application.focus = self + + focus_widget = FocusWidget() + application = SimpleNamespace( + focus=focus_widget, + active_window=window, + popup=None, + ) + qt_core = SimpleNamespace( + Qt=SimpleNamespace( + LeftButton="left", + NoModifier="none", + OtherFocusReason="other", + ) + ) + qt_gui = SimpleNamespace(QCursor=SimpleNamespace(pos=lambda: Point(10, 20))) + qt_widgets = SimpleNamespace( + QTabBar=object, + QToolBar=object, + QApplication=SimpleNamespace( + processEvents=lambda: None, + focusWidget=lambda: application.focus, + activeWindow=lambda: application.active_window, + activePopupWidget=lambda: application.popup, + ), + ) + + class QTest: + @staticmethod + def mouseClick(_widget, _button, _modifiers, _point) -> None: # noqa: N802 + raise AssertionError("action clicks must not use QTest.mouseClick") + + monkeypatch.setitem( + sys.modules, + "PySide", + SimpleNamespace(QtCore=qt_core, QtGui=qt_gui, QtWidgets=qt_widgets), + ) + monkeypatch.setitem( + sys.modules, + "PySide6", + SimpleNamespace(QtTest=SimpleNamespace(QTest=QTest)), + ) + monkeypatch.setattr( + control, + "_gui", + lambda: SimpleNamespace(GuiUp=True, getMainWindow=lambda: window), + ) + + payload = control.dispatch("ui_click", {"kind": "action", "text": "Std_New"}) + assert payload["ok"] is True + assert payload["input_method"] == "qt_in_process_action_trigger" + assert payload["physical_cursor_control"] == "none" + assert payload["object_name"] == "Std_New" + assert payload["semantic_verified"] is True + assert payload["interaction_restored"] is True + assert action.triggered == 1 + assert application.focus is focus_widget + + command_alias = control.dispatch("ui_click", {"kind": "command", "text": "New"}) + assert command_alias["ok"] is True + assert action.triggered == 2 + + +def test_ui_action_click_counts_as_applied_when_focus_moves(monkeypatch) -> None: + class Point: + def __init__(self, x: int, y: int) -> None: + self._x = x + self._y = y + + def x(self) -> int: + return self._x + + def y(self) -> int: + return self._y + + class Action: + def __init__(self) -> None: + self.triggered = 0 + + def text(self) -> str: + return "&New" + + def objectName(self) -> str: # noqa: N802 + return "Std_New" + + def isEnabled(self) -> bool: # noqa: N802 + return True + + def isVisible(self) -> bool: # noqa: N802 + return True + + def menu(self): + return None + + def trigger(self) -> None: + self.triggered += 1 + application.focus = self + + action = Action() + + class ToolBar: + def actions(self) -> list[Action]: + return [action] + + toolbar = ToolBar() + + class MenuBar: + def actions(self) -> list: + return [] + + window = SimpleNamespace( + menuBar=lambda: MenuBar(), + actions=lambda: [], + findChildren=lambda _kind: [toolbar], + ) + + class FocusWidget: + def setFocus(self, _reason=None) -> None: # noqa: N802 + return None + + def clearFocus(self) -> None: # noqa: N802 + return None + + focus_widget = FocusWidget() + application = SimpleNamespace( + focus=focus_widget, + active_window=window, + popup=None, + ) + qt_core = SimpleNamespace( + Qt=SimpleNamespace( + LeftButton="left", + NoModifier="none", + OtherFocusReason="other", + ) + ) + qt_gui = SimpleNamespace(QCursor=SimpleNamespace(pos=lambda: Point(10, 20))) + qt_widgets = SimpleNamespace( + QTabBar=object, + QToolBar=object, + QApplication=SimpleNamespace( + processEvents=lambda: None, + focusWidget=lambda: application.focus, + activeWindow=lambda: application.active_window, + activePopupWidget=lambda: application.popup, + ), + ) + + class QTest: + @staticmethod + def mouseClick(_widget, _button, _modifiers, _point) -> None: # noqa: N802 + raise AssertionError("action clicks must not use QTest.mouseClick") + + monkeypatch.setitem( + sys.modules, + "PySide", + SimpleNamespace(QtCore=qt_core, QtGui=qt_gui, QtWidgets=qt_widgets), + ) + monkeypatch.setitem( + sys.modules, + "PySide6", + SimpleNamespace(QtTest=SimpleNamespace(QTest=QTest)), + ) + monkeypatch.setattr( + control, + "_gui", + lambda: SimpleNamespace(GuiUp=True, getMainWindow=lambda: window), + ) + + payload = control.dispatch("ui_click", {"kind": "action", "text": "Std_New"}) + assert payload["ok"] is True + assert "failure_code" not in payload + assert payload["input_method"] == "qt_in_process_action_trigger" + assert payload["object_name"] == "Std_New" + assert payload["semantic_verified"] is True + assert payload["focus_restored"] is False + assert payload["interaction_restored"] is False + assert action.triggered == 1 + assert application.focus is action + + +def test_ui_action_click_finds_qaction_children_not_on_toolbars(monkeypatch) -> None: + class Point: + def __init__(self, x: int, y: int) -> None: + self._x = x + self._y = y + + def x(self) -> int: + return self._x + + def y(self) -> int: + return self._y + + class QAction: + pass + + class Action: + def __init__(self) -> None: + self.triggered = 0 + + def text(self) -> str: + return "New Body" + + def objectName(self) -> str: # noqa: N802 + return "PartDesign_NewBody" + + def isEnabled(self) -> bool: # noqa: N802 + return True + + def isVisible(self) -> bool: # noqa: N802 + return True + + def menu(self): + return None + + def trigger(self) -> None: + self.triggered += 1 + + action = Action() + + def find_children(kind): + if kind is QAction: + return [action] + return [] + + class MenuBar: + def actions(self) -> list: + return [] + + window = SimpleNamespace( + menuBar=lambda: MenuBar(), + actions=lambda: [], + findChildren=find_children, + ) + application = SimpleNamespace( + focus=None, + active_window=window, + popup=None, + ) + qt_core = SimpleNamespace( + Qt=SimpleNamespace( + LeftButton="left", + NoModifier="none", + OtherFocusReason="other", + ) + ) + qt_gui = SimpleNamespace( + QCursor=SimpleNamespace(pos=lambda: Point(10, 20)), + QAction=QAction, + ) + qt_widgets = SimpleNamespace( + QTabBar=object, + QToolBar=object, + QApplication=SimpleNamespace( + processEvents=lambda: None, + focusWidget=lambda: application.focus, + activeWindow=lambda: application.active_window, + activePopupWidget=lambda: application.popup, + ), + ) + + class QTest: + @staticmethod + def mouseClick(_widget, _button, _modifiers, _point) -> None: # noqa: N802 + raise AssertionError("action clicks must not use QTest.mouseClick") + + monkeypatch.setitem( + sys.modules, + "PySide", + SimpleNamespace(QtCore=qt_core, QtGui=qt_gui, QtWidgets=qt_widgets), + ) + monkeypatch.setitem( + sys.modules, + "PySide6", + SimpleNamespace(QtTest=SimpleNamespace(QTest=QTest)), + ) + monkeypatch.setattr( + control, + "_gui", + lambda: SimpleNamespace(GuiUp=True, getMainWindow=lambda: window), + ) + + missing = control._collect_named_qt_actions(window, qt_widgets, "PartDesign_NewBody") + assert missing == [] + + payload = control.dispatch( + "ui_click", {"kind": "action", "text": "PartDesign_NewBody"} + ) + assert payload["ok"] is True + assert payload["object_name"] == "PartDesign_NewBody" + assert payload["input_method"] == "qt_in_process_action_trigger" + assert action.triggered == 1 + + +def test_pick_clickable_qt_action_skips_the_hidden_disabled_duplicate() -> None: + hidden = SimpleNamespace(isEnabled=lambda: False, isVisible=lambda: False) + visible = SimpleNamespace(isEnabled=lambda: True, isVisible=lambda: True) + picked, reason = control._pick_clickable_qt_action([(0, hidden), (1, visible)]) + assert reason == "enabled_visible" + assert picked == (1, visible) + + +def test_pick_clickable_qt_action_uses_the_enabled_hidden_command_action() -> None: + hidden = SimpleNamespace(isEnabled=lambda: True, isVisible=lambda: False) + picked, reason = control._pick_clickable_qt_action([(0, hidden)]) + assert reason == "enabled" + assert picked == (0, hidden) + + +def test_ui_action_click_triggers_enabled_copy_when_a_hidden_duplicate_exists( + monkeypatch, +) -> None: + class Point: + def __init__(self, x: int, y: int) -> None: + self._x = x + self._y = y + + def x(self) -> int: + return self._x + + def y(self) -> int: + return self._y + + class QAction: + pass + + class Action: + def __init__(self, enabled: bool, visible: bool) -> None: + self.triggered = 0 + self._enabled = enabled + self._visible = visible + + def text(self) -> str: + return "Extrude" + + def objectName(self) -> str: # noqa: N802 + return "PartDesign_DesignExtrude" + + def isEnabled(self) -> bool: # noqa: N802 + return self._enabled + + def isVisible(self) -> bool: # noqa: N802 + return self._visible + + def menu(self): + return None + + def trigger(self) -> None: + self.triggered += 1 + + hidden = Action(enabled=False, visible=False) + visible = Action(enabled=True, visible=True) + + def find_children(kind): + if kind is QAction: + return [hidden, visible] + return [] + + class MenuBar: + def actions(self) -> list: + return [] + + window = SimpleNamespace( + menuBar=lambda: MenuBar(), + actions=lambda: [], + findChildren=find_children, + ) + application = SimpleNamespace( + focus=None, + active_window=window, + popup=None, + ) + qt_core = SimpleNamespace( + Qt=SimpleNamespace( + LeftButton="left", + NoModifier="none", + OtherFocusReason="other", + ) + ) + qt_gui = SimpleNamespace( + QCursor=SimpleNamespace(pos=lambda: Point(10, 20)), + QAction=QAction, + ) + qt_widgets = SimpleNamespace( + QTabBar=object, + QToolBar=object, + QApplication=SimpleNamespace( + processEvents=lambda: None, + focusWidget=lambda: application.focus, + activeWindow=lambda: application.active_window, + activePopupWidget=lambda: application.popup, + ), + ) + + class QTest: + @staticmethod + def mouseClick(_widget, _button, _modifiers, _point) -> None: # noqa: N802 + raise AssertionError("action clicks must not use QTest.mouseClick") + + monkeypatch.setitem( + sys.modules, + "PySide", + SimpleNamespace(QtCore=qt_core, QtGui=qt_gui, QtWidgets=qt_widgets), + ) + monkeypatch.setitem( + sys.modules, + "PySide6", + SimpleNamespace(QtTest=SimpleNamespace(QTest=QTest)), + ) + monkeypatch.setattr( + control, + "_gui", + lambda: SimpleNamespace(GuiUp=True, getMainWindow=lambda: window), + ) + + payload = control.dispatch( + "ui_click", {"kind": "action", "text": "PartDesign_DesignExtrude"} + ) + assert payload["ok"] is True + assert payload["action_pick"] == "enabled_visible" + assert payload["action_match_count"] == 2 + assert hidden.triggered == 0 + assert visible.triggered == 1 + + +def test_ui_action_click_runs_command_when_every_qaction_is_disabled( + monkeypatch, +) -> None: + class Point: + def __init__(self, x: int, y: int) -> None: + self._x = x + self._y = y + + def x(self) -> int: + return self._x + + def y(self) -> int: + return self._y + + class QAction: + pass + + class Action: + def __init__(self) -> None: + self.triggered = 0 + + def text(self) -> str: + return "Extrude" + + def objectName(self) -> str: # noqa: N802 + return "PartDesign_DesignExtrude" + + def isEnabled(self) -> bool: # noqa: N802 + return False + + def isVisible(self) -> bool: # noqa: N802 + return False + + def menu(self): + return None + + def trigger(self) -> None: + self.triggered += 1 + + class Command: + def __init__(self) -> None: + self.ran = 0 + + def run(self) -> None: + self.ran += 1 + + command = Command() + scheduled: list[Any] = [] + + class QTimer: + @staticmethod + def singleShot(_milliseconds: int, callback) -> None: # noqa: N802 + scheduled.append(callback) + + action = Action() + + def find_children(kind): + if kind is QAction: + return [action] + return [] + + class MenuBar: + def actions(self) -> list: + return [] + + window = SimpleNamespace( + menuBar=lambda: MenuBar(), + actions=lambda: [], + findChildren=find_children, + ) + application = SimpleNamespace( + focus=None, + active_window=window, + popup=None, + ) + qt_core = SimpleNamespace( + Qt=SimpleNamespace( + LeftButton="left", + NoModifier="none", + OtherFocusReason="other", + ), + QTimer=QTimer, + ) + qt_gui = SimpleNamespace( + QCursor=SimpleNamespace(pos=lambda: Point(10, 20)), + QAction=QAction, + ) + qt_widgets = SimpleNamespace( + QTabBar=object, + QToolBar=object, + QApplication=SimpleNamespace( + processEvents=lambda: None, + focusWidget=lambda: application.focus, + activeWindow=lambda: application.active_window, + activePopupWidget=lambda: application.popup, + ), + ) + + class QTest: + @staticmethod + def mouseClick(_widget, _button, _modifiers, _point) -> None: # noqa: N802 + raise AssertionError("action clicks must not use QTest.mouseClick") + + monkeypatch.setitem( + sys.modules, + "PySide", + SimpleNamespace(QtCore=qt_core, QtGui=qt_gui, QtWidgets=qt_widgets), + ) + monkeypatch.setitem( + sys.modules, + "PySide6", + SimpleNamespace(QtTest=SimpleNamespace(QTest=QTest)), + ) + monkeypatch.setattr( + control, + "_gui", + lambda: SimpleNamespace( + GuiUp=True, + getMainWindow=lambda: window, + isCommandActive=lambda name: name == "PartDesign_DesignExtrude", + Command=SimpleNamespace( + update=lambda: None, + get=lambda name: command if name == "PartDesign_DesignExtrude" else None, + ), + ), + ) + + payload = control.dispatch( + "ui_click", {"kind": "action", "text": "PartDesign_DesignExtrude"} + ) + assert payload["ok"] is True + assert payload["action_pick"] == "command_active" + assert payload["command_active"] is True + assert payload["action_match_count"] == 1 + assert payload["click_queued"] is True + assert action.triggered == 0 + assert command.ran == 0 + assert len(scheduled) == 1 + scheduled[0]() + assert command.ran == 1 + assert action.triggered == 0 + + +def test_ui_action_click_disabled_payload_includes_action_pick_when_inactive( + monkeypatch, +) -> None: + class Point: + def __init__(self, x: int, y: int) -> None: + self._x = x + self._y = y + + def x(self) -> int: + return self._x + + def y(self) -> int: + return self._y + + class QAction: + pass + + class Action: + def text(self) -> str: + return "Extrude" + + def objectName(self) -> str: # noqa: N802 + return "PartDesign_DesignExtrude" + + def isEnabled(self) -> bool: # noqa: N802 + return False + + def isVisible(self) -> bool: # noqa: N802 + return False + + def menu(self): + return None + + def trigger(self) -> None: + raise AssertionError("disabled QAction.trigger must not run") + + action = Action() + + def find_children(kind): + if kind is QAction: + return [action] + return [] + + class MenuBar: + def actions(self) -> list: + return [] + + window = SimpleNamespace( + menuBar=lambda: MenuBar(), + actions=lambda: [], + findChildren=find_children, + ) + application = SimpleNamespace( + focus=None, + active_window=window, + popup=None, + ) + qt_core = SimpleNamespace( + Qt=SimpleNamespace( + LeftButton="left", + NoModifier="none", + OtherFocusReason="other", + ) + ) + qt_gui = SimpleNamespace( + QCursor=SimpleNamespace(pos=lambda: Point(10, 20)), + QAction=QAction, + ) + qt_widgets = SimpleNamespace( + QTabBar=object, + QToolBar=object, + QApplication=SimpleNamespace( + processEvents=lambda: None, + focusWidget=lambda: application.focus, + activeWindow=lambda: application.active_window, + activePopupWidget=lambda: application.popup, + ), + ) + + class QTest: + @staticmethod + def mouseClick(_widget, _button, _modifiers, _point) -> None: # noqa: N802 + raise AssertionError("action clicks must not use QTest.mouseClick") + + monkeypatch.setitem( + sys.modules, + "PySide", + SimpleNamespace(QtCore=qt_core, QtGui=qt_gui, QtWidgets=qt_widgets), + ) + monkeypatch.setitem( + sys.modules, + "PySide6", + SimpleNamespace(QtTest=SimpleNamespace(QTest=QTest)), + ) + monkeypatch.setattr( + control, + "_gui", + lambda: SimpleNamespace( + GuiUp=True, + getMainWindow=lambda: window, + isCommandActive=lambda _name: False, + Command=SimpleNamespace(update=lambda: None, get=lambda _name: None), + ), + ) + + payload = control.dispatch( + "ui_click", {"kind": "action", "text": "PartDesign_DesignExtrude"} + ) + assert payload["ok"] is False + assert payload["failure_code"] == "UI_TARGET_DISABLED" + assert payload["action_pick"] == "disabled" + assert payload["action_match_count"] == 1 + assert payload["command_active"] is False + + +def test_ui_action_click_queues_trigger_so_a_modal_cannot_hold_http( + monkeypatch, +) -> None: + class Point: + def __init__(self, x: int, y: int) -> None: + self._x = x + self._y = y + + def x(self) -> int: + return self._x + + def y(self) -> int: + return self._y + + class QAction: + pass + + class Action: + def __init__(self) -> None: + self.triggered = 0 + + def text(self) -> str: + return "New Sketch" + + def objectName(self) -> str: # noqa: N802 + return "Sketcher_NewSketch" + + def isEnabled(self) -> bool: # noqa: N802 + return True + + def isVisible(self) -> bool: # noqa: N802 + return True + + def menu(self): + return None + + def trigger(self) -> None: + self.triggered += 1 + raise AssertionError("modal exec would hold the HTTP request") + + action = Action() + scheduled: list[Any] = [] + + class QTimer: + @staticmethod + def singleShot(_milliseconds: int, callback) -> None: # noqa: N802 + scheduled.append(callback) + + def find_children(kind): + if kind is QAction: + return [action] + return [] + + class MenuBar: + def actions(self) -> list: + return [] + + window = SimpleNamespace( + menuBar=lambda: MenuBar(), + actions=lambda: [], + findChildren=find_children, + ) + application = SimpleNamespace( + focus=None, + active_window=window, + popup=None, + ) + qt_core = SimpleNamespace( + Qt=SimpleNamespace( + LeftButton="left", + NoModifier="none", + OtherFocusReason="other", + ), + QTimer=QTimer, + ) + qt_gui = SimpleNamespace( + QCursor=SimpleNamespace(pos=lambda: Point(10, 20)), + QAction=QAction, + ) + qt_widgets = SimpleNamespace( + QTabBar=object, + QToolBar=object, + QApplication=SimpleNamespace( + processEvents=lambda: None, + focusWidget=lambda: application.focus, + activeWindow=lambda: application.active_window, + activePopupWidget=lambda: application.popup, + ), + ) + + class QTest: + @staticmethod + def mouseClick(_widget, _button, _modifiers, _point) -> None: # noqa: N802 + raise AssertionError("action clicks must not use QTest.mouseClick") + + monkeypatch.setitem( + sys.modules, + "PySide", + SimpleNamespace(QtCore=qt_core, QtGui=qt_gui, QtWidgets=qt_widgets), + ) + monkeypatch.setitem( + sys.modules, + "PySide6", + SimpleNamespace(QtTest=SimpleNamespace(QTest=QTest)), + ) + monkeypatch.setattr( + control, + "_gui", + lambda: SimpleNamespace(GuiUp=True, getMainWindow=lambda: window), + ) + + payload = control.dispatch( + "ui_click", {"kind": "action", "text": "Sketcher_NewSketch"} + ) + assert payload["ok"] is True + assert payload["click_queued"] is True + assert payload["object_name"] == "Sketcher_NewSketch" + assert action.triggered == 0 + assert len(scheduled) == 1 + + +def test_ui_dialog_click_accepts_choose_orientation_ok(monkeypatch) -> None: + class Point: + def __init__(self, x: int, y: int) -> None: + self._x = x + self._y = y + + def x(self) -> int: + return self._x + + def y(self) -> int: + return self._y + + class QDialog: + pass + + class QDialogButtonBox: + Ok = "ok" + + class QRadioButton: + pass + + class Radio: + def isChecked(self) -> bool: # noqa: N802 + return True + + def click(self) -> None: + raise AssertionError("XY-plane was already selected") + + class Button: + def __init__(self) -> None: + self.clicked = 0 + + def click(self) -> None: + self.clicked += 1 + dialog.visible = False + dialog.accepted += 1 + + ok_button = Button() + xy = Radio() + + class ButtonBox: + def button(self, flag): + assert flag == QDialogButtonBox.Ok + return ok_button + + button_box = ButtonBox() + + class Dialog: + def __init__(self) -> None: + self.visible = True + self.accepted = 0 + + def windowTitle(self) -> str: # noqa: N802 + return "Choose Orientation" + + def isVisible(self) -> bool: # noqa: N802 + return self.visible + + def findChild(self, kind, name=None): # noqa: N802 + if kind is QRadioButton and name == "XY_radioButton": + return xy + if kind is QDialogButtonBox: + return button_box + return None + + def accept(self) -> None: + self.accepted += 1 + self.visible = False + + dialog = Dialog() + + def find_children(kind): + if kind is QDialog: + return [dialog] + return [] + + class MenuBar: + def actions(self) -> list: + return [] + + window = SimpleNamespace( + menuBar=lambda: MenuBar(), + actions=lambda: [], + findChildren=find_children, + ) + application = SimpleNamespace( + focus=None, + active_window=window, + popup=None, + modal=dialog, + ) + qt_core = SimpleNamespace( + Qt=SimpleNamespace( + LeftButton="left", + NoModifier="none", + OtherFocusReason="other", + ) + ) + qt_gui = SimpleNamespace(QCursor=SimpleNamespace(pos=lambda: Point(10, 20))) + qt_widgets = SimpleNamespace( + QTabBar=object, + QToolBar=object, + QDialog=QDialog, + QDialogButtonBox=QDialogButtonBox, + QRadioButton=QRadioButton, + QApplication=SimpleNamespace( + processEvents=lambda: None, + focusWidget=lambda: application.focus, + activeWindow=lambda: application.active_window, + activePopupWidget=lambda: application.popup, + activeModalWidget=lambda: application.modal, + ), + ) + + class QTest: + @staticmethod + def mouseClick(_widget, _button, _modifiers, _point) -> None: # noqa: N802 + raise AssertionError("dialog clicks must not use QTest.mouseClick") + + monkeypatch.setitem( + sys.modules, + "PySide", + SimpleNamespace(QtCore=qt_core, QtGui=qt_gui, QtWidgets=qt_widgets), + ) + monkeypatch.setitem( + sys.modules, + "PySide6", + SimpleNamespace(QtTest=SimpleNamespace(QTest=QTest)), + ) + monkeypatch.setattr( + control, + "_gui", + lambda: SimpleNamespace(GuiUp=True, getMainWindow=lambda: window), + ) + + payload = control.dispatch("ui_click", {"kind": "dialog", "text": "OK"}) + assert payload["ok"] is True + assert payload["input_method"] == "qt_in_process_dialog_button" + assert payload["physical_cursor_control"] == "none" + assert payload["object_name"] == "Choose Orientation" + assert ok_button.clicked == 1 + assert dialog.accepted == 1 + assert dialog.visible is False + + def test_screenshot_captures_the_visible_vibecad_window(tmp_path, monkeypatch) -> None: target = tmp_path / "visible-vibecad.png" @@ -2908,6 +3912,17 @@ def test_cli_maps_semantic_menu_snapshot_and_independent_ui_click() -> None: "expected_index": 7, } + action_click = cli.build_parser().parse_args( + ["ui-click", "--kind", "action", "--text", "Std_New"] + ) + assert cli._command_arguments(action_click)["kind"] == "action" + assert cli._http_route(action_click.command) == ("POST", "/v1/ui/click") + + dialog_click = cli.build_parser().parse_args( + ["ui-click", "--kind", "dialog", "--text", "OK"] + ) + assert cli._command_arguments(dialog_click)["kind"] == "dialog" + screenshot = cli.build_parser().parse_args( ["screenshot", "--path", "C:\\Evidence\\vibecad.png"] ) diff --git a/src/Mod/VibeCAD/vibecad_tests/test_codex_runtime_package.py b/src/Mod/VibeCAD/vibecad_tests/test_codex_runtime_package.py index 43d0c309..8210e72b 100644 --- a/src/Mod/VibeCAD/vibecad_tests/test_codex_runtime_package.py +++ b/src/Mod/VibeCAD/vibecad_tests/test_codex_runtime_package.py @@ -20,7 +20,7 @@ def test_codex_runtime_release_is_consistent(): for t in node.targets) ) installer = (root / "package/rattler-build/scripts/install_vibecad_codex_runtime.sh").read_text() - assert version == "0.153.4" + assert version == "0.154.0" assert f'codex_version="{version}"' in installer archives = re.findall(r'archive="([^"]+)"\s+archive_sha256="([a-f0-9]{64})"', installer) assert len(archives) == 6 diff --git a/src/Mod/VibeCAD/vibecad_tests/test_native_codex_runtime_build.py b/src/Mod/VibeCAD/vibecad_tests/test_native_codex_runtime_build.py index 3e125193..06b6e568 100644 --- a/src/Mod/VibeCAD/vibecad_tests/test_native_codex_runtime_build.py +++ b/src/Mod/VibeCAD/vibecad_tests/test_native_codex_runtime_build.py @@ -77,12 +77,12 @@ def test_native_runtime_opt_out_does_not_install_or_download(project): def test_fresh_runtime_is_installed_with_companions(installed, tmp_path): run, build, runtime = installed - assert "0.153.4" in run(str(runtime / "bin/codex-app-server"), "--version") + assert "0.154.0" in run(str(runtime / "bin/codex-app-server"), "--version") assert (runtime / "bin/codex-code-mode-host").is_file() prefix = tmp_path / "installed" run("cmake", "--install", str(build), "--prefix", str(prefix)) packaged = prefix / "Mod/VibeCAD/codex_runtime" - assert "0.153.4" in run(str(packaged / "bin/codex-app-server"), "--version") + assert "0.154.0" in run(str(packaged / "bin/codex-app-server"), "--version") assert (packaged / "bin/codex-code-mode-host").is_file() @@ -101,7 +101,7 @@ def test_rebuild_repairs_stale_version_metadata_with_newer_mtime(installed): metadata["version"] = "0.144.5" manifest.write_text(json.dumps(metadata)) run("cmake", "--build", str(build), "--parallel", "12") - assert json.loads(manifest.read_text())["version"] == "0.153.4" + assert json.loads(manifest.read_text())["version"] == "0.154.0" def test_rebuild_replaces_old_executable_even_with_current_metadata(installed): @@ -111,7 +111,7 @@ def test_rebuild_replaces_old_executable_even_with_current_metadata(installed): binary.chmod(0o755) assert "0.144.5" in run(str(binary), "--version") run("cmake", "--build", str(build), "--parallel", "12") - assert "0.153.4" in run(str(binary), "--version") + assert "0.154.0" in run(str(binary), "--version") def test_current_runtime_is_verified_without_replacing_binaries(installed): diff --git a/src/Mod/VibeCAD/vibecad_tests/test_workflow_harness_contract.py b/src/Mod/VibeCAD/vibecad_tests/test_workflow_harness_contract.py new file mode 100644 index 00000000..914587de --- /dev/null +++ b/src/Mod/VibeCAD/vibecad_tests/test_workflow_harness_contract.py @@ -0,0 +1,75 @@ +# SPDX-License-Identifier: LGPL-2.1-or-later + +"""Contracts for the workflow harness that reuses the visible-tour click path.""" + +from __future__ import annotations + +from pathlib import Path + + +REPOSITORY_ROOT = Path(__file__).resolve().parents[4] +TOUR_SCRIPT = REPOSITORY_ROOT / "Invoke-VibeCAD-VisibleTour.ps1" +HARNESS = REPOSITORY_ROOT / "tools" / "vibecad_workflow_harness.py" +CHANNEL = REPOSITORY_ROOT / "tools" / "vibecad_workflow_channel.py" +WORKFLOWS = REPOSITORY_ROOT / "tools" / "vibecad_workflows.json" + + +def test_workflow_harness_reuses_the_tour_click_route() -> None: + tour = TOUR_SCRIPT.read_text(encoding="utf-8") + harness = HARNESS.read_text(encoding="utf-8") + channel = CHANNEL.read_text(encoding="utf-8") + workflows = WORKFLOWS.read_text(encoding="utf-8") + + assert "/v1/ui/click" in tour + assert "Invoke-VibeCADAgentPost" in tour + assert "/v1/ui/click" in harness + assert "/v1/ui/click" in channel + assert "new_document" in workflows + assert "sketch_then_pad" in workflows + assert "\"export\"" in workflows + assert '"kind": "menu"' not in workflows + assert "PartDesign_NewBody" in workflows + assert "Sketcher_NewSketch" in workflows + assert "leave_active_sketch" in workflows + assert "leaveActiveSketch" in channel + assert "PartDesign_DesignExtrude" in workflows + assert "place_closed_circle" in workflows + assert "export_step" in workflows + assert workflows.count("Std_New") == 1 + assert "InternalFace1" in channel + assert "Document Recovery" in channel + assert "Start Recovery" in channel + assert "Cancel" in channel + assert "PartDesign_Pad" not in workflows + assert "Part.Circle" in channel + assert "Import.export" in channel + assert "exportStep" in channel + assert "addGeometry" in channel + assert "/v1/run" in channel + agent_control = ( + REPOSITORY_ROOT / "src" / "Mod" / "VibeCAD" / "VibeCADAgentControl.py" + ).read_text(encoding="utf-8") + assert "_pick_clickable_qt_action" in agent_control + assert "_command_is_active" in agent_control + assert "_named_command_runner" in agent_control + assert "command_active" in agent_control + assert '"kind": "dialog"' in workflows + assert '"text": "OK"' in workflows + assert "Std_New" in workflows + assert "Std_Export" not in workflows + assert "pyautogui" not in channel + assert "SetCursorPos" not in channel + assert "SendInput" not in channel + assert "accept_design_task" in workflows + assert "accept_design_task" in channel + assert "QDialogButtonBox" in channel + assert "DesignBodyPublication" in channel + assert "DesignBodyPublication" in workflows + + +def test_workflow_harness_does_not_turn_the_tour_into_a_test() -> None: + tour = TOUR_SCRIPT.read_text(encoding="utf-8") + harness = HARNESS.read_text(encoding="utf-8") + assert "tour_remains_demo" in harness + assert "workflow-harness" not in tour.lower() + assert "TYPESAFE_API_KEY" not in tour diff --git a/src/Tools/tests/test_macos_build_toolchain.py b/src/Tools/tests/test_macos_build_toolchain.py index 826040dd..fae22a73 100644 --- a/src/Tools/tests/test_macos_build_toolchain.py +++ b/src/Tools/tests/test_macos_build_toolchain.py @@ -1,5 +1,6 @@ # SPDX-License-Identifier: LGPL-2.1-or-later +import fnmatch import os import subprocess import tempfile @@ -21,6 +22,55 @@ class TestMacOSBuildToolchain(unittest.TestCase): + def test_prebuilt_codex_runtime_is_not_relinked_by_rattler(self) -> None: + # These upstream executables use system libraries, not the conda + # prefix. Adding conda RPATHs can overflow rg/zsh's Mach-O headers. + # Keep relocation enabled for every other native file. + recipe = RECIPE.read_text(encoding="utf-8") + self.assertIn( + " dynamic_linking:\n" + " binary_relocation:\n" + " - if: osx\n" + " then:\n" + " - bin/**\n" + " - lib/**\n" + " - Library/**\n" + " - PlugIns/**\n" + " - '**/*.so'\n" + " - '**/*.dylib'\n" + " - '**/*.bundle'\n" + " else:\n" + " - '**'\n", + recipe, + ) + self.assertNotIn("binary_relocation: false", recipe) + + def test_macos_relocation_covers_nested_bundle_executables(self) -> None: + recipe = RECIPE.read_text(encoding="utf-8") + selection = recipe.split(" binary_relocation:\n", 1)[1].split( + " then:\n", 1 + )[1].split(" else:\n", 1)[0] + patterns = [line.strip()[2:].strip("'") for line in selection.splitlines()] + for path in ( + "bin/freecad", + "lib/libFreeCADApp.dylib", + "Mod/Part/Part.so", + "Library/QuickLook/QuicklookFCStd.qlgenerator/Contents/MacOS/QuicklookFCStd", + "PlugIns/FreeCADThumbnailExtension.appex/Contents/MacOS/FreeCADThumbnailExtension", + "PlugIns/FreeCADPreviewExtension.appex/Contents/MacOS/FreeCADPreviewExtension", + ): + with self.subTest(path=path): + self.assertTrue(any(fnmatch.fnmatchcase(path, p) for p in patterns), path) + for path in ( + "bin/codex-app-server", + "bin/codex-code-mode-host", + "codex-path/rg", + "codex-resources/zsh/bin/zsh", + ): + with self.subTest(path=path): + installed = "Mod/VibeCAD/codex_runtime/" + path + self.assertFalse(any(fnmatch.fnmatchcase(installed, p) for p in patterns)) + def test_macos_build_exports_flags_to_cmake(self) -> None: # Execute the complete recipe script, replacing external build/install # commands with stubs. CMake's stub starts a child shell so a shell-only diff --git a/tools/vibecad_workflow_channel.py b/tools/vibecad_workflow_channel.py new file mode 100644 index 00000000..2e97340b --- /dev/null +++ b/tools/vibecad_workflow_channel.py @@ -0,0 +1,1250 @@ +# SPDX-License-Identifier: LGPL-2.1-or-later +"""Loopback click channel used by the VibeCAD workflow harness. + +This is the same authenticated HTTP surface the visible tour uses: +``GET /v1/status``, ``GET /v1/documents``, ``GET /v1/ui/ribbon``, +``GET /v1/ui/menus``, ``POST /v1/ui/click``, and ``POST /v1/run``. +Clicks stay on that one route. Nothing here drives the OS mouse. +""" + +from __future__ import annotations + +from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer +import json +from pathlib import Path +import threading +from typing import Any +from urllib import error, request +from urllib.parse import urlsplit + + +LOOPBACK_HOSTS = frozenset({"127.0.0.1", "localhost", "::1"}) +TREE_INSPECT_PYTHON = """ +# vibecad.workflow-harness:inspect_tree +doc = App.ActiveDocument +result = { + "document": None if doc is None else str(doc.Name), + "objects": [] + if doc is None + else [ + { + "name": str(obj.Name), + "type_id": str(obj.TypeId), + "label": str(getattr(obj, "Label", "") or ""), + "geometry_count": int(getattr(obj, "GeometryCount", 0) or 0), + } + for obj in list(doc.Objects) + ], +} +""" + +# Same in-process SketchObject.addGeometry + Part.Circle call +# TestConsolidatedPartTools._circle_sketch, +# TestSketcherSolver.CreateCircleSketch, and +# TestDesignProfileRegionsGui._master_sketch already use in this tree. +# Sketcher_CreateRectangle is a real command, but activated() only +# starts DrawSketchHandlerRectangle; the two corners need view clicks. +PLACE_CLOSED_CIRCLE_PYTHON = """ +# vibecad.workflow-harness:place_closed_circle +import Part +import PartDesign + +doc = App.ActiveDocument +if doc is None: + raise RuntimeError("No active document") +sketch = next( + ( + obj + for obj in list(doc.Objects) + if obj.isDerivedFrom("Sketcher::SketchObject") + ), + None, +) +if sketch is None: + raise RuntimeError("No Sketcher::SketchObject") +if Gui.editDocument() is not None: + raise RuntimeError("Sketch is still in edit mode") +if int(sketch.GeometryCount) == 0: + sketch.addGeometry( + Part.Circle(App.Vector(0, 0, 0), App.Vector(0, 0, 1), 10), + False, + ) +PartDesign.finalizeDesignDefinition(sketch) +doc.recompute() +if hasattr(doc, "hasPendingTransaction") and doc.hasPendingTransaction(): + doc.commitTransaction() +result = { + "sketch": str(sketch.Name), + "geometry_count": int(sketch.GeometryCount), +} +""" + +# DocumentRecovery.ui title is "Document Recovery". +# DocumentRecovery.cpp relabels Ok to "Start Recovery". +# The other standard button is Cancel (QDialogButtonBox::Cancel). +DISMISS_DOCUMENT_RECOVERY_PYTHON = """ +# vibecad.workflow-harness:dismiss_document_recovery +QtWidgets = None +for module_name in ("PySide6.QtWidgets", "PySide2.QtWidgets", "PySide.QtWidgets"): + try: + QtWidgets = __import__(module_name, fromlist=["QDialog"]) + break + except ImportError: + continue +if QtWidgets is None: + raise RuntimeError("Qt widgets are unavailable") + +dismissed = [] +main = Gui.getMainWindow() +for dialog in list(main.findChildren(QtWidgets.QDialog) or []): + if not bool(getattr(dialog, "isVisible", lambda: False)()): + continue + title = str(dialog.windowTitle() or "") + if title != "Document Recovery": + continue + box = dialog.findChild(QtWidgets.QDialogButtonBox) + cancel = box.button(QtWidgets.QDialogButtonBox.Cancel) if box is not None else None + if cancel is None: + raise RuntimeError("Document Recovery has no Cancel button") + cancel.click() + dismissed.append("Cancel") +result = {"dismissed": dismissed, "button": "Cancel"} +""" + +# SketcherGui.leaveActiveSketch is the exact-edit finish +# (SketchEditControl.cpp). CmdSketcherLeaveSketch::activated() passes +# getActiveGuiDocument(), which is Application::activeDocument(). +# requireExactEditState throws "The exact Sketch document is no longer +# active in edit mode" when activeDocument() != editDocument(). +# A queued LeaveSketch click plus Std_New per workflow creates that +# split. This recipe uses the edit document, not the active tab. +LEAVE_ACTIVE_SKETCH_PYTHON = """ +# vibecad.workflow-harness:leave_active_sketch +import SketcherGui + +edit = Gui.editDocument() +if edit is None: + result = { + "edit_mode": "closed", + "left": False, + "document": None, + "sketch": None, + } +else: + app_doc = edit.Document + view = edit.getInEdit() + sketch = getattr(view, "Object", None) if view is not None else None + if sketch is None or not sketch.isDerivedFrom("Sketcher::SketchObject"): + raise RuntimeError( + "The exact requested Sketch is no longer the active edit target." + ) + native = SketcherGui.leaveActiveSketch( + str(app_doc.Name), + str(app_doc.Uid), + str(sketch.Name), + ) + result = { + "edit_mode": native.get("edit_mode"), + "left": True, + "document": str(app_doc.Name), + "sketch": str(sketch.Name), + "accepted_task_dialog": native.get("accepted_task_dialog"), + } +if Gui.editDocument() is not None: + raise RuntimeError("The exact Sketch edit session did not close") +""" + +# designProfileOperationActive() requires a reusable sketch or InternalFace* +# (ReferenceSelection.cpp). Edge1 disables Extrude. The live enable path in +# TestDesignProfileRegionsGui is addSelection(sketch, "InternalFace1") plus +# Gui.Command.update() so the QAction isEnabled flag matches isActive(). +SELECT_SKETCH_PYTHON = """ +# vibecad.workflow-harness:select_sketch +import PartDesign + +doc = App.ActiveDocument +if doc is None: + raise RuntimeError("No active document") +sketch = next( + ( + obj + for obj in list(doc.Objects) + if obj.isDerivedFrom("Sketcher::SketchObject") + ), + None, +) +if sketch is None: + raise RuntimeError("No Sketcher::SketchObject") +if int(sketch.GeometryCount) < 1: + raise RuntimeError("Sketch has no geometry to select") +PartDesign.finalizeDesignDefinition(sketch) +doc.recompute() +faces = list(getattr(getattr(sketch, "InternalShape", None), "Faces", []) or []) +if not faces: + raise RuntimeError( + "Sketch has no InternalFace; closed profile did not produce a filled area" + ) +Gui.Selection.clearSelection() +Gui.Selection.addSelection(sketch, "InternalFace1") +if hasattr(doc, "hasPendingTransaction") and doc.hasPendingTransaction(): + doc.commitTransaction() +if hasattr(Gui, "Command") and hasattr(Gui.Command, "update"): + Gui.Command.update() +command_active = False +if hasattr(Gui, "isCommandActive"): + command_active = bool(Gui.isCommandActive("PartDesign_DesignExtrude")) +result = { + "selected": str(sketch.Name), + "sub": "InternalFace1", + "command_active": command_active, +} +""" + +# TestDesignProfileRegionsGui._close_task clicks the visible Tasks-dock +# QDialogButtonBox OK, then waits until Gui.Control.activeDialog is gone. +# Kind dialog looks for a QDialog titled OK and finds none: Extrude OK +# is not a modal. After that OK, DesignExtrude.Shape stays null +# (test_final_result_checkbox_renders_unpublished_design_output). +# The solid is BodyResult, type PartDesign::DesignBodyPublication. +ACCEPT_DESIGN_TASK_PYTHON = """ +# vibecad.workflow-harness:accept_design_task +QtWidgets = None +QtCore = None +for module_name in ("PySide6", "PySide2", "PySide"): + try: + QtWidgets = __import__( + module_name + ".QtWidgets", + fromlist=["QDialogButtonBox"], + ) + QtCore = __import__( + module_name + ".QtCore", + fromlist=["QElapsedTimer", "QEventLoop", "QTimer"], + ) + break + except ImportError: + continue +if QtWidgets is None or QtCore is None: + raise RuntimeError("Qt widgets are unavailable") + +def _process_events(wait_ms=20): + if hasattr(Gui, "updateGui"): + Gui.updateGui() + application = QtWidgets.QApplication.instance() + if application is not None: + application.processEvents() + if wait_ms: + loop = QtCore.QEventLoop() + QtCore.QTimer.singleShot(wait_ms, loop.quit) + loop.exec() + +def _task_button(standard_button): + _process_events() + for button_box in Gui.getMainWindow().findChildren( + QtWidgets.QDialogButtonBox + ): + if not button_box.isVisible(): + continue + button = button_box.button(standard_button) + if button is not None and button.isVisible() and button.isEnabled(): + return button + return None + +if not Gui.Control.activeDialog(): + raise RuntimeError("The Design operation task panel did not open") +button = _task_button(QtWidgets.QDialogButtonBox.Ok) +if button is None: + raise RuntimeError("No visible task QDialogButtonBox OK") +button.click() +timer = QtCore.QElapsedTimer() +timer.start() +finished = False +while timer.elapsed() < 5000: + _process_events() + if not Gui.Control.activeDialog(): + finished = True + break +if not finished: + raise RuntimeError("The task did not finish accepting or cancelling") +doc = App.ActiveDocument +if doc is not None: + doc.recompute() +publication = next( + ( + obj + for obj in list(getattr(doc, "Objects", []) or []) + if str(obj.TypeId) == "PartDesign::DesignBodyPublication" + ), + None, +) +if publication is None: + raise RuntimeError("No PartDesign::DesignBodyPublication after task OK") +result = { + "accepted": True, + "button": "Ok", + "publication": str(publication.Name), + "type_id": str(publication.TypeId), +} +""" + +# Import.export is the in-process exporter Std_Export calls after +# FileDialog::getSaveFileName (src/Mod/Import/App/AppImportPy.cpp). +# WriterStep::write only throws on IFSelect RetError/RetFail/RetStop. +# Live 6714cd65: Import.export returned, then os.path.getsize raised +# FileNotFoundError on the Windows temp path. TopoShape.exportStep +# (TopoShapePy / TopoShape.cpp) is the STEP write Part.export uses. +# Live after Extrude task OK: DesignExtrude.Shape is still null. The +# solid with Faces is BodyResult (PartDesign::DesignBodyPublication). +EXPORT_STEP_PYTHON = """ +# vibecad.workflow-harness:export_step +import Import +import os + +doc = App.ActiveDocument +if doc is None: + raise RuntimeError("No active document") +doc.recompute() + +def _faces(obj): + shape = getattr(obj, "Shape", None) + is_null = getattr(shape, "isNull", None) + if shape is None or (callable(is_null) and bool(is_null())): + return [] + return list(getattr(shape, "Faces", []) or []) + +with_faces = [ + (obj, _faces(obj)) + for obj in list(doc.Objects) +] +with_faces = [(obj, faces) for obj, faces in with_faces if faces] +solid = next( + ( + obj + for obj, _ignored in with_faces + if str(obj.TypeId) == "PartDesign::DesignBodyPublication" + ), + None, +) +if solid is None and with_faces: + solid = with_faces[0][0] +if solid is None: + raise RuntimeError("No object with Faces to export") +shape = solid.Shape +path = __EXPORT_PATH__ +parent = os.path.dirname(path) +if parent: + os.makedirs(parent, exist_ok=True) +writer = "Import.export" +Import.export([solid], path) +if not os.path.isfile(path) or os.path.getsize(path) < 1: + exporter = getattr(shape, "exportStep", None) + if not callable(exporter): + raise RuntimeError("Shape.exportStep is unavailable") + exporter(path) + writer = "Shape.exportStep" +if not os.path.isfile(path): + raise RuntimeError("STEP was not written at " + path) +size = os.path.getsize(path) +if size < 1: + raise RuntimeError("STEP at " + path + " is empty") +result = { + "exported_path": path, + "bytes": size, + "object": str(solid.Name), + "type_id": str(solid.TypeId), + "writer": writer, + "face_count": len(_faces(solid)), +} +""" + + +def workflow_run_python(recipe_id: str, *, export_path: str = "") -> str: + recipes = { + "place_closed_circle": PLACE_CLOSED_CIRCLE_PYTHON, + "select_sketch": SELECT_SKETCH_PYTHON, + "leave_active_sketch": LEAVE_ACTIVE_SKETCH_PYTHON, + "dismiss_document_recovery": DISMISS_DOCUMENT_RECOVERY_PYTHON, + "accept_design_task": ACCEPT_DESIGN_TASK_PYTHON, + "export_step": EXPORT_STEP_PYTHON.replace( + "__EXPORT_PATH__", json.dumps(str(export_path)) + ), + "inspect_tree": TREE_INSPECT_PYTHON, + } + try: + return recipes[str(recipe_id or "").strip()] + except KeyError as exc: + raise ValueError(f"Unknown workflow run recipe {recipe_id!r}.") from exc + + +def require_loopback_url(base_url: str) -> str: + parsed = urlsplit(str(base_url or "").strip()) + host = str(parsed.hostname or "").strip().lower() + if ( + parsed.scheme.lower() != "http" + or host not in LOOPBACK_HOSTS + or parsed.username is not None + or parsed.password is not None + or parsed.query + or parsed.fragment + ): + raise ValueError( + "The workflow harness only talks to an authenticated loopback " + "http://127.0.0.1: agent." + ) + return str(base_url).rstrip("/") + + +def click_input_method(kind: str) -> str: + return { + "ribbon": "qt_in_process_mouse_click", + "menu": "qt_in_process_menu_popup", + "action": "qt_in_process_action_trigger", + "command": "qt_in_process_action_trigger", + "button": "qt_in_process_action_trigger", + "dialog": "qt_in_process_dialog_button", + }.get(str(kind or "").strip().lower(), "") + + +class AgentClickChannel: + """HTTP client for the existing loopback ``/v1/ui/click`` path.""" + + def __init__( + self, + base_url: str, + token: str, + *, + timeout_seconds: float = 30.0, + ) -> None: + self.base_url = require_loopback_url(base_url) + self.token = str(token or "").strip() + if len(self.token) < 40: + raise ValueError("The checkout-scoped bearer token is invalid.") + self.timeout_seconds = float(timeout_seconds) + + def request(self, method: str, route: str, body: dict[str, Any] | None = None) -> dict[str, Any]: + headers = { + "Accept": "application/json", + "Authorization": f"Bearer {self.token}", + } + data = None + if method == "POST": + headers["Content-Type"] = "application/json" + data = json.dumps(body or {}).encode("utf-8") + http_request = request.Request( + f"{self.base_url}{route}", + data=data, + headers=headers, + method=method, + ) + try: + response = request.urlopen(http_request, timeout=self.timeout_seconds) + try: + payload = json.loads(response.read().decode("utf-8")) + finally: + response.close() + except error.HTTPError as exc: + try: + raw = exc.read() if hasattr(exc, "read") else b"" + try: + payload = json.loads(raw.decode("utf-8")) if raw else {} + except (UnicodeDecodeError, json.JSONDecodeError): + payload = {} + finally: + exc.close() + if isinstance(payload, dict) and payload: + return payload + return { + "ok": False, + "failure_code": "HTTP_ERROR", + "error": f"Agent control HTTP {exc.code}.", + } + if not isinstance(payload, dict): + return { + "ok": False, + "failure_code": "INVALID_RESPONSE", + "error": "Agent control did not return a JSON object.", + } + return payload + + def status(self) -> dict[str, Any]: + return self.request("GET", "/v1/status") + + def documents(self) -> dict[str, Any]: + return self.request("GET", "/v1/documents") + + def click( + self, + kind: str, + text: str, + *, + expected_process_id: int | None = None, + expected_index: int | None = None, + ) -> dict[str, Any]: + body: dict[str, Any] = {"kind": kind, "text": text} + if expected_process_id is not None: + body["expected_process_id"] = expected_process_id + if expected_index is not None: + body["expected_index"] = expected_index + return self.request("POST", "/v1/ui/click", body) + + def inspect_tree(self) -> dict[str, Any]: + return self.request("POST", "/v1/run", {"python": TREE_INSPECT_PYTHON, "recompute": False}) + + def run(self, python: str, *, recompute: bool = True) -> dict[str, Any]: + return self.request( + "POST", + "/v1/run", + {"python": python, "recompute": bool(recompute)}, + ) + + +class FakeAgentState: + """In-memory GUI/document state for CI when no display is available.""" + + def __init__(self, export_dir: str, *, recovery_dialog: bool = False) -> None: + self.export_dir = export_dir + self.documents: list[dict[str, Any]] = [] + self.active_index = -1 + self.selected_ribbon = "Model" + self.exported_path = "" + self.click_count = 0 + self.orientation_dialog = False + self.sketch_edit = False + self.recovery_dialog = bool(recovery_dialog) + self.selection: list[dict[str, str]] = [] + + def active_document(self) -> dict[str, Any] | None: + if self.active_index < 0 or self.active_index >= len(self.documents): + return None + return self.documents[self.active_index] + + def document_summaries(self) -> list[dict[str, Any]]: + summaries = [] + for index, document in enumerate(self.documents): + summaries.append( + { + "document": document["name"], + "label": document["name"], + "path": "", + "active": index == self.active_index, + "object_count": len(document["objects"]), + "modified": True, + } + ) + return summaries + + def tree_result(self) -> dict[str, Any]: + document = self.active_document() + if document is None: + return {"document": None, "objects": []} + return { + "document": document["name"], + "objects": [ + { + "name": obj["name"], + "type_id": obj["type_id"], + "label": obj["label"], + "geometry_count": int(obj.get("geometry_count") or 0), + } + for obj in document["objects"] + ], + } + + def run_python(self, source: str) -> dict[str, Any]: + document = self.active_document() + if "vibecad.workflow-harness:place_closed_circle" in source: + if document is None: + return { + "ok": False, + "failure_code": "SCRIPT_FAILED", + "error": "No active document", + } + sketch = next( + ( + obj + for obj in document["objects"] + if obj["type_id"] == "Sketcher::SketchObject" + ), + None, + ) + if sketch is None: + return { + "ok": False, + "failure_code": "SCRIPT_FAILED", + "error": "No Sketcher::SketchObject", + } + if self.sketch_edit: + return { + "ok": False, + "failure_code": "SCRIPT_FAILED", + "error": "Sketch is still in edit mode", + } + sketch["closed_profile"] = True + sketch["geometry_count"] = max(int(sketch.get("geometry_count") or 0), 1) + # Adding geometry resolves a leftover tree selection to an edge. + # Edge1 is the live disable path in TestDesignProfileRegionsGui. + self.selection = [{"name": sketch["name"], "sub": "Edge1"}] + return { + "ok": True, + "result": { + "sketch": sketch["name"], + "geometry_count": sketch["geometry_count"], + }, + } + if "vibecad.workflow-harness:leave_active_sketch" in source: + if not self.sketch_edit: + return { + "ok": True, + "result": { + "edit_mode": "closed", + "left": False, + "document": None, + "sketch": None, + }, + } + document = self.active_document() + sketch = next( + ( + obj + for obj in (document or {}).get("objects") or [] + if obj["type_id"] == "Sketcher::SketchObject" + ), + None, + ) + self.sketch_edit = False + self.selected_ribbon = "Model" + return { + "ok": True, + "result": { + "edit_mode": "closed", + "left": True, + "document": None if document is None else document["name"], + "sketch": None if sketch is None else sketch["name"], + "accepted_task_dialog": True, + }, + } + if "vibecad.workflow-harness:dismiss_document_recovery" in source: + dismissed = [] + if self.recovery_dialog: + self.recovery_dialog = False + dismissed.append("Cancel") + return { + "ok": True, + "result": {"dismissed": dismissed, "button": "Cancel"}, + } + if "vibecad.workflow-harness:select_sketch" in source: + if document is None: + return { + "ok": False, + "failure_code": "SCRIPT_FAILED", + "error": "No active document", + } + sketch = next( + ( + obj + for obj in document["objects"] + if obj["type_id"] == "Sketcher::SketchObject" + ), + None, + ) + if sketch is None: + return { + "ok": False, + "failure_code": "SCRIPT_FAILED", + "error": "No Sketcher::SketchObject", + } + if int(sketch.get("geometry_count") or 0) < 1: + return { + "ok": False, + "failure_code": "SCRIPT_FAILED", + "error": "Sketch has no geometry to select", + } + if "InternalFace1" in source: + self.selection = [{"name": sketch["name"], "sub": "InternalFace1"}] + command_active = True + sub = "InternalFace1" + else: + # Name-pair addSelection after a circle is the live miss. + self.selection = [{"name": sketch["name"], "sub": "Edge1"}] + command_active = False + sub = "Edge1" + return { + "ok": True, + "result": { + "selected": sketch["name"], + "sub": sub, + "command_active": command_active, + }, + } + if "vibecad.workflow-harness:accept_design_task" in source: + if document is None: + return { + "ok": False, + "failure_code": "SCRIPT_FAILED", + "error": "No active document", + } + if not any(bool(obj.get("design_task")) for obj in document["objects"]): + return { + "ok": False, + "failure_code": "SCRIPT_FAILED", + "error": "The Design operation task panel did not open", + } + if "QDialogButtonBox" not in source: + return { + "ok": False, + "failure_code": "SCRIPT_FAILED", + "error": "No visible task QDialogButtonBox OK", + } + for obj in document["objects"]: + obj["design_task"] = False + publication = next( + ( + obj + for obj in document["objects"] + if obj["type_id"] == "PartDesign::DesignBodyPublication" + ), + None, + ) + if publication is None: + publication = { + "name": "BodyResult", + "type_id": "PartDesign::DesignBodyPublication", + "label": "BodyResult", + "shape_null": False, + "face_count": 3, + } + document["objects"].append(publication) + return { + "ok": True, + "result": { + "accepted": True, + "button": "Ok", + "publication": publication["name"], + "type_id": publication["type_id"], + }, + } + if "vibecad.workflow-harness:export_step" in source: + if document is None: + return { + "ok": False, + "failure_code": "SCRIPT_FAILED", + "error": "No active document", + } + + def _has_faces(obj: dict[str, Any]) -> bool: + return ( + int(obj.get("face_count") or 0) > 0 + and not bool(obj.get("shape_null")) + ) + + solid = next( + ( + obj + for obj in document["objects"] + if obj["type_id"] == "PartDesign::DesignBodyPublication" + and _has_faces(obj) + ), + None, + ) + if solid is None: + solid = next( + (obj for obj in document["objects"] if _has_faces(obj)), + None, + ) + if solid is None: + return { + "ok": False, + "failure_code": "SCRIPT_FAILED", + "error": "No object with Faces to export", + } + path = Path(self.export_dir) / f"{document['name']}.step" + path.write_text( + "ISO-10303-21; /* fake STEP from workflow harness */\n", + encoding="utf-8", + ) + self.exported_path = str(path) + return { + "ok": True, + "result": { + "exported_path": self.exported_path, + "bytes": path.stat().st_size, + "object": solid["name"], + "type_id": solid["type_id"], + "writer": "Import.export", + "face_count": int(solid.get("face_count") or 0), + }, + "exported_path": self.exported_path, + } + return { + "ok": True, + "result": self.tree_result(), + "exported_path": self.exported_path, + } + + def click_payload(self, body: dict[str, Any]) -> dict[str, Any]: + kind = str(body.get("kind") or "").strip().lower().replace("-", "_") + if kind in {"tab", "ribbon_tab"}: + kind = "ribbon" + if kind in {"command", "button"}: + kind = "action" + text = str(body.get("text") or "").strip() + self.click_count += 1 + if kind not in {"ribbon", "menu", "action", "dialog"}: + return { + "ok": False, + "failure_code": "UI_TARGET_KIND_INVALID", + "error": "kind must be 'ribbon', 'menu', 'action', or 'dialog'.", + } + if not text: + return { + "ok": False, + "failure_code": "UI_TARGET_TEXT_REQUIRED", + "error": "text must name one visible semantic UI target.", + } + + details = { + "target_kind": kind, + "target_text": text, + "target_index": 0, + "click_queued": False, + "focus_restored": True, + "active_window_unchanged": True, + "popup_restored": True, + "active_action_restored": True, + "interaction_restored": True, + "input_method": click_input_method(kind), + "physical_cursor_control": "none", + "physical_cursor_unchanged": True, + "semantic_verified": True, + } + + if kind == "ribbon": + if text not in {"Model", "Sketch", "Aero"}: + return { + "ok": False, + "failure_code": "UI_TARGET_NOT_UNIQUE", + "error": f"Expected exactly one ribbon tab named {text!r}; found 0.", + } + if text == "Model" and self.sketch_edit: + return { + "ok": False, + "failure_code": "UI_TARGET_DISABLED", + "error": f"Ribbon tab {text!r} is disabled.", + **details, + "semantic_verified": False, + } + self.selected_ribbon = text + details["selected_after"] = text + return {"ok": True, **details} + + if kind == "menu": + if text != "File": + return { + "ok": False, + "failure_code": "UI_TARGET_NOT_UNIQUE", + "error": f"Expected exactly one top-level menu named {text!r}; found 0.", + } + details.update( + { + "menu_visible": True, + "menu_open_after": False, + "preview_duration_milliseconds": 240, + } + ) + return {"ok": True, **details} + + if kind == "dialog": + return self._click_dialog(text, details) + + return self._click_action(text, details) + + def _click_action(self, text: str, details: dict[str, Any]) -> dict[str, Any]: + if text in {"Std_New", "New"}: + name = f"Unnamed{len(self.documents) + 1}" + self.documents.append({"name": name, "objects": []}) + self.active_index = len(self.documents) - 1 + self.selection = [] + details["object_name"] = "Std_New" + # Creating a document moves focus. The live agent reports that + # as UI_CLICK_NOT_APPLIED; the harness must still pass on the + # new document. + details["focus_restored"] = False + details["interaction_restored"] = False + details["semantic_verified"] = False + return { + "ok": False, + "failure_code": "UI_CLICK_NOT_APPLIED", + "error": f"Qt click did not activate action target {text!r}.", + **details, + } + + document = self.active_document() + if document is None: + return { + "ok": False, + "failure_code": "UI_CLICK_NOT_APPLIED", + "error": f"Qt click did not activate action target {text!r}.", + **details, + "semantic_verified": False, + } + + type_ids = [str(obj["type_id"]) for obj in document["objects"]] + if text in {"PartDesign_NewBody", "New Body"}: + document["objects"].append( + { + "name": "Body", + "type_id": "PartDesign::Body", + "label": "Body", + } + ) + details["object_name"] = "PartDesign_NewBody" + return {"ok": True, **details} + + if text in { + "PartDesign_NewSketch", + "Sketcher_NewSketch", + "Create Sketch", + "Sketch", + }: + if "PartDesign::Body" not in type_ids: + return { + "ok": False, + "failure_code": "UI_CLICK_NOT_APPLIED", + "error": "Sketch has no Body to own it.", + **details, + "semantic_verified": False, + } + command_name = ( + "PartDesign_NewSketch" + if text == "PartDesign_NewSketch" + else "Sketcher_NewSketch" + ) + self.orientation_dialog = True + details["object_name"] = command_name + details["click_queued"] = True + return {"ok": True, **details} + + if text in { + "Sketcher_CreateRectangle", + "Sketcher_CompCreateRectangles", + "Rectangle", + }: + # Real sketch.edit Geometry control. activated() only starts + # DrawSketchHandlerRectangle; it does not add a closed profile. + if not self.sketch_edit: + return { + "ok": False, + "failure_code": "UI_TARGET_NOT_UNIQUE", + "error": f"Expected exactly one action named {text!r}; found 0.", + } + details["object_name"] = "Sketcher_CreateRectangle" + details["click_queued"] = True + return {"ok": True, **details} + + if text in {"Sketcher_LeaveSketch", "Leave Sketch"}: + if not self.sketch_edit: + return { + "ok": False, + "failure_code": "UI_TARGET_NOT_UNIQUE", + "error": f"Expected exactly one action named {text!r}; found 0.", + } + self.sketch_edit = False + self.selected_ribbon = "Model" + details["object_name"] = "Sketcher_LeaveSketch" + details["click_queued"] = True + return {"ok": True, **details} + + if text in {"PartDesign_Pad", "Pad"}: + # Command.cpp still registers PartDesign_Pad, but createAction() + # runs only when a command is addTo()'d. The Model ribbon and + # sketch.edit Finish group never surface it, so findChildren + # reports found 0 before and after leaving the sketch. + return { + "ok": False, + "failure_code": "UI_TARGET_NOT_UNIQUE", + "error": f"Expected exactly one action named {text!r}; found 0.", + } + + if text in {"PartDesign_DesignExtrude", "Extrude"}: + if self.sketch_edit or not self._design_extrude_active(): + return { + "ok": False, + "failure_code": "UI_TARGET_DISABLED", + "failure_stage": "precondition", + "error": f"Action {text!r} is disabled or hidden.", + "action_pick": "disabled", + "action_match_count": 1, + "command_active": False, + **details, + "semantic_verified": False, + } + if "Sketcher::SketchObject" not in type_ids: + return { + "ok": False, + "failure_code": "UI_CLICK_NOT_APPLIED", + "error": "Extrude has no sketch to consume.", + **details, + "semantic_verified": False, + } + has_closed_profile = any( + str(obj.get("type_id") or "") == "Sketcher::SketchObject" + and bool(obj.get("closed_profile")) + for obj in document["objects"] + ) + details["object_name"] = "PartDesign_DesignExtrude" + details["click_queued"] = True + if not has_closed_profile: + # Live queues the trigger, then FeatureSketchBased throws + # "Linked shape object is empty" and Command.cpp aborts + # after setEdit finds no task panel. No DesignExtrude object. + details["error"] = "Linked shape object is empty" + return {"ok": True, **details} + document["objects"].append( + { + "name": "Extrude", + "type_id": "PartDesign::DesignExtrude", + "label": "Extrude", + "shape_null": True, + "face_count": 0, + "design_task": True, + } + ) + return {"ok": True, **details} + + if text in {"Std_Export", "Export"}: + path = Path(self.export_dir) / f"{document['name']}.step" + path.write_text("ISO-10303-21; /* fake STEP from workflow harness */\n", encoding="utf-8") + self.exported_path = str(path) + details["object_name"] = "Std_Export" + details["exported_path"] = self.exported_path + return {"ok": True, **details} + + return { + "ok": False, + "failure_code": "UI_TARGET_NOT_UNIQUE", + "error": f"Expected exactly one action named {text!r}; found 0.", + } + + def _design_extrude_active(self) -> bool: + document = self.active_document() + if document is None: + return False + sketch = next( + ( + obj + for obj in document["objects"] + if obj["type_id"] == "Sketcher::SketchObject" + ), + None, + ) + if sketch is None: + return False + for item in self.selection: + if item.get("name") != sketch["name"]: + continue + sub = str(item.get("sub") or "") + if sub.startswith("Edge"): + return False + if sub.startswith("InternalFace") or sub == "": + return True + return False + + def _click_dialog(self, text: str, details: dict[str, Any]) -> dict[str, Any]: + visible_titles = [] + if self.recovery_dialog: + visible_titles.append("Document Recovery") + if self.orientation_dialog: + visible_titles.append("Choose Orientation") + if text in {"Cancel"}: + if not self.recovery_dialog: + return { + "ok": False, + "failure_code": "UI_TARGET_NOT_UNIQUE", + "error": f"Expected exactly one visible dialog named {text!r}; found 0.", + } + self.recovery_dialog = False + details["object_name"] = "Document Recovery" + return {"ok": True, **details} + if text in {"OK", "Ok"}: + # Live dialog kind treats OK as matching every visible QDialog. + if len(visible_titles) != 1: + return { + "ok": False, + "failure_code": "UI_TARGET_NOT_UNIQUE", + "error": ( + f"Expected exactly one visible dialog named {text!r}; " + f"found {len(visible_titles)}." + ), + } + if visible_titles[0] == "Document Recovery": + return { + "ok": False, + "failure_code": "UI_CLICK_NOT_APPLIED", + "error": "Document Recovery OK is Start Recovery.", + **details, + "semantic_verified": False, + } + if text not in {"OK", "Ok", "Choose Orientation"}: + return { + "ok": False, + "failure_code": "UI_TARGET_NOT_UNIQUE", + "error": f"Expected exactly one visible dialog named {text!r}; found 0.", + } + if not self.orientation_dialog: + return { + "ok": False, + "failure_code": "UI_TARGET_NOT_UNIQUE", + "error": f"Expected exactly one visible dialog named {text!r}; found 0.", + } + document = self.active_document() + if document is None: + return { + "ok": False, + "failure_code": "UI_CLICK_NOT_APPLIED", + "error": f"Qt click did not activate dialog target {text!r}.", + **details, + "semantic_verified": False, + } + document["objects"].append( + { + "name": "Sketch", + "type_id": "Sketcher::SketchObject", + "label": "Sketch", + "closed_profile": False, + "geometry_count": 0, + } + ) + self.selection = [{"name": "Sketch", "sub": ""}] + self.orientation_dialog = False + self.sketch_edit = True + self.selected_ribbon = "Sketch" + details["object_name"] = "Choose Orientation" + return {"ok": True, **details} + + +class _FakeHandler(BaseHTTPRequestHandler): + server: "FakeAgentServer" + + def log_message(self, _format: str, *_args: object) -> None: + return + + def _read_json(self) -> dict[str, Any]: + length = int(self.headers.get("Content-Length") or 0) + if length <= 0: + return {} + raw = self.rfile.read(length) + try: + payload = json.loads(raw.decode("utf-8")) + except (UnicodeDecodeError, json.JSONDecodeError): + return {} + return payload if isinstance(payload, dict) else {} + + def _write_json(self, status: int, payload: dict[str, Any]) -> None: + body = json.dumps(payload).encode("utf-8") + self.send_response(status) + self.send_header("Content-Type", "application/json") + self.send_header("Content-Length", str(len(body))) + self.end_headers() + self.wfile.write(body) + + def _authorized(self) -> bool: + expected = f"Bearer {self.server.token}" + return self.headers.get("Authorization") == expected + + def do_GET(self) -> None: # noqa: N802 + if not self._authorized(): + self._write_json(401, {"ok": False, "failure_code": "UNAUTHORIZED"}) + return + state = self.server.state + if self.path == "/v1/status": + self._write_json( + 200, + { + "ok": True, + "channel": "vibecad-agent-control", + "gui_up": True, + "documents": state.document_summaries(), + "exported_path": state.exported_path, + }, + ) + return + if self.path == "/v1/documents": + documents = state.document_summaries() + self._write_json( + 200, + { + "ok": True, + "document_count": len(documents), + "documents": documents, + }, + ) + return + if self.path == "/v1/ui/ribbon": + self._write_json( + 200, + { + "ok": True, + "selected_text": state.selected_ribbon, + "tabs": [ + { + "text": "Model", + "index": 0, + "enabled": not state.sketch_edit, + }, + {"text": "Sketch", "index": 1, "enabled": True}, + ], + }, + ) + return + if self.path == "/v1/ui/menus": + self._write_json( + 200, + { + "ok": True, + "menus": [ + { + "text": "File", + "index": 0, + "enabled": True, + "visible": True, + "menu_visible": False, + } + ], + }, + ) + return + self._write_json(404, {"ok": False, "failure_code": "NOT_FOUND"}) + + def do_POST(self) -> None: # noqa: N802 + if not self._authorized(): + self._write_json(401, {"ok": False, "failure_code": "UNAUTHORIZED"}) + return + body = self._read_json() + state = self.server.state + if self.path == "/v1/ui/click": + self._write_json(200, state.click_payload(body)) + return + if self.path == "/v1/run": + self._write_json(200, state.run_python(str(body.get("python") or ""))) + return + self._write_json(404, {"ok": False, "failure_code": "NOT_FOUND"}) + + +class FakeAgentServer(ThreadingHTTPServer): + def __init__(self, token: str, state: FakeAgentState) -> None: + super().__init__(("127.0.0.1", 0), _FakeHandler) + self.token = token + self.state = state + + +def start_fake_channel( + export_dir: str, + token: str, + *, + recovery_dialog: bool = False, +) -> tuple[FakeAgentServer, str, FakeAgentState]: + state = FakeAgentState(export_dir, recovery_dialog=recovery_dialog) + server = FakeAgentServer(token, state) + thread = threading.Thread(target=server.serve_forever, daemon=True) + thread.start() + host, port = server.server_address[:2] + return server, f"http://{host}:{port}", state diff --git a/tools/vibecad_workflow_harness.py b/tools/vibecad_workflow_harness.py new file mode 100755 index 00000000..63b1fec4 --- /dev/null +++ b/tools/vibecad_workflow_harness.py @@ -0,0 +1,478 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: LGPL-2.1-or-later +"""Run named VibeCAD button workflows through the existing loopback click path. + +The visible tour remains a demo. This harness posts the same +``/v1/ui/click`` body the tour posts, then checks documents and the model +tree after every step. Closed profiles and the export file go through the +existing ``/v1/run`` route (``SketchObject.addGeometry(Part.Circle)`` and +``Import.export``), not a second clicker. Code owns the click, the +timeout, and pass/fail. + +Use ``--fake`` when no display is available. That still speaks HTTP on +127.0.0.1. Jev is optional and off unless ``--judge`` is set and +``TYPESAFE_API_KEY`` is present. +""" + +from __future__ import annotations + +import argparse +import json +import secrets +import sys +import tempfile +import time +from pathlib import Path +from typing import Any + +import vibecad_workflow_channel as channel +import vibecad_workflow_judge as judge + + +TOOLS_DIR = Path(__file__).resolve().parent +REPO_ROOT = TOOLS_DIR.parent +DEFAULT_WORKFLOWS = TOOLS_DIR / "vibecad_workflows.json" +TOUR_SCRIPT = REPO_ROOT / "Invoke-VibeCAD-VisibleTour.ps1" +CLICK_NEVER_REACHED_CODES = frozenset( + { + "GUI_REQUIRED", + "HTTP_ERROR", + "INVALID_RESPONSE", + "MAIN_WINDOW_UNAVAILABLE", + "MENU_BAR_UNAVAILABLE", + "NOT_FOUND", + "RIBBON_TABS_UNAVAILABLE", + "UNAUTHORIZED", + "UI_PROCESS_ID_INVALID", + "UI_PROCESS_MISMATCH", + "UI_TARGET_DISABLED", + "UI_TARGET_HAS_NO_MENU", + "UI_TARGET_INDEX_INVALID", + "UI_TARGET_INDEX_MISMATCH", + "UI_TARGET_KIND_INVALID", + "UI_TARGET_NOT_TRIGGERABLE", + "UI_TARGET_NOT_UNIQUE", + "UI_TARGET_TEXT_REQUIRED", + } +) +ALLOWED_CLICK_INPUT_METHODS = frozenset( + { + "qt_in_process_mouse_click", + "qt_in_process_menu_popup", + "qt_in_process_action_trigger", + "qt_in_process_dialog_button", + } +) + + +def load_workflows(path: Path) -> list[dict[str, Any]]: + payload = json.loads(path.read_text(encoding="utf-8")) + workflows = payload.get("workflows") + if not isinstance(workflows, list) or not workflows: + raise ValueError("Workflow file has no workflows.") + return [item for item in workflows if isinstance(item, dict)] + + +def load_live_endpoint(agent_home: Path) -> tuple[str, str]: + endpoint_path = agent_home / "endpoint.json" + endpoint = json.loads(endpoint_path.read_text(encoding="utf-8")) + token_path = Path(str(endpoint.get("token_path") or agent_home / "token")) + token = token_path.read_text(encoding="utf-8").strip() + return str(endpoint.get("base_url") or ""), token + + +def click_reached_target(payload: dict[str, Any]) -> bool: + return bool( + payload.get("ok") + or payload.get("click_queued") + or payload.get("object_name") + or payload.get("input_method") + or payload.get("semantic_verified") + or str(payload.get("failure_code") or "") == "UI_CLICK_NOT_APPLIED" + ) + + +def click_accepted(payload: dict[str, Any]) -> tuple[bool, str]: + """Reject clicks that never reached a target. Checks own pass/fail. + + Creating a document moves Qt focus. The live agent may then return + ``UI_CLICK_NOT_APPLIED`` with ``focus_restored`` false even though + ``GET /v1/documents`` shows an active document. Restoration fields + are evidence, not a harness hard-fail. + """ + + if str(payload.get("physical_cursor_control") or "none") != "none": + return False, "click used physical cursor control" + input_method = str(payload.get("input_method") or "") + if input_method and input_method not in ALLOWED_CLICK_INPUT_METHODS: + return False, f"unsupported click input_method {input_method!r}" + failure_code = str(payload.get("failure_code") or "") + if failure_code in CLICK_NEVER_REACHED_CODES: + return False, str(payload.get("error") or failure_code or "click failed") + if payload.get("ok") or click_reached_target(payload): + return True, "" + return False, str(payload.get("error") or failure_code or "click failed") + + +def evaluate_check( + check: dict[str, Any], + *, + click_payload: dict[str, Any], + documents_payload: dict[str, Any], + tree_payload: dict[str, Any], +) -> list[str]: + errors: list[str] = [] + documents = documents_payload.get("documents") + if not isinstance(documents, list): + documents = [] + tree = tree_payload.get("result") if isinstance(tree_payload.get("result"), dict) else {} + objects = tree.get("objects") if isinstance(tree.get("objects"), list) else [] + type_ids = {str(item.get("type_id") or "") for item in objects if isinstance(item, dict)} + + if check.get("click_ok") and not click_payload.get("ok"): + errors.append("click_ok expected a successful click") + minimum_docs = check.get("document_count_min") + if minimum_docs is not None and len(documents) < int(minimum_docs): + errors.append( + f"document_count_min {minimum_docs} failed; found {len(documents)}" + ) + if check.get("active_document"): + if not any(item.get("active") for item in documents if isinstance(item, dict)): + errors.append("no active document after click") + if tree.get("document") in {None, ""}: + errors.append("tree inspect reported no active document") + required_types = check.get("tree_type_ids") + if isinstance(required_types, list): + missing = [item for item in required_types if item not in type_ids] + if missing: + errors.append(f"tree missing type ids {missing}; have {sorted(type_ids)}") + minimum_geometry = check.get("sketch_geometry_min") + if minimum_geometry is not None: + geometry_counts = [ + int(item.get("geometry_count") or 0) + for item in objects + if isinstance(item, dict) + and str(item.get("type_id") or "") == "Sketcher::SketchObject" + ] + have = max(geometry_counts) if geometry_counts else 0 + if have < int(minimum_geometry): + errors.append( + f"sketch_geometry_min {minimum_geometry} failed; found {have}" + ) + run_result = ( + click_payload.get("result") + if isinstance(click_payload.get("result"), dict) + else {} + ) + if check.get("run_ok") and not click_payload.get("ok"): + errors.append("run_ok expected POST /v1/run to succeed") + required_command = check.get("command_active") + if required_command: + if not bool(run_result.get("command_active")): + errors.append( + f"command_active {required_command} failed; " + "PartDesign_DesignExtrude is not enabled" + ) + if check.get("exported"): + exported = str( + run_result.get("exported_path") + or click_payload.get("exported_path") + or tree_payload.get("exported_path") + or "" + ) + path = Path(exported) if exported else None + if path is not None and path.is_file(): + size = path.stat().st_size + minimum = int(check.get("export_bytes_min") or 1) + if size < minimum: + errors.append(f"export file is {size} bytes; expected at least {minimum}") + else: + triggered = str( + click_payload.get("object_name") or click_payload.get("target_text") or "" + ) + if triggered not in {"Std_Export", "Export"}: + errors.append("export did not produce a file") + return errors + + +def run_step( + step: dict[str, Any], + client: channel.AgentClickChannel, + *, + timeout_seconds: float, + judge_enabled: bool, + judge_transport: Any = None, + export_path: str = "", +) -> dict[str, Any]: + click_spec = dict(step.get("click") or {}) + run_spec = dict(step.get("run") or {}) + started = time.monotonic() + if run_spec: + recipe = str(run_spec.get("id") or "") + click_payload = client.run( + channel.workflow_run_python(recipe, export_path=export_path) + ) + accepted = bool(click_payload.get("ok")) + click_error = str( + click_payload.get("error") + or click_payload.get("failure_code") + or "POST /v1/run failed" + ) + else: + click_payload = client.click( + str(click_spec.get("kind") or ""), + str(click_spec.get("text") or ""), + expected_process_id=click_spec.get("expected_process_id"), + expected_index=click_spec.get("expected_index"), + ) + accepted, click_error = click_accepted(click_payload) + documents_payload = client.documents() + tree_payload = client.inspect_tree() + elapsed = time.monotonic() - started + errors = [] if accepted else [click_error] + if elapsed > timeout_seconds: + errors.append(f"step exceeded timeout of {timeout_seconds:g}s") + errors.extend( + evaluate_check( + dict(step.get("check") or {}), + click_payload=click_payload, + documents_payload=documents_payload, + tree_payload=tree_payload, + ) + ) + code_passed = not errors + judge_state = { + "workflow_step": step.get("id"), + "click": click_spec or run_spec, + "click_response": { + key: click_payload.get(key) + for key in ( + "ok", + "failure_code", + "target_kind", + "target_text", + "object_name", + "semantic_verified", + "input_method", + "result", + ) + }, + "documents": documents_payload.get("documents"), + "tree": tree_payload.get("result"), + "code_errors": list(errors), + "code_passed": code_passed, + } + judged = judge.judge_step( + judge_state, + enabled=judge_enabled, + transport=judge_transport, + ) + # Code owns pass/fail. A judge can never turn a failed step into a pass. + # Low confidence also never counts as a pass. + passed = code_passed + return { + "id": step.get("id"), + "click": click_spec or None, + "run": run_spec or None, + "elapsed_s": round(elapsed, 3), + "click_response": click_payload, + "documents": documents_payload, + "tree": tree_payload.get("result"), + "errors": errors, + "judge": judged, + "passed": passed, + } + + +def run_workflow( + workflow: dict[str, Any], + client: channel.AgentClickChannel, + *, + timeout_seconds: float, + judge_enabled: bool, + judge_transport: Any = None, + export_path: str = "", +) -> dict[str, Any]: + steps = [] + passed = True + for step in workflow.get("steps") or []: + if not isinstance(step, dict): + continue + result = run_step( + step, + client, + timeout_seconds=timeout_seconds, + judge_enabled=judge_enabled, + judge_transport=judge_transport, + export_path=export_path, + ) + steps.append(result) + if not result["passed"]: + passed = False + break + return { + "id": workflow.get("id"), + "title": workflow.get("title"), + "passed": passed, + "steps": steps, + } + + +def run_harness( + *, + workflows: list[dict[str, Any]], + client: channel.AgentClickChannel, + timeout_seconds: float, + judge_enabled: bool, + judge_transport: Any = None, + export_path: str = "", +) -> dict[str, Any]: + preflight_steps = [ + { + "id": "dismiss_document_recovery", + "run": {"id": "dismiss_document_recovery"}, + "check": {"run_ok": True}, + }, + { + "id": "leave_leftover_sketch", + "run": {"id": "leave_active_sketch"}, + "check": {"run_ok": True}, + }, + ] + preflight = [] + preflight_passed = True + for step in preflight_steps: + result = run_step( + step, + client, + timeout_seconds=timeout_seconds, + judge_enabled=judge_enabled, + judge_transport=judge_transport, + export_path=export_path, + ) + preflight.append(result) + if not result["passed"]: + preflight_passed = False + break + results = [] + if preflight_passed: + results = [ + run_workflow( + workflow, + client, + timeout_seconds=timeout_seconds, + judge_enabled=judge_enabled, + judge_transport=judge_transport, + export_path=export_path, + ) + for workflow in workflows + ] + return { + "schema": "vibecad.workflow-harness-report.v1", + "click_route": "/v1/ui/click", + "tour_remains_demo": TOUR_SCRIPT.is_file(), + "judge_requested": bool(judge_enabled), + "passed": bool(preflight_passed and results and all(item["passed"] for item in results)), + "preflight": preflight, + "workflows": results, + } + + +def build_parser() -> argparse.ArgumentParser: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument( + "--workflows", + type=Path, + default=DEFAULT_WORKFLOWS, + help="Workflow definition JSON (default: tools/vibecad_workflows.json).", + ) + parser.add_argument( + "--workflow", + action="append", + dest="workflow_ids", + help="Run only this workflow id. Repeatable.", + ) + parser.add_argument( + "--fake", + action="store_true", + help="Use the fake loopback click channel (no display required).", + ) + parser.add_argument( + "--agent-home", + type=Path, + default=REPO_ROOT / ".vibecad-dev" / "agent", + help="Checkout-scoped agent home with endpoint.json and token.", + ) + parser.add_argument("--base-url", help="Override the loopback agent URL.") + parser.add_argument("--token", help="Override the bearer token.") + parser.add_argument("--timeout", type=float, default=30.0) + parser.add_argument( + "--judge", + action="store_true", + help="Ask Jev after each step. Still skipped without TYPESAFE_API_KEY.", + ) + parser.add_argument("--output", type=Path, help="Write the JSON report here.") + return parser + + +def main(argv: list[str] | None = None) -> int: + args = build_parser().parse_args(argv) + workflows = load_workflows(args.workflows) + if args.workflow_ids: + wanted = set(args.workflow_ids) + workflows = [item for item in workflows if item.get("id") in wanted] + if not workflows: + raise SystemExit(f"No workflows matched {sorted(wanted)}.") + + server = None + export_home = None + export_path = str(Path(tempfile.gettempdir()) / "vibecad-workflow-harness.step") + if args.fake: + if args.output is not None: + export_dir = args.output.parent + export_dir.mkdir(parents=True, exist_ok=True) + else: + export_home = tempfile.TemporaryDirectory(prefix="vibecad-workflow-export-") + export_dir = Path(export_home.name) + export_path = str(export_dir / "vibecad-workflow-harness.step") + token = args.token or secrets.token_hex(24) + server, base_url, _state = channel.start_fake_channel(str(export_dir), token) + client = channel.AgentClickChannel(base_url, token, timeout_seconds=args.timeout) + else: + base_url = args.base_url + token = args.token + if not base_url or not token: + live_url, live_token = load_live_endpoint(args.agent_home) + base_url = base_url or live_url + token = token or live_token + client = channel.AgentClickChannel( + str(base_url), + str(token), + timeout_seconds=args.timeout, + ) + + try: + report = run_harness( + workflows=workflows, + client=client, + timeout_seconds=args.timeout, + judge_enabled=args.judge, + export_path=export_path, + ) + finally: + if server is not None: + server.shutdown() + server.server_close() + if export_home is not None: + export_home.cleanup() + + text = json.dumps(report, indent=2, sort_keys=True) + if args.output: + args.output.parent.mkdir(parents=True, exist_ok=True) + args.output.write_text(text + "\n", encoding="utf-8") + print(text) + return 0 if report["passed"] else 1 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tools/vibecad_workflow_harness_selftest.py b/tools/vibecad_workflow_harness_selftest.py new file mode 100755 index 00000000..4785d509 --- /dev/null +++ b/tools/vibecad_workflow_harness_selftest.py @@ -0,0 +1,483 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: LGPL-2.1-or-later +"""Self-test the workflow harness against the fake loopback click channel.""" + +from __future__ import annotations + +import json +import os +from pathlib import Path +import secrets +import tempfile +from typing import Any + +import vibecad_workflow_channel as channel +import vibecad_workflow_harness as harness +import vibecad_workflow_judge as judge + + +TOOLS_DIR = Path(__file__).resolve().parent +REPO_ROOT = TOOLS_DIR.parent +TOUR_SCRIPT = REPO_ROOT / "Invoke-VibeCAD-VisibleTour.ps1" + + +def scenario(name: str, passed: bool, details: dict[str, Any]) -> dict[str, Any]: + return {"name": name, "result": "pass" if passed else "fail", "details": details} + + +def _run_fake( + workflows: list[dict[str, Any]], **kwargs: Any +) -> tuple[dict[str, Any], channel.FakeAgentState, str]: + with tempfile.TemporaryDirectory(prefix="vibecad-workflow-harness-") as temp: + token = secrets.token_hex(24) + server, base_url, state = channel.start_fake_channel(temp, token) + try: + client = channel.AgentClickChannel(base_url, token, timeout_seconds=5) + report = harness.run_harness( + workflows=workflows, + client=client, + timeout_seconds=5, + judge_enabled=kwargs.get("judge_enabled", False), + judge_transport=kwargs.get("judge_transport"), + export_path=str(Path(temp) / "vibecad-workflow-harness.step"), + ) + return report, state, base_url + finally: + server.shutdown() + server.server_close() + + +def main() -> int: + os.environ.pop("TYPESAFE_API_KEY", None) + workflows = harness.load_workflows(TOOLS_DIR / "vibecad_workflows.json") + scenarios = [] + + report, _state, _base_url = _run_fake(workflows) + workflow_ids = [item["id"] for item in report["workflows"]] + new_step = report["workflows"][0]["steps"][0] + new_click = new_step.get("click_response") or {} + sketch_clicks = [ + str((step.get("click") or {}).get("text") or "") + for item in report["workflows"] + if item["id"] == "sketch_then_pad" + for step in item["steps"] + ] + workflow_source = (TOOLS_DIR / "vibecad_workflows.json").read_text(encoding="utf-8") + scenarios.append( + scenario( + "three_workflows_pass_without_typesafe_key", + report["passed"] is True + and workflow_ids == ["new_document", "sketch_then_pad", "export"] + and all(item["passed"] for item in report["workflows"]) + and all( + step.get("judge", {}).get("called") is False + for item in report["workflows"] + for step in item["steps"] + ), + { + "passed": report["passed"], + "workflow_ids": workflow_ids, + }, + ) + ) + scenarios.append( + scenario( + "workflows_use_visible_ribbon_and_part_design_commands", + '"kind": "menu"' not in workflow_source + and "PartDesign_NewBody" in sketch_clicks + and "Sketcher_NewSketch" in sketch_clicks + and "OK" in sketch_clicks + and "leave_active_sketch" in workflow_source + and "PartDesign_DesignExtrude" in sketch_clicks + and "PartDesign_Pad" not in sketch_clicks, + {"sketch_clicks": sketch_clicks}, + ) + ) + scenarios.append( + scenario( + "new_document_lands_when_focus_moves", + new_step.get("passed") is True + and new_click.get("ok") is False + and new_click.get("failure_code") == "UI_CLICK_NOT_APPLIED" + and new_click.get("focus_restored") is False + and any( + isinstance(item, dict) and item.get("active") + for item in (new_step.get("documents") or {}).get("documents") or [] + ), + { + "errors": new_step.get("errors"), + "failure_code": new_click.get("failure_code"), + "focus_restored": new_click.get("focus_restored"), + }, + ) + ) + + tour_source = TOUR_SCRIPT.read_text(encoding="utf-8") + harness_source = (TOOLS_DIR / "vibecad_workflow_harness.py").read_text(encoding="utf-8") + channel_source = (TOOLS_DIR / "vibecad_workflow_channel.py").read_text(encoding="utf-8") + scenarios.append( + scenario( + "reuses_tour_click_route_and_leaves_tour_as_demo", + "/v1/ui/click" in tour_source + and "Invoke-VibeCADAgentPost" in tour_source + and report["click_route"] == "/v1/ui/click" + and "tour_remains_demo" in harness_source + and "pyautogui" not in channel_source + and "SetCursorPos" not in channel_source + and "SendInput" not in channel_source, + {"click_route": report["click_route"]}, + ) + ) + + failing = json.loads(json.dumps(workflows)) + failing[0]["steps"][-1]["check"]["document_count_min"] = 9 + failed_report, _state, _base_url = _run_fake(failing[:1]) + scenarios.append( + scenario( + "failed_tree_or_document_check_fails_the_workflow", + failed_report["passed"] is False + and failed_report["workflows"][0]["passed"] is False + and any( + "document_count_min" in error + for error in failed_report["workflows"][0]["steps"][-1]["errors"] + ), + {"errors": failed_report["workflows"][0]["steps"][-1]["errors"]}, + ) + ) + + refused = False + try: + channel.require_loopback_url("https://example.invalid/v1") + except ValueError: + refused = True + scenarios.append( + scenario( + "refuses_non_loopback_endpoints", + refused, + {}, + ) + ) + + skipped = judge.judge_step({"code_passed": True}, enabled=True) + disabled = judge.judge_step({"code_passed": True}, enabled=False) + low_confidence = judge.interpret_judge( + { + "model": "jev-latest", + "answers": { + "landed": {"type": "noul", "noul": 0.91}, + "failure_class": { + "type": "choice", + "choice": "none", + "confidence": 0.12, + }, + }, + } + ) + high_confidence = judge.interpret_judge( + { + "model": "jev-latest", + "answers": { + "landed": {"type": "noul", "noul": 0.91}, + "failure_class": { + "type": "choice", + "choice": "none", + "confidence": 0.88, + }, + }, + } + ) + scenarios.append( + scenario( + "judge_skips_without_key_and_low_confidence_is_not_pass", + skipped.get("skipped") == "typesafe_key_absent" + and disabled.get("skipped") == "judge_disabled" + and low_confidence["counts_as_pass"] is False + and high_confidence["counts_as_pass"] is True, + { + "skipped": skipped, + "low_confidence": low_confidence, + "high_confidence": high_confidence, + }, + ) + ) + + judged_report, _state, _base_url = _run_fake( + workflows[:1], + judge_enabled=True, + judge_transport=lambda _state, _key: { + "model": "mock-jev", + "answers": { + "landed": {"type": "noul", "noul": 0.8}, + "failure_class": { + "type": "choice", + "choice": "none", + "confidence": 0.9, + }, + }, + }, + ) + first_judge = judged_report["workflows"][0]["steps"][0]["judge"] + scenarios.append( + scenario( + "optional_judge_can_use_a_mock_transport", + judged_report["passed"] is True + and first_judge.get("called") is True + and first_judge.get("counts_as_pass") is True, + {"judge": first_judge}, + ) + ) + + sketch_workflow = next( + item for item in report["workflows"] if item["id"] == "sketch_then_pad" + ) + export_workflow = next( + item for item in report["workflows"] if item["id"] == "export" + ) + sketch_step_ids = [str(step.get("id") or "") for step in sketch_workflow["steps"]] + extrude_step = next( + step for step in sketch_workflow["steps"] if step.get("id") == "click_extrude" + ) + accept_step = next( + step + for step in sketch_workflow["steps"] + if step.get("id") == "accept_design_task" + ) + place_step = next( + step + for step in sketch_workflow["steps"] + if step.get("id") == "place_closed_profile" + ) + export_step = next( + step for step in export_workflow["steps"] if step.get("id") == "export_step" + ) + extrude_type_ids = { + str(item.get("type_id") or "") + for item in (extrude_step.get("tree") or {}).get("objects") or [] + if isinstance(item, dict) + } + accept_type_ids = { + str(item.get("type_id") or "") + for item in (accept_step.get("tree") or {}).get("objects") or [] + if isinstance(item, dict) + } + export_result = (export_step.get("click_response") or {}).get("result") or {} + exported_path = str(export_result.get("exported_path") or "") + export_bytes = int(export_result.get("bytes") or 0) + pad_while_editing = None + model_while_editing = None + rectangle_while_editing = None + with tempfile.TemporaryDirectory(prefix="vibecad-workflow-harness-") as temp: + token = secrets.token_hex(24) + server, base_url, _state = channel.start_fake_channel(temp, token) + try: + client = channel.AgentClickChannel(base_url, token, timeout_seconds=5) + client.click("action", "Std_New") + client.click("ribbon", "Model") + client.click("action", "PartDesign_NewBody") + client.click("action", "Sketcher_NewSketch") + client.click("dialog", "OK") + pad_while_editing = client.click("action", "PartDesign_Pad") + model_while_editing = client.click("ribbon", "Model") + rectangle_while_editing = client.click("action", "Sketcher_CreateRectangle") + after_rectangle = client.inspect_tree() + leave = client.click("action", "Sketcher_LeaveSketch") + empty_extrude = client.click("action", "PartDesign_DesignExtrude") + empty_tree = client.inspect_tree() + placed = client.run(channel.workflow_run_python("place_closed_circle")) + disabled_after_place = client.click("action", "PartDesign_DesignExtrude") + selected = client.run(channel.workflow_run_python("select_sketch")) + filled_extrude = client.click("action", "PartDesign_DesignExtrude") + filled_tree = client.inspect_tree() + export_before_accept = client.run( + channel.workflow_run_python( + "export_step", + export_path=str(Path(temp) / "before-accept.step"), + ) + ) + accepted_task = client.run( + channel.workflow_run_python("accept_design_task") + ) + after_accept = client.inspect_tree() + export_after_accept = client.run( + channel.workflow_run_python( + "export_step", + export_path=str(Path(temp) / "after-accept.step"), + ) + ) + finally: + server.shutdown() + server.server_close() + after_rectangle_objects = ( + (after_rectangle.get("result") or {}).get("objects") or [] + ) + empty_type_ids = { + str(item.get("type_id") or "") + for item in ((empty_tree.get("result") or {}).get("objects") or []) + if isinstance(item, dict) + } + filled_type_ids = { + str(item.get("type_id") or "") + for item in ((filled_tree.get("result") or {}).get("objects") or []) + if isinstance(item, dict) + } + after_accept_type_ids = { + str(item.get("type_id") or "") + for item in ((after_accept.get("result") or {}).get("objects") or []) + if isinstance(item, dict) + } + rectangle_added_profile = any( + isinstance(item, dict) and item.get("closed_profile") + for item in after_rectangle_objects + ) + scenarios.append( + scenario( + "closed_profile_then_extrude_creates_solid_and_export_writes_file", + sketch_step_ids + == [ + "select_model_ribbon", + "click_body", + "click_sketch", + "accept_sketch_orientation", + "leave_sketch", + "place_closed_profile", + "select_sketch", + "click_extrude", + "accept_design_task", + ] + and place_step.get("passed") is True + and extrude_step.get("passed") is True + and accept_step.get("passed") is True + and "PartDesign::DesignExtrude" in extrude_type_ids + and "PartDesign::DesignBodyPublication" not in extrude_type_ids + and "PartDesign::DesignBodyPublication" in accept_type_ids + and export_step.get("passed") is True + and exported_path.endswith(".step") + and export_bytes >= 1 + and export_result.get("type_id") == "PartDesign::DesignBodyPublication" + and int(export_result.get("face_count") or 0) >= 1 + and "place_closed_circle" in channel_source + and "leave_active_sketch" in channel_source + and "leaveActiveSketch" in channel_source + and "accept_design_task" in channel_source + and "QDialogButtonBox" in channel_source + and "Part.Circle" in channel_source + and "addGeometry" in channel_source + and "Import.export" in channel_source + and "exportStep" in channel_source, + { + "sketch_step_ids": sketch_step_ids, + "extrude_type_ids": sorted(extrude_type_ids), + "accept_type_ids": sorted(accept_type_ids), + "exported_path": exported_path, + "export_bytes": export_bytes, + "export_type_id": export_result.get("type_id"), + }, + ) + ) + scenarios.append( + scenario( + "empty_sketch_and_rectangle_handler_do_not_create_a_solid", + pad_while_editing.get("failure_code") == "UI_TARGET_NOT_UNIQUE" + and "found 0" in str(pad_while_editing.get("error") or "") + and model_while_editing.get("failure_code") == "UI_TARGET_DISABLED" + and rectangle_while_editing.get("ok") is True + and rectangle_while_editing.get("object_name") == "Sketcher_CreateRectangle" + and rectangle_added_profile is False + and leave.get("ok") is True + and leave.get("object_name") == "Sketcher_LeaveSketch" + and empty_extrude.get("ok") is True + and empty_extrude.get("error") == "Linked shape object is empty" + and "PartDesign::DesignExtrude" not in empty_type_ids + and "Sketcher::SketchObject" in empty_type_ids + and placed.get("ok") is True + and int((placed.get("result") or {}).get("geometry_count") or 0) >= 1 + and disabled_after_place.get("failure_code") == "UI_TARGET_DISABLED" + and selected.get("ok") is True + and (selected.get("result") or {}).get("sub") == "InternalFace1" + and (selected.get("result") or {}).get("command_active") is True + and filled_extrude.get("ok") is True + and not filled_extrude.get("error") + and "PartDesign::DesignExtrude" in filled_type_ids + and "PartDesign::DesignBodyPublication" not in filled_type_ids + and export_before_accept.get("ok") is False + and "Faces" in str(export_before_accept.get("error") or "") + and accepted_task.get("ok") is True + and (accepted_task.get("result") or {}).get("type_id") + == "PartDesign::DesignBodyPublication" + and "PartDesign::DesignBodyPublication" in after_accept_type_ids + and export_after_accept.get("ok") is True + and (export_after_accept.get("result") or {}).get("type_id") + == "PartDesign::DesignBodyPublication" + and int((export_after_accept.get("result") or {}).get("face_count") or 0) + >= 1, + { + "pad_while_editing": pad_while_editing.get("failure_code"), + "model_while_editing": model_while_editing.get("failure_code"), + "rectangle_added_profile": rectangle_added_profile, + "empty_extrude_error": empty_extrude.get("error"), + "disabled_after_place": disabled_after_place.get("failure_code"), + "selected_sub": (selected.get("result") or {}).get("sub"), + "empty_type_ids": sorted(empty_type_ids), + "filled_type_ids": sorted(filled_type_ids), + "export_before_accept": export_before_accept.get("error"), + "after_accept_type_ids": sorted(after_accept_type_ids), + "export_after_type": (export_after_accept.get("result") or {}).get( + "type_id" + ), + }, + ) + ) + + recovery_ok = None + dismissed = None + with tempfile.TemporaryDirectory(prefix="vibecad-workflow-harness-") as temp: + token = secrets.token_hex(24) + server, base_url, _state = channel.start_fake_channel( + temp, token, recovery_dialog=True + ) + try: + client = channel.AgentClickChannel(base_url, token, timeout_seconds=5) + client.click("action", "Std_New") + client.click("ribbon", "Model") + client.click("action", "PartDesign_NewBody") + client.click("action", "Sketcher_NewSketch") + recovery_ok = client.click("dialog", "OK") + dismissed = client.run( + channel.workflow_run_python("dismiss_document_recovery") + ) + after_cancel = client.click("dialog", "OK") + finally: + server.shutdown() + server.server_close() + scenarios.append( + scenario( + "document_recovery_is_dismissed_with_cancel_not_start_recovery", + recovery_ok.get("failure_code") == "UI_TARGET_NOT_UNIQUE" + and "found 2" in str(recovery_ok.get("error") or "") + and dismissed.get("ok") is True + and (dismissed.get("result") or {}).get("button") == "Cancel" + and (dismissed.get("result") or {}).get("dismissed") == ["Cancel"] + and after_cancel.get("ok") is True + and after_cancel.get("object_name") == "Choose Orientation", + { + "recovery_ok": recovery_ok.get("failure_code"), + "dismissed": dismissed.get("result"), + "after_cancel": after_cancel.get("object_name"), + }, + ) + ) + + failed = [item for item in scenarios if item["result"] != "pass"] + payload = { + "schema": "vibecad-workflow-harness-selftest-v1", + "ok": not failed, + "scenario_count": len(scenarios), + "failed_scenarios": [item["name"] for item in failed], + "scenarios": scenarios, + } + print(json.dumps(payload, indent=2, sort_keys=True)) + return 0 if payload["ok"] else 1 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tools/vibecad_workflow_judge.py b/tools/vibecad_workflow_judge.py new file mode 100644 index 00000000..0a38be01 --- /dev/null +++ b/tools/vibecad_workflow_judge.py @@ -0,0 +1,160 @@ +# SPDX-License-Identifier: LGPL-2.1-or-later +"""Optional TypeSafe/Jev step judge for the VibeCAD workflow harness. + +Default off. A live System One call happens only when the caller asks for a +judge and ``TYPESAFE_API_KEY`` is set. The harness still owns click, timeout, +and pass/fail. Low confidence never counts as a pass. This module never +prints or writes the API key. +""" + +from __future__ import annotations + +import json +import os +from typing import Any +from urllib import error, request + + +TYPESAFE_URL = "https://api.typesafe.ai/v1/systemone" +TYPESAFE_MODEL = "jev-latest" +JUDGE_CONFIDENCE_FLOOR = 0.5 +FAILURE_CLASSES = ( + "none", + "geometry_bug", + "wrong_button", + "model_claim", + "flake", +) + + +def judge_requested(enabled: bool) -> bool: + return bool(enabled) and bool(str(os.environ.get("TYPESAFE_API_KEY") or "").strip()) + + +def _system_one_request(state: dict[str, Any], api_key: str) -> dict[str, Any]: + payload = { + "state": state, + "model": TYPESAFE_MODEL, + "questions": { + "landed": { + "type": "noul", + "instructions": ( + "Did this VibeCAD workflow step land? The click targeted " + "the named control, post-click UI state matches the step, " + "and the document tree shows the claimed objects." + ), + }, + "failure_class": { + "type": "choice", + "instructions": ( + "If the step failed, classify the failure. Use none when " + "the step landed." + ), + "criteria": { + "none": "The step landed or no failure classification is needed.", + "geometry_bug": "The geometry or CAD kernel result is wrong.", + "wrong_button": "The click named or hit the wrong control.", + "model_claim": ( + "Status or the model claims work the document tree " + "does not show." + ), + "flake": "A timing, focus, or transient UI flake.", + }, + }, + }, + } + http_request = request.Request( + TYPESAFE_URL, + data=json.dumps(payload).encode("utf-8"), + headers={ + "Accept": "application/json", + "Authorization": f"Bearer {api_key}", + "Content-Type": "application/json", + }, + method="POST", + ) + response = request.urlopen(http_request, timeout=30.0) + try: + body = json.loads(response.read().decode("utf-8")) + finally: + response.close() + if not isinstance(body, dict): + raise RuntimeError("TypeSafe did not return a JSON object.") + return body + + +def interpret_judge(response: dict[str, Any]) -> dict[str, Any]: + answers = response.get("answers") if isinstance(response, dict) else None + if not isinstance(answers, dict): + return { + "called": True, + "landed": False, + "failure_class": "none", + "confidence": 0.0, + "counts_as_pass": False, + "error": "TypeSafe response had no answers.", + } + landed_answer = answers.get("landed") if isinstance(answers.get("landed"), dict) else {} + class_answer = ( + answers.get("failure_class") + if isinstance(answers.get("failure_class"), dict) + else {} + ) + noul = landed_answer.get("noul") + landed = isinstance(noul, (int, float)) and float(noul) >= 0.5 + failure_class = str(class_answer.get("choice") or "none") + if failure_class not in FAILURE_CLASSES: + failure_class = "none" + confidence = class_answer.get("confidence") + try: + confidence_value = float(confidence) + except (TypeError, ValueError): + confidence_value = 0.0 + # Low confidence does not count as pass, even if noul looks positive. + counts_as_pass = bool(landed and confidence_value >= JUDGE_CONFIDENCE_FLOOR) + return { + "called": True, + "landed": landed, + "noul": None if not isinstance(noul, (int, float)) else float(noul), + "failure_class": failure_class, + "confidence": confidence_value, + "counts_as_pass": counts_as_pass, + "model": str(response.get("model") or TYPESAFE_MODEL), + } + + +def judge_step( + state: dict[str, Any], + *, + enabled: bool, + transport: Any = None, +) -> dict[str, Any]: + if not enabled: + return {"called": False, "skipped": "judge_disabled"} + api_key = str(os.environ.get("TYPESAFE_API_KEY") or "").strip() + # A provided transport is a test/mock. Live HTTP still requires a key. + if transport is None and not api_key: + return {"called": False, "skipped": "typesafe_key_absent"} + try: + if transport is None: + response = _system_one_request(state, api_key) + else: + response = transport(state, api_key) + return interpret_judge(response) + except ( + error.URLError, + TimeoutError, + ConnectionError, + OSError, + UnicodeDecodeError, + json.JSONDecodeError, + RuntimeError, + ) as exc: + return { + "called": True, + "landed": False, + "failure_class": "none", + "confidence": 0.0, + "counts_as_pass": False, + "error": str(exc), + } diff --git a/tools/vibecad_workflows.json b/tools/vibecad_workflows.json new file mode 100644 index 00000000..6c5b643f --- /dev/null +++ b/tools/vibecad_workflows.json @@ -0,0 +1,150 @@ +{ + "schema": "vibecad.workflow-harness.v1", + "workflows": [ + { + "id": "new_document", + "title": "New document", + "steps": [ + { + "id": "click_new", + "click": { + "kind": "action", + "text": "Std_New" + }, + "check": { + "document_count_min": 1, + "active_document": true + } + } + ] + }, + { + "id": "sketch_then_pad", + "title": "Sketch then pad", + "steps": [ + { + "id": "select_model_ribbon", + "click": { + "kind": "ribbon", + "text": "Model" + }, + "check": { + "click_ok": true + } + }, + { + "id": "click_body", + "click": { + "kind": "action", + "text": "PartDesign_NewBody" + }, + "check": { + "tree_type_ids": [ + "PartDesign::Body" + ] + } + }, + { + "id": "click_sketch", + "click": { + "kind": "action", + "text": "Sketcher_NewSketch" + }, + "check": { + "click_ok": true + } + }, + { + "id": "accept_sketch_orientation", + "click": { + "kind": "dialog", + "text": "OK" + }, + "check": { + "tree_type_ids": [ + "PartDesign::Body", + "Sketcher::SketchObject" + ] + } + }, + { + "id": "leave_sketch", + "run": { + "id": "leave_active_sketch" + }, + "check": { + "run_ok": true, + "tree_type_ids": [ + "PartDesign::Body", + "Sketcher::SketchObject" + ] + } + }, + { + "id": "place_closed_profile", + "run": { + "id": "place_closed_circle" + }, + "check": { + "sketch_geometry_min": 1 + } + }, + { + "id": "select_sketch", + "run": { + "id": "select_sketch" + }, + "check": { + "run_ok": true, + "command_active": "PartDesign_DesignExtrude" + } + }, + { + "id": "click_extrude", + "click": { + "kind": "action", + "text": "PartDesign_DesignExtrude" + }, + "check": { + "tree_type_ids": [ + "PartDesign::Body", + "Sketcher::SketchObject", + "PartDesign::DesignExtrude" + ] + } + }, + { + "id": "accept_design_task", + "run": { + "id": "accept_design_task" + }, + "check": { + "run_ok": true, + "tree_type_ids": [ + "PartDesign::Body", + "Sketcher::SketchObject", + "PartDesign::DesignExtrude", + "PartDesign::DesignBodyPublication" + ] + } + } + ] + }, + { + "id": "export", + "title": "Export", + "steps": [ + { + "id": "export_step", + "run": { + "id": "export_step" + }, + "check": { + "exported": true, + "export_bytes_min": 1 + } + } + ] + } + ] +} diff --git a/version.json b/version.json index 8542a906..5fc9778d 100644 --- a/version.json +++ b/version.json @@ -6,6 +6,6 @@ "version_patch_note": "number of patch release (e.g. 4 for the 0.18.4 release)", "version_suffix": "RC6", "version_suffix_note": "either 'dev' for development snapshot, 'RC1' etc. for release candidate, or empty string for release", - "build_version": 5, + "build_version": 6, "build_version_note": "used when the same VibeCAD version is re-released (for example using an updated LibPack)" }