Look at a window before calling it done: screenshot actions and Qt warnings, unseen-window check, steadier repeats, project-Python checks

CHANGELOG.md

@@ -6,6 +6,36 @@ All notable changes to lcode are documented here. The format follows
 
 ## [Unreleased]
 
+### Added
+
+- The screenshot tool can click first. For a Python Qt program, `actions` run before the picture,
+  the way a user would: `double_click Camera 1` (a widget by its text) or `double_click 300,200`,
+  `click`, `right_click`, `key Escape`, `key ctrl+s`, `type …`, `wait 2`. The result says what each
+  step hit. Bugs that only show after a click were out of reach before.
+- Qt's warnings are listed at the top of a screenshot's report, taken from everything the program
+  printed. In a real run the cause (`setLayout: … which already has a layout`) was there, among 30
+  lines of the program's logs, and the model looked past it.
+
+### Fixed
+
+From a real session that "fixed" a double-click feature four times without once looking at the
+window, while the user saw a blank screen each time:
+
+- **Windows it never looked at** are sent back once before the answer counts. When a request
+  changed a Python file that draws a window (PyQt, PySide, tkinter, Kivy, wx, GTK) and the model
+  never took a screenshot or looked at an image, it's told to look (with clicks first, if that's
+  where the change shows) or to say plainly that it's unchecked. The real cause was in what Qt
+  printed: `setLayout: … which already has a layout`.
+- **A rerun that only took a different time** counts as a repeat: the model ran the same tests four
+  times in a row (`75 passed in 0.14s`, then `0.12s`…) without the repeat check noticing.
+- **A reply lcode drops** for going round in circles is saved, and the message says where, so you
+  can read what the model was thinking instead of taking the stop on trust.
+- **The checks use the project's own Python.** They ran `python -m …` from PATH, which doesn't have
+  the project's dependencies, so a project with a `.venv` "failed" its checks on imports. Now
+  `.venv/bin/python` (or `venv`, `env`) runs them.
+- **Tests written for pytest get pytest** when the project has no pytest configuration: the tests
+  import it, or a requirements file lists it. They used to get `unittest`, which can't run them.
+
 ## [0.37.0] - 2026-10-20
 
 ### Added

docs/how-it-works.md

@@ -56,7 +56,8 @@ same tool, the same arguments and the same result.
   `loop`.
 
 A result that changes isn't a repeat, so a test that's run again after a fix, or a background job
-that's checked while it runs, never adds up. A call that may have changed something, like an edit or
+that's checked while it runs, never adds up. Timings don't count as a change: `75 passed in 0.12s`
+and then `in 0.14s` is the same result. A call that may have changed something, like an edit or
 a shell command that isn't read-only, starts the count again for the other calls: checking the build
 after each edit is fine. Change the threshold with `lcode config set repeat_limit 5`, or turn it off
 with `0` (that also turns off the checks for unreachable sites and for thinking in circles, below).
@@ -85,7 +86,8 @@ aren't tracked.
 Small models sometimes think in circles: a few paragraphs, over and over, until the context is full.
 That takes minutes behind the spinner and ends without an answer. lcode watches the thinking as it
 streams, and when its sentences keep repeating, it stops the model, drops that reply and asks for a
-short one: the next tool call, or what's blocking it. An empty reply, or one that filled the context
+short one: the next tool call, or what's blocking it. The dropped reply is saved, and the message
+says where, so you can read what went round in circles. An empty reply, or one that filled the context
 before it said anything, gets the same second chance. The second time in a request, lcode stops it
 (status `loop`), offering the [stronger model](usage.md#a-stronger-model-when-stuck) first when one
 is set. An answer the context cut off midway is kept, with a warning to `/compact` or use a larger

docs/usage.md

@@ -170,7 +170,9 @@ runs the project's checks itself before the answer is final:
 - **Programs** using `lcode -p --output json` see the runs in the result's `checks`.
 
 **Which checks.** lcode looks at the project: pytest when the configuration mentions it (`uv run
-pytest -q` with a `uv.lock`), otherwise `python -m unittest` for `test_*.py` files. Then
+pytest -q` with a `uv.lock`), when the tests import it, or when a requirements file lists it;
+otherwise `python -m unittest` for `test_*.py` files. Python checks run with the project's own
+environment when it has one (`.venv/bin/python`, `venv`, `env`), where its dependencies are. Then
 `npm test` (or pnpm, yarn, bun) when `package.json` has a test script, `cargo test`, `go test ./...`
 and `make test`. To choose the command yourself, set it for the repository in
 `.lcode/settings.toml`. That file is used once you trust the repository, like its hooks:
@@ -183,6 +185,13 @@ command = "uv run pytest -q && uv run ruff check ."
 Or set it for yourself with `lcode config set checks "make check"`; `lcode config set checks off`
 turns checks off.
 
+**Windows it never looked at.** A model that changes a window checks it with an import and the
+project's tests, which don't show what's on the screen, and calls it done: you find a blank
+window. When a request changed a Python file that draws one (it imports PyQt, PySide, tkinter,
+Kivy, wx or GTK) and the model never took a [screenshot](#screenshots) or looked at an image,
+lcode sends it back once: look at it, with clicks first if that's where the change shows, or say
+plainly that it's unchecked. `checks = "off"` turns this off.
+
 **Scripts it never ran.** A model that writes a build or install script tends to call it done
 without running it once, and the first thing you see is its first error. When a request wrote a
 shell script (a `.sh` file, or a file with a shell shebang) and no command in it ran the script,
@@ -427,6 +436,20 @@ a screen and sees what you would, measured:
 
 - For a web app, the model gives the command that starts the server and the page's URL: lcode
   starts the server, waits for the page to answer, takes the shot and stops the server.
+- **Clicks and keys first.** For something that only shows after a click, a Python Qt program can
+  take `actions` before the picture, the way you would: `double_click 300,200` (a point in the
+  window), `double_click Camera 1` (the widget showing that text), `click Save`, `right_click …`,
+  `key Escape`, `key ctrl+s`, `type some text`, `wait 2`. The result says what each step hit.
+- **Qt's warnings come first.** Qt names many layout mistakes in what the program prints, often
+  among the program's own logs. lcode lists them at the top of the report:
+
+    ```text
+    ● screenshot(python camera_wall.py)  actions: double_click Camera 1
+    Qt printed warnings (they often name the cause):
+    - QWidget::setLayout: Attempting to set QLayout "" on FocusedCamera "", which already has a layout
+    Steps, before the picture:
+    - double_click Camera 1: on QLabel "Camera 1 — Connected" at (207, 317)
+    ```
 - A model that can see describes the picture with the model's question in mind (see
   [Which model looks](#images)). Without one, the measurements still come back.
 - Shots are saved in `.lcode/screenshots/` in the project, which keeps itself out of git, so you

src/lcode/agent.py

@@ -99,7 +99,9 @@ def stall_note(trouble: str) -> str:
     }[trouble]
     return (
         f"[lcode] {first} Don't deliberate at length now. Either make your next tool call, or, if you're stuck, "
-        "tell the user in a few lines what you did, what fails, and what they could do."
+        "tell the user in a few lines what you did, what fails, and what they could do. If you were about to check "
+        "something by running it, run it now instead of reasoning about it: for a window, take a screenshot (with "
+        "actions for clicks or keys)."
     )
 
 
@@ -961,6 +963,12 @@ def unrun_scripts(self, commands: list[str]) -> list[str]:
             return []
         return checks.unrun_scripts(project_root_or_cwd(self.cwd), list(self.tools.before), commands)
 
+    def unseen_windows(self, tools_used: set[str]) -> dict[str, str]:
+        """Window code the model changed in this request and never looked at (only where the checks would run)."""
+        if self.allowed_tools is not None or self.planning() or self.no_changes or self.settings.checks == "off":
+            return {}
+        return checks.unseen_windows(project_root_or_cwd(self.cwd), list(self.tools.before), tools_used)
+
     def project_check(self) -> checks.Check | None:
         """The project's check command, for the main agent outside plan mode and reviews (None otherwise)."""
         if self.allowed_tools is not None or self.planning() or self.no_changes or self.settings.checks == "off":
@@ -1164,6 +1172,8 @@ def _run_turn(self, user_text: str) -> None:
         stalls = 0  # replies that went round in circles or ended without a word
         commands: list[str] = []  # shell commands run in this request
         scripts_noted = False  # the model was told about scripts it wrote and never ran
+        tools_used: set[str] = set()  # tools called in this request
+        windows_noted = False  # the model was told about windows it changed and never looked at
         for step in range(self.max_steps):
             if self.cancel is not None and self.cancel.is_set():
                 raise KeyboardInterrupt
@@ -1215,10 +1225,11 @@ def _run_turn(self, user_text: str) -> None:
                 continue
             if not calls and (self.reply_trouble == "circles" or not (reply.get("content") or "").strip()):
                 stalls += 1
-                self.messages.pop()  # nothing to keep: no answer, and circles would only fill the context
+                dropped = self.messages.pop()  # no answer, and circles would only fill the context
+                kept = self.keep_dropped(dropped)
                 if stalls < MAX_STALLS:
                     self.console.print(
-                        Text(f"  ⎿ {STALLED[self.reply_trouble]}: asking for a short answer", style="yellow")
+                        Text(f"  ⎿ {STALLED[self.reply_trouble]}: asking for a short answer{kept}", style="yellow")
                     )
                     self.messages.append({"role": "user", "content": stall_note(self.reply_trouble)})
                     continue
@@ -1234,7 +1245,7 @@ def _run_turn(self, user_text: str) -> None:
                 self.console.print(
                     Text(
                         "Stopped: the model's replies went round in circles or ended without an answer, twice in "
-                        "this request. Try again with a smaller step, or tell it what to do differently.",
+                        f"this request{kept}. Try again with a smaller step, or tell it what to do differently.",
                         style="yellow",
                     )
                 )
@@ -1253,6 +1264,14 @@ def _run_turn(self, user_text: str) -> None:
                     self.console.print(Text(f"  ⎿ never ran {', '.join(unrun)}: asking the model to", style="yellow"))
                     self.messages.append({"role": "user", "content": checks.unrun_note(unrun)})
                     continue
+                unseen = {} if windows_noted else self.unseen_windows(tools_used)
+                if unseen:
+                    windows_noted = True
+                    self.console.print(
+                        Text(f"  ⎿ never looked at {', '.join(unseen)}: asking the model to", style="yellow")
+                    )
+                    self.messages.append({"role": "user", "content": checks.unseen_note(unseen)})
+                    continue
                 check = self.project_check() if changed and not verified else None
                 if check is not None and check_rounds < checks.MAX_ROUNDS:
                     check_rounds += 1
@@ -1345,6 +1364,7 @@ def _run_turn(self, user_text: str) -> None:
                     )
                     if out_of_reach.stop:
                         blocked = out_of_reach
+                tools_used.add(name)
                 if name == "bash":
                     commands.append(str(args.get("command", "")))
                 if name in ("edit_file", "write_file", "apply_patch") and not result.startswith(REFUSED):
@@ -1412,6 +1432,14 @@ def _run_turn(self, user_text: str) -> None:
         self.turn_status = "success"
         self.print_stats()
 
+    def keep_dropped(self, reply: dict) -> str:
+        """Save a reply lcode dropped (its thinking too), so the user can see what went round in circles."""
+        text = "\n\n".join(part for part in (reply.get("thinking"), reply.get("content")) if part)
+        if not text.strip():
+            return ""
+        path = context_tools.save_output(text, "dropped reply")
+        return f" (the dropped reply is in {path})" if path else ""
+
     def can_escalate(self) -> bool:
         """Whether a stronger model may be asked here: one is set, and this isn't a subagent or plan mode."""
         model = (self.settings.escalate_model or "").strip().lower()

src/lcode/checks.py

@@ -50,49 +50,78 @@ def find(root: Path, setting: str, trusted: bool) -> Check | None:
     return detect(root)
 
 
-def python() -> str:
+VENVS = (".venv", "venv", "env")
+PYTEST_USE = re.compile(r"^\s*(?:import pytest\b|from pytest\b)|^\s*@pytest\.", re.M)
+
+
+def python(root: Path | None = None) -> str:
+    """The project's own Python (its .venv, venv or env), where its dependencies are; else the one on PATH."""
+    if root is not None:
+        for env in VENVS:
+            for exe in ("bin/python", "Scripts/python.exe"):
+                if (root / env / exe).is_file():
+                    return f"{env}/{exe}"  # the checks run in the project's root
     return "python" if shutil.which("python") else "python3"
 
 
+def uses_pytest(root: Path) -> bool:
+    """Whether the tests are written for pytest (they import it), or the requirements list it, without a config."""
+    for name in ("requirements.txt", "requirements-dev.txt", "requirements_dev.txt", "dev-requirements.txt"):
+        if re.search(r"^\s*pytest\b", text(root / name), re.M | re.I):
+            return True
+    return any(PYTEST_USE.search(text(path)[:20000]) for path in test_files(root)[:50])
+
+
 def text(path: Path) -> str:
     try:
         return path.read_text(errors="replace") if path.is_file() else ""
     except OSError:
         return ""
 
 
-def python_tests(root: Path) -> bool:
-    """Whether the project has Python test files (a few levels deep, outside environments and caches)."""
+def test_files(root: Path, limit: int = 200) -> list[Path]:
+    """The project's Python test files, a few levels deep, outside environments and caches."""
     from lcode.tools import IGNORE_DIRS
 
-    def walk(folder: Path, depth: int) -> bool:
+    found: list[Path] = []
+
+    def walk(folder: Path, depth: int) -> None:
         try:
-            entries = list(folder.iterdir())
+            entries = sorted(folder.iterdir())
         except OSError:
-            return False
-        if any(e.is_file() and re.fullmatch(r"test_.*\.py|.*_test\.py", e.name) for e in entries):
-            return True
-        return depth > 0 and any(
-            walk(e, depth - 1)
-            for e in entries
-            if e.is_dir() and e.name not in IGNORE_DIRS and not e.name.startswith(".")
-        )
+            return
+        found.extend(e for e in entries if e.is_file() and re.fullmatch(r"test_.*\.py|.*_test\.py", e.name))
+        if depth > 0:
+            for e in entries:
+                if len(found) >= limit:
+                    return
+                if e.is_dir() and e.name not in IGNORE_DIRS and e.name not in VENVS and not e.name.startswith("."):
+                    walk(e, depth - 1)
 
-    return walk(root, 3)
+    walk(root, 3)
+    return found[:limit]
+
+
+def python_tests(root: Path) -> bool:
+    """Whether the project has Python test files (a few levels deep, outside environments and caches)."""
+    return bool(test_files(root, limit=1))
 
 
 def detect(root: Path) -> Check | None:
     """A check command from what the project looks like. Python checks need test files: pytest configured in a
     project without tests "fails" with nothing to run."""
     if python_tests(root):
+        py = python(root)
         configs = text(root / "pyproject.toml") + text(root / "setup.cfg") + text(root / "tox.ini")
         if "pytest" in configs or any((root / name).is_file() for name in ("pytest.ini", "conftest.py")):
-            runner = "uv run pytest -q" if (root / "uv.lock").is_file() else f"{python()} -m pytest -q"
+            runner = "uv run pytest -q" if (root / "uv.lock").is_file() else f"{py} -m pytest -q"
             return Check(runner, "pytest in the project's configuration")
+        if uses_pytest(root):
+            return Check(f"{py} -m pytest -q", "tests written for pytest")
         if list(root.glob("test_*.py")) or list(root.glob("*_test.py")):
-            return Check(f"{python()} -m unittest", "test files")
+            return Check(f"{py} -m unittest", "test files")
         if (root / "tests").is_dir():
-            return Check(f"{python()} -m unittest discover -s tests -t .", "tests/")
+            return Check(f"{py} -m unittest discover -s tests -t .", "tests/")
     package = text(root / "package.json")
     if package:
         try:
@@ -229,3 +258,52 @@ def unrun_note(scripts: list[str]) -> str:
         f"deploy, delete data) or needs something you don't have, don't: tell the user plainly that {it} is untested "
         "and why."
     )
+
+
+# ----------------------------------------------------------------------------- windows it never looked at
+
+GUI_IMPORT = re.compile(r"^\s*(?:from|import)\s+(PyQt[56]|PySide[26]|tkinter|kivy|wx|gi\.repository)\b", re.M)
+LOOKED = frozenset({"screenshot", "view_image"})
+
+
+def unseen_windows(root: Path, changed: list[str], tools_used: set[str]) -> dict[str, str]:
+    """Python files that draw a window (they import a GUI toolkit), changed in this request when the model never
+    looked at one: file -> toolkit.
+
+    A small model checks a window change by importing the module and running unrelated tests, then calls it done;
+    the user finds a blank window. Tests and imports don't show what's on the screen; a screenshot does.
+    """
+    if tools_used & LOOKED:
+        return {}
+    found = {}
+    for rel in changed:
+        path = root / rel
+        if path.suffix != ".py" or not path.is_file():
+            continue
+        try:
+            text = path.read_text(errors="replace")[:200_000]
+        except OSError:
+            continue
+        match = GUI_IMPORT.search(text)
+        if match:
+            found[rel] = match.group(1)
+    return found
+
+
+def unseen_note(found: dict[str, str]) -> str:
+    names = ", ".join(found)
+    kits = sorted(set(found.values()))
+    qt = any(kit.startswith(("PyQt", "PySide")) for kit in kits)
+    how = (
+        "take a screenshot with the screenshot tool: it runs the program without a screen and shows what it draws, "
+        "and what the program printed (Qt names layout mistakes there). For something that only shows after a click "
+        'or a key, give actions, e.g. actions=["double_click Camera 1"]'
+        if qt
+        else "take a screenshot with the screenshot tool: it runs the program on a virtual display and shows what it "
+        "draws"
+    )
+    return (
+        f"[lcode] You changed {names}, which draws a window ({', '.join(kits)}), but never looked at it: tests and "
+        f"imports don't show what's on the screen. Before you say it works, {how}. Check the picture against what the "
+        "user asked for. If the program can't run here, tell the user plainly that you haven't seen it working."
+    )