Skip to content

[Feat] Recover from stalled browser and desktop tasks with bounded replanning #16

Description

@Yunaik

Why

Browser and desktop tasks can stall after repeated ineffective actions. The existing rethink mechanism provides a starting point for detecting stalled execution and requesting a revised plan.

Scope

Detect repeated ineffective actions, refresh the observation, and ask the existing chat model for a revised plan. Limit recovery attempts and total recovery time. Stop or ask the user when the budget is exhausted.

Reuse the existing rethink mechanism where applicable. Preserve permission checks during retries. Calibrate any confidence thresholds separately for each model using labelled examples.

Acceptance criteria

  1. Behavioural tests cover repeated ineffective actions, delayed progress, observation refresh, successful recovery, and exhausted recovery budgets.
  2. Deliberately blocked browser and desktop tasks either recover with verified completion or end with a clear explanation and an actionable escalation.
  3. Recovery records include the triggering actions, refreshed observation, revised plan, attempts, and termination reason.
  4. The shared task suite compares recovery enabled and disabled, reporting completion rate, extra model calls, added time, and wasted actions.
  5. Retries stop within the configured attempt and time limits. Tests verify that permission checks remain active.

Dependencies and starting points

Use the repeatable browser and desktop task suite linked below. Begin with existing operations, then include the desktop extensions as they become available.

s1a/tool/rethink.py, evals/README.md:33, and docs/browser-front.md.

Evaluation dependency: #13.

Related desktop extension: #15.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions