Skip to content

Harness: prompt-error orphans the run, unclosed think tag hides the answer, mid-stream 5xx/429 fatal while severed connections recover #1155

Description

@nordicnode

While investigating recent reports of garbled/empty output and runs dying mid-task, I traced three bugs in the harness. All three are in this repo (file:line below). The latter two directly match the reported symptoms.

1. prompt-error doesn't stop the run: it orphans a run that keeps executing

In packages/agent-runtime/src/main-prompt.ts (callMainPrompt), when a local agent template fails validation, the code sends the prompt-error action and then unconditionally continues into await mainPrompt(...). Host-side, run() already resolved with the error (sdk/src/run.ts: callMainPrompt is invoked fire-and-forget behind the resolving promise).

Consequences:

  • After the "Invalid agent config" banner, the run silently runs to completion: credits burned, tools executed (real file edits / terminal commands), response chunks streamed after the UI already showed the error.
  • The error branch hands back the live state object the orphan keeps mutating. The next run deep-clones it only at its own start, so a follow-up message sent while the orphan is still running snapshots a half-written history (possibly with orphaned tool calls and no results).
  • The orphan's final prompt-response resolves an already-settled promise, so its outcome is dropped.

Repro sketch: drop a malformed agents/*.md into a project, send any message, get the error banner, and the run continues in the background.

Suggested fix: return (or abort the run signal) after sending prompt-error.

2. A literal `

` in content reclassifies the rest of the answer as "thinking"

In packages/agent-runtime/src/util/think-tag-stream.ts, an explicit open tag arms the in-think state and everything after it in the step is emitted as reasoning_delta (thinking box) instead of text; flush() releases the remainder as reasoning if the close never arrives.

So whenever a model emits the tag as prose (writing docs, quoting a template, or a lane whose chat template is broken), the visible answer lands in the thinking box and the user sees an empty or short reply. This matches the "garbled/empty output" reports. History still stores the raw text, so this is a display/reclassification bug rather than context loss, which is also why it's easy to miss from the server side.

Suggested fix: treat an unclosed explicit open like an orphan close (release the held content as text at flush), or require the close within a bounded window.

3. Mid-stream 5xx/429 kills the whole run; an identical severed connection recovers

In sdk/src/impl/llm.ts (error-chunk branch), recovery is offered only when isTransientNetworkError matches (a socket-level allowlist). A provider-reported 500/429 arriving mid-stream (the openai-compatible shim enqueues it as an error part with finishReason='error') is thrown straight out, and run-agent-step.ts ends the entire run with an error.

The same underlying network death, when it surfaces as a severed body, takes the other path: a recovery note is injected into the conversation and a retry step is forced (capped at 3 consecutive, MAX_CONSECUTIVE_STREAM_RECOVERIES in stream-parser.ts).

So the recoverable class is "the connection failed to speak" and the fatal class is "the provider reported a failure", which is backwards for flaky endpoints, where both are the same transient event.

Suggested fix: route retryable APICallErrors (5xx/429) in mid-stream error chunks through the same capped recovery path.

Happy to open PRs for any of these.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions