fix: stop image blocks breaking turns on text-only models (runtime modality recovery)

[alecuba16]

Closes #1302.

Text-only models served behind optimistic OpenAI-compatible endpoints hard-fail the moment a tool result carries pixels, and because the image block stays in history every subsequent turn of the session is poisoned. This adds runtime modality recovery in three coordinated layers:

1. Prevent — tool-result injection is capability-aware (tool_output_to_content_blocks_with_image_support at all three agent injection sites): for text-only providers, image blocks become an explicit text omission note, so pixels never reach the request.
2. Advise — PromptCapabilities.text_only_model adds a Model Image Capability section to the system prompt so the model knows up front to use text alternatives (OCR, file contents) or ask the user.
3. Self-heal — a new image_capability record (per provider+model, 30-min TTL) stores runtime image-input rejections; supports_image_input consults it, and both the failover layer and the OpenRouter streaming path replay a modality-rejected request once with images filtered to text markers on the same provider (no failover burn, no retry slot burned: deterministic failures replay inside the attempt).

Subtleties covered per the review guidance on #1302:

• Streaming rejections are handled inside run_stream_with_retries because OpenAI-compat 400s surface asynchronously in the stream, after the outer retry decision point.
• The record key is the stripped model id, so records are keyed by the actual provider/profile/model the rejection came from even when the runtime model string transiently carries a session-profile prefix after session restore.
• Replay cannot duplicate partial output or tool calls: the replay path filters image blocks to text markers before the first streamed token is emitted downstream.
• max_retries=1 replay covered by wire-level red-green tests (record key, stream replay, filtered replay).

Validation: wire-level red-green tests plus crate suites at baseline (jcode-base image capability tests 27/27 green after rebase). One caveat from the issue: the endpoint side of the end-to-end validation used a scripted mock OpenAI-compat server; no live text-only model was reachable from the test machine, so the real-endpoint path is recorded as blocked, not silently claimed.

Branch is a single commit rebased onto current master.

[greptile-apps]

Confidence Score: 4/5

[High impact] Adds runtime image-capability tracking and modality-aware prompting.

The unresolved case-sensitive model isolation issue must be fixed before merging; a text-only model's rejection can still suppress valid images for a distinct vision-capable model on the same endpoint.

Findings

1.  Distinct models lose image support ▶

Fix with agent prompt

### Issue 1
crates/jcode-provider-core/src/image_capability.rs:124-127
If a direct endpoint serves distinct case-sensitive IDs such as text-only `model-a` and vision-capable `Model-A`, an image rejection from the first disables image input for the second. The new process-wide rejection cache lowercases both IDs, although custom-endpoint model selection preserves their exact spelling. This affects independent sessions on the same endpoint: the vision session replaces valid images with omission markers for up to 30 minutes, and new tool images are left out of its conversation.

Preserve the exact trimmed, session-prefix-stripped model ID in rejection records and lookups, including the runtime capability lookup. Keep catalog-specific case normalization separate.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

This PR recovers from image-input rejections without removing tool images from saved history.

• Image rejections teach providers to retry without pixels.
• OpenRouter streams replay image rejections as text-only requests.
• Text-only models get guidance to use text instead of images.

Scope acknowledgment: alecuba16 stated that the shared manifest-update, aged live-lock, replacement-lock, indeterminate Windows-holder, and reused-PID issues concern installer-lock code removed from this branch because it duplicated PR #1392's territory and the runtime half was never on master. Those issues do not apply to the current modality-recovery diff.

_{Reviews (33) · Last reviewed commit: "fix: stop image blocks breaking turns on..." · Reviewed by Greptile}

[greptile-apps]

Comments Outside Diff

These findings could not be posted inline.

•  Identify lid override owners crates/jcode-base/src/lid_override.rs:166 ▶

  If a daemon dies while its lid setting is overridden and its PID is later reused, recovery treats the unrelated process as the journal’s live owner. Subsequent daemons leave the stale setting in place, so lid close can remain blocked from sleeping. Record a process-lifetime identity alongside the PID while retaining genuine re-exec recovery.

•   Authorize applet controls crates/jcode-app-core/src/server/client_actions.rs:188 ▶

  A client attached to one session can supply another session’s ID in an applet action or close request. Subscribed and one-shot requests change or remove the other session’s applet without checking the caller’s attachment, allowing cross-session state tampering by a client with access to the daemon socket. Bind these controls to an authorized session.

  How this was verified: Requests targeting a different session returned success and changed that session’s applet snapshots.

•  Validate patched documents crates/jcode-applet-types/src/patch.rs:78 ▶

  apply_patch says its result is revalidated, but it only deserializes the patched document. Patches introducing a missing asset, an undeclared capability, or over-limit state return success with a higher revision; callers relying on that success must perform a separate validation to avoid accepting the invalid document.

•  Count all visible text crates/jcode-applet-types/src/validate.rs:287 ▶

  The document text limit omits button labels, key-value rows, table headings, badges, and other visible strings. A document with 1,101 bytes of visible text passed an 80-byte limit, so hosts relying on that budget may have to render substantially more text than they allowed.

•  Stable status remains stale scripts/install.sh:320 ▶

  After installing a new stable version, this script updates the binary link and stable-version marker but leaves manifest.stable unchanged. selfdev status reads the manifest, so it reports the previous stable version-or none-instead of the installed one. This non-blocking error makes the installation status unreliable.

[greptile-apps]

Want your agent to iterate on Greptile's feedback? Try greploops.

[alecuba16]

Rebased onto current master and CI is green. This follows the review recommendation from your comment on #1302 (replay cannot duplicate partial output/tools, capability records keyed by actual provider/profile/model). Ready for review whenever you have time, happy to address any feedback.

[alecuba16]

Small correction to my earlier note: upstream CI never actually ran on this branch. The runs are sitting in action_required because fork PRs need a maintainer to approve workflows first, so "CI is green" was wrong of me. What I actually verified: full local test suite passing on the branch, plus a local mirror of your CI gates (fmt, check, clippy, ratchets) clean relative to master baselines. Sorry for the confusion.

[alecuba16]

I slimmed the branch while re-verifying: the manifest-lock work (file_lock.rs, the install scripts, the BuildManifest update_with machinery) was riding along from my fork history and overlaps PR #1392, so I removed it from this one. The branch is now the modality recovery alone. The lock work is preserved on my side for a separate PR if there is interest. Local gates (fmt, check, clippy, budget scripts, e2e cohorts) now match master's baseline exactly, with no new findings introduced by the branch.

[alecuba16]

Fresh rebased single-commit head (fae6dbe). The latest deferred-tool-references finding from the review bot is fixed with a pinning test, and every earlier thread is addressed. Ready for review.

[greptile-apps]

Distinct models lose image support

If a direct endpoint serves distinct case-sensitive IDs such as text-only model-a and vision-capable Model-A, an image rejection from the first disables image input for the second. The new process-wide rejection cache lowercases both IDs, although custom-endpoint model selection preserves their exact spelling. This affects independent sessions on the same endpoint: the vision session replaces valid images with omission markers for up to 30 minutes, and new tool images are left out of its conversation.

Preserve the exact trimmed, session-prefix-stripped model ID in rejection records and lookups, including the runtime capability lookup. Keep catalog-specific case normalization separate.

_{Ran code and verified through T-Rex}

Prompt To Fix With AI

This is a comment left during a code review.
Path: crates/jcode-provider-core/src/image_capability.rs
Line: 124-127

Comment:
**Distinct models lose image support**

If a direct endpoint serves distinct case-sensitive IDs such as text-only `model-a` and vision-capable `Model-A`, an image rejection from the first disables image input for the second. The new process-wide rejection cache lowercases both IDs, although custom-endpoint model selection preserves their exact spelling. This affects independent sessions on the same endpoint: the vision session replaces valid images with omission markers for up to 30 minutes, and new tool images are left out of its conversation.

Preserve the exact trimmed, session-prefix-stripped model ID in rejection records and lookups, including the runtime capability lookup. Keep catalog-specific case normalization separate.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

[alecuba16]

Fixed in 159ff39: rejection records and lookups now keep the exact case-sensitive model id, so a text-only model-a no longer disables images for a distinct vision-capable Model-A on the same endpoint. Added a regression test covering the mixed-case pair.