Skip to content

fix: render image/file attachments as identity placeholders in the review context - #162

Merged
btspoony merged 1 commit into
mainfrom
fix/161-image-placeholder
Nov 17, 2026
Merged

btspoony merged 1 commit into
mainfrom
fix/161-image-placeholder

Conversation

@btspoony

Copy link
Copy Markdown
Member

Fixes #161

Problem

The advisor reviewer model receives the transcript as a single text prompt (buildOptions sends one text block), so it never sees image bytes. blockText() dropped image and file content blocks entirely, which made a screenshot-only user message render as **user**: <empty>. The reviewer read that as ground truth — "the user supplied no screenshot" — and steered the primary agent into retracting a correct claim.

Change

  • blockText() (src/transcript.ts) renders image blocks as identity placeholders — [image attachment present: sha256:<8 hex>, <w>x<h> <mediaType>; pixels are not included in this review context] — and file blocks as name/size placeholders. A non-digest or truncated attachment id is echoed verbatim; the placeholder never fabricates a digest.
  • The wording deliberately avoids dsh-llm's textOnlyImageText() phrasing ("because this model accepts text only"): that line is written for the request layer and would invite the reviewer to assume the primary model cannot see the attachment either — the defect's mirror image.
  • No branching on model inputModalities: the field is optional and absent means unknown, so the placeholder is required unconditionally. No image passthrough: the reviewer only needs the fact that an attachment exists, and a content-blocks pipeline would drag in char budgets, fingerprints, and the IMAGE_OFFLOAD_REQUIRED retry path.
  • Documentation: one README limitations bullet and a settings-card hint (en/zh) stating the review context is text-only and that file content the primary agent reads still enters the transcript as text.

Verification

  • pnpm typecheck exit 0.
  • pnpm vitest run tests/transcript.test.ts 41/41, including a new regression case: an image-only user message must contain image attachment present and must not contain **user**: <empty> (confirmed failing when the image arm is removed).
  • QC: correctness seat 6/6 clean (digest guard arithmetic, wording mirror-risk, <empty> reachability, locale key completeness, card hint placement/a11y, regression-test validity); risk seat findings (hint overclaim on read-file text, README metadata accuracy) fixed before commit.

Risk

The reviewer still cannot comment on the visual content of a screenshot — it only knows one exists. Whether the advisor should ever see pixels is a product decision, intentionally out of scope here.

@m-inspector

m-inspector Bot commented Nov 17, 2026

Copy link
Copy Markdown

✅ Verdict: ship it · Score 92% (mergeable)

Findings Count
🔴 must-fix 0
🟠 should-fix 0
🟡 may-fix 1
🔵 nit 1

📋 Review

What this PR does: Fixes the advisor reviewer's blind spot for attachment-bearing user turns: blockText() in src/transcript.ts now renders image blocks as identity placeholders (short digest, dimensions, media type) and file blocks as name/size placeholders, so a screenshot-only message no longer collapses to **user**: &lt;empty> and the reviewer cannot read "no screenshot supplied" out of an empty delta. The change is text-only by design — attachment bytes never enter the review context, no branching is done on the optional inputModalities field, and the bound is documented in a README accepted-gap bullet plus an en/zh settings-card hint. Scope is one rendering helper, two locale strings, one card hint, and docs; review verdict is ship it at 92%, with one docs-only may-fix for the untranslated README bullet and the stale bilingual record.

Findings

🟡 New README gap bullet leaves the bilingual pair and its consistency record stale

  • Evidence:

    [DOCS-01] New README gap bullet leaves the bilingual pair and its consistency record stale

    • Evidence: README.md:141 — the diff adds a new - **Text-only review context** — … bullet to the accepted-gaps list under "Limitations & roadmap".
    • Evidence: README.zh.md:137-140 — the corresponding Chinese gap list still ends at the advisor 上下文有界 bullet; every other accepted gap in that list is mirrored, and the new one is absent.
    • Evidence: README.i18n.yaml:1-6 — the repo-root bilingual-pair record file states "Both languages carry equal authority; after editing either side, bring the other along and re-record with: git hash-object README.md / git hash-object README.zh.md", and pins README.md: 4b9a16be… / README.zh.md: c12e579c… as "the last confirmed-consistent state". Neither README.zh.md nor README.i18n.yaml is in this changeset.
    • Impact: (a) the Chinese README — declared to carry equal authority — does not tell zh readers that image/file attachments reach the reviewer only as metadata placeholders, i.e. that screenshots they paste are not reviewed as pixels; (b) the recorded README.md blob hash now describes a file that no longer exists, so the consistency record asserts a pairing that is false and the next sync pass starts from a wrong baseline.
    • Effort: XS — translate the one bullet and re-record both hashes.
    • Risk: LOW — docs only; no code or runtime behavior touched.
    • Confidence: HIGH — read the new bullet, the zh gap list, and the record file itself.
    • Merge class: may-fix
    • Fix sketch: Add the mirrored bullet to README.zh.md's accepted-gaps list and re-record both blob hashes in README.i18n.yaml in the same commit, per the procedure documented in that file.
  • Location: README.md:141

  • Category: DOCS

  • Merge class: may-fix

🔵 Seat coverage declaration — changeset-2-root-src (not a finding, do not tally)

  • Evidence:
    Coverage note, not a finding: this seat is changeset-2-root-src.

    Scanned in full: the pinned diff (README.md, src/client/advisor-card.tsx, src/client/locales.ts, src/transcript.ts) plus the changed regions and their consumers — blockText/renderMessage (src/transcript.ts:151-206), the DeltaRenderer render path (:400-424), the card's model-field block (src/client/advisor-card.tsx, diff hunk at :315-318), both locale dictionaries (src/client/locales.ts:28,126), and the README gap list (README.md:124-145).

    Not scanned: tests/transcript.test.ts (explicitly outside this seat's file scope; the added image-only assertion uses as unknown as ContentBlock, so it cannot prove the V4 attachment shape — worth a look by the owning seat); src/** outside the four scoped files; node_modules/ is absent in this worktree, so the @deepseek-ai/dsh-llm ContentBlock image/file arm shape (attachment.attachmentId / mediaType / width / height, and attachment.name / bytes on the file arm) could not be verified against installed types — the four new field reads are type-checked only when deps are installed (pnpm typecheck).

    Considered & rejected (leads, do not re-chase without new evidence): (1) the assistant-role branch of renderMessage still drops image/file blocks (src/transcript.ts:194-206) — not reachable, assistant content arms in this stack are text/tool-call/reasoning, and the behavior is unchanged by the diff; (2) an attachment.name or fallback attachmentId containing newlines could not forge **agent**: turns in the delta — transcript message text is already arbitrary unescaped user content and README explicitly accepts "no quarantine" as a documented bound; (3) the id.length >= 15 digest guard is correct for sha256: + ≥8 hex and safely falls back to the raw id otherwise; (4) unconditional textOnlyContextHint placement inside the Model field is layout preference, not a wrong outcome.

  • Location: repo-wide

  • Category: process

  • Merge class: nit

Revalidated findings

none

Linked-issue AC

- Placeholder signalling that an image attachment exists (issue suggestion 1) — met: blockText() emits [image attachment present: sha256:&lt;8 hex>, &lt;w>x&lt;h> &lt;mediaType>; pixels are not included in this review context], and file blocks get a name/bytes placeholder; the digest guard falls back to the raw id instead of fabricating one.
- Advisor must not interpret an image-only user message as "the user provided no screenshot" (issue's example defect) — met: the empty-text collapse is gone for image/file blocks, and the PR reports a regression case asserting image attachment present and absence of **user**: &lt;empty>; the case was not re-run in this read-only review.
- Pass image bytes through to the advisor (issue suggestion 2) — cut: the PR explicitly scopes pixels out (no content-blocks pipeline, no char budgets/fingerprints/IMAGE_OFFLOAD_REQUIRED), and the README gap bullet states passthrough is unsupported.
- Select the approach by whether the advisor model supports images (issue suggestion 3) — cut: inputModalities is optional and absent means unknown, so the placeholder is applied unconditionally; documented as a deliberate bound rather than a per-model branch.

✅ Verified

- Read README.md:141 inside the Limitations gap list (README.md:124-145): the diff adds the Text-only review context accepted-gap bullet, stating image blocks become identity placeholders and file blocks name/size placeholders.
- Read README.zh.md:124-146: the mirrored gap list carries every other bullet but ends at advisor 上下文有界; the text-only bullet is absent.
- Read README.i18n.yaml:1-6: the record pins README.md: 4b9a16be… and README.zh.md: c12e579c… and states both languages carry equal authority; neither README.zh.md nor the yaml is in the changeset.
- Read src/transcript.ts:145-206: image arm uses the sha256: + id.length >= 15 guard and falls back to the verbatim id; file arm emits name and byte count; the user/tool arms join non-empty block text.
- Read the assistant branch of renderMessage (src/transcript.ts:194-206) and the DeltaRenderer render path (:400-424): the assistant arm still drops image/file blocks, unchanged by this diff.
- Read src/client/locales.ts:28,126: textOnlyContextHint exists in both dictionaries with matching intent (en: text-only context, file content read by the primary agent still reviewed; zh mirror).
- Read src/client/advisor-card.tsx model field: the hint renders unconditionally inside the Model field, alongside the stale/required hints.
- ❓ tests/transcript.test.ts was outside this seat's file scope; the PR's new image-only assertion builds its input with as unknown as ContentBlock, so it cannot by itself prove the V4 attachment arm shape.
- ❓ The installed @deepseek-ai/dsh-llm ContentBlock image/file arms (attachmentId/mediaType/width/height, name/bytes) could not be checked against real types — node_modules/ is absent in this worktree; those four field reads are type-checked only by pnpm typecheck.
- PR body reports pnpm typecheck exit 0 and tests/transcript.test.ts 41/41; this read-only review re-ran neither command.

🗑️ Considered & rejected

- Assistant-role renderMessage still dropping image/file blocks — not reachable in this stack (assistant content arms are text/tool-call/reasoning) and the behavior is unchanged by the diff.
- Transcript-forgery via a newline-bearing attachment.name or fallback attachmentId injecting **agent**: turns — transcript message text is already arbitrary unescaped user content and the README accepts "no quarantine" as a documented bound.
- The id.length >= 15 digest guard — correct for sha256: plus at least 8 hex and safely falls back to the raw id otherwise, so no fabricated digest.
- Unconditional textOnlyContextHint placement inside the Model field — a layout preference, not a wrong outcome; the copy itself does not overclaim.
- Direct image passthrough / inputModalities branching as missing functionality — deliberately out of scope per the PR description and the accepted-gap documentation, not a defect in this changeset.

🛠️ Plan to fix

Expand fix plan
Only the may-fix remains; the code paths need no changes (verdict ship it, must-fix=0, should-fix=0). The nit is a seat coverage note, not a finding, and requires no action.

1. **Mirror the bullet in `README.zh.md`** — in the `## 限制与路线图` gap list, insert a translation of the English `Text-only review context` bullet immediately after the `**advisor 上下文有界**` entry, preserving list order and style: 审阅上下文仅含文本;image 块以身份占位符(短附件摘要、尺寸、媒体类型)出现,file 块以名称/字节数占位,附件字节不会到达评审者;主代理读入 transcript 的内容仍会被审阅;透传图片不受支持。 Keep the wording aligned with the existing zh `textOnlyContextHint` string so the card and README do not diverge.
2. **Re-record the bilingual pair in `README.i18n.yaml`** — after step 1, run `git hash-object README.md` and `git hash-object README.zh.md`, then replace both pinned values (`4b9a16be…` for `README.md`, `c12e579c…` for `README.zh.md`) with the new blobs, leaving the header comment and procedure untouched.
3. **Land steps 1-2 in one commit** with no other README edits on either side, so the recorded hashes describe exactly the commit that made the pair consistent again.
4. **Verification gates** — (a) `git hash-object README.md` / `git hash-object README.zh.md` output matches both recorded values byte-for-byte; (b) the zh accepted-gap list now has the same entry count and order as the en list (`README.md:124-145` vs the zh block); (c) the zh bullet says the same three things as the English one — placeholders only, no attachment bytes, primary-agent-read file content still reviewed; (d) docs-only change, so no test or typecheck re-run is required.
5. **Follow-up note (not a code change)** — if the owning seat picks up `tests/transcript.test.ts`, tighten the new image-only case to construct a real V4 `ContentBlock` image arm instead of `as unknown as ContentBlock`, so the attachment-shape assumption is asserted rather than cast away; and run `pnpm typecheck` once dependencies are installed to confirm the four attachment field reads against the `@deepseek-ai/dsh-llm` types.

Review 1 · commit b012408

@m-inspector m-inspector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mstar-inspector line comments · round 1 · b012408

Comment thread README.md Resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Advisor 审阅上下文未附带图片,可能导致错误纠正 / Advisor model can not read photo attachment, and may cause mistakes.

1 participant