Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions docs/changes/unreleased/1026-image-step-names-model.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
---
kind: added
title: a picture's step row names the image model, and opening it shows the prompt
pr: 1026
surface: [chat]
invalidates:
- "A `generate_image` step row said nothing but the tool's name and a clock. It now reads `generate_image circle.png · seedream-5` — the file it wrote, and behind the middle dot, dim, the image model that drew it, in the short spelling (`modelui.ModelWord`) this surface uses for a model everywhere else. An unknown model draws nothing there rather than a placeholder."
- "Opening a `generate_image` step showed the picture and the path and nothing else. It now leads with what went IN, as prose: the prompt quoted and wrapped (capped at twelve lines, lifted by the usual `… N more lines`), one dim line per other input the call gave, then `drawn with <model>` — or `asked for best · drawn with seedream-5` when the call asked in its own words. The picture and its whole path still close the block."
- "A RUNNING `generate_image` step showed only its clock when opened. It now shows the same prompt block above the clock, the way a running `bash` call already shows its command."
- "Nothing on the wire carried which image model a call used. Nothing on the wire carries it now either: the row reads the tail of the tool's own result line, `… , generated on <model>`, which is where the model that the request actually carried has always been written. There is no new event field and no second derivation — internal/session's `describeGeneratedImage` remains the one authority, and `internal/session/tools_image_test.go` pins that sentence's shape."
- "`generate_image` and `view_image` rows had no target at all, so no path on them was clickable. Both now draw the file they are about, resolved by the same `app.picturePath` rule the preview uses (arguments first, then the result, absolute beating relative), and the path on the row is a door like every other path on this surface."
---

`generate_image` is the one hand whose model a person chooses per call, so the
step row is where "which image model is this?" gets asked — and it was the one
row that could not answer. Both halves ride roads the row already draws from: the
call's arguments for what was asked, and the tool's own result for what actually
drew. The model is resolved inside the tool against the live catalog, so the
result is the only place the answer exists; a surface that showed its own default
instead would be wrong the first time a hosted task room drew a row.
50 changes: 50 additions & 0 deletions internal/manual/chat/making-pictures-audio-and-video.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,56 @@ This is the same freedom you have yourself in `/settings` → Providers, handed
to aforge per call: ask it to "draw this one with gemini" or "try the best
image model" and it can, just in time.

## Which image model was used, and how do I see the prompt you sent?

**The step row names it, and opening the step shows the input.**

While a picture is being drawn the row is the tool and its clock. When it
finishes the row reads:

```
generate_image circle.png · seedream-5 1.2 MB · 2.4s
```

The file is what you are reading for, so it leads; the image model behind the
middle dot is dim, because it qualifies the file rather than being the point of
it. It is the **short** spelling — the vendor prefix comes off, so
`bytedance/seedream-5` reads `seedream-5`. If the model is not known — the call
is still running, or the conversation was recorded by an older build — the row
says **nothing at all** there rather than a placeholder.

Click the row, or select it with ↑/↓ and press **enter**, to open the step. It
shows what went in, as prose rather than as JSON:

```
"a small red circle, flat vector, centred on white"
1024x1024
from art/sketch.png
drawn with seedream-5
```

The prompt first, in quotes, exactly as it was sent — a very long one folds at
twelve lines and `… N more lines` lifts the rest. Then one line for each other
input you gave: the size or the aspect ratio, and the pictures it worked from.
An input you left out has no line, because leaving `size` out means the image
model's own default and aforge will not guess at what that is. The picture
itself, and the path under it, follow below.

If the call asked for a model in its own words, the line keeps both halves —
`asked for best · drawn with seedream-5` — so you can see what was requested and
what actually ran. A running call that asked for one says `asked for seedream`
until it finishes.

A call that **failed** has no file and no model beside it on the row — there was
no picture to attribute. The model it tried is named inside the failure the
expansion shows, e.g. `Image generation failed (bytedance-seed/seedream-5-0-pro):
…`.

To change which model draws, just say so: "draw this one with gemini", "use the
best image model". That is the `model` argument on the tool, described in the
section above, and it changes nothing permanently. For a new default, use
`/settings` → Providers.

## Why does a picture or video look generic, blurry, or like AI slop?

Four causes, all fixable — none of them is "the model is bad at this".
Expand Down
6 changes: 6 additions & 0 deletions internal/manual/chat_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -1379,6 +1379,12 @@ func TestTheChatManualAnswersTheQuestionsPeopleAsk(t *testing.T) {
"making-pictures-audio-and-video"},
{"how do I find the file for the image you generated", "making-pictures-audio-and-video"},
{"why is the picture you generated over --host not painted in my terminal", "making-pictures-audio-and-video"},
// The step row names the image model behind the file, and opening the
// step shows what was sent to it. Both are asked about the PICTURE — the
// person is looking at a row in their transcript — so they land on the
// making page rather than on the screen or the model pages.
{"which image model drew this picture", "making-pictures-audio-and-video"},
{"show me the prompt you sent to the image model", "making-pictures-audio-and-video"},
// Written from a real run: twelve clips rendered in parallel with
// nothing shared, stitched with a video-only crossfade — the person
// asked all three of these, in these words, and the answers were
Expand Down
54 changes: 54 additions & 0 deletions internal/session/tools_image_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -723,3 +723,57 @@ func TestGenerateImageRefusesAReferenceItCannotRead(t *testing.T) {
t.Fatalf("a refused reference still cost %d generations", len(painter.seen))
}
}

// ── the model the surface reads back ────────────────────────────────────────

// THE RESULT LINE IS THE SURFACE'S ONE ROAD TO THE IMAGE MODEL, and the shape
// of the sentence is the contract.
//
// internal/tui3 draws "which model drew this" on the step row by reading the
// tail of this line (its imagepreview.go's [generatedPictureModel]) — there is
// no second field on the wire carrying it, because the model is CHOSEN inside
// this tool, after the call's own word has been resolved against the catalog,
// and a name resolved in one place and re-derived in another is a name that
// will one day disagree with itself.
//
// So this test asserts the parse rather than the prose: the model that rode the
// request is what stands after the marker, at the end of the line, whether it
// came from the slot default or from the call's own word. Reword the sentence
// and this fails here, where the fix is cheap — rather than in a terminal,
// where the row simply stops saying anything and nobody is told why.
func TestTheImageResultNamesTheModelWhereASurfaceCanReadIt(t *testing.T) {
// The same marker internal/tui3 looks for. It is written down twice on
// purpose: this copy is the test's own, so a change to either side of the
// contract has to be made deliberately on both.
const marker = ", generated on "

painter := &scriptedMedia{base64: base64.StdEncoding.EncodeToString(pngOfSize(t, 8, 8))}
agent, _ := newTestAgent(t, &scriptedCompleter{}, func(config *Config) {
config.Media = painter
config.MediaModel = mediaModels(map[string]string{modalityImage: "paint/default"})
config.MediaPick = pickTable(map[string]string{"image/crayon": "paint/crayon-2"})
})

for index, testCase := range []struct{ args, want string }{
{`{"prompt":"a lighthouse"}`, "paint/default"},
{`{"prompt":"a lighthouse","model":"crayon"}`, "paint/crayon-2"},
} {
result, isError := runTool(t, agent, "generate_image", testCase.args)
if isError {
t.Fatalf("%s failed: %s", testCase.args, result)
}
line, _, _ := strings.Cut(strings.TrimSpace(result), "\n")
at := strings.LastIndex(line, marker)
if at < 0 {
t.Fatalf("the result carries no %q marker for a surface to read: %q", marker, line)
}
if got := strings.TrimSpace(line[at+len(marker):]); got != testCase.want {
t.Fatalf("the line ends on %q, want the model %q", got, testCase.want)
}
// And it is the model the REQUEST carried, which is the whole point of
// reading it here rather than guessing it there.
if got := painter.request(index).Model; got != testCase.want {
t.Fatalf("the request rode %q while the line said %q", got, testCase.want)
}
}
}
177 changes: 177 additions & 0 deletions internal/tui3/imagecall.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,177 @@
package tui3

import (
"encoding/json"
"strings"

"github.com/Agent-Field/aforge-v2/internal/tui2/modelui"
)

// WHAT WENT INTO A PICTURE, when somebody opens the step.
//
// The row above says which file came out and which image model drew it
// (toolstat.go's [app.pictureTarget]); this is the other half of the same
// question, and it is the half a person opens the step for — they want to read
// the prompt, because the prompt is the part they would change. It is drawn as
// PROSE and never as the JSON it arrived in: a person opening a row to read one
// sentence should not have to find it inside a payload, and every other
// expansion on this surface already shapes its tool's arguments for reading
// (toolview.go's [app.detailBody] — an edit's diff, a bash call's command).
//
// The two tiers are the row's own (toolview.go's parameter hierarchy). The
// prompt is the substance and takes the primary ink; everything that QUALIFIES
// it — the frame it was asked for, the pictures it worked from, the model that
// drew — recedes to dim, one fact per line.
//
// EVERYTHING HERE COMES OFF THE ROADS THE ROW ALREADY DRAWS FROM: the arguments
// session already sends for this call (Event.Args, bounded by its own
// argsLimit and marked where it was cut, so a prompt too long to carry arrives
// saying so and is never fetched a second way), and the model out of the tool's
// own result. Nothing is asked of the disk and nothing new is put on the wire.

// promptWindow caps the prompt block, in lines, like every other expansion cap
// (toolstat.go's table). Twelve wrapped lines is a long and specific prompt
// read whole; past that the block folds and "… N more lines" lifts it, which is
// the same gesture the rest of this surface answers a long expansion with.
const promptWindow = 12

// imageCallRows is the prose head of an open `generate_image` step: the prompt,
// then one dim line per other input, then the model line. It returns the rows
// it kept and how many it dropped, exactly as [app.cap] does, because the
// prompt is the one part of it that can run long.
//
// It answers nothing at all for a call whose arguments never arrived — a begin
// with no payload, which is what a provider that does not stream tool calls
// sends — and then the expansion is the picture alone, exactly as it was.
func (a *app) imageCallRows(e *entry, width int) ([]string, int) {
// IT ANSWERS FOR ONE HAND AND SAYS SO ITSELF, the way [app.commandRows]
// answers only for a call carrying a command. The running expansion asks
// this block without knowing which tool it is drawing (toolview.go's
// [app.liveDetail]), and the other making verbs carry a `prompt` field too —
// a video's, a tune's — which this block's words are wrong about: "drawn
// with" is a sentence about a picture. So the gate is here, once, rather
// than in each caller.
if e == nil || e.tool != "generate_image" {
return nil, 0
}
fields := argsOf(e.detail.Args)
rows, more := a.cappedPromptRows(e, argString(fields, "prompt"), width)
for _, line := range imageInputWords(fields) {
rows = append(rows, a.pal.dim(fit(line, width)))
}
if line := imageModelLine(fields, e.detail.Output); line != "" {
rows = append(rows, a.pal.dim(fit(line, width)))
}
return rows, more
}

// cappedPromptRows is the prompt, quoted and wrapped to the block's width.
//
// It is QUOTED because it is somebody's own words sitting among facts about
// them, and the quotes are the plain ASCII pair rather than the typographic
// one: this block is drawn on every terminal this surface reaches, including
// the ones with no repertoire beyond ASCII, and a quote mark is not a mark from
// the vocabulary that could answer for those (docs/design/icons/DESIGN.md
// governs glyphs, and this is punctuation).
//
// A call with no prompt draws nothing rather than an empty pair of quotes.
func (a *app) cappedPromptRows(e *entry, prompt string, width int) ([]string, int) {
// The prompt is SOMEBODY ELSE'S BYTES and reaches this block with whatever
// the model put in it, so every line of it goes through [drawableLine] —
// which drops the newline with the other control bytes. The paragraph
// breaks the author wrote are kept by splitting first and rejoining after,
// because a prompt written in stanzas is a prompt read in stanzas.
lines := strings.Split(strings.ReplaceAll(prompt, "\r\n", "\n"), "\n")
for index, line := range lines {
lines[index] = drawableLine(line)
}
prompt = strings.TrimSpace(strings.Join(lines, "\n"))
if prompt == "" {
return nil, 0
}
wrapped := wrap(`"`+prompt+`"`, width)
painted := make([]string, 0, len(wrapped))
for _, line := range wrapped {
painted = append(painted, a.pal.ink(fit(line, width)))
}
if e.full || len(painted) <= promptWindow {
return painted, 0
}
return painted[:promptWindow], len(painted) - promptWindow
}

// imageInputWords is every OTHER input the call carried, one short line each,
// in the order a person asks about them: the shape of the frame, then the
// pictures it was told to work from.
//
// An input nobody gave is not a line. There is no "default" row and no "—":
// leaving `size` out means the image model's own default, and a surface that
// wrote the word "default" would be claiming to know what that is.
func imageInputWords(fields map[string]json.RawMessage) []string {
var out []string
// The two frame arguments are one fact — how big, or what shape — and the
// tool passes whichever arrived through untouched, so the line says the
// caller's own spelling rather than a normalized one.
if size := strings.TrimSpace(argString(fields, "size")); size != "" {
out = append(out, size)
}
if ratio := strings.TrimSpace(argString(fields, "aspect_ratio")); ratio != "" {
out = append(out, ratio)
}
if from := argPaths(fields, "reference_paths"); len(from) > 0 {
out = append(out, "from "+strings.Join(from, ", "))
}
return out
}

// imageModelLine is what the block says about which model drew.
//
// THREE SHAPES, AND WHICH ONE APPEARS IS A FACT RATHER THAN A CHOICE:
//
// - `drawn with paint-5` — the call left the model out, or asked for the name
// it got. This is the ordinary line.
// - `asked for best · drawn with paint-5` — the call asked in its own words
// and the catalog answered with a name. Both halves are worth keeping: the
// word is what somebody would repeat, the slug is what actually ran.
// - `asked for seedream` alone — the call is still running, so the request is
// known and the answer is not yet.
//
// A call that named no model and has not finished produces NOTHING, which is
// the emptiness law: the default is resolved inside the tool against the live
// catalog, and a surface that printed its own guess at it would be inventing a
// name three layers from anyone who could check it.
func imageModelLine(fields map[string]json.RawMessage, output string) string {
asked := strings.TrimSpace(argString(fields, "model"))
drew := modelui.ModelWord(generatedPictureModel(output))
switch {
case drew == "" && asked == "":
return ""
case drew == "":
return "asked for " + asked
case asked == "" || strings.EqualFold(asked, drew):
return "drawn with " + drew
default:
return "asked for " + asked + " · drawn with " + drew
}
}

// argPaths reads one array-of-strings argument as the drawable words a row can
// show. A field that is not an array of strings yields nothing rather than its
// raw JSON, because this block's whole promise is that it is prose.
func argPaths(fields map[string]json.RawMessage, key string) []string {
raw, ok := fields[key]
if !ok {
return nil
}
var values []string
if err := json.Unmarshal(raw, &values); err != nil {
return nil
}
out := make([]string, 0, len(values))
for _, value := range values {
if value = strings.TrimSpace(drawableLine(value)); value != "" {
out = append(out, value)
}
}
return out
}
Loading
Loading