Make Chat aware of the viewed model, and trace exports to their source - #246
Open
OmarB97 wants to merge 10 commits into
Open
Make Chat aware of the viewed model, and trace exports to their source#246OmarB97 wants to merge 10 commits into
OmarB97 wants to merge 10 commits into
Conversation
Rigged models (from the rigging extensions, or any GLB with clips) render frozen in bind pose today: the viewer loads the glTF but never creates an AnimationMixer, so the clips it just parsed never run. That reads as a failed rig rather than a missing feature. Create a mixer for models that carry clips, tick it from useFrame, and dispose it with the model. When a file carries more than one clip, offer a small picker so each motion can be reviewed without leaving the app. Co-Authored-By: Claude Fable 5 <[email protected]>
Asset rows now render their pre-rendered <name>.thumb.png beside the filename instead of showing bare text, falling back to the existing text-only row when no thumbnail exists. The Library panel can also be resized by dragging its edge, with the chosen width persisted across restarts.
…' into feat/desktop-ergonomics
npm ci fails on main because the committed lockfile is stale against package.json; npm install (what upstream CI itself falls back to) brings it back in sync. Committing the result so ci is usable again on this branch. Co-Authored-By: Claude Fable 5 <[email protected]>
Ctrl/Cmd with +, -, or 0 now scales the whole window like a browser, clamped to [-2, 4] and persisted to <userData>/ui-zoom.json so it survives restarts. This extends the existing before-input-event handler (where the macOS quit shortcut already lives) and calls webContents.setZoomLevel directly, deliberately not CSS zoom: a CSS transform scales the rendered pixels without telling the 3D viewport, so pointer math there (raycasting, gizmo dragging) goes wrong as soon as the page is scaled. Co-Authored-By: Claude Fable 5 <[email protected]>
The panel's resize handle was a 1px strip with no resting affordance — easy to miss entirely. Widen it to 2px with a visible resting tint plus a stronger hover/active highlight, mark it up as a proper separator (role, aria-orientation, title) matching the Library panel's handle, raise the width cap from 520 to 900 so wide parameter forms have room, and persist the chosen width to localStorage (modly-panel-width) the same way the Library panel already remembers its own width. Co-Authored-By: Claude Fable 5 <[email protected]>
- Parameter labels used a truncating 5rem column that clipped anything longer than a couple of words; widen it to 8rem and let it wrap. - The prompt textarea was a fixed 3-row, non-resizable box; make it 14 rows by default and user-resizable (resize-y, 14rem min-height). - Say plainly which Generate inputs are required vs optional: "Your sketch or photo -- required" and "Extra details -- optional". - Add a text param type (a plain input, unlike the existing string type folder picker) so free-text fields like the exporter new name, project, and tags params render as usable text boxes instead of falling through to a numeric input. - Render clickable tag-suggestion pills beneath any tags param: slugified words from that node typed name/project fields, plus a small fixed game-asset vocabulary, deduped and capped at 10. Clicking a pill toggles it in the comma-separated value; no network calls. Co-Authored-By: Claude Fable 5 <[email protected]>
The clip picker was a raw select mounted via drei Html at the model origin, recreating the whole AnimationMixer every time the clip changed (a hard cut, not a blend) and easy to lose behind the scene. Replace it with a proper bottom-left overlay component (MotionBar), matching the existing ViewerToolbar/stats-overlay idiom: - The mixer and active AnimationAction now live for as long as the model does; switching clips fades the old action out and the new one in over 0.25s instead of recreating anything. - GltfMeshModel reports its clip list up to Viewer3D (the same onStats/onObject callback pattern already used here), which owns the active index and renders MotionBar as a plain sibling of the Canvas. - Zero clips renders nothing, exactly one clip shows its name read-only, more than one gets a labelled dropdown plus a Next button that cycles with a cross-fade. All playback loops. Co-Authored-By: Claude Fable 5 <[email protected]>
The built-in exporter named every file export-<epoch-ms>.glb, so a folder of work told you nothing. Add three optional params: model_name (used to slug the file name, falling back to a dated model-YYYY-MM-DD-HHMM name), project (files land under Exports/<project-slug>/ so a game groups together instead of one long list), and tags. Output is always <slug>-<4-char-unique>.<ext> so re-exports never collide. Tags are the operator typed list plus what the file itself can tell: rigged (has skins), animated plus clip-<name> per animation, textured, and one of low-poly/mid-poly/high-poly by triangle count, plus the project slug. Tags are written both as a <file>.tags.json sidecar and embedded under the glTF asset extras (extras.modly), so they travel with the file either way. Ported from the already-working implementation at /mnt/tank/modly/app/extracted/resources/builtin-extensions/mesh-exporter on the operator machine. Co-Authored-By: Claude Fable 5 <[email protected]>
The agent had no idea what was actually on screen beyond a mesh path and triangle count -- it couldn't tell you which clip was playing, what the model was named, or what game it belonged to. Extend both ends of that pipeline: - ChatPanel now mirrors the viewer's selected clip and full clip list (Viewer3D reports them into appStore alongside the existing meshStats/meshSelected pattern) into the /agent/chat context, so requests like "make the stroll slower" resolve without re-asking. - agent.py resolves the current mesh's name/project/tags from its .tags.json sidecar, falling back to the GLB's own embedded extras.modly when no sidecar exists, and renders them into the system prompt in the same style as the existing scene-context lines. Lineage: when mesh-exporter's input traces back to an existing workspace asset, record derived_from in the sidecar -- the source's workspace- relative path plus its name if it had one. The workflow runner threads a sourceAssetPath through NodeOutput/ProcessInput from wherever a mesh first resolves (the Load 3D Mesh node) down to whichever node eventually exports, so multi-step chains still carry it. Re-deriving from an already -derived asset keeps just the immediate parent plus the chain's root (read from the parent's own derived_from, if any) instead of growing an unbounded list. Surfaced in the chat context alongside name/project/tags. Also fixes an open question: the rig-and-animate extension's "Describe the movement" param sets multiline: true, a flag ParamField never checked -- it fell into the plain "string" case and rendered as a single -line input with a stray folder-browse button. ParamField now renders any multiline-flagged param as a resizable textarea (matching the existing "Extra details" box), independent of its base type. Co-Authored-By: Claude Fable 5 <[email protected]>
This was referenced Jul 28, 2026
Open
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Builds on feat/desktop-ergonomics (PR #245, still open) -- same head-branch-per-PR-against-main pattern as #242/#243/#244. Three related changes, all scoped to the Generate area and the mesh-exporter:
Richer chat context. `/agent/chat` now also receives the selected animation clip name, the full list of available clips, and the current model's name/project/tags (and lineage, see below). Clips come from Viewer3D via a new mirrored pair on `appStore` (`motionClips`/`activeClipIndex`), following the same "set by Viewer3D, read elsewhere" pattern already used for `meshStats`/`meshSelected`. Name/project/tags are resolved server-side in `agent.py` from the model's `.tags.json` sidecar, falling back to the GLB's own `extras.modly` when no sidecar exists -- the renderer only ever sends the raw signals it already has (mesh path, clip state); it does not itself read sidecars or parse GLBs.
Lineage. When mesh-exporter's input traces back to an existing workspace asset, the sidecar now also records `derived_from`: `{ parent, root }`, each with a workspace-relative `path` and `name` (from the source's own sidecar/extras, or `null` if it has none). `root` chains through the parent's own `derived_from.root` when present, so a v4 derived from a v3 derived from a v2 derived from v1 still only ever stores two entries, not a growing list. Plumbing: `workflowRunStore` now threads a `sourceAssetPath` through `NodeOutput`/`ProcessInput` from wherever a mesh first resolves (a "Load 3D Mesh" node) down to whatever node eventually calls the exporter, so multi-step chains carry it correctly.
Multiline param fix. The rig-and-animate extension's "Describe the movement" param sets `multiline: true`, a flag `ParamField` never actually checked -- it fell through to the `"string"` case and rendered as a single-line input with a stray folder-browse button next to it. `ParamField` now checks `param.multiline` first and renders a resizable textarea (matching the existing "Extra details" box) regardless of the param's base type. Confirmed on screen before and after the fix (see Verification).
Files changed
Verification
Built, packaged (`npm run package`, AppImage target), and installed on the operator's machine the same way the current build was installed (asar extracted into `resources/app`, original parked as `app.asar.orig`), replacing the running app.
```
Current mesh path: Exports/lineage-test/squid-lineage-test-v4-nvvo.glb
Current mesh triangles: 29,360
Current animation clip: idle_breathe
Available animation clips: idle_breathe, startle_flick
Model name: squid-lineage-test-v4
Project: lineage-test
Tags: lineage-proof, rigged, animated, clip-idle-breathe, clip-startle-flick, textured, mid-poly, lineage-test
Derived from: squid-lineage-test-v3 (originally: Exports/sketch-to-3d/squid-seamless-loop.glb)
```
Test plan