UE Copilot Overlay

An always-on-top desktop overlay that answers Unreal Engine questions and can look at a screenshot of your editor to tell you where you are and what to do next.

This is v1: overlay window + chat + screenshot-aware guidance (steps 1–4 of the roadmap). Blueprint-graph verification against the actual project (step 6) is not built yet — see "What's not here yet" below.

Setup

1. Install dependencies (already done if you're reading this from the initial scaffold):

   npm install
   

2. Copy .env.example to .env and add your Anthropic API key:

   cp .env.example .env
   

   Then edit .env:

   ANTHROPIC_API_KEY=sk-ant-...
   

3. Start the app:

   npm start
   

A small chat panel appears in the top-right corner of your screen, always on top of other windows.

Using it

• Type a question and press Enter (Shift+Enter for a newline).
• Click the 📷 button to attach a screenshot of your current screen (the overlay hides itself for a moment so it doesn't appear in its own screenshot). Then ask something like "am I in the right place to create a Blueprint?" or "what should I click next?".
• Ctrl+Shift+U toggles the overlay's visibility from anywhere, even while Unreal Editor is focused.
• Click ✕ in the title bar to quit.
• Drag the title bar to reposition the window.

Architecture

• src/main.js — Electron main process. Creates the always-on-top overlay window, registers the global toggle shortcut, captures screenshots via desktopCapturer, and brokers chat requests over IPC.
• src/preload.js — Context-isolated bridge exposing a minimal window.ueCopilot API to the renderer (no direct Node/Electron access from the UI).
• src/anthropicClient.js — Builds the Claude request (system prompt tuned for Unreal Engine assistance, optional image content block) and streams the response using the Anthropic SDK.
• src/renderer/ — The chat UI (plain HTML/CSS/JS, no framework or bundler).

Model: claude-opus-5, streamed. Conversation history is kept client-side in the renderer (text only) and resent each turn; screenshots are only sent for the turn they're attached to, not stored in history, to keep context size down.

What's not here yet

• Blueprint/graph verification (step 6 from the original plan): currently "verify this is wired correctly" relies purely on the vision model reading a screenshot, which is unreliable for dense node graphs. A more accurate version would also read the Unreal project's actual Blueprint data (e.g. via Unreal's Python API) rather than just pixels — see the architecture discussion this project started from.
• Packaging/distribution (electron-builder) — for now this only runs via npm start from source.
• Persisted chat history across restarts.