Skip to content

About

A compiler-native IDE for the Datara language

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

Datara Studio

A working IDE for Datara. Human-first, compiler-native, and small enough to read in one sitting.

datara-studio/
  src/main.dtr        entry point, accept loop, routing
  src/http.dtr        HTTP/1.1 over raw sockets, request parsing, JSON escaping
  src/api.dtr         endpoints: tree, read, write, new, run, action, layout
  src/explorer.dtr    the filesystem browser, and the console-codepage fix
  ui/app.js           the interface
  src-tauri/          the desktop shell - Tauri is only the window
  PORTING.md          every compiler workaround and its removal plan

Install it

Releases carry one installer per platform.

Windows is the only one that has ever been built and run. The Linux and macOS targets are wired into .github/workflows/release.yml, which has not had its first run, and no Linux or macOS artifact exists yet - release v0.3.0 carries exactly one file, Datara.Studio_0.3.0_x64-setup.exe. Read the last two rows as "the build is configured", not "the build is known good".

Platform Asset Status
Windows x64 Datara Studio_<version>_x64-setup.exe (NSIS) built here and launched
Linux x64 datara-studio_<version>_amd64.deb workflow added, never run
Linux x64 datara-studio_<version>_amd64.AppImage workflow added, never run
macOS, Intel and Apple Silicon Datara Studio_<version>_universal.dmg workflow added, never run

macOS is built universal rather than for the runner's own architecture, because a macos-latest runner is Apple Silicon and a default build would produce a disk image that does not open on an Intel Mac.

None of the installers contain forgen. This is a compiler-native IDE: the shell starts the Datara server, and the server is forgen run src/main.dtr. The installers ship the shell, the studio's Datara sources and the built interface, and they need the compiler on PATH to do anything. Without it the window opens on the splash page and stays there. There is no public download for forgen yet, so installing the studio means having the toolchain already.

The .deb and .dmg install the app and register .dtr with it. The AppImage needs no installation at all - chmod +x it and run it.

Run it

From a release, the installed app is enough: launch it from the menu, or on Linux run the AppImage directly.

From a clone, there is one launcher per platform and they do the same four things - build the single-file interface if it is missing, start two servers, start the optional AI companion, open the browser, and then watch the pair, restarting it if neither port answers:

start.cmd          # Windows
./start.sh         # Linux and macOS

Two servers is not redundancy for its own sake: the Datara runtime cannot time out a socket, so one idle connection can wedge a server permanently, and the interface falls back to the second port. See PORTING.md SEAM-6.

The long way, which is also the way to see what the server is doing:

cd datara-studio
forgen run src/main.dtr

Then open http://127.0.0.1:7878.

What runs, and where.

Port What Started by
7878 the server the browser opens first either launcher, or forgen run src/main.dtr
7879 the second server, for when 7878 is wedged either launcher
7880, 7881 a fresh pair, when a killed server left a bound-but-silent socket on the first two start.cmd; start.sh picks from the same four before it starts anything
7890 the optional AI companion either launcher, if the companion is found

The interface tries all four in order and remembers which one answered, so a reload does not re-pay the failed attempts. Any of these can be moved: DATARA_STUDIO_PORT=7879 forgen run src/main.dtr, and the launchers read DATARA_STUDIO_PORT_B, DATARA_STUDIO_PORT_C, DATARA_STUDIO_PORT_D and DATARA_AI_PORT the same way.

Prerequisites.

Needed for Notes
forgen everything - it compiles the server on PATH, or set DATARA_FORGEN to it. start.sh and the shell both search the conventional install locations (~/.local/bin, /usr/local/bin, /usr/bin, %LOCALAPPDATA%\Programs\Datara\bin) before giving up
Node.js rebuilding ui/studio.html, and the launchers' search for the AI companion not needed to run a release
Python 3 the AI companion, which is optional the IDE runs without it, with no suggestions
Rust + a C++ toolset building the desktop shell from source see below

Nothing else: the servers are native binaries and the interface is one file with no subresources.

Where the AI companion is looked for. The launchers do not guess: they call scripts/find-companion.mjs, and it prints the first directory that actually contains forgen_ai/ide_daemon.py, in this order - FORGEN_AI_DIR, then $ROOT/../python, $ROOT/../../python, $HOME/.datara/python, then siblings of the studio's parent. Set FORGEN_AI_DIR to the directory that contains forgen_ai/ to skip the search. That variable is not the same thing as the aiDir setting in the IDE's Settings panel, which wants the directory containing python/ - one level higher.

The desktop build

src-tauri/ is a Tauri 2 shell. It is a window and nothing else - no bundled interface, no filesystem access, no commands of its own - because the point of this IDE is that it can read the project and run the compiler, and a frozen asset bundle could neither. The shell starts the servers, waits for a port to answer, opens the window at it, and kills the children on exit.

It builds and runs on Windows, Linux and macOS. The shell has no platform-specific code paths except two: where it looks for forgen, and CREATE_NO_WINDOW on the child processes, which is a Windows flag and is compiled out elsewhere. cargo check --all-targets and cargo test run on all three platforms in CI.

Building it. Windows:

start-tauri.cmd                     # builds the shell if it is missing, then runs it

or by hand, on any platform:

node scripts/build-wasm.mjs         # ui/vendor/textcore.js - gitignored, and inlined
node scripts/build-icons.mjs        # src-tauri/icons/ - gitignored, and bundled
node scripts/build-ui.mjs && node scripts/build-ui.mjs --core
tauri build                         # or `cargo tauri build`, see below

Those first two lines are not optional and the order matters. Both write files that are in .gitignore and that the steps after them read: ui/vendor/textcore.js is inlined into the interface by build-ui.mjs, which fails outright if it is missing, and src-tauri/icons/ is a bundled resource. A fresh checkout has neither.

tauri or cargo tauri, pick one. They are different executables and the project does not install either for you:

You have Run Notes
cargo install tauri-cli --version "^2" cargo tauri build compiles the CLI from source, minutes
npm install -g @tauri-apps/cli@^2 tauri build a prebuilt binary, seconds - but it provides tauri and not cargo-tauri, so cargo tauri still fails

The release workflow uses the npm one. On Windows, start-tauri.cmd checks for either.

The output lands in src-tauri/target/release/bundle/. On Linux, install the Tauri system dependencies first - the bundler fails at the point it links the webview, which reads as a Rust error rather than a missing system package:

sudo apt-get install -y libwebkit2gtk-4.1-dev libgtk-3-dev librsvg2-dev \
  patchelf libayatana-appindicator3-dev libxdo-dev libssl-dev libfuse2

--bundles takes only the formats the host can produce and rejects anything else - measured, cargo tauri build --bundles nsis,deb,appimage,app,dmg on Windows exits with invalid value 'deb' for '--bundles'. The bundle.targets list in tauri.conf.json is not restricted that way: the bundler skips inapplicable targets with a warning, so one list serves all three platforms. That is why tauri.conf.json names dmg and app even on a Windows checkout.

On Windows, scripts/msvc-env.sh must be sourced first for anything that produces a native binary, because MSVC's link.exe has to precede Git-Bash's GNU link. The JIT path (forgen run) needs no setup.

The one thing the page asks the shell for is the window itself: minimise, maximise and close. That is a single capability file (src-tauri/capabilities/default.json) granting exactly those permissions and nothing else.

The title bar is the interface's, not Windows'. The window is created undecorated (.decorations(false)), and the interface draws the bar - the mark, the open file's name, and three caption buttons in the app's own palette. Three pieces have to agree, and missing each one breaks it differently:

Piece Missing it means
.decorations(false) in src-tauri/src/main.rs two title bars
capabilities/default.json a bar whose buttons the access-control list refuses
ui/tauri-bridge.js no bar at all, and a window with no chrome

The capability is not optional even though the server is on loopback: Tauri treats every origin that is not its own as remote, and the studio page is served by the Datara server at http://127.0.0.1:7878. Without the remote.urls list the bar draws and its buttons silently do nothing, which reads as broken buttons rather than a missing permission.

tauri-bridge.js is also what keeps the browser build honest: it creates window.__DS_SHELL__ only when window.isTauri is set, so in a browser the bar is never drawn and .shell keeps its three rows. drive.mjs asserts both halves.

Nothing needs to be running first. The window starts its own servers, on the first two of the four ports that are free, and probes them itself.

The window opens immediately, on a bundled splash, and does not wait for the server. The splash polls all four ports and hands the window over the moment one answers, and only admits failure after nine seconds. The first version waited up to twenty seconds before creating any window at all, so a slow start looked like nothing happening - and the wait was never buying anything, because the page can do the waiting better than the shell can.

No console windows. start.cmd launches the servers through scripts/hidden.vbs and the shell passes CREATE_NO_WINDOW, because a GUI-subsystem process spawning a console application makes Windows create one - so opening the IDE used to flash two black windows and leave them in the taskbar for as long as it was open. There is no flag that means "minimised and invisible"; /min only minimises.

scripts/build-icons.mjs generates every piece of icon artwork from the one geometry in scripts/mark.mjs, which is the same shape ui/icon.svg describes. It has no image-library dependency: the mark is a rounded square, two strokes and a dot, so it is rasterised from signed distances and encoded as PNG with zlib. The build fails if ui/icon.svg stops containing the mark's three colours, because that is the one way the artwork and the drawing can silently separate - which is exactly what had happened.

scripts/build-desktop.cmd checks for Rust and the MSVC C++ toolset and says which one is missing.

What works today

Area Status
Problems the compiler's own diagnostics, underlined at the exact column while you type, with no AI at all; the = help: line is kept
Project problems forgen check over the whole project - every error in every module, grouped by file, click to jump to the line. Aimed at the project that governs the open file, because a check pointed above it reports every import as a missing package. 301 ms measured
Typing aids auto-indent (plus one level after an opening brace), bracket and quote pairs, backspace inside an empty pair, Ctrl+/ comment toggle, Tab / Shift+Tab indent and outdent a selection - each one switchable in settings
Go to definition F12 or Ctrl+click, across the whole workspace
Hover help rests on a name and shows where it is declared, or what the builtin does - including the traps
Completion this file's symbols, keywords, types and builtins, local and instant; the companion folds in when present
Snippets accepting a declaration keyword writes the shape it is always followed by. fn Tab gives fn name() -> Int { ... } with the name selected, so typing replaces it, and the body indented to wherever you typed. 19 of them: fn struct class entity record behavior trait impl enum type if while for match unsafe comptime defer use mod. Keywords with no shape (let, return, pub, true) still insert as the plain word - a skeleton on true would be noise
Ghost text Tab accepts; the suggestion appears at the caret before you finish typing
Resizable panels drag the dividers, fold either side; widths are remembered
Zoom Ctrl+wheel in the editor, Ctrl+0 to reset; survives a reload
Language providers the editor does not know what Datara is - see below
File tree every file and folder, language icons, drag a file onto a folder to move it
Create one create control, File or Folder, named inline in the folder you are working in
Dialogs every question is drawn by the interface itself - delete, discarding unsaved changes, rename, Save As, naming a new project. There is no window.confirm, no window.prompt and no platform message box left anywhere, so nothing in the window looks like a different application
Status bar three columns, not a row of loose text: the open file and the server's word on the left, the git branch in the middle (dirty count beside it, click for the Project panel), and problems, caret position and the output drawer on the right
Open file any file on the machine, not only the ones in the workspace
Open folder editable path box, drive row, new folder, failures that say what went wrong
Reopens where you left off last file, last workspace, and the port that last answered
Editor textarea over a highlighted layer, line numbers, dirty marker, semantic rail
Breadcrumb shows the declaration the caret is inside, not just the path
Check / Build / Run / Lint / Audit forgen invoked from the server, output in the output pane
Find references Alt+F7, a real workspace search, or the search box
Layout inspector the layout forgen will choose per struct, and every field-name conflict
Companion optional, starts itself in the background, one dot in the chrome, toggle in settings
Degrades without AI completely. Problems, Structure, Project and Layout all work offline

Keyboard: Ctrl+S save, Ctrl+N new file, Ctrl+B build, Ctrl+Shift+B check the open file, Ctrl+Enter run, Ctrl+P file palette, Ctrl+, settings, F12 or Ctrl+click go to definition, Alt+F7 references, Ctrl+/ comment toggle, Ctrl+wheel zoom, Ctrl+0 reset zoom. Tab resolves in one fixed order: it accepts the open completion list first - and a declaration keyword there writes its whole shape, not the word - then a ghost suggestion, and only then does it indent (outdent with Shift, or indent every line of a selection). Esc dismisses the list. Ctrl+P also offers Check the whole project - forgen check over every module.

The language provider seam

The editor must not know what Datara is. Everything language-specific sits behind one object - DATARA_LANG in ui/app.js - which owns the keywords, the types, the builtins, the hover documentation, symbol extraction, the indent string and the compiler check.

There are two implementations. The second is PLAIN_LANG, which claims no extensions and offers nothing: open a .py file and the editor still works normally, but with no Datara keywords suggested, no Datara symbols in the outline, and no compiler asked to check a file it cannot read.

That second provider is not decoration. A boundary with one implementation is an abstraction nobody has tested, so the plain-text provider exists to prove the seam is real - and 20 assertions hold it to that, including that a plain file never calls the compiler at all.

Adding a language is one object. Adding one with a compiler is one object with a real check. This is the first boundary the project found on its own rather than designed up front, which is the order the roadmap asks for: extract the kernel, do not design it.

The right panel is ordered offline first - Problems, Structure, Project, Layout, then Chat, AI and Generate - because a person who never starts the companion should still get a useful panel. See docs/DATARA-ERGONOMICS.md for the measurements and for what is still missing.

The AI companion

The IDE talks to forgen_ai (D:\ryan) directly over HTTP on port 7890. It is optional: with it absent the status dot goes grey and everything else keeps working. That is the design rule from the spec - no mandatory AI - enforced by architecture rather than by policy.

Start it:

cd D:\ryan
python python/forgen_ai/ide_daemon.py --port 7890

Then the IDE picks it up within five seconds.

PORTING.md SEAM-4 explains the one change made in that project: the neural stack is now imported lazily, so the daemon starts and serves completions without torch. No endpoint needs it now - chat included, which previously answered neural stack unavailable here because it was the one path that did.

Which model is running, and why the answer is on disk

The companion ships two things under the word "model", and the interface used to report the wrong one:

file state
NanoCodeModel data/nano_code_model.json, 131 KB active. An order-3 n-gram over verified Datara; this is what generates code, and it needs no ML runtime
NanoLM data/nano_lm.pt, 6.8 MB dormant. A 1.70M-parameter transformer that measured worse than the n-gram on both holdouts (NEURAL.md §7), so nothing is wired to it

/health used to announce FORGEN-12.6M-MoE, naming a network whose checkpoint was deleted. It now reports what is actually loadable, per candidate, with the reason: ngram-datara active, nanolm-transformer present but dormant: torch not installed; no-ship build (measured worse than the n-gram). The verdict comes from python/forgen_ai/model_registry.py, which probes the directory rather than reading a preset name - so installing torch tomorrow changes the report rather than contradicting it.

Chat, and why it replaced the Generate box alone

Generate is one shot: it composes a program from verified fragments and writes it into the open file. That is still there. What was missing is a surface where the previous turn exists, because a follow-up question has no meaning without one - and the companion has had a /chat endpoint all along with no way to reach it from the IDE.

The Chat tab is that surface. It is served by python/forgen_ai/chat_engine.py in order of how much each answer should be trusted:

resolved by example reported as
the project index «какие функции имеют эффекты» VERIFIED, with Источник: forgen context …
exact arithmetic «сколько будет 2+2*10» VERIFIED, 2+2*10 = 22.
the compositional generator «напиши сортировку вставками» VERIFIED after forgen check, code in the transcript
the curated knowledge base version and feature questions SUPPORTED
honest refusal anything outside those SUPPORTED, and it lists what does work

Context is the feature, so it is visible and it is deletable. The card at the top of the tab shows how many messages are carried, where the context came from (compiler, manual or empty), and the topic the last answer established. Three controls act on it:

  • clear context - deletes the transcript and the attached context. It is final: the session records that the emptying was deliberate, so the project context is not silently re-gathered on the next turn. A clear button that refills itself is worse than no button, and there is an assertion for exactly that in both test suites.
  • undo - drops the last exchange, in the panel and in the companion.
  • use the project - attaches the compiler-gathered context again, which is the only way back once it has been cleared.

Follow-up inheritance is what makes it conversational rather than a log. «а теперь по убыванию» carries a modifier and no subject, so the previous subject is grafted onto it - сортировка + по убыванию - and re-planned. Generated descending, the output is genuinely descending:

=== Сортировка пузырьком по убыванию ===
Исходный:  64 34 25 12 22 11 90
Результат: 90 64 34 25 22 12 11

A cold session asking the same follow-up is refused rather than misread as a request for a bubble sort, which is the boundary that keeps inheritance from turning into guessing.

What the Generate panel actually does

Measured against the real companion, not inferred from the code. Five requests, each with an empty context and verify: true:

asked for task it chose confidence fragments
sort a list of integers sort 0.87 copy_ints, join_ints
read a file and count its lines file_stats 1.00 count_lines, count_words
compute the sha256 of a buffer generic 0.15 sum_ints
add two numbers generic 0.15 sum_ints
parse an integer from a string and return an option string_ops 0.47 is_palindrome, b2s

Two things are worth stating plainly, because the panel does not:

  • The task is chosen by keyword match against a small set of shapes. The first, third and fifth rows are the honest failure: asked to parse an integer, the companion wrote a palindrome checker, in Russian, as a console program with out statements rather than a library function. It reported confidence: 0.47 and verified: true while doing it.
  • verified: true is true and is not reassurance. Verification means forgen check compiled it - and it does: the generated palindrome checker passes forgen check 100% OK. The code is valid Datara. It is just not the code that was asked for, and nothing in the panel distinguishes "compiles" from "answers the question".

So the Generate panel is a fragment composer with a keyword front-end, not a model that understands a request. The panel now says so: the header no longer claims a token count the companion never sends (it read 3-gram · null tokens for everyone - see describeModel in ui/app.js), and shows retrieval and verification as the separate capabilities they are.

confidence is the field a caller should gate on, and nothing gated on it before this. A confidence below roughly 0.5 in the table above means the classifier did not really match; 0.15 means it fell through to generic.

Architecture

Three languages, each doing what it is fastest at. The split is the design, not an accident:

ui/app.js        React 18 + htm   the chrome: title bar, rail, tree, panels,
                                  status bar, command palette
ui/app.js        direct DOM       the text surface, mounted imperatively
crates/textcore  Rust -> wasm     the document, the line index, the lexer
src/*.dtr        Datara           files, the compiler, the layout inspector,
                                  and the server that serves all of the above

Rust owns the text, and here is the measured reason. On a 1 MB / 20k-line document:

operation time
load the document 4.1 ms
lex the whole document 5.6 ms -> 260001 tokens
verify the line index against a rebuild 4.4 ms
one edit in the middle 0.091 ms
one edit at offset 0 (worst case) 0.1 ms

node crates/textcore/test/test.mjs reproduces those numbers, and it asserts the incremental line index against a full rebuild after every edit - a fast path that is never checked against the slow path is a bug waiting to happen. 40 assertions.

React owns the chrome, and explicitly not the text. Re-rendering a 20k-line highlighted document through React on every keystroke is the single biggest performance mistake an editor can make. So the text surface is mounted once into a div React renders and then never touches, and updated imperatively. The rule: React renders things a person clicks.

No wasm-bindgen. The ABI is flat C over linear memory: write UTF-8 into a scratch buffer from alloc, call with (ptr, len), read results as a Uint32Array view over lex_ptr(). That keeps the module at 39 KB, removes a toolchain dependency, and makes the number of boundary crossings explicit - which matters, because that is where wasm speed is lost.

The AI is a separate process (port 7890) and the browser calls it directly. The Datara server never proxies it, so an AI failure cannot take the editor down.

Everything that touches the project goes through the Datara server, which is the only component holding a capability grant.

Build and verify

bash scripts/build.sh

Nine stages, 376 assertions, all green:

stage assertions what it proves
wasm build - the module compiles and embeds
single-file interface + icons - the Tauri icon set is generated from the one geometry in scripts/mark.mjs, and React, htm, app.js and the wasm core inline into one ui/studio.html with zero subresources, with the icon inlined as a data URI. Builds both variants - the full studio and --core without the companion. Fails the build if the mark beside a .dtr file is backed by an icon under 32 px, or if ui/icon.svg no longer matches the generated artwork - see the icon notes below
Rust text core 40 the incremental line index matches a full rebuild after every edit
highlighting pipeline 40 the token-to-line split is lossless on empty input, trailing newlines, unterminated strings, Cyrillic and emoji, and every comment token starts with // and stops at its own line - which is what caught the byte-offset bug
interface renders 192 the component tree renders without throwing, every piece of chrome is present, the create name rule holds, an empty folder survives into the tree, the compiler's real coloured output parses into line, column and span, and a project check is aimed inside the workspace
the editor, driven as the app drives it 74 text, highlight layer, gutter, every editor layer and textarea height agree, and every editing aid leaves the text, the highlight layer and the caret agreeing. 19 of these drive the snippet table: that each keyword expands to a body that parses, that the placeholder is selected, that the caret lands inside it - the unsafe one inside the quotes - and that an indented keyword indents its body to match
boot the built artifact - the shipped ui/studio.html mounts in a DOM, shows the title screen, and defers the editor until a file is open
Datara server - forgen check reports 100% OK

Then:

forgen run src/main.dtr     # open http://127.0.0.1:7878

What is verified and what is not. The text core is verified by execution: the numbers in the table above are measured, not estimated. The interface is verified by structure in build.sh, and separately by pixels and by gestures:

node ui/test/shoot.mjs http://127.0.0.1:7878 shots        # real Chromium, writes PNGs
node ui/test/drive.mjs http://127.0.0.1:7878 shots/drive  # real gestures, then reads the bytes off disk
node ui/test/chat.mjs  http://127.0.0.1:7878              # the chat surface, against a live companion

None of these is part of build.sh, because they all need a running server and a Playwright install. shoot.mjs drives the real page in a real engine at a real viewport; drive.mjs right-clicks the tree, saves with Ctrl+S and presses Run, and then reads the result off the filesystem instead of trusting the screen. It currently makes 70 such checks, and because it walks every panel tab and drives every dialog, it is the only thing here that would notice a control that kills the window when you use it. It is also the only thing that types fn into a real browser and presses Tab - the structure suite can prove the table is right, but only a real key event proves the key is wired to it.

chat.mjs exists because chat is the kind of feature that passes at the API layer and does nothing in the interface: the companion's own gate (D:\ryan\tools\verify_chat.py, 33 assertions) proves /chat and /context behave, and cannot prove the IDE reaches them or that the transcript renders. So it drives the panel - sending a question, a code request and a follow-up, then pressing clear context and asserting that the transcript and the readout both emptied and stayed empty. The companion is optional, so with it offline the suite asserts the honest empty state instead of going red for the wrong reason.

The LLVM backend, and the two bugs that hid behind "falling back"

forgen ships two backends: a fast-to-compile native one (Cranelift) and an optimising --llvm one. Asking for LLVM used to appear to work and quietly did not. There were two independent causes, and both had the same symptom - a successful build - which is why neither was noticed.

One: the linker was shadowed. Git-Bash ships a GNU link.exe at /usr/bin/link.exe, and on a machine where that directory precedes MSVC's, every link the compiler attempted went to a Unix hard-link utility:

/usr/bin/link: extra operand '/DEBUG:NONE'
. Falling back to native Cranelift backend.

The fix is in src/toolchain.dtr, which finds MSVC and sets PATH, LIB and INCLUDE before the compiler runs. LIB and INCLUDE are not optional extras: with only PATH set, the right link.exe runs and then dies with LNK1181: cannot open input file "legacy_stdio_definitions.lib", because link.exe finds its libraries through LIB and nowhere else. The Windows SDK version is discovered, not hardcoded, and the whole probe uses dir_list and path_exists - no process is launched to find a linker.

Worth knowing: env_set stages a value for the next child process and env_get does not read it back (measured). That makes the prepend naturally idempotent - three calls leave exactly one entry in the child's PATH - and it means a "have I already done this?" guard would be dead code.

Two: the LLVM backend itself cannot compile list-heavy code. Even with the linker correct, a real program trips an internal verifier error:

llc.exe: error: '<cache>/main_*.ll:11939:23: error: '%v5244' defined with type
         'ptr' but expected 'i64'
  %v18804 = phi i64 [ %v5244, %bb2 ], [ %v18806, %bb9 ]
. Falling back to native Cranelift backend.

This is a codegen defect in forgen 1.4.2, not environment. It is isolated to a List-typed local reassigned inside a while loop, reproducible in the smallest form in docs/llvm-backend-bug.dtr:

fn f(items: List<Str>) -> Int {
    mut acc = items
    mut i = 0
    while i < items.len() {
        acc = acc.push(acc[i])   // <- removing this line lets LLVM finish
        i = i + 1
    }
    return acc.len()
}

Three of this project's six modules (main, api, toolchain) trigger it, so --llvm cannot build the studio until that is fixed upstream. It is a forgen bug, not a studio one, and the honest thing is to say so rather than work around it.

Measured, because the choice should be made on numbers. Same source, same MSVC linker, this machine:

native (Cranelift) --llvm
fib(30), recursive calls 472-490 ms 491-503 ms
40M-iteration arithmetic loop 531-564 ms 499-517 ms
build time 292-294 ms 1055-1998 ms
output size 374.5 KB 372-464 KB

LLVM wins a tight arithmetic loop by about 5% and costs 3.5x-7x the build time. Build time is paid on every Run; 5% is invisible outside a benchmark. So the default is the native backend - which is what "the fastest variant" actually means for someone using the IDE - and LLVM is one environment variable away:

FORGEN_LLVM=1 forgen run src/main.dtr   # every build in this session asks for --llvm

The build now reports which backend it used. Both facts are carried back from the server (linker_ok, fell_back) and named in the Run drawer:

forgen build  ·  ok  ·  exit 0  ·  took 894 ms  ·  native (LLVM ready: set FORGEN_LLVM=1)
forgen build  ·  ok  ·  exit 0  ·  took 2103 ms ·  Cranelift (LLVM fell back - linker missing)

A silent fallback used to be indistinguishable from success. It no longer is.

The icon is measured, not assumed

The mark beside a .dtr file and in the title bar is drawn at 13-16 px CSS, which is not 13-16 device px: a 16 px box is 20 device px at 125% display scaling, 24 at 150%, 32 at 200%, and the browser upsamples whatever source it has to fill it. Backing that with the ICO's own 16 px entry - the obvious choice - meant a 3x upsample on a 200% display. Measured, focus = variance of Laplacian / variance of luma, the mark drawn at 16 px CSS:

display scaling 16 px entry 32 px entry
100% 6.57 9.59
125% 1.51 12.46
150% 1.02 9.50
200% 0.34 8.01

The 16 px entry is the worst of the seven entries at every scaling, and it is not even best at 100%, so "the only size at which this artwork is crisp" - which is what the stylesheet claimed - was simply wrong. The 32 px entry covers 16 px CSS exactly up to 200% and won on the sum, so that is what backs it now, and build-ui.mjs refuses to build if that ever drops below 32.

The 128 px entry the title screen uses was checked the same way, and it is genuine: it differs from a proper 256 -> 128 resample by a mean of 1.77 of 255 per channel, against 7.62 for a 64 -> 128 upscale. It is not a fake upscale, which is what it looks like when magnified 4x.

There was a second mark, and the taskbar was showing it

The measurements above were all correct and they were all about the wrong artwork. assets/datara.ico - the file the window icon, the taskbar, the browser tab and the title screen were all built from - held an orange-and-yellow palm inside a yellow rounded square. Everything inside the interface drew something else: two light brackets around a mint dot, which is what ui/icon.svg, the Mark component and the mark beside a .dtr file all show. Two marks were live in one window and the one on the taskbar was the wrong one. Amber is also the reserved warning colour in docs/DESIGN-UI.md, and the accent is mint, so the icon was arguing with the interface in colour as well as in shape.

It had been made worse by a well-intentioned change: build-icons.mjs used to draw the mark itself, and was rewritten to read assets/datara.ico directly, on the argument that three approximations of one design are three designs. The argument is right. The conclusion was not, because that file was a fourth design rather than the first one.

The geometry now lives in scripts/mark.mjs, scripts/build-icons.mjs writes it to ui/mark.ico, and everything else derives from those bytes. build-ui.mjs reads ui/mark.ico rather than assets/, so a stale file in assets/ can no longer put a different icon in the browser tab than the one in the window.

Review icons on a light surface as well as a dark one. The first attempt at the small entries dropped the dark plate at 16 and 24 px, because a 64-unit corner radius lands on the pixel grid there and produced dirty corners. It looked right on the dark review image. On the light one the icon had vanished - the brackets are #E9E9EE, which is invisible on white, so all that remained was the mint dot. scripts/icon-sheet.mjs now renders every entry at its real size, magnified 8x, on both surfaces, and that is the image to look at:

node scripts/icon-sheet.mjs shots/icons
  shots/icons/sheet-light.png   and  shots/icons/sheet-dark.png

Zen mode

Ctrl+Shift+Z, or the button beside the panel toggle. Everything that is not the code goes - title bar, toolbar, explorer, right panel, breadcrumbs, status bar - and Esc brings it back. What survives is one 26px line: the file name, the cursor position, and a clickable error and warning count that leaves Zen and opens the Problems tab. Each of those can be turned off in Settings → Zen, along with soft wrap, a centred column width, extra breathing room and a larger face for reading. With the informational parts all off the bar collapses entirely, so this is configurable from "one quiet line" down to "literally just code".

The mode is carried on <html> as data-zen rather than as a React class, because the rules it drives are about the shell's own grid and a grid track cannot be removed from inside the grid it belongs to: hiding five children with display:none leaves the reserved tracks behind, and the code ends up in a 1fr row with dead space above and below. Collapsing grid-template-rows to a single row is what makes the editor actually fill the window.

Two things this got wrong first, both worth recording:

  • The shortcut was registered in the editor mount, which begins if (!edRef.current) return; - so with no file open the handler was never installed, the key arrived with nobody listening, and Zen looked broken rather than unavailable. A shortcut that switches the window's layout must not depend on the window's contents.
  • The centred column insets the text surface, not the text. A max-width on the code alone leaves the line-number gutter at the window edge and puts the caret in the wrong place, because the gutter is positioned from the surface's own box.

node ui/test/zen.mjs http://127.0.0.1:7878 <a-directory-with-dtr-files> drives all of it in a real browser and measures the result - 27 assertions covering the shortcut, the button, both exits, the grid collapse and every setting.

The core build, without the companion

node scripts/build-ui.mjs --core writes ui/studio-core.html, and the server serves it at /?core=1. It is the same interface with the AI tabs taken out: four compiler tabs instead of seven, no plug in the bar, no "start the companion" in the palette, no companion section in Settings. Everything else - the compiler panels, the editor, the tree, the explorer, Layout, Project, git and search - is identical, because none of it was ever an AI feature.

It is a flag rather than a second source file. ui/app.js reads window.__DS_CORE__ in four places, and a conditional in four places can be audited in one sitting where two copies of a 4000-line file cannot. The build refuses to emit if the flag is not settled before app.js loads - the failure that would otherwise produce a file named studio-core.html containing the full studio.

The panel tab strip, and a test that lied twice

Eight tabs want 404 px in a 264 px panel, so the row runs off the edge and scrolls. Two things were wrong with that, and neither was visible in a test until there was one that drove a browser.

The auto-scroll that keeps the selected tab in view was not clamped, so selecting the last tab scrolled the row past its own end and pushed the first one out the left edge. Measured at 1440x900: Problems reported at x 1066 with the strip's own left edge at 1176, sitting over the code column, outside its container. And nothing indicated overflow at all - no arrows, no fade, a scrollbar hidden by overflow:hidden - so the tabs at the end were found by accident or not at all. Now: the scroll is clamped, two arrow buttons appear when the measured row overflows and disable at each end, and the long labels are abbreviated (Issues, Symbols), which brings 380 px down to 268 and fits the default panel outright.

ui/test/tabs.mjs drives the real page and asserts reachability rather than appearance: every tab, when clicked, is brought fully into view; the row starts at scrollLeft 0; the arrows exist exactly when they are needed. Two traps in writing it, both kept in the comments because both cost a round:

  • getBoundingClientRect does not know about clipping. A child scrolled past its container still reports its geometric position, so "no tab may have left < strip.left" fails against a strip that is working perfectly. A screenshot settles it - at full scroll Project is cut mid-word at the panel edge, which is clipping. Assert on whether hiding is in effect, not on rects.
  • The first version asserted that every tab is visible at once, which contradicts the entire design. The claim is reachability.

The boot test caught one more real defect here: calling el.scrollTo(...) directly throws is not a function in a DOM without it, and because the throw lands inside React's commit the whole interface rendered as the crash screen. It is guarded now.

drive.mjs also covers the desktop shell's title bar without needing the shell binary: it stubs what Tauri injects before any page script runs, and then asserts that the real ui/tauri-bridge.js picks it up, that the bar appears with three controls, and that each control reaches the window plugin with the window label the shell actually created. The converse is asserted too, because it is the property easiest to lose silently - a browser must draw no title bar at all.

That pair is how the worst defects in this studio were found, and none of them was visible to a structure test:

  • Opening the Project tab destroyed the interface. Panel's proj() read git.branch, git.files and git.log - and git is state that lives in App's scope, which Panel was never given. The tab threw ReferenceError: git is not defined, the error boundary caught it, and the whole window was replaced by "The interface stopped". It had been that way for as long as the panel has existed, and every other check in this file passed while it was true, because nothing ever clicked a tab. Found by clicking one, and now asserted tab by tab.

  • A .wbtn of the same name squashed the title screen. The title bar's caption buttons were .wbtn, which the title screen already used for its three ways in. Both rules are one class deep, so the one declared earlier in the stylesheet won every property the later one did not also set - and the caption's width:46px landed on "Create new project", wrapping it into a column of single words. No assertion noticed; the screenshot did. The classes are now .capbtn, and drive.mjs measures the three cards' width so the next one is caught by the suite rather than by eye.

  • The editor was never mounted on a cold boot - the mount effect ran once, before there was a surface to mount into, and never retried, so the code area was empty while the breadcrumb and the status bar both reported the file as open.

  • The title screen could never appear, because the workspace defaulted to the server's own directory, which is truthy, so the "create a project / open a folder" screen was unreachable code.

  • The companion never suggested anything. Editor.onInput is installed by the mount effect, so the runAI it closed over was built before the first /health answer and had aiOnline frozen at false for the life of the page. The indicator still read "companion: on", because it renders from state rather than from the closure - so reading the code did not find it. Proved by running identical keystrokes with the surface mounted before and after the health poll: two /complete requests and ghost text in one ordering, nothing in the other.

  • The squiggles described the last save. /api/check is aimed at a path and the compiler reads that path, so a check issued with a dirty buffer reported the file as it was before you typed; with autosave on, nothing re-checked after the write, so an error you had just typed appeared only after the next keystroke. runCheck now writes the buffer first.

  • A new project was created inside the IDE's own source tree. Create new project asked for a name and never for a location, so with no workspace open the name became a path relative to the server's working directory - measured on a cold launch, demo-project landed in datara-studio/. Two things had to be true at once and both were: the folder dialog opened on the server's own directory, and the 8 s tree poll ran with no workspace, took the root: "." the server answers such a request with, and overwrote the folder the reader had just typed. Both are checked on disk now, because a project in the wrong place looks exactly like a project in the right place.

A screenshot alone would not have caught any of them; a screenshot plus a filesystem check did.

linkedom is a development dependency of the structure tests, installed with npm install in this directory. The studio ships no runtime npm packages: the interface is one generated HTML file with React, htm and the wasm core inlined.

Endpoints

All POST, plain-text bodies, JSON responses.

Endpoint Body Returns
GET / - the interface
GET /api/health - service info
POST /api/tree root path {root, files:[...], dirs:[...]} recursive, with the resolved absolute root
POST /api/read path {content, bytes}
POST /api/write path newline content {bytes}
POST /api/new path newline content creates the file, making its folder first
POST /api/mkdir path creates a directory, parents included
POST /api/move from newline to moves or renames, creating the destination folder
POST /api/delete kind newline path deletes a file, or a folder and everything in it
POST /api/list path one level: dirs, files, plus parent and roots
POST /api/check path the compiler's own output for that file
POST /api/find root newline query content search: {file, line, text}[]
POST /api/git workspace root {branch, dirty, files:[{status,path}], log:[{shorthand,subject}]}
POST /api/ai/start companion dir starts the optional companion, detached and minimised
POST /api/run a whitelisted command {status, output}
POST /api/action action newline target {status, output}
POST /api/layout workspace root {structs, collisions}

/api/run accepts only forgen, git, python and bash prefixes. An HTTP endpoint that runs arbitrary commands is a backdoor even on loopback.

/api/check is the important one. The red underlines come from forgen itself, parsed into line/column/span, so they appear while you type and do not need the companion to be running. The diagnostics panel used to be fed only by the AI's linter, which meant that with the AI off the editor looked like it was paying no attention - while the compiler, which already had the answer, sat unused.

Creation endpoints report failure by checking the result, not by reading the shell's message: st_mkdir enters the folder afterwards and st_new_file asks the runtime whether the file is there. Matching the English "already exists" would still have missed the localised wording, and on this machine it did - the first version turned the shell's Russian error text into a folder entry named after the error.

The layout inspector

The Layout tab reimplements forgen's layout optimizer (layout.rs) in Datara and shows what it will decide, because the compiler reports none of it:

  • field order - descending type rank (Float4/Int4=16, Int/Float/Str/ptr=8, Int32=4, Int16=2, Bool/Byte=1), alphabetical on ties, offsets as index * 8;
  • alignment and size - 64 bytes at 8+ fields, 32 at 4+, else 16;
  • the SoA rule - a struct whose name merely contains soa or layout, or that has 4+ fields, is flagged as columnar.

Then it cross-references every struct and reports each field name that two structs place at different offsets. That is the condition that silently corrupts one of them, and the reason the kernel's struct fields carry a prefix.

Verified against the two probe projects that reproduce the defect:

probes/fieldnames_shared   3 structs, 6 conflicts
    scope_id       SA 40   SB 16   SC 16     <- different offsets, one field name
    deferred_mode  SA 48   SB 24   SC 24
    cancelled      SA 56   SB 32   SC 32
    delivery_count SA 64            SC 40

probes/fieldnames_unique   3 structs, 0 conflicts

The field ordering it produces matches the compiler's own choices, which is how the reimplementation was checked: it is not a guess, it is the same algorithm.

This is the tool that would have turned a day of bisecting probes into a five-minute look. It is also the first Datara-specific surface in the IDE - the kind of thing no general-purpose editor can offer, because the information exists only inside forgen.

Honest limitations

  1. --llvm cannot build this project yet, and the reason is upstream. Three of the six modules (main, api, toolchain) trip an internal LLVM verifier error on any List-typed local reassigned inside a while loop; see docs/llvm-backend-bug.dtr for the three-line reproduction. The default backend is unaffected, the linker discovery is in place, and the moment forgen fixes the codegen, FORGEN_LLVM=1 starts using it with no change here.
  2. Build status is authoritative again. system() returns the process's real exit code on 1.4.0, so /api/run reports exit and derives status from the number rather than from the text - status_source is no longer inferred. SEAM-3 is retired; see PORTING.md. The output is still captured to a file rather than a pipe, because that is one process launch instead of two.
  3. No document model on the server. The browser holds the text and sends it on save; the server re-reads files. There is no revision tracking, no undo across sessions, no stale-result suppression yet - the kernel's Freshness and Generation counters are not wired in. That is the next real step.
  4. The server holds no state and no structs. Deliberate: it keeps every string raw and converts only at the JSON boundary, which is what makes the codepage handling tractable; see PORTING.md finding 1.
  5. Single-threaded. A slow forgen build blocks the IDE's own file reads for its duration. Suggestions are unaffected because the AI is a separate process. This is why /api/health must never spawn a process - it used to, and it made the cheapest endpoint in the server the slowest one.
  6. There is no terminal. /api/run will execute a whitelisted command (forgen, git, python, bash) and return its real output and real exit code, but nothing in the interface types into it - the UI only calls /api/action, which runs a fixed forgen action. The output drawer is the closest thing that exists. An interactive console is unbuilt, not hidden.
  7. The file tree is native now. dir_list replaced the two shell listings, so /api/tree is 26 ms on this workspace (was 487 ms). The walk is breadth-first with a budget, because a depth-first walk spends the whole budget inside the first folder it meets - on D:/ that showed 5998 files and 2 folders instead of 157. A drive root is still listed one level deep on purpose (shallow: true in the response): a recursive listing of D:/ grew the server to 3.9 GB and stopped it answering, which is how a workspace saved as D:/ became unreadable.
  8. Non-ASCII paths work, except for creating one. dir_list returns cp1251 and the server keeps entry names raw on the inside, so a Cyrillic folder now lists, opens, reads and writes correctly. st_fs_path resolves a path from the browser by finding each segment in its parent's listing rather than reconstructing it, because a Datara string cannot be built from byte values - which is also why a new folder or file whose name is not ASCII is refused with a reason instead of silently creating a misnamed one. See PORTING.md finding 1.
  9. Syntax highlighting is a single-pass line tokenizer. It does not understand multi-line strings or nested block comments, and it is not the semantic highlighting the spec asks for. Real semantic regions need the Datara provider, which does not exist yet.
  10. Reading mode is a layout change, not a different mode. The spec's Reading Mode wants call graphs, dependency views and ownership surfaces. Today it widens the viewport and hides the tree.
  11. The layout inspector is a text scan, not a parser. It reads multi-line declaration bodies only, so a single-line pub struct X { a: Int } is skipped, and generics are taken as text. It recognises all four declaration keywords (struct, class, entity, record), which it did not before - it matched struct alone, so a project using class reported zero structs. It mirrors forgen's optimizer rather than calling it, so if the optimizer changes the mirror has to be updated; that file is the thing to fix, and it says so at the top. Its field sort compares bytes as integers rather than using Str <, because Str comparison is not deterministic on 1.4.0 (finding 13 in docs/COMPILER-NOTES.md).
  12. Deleting is permanent, and an ASCII-only operation. Delete and rename both work in the explorer now, but delete shells out - forgen 1.4.0 has no file-delete builtin - so a name that is not ASCII cannot be spelled for the shell and is refused, with the reason, rather than silently deleting the wrong thing. There is no trash and no undo: the confirmation is the only thing between a mis-click and the loss, which is why it says so in as many words and why the destructive button is the one that has to be aimed at.

Next, in order

  1. An in-process check. /api/check costs 299 ms and forgen reports 0 ms of compile time for one file and for 2828 alike - all of it is process start. Reusing the compiler as a library removes nearly the whole number, and it is the single largest latency win left in the IDE.
  2. An interactive console. /api/run already runs a whitelisted command and returns the real exit code; what is missing is the input line, the history and the streaming. Roughly a day, and it is the one thing people expect an IDE to have that this one does not.
  3. A real document model on the server: open buffers, revisions, Freshness wired in, so a slow analysis can never overwrite a newer edit.
  4. A Datara provider: reuse forgen's own lexer and symbol table instead of the regex highlighter. This is the step that makes the IDE compiler-native rather than a text editor with a colour scheme.
  5. Semantic lenses: types, ownership, effects, capabilities, and the struct layout inspector - the panel that would have turned the layout bug into a five-minute diagnosis instead of a day.

Licence

MIT OR Apache-2.0, at your option. Both texts are in the repository: LICENSE-MIT and LICENSE-APACHE.

SPDX-License-Identifier: MIT OR Apache-2.0

The dual form is the Rust convention, and this project is a Rust, wasm and Datara stack, so it is what contributors and downstream packagers already expect to find. The two licences are not redundant: MIT is the shortest thing to read and the easiest to satisfy, while Apache-2.0 adds an explicit patent grant, a trademark clause and a termination condition. Offering the choice lets a user pick whichever fits their own legal constraints, and for a project that is not trying to sell anything, that is the point - the licence should never be the reason someone does not use it.

Why not copyleft. GPL and AGPL exist to force commercial users to share their changes. With no revenue to protect, that buys little here, while its cost is real: it would block anyone from embedding this interface in a proprietary tool, and it would make the project's own future relicensing impossible without the agreement of every contributor who had ever sent a patch. AGPL is aimed at network services in any case, and a desktop IDE is not one.

Why not public domain. A dedication like Unlicense carries no patent grant and is on uncertain ground in some jurisdictions. For a compiler toolchain, where the patent grant is the clause most likely to matter one day, that is the wrong trade.

The one alternative worth naming is MPL-2.0, which is file-level copyleft: changes to files that are already MPL must be published, but MPL code can be combined with proprietary code freely. If the intent ever changes to "improvements to this must come back", MPL-2.0 is the switch to make, and it is the least restrictive copyleft available.

Contributions. There is no CLA. Contributions come in under the same dual licence they were received under, which is the norm for MIT/Apache-2.0 projects and keeps the barrier to a first patch at zero.

Third-party components that ship inside the built interface are listed, with their licences and copyright holders, in THIRD-PARTY-NOTICES.md.

About

A compiler-native IDE for the Datara language

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages