A working IDE for Datara. Human-first, compiler-native, and small enough to read in one sitting.
datara-studio/
src/main.dtr entry point, accept loop, routing
src/http.dtr HTTP/1.1 over raw sockets, request parsing, JSON escaping
src/api.dtr endpoints: tree, read, write, new, run, action, layout
src/explorer.dtr the filesystem browser, and the console-codepage fix
ui/app.js the interface
src-tauri/ the desktop shell - Tauri is only the window
PORTING.md every compiler workaround and its removal plan
Releases carry one installer per platform.
Windows is the only one that has ever been built and run. The Linux and macOS
targets are wired into .github/workflows/release.yml, which has not had its
first run, and no Linux or macOS artifact exists yet - release v0.3.0 carries
exactly one file, Datara.Studio_0.3.0_x64-setup.exe. Read the last two rows as
"the build is configured", not "the build is known good".
| Platform | Asset | Status |
|---|---|---|
| Windows x64 | Datara Studio_<version>_x64-setup.exe (NSIS) |
built here and launched |
| Linux x64 | datara-studio_<version>_amd64.deb |
workflow added, never run |
| Linux x64 | datara-studio_<version>_amd64.AppImage |
workflow added, never run |
| macOS, Intel and Apple Silicon | Datara Studio_<version>_universal.dmg |
workflow added, never run |
macOS is built universal rather than for the runner's own architecture, because a
macos-latest runner is Apple Silicon and a default build would produce a disk
image that does not open on an Intel Mac.
None of the installers contain forgen. This is a compiler-native IDE: the
shell starts the Datara server, and the server is forgen run src/main.dtr. The
installers ship the shell, the studio's Datara sources and the built interface,
and they need the compiler on PATH to do anything. Without it the window opens on
the splash page and stays there. There is no public download for forgen yet, so
installing the studio means having the toolchain already.
The .deb and .dmg install the app and register .dtr with it. The AppImage
needs no installation at all - chmod +x it and run it.
From a release, the installed app is enough: launch it from the menu, or on Linux run the AppImage directly.
From a clone, there is one launcher per platform and they do the same four things - build the single-file interface if it is missing, start two servers, start the optional AI companion, open the browser, and then watch the pair, restarting it if neither port answers:
start.cmd # Windows
./start.sh # Linux and macOS
Two servers is not redundancy for its own sake: the Datara runtime cannot time
out a socket, so one idle connection can wedge a server permanently, and the
interface falls back to the second port. See PORTING.md SEAM-6.
The long way, which is also the way to see what the server is doing:
cd datara-studio
forgen run src/main.dtrThen open http://127.0.0.1:7878.
What runs, and where.
| Port | What | Started by |
|---|---|---|
| 7878 | the server the browser opens first | either launcher, or forgen run src/main.dtr |
| 7879 | the second server, for when 7878 is wedged | either launcher |
| 7880, 7881 | a fresh pair, when a killed server left a bound-but-silent socket on the first two | start.cmd; start.sh picks from the same four before it starts anything |
| 7890 | the optional AI companion | either launcher, if the companion is found |
The interface tries all four in order and remembers which one answered, so a
reload does not re-pay the failed attempts. Any of these can be moved:
DATARA_STUDIO_PORT=7879 forgen run src/main.dtr, and the launchers read
DATARA_STUDIO_PORT_B, DATARA_STUDIO_PORT_C, DATARA_STUDIO_PORT_D and
DATARA_AI_PORT the same way.
Prerequisites.
| Needed for | Notes | |
|---|---|---|
forgen |
everything - it compiles the server | on PATH, or set DATARA_FORGEN to it. start.sh and the shell both search the conventional install locations (~/.local/bin, /usr/local/bin, /usr/bin, %LOCALAPPDATA%\Programs\Datara\bin) before giving up |
| Node.js | rebuilding ui/studio.html, and the launchers' search for the AI companion |
not needed to run a release |
| Python 3 | the AI companion, which is optional | the IDE runs without it, with no suggestions |
| Rust + a C++ toolset | building the desktop shell from source | see below |
Nothing else: the servers are native binaries and the interface is one file with no subresources.
Where the AI companion is looked for. The launchers do not guess: they call
scripts/find-companion.mjs, and it prints the first directory that actually
contains forgen_ai/ide_daemon.py, in this order - FORGEN_AI_DIR, then
$ROOT/../python, $ROOT/../../python, $HOME/.datara/python, then siblings of
the studio's parent. Set FORGEN_AI_DIR to the directory that contains
forgen_ai/ to skip the search. That variable is not the same thing as the
aiDir setting in the IDE's Settings panel, which wants the directory containing
python/ - one level higher.
src-tauri/ is a Tauri 2 shell. It is a window and nothing else - no bundled
interface, no filesystem access, no commands of its own - because the point of
this IDE is that it can read the project and run the compiler, and a frozen asset
bundle could neither. The shell starts the servers, waits for a port to answer,
opens the window at it, and kills the children on exit.
It builds and runs on Windows, Linux and macOS. The shell has no
platform-specific code paths except two: where it looks for forgen, and
CREATE_NO_WINDOW on the child processes, which is a Windows flag and is
compiled out elsewhere. cargo check --all-targets and cargo test run on all
three platforms in CI.
Building it. Windows:
start-tauri.cmd # builds the shell if it is missing, then runs itor by hand, on any platform:
node scripts/build-wasm.mjs # ui/vendor/textcore.js - gitignored, and inlined
node scripts/build-icons.mjs # src-tauri/icons/ - gitignored, and bundled
node scripts/build-ui.mjs && node scripts/build-ui.mjs --core
tauri build # or `cargo tauri build`, see belowThose first two lines are not optional and the order matters. Both write
files that are in .gitignore and that the steps after them read:
ui/vendor/textcore.js is inlined into the interface by build-ui.mjs, which
fails outright if it is missing, and src-tauri/icons/ is a bundled resource.
A fresh checkout has neither.
tauri or cargo tauri, pick one. They are different executables and the
project does not install either for you:
| You have | Run | Notes |
|---|---|---|
cargo install tauri-cli --version "^2" |
cargo tauri build |
compiles the CLI from source, minutes |
npm install -g @tauri-apps/cli@^2 |
tauri build |
a prebuilt binary, seconds - but it provides tauri and not cargo-tauri, so cargo tauri still fails |
The release workflow uses the npm one. On Windows, start-tauri.cmd checks for
either.
The output lands in src-tauri/target/release/bundle/. On Linux, install the
Tauri system dependencies first - the bundler fails at the point it links the
webview, which reads as a Rust error rather than a missing system package:
sudo apt-get install -y libwebkit2gtk-4.1-dev libgtk-3-dev librsvg2-dev \
patchelf libayatana-appindicator3-dev libxdo-dev libssl-dev libfuse2--bundles takes only the formats the host can produce and rejects anything
else - measured, cargo tauri build --bundles nsis,deb,appimage,app,dmg on
Windows exits with invalid value 'deb' for '--bundles'. The bundle.targets
list in tauri.conf.json is not restricted that way: the bundler skips
inapplicable targets with a warning, so one list serves all three platforms. That
is why tauri.conf.json names dmg and app even on a Windows checkout.
On Windows, scripts/msvc-env.sh must be sourced first for anything that
produces a native binary, because MSVC's link.exe has to precede Git-Bash's GNU
link. The JIT path (forgen run) needs no setup.
The one thing the page asks the shell for is the window itself: minimise,
maximise and close. That is a single capability file
(src-tauri/capabilities/default.json) granting exactly those permissions and
nothing else.
The title bar is the interface's, not Windows'. The window is created
undecorated (.decorations(false)), and the interface draws the bar - the mark,
the open file's name, and three caption buttons in the app's own palette. Three
pieces have to agree, and missing each one breaks it differently:
| Piece | Missing it means |
|---|---|
.decorations(false) in src-tauri/src/main.rs |
two title bars |
capabilities/default.json |
a bar whose buttons the access-control list refuses |
ui/tauri-bridge.js |
no bar at all, and a window with no chrome |
The capability is not optional even though the server is on loopback: Tauri
treats every origin that is not its own as remote, and the studio page is
served by the Datara server at http://127.0.0.1:7878. Without the remote.urls
list the bar draws and its buttons silently do nothing, which reads as broken
buttons rather than a missing permission.
tauri-bridge.js is also what keeps the browser build honest: it creates
window.__DS_SHELL__ only when window.isTauri is set, so in a browser the bar
is never drawn and .shell keeps its three rows. drive.mjs asserts both halves.
Nothing needs to be running first. The window starts its own servers, on the first two of the four ports that are free, and probes them itself.
The window opens immediately, on a bundled splash, and does not wait for the server. The splash polls all four ports and hands the window over the moment one answers, and only admits failure after nine seconds. The first version waited up to twenty seconds before creating any window at all, so a slow start looked like nothing happening - and the wait was never buying anything, because the page can do the waiting better than the shell can.
No console windows. start.cmd launches the servers through
scripts/hidden.vbs and the shell passes CREATE_NO_WINDOW, because a
GUI-subsystem process spawning a console application makes Windows create one -
so opening the IDE used to flash two black windows and leave them in the taskbar
for as long as it was open. There is no flag that means "minimised and invisible";
/min only minimises.
scripts/build-icons.mjs generates every piece of icon artwork from the one
geometry in scripts/mark.mjs, which is the same shape ui/icon.svg describes.
It has no image-library dependency: the mark is a rounded square, two strokes and
a dot, so it is rasterised from signed distances and encoded as PNG with zlib.
The build fails if ui/icon.svg stops containing the mark's three colours,
because that is the one way the artwork and the drawing can silently separate -
which is exactly what had happened.
scripts/build-desktop.cmd checks for Rust and the MSVC C++ toolset and says
which one is missing.
| Area | Status |
|---|---|
| Problems | the compiler's own diagnostics, underlined at the exact column while you type, with no AI at all; the = help: line is kept |
| Project problems | forgen check over the whole project - every error in every module, grouped by file, click to jump to the line. Aimed at the project that governs the open file, because a check pointed above it reports every import as a missing package. 301 ms measured |
| Typing aids | auto-indent (plus one level after an opening brace), bracket and quote pairs, backspace inside an empty pair, Ctrl+/ comment toggle, Tab / Shift+Tab indent and outdent a selection - each one switchable in settings |
| Go to definition | F12 or Ctrl+click, across the whole workspace |
| Hover help | rests on a name and shows where it is declared, or what the builtin does - including the traps |
| Completion | this file's symbols, keywords, types and builtins, local and instant; the companion folds in when present |
| Snippets | accepting a declaration keyword writes the shape it is always followed by. fn Tab gives fn name() -> Int { ... } with the name selected, so typing replaces it, and the body indented to wherever you typed. 19 of them: fn struct class entity record behavior trait impl enum type if while for match unsafe comptime defer use mod. Keywords with no shape (let, return, pub, true) still insert as the plain word - a skeleton on true would be noise |
| Ghost text | Tab accepts; the suggestion appears at the caret before you finish typing |
| Resizable panels | drag the dividers, fold either side; widths are remembered |
| Zoom | Ctrl+wheel in the editor, Ctrl+0 to reset; survives a reload |
| Language providers | the editor does not know what Datara is - see below |
| File tree | every file and folder, language icons, drag a file onto a folder to move it |
| Create | one create control, File or Folder, named inline in the folder you are working in |
| Dialogs | every question is drawn by the interface itself - delete, discarding unsaved changes, rename, Save As, naming a new project. There is no window.confirm, no window.prompt and no platform message box left anywhere, so nothing in the window looks like a different application |
| Status bar | three columns, not a row of loose text: the open file and the server's word on the left, the git branch in the middle (dirty count beside it, click for the Project panel), and problems, caret position and the output drawer on the right |
| Open file | any file on the machine, not only the ones in the workspace |
| Open folder | editable path box, drive row, new folder, failures that say what went wrong |
| Reopens where you left off | last file, last workspace, and the port that last answered |
| Editor | textarea over a highlighted layer, line numbers, dirty marker, semantic rail |
| Breadcrumb | shows the declaration the caret is inside, not just the path |
| Check / Build / Run / Lint / Audit | forgen invoked from the server, output in the output pane |
| Find references | Alt+F7, a real workspace search, or the search box |
| Layout inspector | the layout forgen will choose per struct, and every field-name conflict |
| Companion | optional, starts itself in the background, one dot in the chrome, toggle in settings |
| Degrades without AI | completely. Problems, Structure, Project and Layout all work offline |
Keyboard: Ctrl+S save, Ctrl+N new file, Ctrl+B build, Ctrl+Shift+B check
the open file, Ctrl+Enter run, Ctrl+P file palette, Ctrl+, settings, F12
or Ctrl+click go to definition, Alt+F7 references, Ctrl+/ comment toggle,
Ctrl+wheel zoom, Ctrl+0 reset zoom. Tab resolves in one fixed order: it
accepts the open completion list first - and a declaration keyword there writes
its whole shape, not the word - then a ghost suggestion, and only then does it
indent (outdent with Shift, or indent every line of a selection). Esc
dismisses the list. Ctrl+P also offers Check the whole project -
forgen check over every module.
The editor must not know what Datara is. Everything language-specific sits behind
one object - DATARA_LANG in ui/app.js - which owns the keywords, the types,
the builtins, the hover documentation, symbol extraction, the indent string and
the compiler check.
There are two implementations. The second is PLAIN_LANG, which claims no
extensions and offers nothing: open a .py file and the editor still works
normally, but with no Datara keywords suggested, no Datara symbols in the
outline, and no compiler asked to check a file it cannot read.
That second provider is not decoration. A boundary with one implementation is an abstraction nobody has tested, so the plain-text provider exists to prove the seam is real - and 20 assertions hold it to that, including that a plain file never calls the compiler at all.
Adding a language is one object. Adding one with a compiler is one object with a
real check. This is the first boundary the project found on its own rather than
designed up front, which is the order the roadmap asks for: extract the kernel,
do not design it.
The right panel is ordered offline first - Problems, Structure, Project,
Layout, then Chat, AI and Generate - because a person who never starts the
companion should still get a useful panel. See docs/DATARA-ERGONOMICS.md for the
measurements and for what is still missing.
The IDE talks to forgen_ai (D:\ryan) directly over HTTP on port 7890. It is
optional: with it absent the status dot goes grey and everything else keeps
working. That is the design rule from the spec - no mandatory AI - enforced by
architecture rather than by policy.
Start it:
cd D:\ryan
python python/forgen_ai/ide_daemon.py --port 7890Then the IDE picks it up within five seconds.
PORTING.md SEAM-4 explains the one change made in that project: the neural
stack is now imported lazily, so the daemon starts and serves completions without
torch. No endpoint needs it now - chat included, which previously answered
neural stack unavailable here because it was the one path that did.
The companion ships two things under the word "model", and the interface used to report the wrong one:
| file | state | |
|---|---|---|
NanoCodeModel |
data/nano_code_model.json, 131 KB |
active. An order-3 n-gram over verified Datara; this is what generates code, and it needs no ML runtime |
NanoLM |
data/nano_lm.pt, 6.8 MB |
dormant. A 1.70M-parameter transformer that measured worse than the n-gram on both holdouts (NEURAL.md §7), so nothing is wired to it |
/health used to announce FORGEN-12.6M-MoE, naming a network whose checkpoint
was deleted. It now reports what is actually loadable, per candidate, with the
reason: ngram-datara active, nanolm-transformer present but dormant: torch not installed; no-ship build (measured worse than the n-gram). The verdict comes
from python/forgen_ai/model_registry.py, which probes the directory rather than
reading a preset name - so installing torch tomorrow changes the report rather
than contradicting it.
Generate is one shot: it composes a program from verified fragments and writes it
into the open file. That is still there. What was missing is a surface where the
previous turn exists, because a follow-up question has no meaning without one -
and the companion has had a /chat endpoint all along with no way to reach it
from the IDE.
The Chat tab is that surface. It is served by python/forgen_ai/chat_engine.py in
order of how much each answer should be trusted:
| resolved by | example | reported as |
|---|---|---|
| the project index | «какие функции имеют эффекты» | VERIFIED, with Источник: forgen context … |
| exact arithmetic | «сколько будет 2+2*10» | VERIFIED, 2+2*10 = 22. |
| the compositional generator | «напиши сортировку вставками» | VERIFIED after forgen check, code in the transcript |
| the curated knowledge base | version and feature questions | SUPPORTED |
| honest refusal | anything outside those | SUPPORTED, and it lists what does work |
Context is the feature, so it is visible and it is deletable. The card at the
top of the tab shows how many messages are carried, where the context came from
(compiler, manual or empty), and the topic the last answer established. Three
controls act on it:
- clear context - deletes the transcript and the attached context. It is final: the session records that the emptying was deliberate, so the project context is not silently re-gathered on the next turn. A clear button that refills itself is worse than no button, and there is an assertion for exactly that in both test suites.
- undo - drops the last exchange, in the panel and in the companion.
- use the project - attaches the compiler-gathered context again, which is the only way back once it has been cleared.
Follow-up inheritance is what makes it conversational rather than a log. «а теперь
по убыванию» carries a modifier and no subject, so the previous subject is
grafted onto it - сортировка + по убыванию - and re-planned. Generated
descending, the output is genuinely descending:
=== Сортировка пузырьком по убыванию ===
Исходный: 64 34 25 12 22 11 90
Результат: 90 64 34 25 22 12 11
A cold session asking the same follow-up is refused rather than misread as a request for a bubble sort, which is the boundary that keeps inheritance from turning into guessing.
Measured against the real companion, not inferred from the code. Five requests,
each with an empty context and verify: true:
| asked for | task it chose | confidence | fragments |
|---|---|---|---|
| sort a list of integers | sort |
0.87 | copy_ints, join_ints |
| read a file and count its lines | file_stats |
1.00 | count_lines, count_words |
| compute the sha256 of a buffer | generic |
0.15 | sum_ints |
| add two numbers | generic |
0.15 | sum_ints |
| parse an integer from a string and return an option | string_ops |
0.47 | is_palindrome, b2s |
Two things are worth stating plainly, because the panel does not:
- The task is chosen by keyword match against a small set of shapes. The
first, third and fifth rows are the honest failure: asked to parse an integer,
the companion wrote a palindrome checker, in Russian, as a console program
with
outstatements rather than a library function. It reportedconfidence: 0.47andverified: truewhile doing it. verified: trueis true and is not reassurance. Verification meansforgen checkcompiled it - and it does: the generated palindrome checker passesforgen check100% OK. The code is valid Datara. It is just not the code that was asked for, and nothing in the panel distinguishes "compiles" from "answers the question".
So the Generate panel is a fragment composer with a keyword front-end, not a
model that understands a request. The panel now says so: the header no longer
claims a token count the companion never sends (it read 3-gram · null tokens
for everyone - see describeModel in ui/app.js), and shows retrieval and
verification as the separate capabilities they are.
confidence is the field a caller should gate on, and nothing gated on it
before this. A confidence below roughly 0.5 in the table above means the
classifier did not really match; 0.15 means it fell through to generic.
Three languages, each doing what it is fastest at. The split is the design, not an accident:
ui/app.js React 18 + htm the chrome: title bar, rail, tree, panels,
status bar, command palette
ui/app.js direct DOM the text surface, mounted imperatively
crates/textcore Rust -> wasm the document, the line index, the lexer
src/*.dtr Datara files, the compiler, the layout inspector,
and the server that serves all of the above
Rust owns the text, and here is the measured reason. On a 1 MB / 20k-line document:
| operation | time |
|---|---|
| load the document | 4.1 ms |
| lex the whole document | 5.6 ms -> 260001 tokens |
| verify the line index against a rebuild | 4.4 ms |
| one edit in the middle | 0.091 ms |
| one edit at offset 0 (worst case) | 0.1 ms |
node crates/textcore/test/test.mjs reproduces those numbers, and it asserts the
incremental line index against a full rebuild after every edit - a fast path that
is never checked against the slow path is a bug waiting to happen. 40 assertions.
React owns the chrome, and explicitly not the text. Re-rendering a 20k-line highlighted document through React on every keystroke is the single biggest performance mistake an editor can make. So the text surface is mounted once into a div React renders and then never touches, and updated imperatively. The rule: React renders things a person clicks.
No wasm-bindgen. The ABI is flat C over linear memory: write UTF-8 into a
scratch buffer from alloc, call with (ptr, len), read results as a
Uint32Array view over lex_ptr(). That keeps the module at 39 KB, removes a
toolchain dependency, and makes the number of boundary crossings explicit -
which matters, because that is where wasm speed is lost.
The AI is a separate process (port 7890) and the browser calls it directly. The Datara server never proxies it, so an AI failure cannot take the editor down.
Everything that touches the project goes through the Datara server, which is the only component holding a capability grant.
bash scripts/build.shNine stages, 376 assertions, all green:
| stage | assertions | what it proves |
|---|---|---|
| wasm build | - | the module compiles and embeds |
| single-file interface + icons | - | the Tauri icon set is generated from the one geometry in scripts/mark.mjs, and React, htm, app.js and the wasm core inline into one ui/studio.html with zero subresources, with the icon inlined as a data URI. Builds both variants - the full studio and --core without the companion. Fails the build if the mark beside a .dtr file is backed by an icon under 32 px, or if ui/icon.svg no longer matches the generated artwork - see the icon notes below |
| Rust text core | 40 | the incremental line index matches a full rebuild after every edit |
| highlighting pipeline | 40 | the token-to-line split is lossless on empty input, trailing newlines, unterminated strings, Cyrillic and emoji, and every comment token starts with // and stops at its own line - which is what caught the byte-offset bug |
| interface renders | 192 | the component tree renders without throwing, every piece of chrome is present, the create name rule holds, an empty folder survives into the tree, the compiler's real coloured output parses into line, column and span, and a project check is aimed inside the workspace |
| the editor, driven as the app drives it | 74 | text, highlight layer, gutter, every editor layer and textarea height agree, and every editing aid leaves the text, the highlight layer and the caret agreeing. 19 of these drive the snippet table: that each keyword expands to a body that parses, that the placeholder is selected, that the caret lands inside it - the unsafe one inside the quotes - and that an indented keyword indents its body to match |
| boot the built artifact | - | the shipped ui/studio.html mounts in a DOM, shows the title screen, and defers the editor until a file is open |
| Datara server | - | forgen check reports 100% OK |
Then:
forgen run src/main.dtr # open http://127.0.0.1:7878What is verified and what is not. The text core is verified by execution:
the numbers in the table above are measured, not estimated. The interface is
verified by structure in build.sh, and separately by pixels and by
gestures:
node ui/test/shoot.mjs http://127.0.0.1:7878 shots # real Chromium, writes PNGs
node ui/test/drive.mjs http://127.0.0.1:7878 shots/drive # real gestures, then reads the bytes off disk
node ui/test/chat.mjs http://127.0.0.1:7878 # the chat surface, against a live companionNone of these is part of build.sh, because they all need a running server and a
Playwright install. shoot.mjs drives the real page in a real engine at a real
viewport; drive.mjs right-clicks the tree, saves with Ctrl+S and presses Run,
and then reads the result off the filesystem instead of trusting the screen. It
currently makes 70 such checks, and because it walks every panel tab and
drives every dialog, it is the only thing here that would notice a control that
kills the window when you use it. It is also the only thing that types fn into
a real browser and presses Tab - the structure suite can prove the table is
right, but only a real key event proves the key is wired to it.
chat.mjs exists because chat is the kind of feature that passes at the API
layer and does nothing in the interface: the companion's own gate
(D:\ryan\tools\verify_chat.py, 33 assertions) proves /chat and /context
behave, and cannot prove the IDE reaches them or that the transcript renders. So
it drives the panel - sending a question, a code request and a follow-up, then
pressing clear context and asserting that the transcript and the readout
both emptied and stayed empty. The companion is optional, so with it offline the
suite asserts the honest empty state instead of going red for the wrong reason.
forgen ships two backends: a fast-to-compile native one (Cranelift) and an
optimising --llvm one. Asking for LLVM used to appear to work and quietly did
not. There were two independent causes, and both had the same symptom - a
successful build - which is why neither was noticed.
One: the linker was shadowed. Git-Bash ships a GNU link.exe at
/usr/bin/link.exe, and on a machine where that directory precedes MSVC's, every
link the compiler attempted went to a Unix hard-link utility:
/usr/bin/link: extra operand '/DEBUG:NONE'
. Falling back to native Cranelift backend.
The fix is in src/toolchain.dtr, which finds MSVC and sets PATH, LIB and
INCLUDE before the compiler runs. LIB and INCLUDE are not optional
extras: with only PATH set, the right link.exe runs and then dies with
LNK1181: cannot open input file "legacy_stdio_definitions.lib", because
link.exe finds its libraries through LIB and nowhere else. The Windows SDK
version is discovered, not hardcoded, and the whole probe uses dir_list and
path_exists - no process is launched to find a linker.
Worth knowing: env_set stages a value for the next child process and
env_get does not read it back (measured). That makes the prepend naturally
idempotent - three calls leave exactly one entry in the child's PATH - and it
means a "have I already done this?" guard would be dead code.
Two: the LLVM backend itself cannot compile list-heavy code. Even with the linker correct, a real program trips an internal verifier error:
llc.exe: error: '<cache>/main_*.ll:11939:23: error: '%v5244' defined with type
'ptr' but expected 'i64'
%v18804 = phi i64 [ %v5244, %bb2 ], [ %v18806, %bb9 ]
. Falling back to native Cranelift backend.
This is a codegen defect in forgen 1.4.2, not environment. It is isolated to
a List-typed local reassigned inside a while loop, reproducible in the
smallest form in docs/llvm-backend-bug.dtr:
fn f(items: List<Str>) -> Int {
mut acc = items
mut i = 0
while i < items.len() {
acc = acc.push(acc[i]) // <- removing this line lets LLVM finish
i = i + 1
}
return acc.len()
}
Three of this project's six modules (main, api, toolchain) trigger it, so
--llvm cannot build the studio until that is fixed upstream. It is a forgen
bug, not a studio one, and the honest thing is to say so rather than work around
it.
Measured, because the choice should be made on numbers. Same source, same MSVC linker, this machine:
| native (Cranelift) | --llvm |
|
|---|---|---|
fib(30), recursive calls |
472-490 ms | 491-503 ms |
| 40M-iteration arithmetic loop | 531-564 ms | 499-517 ms |
| build time | 292-294 ms | 1055-1998 ms |
| output size | 374.5 KB | 372-464 KB |
LLVM wins a tight arithmetic loop by about 5% and costs 3.5x-7x the build time. Build time is paid on every Run; 5% is invisible outside a benchmark. So the default is the native backend - which is what "the fastest variant" actually means for someone using the IDE - and LLVM is one environment variable away:
FORGEN_LLVM=1 forgen run src/main.dtr # every build in this session asks for --llvmThe build now reports which backend it used. Both facts are carried back from
the server (linker_ok, fell_back) and named in the Run drawer:
forgen build · ok · exit 0 · took 894 ms · native (LLVM ready: set FORGEN_LLVM=1)
forgen build · ok · exit 0 · took 2103 ms · Cranelift (LLVM fell back - linker missing)
A silent fallback used to be indistinguishable from success. It no longer is.
The mark beside a .dtr file and in the title bar is drawn at 13-16 px CSS,
which is not 13-16 device px: a 16 px box is 20 device px at 125% display
scaling, 24 at 150%, 32 at 200%, and the browser upsamples whatever source it has
to fill it. Backing that with the ICO's own 16 px entry - the obvious choice -
meant a 3x upsample on a 200% display. Measured, focus = variance of Laplacian /
variance of luma, the mark drawn at 16 px CSS:
| display scaling | 16 px entry | 32 px entry |
|---|---|---|
| 100% | 6.57 | 9.59 |
| 125% | 1.51 | 12.46 |
| 150% | 1.02 | 9.50 |
| 200% | 0.34 | 8.01 |
The 16 px entry is the worst of the seven entries at every scaling, and it is
not even best at 100%, so "the only size at which this artwork is crisp" - which
is what the stylesheet claimed - was simply wrong. The 32 px entry covers 16 px
CSS exactly up to 200% and won on the sum, so that is what backs it now, and
build-ui.mjs refuses to build if that ever drops below 32.
The 128 px entry the title screen uses was checked the same way, and it is genuine: it differs from a proper 256 -> 128 resample by a mean of 1.77 of 255 per channel, against 7.62 for a 64 -> 128 upscale. It is not a fake upscale, which is what it looks like when magnified 4x.
The measurements above were all correct and they were all about the wrong
artwork. assets/datara.ico - the file the window icon, the taskbar, the
browser tab and the title screen were all built from - held an orange-and-yellow
palm inside a yellow rounded square. Everything inside the interface drew
something else: two light brackets around a mint dot, which is what ui/icon.svg,
the Mark component and the mark beside a .dtr file all show. Two marks were
live in one window and the one on the taskbar was the wrong one. Amber is also
the reserved warning colour in docs/DESIGN-UI.md, and the accent is mint, so
the icon was arguing with the interface in colour as well as in shape.
It had been made worse by a well-intentioned change: build-icons.mjs used to
draw the mark itself, and was rewritten to read assets/datara.ico directly, on
the argument that three approximations of one design are three designs. The
argument is right. The conclusion was not, because that file was a fourth
design rather than the first one.
The geometry now lives in scripts/mark.mjs, scripts/build-icons.mjs writes it
to ui/mark.ico, and everything else derives from those bytes. build-ui.mjs
reads ui/mark.ico rather than assets/, so a stale file in assets/ can no
longer put a different icon in the browser tab than the one in the window.
Review icons on a light surface as well as a dark one. The first attempt at
the small entries dropped the dark plate at 16 and 24 px, because a 64-unit
corner radius lands on the pixel grid there and produced dirty corners. It looked
right on the dark review image. On the light one the icon had vanished - the
brackets are #E9E9EE, which is invisible on white, so all that remained was the
mint dot. scripts/icon-sheet.mjs now renders every entry at its real size,
magnified 8x, on both surfaces, and that is the image to look at:
node scripts/icon-sheet.mjs shots/icons
shots/icons/sheet-light.png and shots/icons/sheet-dark.png
Ctrl+Shift+Z, or the button beside the panel toggle. Everything that is not the
code goes - title bar, toolbar, explorer, right panel, breadcrumbs, status bar -
and Esc brings it back. What survives is one 26px line: the file name, the
cursor position, and a clickable error and warning count that leaves Zen and opens
the Problems tab. Each of those can be turned off in Settings → Zen, along
with soft wrap, a centred column width, extra breathing room and a larger face
for reading. With the informational parts all off the bar collapses entirely, so
this is configurable from "one quiet line" down to "literally just code".
The mode is carried on <html> as data-zen rather than as a React class,
because the rules it drives are about the shell's own grid and a grid track cannot
be removed from inside the grid it belongs to: hiding five children with
display:none leaves the reserved tracks behind, and the code ends up in a 1fr
row with dead space above and below. Collapsing grid-template-rows to a single
row is what makes the editor actually fill the window.
Two things this got wrong first, both worth recording:
- The shortcut was registered in the editor mount, which begins
if (!edRef.current) return;- so with no file open the handler was never installed, the key arrived with nobody listening, and Zen looked broken rather than unavailable. A shortcut that switches the window's layout must not depend on the window's contents. - The centred column insets the text surface, not the text. A
max-widthon the code alone leaves the line-number gutter at the window edge and puts the caret in the wrong place, because the gutter is positioned from the surface's own box.
node ui/test/zen.mjs http://127.0.0.1:7878 <a-directory-with-dtr-files> drives
all of it in a real browser and measures the result - 27 assertions covering the
shortcut, the button, both exits, the grid collapse and every setting.
node scripts/build-ui.mjs --core writes ui/studio-core.html, and the server
serves it at /?core=1. It is the same interface with the AI tabs taken out:
four compiler tabs instead of seven, no plug in the bar, no "start the companion"
in the palette, no companion section in Settings. Everything else - the compiler
panels, the editor, the tree, the explorer, Layout, Project, git and search - is
identical, because none of it was ever an AI feature.
It is a flag rather than a second source file. ui/app.js reads
window.__DS_CORE__ in four places, and a conditional in four places can be
audited in one sitting where two copies of a 4000-line file cannot. The build
refuses to emit if the flag is not settled before app.js loads - the failure
that would otherwise produce a file named studio-core.html containing the full
studio.
Eight tabs want 404 px in a 264 px panel, so the row runs off the edge and scrolls. Two things were wrong with that, and neither was visible in a test until there was one that drove a browser.
The auto-scroll that keeps the selected tab in view was not clamped, so selecting
the last tab scrolled the row past its own end and pushed the first one
out the left edge. Measured at 1440x900: Problems reported at x 1066 with
the strip's own left edge at 1176, sitting over the code column, outside its
container. And nothing indicated overflow at all - no arrows, no fade, a
scrollbar hidden by overflow:hidden - so the tabs at the end were found by
accident or not at all. Now: the scroll is clamped, two arrow buttons appear when
the measured row overflows and disable at each end, and the long labels are
abbreviated (Issues, Symbols), which brings 380 px down to 268 and fits the
default panel outright.
ui/test/tabs.mjs drives the real page and asserts reachability rather than
appearance: every tab, when clicked, is brought fully into view; the row starts at
scrollLeft 0; the arrows exist exactly when they are needed. Two traps in
writing it, both kept in the comments because both cost a round:
getBoundingClientRectdoes not know about clipping. A child scrolled past its container still reports its geometric position, so "no tab may haveleft < strip.left" fails against a strip that is working perfectly. A screenshot settles it - at full scrollProjectis cut mid-word at the panel edge, which is clipping. Assert on whether hiding is in effect, not on rects.- The first version asserted that every tab is visible at once, which contradicts the entire design. The claim is reachability.
The boot test caught one more real defect here: calling el.scrollTo(...)
directly throws is not a function in a DOM without it, and because the throw
lands inside React's commit the whole interface rendered as the crash screen. It
is guarded now.
drive.mjs also covers the desktop shell's title bar without needing the shell
binary: it stubs what Tauri injects before any page script runs, and then asserts
that the real ui/tauri-bridge.js picks it up, that the bar appears with three
controls, and that each control reaches the window plugin with the window label
the shell actually created. The converse is asserted too, because it is the
property easiest to lose silently - a browser must draw no title bar at all.
That pair is how the worst defects in this studio were found, and none of them was visible to a structure test:
-
Opening the Project tab destroyed the interface.
Panel'sproj()readgit.branch,git.filesandgit.log- andgitis state that lives inApp's scope, whichPanelwas never given. The tab threwReferenceError: git is not defined, the error boundary caught it, and the whole window was replaced by "The interface stopped". It had been that way for as long as the panel has existed, and every other check in this file passed while it was true, because nothing ever clicked a tab. Found by clicking one, and now asserted tab by tab. -
A
.wbtnof the same name squashed the title screen. The title bar's caption buttons were.wbtn, which the title screen already used for its three ways in. Both rules are one class deep, so the one declared earlier in the stylesheet won every property the later one did not also set - and the caption'swidth:46pxlanded on "Create new project", wrapping it into a column of single words. No assertion noticed; the screenshot did. The classes are now.capbtn, anddrive.mjsmeasures the three cards' width so the next one is caught by the suite rather than by eye. -
The editor was never mounted on a cold boot - the mount effect ran once, before there was a surface to mount into, and never retried, so the code area was empty while the breadcrumb and the status bar both reported the file as open.
-
The title screen could never appear, because the workspace defaulted to the server's own directory, which is truthy, so the "create a project / open a folder" screen was unreachable code.
-
The companion never suggested anything.
Editor.onInputis installed by the mount effect, so therunAIit closed over was built before the first/healthanswer and hadaiOnlinefrozen atfalsefor the life of the page. The indicator still read "companion: on", because it renders from state rather than from the closure - so reading the code did not find it. Proved by running identical keystrokes with the surface mounted before and after the health poll: two/completerequests and ghost text in one ordering, nothing in the other. -
The squiggles described the last save.
/api/checkis aimed at a path and the compiler reads that path, so a check issued with a dirty buffer reported the file as it was before you typed; with autosave on, nothing re-checked after the write, so an error you had just typed appeared only after the next keystroke.runChecknow writes the buffer first. -
A new project was created inside the IDE's own source tree.
Create new projectasked for a name and never for a location, so with no workspace open the name became a path relative to the server's working directory - measured on a cold launch,demo-projectlanded indatara-studio/. Two things had to be true at once and both were: the folder dialog opened on the server's own directory, and the 8 s tree poll ran with no workspace, took theroot: "."the server answers such a request with, and overwrote the folder the reader had just typed. Both are checked on disk now, because a project in the wrong place looks exactly like a project in the right place.
A screenshot alone would not have caught any of them; a screenshot plus a filesystem check did.
linkedom is a development dependency of the structure tests, installed with
npm install in this directory. The studio ships no runtime npm packages: the
interface is one generated HTML file with React, htm and the wasm core inlined.
All POST, plain-text bodies, JSON responses.
| Endpoint | Body | Returns |
|---|---|---|
GET / |
- | the interface |
GET /api/health |
- | service info |
POST /api/tree |
root path | {root, files:[...], dirs:[...]} recursive, with the resolved absolute root |
POST /api/read |
path | {content, bytes} |
POST /api/write |
path newline content |
{bytes} |
POST /api/new |
path newline content |
creates the file, making its folder first |
POST /api/mkdir |
path | creates a directory, parents included |
POST /api/move |
from newline to |
moves or renames, creating the destination folder |
POST /api/delete |
kind newline path |
deletes a file, or a folder and everything in it |
POST /api/list |
path | one level: dirs, files, plus parent and roots |
POST /api/check |
path | the compiler's own output for that file |
POST /api/find |
root newline query |
content search: {file, line, text}[] |
POST /api/git |
workspace root | {branch, dirty, files:[{status,path}], log:[{shorthand,subject}]} |
POST /api/ai/start |
companion dir | starts the optional companion, detached and minimised |
POST /api/run |
a whitelisted command | {status, output} |
POST /api/action |
action newline target |
{status, output} |
POST /api/layout |
workspace root | {structs, collisions} |
/api/run accepts only forgen, git, python and bash prefixes. An HTTP
endpoint that runs arbitrary commands is a backdoor even on loopback.
/api/check is the important one. The red underlines come from forgen
itself, parsed into line/column/span, so they appear while you type and do not
need the companion to be running. The diagnostics panel used to be fed only by
the AI's linter, which meant that with the AI off the editor looked like it was
paying no attention - while the compiler, which already had the answer, sat
unused.
Creation endpoints report failure by checking the result, not by reading the
shell's message: st_mkdir enters the folder afterwards and st_new_file asks
the runtime whether the file is there. Matching the English "already exists"
would still have missed the localised wording, and on this machine it did - the
first version turned the shell's Russian error text into a folder entry named
after the error.
The Layout tab reimplements forgen's layout optimizer (layout.rs) in Datara and
shows what it will decide, because the compiler reports none of it:
- field order - descending type rank (Float4/Int4=16, Int/Float/Str/ptr=8,
Int32=4, Int16=2, Bool/Byte=1), alphabetical on ties, offsets as
index * 8; - alignment and size - 64 bytes at 8+ fields, 32 at 4+, else 16;
- the SoA rule - a struct whose name merely contains
soaorlayout, or that has 4+ fields, is flagged as columnar.
Then it cross-references every struct and reports each field name that two structs place at different offsets. That is the condition that silently corrupts one of them, and the reason the kernel's struct fields carry a prefix.
Verified against the two probe projects that reproduce the defect:
probes/fieldnames_shared 3 structs, 6 conflicts
scope_id SA 40 SB 16 SC 16 <- different offsets, one field name
deferred_mode SA 48 SB 24 SC 24
cancelled SA 56 SB 32 SC 32
delivery_count SA 64 SC 40
probes/fieldnames_unique 3 structs, 0 conflicts
The field ordering it produces matches the compiler's own choices, which is how the reimplementation was checked: it is not a guess, it is the same algorithm.
This is the tool that would have turned a day of bisecting probes into a five-minute look. It is also the first Datara-specific surface in the IDE - the kind of thing no general-purpose editor can offer, because the information exists only inside forgen.
--llvmcannot build this project yet, and the reason is upstream. Three of the six modules (main,api,toolchain) trip an internal LLVM verifier error on anyList-typed local reassigned inside awhileloop; seedocs/llvm-backend-bug.dtrfor the three-line reproduction. The default backend is unaffected, the linker discovery is in place, and the moment forgen fixes the codegen,FORGEN_LLVM=1starts using it with no change here.- Build status is authoritative again.
system()returns the process's real exit code on 1.4.0, so/api/runreportsexitand derivesstatusfrom the number rather than from the text -status_sourceis no longerinferred. SEAM-3 is retired; seePORTING.md. The output is still captured to a file rather than a pipe, because that is one process launch instead of two. - No document model on the server. The browser holds the text and sends it
on save; the server re-reads files. There is no revision tracking, no undo
across sessions, no stale-result suppression yet - the kernel's
FreshnessandGenerationcounters are not wired in. That is the next real step. - The server holds no state and no structs. Deliberate: it keeps every
string raw and converts only at the JSON boundary, which is what makes the
codepage handling tractable; see
PORTING.mdfinding 1. - Single-threaded. A slow
forgen buildblocks the IDE's own file reads for its duration. Suggestions are unaffected because the AI is a separate process. This is why/api/healthmust never spawn a process - it used to, and it made the cheapest endpoint in the server the slowest one. - There is no terminal.
/api/runwill execute a whitelisted command (forgen,git,python,bash) and return its real output and real exit code, but nothing in the interface types into it - the UI only calls/api/action, which runs a fixedforgenaction. Theoutputdrawer is the closest thing that exists. An interactive console is unbuilt, not hidden. - The file tree is native now.
dir_listreplaced the two shell listings, so/api/treeis 26 ms on this workspace (was 487 ms). The walk is breadth-first with a budget, because a depth-first walk spends the whole budget inside the first folder it meets - onD:/that showed 5998 files and 2 folders instead of 157. A drive root is still listed one level deep on purpose (shallow: truein the response): a recursive listing ofD:/grew the server to 3.9 GB and stopped it answering, which is how a workspace saved asD:/became unreadable. - Non-ASCII paths work, except for creating one.
dir_listreturns cp1251 and the server keeps entry names raw on the inside, so a Cyrillic folder now lists, opens, reads and writes correctly.st_fs_pathresolves a path from the browser by finding each segment in its parent's listing rather than reconstructing it, because a Datara string cannot be built from byte values - which is also why a new folder or file whose name is not ASCII is refused with a reason instead of silently creating a misnamed one. SeePORTING.mdfinding 1. - Syntax highlighting is a single-pass line tokenizer. It does not understand multi-line strings or nested block comments, and it is not the semantic highlighting the spec asks for. Real semantic regions need the Datara provider, which does not exist yet.
- Reading mode is a layout change, not a different mode. The spec's Reading Mode wants call graphs, dependency views and ownership surfaces. Today it widens the viewport and hides the tree.
- The layout inspector is a text scan, not a parser. It reads multi-line
declaration bodies only, so a single-line
pub struct X { a: Int }is skipped, and generics are taken as text. It recognises all four declaration keywords (struct,class,entity,record), which it did not before - it matchedstructalone, so a project usingclassreported zero structs. It mirrors forgen's optimizer rather than calling it, so if the optimizer changes the mirror has to be updated; that file is the thing to fix, and it says so at the top. Its field sort compares bytes as integers rather than usingStr<, becauseStrcomparison is not deterministic on 1.4.0 (finding 13 indocs/COMPILER-NOTES.md). - Deleting is permanent, and an ASCII-only operation. Delete and rename both work in the explorer now, but delete shells out - forgen 1.4.0 has no file-delete builtin - so a name that is not ASCII cannot be spelled for the shell and is refused, with the reason, rather than silently deleting the wrong thing. There is no trash and no undo: the confirmation is the only thing between a mis-click and the loss, which is why it says so in as many words and why the destructive button is the one that has to be aimed at.
- An in-process check.
/api/checkcosts 299 ms and forgen reports 0 ms of compile time for one file and for 2828 alike - all of it is process start. Reusing the compiler as a library removes nearly the whole number, and it is the single largest latency win left in the IDE. - An interactive console.
/api/runalready runs a whitelisted command and returns the real exit code; what is missing is the input line, the history and the streaming. Roughly a day, and it is the one thing people expect an IDE to have that this one does not. - A real document model on the server: open buffers, revisions,
Freshnesswired in, so a slow analysis can never overwrite a newer edit. - A Datara provider: reuse forgen's own lexer and symbol table instead of the regex highlighter. This is the step that makes the IDE compiler-native rather than a text editor with a colour scheme.
- Semantic lenses: types, ownership, effects, capabilities, and the struct layout inspector - the panel that would have turned the layout bug into a five-minute diagnosis instead of a day.
MIT OR Apache-2.0, at your option. Both texts are in the repository:
LICENSE-MIT and LICENSE-APACHE.
SPDX-License-Identifier: MIT OR Apache-2.0
The dual form is the Rust convention, and this project is a Rust, wasm and Datara stack, so it is what contributors and downstream packagers already expect to find. The two licences are not redundant: MIT is the shortest thing to read and the easiest to satisfy, while Apache-2.0 adds an explicit patent grant, a trademark clause and a termination condition. Offering the choice lets a user pick whichever fits their own legal constraints, and for a project that is not trying to sell anything, that is the point - the licence should never be the reason someone does not use it.
Why not copyleft. GPL and AGPL exist to force commercial users to share their changes. With no revenue to protect, that buys little here, while its cost is real: it would block anyone from embedding this interface in a proprietary tool, and it would make the project's own future relicensing impossible without the agreement of every contributor who had ever sent a patch. AGPL is aimed at network services in any case, and a desktop IDE is not one.
Why not public domain. A dedication like Unlicense carries no patent grant and is on uncertain ground in some jurisdictions. For a compiler toolchain, where the patent grant is the clause most likely to matter one day, that is the wrong trade.
The one alternative worth naming is MPL-2.0, which is file-level copyleft: changes to files that are already MPL must be published, but MPL code can be combined with proprietary code freely. If the intent ever changes to "improvements to this must come back", MPL-2.0 is the switch to make, and it is the least restrictive copyleft available.
Contributions. There is no CLA. Contributions come in under the same dual licence they were received under, which is the norm for MIT/Apache-2.0 projects and keeps the barrier to a first patch at zero.
Third-party components that ship inside the built interface are listed, with
their licences and copyright holders, in THIRD-PARTY-NOTICES.md.