From 93d7a6ef369fbe6a03658337ba92e579cde91422 Mon Sep 17 00:00:00 2001 From: KernelGhost-gif Date: Wed, 18 Nov 2026 18:35:02 +0530 Subject: [PATCH 1/4] Add opt-in typed Go runtime and begin Rust GPUI migration --- .github/workflows/studio.yml | 31 +++++ .github/workflows/tellc.yml | 10 +- README.md | 25 ++-- TellC/docs/CI.md | 10 ++ docs/ARCHITECTURE.md | 12 +- docs/IDE.md | 57 ++++---- docs/TYPED_TELL.md | 62 +++++++++ examples/native-typed.tell | 9 ++ native/cmd/tell/main.go | 35 +++-- native/internal/tell/bytecode.go | 58 ++++++-- native/internal/tell/execute.go | 4 +- native/internal/tell/modules.go | 32 ++++- native/internal/tell/parser.go | 146 ++++++++++++++------ native/internal/tell/runtime.go | 18 ++- native/internal/tell/semantic.go | 146 ++++++++++++++++++-- native/internal/tell/typed_dialect.go | 66 +++++++++ native/internal/tell/typed_dialect_test.go | 152 +++++++++++++++++++++ pyproject.toml | 3 +- studio/Cargo.toml | 10 ++ studio/README.md | 29 ++++ studio/src/buffer.rs | 82 +++++++++++ studio/src/main.rs | 104 ++++++++++++++ tell/__main__.py | 4 + tell/ide.py | 6 +- tell/interpreter.py | 7 + 25 files changed, 988 insertions(+), 130 deletions(-) create mode 100644 .github/workflows/studio.yml create mode 100644 TellC/docs/CI.md create mode 100644 docs/TYPED_TELL.md create mode 100644 examples/native-typed.tell create mode 100644 native/internal/tell/typed_dialect.go create mode 100644 native/internal/tell/typed_dialect_test.go create mode 100644 studio/Cargo.toml create mode 100644 studio/README.md create mode 100644 studio/src/buffer.rs create mode 100644 studio/src/main.rs diff --git a/.github/workflows/studio.yml b/.github/workflows/studio.yml new file mode 100644 index 0000000..f78f5e4 --- /dev/null +++ b/.github/workflows/studio.yml @@ -0,0 +1,31 @@ +name: Tell Studio (Rust / GPUI) + +on: + workflow_dispatch: + schedule: + - cron: "41 3 * * 1" + +permissions: + contents: read + +concurrency: + group: tell-studio-${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: true + +jobs: + cargo-test: + runs-on: ubuntu-24.04 + steps: + - uses: actions/checkout@v4 + - uses: dtolnay/rust-toolchain@stable + - name: Install GPUI Linux dependencies + run: | + sudo apt-get update + sudo apt-get install -y \ + libasound2-dev libfontconfig1-dev libfreetype6-dev \ + libx11-dev libx11-xcb-dev libxkbcommon-dev libxkbcommon-x11-dev \ + libxcb-render0-dev libxcb-shape0-dev libxcb-xfixes0-dev \ + libxcb-xkb-dev libxrandr-dev libxi-dev libxcursor-dev \ + libwayland-dev libvulkan-dev pkg-config + - name: Check the GPUI application and run buffer tests + run: cargo test --manifest-path studio/Cargo.toml diff --git a/.github/workflows/tellc.yml b/.github/workflows/tellc.yml index 68904c7..12fdc7d 100644 --- a/.github/workflows/tellc.yml +++ b/.github/workflows/tellc.yml @@ -1,12 +1,10 @@ name: TellC LLVM compiler on: - pull_request: - paths: - - "TellC/**" - - "scripts/build-tellc-macos.sh" - - "README.md" - - ".github/workflows/tellc.yml" + # The embedded LLVM/LLD build is intentionally excluded from normal PRs: + # upstream builds are expensive and the native Go/runtime suites run on PRs. + schedule: + - cron: "17 2 * * 1" workflow_dispatch: permissions: diff --git a/README.md b/README.md index 54740f3..433b705 100644 --- a/README.md +++ b/README.md @@ -2,10 +2,7 @@ **Tell is a sentence-first programming language.** Its goal is to make code read like clear instructions written in ordinary language, while keeping familiar programming capabilities underneath. -Tell has two runtimes: - -- **Native runtime:** a standalone executable written in Go. It runs `.tell` programs directly and does not need Python or Go installed on the user's machine. -- **Python runtime:** the original interpreter, kept for compatibility, experimentation, and people who want to embed or extend Tell from Python. +Tell's primary runtime is the standalone Go executable. The Python interpreter is deprecated and remains available only as a compatibility bridge while existing projects migrate. New language features target Go; new desktop editor work targets Rust/GPUI Studio. Current version: **0.3.7**. @@ -111,9 +108,9 @@ build/tellc-macos-arm64/tellc/tellc TellC/examples/factorial.tell -o factorial The example prints `120`. The standalone compiler embeds LLVM code generation and the Mach-O linker; it and its output target macOS 11 or later. A Linux build uses a compatible LLVM installation, `llc`, and Clang. Build details, dependencies, and tests are in the [TellC guide](TellC/README.md), [build guide](TellC/docs/BUILDING.md), and [test guide](TellC/docs/TESTING.md). `tellc source.tell` defaults to `a.out`; use `-o` to choose the output and `tellc --llvm-version` to inspect its LLVM version. -## Run the Python version +## Legacy Python runtime (deprecated) -Python 3.10 or newer is required for this runtime only. +Python 3.10 or newer is required only for this legacy compatibility runtime. New development should target the native Go runtime. ```bash python tell.py examples/hello.tell @@ -132,15 +129,17 @@ python -m pip install . tell examples/hello.tell ``` -## Desktop IDE +## Native desktop editor -Tell includes a Python Tk desktop IDE with a tabbed `.tell` editor, line numbers, syntax colours, block-aware indentation, file open/save, Find, and a persistent interactive shell. Run, syntax check, bytecode build, and project tests invoke the native Go runtime when it is available. The Python runtime is the fallback. The IDE is optional and does not change the standalone native runtime. +The replacement editor is the early Rust/GPUI application in [studio/](studio/README.md). It opens one Tell source file, supports basic editing and saving, and runs the native Go executable. Build it from the repository root: -```powershell -python -m tell ide -``` +~~~sh +cargo run --manifest-path studio/Cargo.toml -- path/to/program.tell +~~~ + +Set TELL_NATIVE_EXECUTABLE to the Go executable path when it is not available as tell on PATH. This is an initial editor foundation, not yet a full Zed-style IDE; tabs, diagnostics, language-server integration, undo/redo, and project navigation remain future work. -After installing Tell with `python -m pip install .`, launch it with `tell-ide` (or `tell ide`). Tkinter is included with the standard Windows Python installer; on some Linux distributions install `python3-tk`. See the [IDE guide](docs/IDE.md) for shortcuts and runtime selection. +The Python/Tk IDE is deprecated and remains temporarily available as a compatibility fallback through python -m tell ide, which prints a deprecation warning. New installations no longer include the separate tell-ide console command. See the [migration guide](docs/IDE.md). ## Language features @@ -169,7 +168,7 @@ After installing Tell with `python -m pip install .`, launch it with `tell-ide` | Static methods, access modifiers, multiple inheritance | Planned | Planned | | Async, package manager | Planned | Planned | -The native and Python runtimes aim to share the same core syntax. The Python runtime currently has a broader set of built-ins and standard-library behavior. This is an evolving language, not a claim of complete Python compatibility. +The Go runtime is the primary implementation and the Python runtime is maintained only for compatibility during migration. Cross-runtime differences remain; use the Go executable for new projects. This is an evolving language, not a claim of complete Python compatibility. ## Syntax guide diff --git a/TellC/docs/CI.md b/TellC/docs/CI.md new file mode 100644 index 0000000..502d0c9 --- /dev/null +++ b/TellC/docs/CI.md @@ -0,0 +1,10 @@ +# LLVM compiler workflow + +TellC's full LLVM/LLD jobs are resource-intensive. They run on a weekly +schedule and through manual workflow dispatch, not on ordinary pull requests. +Fast Go/runtime checks remain part of normal PR validation. + +To run the full matrix manually, open **Actions → TellC LLVM compiler → +Run workflow**. The workflow validates Linux with the external LLVM toolchain, +Apple silicon with embedded LLVM + LLD, and Intel macOS with the external +toolchain. diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index d3cc0cb..3636186 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -4,14 +4,14 @@ Both runtimes parse block headers with explicit prefix and delimiter routines, including nested otherwise-if chains. The parser now builds expression ASTs for common `Let`, `Set`, instance member assignment, output, assertion, return, raise, loop-control, bare expression, and `Tell object to method` statements. The expression AST includes tuple literals with shared runtime semantics. Other statements still use the line-oriented compatibility recognizers. -Tell has two independent runtimes: +Tell is moving toward one primary native runtime. The Python implementation is deprecated and remains only as a compatibility runtime during migration. New language features should target Go; new desktop editor features should target Rust/GPUI Studio. - `native/internal/tell` is a Go interpreter compiled into standalone binaries. Its block parser retains source line/column information and represents migrated statements with a `simpleStatement` containing a parsed expression AST. The runtime evaluates that AST directly; the bytecode compiler also lowers it directly rather than reparsing the statement text. Remaining statement families still use compatibility recognizers. Its expression lexer recognizes human-language operators directly, supports Unicode expression identifiers, and records offsets and Unicode-aware line/column coordinates. A few prefix and object-construction phrases remain syntax rewrites. -- `tell/interpreter.py` parses line-oriented blocks into nodes with typed dictionaries for migrated statements, and uses its own Tell lexer, Pratt parser, and expression nodes for the shared expression grammar. Python expression evaluation now uses Tell ASTs exclusively; host-only Python expression forms fail Tell parsing rather than falling through to a second expression language. Control-flow conditions are parsed into cached ASTs before execution. +- `tell/interpreter.py` is the deprecated Python compatibility interpreter. It parses line-oriented blocks into nodes with typed dictionaries for migrated statements, and uses its own Tell lexer, Pratt parser, and expression nodes for the shared expression grammar. Python expression evaluation now uses Tell ASTs exclusively; host-only Python expression forms fail Tell parsing rather than falling through to a second expression language. Control-flow conditions are parsed into cached ASTs before execution. Both runtimes own separate environments, built-ins, class/instance implementations, module loaders, and standard-library adapters. Local modules execute in an environment parented by the runtime globals, export public top-level bindings, and are cached by resolved file path per interpreter. `conformance/core.json` is the shared executable corpus. Each runtime test runner executes those same programs and compares output against the declared expected output. -The optional Python adapter is deliberately outside ordinary runtime startup. The Python implementation imports requested modules in its current process. The native Go runtime invokes a configured Python executable per foreign function call and exchanges JSON values over stdin/stdout. Named and positional arguments work; opaque Python objects and persistent Python-side module state do not cross calls. This keeps the default Go runtime independent of Python while making the dependency explicit for programs that choose the adapter. +The optional Python adapter is deliberately outside ordinary Go runtime startup. The Python runtime is deprecated and should not be used as the target for new features. The Python implementation imports requested modules in its current process. The native Go runtime invokes a configured Python executable per foreign function call and exchanges JSON values over stdin/stdout. Named and positional arguments work; opaque Python objects and persistent Python-side module state do not cross calls. This keeps the default Go runtime independent of Python while making the dependency explicit for programs that choose the adapter. ## Current execution path @@ -36,7 +36,7 @@ It supports `Let`, `Set`, `Remember` field initialization, output statements, ba ## Persistent sessions -The Python `Interpreter` retains its global environment between `run` calls. The native `Session` API provides the same behavior with `NewSession` or `NewSessionWithLimits` and `Exec`. Both REPLs rely on this persistence and pass the same execution-limit settings used by command-line program runs. +The deprecated Python `Interpreter` retains its global environment between `run` calls. The native `Session` API provides the same behavior with `NewSession` or `NewSessionWithLimits` and `Exec`. Both REPLs rely on this persistence and pass the same execution-limit settings used by command-line program runs. ## Architectural priorities @@ -44,6 +44,8 @@ The Python `Interpreter` retains its global environment between `run` calls. The 2. Replace expression phrase rewriting with explicit tokens in both runtimes while retaining the English operator grammar. 3. Separate parser, AST, diagnostics, runtime, standard library, and CLI code into focused modules/packages. 4. Build shared semantic analysis and a stable IR, then expand bytecode lowering from the current syntax-limited subset. -5. Align numeric, collection, error, and library behavior with cross-runtime conformance cases as each statement family moves to the AST. +5. Continue migrating the supported typed dialect into the Go parser, semantic checker, runtime, and bytecode format. +6. Keep the Python runtime in compatibility-only mode until native feature coverage and migration guidance are sufficient. +7. Build the Rust/GPUI Studio editor around the Go CLI and the existing Tell language server. See [ENGINEERING_AUDIT.md](ENGINEERING_AUDIT.md) for measured gaps and current validation evidence. diff --git a/docs/IDE.md b/docs/IDE.md index 6702b13..3a4b4ec 100644 --- a/docs/IDE.md +++ b/docs/IDE.md @@ -1,35 +1,46 @@ -# Tell IDE +# Tell Studio and IDE migration -Tell IDE is an optional desktop editor and interactive shell built with Python's Tk interface. It follows the editing and run loop of Python IDLE while executing programs with Tell itself. +The Python/Tk desktop IDE is deprecated. It remains available temporarily as a +compatibility fallback, but new editor work belongs in the Rust/GPUI +application under [studio/](../studio/README.md). -## Start +## Current replacement: Tell Studio -From a Tell source checkout: +Tell Studio is an early native desktop shell built on GPUI, the Rust UI +framework used by Zed. It provides a single-file source workspace, basic +keyboard editing, save, and execution through the native Go runtime. -```console -python -m tell ide -``` +Build and launch it from the repository root: + +~~~sh +cargo run --manifest-path studio/Cargo.toml -- path/to/program.tell +~~~ -After installing the Python package, use either command: +The native Go executable must be available as tell on PATH, or set +TELL_NATIVE_EXECUTABLE to its absolute path. Studio is an early foundation, +not a full Zed-style IDE yet: multi-tab editing, diagnostics, language-server +integration, debugger tooling, and project navigation are future work. -```console -tell ide -tell-ide -``` +## Legacy Python/Tk editor -The UI requires Python 3.10 or newer and Tkinter. Windows Python installers normally include Tkinter. On Linux, install the distribution's `python3-tk` package if needed. The Go runtime is preferred for execution. Set `TELL_NATIVE_EXECUTABLE` to a native Tell executable to select one explicitly. If no native executable is found, the editor falls back to the Python Tell runtime. A Python fallback requires the Tell Python package to be importable by the child process. +For compatibility during the transition only: -## Editor and shell +~~~sh +python -m tell ide +~~~ -- Open multiple `.tell` files in tabs; create, save, and close files with unsaved-change prompts. -- Use syntax colours, line numbers, block-aware indentation, horizontal and vertical scrolling, and Find. -- The bottom shell is persistent: variables and functions survive between entries. Use Shift+Enter to write a multi-line shell entry, or enter a block one line at a time. -- Shell input is routed to a running program while it is waiting for `Ask`; otherwise, it goes to the persistent Tell REPL. -- Run a module with F5, check syntax with F7, build native bytecode with Ctrl+B, and run project tests with Ctrl+T. -- Build output is written beside the source as `.tbc`. The IDE asks before replacing an existing bytecode file. -- Run and check can execute the current unsaved buffer as a temporary source file in the same directory, preserving relative module lookup. Temporary files are removed when the command exits. +This command prints a deprecation warning. The standalone tell-ide Python +console entry point is no longer installed by new package builds. -The IDE is an editor and runtime console, not a debugger or language server. Highlighting is lexical and provides no static diagnostics while typing. Bytecode building requires the native Go runtime; ordinary run/check operations can use either runtime. +The legacy UI requires Python 3.10 or newer and Tkinter. Existing projects may +continue using it while migrating, but new features should be added to +studio/ and the native Go toolchain instead. +## Migration policy -For editor integration with the typed TellC language, see the separate [language server](../lsp/README.md). +The Python interpreter and IDE are not deleted in this change. They remain +compatibility components until supported Go runtime features and the Rust +editor reach sufficient parity. The Python runtime should receive only +security, compatibility, and migration fixes; new runtime features belong in +Go. The IDE should receive only critical compatibility fixes; new editor +features belong in GPUI Studio. diff --git a/docs/TYPED_TELL.md b/docs/TYPED_TELL.md new file mode 100644 index 0000000..8af6e27 --- /dev/null +++ b/docs/TYPED_TELL.md @@ -0,0 +1,62 @@ +# Typed Tell on the native Go runtime + +The Go runtime now has an opt-in typed source dialect. It extends the native +Go parser and runtime; it is separate from the experimental LLVM-based TellC +compiler. + +## Syntax + +Declare variables with their type before the name: + +~~~tell +let number count be 42 +let text title be "Typed Tell" +let bool enabled be true +~~~ + +Declare function parameters with type: name and the return type with +out type: + +~~~tell +to factorial(number: n) out number: + if n <= 1: + give back 1 + end if + give back n * factorial(n - 1) +end function + +let number answer be factorial(5) +say answer +~~~ + +Use out void for a function that does not return a value. Built-in type names +are case-insensitive; custom class names are resolved against the program's +declared types. The runtime currently maps decimal to its numeric +representation and bool to boolean. + +## Enable strict validation + +~~~sh +tell --typed program.tell +tell check --typed program.tell +tell build --typed program.tell +~~~ + +Strict mode requires explicit types on let declarations and typed function +parameters and return types. It performs conservative static checks for +known expressions and enforces parameter and return types at runtime. The +compiler stores signatures in bytecode so the checks also apply to compiled +programs. + +The ordinary dynamic syntax remains the default to preserve existing code. +The Go type system is an incremental typed dialect, not yet a whole-program +type system: unresolved/dynamic values are checked at runtime, collection +element types and generics are not yet supported, and class method overloads, +traits, interfaces, and typed FFI are not part of this initial slice. + +## Migration + +For new programs, prefer the typed form. Existing programs can migrate one +declaration and function at a time, then use --typed to make omissions and +incompatibilities explicit. The Python runtime is deprecated and should not +receive new typed-language features. diff --git a/examples/native-typed.tell b/examples/native-typed.tell new file mode 100644 index 0000000..42a1b9a --- /dev/null +++ b/examples/native-typed.tell @@ -0,0 +1,9 @@ +to factorial(number: n) out number: + if n <= 1: + give back 1 + end if + give back n * factorial(n - 1) +end function + +let number result be factorial(5) +say result diff --git a/native/cmd/tell/main.go b/native/cmd/tell/main.go index c82bb66..6c8a550 100644 --- a/native/cmd/tell/main.go +++ b/native/cmd/tell/main.go @@ -19,7 +19,8 @@ var version = "0.3.7" func main() { check := flag.Bool("check", false, "parse the program without running it") - typeCheck := flag.Bool("types", false, "check statically knowable type annotations (native runtime)") + typeCheck := flag.Bool("types", false, "check statically knowable type annotations") + typedMode := flag.Bool("typed", false, "require explicit variable, parameter, and return types") showVersion := flag.Bool("version", false, "print the Tell version") quietREPL := flag.Bool("quiet-repl", false, "omit terminal prompts and banner (for IDE integrations)") buildOutput := flag.String("output", "", "write compiled bytecode to this path") @@ -192,6 +193,11 @@ func main() { if outputPath == "" { outputPath = strings.TrimSuffix(filename, filepath.Ext(filename)) + ".tbc" } + if *typedMode { + if err := tell.ValidateTypedProgram(string(data), filename); err != nil { + exitWithDiagnostic(err, filename, diagnostic.TypeError) + } + } compiled, err := tell.Compile(string(data), filename) if err != nil { exitWithDiagnostic(err, filename, diagnostic.SyntaxError) @@ -213,16 +219,22 @@ func main() { if filename == "-" { displayName = "" } - checkSource := tell.Check - if *typeCheck { - checkSource = tell.Analyze - } - if err := checkSource(string(data), displayName); err != nil { - kind := diagnostic.SyntaxError + if *typedMode { + if err := tell.ValidateTypedProgram(string(data), displayName); err != nil { + exitWithDiagnostic(err, displayName, diagnostic.TypeError) + } + } else { + checkSource := tell.Check if *typeCheck { - kind = diagnostic.TypeError + checkSource = tell.Analyze + } + if err := checkSource(string(data), displayName); err != nil { + kind := diagnostic.SyntaxError + if *typeCheck { + kind = diagnostic.TypeError + } + exitWithDiagnostic(err, displayName, kind) } - exitWithDiagnostic(err, displayName, kind) } fmt.Printf("Looks good: %s\n", displayName) return @@ -231,6 +243,11 @@ func main() { if filename == "-" { runName = "" } + if *typedMode { + if err := tell.ValidateTypedProgram(string(data), runName); err != nil { + exitWithDiagnostic(err, runName, diagnostic.TypeError) + } + } if err := tell.NewSessionWithLimits(os.Stdin, os.Stdout, limits).Exec(string(data), runName); err != nil { exitWithDiagnostic(err, runName, diagnostic.RuntimeError) } diff --git a/native/internal/tell/bytecode.go b/native/internal/tell/bytecode.go index 4dea1ec..a98cf64 100644 --- a/native/internal/tell/bytecode.go +++ b/native/internal/tell/bytecode.go @@ -43,9 +43,11 @@ type BytecodeModule struct { // BytecodeFunction stores a separately compiled function body and its signature. type BytecodeFunction struct { - Name string `json:"name"` - Parameters []string `json:"parameters"` - Defaults []int `json:"default_functions,omitempty"` + Name string `json:"name"` + Parameters []string `json:"parameters"` + ParameterTypes []string `json:"parameter_types,omitempty"` + ReturnType string `json:"return_type,omitempty"` + Defaults []int `json:"default_functions,omitempty"` Variadic int `json:"variadic_index,omitempty"` HasVariadic bool `json:"has_variadic,omitempty"` Receiver bool `json:"receiver,omitempty"` @@ -162,7 +164,7 @@ func (c *bytecodeCompiler) statements(nodes []node) error { c.line = statement.line switch statement.kind { case "function": - index, err := c.compileFunction(statement.name, statement.params, statement.body, false) + index, err := c.compileFunction(statement.name, statement.params, statement.paramTypes, statement.returnType, statement.body, false) if err != nil { return err } @@ -173,7 +175,7 @@ func (c *bytecodeCompiler) statements(nodes []node) error { compiledClass := BytecodeClass{Name: statement.name, Base: statement.base, Methods: map[string]int{}} hasConstructor := false for _, member := range statement.body { - index, err := c.compileFunction(member.name, member.params, member.body, true) + index, err := c.compileFunction(member.name, member.params, member.paramTypes, member.returnType, member.body, true) if err != nil { return err } @@ -299,7 +301,7 @@ func (c *bytecodeCompiler) statements(nodes []node) error { return nil } -func (c *bytecodeCompiler) compileFunction(name string, params []string, body []node, receiver bool) (int, error) { +func (c *bytecodeCompiler) compileFunction(name string, params, parameterTypes []string, returnType string, body []node, receiver bool) (int, error) { parameterNames := make([]string, len(params)) defaultFunctions := make([]int, len(params)) hasDefaults := false @@ -333,7 +335,7 @@ func (c *bytecodeCompiler) compileFunction(name string, params []string, body [] } functionCompiler.emit("PUSH") functionCompiler.emit("RETURN") - function := BytecodeFunction{Name: name, Parameters: parameterNames, Variadic: max(variadicIndex, 0), HasVariadic: variadicIndex >= 0, Receiver: receiver, Instructions: functionCompiler.code} + function := BytecodeFunction{Name: name, Parameters: parameterNames, ParameterTypes: append([]string(nil), parameterTypes...), ReturnType: returnType, Variadic: max(variadicIndex, 0), HasVariadic: variadicIndex >= 0, Receiver: receiver, Instructions: functionCompiler.code} if hasDefaults { function.Defaults = defaultFunctions } @@ -1194,6 +1196,18 @@ func validateExecutableBytecode(program BytecodeProgram) error { if function.Name == "" || len(function.Instructions) == 0 { return fail("invalid bytecode function at index %d", index) } + if len(function.ParameterTypes) != 0 && len(function.ParameterTypes) != len(function.Parameters) { + return fail("invalid parameter type table for bytecode function %q", function.Name) + } + for _, typeName := range function.ParameterTypes { + // Empty entries preserve the dynamic parameters of legacy functions. + if strings.TrimSpace(typeName) != "" && !identifier(typeName) { + return fail("invalid parameter type for bytecode function %q", function.Name) + } + } + if function.ReturnType != "" && !identifier(function.ReturnType) { + return fail("invalid return type for bytecode function %q", function.Name) + } if len(function.Defaults) != 0 && len(function.Defaults) != len(function.Parameters) { return fail("invalid default-argument table for bytecode function %q", function.Name) } @@ -2059,10 +2073,38 @@ func callBytecodeFunction(r *runtime, function BytecodeFunction, closure *env, f } local.set(parameter, value) } + for index, parameter := range function.Parameters { + if index >= len(function.ParameterTypes) || strings.TrimSpace(function.ParameterTypes[index]) == "" { + continue + } + if value, exists := local.values[parameter]; exists { + if err := r.checkType(function.ParameterTypes[index], value, local, parameter); err != nil { + return nil, err + } + } + } previous := r.global r.global = local defer func() { r.global = previous }() - return executeBytecodeCode(r, function.Instructions, functions, classes) + result, err := executeBytecodeCode(r, function.Instructions, functions, classes) + if err != nil { + return nil, err + } + if function.ReturnType != "" { + if strings.EqualFold(function.ReturnType, "void") { + if result != nil { + return nil, fail("%q declares out void but returned a value", function.Name) + } + } else { + if result == nil { + return nil, fail("%q must return a value of type %s", function.Name, function.ReturnType) + } + if err := r.checkType(function.ReturnType, result, local, function.Name+" return value"); err != nil { + return nil, err + } + } + } + return result, nil } func constructBytecodeInstance(r *runtime, class *bytecodeClassValue, args []any, named map[string]any) (any, error) { diff --git a/native/internal/tell/execute.go b/native/internal/tell/execute.go index 20baa64..631d7aa 100644 --- a/native/internal/tell/execute.go +++ b/native/internal/tell/execute.go @@ -141,7 +141,7 @@ func (r *runtime) execNode(n node, e *env) error { } return nil case "function": - e.set(n.name, &userFunction{name: n.name, params: n.params, body: n.body, closure: e, runtime: r}) + e.set(n.name, &userFunction{name: n.name, params: n.params, paramTypes: n.paramTypes, returnType: n.returnType, body: n.body, closure: e, runtime: r}) return nil case "class": class := &userClass{name: n.name, methods: make(map[string]*userFunction)} @@ -159,7 +159,7 @@ func (r *runtime) execNode(n node, e *env) error { } hasConstructor := false for _, member := range n.body { - function := &userFunction{name: member.name, params: member.params, body: member.body, closure: e, runtime: r} + function := &userFunction{name: member.name, params: member.params, paramTypes: member.paramTypes, returnType: member.returnType, body: member.body, closure: e, runtime: r} if member.kind == "constructor" { if strings.ToLower(member.name) != strings.ToLower(n.name) { return fail("constructor name %q does not match class %q", member.name, n.name) diff --git a/native/internal/tell/modules.go b/native/internal/tell/modules.go index 75831b5..9c43826 100644 --- a/native/internal/tell/modules.go +++ b/native/internal/tell/modules.go @@ -156,15 +156,43 @@ func (f *userFunction) callNamed(args []any, named map[string]any, receiver *use } local.set(parameter, value) } + for index, parameter := range parameterNames { + if index >= len(f.paramTypes) || strings.TrimSpace(f.paramTypes[index]) == "" { + continue + } + if value, exists := local.values[parameter]; exists { + if err := f.runtime.checkType(f.paramTypes[index], value, local, parameter); err != nil { + return nil, err + } + } + } + validateReturn := func(value any) (any, error) { + if f.returnType == "" { + return value, nil + } + if strings.EqualFold(f.returnType, "void") { + if value != nil { + return nil, fail("%q declares out void but returned a value", f.name) + } + return nil, nil + } + if value == nil { + return nil, fail("%q must return a value of type %s", f.name, f.returnType) + } + if err := f.runtime.checkType(f.returnType, value, local, f.name+" return value"); err != nil { + return nil, err + } + return value, nil + } err := f.runtime.execute(f.body, local) if err != nil { var s signal if errors.As(err, &s) && s.kind == "return" { - return s.value, nil + return validateReturn(s.value) } return nil, err } - return nil, nil + return validateReturn(nil) } func invoke(v any, args []any) (any, error) { diff --git a/native/internal/tell/parser.go b/native/internal/tell/parser.go index 408bda4..924233f 100644 --- a/native/internal/tell/parser.go +++ b/native/internal/tell/parser.go @@ -87,6 +87,8 @@ type node struct { condition, name, base string conditionExpr expression params []string + paramTypes []string + returnType string body, otherwise, handler []node simple *simpleStatement } @@ -307,7 +309,13 @@ func parseSimpleStatement(source string, line, column int) *simpleStatement { if separated { name, annotation, annotated := splitFoldOnce(binding, " as ") if !annotated { - name = binding + // Also accept type-first declarations such as "Let number count be 3". + parts := strings.Fields(binding) + if len(parts) == 2 && identifier(parts[0]) && identifier(parts[1]) { + annotation, name, annotated = parts[0], parts[1], true + } else { + name = binding + } } if strings.Contains(binding, ",") && !annotated { if names, valid := parseLoopNames(binding); valid && len(names) > 1 { @@ -658,78 +666,130 @@ func newSetValue(values []any) setValue { return result } -func parseConstructorHeader(header string) (name string, params []string, ok bool) { +func parseConstructorHeader(header string) (name string, params, paramTypes []string, ok bool) { rest, ok := prefixFold(header, "when creating ") if !ok { - return "", nil, false + return "", nil, nil, false } - articleRest := rest - rest, hasArticle := prefixFold(articleRest, "an ") + rest, hasArticle := prefixFold(rest, "an ") if !hasArticle { - rest, hasArticle = prefixFold(articleRest, "a ") + rest, hasArticle = prefixFold(header[len("when creating "):], "a ") } if !hasArticle { - return "", nil, false + return "", nil, nil, false } name, tail := firstWord(rest) if !identifier(name) { - return "", nil, false + return "", nil, nil, false } if tail != "" { tail, ok = prefixFold(tail, "with ") if !ok { - return "", nil, false - } - params = splitAnd(tail) - for index, param := range params { - parsed, valid := parseParameterText(param) - if !valid { - return "", nil, false + return "", nil, nil, false + } + parts := splitAnd(tail) + typed := len(parts) > 0 && strings.Contains(parts[0], ":") + for _, parameter := range parts { + if typed { + typeName, parameterName, separated := splitFoldOnce(parameter, ":") + if !separated || !identifier(typeName) || !identifier(parameterName) { + return "", nil, nil, false + } + params = append(params, parameterName) + paramTypes = append(paramTypes, typeName) + } else { + parsed, valid := parseParameterText(parameter) + if !valid { + return "", nil, nil, false + } + params = append(params, parsed) + paramTypes = append(paramTypes, "") } - params[index] = parsed } if !validParameters(params) { - return "", nil, false + return "", nil, nil, false } } - return name, params, true + return name, params, paramTypes, true } -func parseFunctionHeader(header string) (name string, params []string, ok bool) { +func parseFunctionHeader(header string) (name string, params, paramTypes []string, returnType string, ok bool) { rest, ok := prefixFold(header, "to ") if !ok { rest, ok = prefixFold(header, "define a function called ") } if !ok { - return "", nil, false + return "", nil, nil, "", false } name, tail := firstWord(rest) if !identifier(name) { - return "", nil, false + return "", nil, nil, "", false } - if strings.HasPrefix(tail, "(") && strings.HasSuffix(tail, ")") { - inner := strings.TrimSpace(tail[1 : len(tail)-1]) + + if strings.HasPrefix(tail, "(") { + close := strings.Index(tail, ")") + if close < 0 { + return "", nil, nil, "", false + } + inner := strings.TrimSpace(tail[1:close]) + suffix := strings.TrimSpace(tail[close+1:]) + if suffix != "" { + resultType, hasOut := prefixFold(suffix, "out ") + if !hasOut || !identifier(resultType) { + return "", nil, nil, "", false + } + returnType = resultType + } if inner != "" { - params = splitTop(inner, ",") + parts := splitTop(inner, ",") + typed := strings.Contains(parts[0], ":") + for _, part := range parts { + if typed { + typeName, parameterName, separated := splitFoldOnce(part, ":") + if !separated || !identifier(typeName) || !identifier(parameterName) { + return "", nil, nil, "", false + } + params = append(params, parameterName) + paramTypes = append(paramTypes, typeName) + } else { + parsed, valid := parseParameterText(part) + if !valid { + return "", nil, nil, "", false + } + params = append(params, parsed) + paramTypes = append(paramTypes, "") + } + } + if !validParameters(params) { + return "", nil, nil, "", false + } } } else if tail != "" { - tail, ok = prefixFold(tail, "with ") - if !ok { - return "", nil, false - } - params = splitAnd(tail) - } - for index, param := range params { - parsed, valid := parseParameterText(param) - if !valid { - return "", nil, false + if resultType, hasOut := prefixFold(tail, "out "); hasOut { + if !identifier(resultType) { + return "", nil, nil, "", false + } + returnType = resultType + } else { + tail, hasWith := prefixFold(tail, "with ") + if !hasWith { + return "", nil, nil, "", false + } + parts := splitAnd(tail) + for _, part := range parts { + parsed, valid := parseParameterText(part) + if !valid { + return "", nil, nil, "", false + } + params = append(params, parsed) + paramTypes = append(paramTypes, "") + } + if !validParameters(params) { + return "", nil, nil, "", false + } } - params[index] = parsed - } - if !validParameters(params) { - return "", nil, false } - return name, params, true + return name, params, paramTypes, returnType, true } func stripOptionalThen(condition string) string { @@ -856,7 +916,7 @@ func (p *parser) block(stops map[string]bool) ([]node, string, error) { out = append(out, node{kind: "class", name: name, base: base, body: body, line: lineNo, column: p.columnAt(lineNo)}) continue } - if name, params, ok := parseConstructorHeader(header); hasHeader && ok { + if name, params, paramTypes, ok := parseConstructorHeader(header); hasHeader && ok { body, stop, err := p.block(map[string]bool{"end constructor.": true}) if err != nil { return nil, "", err @@ -864,7 +924,7 @@ func (p *parser) block(stops map[string]bool) ([]node, string, error) { if stop != "end constructor." { return nil, "", fail("constructor on line %d needs 'End constructor.'", lineNo) } - out = append(out, node{kind: "constructor", name: name, params: params, body: body, line: lineNo, column: p.columnAt(lineNo)}) + out = append(out, node{kind: "constructor", name: name, params: params, paramTypes: paramTypes, body: body, line: lineNo, column: p.columnAt(lineNo)}) continue } if condition, ok := prefixFold(header, "if "); hasHeader && ok && condition != "" { @@ -913,7 +973,7 @@ func (p *parser) block(stops map[string]bool) ([]node, string, error) { out = append(out, node{kind: "for", name: strings.Join(loopNames, ","), condition: items, body: body, line: lineNo, column: p.columnAt(lineNo)}) continue } - if name, params, ok := parseFunctionHeader(header); hasHeader && ok { + if name, params, paramTypes, returnType, ok := parseFunctionHeader(header); hasHeader && ok { body, stop, err := p.block(map[string]bool{"end function.": true}) if err != nil { return nil, "", err @@ -921,7 +981,7 @@ func (p *parser) block(stops map[string]bool) ([]node, string, error) { if stop != "end function." { return nil, "", fail("function %q on line %d needs 'End function.'", name, lineNo) } - out = append(out, node{kind: "function", name: name, params: params, body: body, line: lineNo, column: p.columnAt(lineNo)}) + out = append(out, node{kind: "function", name: name, params: params, paramTypes: paramTypes, returnType: returnType, body: body, line: lineNo, column: p.columnAt(lineNo)}) continue } if low == "try:" { diff --git a/native/internal/tell/runtime.go b/native/internal/tell/runtime.go index 71a9173..b230fcc 100644 --- a/native/internal/tell/runtime.go +++ b/native/internal/tell/runtime.go @@ -41,11 +41,13 @@ type namedCallable func([]any, map[string]any) (any, error) type pythonModule struct{ name string } type pythonFunction struct{ module, name string } type userFunction struct { - name string - params []string - body []node - closure *env - runtime *runtime + name string + params []string + paramTypes []string + returnType string + body []node + closure *env + runtime *runtime } type userClass struct { name string @@ -95,10 +97,12 @@ func typeMatches(typeName string, value any, e *env) (bool, error) { return true, nil case "nothing": return value == nil, nil - case "boolean": + case "boolean", "bool": _, ok := value.(bool) return ok, nil - case "number": + case "void": + return value == nil, nil + case "number", "decimal": switch value.(type) { case float64, int, int64: return true, nil diff --git a/native/internal/tell/semantic.go b/native/internal/tell/semantic.go index 0785896..196e519 100644 --- a/native/internal/tell/semantic.go +++ b/native/internal/tell/semantic.go @@ -7,9 +7,17 @@ type semanticBinding struct { inferred string } +type semanticSignature struct { + parameters []string + types []string + returnType string +} + type semanticAnalyzer struct { filename string - classes map[string]string + classes map[string]string + functions map[string]semanticSignature + currentReturn string } var semanticBuiltins = map[string]string{ @@ -23,7 +31,8 @@ var semanticBuiltins = map[string]string{ } var semanticBuiltinTypes = map[string]bool{ - "any": true, "nothing": true, "boolean": true, "number": true, + "any": true, "nothing": true, "boolean": true, "bool": true, "void": true, + "number": true, "decimal": true, "text": true, "list": true, "tuple": true, "set": true, "dictionary": true, "function": true, "class": true, } @@ -39,8 +48,9 @@ func Analyze(source, filename string) error { } func analyzeBytecodeSemantics(nodes []node, filename string) error { - analyzer := semanticAnalyzer{filename: filename, classes: make(map[string]string)} + analyzer := semanticAnalyzer{filename: filename, classes: make(map[string]string), functions: make(map[string]semanticSignature)} analyzer.collectClasses(nodes) + analyzer.collectFunctions(nodes) return analyzer.statements(nodes, map[string]semanticBinding{}, false) } @@ -55,6 +65,34 @@ func (a *semanticAnalyzer) collectClasses(nodes []node) { } } +func (a *semanticAnalyzer) collectFunctions(nodes []node) { + for _, statement := range nodes { + if statement.kind == "function" { + a.functions[statement.name] = semanticSignature{ + parameters: append([]string(nil), statement.params...), + types: append([]string(nil), statement.paramTypes...), + returnType: statement.returnType, + } + } + a.collectFunctions(statement.body) + a.collectFunctions(statement.otherwise) + a.collectFunctions(statement.handler) + } +} + +func normalizeSemanticType(typeName string) string { + switch strings.ToLower(strings.TrimSpace(typeName)) { + case "bool": + return "boolean" + case "void": + return "nothing" + case "decimal": + return "number" + default: + return strings.ToLower(strings.TrimSpace(typeName)) + } +} + func (a *semanticAnalyzer) statements(nodes []node, bindings map[string]semanticBinding, inFunction bool) error { for _, statement := range nodes { switch statement.kind { @@ -68,22 +106,52 @@ func (a *semanticAnalyzer) statements(nodes []node, bindings map[string]semantic local := map[string]semanticBinding{ "this": {inferred: "class:" + statement.name}, } - for _, parameter := range member.params { - local[parameter] = semanticBinding{inferred: "any"} + for index, parameter := range member.params { + typeName := "" + if index < len(member.paramTypes) { + typeName = member.paramTypes[index] + } + if typeName == "" { + local[parameter] = semanticBinding{inferred: "any"} + } else { + if err := a.checkAnnotation(parameter, typeName, "any"); err != nil { + return a.at(member, err) + } + local[parameter] = semanticBinding{annotation: typeName} + } } + previousReturn := a.currentReturn + a.currentReturn = member.returnType if err := a.statements(member.body, local, true); err != nil { + a.currentReturn = previousReturn return err } + a.currentReturn = previousReturn } case "function": bindings[statement.name] = semanticBinding{inferred: "function"} local := map[string]semanticBinding{statement.name: {inferred: "function"}} - for _, parameter := range statement.params { - local[parameter] = semanticBinding{inferred: "any"} + for index, parameter := range statement.params { + typeName := "" + if index < len(statement.paramTypes) { + typeName = statement.paramTypes[index] + } + if typeName == "" { + local[parameter] = semanticBinding{inferred: "any"} + } else { + if err := a.checkAnnotation(parameter, typeName, "any"); err != nil { + return a.at(statement, err) + } + local[parameter] = semanticBinding{annotation: typeName} + } } + previousReturn := a.currentReturn + a.currentReturn = statement.returnType if err := a.statements(statement.body, local, true); err != nil { + a.currentReturn = previousReturn return err } + a.currentReturn = previousReturn case "if": if statement.conditionExpr != nil { if _, err := a.infer(statement.conditionExpr, bindings); err != nil { @@ -138,6 +206,22 @@ func (a *semanticAnalyzer) statement(statement node, bindings map[string]semanti } switch simple.kind { + case "return": + inferred, err := inferExpression(simple.expression) + if err != nil { + return a.at(statement, err) + } + if a.currentReturn != "" { + if strings.EqualFold(a.currentReturn, "void") { + if simple.hasExpression && inferred != "nothing" { + return a.at(statement, fail("void function cannot return a value")) + } + } else if !simple.hasExpression { + return a.at(statement, fail("%q must return a value of type %s", "function", a.currentReturn)) + } else if err := a.checkAnnotation("return value", a.currentReturn, inferred); err != nil { + return a.at(statement, err) + } + } case "let": inferred, err := inferExpression(simple.expression) if err != nil { @@ -215,6 +299,10 @@ func (a *semanticAnalyzer) checkAnnotation(name, annotation, inferred string) er if inferred == "" || inferred == "any" || normalized == "any" { return nil } + normalized = normalizeSemanticType(normalized) + if !strings.HasPrefix(inferred, "class:") { + inferred = normalizeSemanticType(inferred) + } if strings.HasPrefix(inferred, "class:") { actual := strings.TrimPrefix(inferred, "class:") if _, expectedIsClass := a.classes[typeName]; expectedIsClass && a.isSubtype(actual, typeName) { @@ -415,10 +503,13 @@ func (a *semanticAnalyzer) infer(expr expression, bindings map[string]semanticBi } return "any", nil case callExpression: - for _, argument := range value.args { - if _, err := a.infer(argument.value, bindings); err != nil { + argumentTypes := make([]string, len(value.args)) + for index, argument := range value.args { + inferred, err := a.infer(argument.value, bindings) + if err != nil { return "", err } + argumentTypes[index] = inferred } if callee, ok := value.callee.(nameExpression); ok { if _, isClass := a.classes[callee.name]; isClass { @@ -427,6 +518,43 @@ func (a *semanticAnalyzer) infer(expr expression, bindings map[string]semanticBi if result, isBuiltin := semanticBuiltins[callee.name]; isBuiltin { return result, nil } + if signature, exists := a.functions[callee.name]; exists { + positionalIndex := 0 + used := make(map[string]bool) + for index, argument := range value.args { + parameterIndex := -1 + if argument.name != "" { + for candidate, parameter := range signature.parameters { + if parameter == argument.name { + parameterIndex = candidate + break + } + } + } else { + parameterIndex = positionalIndex + positionalIndex++ + } + if parameterIndex < 0 || parameterIndex >= len(signature.parameters) { + continue + } + parameterName := signature.parameters[parameterIndex] + if used[parameterName] { + return "", fail("%q got multiple values for %q", callee.name, parameterName) + } + used[parameterName] = true + if parameterIndex < len(signature.types) && signature.types[parameterIndex] != "" { + if err := a.checkAnnotation(parameterName, signature.types[parameterIndex], argumentTypes[index]); err != nil { + return "", err + } + } + } + if signature.returnType != "" { + if _, class := a.classes[signature.returnType]; class { + return "class:" + signature.returnType, nil + } + return normalizeSemanticType(signature.returnType), nil + } + } } return "any", nil } diff --git a/native/internal/tell/typed_dialect.go b/native/internal/tell/typed_dialect.go new file mode 100644 index 0000000..31b90c6 --- /dev/null +++ b/native/internal/tell/typed_dialect.go @@ -0,0 +1,66 @@ +package tell + +import "strings" + +// ValidateTypedProgram enforces the opt-in statically declared dialect used by +// the native Go runtime. It preserves the dynamic language as the default: +// callers must explicitly select typed mode. +func ValidateTypedProgram(source, filename string) error { + nodes, err := parse(source, filename) + if err != nil { + return err + } + analyzer := semanticAnalyzer{ + filename: filename, + classes: make(map[string]string), + functions: make(map[string]semanticSignature), + } + analyzer.collectClasses(nodes) + analyzer.collectFunctions(nodes) + + var validate func([]node) error + validate = func(statements []node) error { + for _, statement := range statements { + if statement.kind == "function" || statement.kind == "constructor" { + for index, parameter := range statement.params { + if index >= len(statement.paramTypes) || strings.TrimSpace(statement.paramTypes[index]) == "" { + return fail("%s:%d:%d: parameter %q in %q needs an explicit type (use type: name)", filename, statement.line, statement.column, parameter, statement.name) + } + if err := analyzer.checkAnnotation(parameter, statement.paramTypes[index], "any"); err != nil { + return fail("%s:%d:%d: %v", filename, statement.line, statement.column, err) + } + } + if statement.kind == "function" { + if strings.TrimSpace(statement.returnType) == "" { + return fail("%s:%d:%d: function %q needs an explicit return type (use out type or out void)", filename, statement.line, statement.column, statement.name) + } + if err := analyzer.checkAnnotation("return type", statement.returnType, "any"); err != nil { + return fail("%s:%d:%d: %v", filename, statement.line, statement.column, err) + } + } + } + if statement.kind == "statement" && statement.simple != nil && statement.simple.kind == "let" { + if strings.TrimSpace(statement.simple.annotation) == "" { + return fail("%s:%d:%d: variable %q needs an explicit type (use 'let type name be value')", filename, statement.line, statement.column, strings.Join(statement.simple.names, ", ")) + } + if err := analyzer.checkAnnotation(strings.Join(statement.simple.names, ", "), statement.simple.annotation, "any"); err != nil { + return fail("%s:%d:%d: %v", filename, statement.line, statement.column, err) + } + } + if err := validate(statement.body); err != nil { + return err + } + if err := validate(statement.otherwise); err != nil { + return err + } + if err := validate(statement.handler); err != nil { + return err + } + } + return nil + } + if err := validate(nodes); err != nil { + return err + } + return analyzer.statements(nodes, map[string]semanticBinding{}, false) +} diff --git a/native/internal/tell/typed_dialect_test.go b/native/internal/tell/typed_dialect_test.go new file mode 100644 index 0000000..198558c --- /dev/null +++ b/native/internal/tell/typed_dialect_test.go @@ -0,0 +1,152 @@ +package tell + +import ( + "bytes" + "encoding/json" + "strings" + "testing" +) + +const typedFactorialSource = `to factorial(number: n) out number: + if n <= 1: + give back 1 + end if + give back n * factorial(n - 1) +end function +let number result be factorial(5) +say result +` + +func TestTypedDialectSourceExecution(t *testing.T) { + var output bytes.Buffer + if err := RunWithIO(typedFactorialSource, "typed-factorial.tell", strings.NewReader(""), &output); err != nil { + t.Fatalf("typed program failed: %v", err) + } + if got := output.String(); got != "120\n" { + t.Fatalf("unexpected output: %q", got) + } +} + +func TestTypedDialectRejectsRuntimeParameterMismatch(t *testing.T) { + source := `to echo(number: value) out number: + give back value +end function +say echo("not a number") +` + var output bytes.Buffer + err := RunWithIO(source, "typed-argument.tell", strings.NewReader(""), &output) + if err == nil || !strings.Contains(err.Error(), "type mismatch") { + t.Fatalf("expected runtime parameter type mismatch, got %v", err) + } +} + +func TestTypedDialectRejectsRuntimeReturnMismatch(t *testing.T) { + source := `to wrong(number: value) out number: + give back "not a number" +end function +say wrong(1) +` + var output bytes.Buffer + err := RunWithIO(source, "typed-return.tell", strings.NewReader(""), &output) + if err == nil || !strings.Contains(err.Error(), "type mismatch") { + t.Fatalf("expected runtime return type mismatch, got %v", err) + } +} + +func TestTypedDialectRejectsMissingReturn(t *testing.T) { + source := `to incomplete(number: value) out number: + say value +end function +say incomplete(1) +` + var output bytes.Buffer + err := RunWithIO(source, "typed-missing-return.tell", strings.NewReader(""), &output) + if err == nil || !strings.Contains(err.Error(), "must return a value") { + t.Fatalf("expected missing return error, got %v", err) + } +} + +func TestValidateTypedProgramRequiresDeclarations(t *testing.T) { + valid := `to add(number: left, number: right) out number: + give back left + right +end function +let number result be add(20, 22) +say result +` + if err := ValidateTypedProgram(valid, "valid-typed.tell"); err != nil { + t.Fatalf("valid typed source rejected: %v", err) + } + + untypedVariable := "let count be 3\nsay count\n" + if err := ValidateTypedProgram(untypedVariable, "untyped-variable.tell"); err == nil || !strings.Contains(err.Error(), "needs an explicit type") { + t.Fatalf("expected untyped variable to be rejected, got %v", err) + } + + untypedFunction := `to identity(value) out number: + give back value +end function +` + if err := ValidateTypedProgram(untypedFunction, "untyped-function.tell"); err == nil || !strings.Contains(err.Error(), "needs an explicit type") { + t.Fatalf("expected untyped parameter to be rejected, got %v", err) + } +} + +func TestTypedDialectStaticCallChecks(t *testing.T) { + source := `to add(number: left, number: right) out number: + give back left + right +end function +let number result be add("wrong", 2) +say result +` + if err := ValidateTypedProgram(source, "static-call-mismatch.tell"); err == nil || !strings.Contains(err.Error(), "type mismatch") { + t.Fatalf("expected compile-time argument mismatch, got %v", err) + } +} + +func TestTypedDialectBytecodePreservesSignatures(t *testing.T) { + compiled, err := Compile(typedFactorialSource, "typed-factorial.tell") + if err != nil { + t.Fatalf("typed source failed to compile: %v", err) + } + var output bytes.Buffer + if err := RunBytecode(compiled, "typed-factorial.tbc", strings.NewReader(""), &output); err != nil { + t.Fatalf("typed bytecode failed: %v", err) + } + if got := output.String(); got != "120\n" { + t.Fatalf("unexpected bytecode output: %q", got) + } + + dynamicArgument := `to echo(number: value) out number: + give back value +end function +say echo(1) +` + compiled, err = Compile(dynamicArgument, "typed-bytecode-mismatch.tell") + if err != nil { + t.Fatalf("valid typed bytecode failed to compile: %v", err) + } + var program BytecodeProgram + if err := json.Unmarshal(compiled, &program); err != nil { + t.Fatalf("could not decode compiled program: %v", err) + } + changed := false + for index := range program.Instructions { + if program.Instructions[index].Op == "PUSH" { + program.Instructions[index].Value = "not a number" + changed = true + break + } + } + if !changed { + t.Fatal("compiled call had no PUSH argument instruction to mutate") + } + tampered, err := json.Marshal(program) + if err != nil { + t.Fatalf("could not encode test bytecode: %v", err) + } + output.Reset() + err = RunBytecode(tampered, "typed-bytecode-mismatch.tbc", strings.NewReader(""), &output) + if err == nil || !strings.Contains(err.Error(), "type mismatch") { + t.Fatalf("expected bytecode parameter type mismatch, got %v", err) + } +} diff --git a/pyproject.toml b/pyproject.toml index d109116..42a2d31 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -5,7 +5,7 @@ build-backend = "setuptools.build_meta" [project] name = "tell-language" version = "0.3.7" -description = "Tell, a sentence-first polyglot programming language with Python, JavaScript, and native runtimes" +description = "Legacy Python compatibility runtime for Tell; the native Go runtime is recommended" readme = "README.md" requires-python = ">=3.10" license = { text = "MIT" } @@ -13,7 +13,6 @@ authors = [{ name = "Tell Language contributors" }] [project.scripts] tell = "tell.__main__:main" -tell-ide = "tell.ide:main" [tool.setuptools] packages = ["tell"] diff --git a/studio/Cargo.toml b/studio/Cargo.toml new file mode 100644 index 0000000..8984d2a --- /dev/null +++ b/studio/Cargo.toml @@ -0,0 +1,10 @@ +[package] +name = "tell-studio" +version = "0.1.0" +edition = "2021" +description = "Native Tell source editor built with Rust and GPUI" +license = "MIT" + +[dependencies] +gpui = { git = "https://github.com/zed-industries/zed", rev = "f3626706fa6855274a561f8f4aebf40666051596" } +gpui_platform = { git = "https://github.com/zed-industries/zed", rev = "f3626706fa6855274a561f8f4aebf40666051596" } diff --git a/studio/README.md b/studio/README.md new file mode 100644 index 0000000..9972ab6 --- /dev/null +++ b/studio/README.md @@ -0,0 +1,29 @@ +# Tell Studio + +Tell Studio is the first native desktop-editor slice for Tell. It is written in +Rust and uses the GPUI framework from Zed. GPUI is pinned to commit +f3626706fa6855274a561f8f4aebf40666051596 so upstream changes do not silently break the app. + +## Current capabilities + +- Opens a Tell file supplied as the first command-line argument. +- Provides a single-file source buffer with UTF-8-safe cursor movement, insertion, deletion, and a visible caret. +- Saves the current file with the Save button or Ctrl/Command+S. +- Runs the saved file through the native Go CLI with the Run button or Ctrl/Command+R. +- Executes the Go process off the UI thread and displays stdout/stderr. + +## Build + +Install the Rust toolchain and the platform development packages required by GPUI. Then run: + +~~~sh +cargo run --manifest-path studio/Cargo.toml -- path/to/program.tell +~~~ + +Tell Studio searches for tell on PATH. Set TELL_NATIVE_EXECUTABLE to use another native Go binary. + +## Scope and next steps + +This is an editor foundation, not a complete Zed replacement. It currently has one open buffer and a deliberately small editing model. Multi-tab workspaces, native file dialogs, selection and clipboard support, undo/redo, syntax highlighting, diagnostics, LSP integration (including the existing TellC server), debugger integration, and project search remain future work. + +The Python/Tk IDE remains available only as a deprecated compatibility fallback while the Rust app and Go toolchain gain the required features. No claim of reduced memory or CPU use is made until the new app is benchmarked against the legacy IDE on the same workloads. diff --git a/studio/src/buffer.rs b/studio/src/buffer.rs new file mode 100644 index 0000000..488c0ba --- /dev/null +++ b/studio/src/buffer.rs @@ -0,0 +1,82 @@ +use std::{ fs, io, path::{Path, PathBuf} }; + +/// UTF-8 source buffer with a byte-safe caret and a single-file save target. +#[derive(Debug, Clone)] +pub struct SourceBuffer { path: PathBuf, text: String, cursor: usize, dirty: bool } + +impl SourceBuffer { + pub fn new(path: Option) -> io::Result { + let path = path.unwrap_or_else(|| PathBuf::from("untitled.tell")); + let text = match fs::read_to_string(&path) { + Ok(contents) => contents, + Err(error) if error.kind() == io::ErrorKind::NotFound => String::new(), + Err(error) => return Err(error), + }; + let cursor = text.len(); + Ok(Self { path, text, cursor, dirty: false }) + } + pub fn path(&self) -> &Path { &self.path } + pub fn text(&self) -> &str { &self.text } + pub fn is_dirty(&self) -> bool { self.dirty } + pub fn editor_text(&self) -> String { let mut value = self.text.clone(); value.insert_str(self.cursor, "▏"); value } + pub fn insert(&mut self, value: &str) { + if value.is_empty() { return; } + self.text.insert_str(self.cursor, value); + self.cursor += value.len(); + self.dirty = true; + } + pub fn backspace(&mut self) { + if self.cursor == 0 { return; } + let start = self.text[..self.cursor].char_indices().next_back().map(|(index, _)| index).unwrap_or(0); + self.text.replace_range(start..self.cursor, ""); + self.cursor = start; + self.dirty = true; + } + pub fn move_left(&mut self) { + if self.cursor == 0 { return; } + self.cursor = self.text[..self.cursor].char_indices().next_back().map(|(index, _)| index).unwrap_or(0); + } + pub fn move_right(&mut self) { + if self.cursor >= self.text.len() { return; } + let width = self.text[self.cursor..].chars().next().map(char::len_utf8).unwrap_or(0); + self.cursor = (self.cursor + width).min(self.text.len()); + } + pub fn save(&mut self) -> io::Result<&Path> { + fs::write(&self.path, self.text.as_bytes())?; + self.dirty = false; + Ok(&self.path) + } +} + +#[cfg(test)] +mod tests { + use super::SourceBuffer; + use std::{fs, path::PathBuf, time::{SystemTime, UNIX_EPOCH}}; + fn unique_path() -> PathBuf { + let nonce = SystemTime::now().duration_since(UNIX_EPOCH).unwrap().as_nanos(); + std::env::temp_dir().join(format!("tell-studio-{nonce}.tell")) + } + #[test] + fn editing_is_utf8_boundary_safe_and_marks_dirty() { + let path = unique_path(); + let mut buffer = SourceBuffer::new(Some(path.clone())).unwrap(); + buffer.insert("λTell"); + assert_eq!(buffer.text(), "λTell"); + buffer.move_left(); + buffer.backspace(); + assert_eq!(buffer.text(), "λTel"); + assert!(buffer.is_dirty()); + let _ = fs::remove_file(path); + } + #[test] + fn save_and_reload_round_trip_source() { + let path = unique_path(); + let mut buffer = SourceBuffer::new(Some(path.clone())).unwrap(); + buffer.insert("say \"hello\"\n"); + buffer.save().unwrap(); + assert!(!buffer.is_dirty()); + let reloaded = SourceBuffer::new(Some(path.clone())).unwrap(); + assert_eq!(reloaded.text(), "say \"hello\"\n"); + let _ = fs::remove_file(path); + } +} diff --git a/studio/src/main.rs b/studio/src/main.rs new file mode 100644 index 0000000..1b9ba13 --- /dev/null +++ b/studio/src/main.rs @@ -0,0 +1,104 @@ +mod buffer; +use buffer::SourceBuffer; +use gpui::{App, Bounds, Context, FocusHandle, KeyDownEvent, Render, Window, WindowBounds, WindowOptions, div, prelude::*, px, rgb, size}; +use gpui_platform::application; +use std::{env, path::PathBuf, process::Command}; + +struct Studio { focus_handle: FocusHandle, buffer: SourceBuffer, status: String, output: String } + +impl Studio { + fn new(_window: &mut Window, cx: &mut Context, source_path: Option) -> Self { + match SourceBuffer::new(source_path) { + Ok(buffer) => Self { focus_handle: cx.focus_handle(), buffer, status: "Ready · Ctrl/⌘+S to save · Ctrl/⌘+R to run".into(), output: String::new() }, + Err(error) => Self { focus_handle: cx.focus_handle(), buffer: SourceBuffer::new(None).expect("create an empty Tell buffer"), status: format!("Could not open file: {error}"), output: String::new() }, + } + } + fn save(&mut self) { + match self.buffer.save() { + Ok(path) => self.status = format!("Saved {}", path.display()), + Err(error) => self.status = format!("Save failed: {error}"), + } + } + fn run_current(&mut self, cx: &mut Context) { + let path = match self.buffer.save() { + Ok(path) => path.to_path_buf(), + Err(error) => { self.status = format!("Save before run failed: {error}"); cx.notify(); return; } + }; + let executable = env::var_os("TELL_NATIVE_EXECUTABLE").unwrap_or_else(|| "tell".into()); + self.status = format!("Running {}", path.display()); + self.output.clear(); + cx.spawn(async move |this, cx| { + let command_path = path.clone(); + let result = cx.background_executor().spawn(async move { + Command::new(executable).arg("run").arg(&command_path).output() + }).await; + let (status, output) = match result { + Ok(result) => { + let mut text = String::from_utf8_lossy(&result.stdout).into_owned(); + text.push_str(&String::from_utf8_lossy(&result.stderr)); + if result.status.success() { (format!("Finished {}", path.display()), text) } + else { (format!("Tell exited with {}", result.status), text) } + } + Err(error) => (format!("Could not start the Go runtime: {error}"), "Set TELL_NATIVE_EXECUTABLE or put the native tell binary on PATH.".into()), + }; + let _ = this.update(cx, |view, cx| { view.status = status; view.output = output; cx.notify(); }); + }).detach(); + cx.notify(); + } + fn on_key_down(&mut self, event: &KeyDownEvent, _window: &mut Window, cx: &mut Context) { + let key = event.keystroke.key.to_ascii_lowercase(); + let modifiers = &event.keystroke.modifiers; + if modifiers.control || modifiers.platform { + match key.as_str() { "s" => self.save(), "r" => self.run_current(cx), _ => return } + cx.notify(); return; + } + match key.as_str() { + "backspace" => self.buffer.backspace(), + "left" => self.buffer.move_left(), + "right" => self.buffer.move_right(), + "enter" | "return" => self.buffer.insert("\n"), + "tab" => self.buffer.insert(" "), + _ => { if modifiers.alt { return; } if let Some(value) = event.keystroke.key_char.as_deref() { self.buffer.insert(value); } else { return; } } + } + self.status = if self.buffer.is_dirty() { format!("{} · unsaved", self.buffer.path().display()) } else { "Ready".into() }; + cx.notify(); + } + fn toolbar_button(label: &'static str, action: impl Fn(&mut Self, &mut Context) + 'static, cx: &mut Context) -> impl IntoElement { + div().id(label).px_3().py_2().rounded_md().bg(rgb(0x263449)).text_color(rgb(0xe7edf5)).cursor_pointer().hover(|style| style.bg(rgb(0x354a65))).child(label).on_click(cx.listener(move |this, _, _, cx| action(this, cx))) + } +} + +impl Render for Studio { + fn render(&mut self, _window: &mut Window, cx: &mut Context) -> impl IntoElement { + let save = Self::toolbar_button("Save", |this, cx| { this.save(); cx.notify(); }, cx); + let run = Self::toolbar_button("Run", |this, cx| this.run_current(cx), cx); + let file_name = self.buffer.path().display().to_string(); + let editor_text = self.buffer.editor_text(); + let output = if self.output.is_empty() { "Run output appears here.".to_string() } else { self.output.clone() }; + div().id("tell-studio").size_full().flex().flex_col().gap_3().p_4().bg(rgb(0x101722)).text_color(rgb(0xe7edf5)) + .child(div().flex().items_center().justify_between() + .child(div().flex().flex_col().gap_1() + .child(div().text_xl().font_weight(gpui::FontWeight::BOLD).child("Tell Studio")) + .child(div().text_sm().text_color(rgb(0x9aa9bd)).child(file_name))) + .child(div().flex().gap_2().child(save).child(run))) + .child(div().flex_1().min_h(px(280.)).p_3().rounded_md().border_1().border_color(rgb(0x35465b)).bg(rgb(0x172231)) + .track_focus(&self.focus_handle) + .on_key_down(cx.listener(Self::on_key_down)) + .on_click(cx.listener(|this, _, window, cx| { window.focus(&this.focus_handle, cx); })) + .child(editor_text)) + .child(div().h(px(26.)).text_sm().text_color(rgb(0xa5b5c8)).child(self.status.clone())) + .child(div().h(px(112.)).p_3().rounded_md().border_1().border_color(rgb(0x263449)).bg(rgb(0x0c121b)).child(output)) + } +} + +fn main() { + let source_path = env::args_os().nth(1).map(PathBuf::from); + application().run(move |cx: &mut App| { + let bounds = Bounds::centered(None, size(px(1100.), px(760.)), cx); + let source_path = source_path.clone(); + cx.open_window(WindowOptions { window_bounds: Some(WindowBounds::Windowed(bounds)), ..Default::default() }, + |window, cx| cx.new(|cx| Studio::new(window, cx, source_path.clone()))) + .expect("failed to open Tell Studio window"); + cx.activate(true); + }); +} diff --git a/tell/__main__.py b/tell/__main__.py index e6a40d7..19342f3 100644 --- a/tell/__main__.py +++ b/tell/__main__.py @@ -202,6 +202,10 @@ def main() -> int: return repl(max_call_depth=args.max_call_depth, max_loop_iterations=args.max_loop_iterations, quiet=args.quiet_repl) if command == "ide" and args.file is None: + print( + "warning: the Python/Tk IDE is deprecated. Use Tell Studio (Rust + GPUI) for new editor development.", + file=sys.stderr, + ) from .ide import main as ide_main return ide_main() diff --git a/tell/ide.py b/tell/ide.py index b4781fb..d45e5cc 100644 --- a/tell/ide.py +++ b/tell/ide.py @@ -1,4 +1,8 @@ -"""Tk desktop IDE for editing, running, and interactively exploring Tell.""" +"""Deprecated Tk compatibility IDE. + +The replacement editor is the Rust/GPUI application under studio/. This +module remains available as a transition fallback for existing installations. +""" from __future__ import annotations diff --git a/tell/interpreter.py b/tell/interpreter.py index fdfe63b..7303b8c 100644 --- a/tell/interpreter.py +++ b/tell/interpreter.py @@ -1,3 +1,10 @@ +"""Deprecated Python compatibility runtime. + +New language and performance work targets the native Go runtime. This module +remains available during the migration so existing Python integrations keep +working while their dependencies are moved. +""" + from __future__ import annotations import csv From 30287a717dbb886f18c6177b21449d2840bf7f6f Mon Sep 17 00:00:00 2001 From: KernelGhost-gif Date: Wed, 18 Nov 2026 18:38:06 +0530 Subject: [PATCH 2/4] Move Go-only typed example out of the legacy shared examples --- docs/TYPED_TELL.md | 3 +++ {examples => native/examples}/native-typed.tell | 0 2 files changed, 3 insertions(+) rename {examples => native/examples}/native-typed.tell (100%) diff --git a/docs/TYPED_TELL.md b/docs/TYPED_TELL.md index 8af6e27..909d31a 100644 --- a/docs/TYPED_TELL.md +++ b/docs/TYPED_TELL.md @@ -54,6 +54,9 @@ type system: unresolved/dynamic values are checked at runtime, collection element types and generics are not yet supported, and class method overloads, traits, interfaces, and typed FFI are not part of this initial slice. + +A runnable native example is available at [native/examples/native-typed.tell](../native/examples/native-typed.tell). + ## Migration For new programs, prefer the typed form. Existing programs can migrate one diff --git a/examples/native-typed.tell b/native/examples/native-typed.tell similarity index 100% rename from examples/native-typed.tell rename to native/examples/native-typed.tell From 16a8497f4825f1f4778a108fc38103e5f3276236 Mon Sep 17 00:00:00 2001 From: KernelGhost-gif Date: Wed, 18 Nov 2026 18:40:12 +0530 Subject: [PATCH 3/4] Document the native Go typed dialect in the project guide --- README.md | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/README.md b/README.md index 433b705..30565d8 100644 --- a/README.md +++ b/README.md @@ -78,6 +78,30 @@ tell run examples/hello.tbc Use `tell build source.tell --output output.tbc` to choose the artifact path. The compiler supports functions, recursion, closures, core control flow, local modules embedded in the artifact, `Try`/`Raise`, and the initial class model with constructors, fields, methods, single inheritance, and overrides. `Finally` and other language features remain unsupported in bytecode; see the [CLI reference](docs/CLI_REFERENCE.md) for its exact subset and limits. [examples/compiled-objects.tell](examples/compiled-objects.tell) demonstrates an object program that can be run by either interpreter or compiled to bytecode by the native runtime. +## Typed Tell on the native Go runtime + +The Go runtime supports an opt-in typed dialect with type-first declarations +and explicit function signatures: + +~~~tell +to factorial(number: n) out number: + if n <= 1: + give back 1 + end if + give back n * factorial(n - 1) +end function + +let number answer be factorial(5) +say answer +~~~ + +Use `tell --typed native/examples/native-typed.tell` to require explicit +types for variables, function parameters, and return values. The same strict +validation is available through `tell check --typed` and `tell build --typed`. +Typed parameter and return contracts are enforced during normal execution +and preserved in bytecode. Existing dynamic Tell programs remain supported +without `--typed`; see the [typed Go dialect guide](docs/TYPED_TELL.md). + ## Compile native executables with TellC TellC is an experimental C++ compiler that translates its typed dialect through LLVM into a native executable. Its dialect is separate from the dynamically typed language used by the Go and Python runtimes; programs are not necessarily interchangeable. For example: From 1dd4d8af770b27ab8f2b51fcb821b60c701cc49b Mon Sep 17 00:00:00 2001 From: KernelGhost-gif Date: Wed, 18 Nov 2026 18:43:04 +0530 Subject: [PATCH 4/4] Document typed runtime performance roadmap --- docs/TYPED_TELL.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/docs/TYPED_TELL.md b/docs/TYPED_TELL.md index 909d31a..ba971f1 100644 --- a/docs/TYPED_TELL.md +++ b/docs/TYPED_TELL.md @@ -57,6 +57,20 @@ traits, interfaces, and typed FFI are not part of this initial slice. A runnable native example is available at [native/examples/native-typed.tell](../native/examples/native-typed.tell). +## Performance status + +This first Go typed-dialect increment adds validation and runtime contract +checks; it does not yet specialize the VM's dynamic value representation. +Go execution still uses the existing general-purpose runtime and bytecode +operations. Do not treat this change as a measured speedup in execution, +memory use, or compilation time. + +The next performance phase should use typed semantic information to lower +verified functions to specialized bytecode operations and compact value +representations where benchmarks justify them. Any such change should retain +a dynamic compatibility path and be measured against representative Tell +programs before performance claims are made. + ## Migration For new programs, prefer the typed form. Existing programs can migrate one