[transfix]

Why

src/cvc/state/state_exec/stdlib.cpp registers the string, bytes, math and collections modules, including the Unicode-aware string.length/char-at/substring and string.encode/bytes.decode from the SE-1/SE-3 work. None of it was reachable from a DSL program. Only stdlib.cpp and two tests referenced stdlib_registry, and builtins::make_default_environment() didn't import it. So on master, (string.length "x") fails with unknown function or special form: string.length on the sync and async schedulers, in Ariadne's program lanes and predicates, in pycvc's Exec and in cvc exec.

The stdlib arrived with the engine in #93 and was never wired in. CVCGL-UI-DSL-ROADMAP.md §7.3 had already decided on "full stdlib by default" for the predicate and action envs.

What

Always loaded, shared, not copied.

• stdlib_registry::standard() is the built-in registry, built once.
• stdlib_registry::shared_scope() is one frozen environment holding every stdlib function, also built once.
• make_default_environment() links that scope as its outer, so every host gets the modules from one place without copying them.
• All names are module-qualified, so nothing clashes with builtins (length, range, slice are unchanged).

Frozen scopes.

• environment::frozen: set() throws on a frozen scope.
• set_existing() binds in the outermost unfrozen scope instead of writing into a frozen one. It is now a loop rather than recursion.
• So a program's (set string.length 42) or (defun string.upper ...) shadows the name for that program only. Nothing writes the shared scope, so threads can read it concurrently.

Ariadne predicates (kAllowed) get the pure, bounded subset, where the result is a scalar or no bigger than the argument:

• string.length, byte-length, char-at, substring, upper, lower, trim, starts-with, ends-with, contains;
• bytes.length and byte-at;
• every math.* function except math.random, which would make a predicate nondeterministic.

The materializers (string.join/replace/split/encode, bytes.slice/concat/decode, collections.*) stay out of predicates until they run as scheduled, sliced work: the executor-jobs follow-up. The init, action and resident lanes get everything.

Per-call allocation limit. A native call builds its whole result inside one step. max_memory is checked between steps and counts only state-tree writes, so it couldn't see a call like (collections.range 1000000000).

• eval_alloc_guard and eval_alloc_check() sit next to eval_deadline_guard in types.h.
• Both schedulers arm the guard around each step with the tighter of two limits:
  • the tree setting limits.max_alloc_bytes, resolved like max_pending_messages: state_exec.schedulers.<id>.limits.max_alloc_bytes, then state_exec.defaults.limits.max_alloc_bytes, then 16 MiB. 0 means no limit of its own.
  • what's left of the process's max_memory.
• The sync scheduler resolves the setting in load_settings() and publishes the effective value. The async scheduler (the app's exec scheduler and pycvc's Exec) reads the defaults key on every submission.
• With no guard armed (a bare evaluator, or a direct C++ call) the limit is the 16 MiB default.
• These functions estimate their result size and throw, naming themselves, before allocating:
  • string.split, string.join, string.replace, bytes.concat;
  • collections.range, collections.zip, collections.flatten, collections.dict-merge;
  • the builtins str-concat and collect.

Ariadne action budget. kActionMaxBytes was a fixed 256 KiB. It's now the tree setting ariadne.limits.action_max_memory, read on every submission. The default is 4 MiB, and 0 means no limit. Since the per-step allocation limit takes what's left of max_memory, keeping 256 KiB would have capped every native call in an action at 256 KiB too. docs/ARIADNE.md has the new table. Test: AriadneStdlib.ActionMemoryBudgetIsATreeSetting checks the default, a 64 KiB budget, 0, and an unparsable value.

Step guard cost (86ac286). An unconditional per-step guard cost a memory-tracker lookup and thread-local writes every step, making a plain loop about 6-8% slower. The schedulers now arm it only for a process with a memory budget or a non-default limits.max_alloc_bytes; for everything else the default applies unarmed, with the same limit.

Fixes found along the way.

• collections.range counts its elements in unsigned arithmetic. The old loop had signed overflow (UB) near INT64_MAX/INT64_MIN.
• string.replace builds its result in one pass with the same non-overlapping, left-to-right semantics. The old version did an in-place replace per match, which is O(n·k).
• collections.unique and collections.dict-merge are quadratic, so they now poll the evaluation deadline.

Docs. The developer guide's §11 is rewritten: how the modules load, shadowing, the allocation limit, the predicate subset, and per-module examples. §7's settings table gets the new key. Every ;; => example in §11 was evaluated through pycvc against this build and matches. The old examples had several mistakes:

• the collections.map argument order was wrong, and it showed lambdas, which aren't accepted;
• substring was documented as (start, end) instead of (start, length);
• floor/ceil/round were shown returning floats;
• booleans were shown as true rather than #t.

Tests

• New state_exec_stdlib_test (20 cases):
  • reach through the sync scheduler, the async scheduler and an explicit make_default_environment() env;
  • every module, and builtins keeping their short names;
  • the shared scope is linked and not copied;
  • set, defun and set_existing shadow; writing a frozen scope throws;
  • the default limit; max_memory tightening it; tree resolution per scheduler and then defaults, including a bad value and the published value; 0 = none; guard nesting;
  • each growing result is checked under a tight limit;
  • replace/split/join semantics, and range at the int64 ends.
• ariadne_runtime_test (127): the predicate subset is shown, the materializers and math.random are hidden and reported, the init and action lanes reach the whole stdlib, and an action over limits.max_alloc_bytes is killed and reported, then succeeds once the limit fits.
• test_pycvc_exec.py: the modules are loaded in Exec, shadowing is per program, and the limit comes from the app tree.
• state_exec_intrinsics_test: StdlibRegistryTest.ImportAll and ImportSpecific now use a bare environment, since a default one already reaches the stdlib.

Local results (MSVC 14.38 Release, Windows, deps from the cvcpkg prefix):

• On master, the new reach tests fail with unknown function or special form: string.length on both schedulers. With this change:
  • all 21 state_exec, Ariadne, async_lane and state_list suites pass, 1309 tests in all (ariadne_runtime 126, state_exec_intrinsics 156, state_exec_stdlib 20, ...);
  • pycvc (CVC_BUILD_PYCVC=ON, SWIG 4.4.1, Python 3.11 from the prefix): test_pycvc_exec, _exec_async, _exec_coro, _ari and _state give 41 passed;
  • cvc exec -e '(string.length "café")' prints 4, and (collections.range 1000000000) exits 1 with the allocation-limit error;
  • CvcCliTest: 171/173 pass, including every Exec* case. The two failures are Vol2ImgExportsSlices and Img2VolImportsSlices: ImageMagick's CoderModulesPath registry lookup fails on this box, which is unrelated;
  • check_test_targets.py is OK, and clang-format 18 is clean.
  • After merging master (gl: physics bodies are drawn at their pose anywhere in the scene hierarchy #654-world: one world_geometry for render, physics and nav; the city bundle loader (G3a) #656, which brings PEGTL): rebuilt, and the same suites plus state_exec_pegtl_test pass (22 suites, 1313 tests); pycvc still gives 41 passed.

Not in this PR

• Scheduled, sliced versions of the materializers, so a long string.join or collections.sort runs a slice per step against max_steps/max_time and can join the predicate subset. This is the resumable-job mechanism planned under "deep-data work on the executor" (stacked on state_exec: read submitted scripts inside the process, a slice per step #653 and state_exec: free, print, compare, copy and snapshot deep values without native recursion #657).
• collections.map, filter and reduce still take only native functions ((collections.map xs math.sqrt) works, a DSL lambda doesn't). Calling a closure from a native needs the same job mechanism.
• The allocation check is a pre-allocation bound per call. Live in-process values still aren't charged to max_memory.
• The snapshot codec encodes the frozen stdlib scope as an empty scope (it skips native bindings, as before). No production path decodes evaluator states today.

Conflicts

This touches lines that #649/#653 (scheduler.cpp/async_scheduler.cpp settings and submit) and #657 (environment::set_existing, now a loop in both) also change. Whichever merges second rebases; each conflict is a few lines.