state_exec: max_memory covers the measured heap; collect reference cycles

bindings/pycvc/test_pycvc_exec.py

@@ -335,6 +335,23 @@ def run_error(src):
     assert ex.run("(length (collections.range 1000))") == "1000"
 
 
+# ── reference cycles ────────────────────────────────────────────────────
+
+
+def test_exec_functions_survive_cycle_collection():
+    app = pycvc.make_app()
+    ex = pycvc.Exec(app)
+    ex.run("(defun twice (x) (* 2 x))")
+    # Each iteration leaks a let scope (a closure bound in the scope it captured). That many make
+    # the scheduler collect cycles between steps, and Exec's own environment, held by Exec, must
+    # come through intact.
+    ex.run(
+        "(begin (set i 0) (while (< i 1000) (begin (let ((h nil)) (set h (lambda () h))) "
+        "(set i (+ i 1)))) i)"
+    )
+    assert ex.run("(twice 21)") == "42"
+
+
 if __name__ == "__main__":
     tests = [v for k, v in sorted(globals().items()) if k.startswith("test_") and callable(v)]
     for t in tests:

docs/ARIADNE.md

@@ -128,9 +128,12 @@ per-activation caps ([`ariadne.cpp:1382`](../src/cvc/ariadne/ariadne.cpp)):
 | wall-time cap | `kActionMaxSeconds` | `0.1 s` |
 | memory budget | tree setting `ariadne.limits.action_max_memory` | `4 MiB` by default; `0` = no limit |
 
-The memory budget is read on every submission. It bounds the action's state-tree writes and
-also each native call's allocations: state_exec's per-step allocation limit takes what is left
-of `max_memory` (developer guide §11, "Allocation limit").
+The memory budget is read on every submission. It covers the action's state-tree writes and its
+measured share of the values it holds (developer guide §7), and it bounds each native call's
+allocations: state_exec's per-step allocation limit takes what is left of `max_memory`
+(developer guide §11, "Allocation limit"). A finished action's process is reaped from the app
+scheduler when the drain sweeps it, and what it left in reference cycles (a function it defined,
+say) is freed by the cycle collector.
 
 A quick action completes within that drain. An action that `await`s, `sleep`s, or
 `msg-recv`s **parks** and resumes on a later frame — this is what makes the async arc

docs/STATE_EXEC_DEVELOPER_GUIDE.md

@@ -819,13 +819,70 @@ Set via `execute_options` at submission time or adjusted at runtime:
 execute_options opts;
 opts.max_steps         = 10000;     // evaluator steps
 opts.max_time          = 5.0;       // wall time in seconds
-opts.max_memory        = 1048576;   // bytes (tracked by memory_tracker)
+opts.max_memory        = 1048576;   // bytes: state-tree writes + the measured heap share
 opts.max_messages      = 100;       // outbound msg-send calls
 opts.max_message_bytes = 65536;     // total outbound payload bytes
 ```
 
 When a limit is exceeded, the process is killed (status becomes `killed`).
 
+### Memory: state-tree writes and the measured heap
+
+`max_memory` covers two things:
+- what the process wrote to the state tree, counted by the scheduler's `memory_tracker`;
+- its share of the values it holds in memory (lists, dicts, strings, closures, environments),
+  measured by walking what it can reach (`heap.h`).
+
+A value can be shared between processes. A broadcast message is one dict held by every
+receiver, and a forked child shares its parent's lists. So a process pays its **share** of each
+node it reaches: the node's size × (its references ÷ the node's owner count), and a node passes
+its share on to the nodes it holds. Summed over all holders, a node's shares come to its size: a
+1 MB message held by 50 residents costs each about 20 KB, and as holders drop it, those left pay
+more at their next measurement.
+
+A measurement is the process's own deep-data work, run a slice per step like other data work
+(`data.step_nodes`) and counted against its steps and time. One starts after a step when any of
+these holds:
+
+- natives have reported allocating 64 KiB, or half the last measurement if that is more, since
+  the last one (the functions with an allocation check, §11, plus copies like `cons` and
+  `slice`);
+- what they reported could cross the remaining budget;
+- `limits.heap_check_steps` steps have passed (default 4096; `0` = only the two above).
+
+The kill decision always uses an actual measurement, never the reported bytes, so temporaries a
+program drops don't count. Only a process with `max_memory > 0` is measured.
+`process_info::heap_memory` and `heap_measurements` report the last measurement. The per-step
+allocation limit (§11) takes what is left after state-tree writes and the measured heap.
+
+### Reference cycles
+
+Values are reference counted, and reference counting can't free a cycle. These all leaked their
+whole scope, with everything in it, before the collector existed:
+- a closure bound in the scope it captured: every `defun`, `(set f (lambda …))`, a `let`-local
+  recursive lambda;
+- a class, whose methods capture the scope the class is bound in;
+- a container that holds itself, as after `(append l l)`.
+
+`collect_cycles()` (`heap.h`) finds the cycles nothing outside them holds, the way CPython does:
+- It tracks the environments that closures, generators and class methods capture, and the
+  containers changed in place to hold another container (`append`, `set-nth`, `set-attr`). Only
+  those can close a cycle.
+- It counts the references these objects (and everything they reach) make to each other. Anything
+  with more owners than that is held from outside: by a live process, a host (pycvc's `Exec`, an
+  Ariadne action context), or anything else. Such an object stays, along with everything it
+  reaches.
+- What is left is garbage. Clearing it lets it free.
+
+Both schedulers collect between steps once a thread has tracked as many new objects as survived
+the last collection, and at least an eighth as many as it walked (at least 256). That keeps a
+collection's cost proportional to what was tracked since the last one. They also collect at most
+a second after a process ended. A finished
+process also releases its evaluation state right away (`process::release_state`; result, error
+and statistics stay), so what it leaked can be collected before the process is reaped. Tracking
+is per thread. A host that runs evaluators without a scheduler calls `collect_cycles()` itself,
+between steps; inside a step it does nothing.
+
 ### Cluster-Level Resource Policy
 
 A `resource_policy` defines cluster-wide constraints that apply to all
@@ -890,6 +947,7 @@ sched.load_settings();         // read and apply settings
 |---------|-------------------|-------------|---------|
 | `max_pending_messages` | `state_exec.schedulers.<id>.max_pending_messages` | `state_exec.defaults.max_pending_messages` | `1024` |
 | `limits.max_alloc_bytes` | `state_exec.schedulers.<id>.limits.max_alloc_bytes` | `state_exec.defaults.limits.max_alloc_bytes` | `16777216` ([§11](#allocation-limit)) |
+| `limits.heap_check_steps` | `state_exec.schedulers.<id>.limits.heap_check_steps` | `state_exec.defaults.limits.heap_check_steps` | `4096` ([heap](#memory-state-tree-writes-and-the-measured-heap)) |
 
 ```cpp
 // Set a global default for all schedulers
@@ -910,6 +968,7 @@ configuration back to the state tree:
 
 - `state_exec.schedulers.<id>.max_pending_messages` — the resolved cap
 - `state_exec.schedulers.<id>.limits.max_alloc_bytes` — the resolved per-call allocation limit
+- `state_exec.schedulers.<id>.limits.heap_check_steps` — the resolved heap re-measure interval
 - `state_exec.schedulers.<id>.policy` — `"round_robin"`, `"priority"`, or `"priority_rr"`
 
 ---
@@ -1341,8 +1400,8 @@ mine.import_module("geo", env, {"geo.area"});         // or a subset
 ### Allocation limit
 
 A native call builds its whole result inside one evaluator step. The per-process
-`max_memory` check runs between steps and counts only state-tree writes, so it cannot see
-`(collections.range 1000000000)`. Each function whose result can be larger than all of its
+`max_memory` check runs between steps, and the heap it counts is measured after the fact (§7), so
+neither can stop `(collections.range 1000000000)` from allocating. Each function whose result can be larger than all of its
 arguments therefore estimates the result's size and fails before allocating when it is over
 the limit:
 
@@ -1353,7 +1412,8 @@ the limit:
 The limit for a step is the tighter of two values:
 
 1. the tree setting `limits.max_alloc_bytes` (table below; `0` means no limit of its own);
-2. what is left of the process's `max_memory`, when it has one.
+2. what is left of the process's `max_memory`, when it has one: after its state-tree writes and
+   its last measured heap share (§7).
 
 | Setting | Per-scheduler path | Global path | Default |
 |---------|-------------------|-------------|---------|

inc/cvc/state/state_exec/async_scheduler.h

@@ -215,6 +215,7 @@ class async_scheduler : public scheduler_base {
   uint64_t total_steps_ = 0;
 
   int current_pid_ = -1;
+  uint32_t cycle_check_ = 0; // steps, for checking every 64th whether a cycle collection is due
   process_ptr current_proc_;
 
   // The scheduled evaluator IS the async_stackless_evaluator (roadmap §8.12): a coroutine
@@ -246,6 +247,9 @@ class async_scheduler : public scheduler_base {
   void check_limits(process &proc);
   void terminate_process(process &proc, value_t result);
   void kill_process(process &proc, const std::string &reason);
+  /// After a process ended: note it for the cycle collector and, unless it is the one stepping
+  /// now, release its evaluation state (process::release_state).
+  void release_if_idle(process &proc);
   process_info make_info(const process &proc) const;
 };
 

inc/cvc/state/state_exec/heap.h

@@ -0,0 +1,110 @@
+/**
+ * @file heap.h
+ * @brief A process's live DSL heap: its share for max_memory, and collecting reference cycles.
+ *
+ * Values are reference counted and may be shared between processes: a broadcast message is one
+ * dict held by every receiver, and a forked child shares its parent's lists. A process is charged
+ * its share of each node it can reach, size x (its references / the node's owner count), and a
+ * node passes its share on to the nodes under it. A node's shares over all its holders add up to
+ * its size, so a value held by k processes costs each of them about 1/k of it.
+ *
+ * Reference counting cannot free a cycle: a closure bound in the environment it captured (every
+ * defun), or a list appended to itself. The collector here finds the cycles nothing outside them
+ * holds, as CPython's does: it counts the references the tracked objects make to each other, and
+ * whatever has more owners than that is held from outside, as is everything it reaches. The rest
+ * is garbage, and clearing it lets it free.
+ */
+#ifndef CVC_STATE_EXEC_HEAP_H
+#define CVC_STATE_EXEC_HEAP_H
+
+#include <cstddef>
+#include <cstdint>
+#include <cvc/state/state_exec/types.h>
+#include <memory>
+#include <string>
+
+namespace cvc {
+class state;
+}
+
+namespace cvc::state_exec {
+
+class data_job;
+struct process;
+
+// ---------------------------------------------------------------------------
+// Measuring a process's heap
+// ---------------------------------------------------------------------------
+
+/// Heap settings. Hosts keep them on the state tree under `limits.*`, like
+/// limits.max_alloc_bytes.
+struct heap_options {
+  /// Re-measure a process that has a memory budget at least every this many steps, besides the
+  /// measurements its allocations call for (0 = only those). State-tree key:
+  /// `limits.heap_check_steps`.
+  std::size_t check_steps = 4096;
+};
+
+/// Resolve heap settings from a state tree, per key: `state_exec.schedulers.<id>.limits.<key>`,
+/// then `state_exec.defaults.limits.<key>`, then `base`. A missing or malformed value is skipped.
+heap_options heap_options_from_state(cvc::state &root, const std::string &scheduler_id = {},
+                                     const heap_options &base = {});
+
+/// A data job that measures `proc`'s heap share; its result is the share in bytes, as an int.
+/// It holds what it measures, so the process (and others) may run between its slices.
+std::unique_ptr<data_job> make_heap_share_job(const process &proc);
+
+/// `proc`'s heap share in bytes, measured now in one go.
+std::size_t measure_heap_share(const process &proc);
+
+/// The bytes a re-measure waits for: natives' reported allocations since the last measurement
+/// trigger one once they reach this (64 KiB, or half the last measurement if that is more).
+std::size_t heap_remeasure_bytes(std::size_t last_measured);
+
+/// The heap accounting a scheduler does after each step of `proc`: take a measurement that just
+/// finished (proc.heap_bytes), add `allocated` (the step guard's eval_alloc_guard::allocated()),
+/// and start the next measurement as the process's data work when one is due
+/// (process::heap_measure_due). `state_bytes` is the memory tracker's count for the process.
+/// Nothing is measured for a process without a memory budget.
+void heap_after_step(process &proc, std::size_t allocated, uint64_t state_bytes);
+
+// ---------------------------------------------------------------------------
+// Reference cycles
+// ---------------------------------------------------------------------------
+
+/// Track an environment that a closure, a generator or a class method captured: it may close a
+/// cycle (the closure bound in it, or in something it holds). Tracking is per thread.
+void track_captured_env(const environment_ptr &env);
+
+/// Track a list or dict that was changed in place (append, set-nth, set-attr): only a change in
+/// place can make a container hold itself.
+void track_mutated(const list_ptr &list);
+void track_mutated(const dict_ptr &dict);
+
+/// What a collection did.
+struct cycle_stats {
+  std::size_t candidates = 0; ///< tracked objects still alive when it started
+  std::size_t examined = 0;   ///< objects reachable from them
+  std::size_t freed = 0;      ///< objects found to be garbage and cleared
+};
+
+/// Find the reference cycles among this thread's tracked objects that nothing outside them holds,
+/// and clear them so they free. Does nothing (and returns zeros) while an evaluator step is
+/// running on this thread: call it between steps. The schedulers call it when one is due.
+cycle_stats collect_cycles();
+
+/// Whether this thread has tracked enough since its last collection for one to be due: as many
+/// new objects as survived the last one, at least an eighth of the objects it walked (so a walk
+/// costs a bounded amount per newly tracked object), and at least 256.
+bool cycle_collection_due();
+
+/// Note that a process ended on this thread: what it leaked is garbage now, so a collection is
+/// due as soon as anything has been tracked since the last one (at most once a second).
+void note_process_exit();
+
+/// Objects this thread tracks: those alive after its last collection plus those tracked since.
+std::size_t tracked_objects();
+
+} // namespace cvc::state_exec
+
+#endif // CVC_STATE_EXEC_HEAP_H

inc/cvc/state/state_exec/process.h

@@ -73,6 +73,17 @@ struct process {
   // limits.max_alloc_bytes, resolved when the process is submitted. See alloc_limit().
   uint64_t max_alloc_bytes = kDefaultMaxAllocBytes;
 
+  // Heap accounting (heap.h), for a process with max_memory > 0: max_memory covers its state-tree
+  // writes plus heap_bytes, its measured share of the values it can reach. A measurement runs as
+  // the process's own data steps; heap_alloc_bytes (what natives reported allocating) and
+  // heap_steps since the last one decide when the next is due.
+  uint64_t heap_bytes = 0;
+  uint64_t heap_alloc_bytes = 0;
+  uint64_t heap_steps = 0;
+  uint64_t heap_measurements = 0;
+  std::size_t heap_check_steps = 4096; // limits.heap_check_steps, resolved at submission
+  bool heap_measuring = false;         // a measurement is in proc.state.data_work
+
   // Evaluator state (stackless — serializable)
   evaluator_state state;
 
@@ -147,11 +158,23 @@ struct process {
 
   /// The allocation limit for the next step's eval_alloc_guard (nullopt when unlimited): the
   /// tighter of max_alloc_bytes and what is left of max_memory once `used_bytes` (the memory
-  /// tracker's count for this process) is taken out.
+  /// tracker's count for this process plus heap_bytes) is taken out.
   std::optional<std::size_t> alloc_limit(std::size_t used_bytes) const;
 
+  /// Whether to start a heap measurement after this step: the process has a memory budget, none
+  /// is running, and natives reported enough allocation (heap_remeasure_bytes), or the next
+  /// allocation could cross the budget, or limits.heap_check_steps steps passed.
+  /// `state_bytes` is the memory tracker's count for this process.
+  bool heap_measure_due(uint64_t state_bytes) const;
+
   /// Step count from evaluator stats.
   uint64_t step_count() const { return state.stats.get_step_count(); }
+
+  /// Drop what a finished process evaluated with (its stack, environment, program, handlers and
+  /// inbox), keeping its result, error and statistics. A scheduler calls it once the process is
+  /// terminated or killed and not running, so what the program left in reference cycles becomes
+  /// garbage the cycle collector can free (heap.h), even before the process is reaped.
+  void release_state();
 };
 
 /// Shared ownership alias for process.

inc/cvc/state/state_exec/scheduler.h

@@ -11,6 +11,7 @@
 #define CVC_STATE_EXEC_SCHEDULER_H
 
 #include <chrono>
+#include <cvc/state/state_exec/heap.h>
 #include <cvc/state/state_exec/memory_tracker.h>
 #include <cvc/state/state_exec/process.h>
 #include <cvc/state/state_exec/scheduler_base.h>
@@ -210,6 +211,8 @@ class scheduler : public scheduler_base {
   scheduling_policy policy_;
   parse_options parse_options_;
   data_options data_options_;
+  heap_options heap_options_;
+  uint32_t cycle_check_ = 0; // steps, for checking every 64th whether a cycle collection is due
   int next_pid_ = 1;
   int rr_index_ = 0;
   bool running_ = false;
@@ -257,6 +260,10 @@ class scheduler : public scheduler_base {
   /// Mark a process as killed with an error.
   void kill_process(process &proc, const std::string &reason);
 
+  /// After a process ended: note it for the cycle collector and, unless it is the one stepping
+  /// now, release its evaluation state (process::release_state).
+  void release_if_idle(process &proc);
+
   /// Build process_info from a process.
   process_info make_info(const process &proc) const;
 };

inc/cvc/state/state_exec/scheduler_base.h

@@ -86,6 +86,8 @@ struct process_info {
   double elapsed_time;
   uint64_t current_memory;
   uint64_t peak_memory;
+  uint64_t heap_memory = 0;       // measured heap share (heap.h); max_memory covers it too
+  uint64_t heap_measurements = 0; // measurements taken so far
   uint64_t max_memory;
   double max_time;
   uint64_t message_count;