state_exec: native calls past the first level are the evaluator's

docs/STATE_EXEC_DEVELOPER_GUIDE.md

@@ -1479,10 +1479,12 @@ read. The document is held to the allocation limit as it grows.
   the work runs whole in the call's step.
 - **Calls.** `collections.map`, `filter` and `reduce` hand each call of their function to the
   evaluator running the program, and so do the builtins `apply` and `send`. A native function
-  is called inside the data step. A DSL function runs as the process's own frames, so its
-  body's steps are eval steps. The job goes on in the data step after it returns, even with
-  `data.step_nodes` = 0. Another native function can do the same with
-  `call_function(fn, args, who)`.
+  is called right away when the native function the step applies asks for it, and by the
+  evaluator, in a data step, when a native function deeper than that asks (so native calls nest
+  at most one level; `docs/STATE_EXEC_GRAMMAR.md`, "Native call nesting"). A DSL function runs
+  as the process's own frames, so its body's steps are eval steps. The job goes on in the data
+  step after it returns, even with `data.step_nodes` = 0. Another native function can do the
+  same with `call_function(fn, args, who)`.
 - **Cost.** `collections.sort` is a stable merge sort, n log n nodes.
   `collections.unique` costs a node per comparison, and compares lists and dicts with a sliced
   equality walk. `collections.dict-merge` keeps a key index, so it is linear. Every job polls the

docs/STATE_EXEC_GRAMMAR.md

@@ -242,10 +242,12 @@ A node is an element visited: a list element or a dict entry walked, or an eleme
 - **Signals and watch events** wait while a walk is pending, because the walk's call frame is
   on top of the evaluation stack.
 - **Function calls.** A job can stop for a call of a function (`collections.map`, `filter`,
-  `reduce`, and `apply`/`send` through `call_function()`), and the evaluator makes it. A native function is called inside the data step. A
-  DSL function runs as frames above the waiting call, in eval steps; `return` and `break` in it
-  unwind no further than that call. The job goes on in the data step after the function
-  returns. Such a job waits on the evaluator even with `step_nodes` = 0.
+  `reduce`, and `apply`/`send` through `call_function()`), and the evaluator makes it. A native
+  function is called inside the step: by the native call the step applies, if the job is its own,
+  and otherwise by the evaluator in a data step (see "Native call nesting"). A DSL function runs
+  as frames above the waiting call, in eval steps; `return` and `break` in it unwind no further
+  than that call. The job goes on in the data step after the function returns. Such a job waits
+  on the evaluator even with `step_nodes` = 0.
 - **Snapshots** keep `step_nodes` but not a walk in flight: after a restore, the waiting call
   or macro expansion runs again. The frames of a DSL function it was calling are left out.
 - **Values changed between slices.** A walk holds the lists and dicts it walks by owning
@@ -270,14 +272,46 @@ decode.
 ### Native call nesting
 
 Some native functions call other functions themselves: `apply`, `send`, a class constructor
-running `init`, an inherited `init` running its superclass's. A generator advancing steps an
-evaluator of its own. Each of these nests on the native stack, so a value or class that refers to
-itself could ask for unbounded nesting: `(apply apply L)` with `L` containing itself, or a class
-whose `init` is its own constructor. A level is refused with an error ("native calls nested too
-deeply") past 256 levels (`kMaxNativeNesting`), or when less of the thread's native stack is left
-than `kNativeStackReserve`: 128 KiB on Windows, where a catch block runs before the frames above
-it unwind, and 32 KiB elsewhere. Levels cost from about 0.5 KiB (`apply`) to a few KiB (a
-generator) in a release build.
+running `init`, an inherited `init` running its superclass's, and the stdlib jobs
+(`collections.map` and friends). Inside a running program they nest at most one level on the
+native stack. The native function the evaluator applies may call a native function itself
+(`apply` calling `+`), but a native call asked for by that function, or from anywhere deeper, is
+the evaluator's: it makes the call at its own level, in a data step (`call_function`,
+`run_data_job`).
+
+- **Runaways are loops.** `(apply apply L)` with `L` containing itself, `send` re-sending through
+  itself, or a class whose `init` is its own constructor is a sequence of steps. `max_steps` and
+  `max_time` stop it as they stop `(while t nil)`, at the same step on every platform and stack
+  size.
+- **Long chains finish.** `apply` applied 10000 levels deep, or a chain of 10000 classes that
+  inherit `init`, runs on any stack.
+- **Inherited `init` and `finalize`** look up the chain of classes that inherit them too, 64
+  classes a call (`kInheritedHops`), to the first class that defines one, and call that. A
+  constructor does the same, and with no class defining `init` it calls nothing. A longer chain,
+  or one that loops, goes on as calls.
+- **Space.** A call whose value is its caller's, unchanged (`apply`, `send`, an inherited
+  `init`), hands its place to the call it makes, so a chain of them runs in constant space. A
+  call with more to do after its callee (a constructor returns the instance once `init` is done)
+  waits, a link of pending work per level, like a function that calls itself before doing
+  anything else. Pending work is not yet part of a process's measured heap (`max_memory`).
+- **Cost.** A call the evaluator makes costs about 300 ns more than one made directly (MSVC
+  Release, `state_exec_bench_test` `NativeCallThroughput`). The one direct level keeps `apply`
+  and `send` at their old cost, and only a native call two levels down pays it. Constructing a
+  class that inherits `init` costs less than it did, since the chain is looked up, not called.
+
+What still nests is a **level**:
+
+- the one call above, made by the step's own native call;
+- a generator advancing, which steps an evaluator of its own (generators of generators nest a
+  level apiece);
+- a native function calling another from C++ where no program is running: a host calling a
+  native function directly, or a job run to completion (`run_to_completion`).
+
+A level is refused with an error ("native calls nested too deeply") past 256 levels
+(`kMaxNativeNesting`), or when less of the thread's native stack is left than
+`kNativeStackReserve`: 128 KiB on Windows, where a catch block runs before the frames above it
+unwind, and 32 KiB elsewhere. Levels cost from about 0.5 KiB (`apply`) to a few KiB (a generator)
+in a release build.
 
 DSL functions are not levels, and neither are class methods. A method is a closure in its class's
 dict, taking `self` first. `send`, `super` and a constructor's `init` call it through the

inc/cvc/state/state_exec/data_job.h

@@ -73,6 +73,11 @@ class data_job {
   /// from that call's arguments.
   virtual void own_inputs() {}
 
+  /// True for a job whose result is the value of the one call it makes, unchanged (a call
+  /// call_function() made a job). When that call leaves work of its own, the evaluator lets the
+  /// work take the job's place, so a chain of such calls runs in constant space.
+  virtual bool forwards_call() const { return false; }
+
 protected:
   std::size_t nodes_ = 0;
   std::optional<call_request> call_;
@@ -132,15 +137,19 @@ data_options data_options_from_state(cvc::state &root, const std::string &schedu
 /// evaluator: it runs a slice per step, makes the job's calls (a DSL function as ordinary
 /// frames of the same process) and completes the call with then's value once the job is done;
 /// what this returns is then ignored, so the native function should return it at once.
-/// Anywhere else the job runs to completion here (see run_to_completion). Without `then`, the
-/// job's result is the call's.
+/// A native function the job calls is called here only from the step's own native call
+/// (data_step_scope::calls_here); deeper, the evaluator calls it. Anywhere else the job runs to
+/// completion here (see run_to_completion). Without `then`, the job's result is the call's.
 value_t run_data_job(std::unique_ptr<data_job> job, std::function<value_t(value_t)> then = {});
 
 /// Apply `fn`, a native function or a DSL closure, to `args` from inside a native function.
-/// A native function is called here. A closure runs through the evaluator, as a one-call job
-/// (run_data_job): inside a stackless evaluator step its body becomes the process's own frames,
-/// and the native function should return what this returns at once; anywhere else a closure
-/// throws. `who` names the caller in errors.
+/// Inside a stackless evaluator step the call is the evaluator's, as a one-call job
+/// (run_data_job): a closure's body becomes the process's own frames, and a native function is
+/// called by the evaluator at its own level, in a data step, unless this is the step's own native
+/// call, which calls it here (data_step_scope::calls_here). The native function should return
+/// what this returns at once. Anywhere else a native function is called here, one level of
+/// native nesting (detail::native_nesting), and a closure throws. `who` names the caller in
+/// errors.
 value_t call_function(const value_t &fn, std::span<const value_t> args, const char *who);
 
 /// As call_function, with `then` applied to the call's value: for a native function that has
@@ -150,16 +159,18 @@ value_t call_function(const value_t &fn, std::span<const value_t> args, const ch
 value_t call_function(const value_t &fn, std::span<const value_t> args, const char *who,
                       std::function<value_t(value_t)> then);
 
-/// How native calls may nest on one thread's native stack. A level is a native function calling
-/// another function itself (apply, send, a class constructor running init, an inherited init
-/// running its superclass's, a job's call made where the job runs) or stepping a generator's own
-/// evaluator. The evaluator applying a native function is not a level, and neither is a DSL
-/// function or class method: those run as evaluator frames, on the heap. A self-containing
-/// argument list, a class whose init is its own constructor, or a long enough chain of such calls
-/// asks for unbounded nesting, so a level is refused (it throws) past kMaxNativeNesting levels, or
-/// when less than kNativeStackReserve bytes of the thread's native stack are left (where the
-/// platform says where its stack ends). Levels cost from about 0.5 KiB (apply) to a few KiB (a
-/// generator's evaluator) in a release build.
+/// How native calls may nest on one thread's native stack. Inside an evaluator step they barely
+/// do: when a native function (apply, send, a class constructor running init, an inherited init
+/// running its superclass's, a job) calls a native function, the call is made here only from the
+/// native function the step applies, and any call deeper than that is the evaluator's, made in a
+/// data step (data_step_scope::calls_here). So a self-containing argument list, a class whose init
+/// is its own constructor, or a long chain of inits is a sequence of steps, stopped by max_steps
+/// and max_time like a loop. What still nests is a level: that one call, a generator stepping its
+/// own evaluator, and a native function calling another from C++ with no evaluator step running
+/// (or one that cannot wait for it, see run_to_completion). A level is refused (it throws) past
+/// kMaxNativeNesting levels, or when less than kNativeStackReserve bytes of the thread's native
+/// stack are left (where the platform says where its stack ends). Levels cost from about 0.5 KiB
+/// (apply) to a few KiB (a generator's evaluator) in a release build.
 constexpr std::size_t kMaxNativeNesting = 256;
 
 /// The native stack a level leaves for the error it may throw. MSVC runs a catch block before
@@ -232,6 +243,12 @@ class data_step_scope {
   /// The job a native call deferred, if any.
   std::unique_ptr<deferred_data_job> take_deferred() { return std::move(deferred_); }
 
+  /// Whether a native function that the native call being applied wants called (call_function,
+  /// a job's call) is called right here, on the native stack: only from that native call itself,
+  /// one level down, and while the slice has room. A call from deeper goes to the evaluator,
+  /// which makes it at its own level, so native calls nest at most one level in a step.
+  bool calls_here() const;
+
   /// Free backlog owners with what is left of the slice (all of them when not slicing).
   void drain_backlog();
 
@@ -241,11 +258,14 @@ class data_step_scope {
 private:
   friend value_t cvc::state_exec::run_data_job(std::unique_ptr<data_job>,
                                                std::function<value_t(value_t)>);
+  friend value_t cvc::state_exec::call_function(const value_t &, std::span<const value_t>,
+                                                const char *);
   friend void cvc::state_exec::count_data_nodes(std::size_t);
 
   std::size_t step_nodes_;
   evaluation_stats &stats_;
   release_backlog *backlog_;
+  std::size_t native_depth_; // native_nesting::depth() when the step began
   std::size_t nodes_ = 0;
   bool may_defer_ = false;
   std::unique_ptr<deferred_data_job> deferred_;

inc/cvc/state/state_exec/stackless_evaluator.h

@@ -90,6 +90,11 @@ struct pending_data_work {
   std::shared_ptr<pending_data_work> outer;
   /// The result is the value of a call outer's job made, not the waiting frame's.
   bool feeds_outer = false;
+
+  /// Frees the chain of outer work a link at a time, not one native frame per link: a native
+  /// call that waits on one like it, with more to do after it (a class constructor whose init
+  /// constructs again), makes a chain as long as the process's steps allow.
+  ~pending_data_work();
 };
 
 /// Serializable evaluator state — the entire execution snapshot.

src/cvc/state/state_exec/builtins.cpp

@@ -129,7 +129,8 @@ value_t dispatch_method(const value_t &obj, const std::string &method,
     if (value_t *m = dict_find(*cls_dict, method)) {
       if (!std::holds_alternative<native_fn>(m->v) && !std::holds_alternative<closure_ptr>(m->v))
         throw std::runtime_error("send: method is not callable");
-      // Invoke: first arg is self. A closure runs as the program's own frames.
+      // Invoke: first arg is self. A closure runs as the program's own frames, and a native
+      // method nests no deeper than one level (call_function).
       std::vector<value_t> call_args;
       call_args.reserve(1 + args.size());
       call_args.push_back(obj);
@@ -628,7 +629,8 @@ value_t builtin_send(std::span<const value_t> args) {
 value_t builtin_apply(std::span<const value_t> args) {
   expect_exact(args, 2, "apply");
   auto &arg_list = as_list(args[1], "apply");
-  // A native function is called here; a closure runs as the program's own frames.
+  // A closure runs as the program's own frames; a native function is called here, unless this
+  // apply was itself called by a native function, when the evaluator calls it (call_function).
   return call_function(args[0], arg_list, "apply");
 }
 

src/cvc/state/state_exec/closure_natives.h

@@ -30,9 +30,18 @@ struct class_inherited_method {
   dict_ptr super_cls; // may be null: no superclass
   std::string name;   // "init" or "finalize"
   value_t operator()(std::span<const value_t> args) const;
+  /// The method a call of this one comes to: up the chain of classes that inherit it too, the
+  /// first class's own `name` method, or nullptr when no class defines one (the default). A
+  /// chain longer than kInheritedHops classes (or one that loops) stops at the inherited method
+  /// it reached, whose call goes on from there.
+  const value_t *resolve() const;
   ~class_inherited_method() { drop_owner(super_cls); }
 };
 
+/// How many classes up one call of an inherited method looks itself (class_inherited_method::
+/// resolve) before it leaves the rest of the chain to a call.
+constexpr int kInheritedHops = 64;
+
 } // namespace cvc::state_exec::detail
 
 #endif // CVC_STATE_EXEC_CLOSURE_NATIVES_H

src/cvc/state/state_exec/data_job.cpp

@@ -503,6 +503,8 @@ class call_walk final : public data_job {
 
   value_t result() override { return std::move(result_); }
 
+  bool forwards_call() const override { return true; }
+
 private:
   value_t fn_;
   std::vector<value_t> args_;
@@ -733,8 +735,9 @@ value_t run_data_job(std::unique_ptr<data_job> job, std::function<value_t(value_
     return then ? then(std::move(r)) : r;
   }
   // One slice here (without slicing, all of it), making the native calls the job needs on the
-  // way. It waits for the evaluator when the slice runs out, for a DSL function, and for the work
-  // a native function it called left of its own.
+  // way if this is the step's own native call. It waits for the evaluator when the slice runs out,
+  // for a DSL function, for a native one called from deeper, and for the work a native function it
+  // called left of its own.
   const bool slicing = scope->step_nodes_ > 0;
   std::unique_ptr<detail::deferred_data_job> waits_on;
   for (;;) {
@@ -747,7 +750,7 @@ value_t run_data_job(std::unique_ptr<data_job> job, std::function<value_t(value_
     }
     const data_job::call_request *pending = job->pending_call();
     const native_fn *fn = pending ? std::get_if<native_fn>(&pending->fn->v) : nullptr;
-    if (!fn || (slicing && scope->nodes_ >= scope->step_nodes_))
+    if (!fn || !scope->calls_here())
       break;
     const data_job::call_request call = *job->take_call();
     value_t value;
@@ -769,13 +772,17 @@ value_t run_data_job(std::unique_ptr<data_job> job, std::function<value_t(value_
 
 value_t call_function(const value_t &fn, std::span<const value_t> args, const char *who) {
   if (const auto *native = std::get_if<native_fn>(&fn.v)) {
-    detail::native_nesting level(who);
-    return (*native)(args);
+    // Here, unless an evaluator step can make the call at its own level instead.
+    const detail::data_step_scope *scope = t_scope;
+    if (!scope || !scope->may_defer_ || scope->deferred_ || scope->calls_here()) {
+      detail::native_nesting level(who);
+      return (*native)(args);
+    }
+  } else if (const auto *cls = std::get_if<closure_ptr>(&fn.v); !cls || !*cls) {
+    throw std::runtime_error(std::string(who) + ": not a function: " + fn.type_name());
   }
-  if (const auto *cls = std::get_if<closure_ptr>(&fn.v); cls && *cls)
-    return run_data_job(
-        std::make_unique<call_walk>(fn, std::vector<value_t>(args.begin(), args.end()), who));
-  throw std::runtime_error(std::string(who) + ": not a function: " + fn.type_name());
+  return run_data_job(
+      std::make_unique<call_walk>(fn, std::vector<value_t>(args.begin(), args.end()), who));
 }
 
 value_t call_function(const value_t &fn, std::span<const value_t> args, const char *who,
@@ -816,7 +823,7 @@ std::size_t detail::native_nesting::depth() { return t_native_depth; }
 detail::data_step_scope::data_step_scope(std::size_t step_nodes, evaluation_stats &stats,
                                          release_backlog *backlog)
     : step_nodes_(step_nodes), stats_(stats), backlog_(step_nodes > 0 ? backlog : nullptr),
-      outer_(t_scope), routing_(backlog_) {
+      native_depth_(t_native_depth), outer_(t_scope), routing_(backlog_) {
   t_scope = this;
 }
 
@@ -826,6 +833,12 @@ std::size_t detail::data_step_scope::remaining() const {
   return step_nodes_ > nodes_ ? step_nodes_ - nodes_ : 1;
 }
 
+bool detail::data_step_scope::calls_here() const {
+  // The evaluator applies a native function without a level of native nesting, so the depth is
+  // the step's own only inside that function.
+  return t_native_depth == native_depth_ && (step_nodes_ == 0 || nodes_ < step_nodes_);
+}
+
 void detail::data_step_scope::drain_backlog() {
   if (!backlog_ || backlog_->empty() || nodes_ >= step_nodes_)
     return;