Skip to content

Apply max_call_levels to native functions calling back into the engine - #1208

Merged
schungx merged 3 commits into
rhaiscript:mainfrom
Rockyrok77:native-call-level-limit
Oct 30, 2026
Merged

schungx merged 3 commits into
rhaiscript:mainfrom
Rockyrok77:native-call-level-limit

Conversation

@Rockyrok77

@Rockyrok77 Rockyrok77 commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Comparing a deeply nested array overflows the Rust stack and aborts the host, even with the sandbox limits set:

// Engine with max_operations(1_000_000), max_array_size(10_000)
let a = [];
for i in 0..9000 { a = [a]; }
a == a      // thread 'main' has overflowed its stack; fatal runtime error: stack overflow, aborting

NativeCallContext::_call_fn_raw bumps global.level on every call from a native function back into the engine, but only call_script_fn ever compares the level with max_call_levels. Array == calls == on each element through the context, so it recurses once per nesting level with nothing to stop it. The same goes for map ==, contains/index_of, and format_array/format_map behind print/to_string/debug. Each level costs about 1 KiB of stack in release, so a 9000-deep array (fine under max_array_size(10_000)) is enough to blow through an 8 MiB main thread, and on a 2 MiB spawned thread it takes about 2000. A stack overflow aborts the process, so no limit or catch_unwind can stop it.

The fix does the max_call_levels check in _call_fn_raw and returns ErrorStackOverflow, the same check script calls get. Printing still works because print_with_func already falls back to Display when the nested call fails. max_call_levels and set_max_call_levels are no longer gated on no_function, since the same recursion happens there too, so the check applies in that build as well. A regression test is in tests/stack.rs. Without the fix it aborts the test binary.

@schungx

schungx commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator

In that case, should max_call_levels be available also for no_function builds? Which means make it public for features. Since, as you pointed out, the problem also exist for native Rust calls...

Before, I was under the (wrong) assumption that, since we control all Rust functions, we definitely can avoid any dangerous calls ourselves. Now you've pointed out the fact that certain built-in Rhai functions, such as ==, will implicitly call recursively that may blow up the stack.

@schungx schungx added the bug label Oct 29, 2026
@Rockyrok77

Copy link
Copy Markdown
Contributor Author

Yes, I think it should. You don't need script functions to hit this, just a nested array and ==, so a no_function build overflows the same way. I pushed a commit that drops the no_function gate from set_max_call_levels and max_call_levels (plus the call_stack_depth limit and MAX_CALL_STACK_DEPTH), so the check in _call_fn_raw now runs in that build too, and the regression test isn't gated on no_function anymore. The full test suite passes with and without no_function.

Worth keeping in mind that the debug default for max_call_levels is 8, so in a debug build, comparing arrays nested more than 8 deep now needs a higher limit. I left the defaults alone.

@schungx
schungx merged commit 0e4eaa7 into rhaiscript:main Oct 30, 2026
55 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants