perf(gc): make the shadow-stack runtime ops cheap

[proggeramlug]

A ceiling measurement put the shadow stack at +71.5 % instructions on
application-shaped code (w6_records, Pi 5) versus not emitting it at all —
roughly 65–70 instructions per shadow op — and named two implementation costs
rather than anything inherent to shadow stacks. This PR removes both, without
changing the root-set contract.

What changed

1. js_shadow_frame_push did three Vec::resize calls per activation

The per-slot state lived in three parallel thread-local Vecs — stack: Vec<u64> (mirrored values), slot_ptrs: Vec<usize> (bindings to compiled
locals), active: Vec<bool> (liveness bits) — and the frame header took two
u64 words. One frame push therefore cost five capacity checks, three length
updates, and, read out of the shipped archive's disassembly, up to three
memset calls for the 2–4-slot frames codegen actually emits.

The three words are now one 16-byte ShadowEntry { value: u64, meta: usize },
the header packs prev_frame_top and slot_count into a single entry
(SHADOW_STACK_HEADER_SLOTS 2 → 1), and the slot clear is a constant-size
store.

aarch64 release disassembly, fast path, libperry_runtime.a:

	before	after
js_shadow_frame_push	1 TLS + 5 capacity checks + up to 3 bl memset	1 TLS + 1 capacity check + movi + 2 × stp q0, q0, no call
js_shadow_slot_set	3 bounds checks against 3 Vec headers, bl root barrier (which does its own TLS lookup)	1 bounds check, barrier inline-gated (ldar + cbnz), no call
js_shadow_slot_bind	same 3 checks + unconditional barrier call	1 bounds check, whole entry written with one stp, no call

Keeping the clear call-free took three attempts, each verified by
disassembling the linked archive rather than assumed:

1. a match on the slot count with a spelled-out arm per size → LLVM re-formed
   it into a compare chain computing a byte count and bl memset;
2. a constant-size 4-entry store plus a variable write_bytes for wide frames
   → LLVM tail-merged the two into one csel-the-length-then-bl memset;
3. constant-size store with the wide-frame path behind #[cold] #[inline(never)] → movi.2d v0, #0 + stp q0, q0 × 2. Shipped.

2. js_shadow_slot_set / js_shadow_slot_bind called the root write barrier on every store

It is still called — it is now gated. The barrier is the incremental-mark
root shading barrier, not the generational remembered-set barrier: old→young
edges are logged by the heap-slot barriers (runtime_write_barrier_slot and
friends), which this PR does not touch. Its whole body is
incremental_mark_barrier_value, which reads the thread-local
INCREMENTAL_MARK_BARRIER_VALID_PTRS and returns immediately when it is null —
so on the overwhelmingly common path the cost was a call plus a second TLS
lookup to learn there was nothing to do.

It is now gated inline on PERRY_INCREMENTAL_MARK_BARRIER_ACTIVE_COUNT, the
same guard codegen already emits around js_write_barrier_root_nanbox for
persistent shadow slots (expr/shadow_slot.rs::emit_persistent_shadow_root_barrier).

The gate is observationally identical, not a narrowing.
incremental_mark_barrier_enable installs the thread-local and then
increments the count, both before returning to the mutator, so on any thread
"this thread's VALID_PTRS is non-null ⟹ the count is ≥ 1". A zero count
therefore proves this thread's pointer is null, i.e. proves the call would
have returned false having done nothing. A non-zero count is conservative in
the harmless direction: another thread's cycle makes us take the call, which
then observes its own null pointer and returns.

Soundness, per property

Liveness — an active entry is still visited by
visit_shadow_stack_root_slots and still marked. The liveness bit moved from
active: Vec<bool> into bit 0 of meta; a compiled local slot is an
i64/double alloca and so 8-byte aligned, leaving bit 0 free. An address
that would collide with the tag is recorded active-but-unbound rather than
truncated — the mirrored value is still marked and still rewritten, and the
collector is never handed a mis-derived address to write a forwarded pointer
into.

Rewritability — unchanged in kind. A bound entry still exposes the compiled
local's address to the visitor, so an evacuating collection rewrites the
storage the mutator reads after the safepoint. An unbound entry exposes the
mirror word itself; ShadowEntry is #[repr(C)] with value at offset 0, so
that is the same correctly-aligned *mut u64 the parallel-Vec layout handed
out.

The value the mutator stored — js_shadow_slot_bind still snapshots
*value_slot once, at the call, and roots that word. Nothing re-reads a slot at
a safepoint.

No stale roots — the zero-fill is load-bearing, not hygiene: the buffer past
the current length still holds the popped frame's entries, so a frame that
inherited them would report dead values and, worse, a live binding into a
stack frame that has since returned. Clearing meta matters as much as clearing
value.

Also hardened while here: the malformed-frame_handle guard in
js_shadow_frame_pop was base + HEADER_SLOTS > len, which wraps for a handle
near usize::MAX and lets exactly the corrupted handles it exists for slip
through into an unchecked read. It is now base >= len.

Tests

New gc::tests::shadow_stack_ops (13 tests) covers the four properties above.
Every test was verified to fail under a targeted sabotage of the thing it
covers — see the table in the PR discussion.

Verification

See the PR discussion for the measured instruction counts, matrix, and ratchet
results.

Summary by CodeRabbit

• Performance Improvements

  • Reduced overhead for shadow-stack frame creation and slot updates.
  • Optimized incremental garbage-collection barrier checks.

• Bug Fixes

  • Improved handling of invalid frame references to prevent crashes or state corruption.
  • Prevented stale values and bindings from being reused in new frames.
  • Preserved correct pointer updates during memory evacuation.

• Tests

  • Added comprehensive coverage for shadow-stack liveness, nested frames, savepoint restoration, garbage collection, and barrier behavior.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: a30171c9-0cee-4f4d-aba3-e87fbed24267

📥 Commits

Reviewing files that changed from the base of the PR and between 9a3967b and 76a7512.

📒 Files selected for processing (7)

• changelog.d/7079-shadow-stack-runtime-ops.md
• crates/perry-runtime/src/gc/roots.rs
• crates/perry-runtime/src/gc/roots/shadow_stack.rs
• crates/perry-runtime/src/gc/tests/debt_pacer.rs
• crates/perry-runtime/src/gc/tests/mod.rs
• crates/perry-runtime/src/gc/tests/shadow_stack_ops.rs
• crates/perry-runtime/src/gc/tests/support.rs

---

📝 Walkthrough

Walkthrough

The shadow stack now uses packed entries instead of parallel vectors. Frame operations, slot binding, liveness tracking, root scanning, savepoint restoration, and barrier behavior were updated, with extensive tests covering GC forwarding, stale-state prevention, frame chaining, and malformed handles.

Changes

Shadow-stack runtime

Layer / File(s)	Summary
Packed storage and runtime operations crates/perry-runtime/src/gc/roots/shadow_stack.rs, changelog.d/*	Shadow-stack frames and slots use contiguous ShadowEntry values with packed liveness and binding metadata. Push, pop, slot access, savepoint, restore, and barrier operations use the new layout.
Packed-entry root scanning crates/perry-runtime/src/gc/roots.rs	Root visitation reads packed headers and entries, skips inactive slots, and visits bound pointers or embedded slot values.
Runtime migration and behavioral tests crates/perry-runtime/src/gc/tests/*	Existing tests and reset helpers adopt the new storage, while new tests cover liveness, forwarding, barriers, stale entries, frame growth, unwinding, and invalid handles.

Estimated code review effort: 4 (Complex) | ~60 minutes

Possibly related PRs

• PerryTS/perry#6972: Modifies the same shadow-stack root scanning and savepoint/restore infrastructure.
• PerryTS/perry#7007: Extends shadow-stack binding behavior for scalar-replaced fields and elements.
• PerryTS/perry#7013: Builds on shadow-slot binding and incremental-mark barrier semantics.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title is concise and accurately summarizes the main shadow-stack runtime optimization.
Description check	✅ Passed	It covers summary, changes, tests, and verification; only non-critical template sections like related issue and checklist are omitted.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch perf/shadow-stack-runtime-ops

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[proggeramlug]

Measurement

Host Raspberry Pi 5, Cortex-A76 @ 2.400 GHz, governor performance,
taskset -c 3, perf stat. Every run samples vcgencmd measure_clock arm and
vcgencmd get_throttled immediately before and after and is dropped if either
moved. Clock held 2 400 017 408 – 2 400 037 120 Hz (±0.001 %), throttle word
0x0 throughout, 62–68 °C. Every run's stdout was compared to the pinned Node
26.5.0 oracle.

Both arms are the same pinned perry compiler binary and the same sources,
linked against two runtime archives via PERRY_RUNTIME_DIR, interleaved
base/opt back to back within each rep. The harness asserts the two arms'
binaries actually differ — a runtime swap that silently didn't take would
otherwise read as a clean "no regression".

A/B: base vs opt (14 reps/arm, 224 runs, 0 dropped, 0 oracle mismatches)

workload	base	opt	instructions	paired spread
w3_treewalk (deep pointer recursion)	19,566,230,920	15,543,615,940	−20.56 %	[−20.57, −20.55]
w4_strchurn	23,786,031,230	21,301,521,602	−10.45 %	[−10.45, −10.45]
w1_calldepth	27,247,446,450	25,147,038,155	−7.71 %	[−7.72, −7.70]
w6_records (application-shaped)	20,234,622,398	19,310,159,870	−4.57 %	[−4.57, −4.57]
w2s_allocring_small	1,715,266,486	1,672,075,263	−2.52 %	[−2.80, −2.26]
w2_allocring	28,768,295,842	28,246,934,792	−1.81 %	[−3.80, +0.03]
w0_noop	1,939,832	1,926,284	−0.70 %	[−3.44, +1.18]
w5_purity (floor)	17,205,601,084	17,205,295,805	−0.0018 %	[−0.02, +0.01]

Instruction-count cv is 0.00 % on every workload above 100 M instructions.
w5_purity and w0_noop emit no shadow-stack ops at all — verified, not
assumed: their binaries are byte-identical with PERRY_SHADOW_STACK on and off.
w5_purity moves 0.0018 % across 17.2 G instructions, so the harness is not
measuring drift.

The ceiling, re-measured rather than inferred

Same experiment as the original ceiling run (PERRY_SHADOW_STACK on vs off),
re-run on the optimised runtime, 12 reps/arm, 72 runs, 0 dropped:

workload	ceiling before	ceiling after	share of the shadow-stack cost removed
w3_treewalk	+77.93 %	+41.34 %	47.0 %
w1_calldepth	+38.71 %	+28.01 %	27.6 %
w6_records	+71.53 %	+63.69 %	11.0 %

The two ceilings are comparable because the OFF arm is the same program in both:

workload	OFF, original session	OFF, this session	delta
w6_records	11,796,757,612	11,796,770,650	+0.0001 %
w3_treewalk	10,996,606,213	10,997,259,874	+0.006 %
w1_calldepth	19,643,910,251	19,644,035,098	+0.0006 %

The ON arms reproduce equally well: this session's ceiling ON counts agree with
the base/opt A/B's opt counts to within 0.0014 %, from independent compiles and
independent measurement runs.

What that means for the stack-map question

The cost profile is not uniform, and the shape of what is left is the useful
result:

• Deep pointer recursion (w3) was mostly implementation overhead. Nearly
  half the shadow-stack cost was three Vec::resize calls per activation.
  77.9 % → 41.3 %.
• Application-shaped code (w6) was mostly not. 71.5 % → 63.7 %. w6
  pushes shallow frames but stores slots constantly, so its residue is the
  per-store extern "C" call itself — the call, its TLS access, and its bounds
  check — none of which this PR can remove without changing what codegen emits.

So: a stack-map redesign has considerably less to win on recursive pointer code
than the original ceiling suggested, and roughly as much as it ever did on
application-shaped code. That is my reading of the numbers, not a measured
claim — attributing the w6 residue to call overhead specifically would need
a profile I did not take.

Sanity check on the mechanism

w3_treewalk performs ~10.5 M walk activations and saves 4.02 G instructions,
i.e. ~383 per activation — the right order for deleting two memset calls, four
capacity checks, two length updates and a barrier call per frame. The win is
where the change is, not somewhere else in the runtime.

Static, from the shipped libperry_runtime.a (aarch64, --release)

symbol	before	after
js_shadow_frame_push	159 instructions, up to 3 memset calls on the fast path	65, 0 calls on the fast path
js_shadow_frame_pop	44	36
js_shadow_slot_set	61, plus a barrier call that did its own TLS lookup	58, barrier inline-gated, 0 calls
js_shadow_slot_bind	62, same barrier call	51, whole entry written with one stp, 0 calls

Every bl remaining in these four symbols is on a cold or panic path.

Wall time

Directional only — wall cv is 2–13 % on this host. Medians: w3 2.625 → 2.147 s,
w1 3.163 → 2.849 s, w4 3.080 → 2.919 s, w6 2.373 → 2.294 s.

w2_allocring is the one arm where wall moved the wrong way (median −6.6 % in
the opt arm) while its instruction count improved 1.8 %. Its paired spread
crosses zero ([−14.8 %, +11.3 %]) and its backend-stall counter is the noisiest
in the set, so it is not a measured regression — but I could not attribute it
either way and record it rather than explain it away.

Gates

scripts/gc_repsel_matrix.sh --arms all --pressure 8, node 26.5.0:
PASS=339 UNVER=100 XFAIL=1 FAIL=0 over 440 cells, byte-exact 439/440 — the
required baseline, unchanged.

gc-ratchet: OK. More usefully, I built a pristine-main reference
runtime in the same worktree, on the same host, and measured it the same way:
all 72 gated retention and evacuation counters are bit-identical between
pristine main and this branch. (The small copied_objects movements the
checker reports against the pinned baseline artifact — ±0.02 % on 5 of 8 probes
— are present identically on pristine main, so they are pre-existing drift
from intervening commits, not this change.)

cargo test -p perry-runtime --lib: 1544 pass, 0 fail single-threaded;
gc:: 522/522. In parallel the run drops 2–3 tests
(gc::tests::copying::large_object_old_born_array_slot_write_keeps_young_child_alive,
gc::tests::teardown::map_set_side_allocations_release_*,
object::tests::closure_name_and_length_ignore_plain_assignment), with an
unstable failing set. I verified the same tests fail in a parallel run of a
pristine main checkout in this worktree, so they are the known
macOS parallel-ordering flake family, not this change.

Sabotage verification

Nine sabotages, each reverted after. Every one is caught.

#	Sabotage	Result
S1	delete the clear_slots call in js_shadow_frame_push	3 named tests FAIL; the full gc:: suite SIGSEGVs
S2	clear only ShadowEntry::value, leave meta (binding + liveness bit)	same 3 FAIL, incl. fresh_frame_does_not_inherit_popped_frame_bindings
S3	meta = 0 instead of meta & SLOT_PTR_MASK on the clear path	clearing_a_bound_slot_keeps_the_binding_for_later_reactivation FAIL
S4	bound_slot_meta returns raw | SLOT_ACTIVE (truncates a misaligned address)	bound_meta_encoding_rejects_addresses_that_would_clobber_the_liveness_bit FAIL
S5	scanner always hands out the mirror, never the bound pointer	3 FAIL, incl. bound_slot_survives_and_is_rewritten_by_a_copying_minor
S6	invert the barrier gate to == 0 (never shades)	slot_set_and_bind_shade_the_stored_value_while_a_cycle_is_active FAIL
S7	scanner ignores the liveness bit	4 FAIL
S8	swap the packed header fields (prev_frame_top ↔ slot_count)	SIGSEGV
S9	remove the js_shadow_frame_pop bounds guard	SIGSEGV

Not verified

• The Linux/aarch64 disassembly. The static instruction counts above are
  macOS/aarch64 (tlv_get_addr TLS); the Pi binaries are stripped, so I read
  the Linux result dynamically (instruction counts) rather than statically.
• w2_allocring's wall-time direction, as above.
• Non-aarch64 targets. No x86-64 or Windows measurement was taken; the
  change is portable Rust and the gates that ran are aarch64 macOS + aarch64
  Linux.

[proggeramlug]

Matrix re-confirmed on the final commit (b37108b8b, after the frame_pop
overflow-guard hardening): a second independent
scripts/gc_repsel_matrix.sh --arms all --pressure 8 run gives the identical
PASS=339 UNVER=100 XFAIL=1 FAIL=0, byte-exact 439/440.

The Pi instruction-count measurement was taken on that same final commit, not
on an intermediate one — the Pi runtime was re-patched and rebuilt so the
measured libperry_runtime.a is the tree this PR ships.