GC: evacuating minor drops an old-to-young field[1] edge, crashing with 'value is not a function'

Summary

A young-generation minor collection can reclaim or relocate a shared function object while other objects still hold a reference to it. When something later calls that reference, the program dies with TypeError: value is not a function (lodash reports the same thing as "Expected a function").

The reference that gets lost always lives in the second field of an object, and the collector never sees it because the old-to-young edge pointing at it is missing from the cycle's remembered set. This reproduces 100% of the time with PERRY_GC_FORCE_EVACUATE=1, and setting PERRY_GC_MOVING_LOOP_POLLS=0 makes it go away entirely, which points at the evacuating young-generation scavenge that became the default in #7019.

This is the same family of failure as #6831, but it happens at the object-field and remembered-set layer rather than the array-layout layer.

How to reproduce

Run any affected Node CLI under a forced evacuation:

PERRY_FORCE_WELL_KNOWN=iovalkey PERRY_GC_FORCE_EVACUATE=1 <binary> --help

It exits 1 with TypeError: value is not a function every time. Setting PERRY_GC_MOVING_LOOP_POLLS=0 produces a clean run, which is the kill switch that isolates this to the moving collector.

Adding PERRY_GC_FROMSPACE_SCAN=1 prints the dangling references described below.

What the diagnostic scan shows

After the array fixes in #7138 removed their false positives, roughly 36 genuine offenders remain per cycle. The rest are type=3 string entries, which are false positives from the conservative scan. The real ones are type=2 objects and type=4 closures:

owner=… type=2 space=Old       +40 nanbox -> <young> DANGLING       dirty_now=true  ever_dirty=true  marked=false
owner=… type=2 space=Survivor0 +40 nanbox -> <same young> DANGLING  dirty_now=false ever_dirty=false marked=true

The +40 offset is field[1]. An ObjectHeader occupies 32 bytes — object_type, class_id, parent_class_id and field_count take 16, the keys array takes 8, and meta takes 8 — so field[1] lands at offset 40.

All of those offenders point at a single shared young object, referenced from field[1] of about 30 different objects. That shared object is a constructor function, which is why the crash surfaces as "value is not a function".

An allocation-site backtrace shows the holders are created in js_object_set_field_by_name, called from proxy::create_or_update_receiver_property, called from zod's $constructor in node_modules/zod/src/v4/core/core.ts.

Two details rule out the obvious explanations. layout_covers_slot is true with the layout state at SIDE_MASK, so the traced layout does visit field[1] and this is not a layout-coverage gap. And the snapshot counters read in_snapshot = 0, not_in_snapshot = 43, meaning none of these old-to-young field[1] edges are in the cycle's remembered-set snapshot, so the minor never scans them at all.

Where the edge is being lost

The old-to-young field[1] edge is dropped across a cycle boundary and never recovered. Four places are worth checking, in rough order of suspicion.

remembered_set_clear in crates/perry-runtime/src/gc/copying.rs around line 1207 clears DIRTY_OLD_PAGES on every cycle.

restore_surviving_dirty_coverage in crates/perry-runtime/src/gc/verify.rs around line 167 only rescans pages that were in the pre-cycle snapshot. Once a page is dropped it is never re-remembered, so the loss cascades. This is the area #5029 covers.

The promotion path is also a candidate: rebuild_evacuated_old_to_young_remembered_set and remember_evacuated_old_copy_young_slots, both in verify.rs, gate on pointer_in_old_gen and only process moved_headers.

Finally, part of the population has ever_dirty=false, meaning the write barrier never fired for that store at all. That suggests a barrier skip on old-object field stores in js_object_set_field_by_name or runtime_store_jsvalue_slot, which is a different bug from the snapshot drop affecting the rest.

Suggested next diagnostic step

The ~36 offenders appear to be two distinct bugs sharing one symptom, so the next step is to split them. Re-run with a global allocation-site tag plus a per-slot barrier trap on js_object_set_field_by_name and runtime_store_jsvalue_slot, restricted to old-object field stores whose child is young.

Offenders with ever_dirty=false are barrier skips, and the fix is to route that store through the write barrier. Offenders with ever_dirty=true are snapshot drops, and the fix belongs in restore_surviving_dirty_coverage or remembered_set_clear so that an old-to-young edge to a still-live shared young object survives the cycle boundary.

One approach that does not work

Widening the collector to trace beyond length or into uninitialized capacity is not a fix. It has been tried and it segfaults on the uninitialized slack in js_array_alloc. The correct layer for this fix is the remembered set and the write barrier.

Relationship to #7138

#7138 is a separate change and does not fix this. It landed two array soundness fixes: layout_note_slot restoring SIDE_MASK, which took within-length array-to-young losses from 202 to 0, and HOLE-initialisation of the slack in js_array_alloc and js_array_grow, which removed roughly 4000 uninitialized-capacity false positives per cycle from PERRY_GC_FROMSPACE_SCAN.

Those fixes matter here only because clearing the false positives is what made the ~36 genuine offenders above visible.

Barrier-trap split done — it is not case (a) barrier-skip or (b) snapshot-drop. The remembered set is doing its job; the residual bug is in the copying collector's evacuation of the shared young target.

Worked from a branch off #7138's tip (e04796621), reproduced RED 100% (PERRY_FORCE_WELL_KNOWN=iovalkey PERRY_GC_FORCE_EVACUATE=1 ./bin-registry --help → TypeError: value is not a function). Added temporary genuine-only (non-string) counters + per-offender layout/coverage instrumentation to fromspace_scan.rs and a layout_debug_slot_coverage probe (all reverted now).

Split result (genuine = `type != 3`; type=3 strings are the accepted conservative-scan FPs)

Representative cycle, ~250 genuine offenders:

bucket	count	meaning
`dirty_but_missed`	249 (97%)	page IS in `DIRTY_OLD_PAGES` and in the pre-cycle snapshot, scanned by the remembered-set pass, slot still not rewritten
`in_snapshot`	249	"
`never_dirty`	7 (~3%)	all freshly-copied Survivor holders (young→young, no barrier expected)
`lost_dirty`	0	case (b) snapshot-drop ruled out — nothing recorded-then-lost across the cycle boundary

So the predicted split ("some (a), most (b)") does not hold on this base — #7138's array fixes shifted the residue. remembered_set_clear / restore_surviving_dirty_coverage are not implicated: 97% of genuine offenders are on pages that were remembered and scanned this cycle.

What the offenders are

Every genuine offender in a cycle is a type=2 object, slot +40 = field[1], fields=3, layout=0x9000 (SIDE_MASK|TYPED_LAYOUT_INTACT), all pointing at a single shared young target (e.g. 0x294866e0700, NurseryEden, DANGLING/not-forwarded), from both:

genuine Old holders (gc_flags=0x22 = ARENA|TENURED, dirty_now=true, remembered), and
freshly-copied Survivor holders (gc_flags=0x43, marked=true).

Layout coverage is correct: layout_debug_slot_coverage = 0x19 for all of them = INTACT(0x01) + shape-shared descriptor present(0x08) + shape descriptor pointer_mask contains field[1] (0x10), with no per-object LAYOUT_SLOT_MASKS shadow. Static trace of the enumerators (heap_payload_slot_selection SIDE_MASK arm → shape_shared_pointer_mask; visit_gc_layout_slot_descriptors Masked arm; scan_dirty_object_slots) shows they should all emit field[1] as a masked Slot on the dirty page.

Refined root cause / why a minimal remembered-set edit is unsound

Decisive fact: within one cycle ~250 holders — Old-remembered and Survivor-copied, all with field[1] masked-covered — reference the same young target and none of them evacuates it. Since those holders are demonstrably scanned (remembered pages) or copied (survivors) with field[1] in the mask, the failure is not in the write barrier / remembered set / layout mask — it is that the shared young target is not evacuated even though its field[1] edges are enumerable. That implicates the copying-minor evacuation/classification of the shared young object (copying::visit_value_bits / CopyingPointerSet / pin/tenure/large-exclude handling), not copying.rs remembered_set_clear (~L1207) or verify.rs restore_surviving_dirty_coverage (~L167).

A remembered-set / snapshot edit (the original plan's fix) would not fix the 97% remembered+scanned offenders and would risk corrupting the heap, so I stopped rather than ship it.

Suggested next diagnostic step (GC owner)

Instrument copying::visit_value_bits / move_young for the shared young target: for each field[1] edge, log whether visit_value_bits(Y) is actually invoked and what it returns (None vs new bits), plus Y's header gc_flags/space/obj_type at that instant (pinned? tenured-young? malloc-GC? large-excluded?). That distinguishes enumerated-but-no-op (evacuation/classification bug) from never-enumerated (a deeper layout-timing bug) — the current data cannot separate those two without a live-trace probe on the evacuation path.

Repro: PERRY_FORCE_WELL_KNOWN=iovalkey PERRY_GC_FORCE_EVACUATE=1 <sfw-registry> --help; PERRY_GC_FROMSPACE_SCAN=1 prints the offender split.

Live-trace evacuation probe done — it is enumerated-but-no-op, and the target is a dangling pointer to an already-reclaimed young closure. `classify` is behaving correctly; the real miss is one cycle upstream. Recommending the stopgap.

Worked from the #7138 tip (e04796621). Added a temporary live-trace probe on the evacuation path (gated on PERRY_GC_EVAC_PROBE; all reverted now): it records every slot handed to visit_slot_with_parent on the evac path, and — in run_fromspace_scan — reports, per genuine offender, whether that slot was enumerated on the evac path plus the target's live header bytes.

Result: enumerated-but-no-op (the "never-enumerated" alternative is ruled out for the observed cycles)

For every genuine (type != 3) offender — type=2 object holder, slot +40 = field[1], both Old-remembered and Survivor-copied:

enumerated_on_evac = true — field[1] is visited by visit_slot_with_parent on the evacuation path.
visit_value_bits → mark_addr → CopyingPointerSet::classify returns None, so the slot is never rewritten and the target is never forwarded (forwarded=false).

So this is the enumerated-but-no-op branch, not never-enumerated. But the reason classify returns None is the decisive part:

The field[1] value is a nanboxed pointer to reclaimed young memory (garbage header)

[evac-probe] ENUMERATED-noop(classify=None) bits=0x7ffd0566faae04c0 nanbox=true target=0x566faae04c0
   t_obj_type=56 t_gc_flags=0x43 t_size=2147419494 t_space=NurseryEden
   plausible_arena=false plausible_malloc=false classify=None parent_obj_type=2 slot_off=40

bits = 0x7ffd… → a genuine NaN-boxed POINTER_TAG value (not a bare/conservative candidate).
target is in NurseryEden, but the header at target-8 is garbage: across all samples plausible_arena=false (100%), and t_obj_type / t_size / t_gc_flags vary run-to-run (obj_type 56/232/184/16/11…, size ≈ 2^31 or 0) as the freed slot is re-bump-allocated in later cycles.

In other words: classify_arena/plausible_gc_header are correctly rejecting a pointer that no longer points at a live GC object. field[1] is a dangling reference — the young closure it named was reclaimed in an earlier evacuating minor, and field[1] was never repaired.

Why this proves the miss is one cycle upstream

The offender population includes freshly-copied Survivor holders (gc_flags=0x43, marked=true). A survivor holder is copied this cycle and its fields are scanned during drain → scan_object_fields → visit_slot_with_parent(field[1]). If the closure were a live, valid young object at that moment, it would be evacuated right there. It is not — it is already garbage — so the closure died in a prior cycle. Every cycle my probe can observe is post-free, which is exactly why the header is always garbage and classify (rightly) says None.

This also means a classify / move_young / CopyingPointerSet edit that "accepts" the target would evacuate garbage → SIGSEGV, matching this issue's own "widening the collector segfaults" note. I did not ship such an edit.

Severity is higher than the title implies

Plain ./bin-registry --help with no env at all also exits 1 with TypeError: value is not a function — the evacuating minor is default-on (#7019), so this is not confined to PERRY_GC_FORCE_EVACUATE=1.

Confirmed stopgap

Disabling the moving loop eliminates the crash, deterministically:

invocation	result
`./bin-registry --help` (default)	crash (exit 1)
`PERRY_GC_FORCE_EVACUATE=1 ./bin-registry --help`	crash (exit 1)
`PERRY_GC_MOVING_LOOP_POLLS=0 ./bin-registry --help`	clean (0), 3/3
`PERRY_GC_FORCE_EVACUATE=1 PERRY_GC_MOVING_LOOP_POLLS=0 …`	clean (0), 3/3

Recommendation (GC owner)

Unblock now: gate the evacuating minor off by default — revert gc: default-on evacuating young-gen scavenge (moving minor at precise safepoints) #7019's default-on behind its flag. Confirmed above to eliminate the crash with no behavior change to correctness (falls back to the non-moving minor).
Root cause to fix: the surviving evidence points at the live cycle where the young closure is referenced from a dynamically-added field[1] (holders are built in proxy::create_or_update_receiver_property from zod's $constructor) but is not evacuated through that edge, so it is reclaimed and field[1] dangles. The most consistent explanation with the "coverage is correct in the observed (post-free) cycles" data is a shape/layout pointer_mask timing gap: field[1]'s masked coverage is not yet in effect in the cycle where the closure is young (so the edge is dropped), and only becomes covered in later cycles (where the target is already garbage). The decisive next probe is to tag closures at allocation and catch the first minor in which a live closure is referenced by an enumerated field[1] yet not forwarded, then compare the holder's shape pointer_mask/SIDE_MASK state in that cycle vs. later.

Probe was temporary and is fully reverted; base unchanged at e04796621.

Live-cycle probe: the drop is a live-cycle old→young remembered-set/scan miss on the dynamically-added field[1] edge — not a coverage-mask gap. Two conservative-coverage fixes were built and both FAIL to stop the crash, which pins the fault to the holder not being scanned at all in the cycle where the closure is still live. Escalating with a deterministic repro + exact fix spec.

Worked from #7138's tip (e04796621) on fix/gc-dynamic-field-young-evac. All instrumentation below is reverted; the branch is back at e04796621 with no diff. I did not ship a fix — both attempts are proven ineffective (see §Fix attempts), so shipping either would be a speculative heap edit.

New, decisive evidence

1. The live-cycle drop is real and it IS field[1] (+40). I extended run_fromspace_scan (runs after the rewrite pass, before flip — from-space still intact) to read each dangling target's still-live header and the holder's layout state. It captured the smoking gun the prior rounds could not — a dangling target that is a live closure, not garbage:

owner=0x5194120f240 type=2 space=Old +40 nanbox -> 0x51940eda328 (Survivor1)
  MISSING-REWRITE (target moved) [dirty_now=true ever_dirty=true owner_flags=0x22 marked=false]
  {PROBE t_type=4(CLOSURE) t_arena_ok=true live_closure=true | owner_state=0x8000(SIDE_MASK) intact=true slot_masked=false}

So in the live cycle the closure IS evacuated (via some other referent) but this holder's field[1] was neither traced nor rewritten → after flip it dangles → later read+called → "value is not a function". This confirms classify is a red herring: by the crashing cycle the target is already garbage (as round 2 found), but the originating miss is a live closure at field[1] of a surviving holder.

2. The offenders are 100% SIDE_MASK + INTACT, never POINTER_FREE. Layout-state histogram of the genuine (type=2, +40) danglers across 5 runs: 342 owner_state=0x8000 intact=true, 0 POINTER_FREE. Holder population is the expected mix: owner_flags=0x22 (Old, remembered, dirty) and 0x43 (freshly-copied Survivor). Owners are the zod $constructor holders (node_modules/zod/src/v4/core/core.ts — inst[k] = proto[k].bind(inst) stores a freshly-bound young closure into a dynamically-added field).

3. The fromspace dangler COUNT is post-death-dominated and cannot verify a live-cycle fix. In any given --help run almost every sampled dangler already has a garbage target (t_type = 192/248/56/99/16/… — the freed slot re-bump-allocated), because a --help does one startup burst of zod construction and the scan mostly sees misses that already happened. A scan-time fix cannot retroactively un-dangle an already-reclaimed target, so dangler count is the wrong metric — the crash itself is the only reliable signal. (Under machine load the crash is now deterministic: PERRY_FORCE_WELL_KNOWN=iovalkey PERRY_GC_FORCE_EVACUATE=1 ./bin-registry --help → 5/5 exit 1, ~20s each.)

The linchpin: conservative enumeration does NOT save the closure

I built two fixes that force the holder to trace all slots (so field[1] can't be skipped by a stale/short mask):

Fix A — gc/layout.rs::shape_shared_pointer_mask: reject a shared shape descriptor whose pointer_mask omits any slot holding a genuine heap pointer → caller falls back to the conservative all-slots scan.
Fix B — gc/layout.rs::heap_payload_slot_selection (SIDE_MASK arm, the single enumeration choke point for marking, evac-drain and the remembered-set scan via visit_gc_rewrite_slot_descriptors): same check applied to the resolved mask (per-object LAYOUT_SLOT_MASKS or shared), downgrading to UNKNOWN+All on any uncovered heap-pointer field.

Both are provably sound (worst case = conservative scan). Both still crash 5/5. Making the holder's field enumeration conservative changes nothing ⟹ the holder is not being enumerated at all in the live cycle. That rules out the coverage-mask hypothesis and points squarely at the remembered-set / write-barrier membership of the old→young field[1] edge in the live cycle: the holder (freshly Old / just-promoted) is not in that cycle's scan set, so its field[1]→young-closure edge is never followed and the closure is reclaimed. (Round 1 correctly found the edge is remembered in the post-death cycles — dirty_but_missed, lost_dirty=0 — but that is one or more cycles after the closure already died; the miss is in the earlier cycle where the holder first goes Old with field[1] still young.)

Exact next probe + fix spec (for the GC owner)

Probe: tag the zod bound-method closure at allocation; on each minor, for the FIRST cycle in which a surviving holder's field[1] points at that still-live closure, log whether the holder is in the cycle's scan set — i.e. for an Old holder, is its page in DIRTY_OLD_PAGES/the snapshot; for a just-promoted holder, did the promotion register the edge. Instrument the promotion remembered-set rebuild (gc/verify.rs::remember_evacuated_old_copy_young_slots / rebuild_evacuated_old_to_young_remembered_set) and the closure-store write barrier. Expected finding: in that first cycle the field[1]→young edge is absent from the remembered set (barrier not yet fired, or promotion enumerated the old copy's young slots via a path that skipped field[1]).

Fix (most likely, in order):

Ensure promotion of a holder that still references a young child records every young pointer slot into the sticky remembered set — audit remember_evacuated_old_copy_young_slots uses the same enumeration that will later scan it, and that a holder promoted while UNKNOWN (dynamic-add leaves it UNKNOWN) still has its young slots remembered (an UNKNOWN/mask mismatch here would drop the edge from the rebuild, mirroring what Fix A/B addressed on the scan side).
Route the zod dynamic-field closure store through the old→young write barrier for the case where the holder is already Old at store time.

Separately (latent, not the crash): SHAPE_LAYOUTS (#6893 shared shape descriptors, gc/layout.rs) is keyed by the keys_array heap address and is only ever inserted — never removed or re-keyed when that array is freed/relocated. A reused keys_array address can serve a stale descriptor to a different shape. Worth fixing independently (lifecycle-invalidate on keys_array free/move, or key by a stable shape id), but it is not the #7154 crash driver — Fix A targeted exactly this and did not help.

Verification scorecard (this round)

config	base `e04796621`	Fix A	Fix B
`iovalkey + FORCE_EVACUATE` `--help`	5/5 crash	4/4 crash	5/5 crash
`MOVING_LOOP_POLLS=0`	clean	—	—
fromspace `type=2 +40` danglers	~290–520 (post-death dominated; not a valid fix metric)	unchanged	unchanged

Net: the stopgap (gate the evacuating minor off by default, #7019 revert-behind-flag) remains necessary until the live-cycle remembered-set miss above is closed. I stopped rather than ship a speculative edit to the remembered-set/barrier without a confirmed pinpoint, per the memory-corruption guardrail.

One root cause traced and fixed — but it is not the whole of #7154. Keep the stopgap.

PR #7179. Posting the status now rather than after a full fix, because the #7161 decision depends on it.

The invariant that was broken

Perry allocates every heap object in the POINTER_FREE layout state and only leaves that state when a store records a pointer through layout_note_slot. heap_payload_slot_selection short-circuits on POINTER_FREE and skips the entire payload without consulting any mask. So a raw *slot = bits into a traced slot does not merely lose precision — it makes the child invisible, and the evacuating minor neither keeps it alive nor rewrites the slot when it moves. The same raw store also skips runtime_write_barrier_slot, so an old→young edge never reaches the remembered set.

A store that publishes a pointer-bearing value into a slot the collector traces — object field, array element, closure capture — must go through the barriered store helper. A raw slot write is sound only for a provably non-pointer value, or an object that is not yet published.

Two sites broke it

bind_reserved_this_slot (symbol/properties.rs), shared by js_object_set_method_by_name (perry-codegen: object literal with computed-key props + computed-key methods + cross-module spread drops keys & mis-resolves methods — Effect HashRing.ts blocker (post-#740) #809 — the ordered object-literal lowering used when a ...spread precedes a this-reading method) and js_object_set_symbol_method. It patched the closure's reserved this capture slot with a raw *slot = f64.
js_weakmap_set's overwrite-existing-key path (weakref.rs) wrote the new value into the entry's field 1 — byte offset +40, the exact offset your scan reports — with write_object_field_bits_raw. The insert path already used js_object_set_field; only the overwrite was raw. Note this one produces ever_dirty=true offenders once the entry has been dirtied by anything else, and dirty_but_missed when the layout mask does not cover the slot — the two buckets your split reported.

Plus two hardening changes in the same family: js_closure_alloc now undefined-initializes its capture slots (previously raw recycled arena bytes — the closure analogue of #7138's array-slack fix, and a likely contributor to the bogus-slot decodes in #7164), and js_object_set_field_by_name's three dynamic-key append paths now widen field_count before publishing the value (gc_field_slot_range bounds the collector's payload view by field_count, so the old order left a window where the new slot was written but invisible).

How it was traced

Reproducer: zod 4.4.3 compiled natively, 400 schemas × 20 rounds of safeParse. Fails in the default configuration, clean under PERRY_GC_MOVING_LOOP_POLLS=0 — your fingerprint.

PERRY_GC_FROMSPACE_SCAN=1 → 294 dangling type=4 (closure) capture-slot edges per cycle, Survivor1 holders, NurseryEden targets, ever_dirty=false.
A probe on the offenders reported _reserved = 0x4008 = GC_LAYOUT_POINTER_FREE, gc_child_slots emitting zero child slots, and the offending slot falling inside the skipped PointerFreeRange. capture_count = 0x80000001 = CAPTURES_THIS_FLAG | 1 → the dangling slot is the reserved this slot.
A #[track_caller] trap on set_layout_state / layout_transfer showed the Eden original was already POINTER_FREE with a live pointer capture and no per-object mask. So the capture was never recorded — it was not lost on the move, and not a remembered-set problem. That leaves exactly one store path: the raw one above.

This is the same shape as your finding, one layer up: your probe correctly concluded classify was right to reject the target and that the miss was upstream. The upstream miss is the store never registering the slot, so the collector's enumerator never offers it to classify in the cycle where the child is still live.

Result: dangling closure-capture edges 294 → 0 per cycle.

Why the stopgap should stay

With that population at zero, the zod binary still exits non-zero in the default configuration and is still clean under PERRY_GC_MOVING_LOOP_POLLS=0. At least one more offender remains. The residue the scan now reports is dominated by type=1 array owners whose own header carries GC_FLAG_FORWARDED — stale relocated copies, most likely scan false positives (scan_object skips from-space owners but not forwarded ones, which looks worth fixing in the instrument itself). The most interesting genuine-looking remainder is a handful of GC_FLAG_SHAPE_SHARED keys-array slots in Survivor0 with un-evacuated nursery string targets.

I have not chased that one. #7179 is necessary but not sufficient — I would not drop #7161 on the strength of it.

Refs #7161, #7138, #7164.

test(gc): register #7154 pointer-publish gap test in the GC x repsel …

Final round: both directed fixes implemented and empirically disproven; the GC heap, root, and remembered-set layers are now exonerated by direct measurement. The stale value is injected by the MUTATOR from a location no GC surface scans — the evidence points at a codegen liveness gap across the loop back-edge moving safepoint, which is exactly what the `PERRY_GC_MOVING_LOOP_POLLS=0` kill switch removes. Full evidence chain + fix spec below.

Worked from #7138's tip (e04796621) on fix/gc-dynamic-field-young-evac; all instrumentation and candidate fixes reverted; branch back at e04796621, no diff. Repro used throughout: PERRY_FORCE_WELL_KNOWN=iovalkey PERRY_GC_FORCE_EVACUATE=1 ./bin-registry --help — under load this is now deterministic (5/5, ~20s per run), which made every experiment below conclusive.

1. The directed fixes do not fix it

Store-side mask coverage ("update pointer coverage before/with the store"): implemented at BOTH ends — a validation in heap_payload_slot_selection (gc/layout.rs SIDE_MASK arm and POINTER_FREE arm) that downgrades to a conservative all-slots scan whenever the recorded layout omits a slot currently holding a nan-boxed heap pointer, plus a faithful mask rebuild at layout_note_slot's POINTER_FREE→SIDE_MASK upgrade (which today mints a mask containing only the just-stored slot — a real latent bug, see §5). Since heap_payload_slot_selection is the single slot-selection choke point for marking, the evac drain, the remembered-set scan (scan_dirty_object_slots) and promotion-time registration (remember_evacuated_old_copy_young_slots → visit_gc_rewrite_slots), this subsumes the intent of both suggested fixes. Result: still crashes 4/4 – 5/5, offender counts unchanged (fromspace type=2 +40: base 293 vs fixed 312/519 across runs — noise, no reduction).
Conservative promotion registration: covered by the same choke-point validation (at registration time field[1] holds the young pointer, so an understating mask forces the all-slots path there too). Same result.

2. Systematic elimination — each layer measured in the live process

(a) Only one collector ever runs. PERRY_GC_DIAG=1 on a crashing run: 695/695 cycles are [gc-copy-minor] eligible=true fallback=none — no non-moving minor, no full cycle. Every cycle is fromspace-scanned, so nothing is happening in an unobserved cycle kind.

(b) Heap-side enumeration is sound. Temporary probe: record every slot address handed to visit_slot_with_parent per cycle; cross-reference in run_fromspace_scan (same cycle, pre-flip). 315/315 sampled +40 offender slots: slot_enum=true, hdr_scanned=true — every offender slot is visited every cycle; classify correctly no-ops because the target is already garbage.

(c) No visit ever mishandles a live or forwarded target. Anomaly logger inside visit_slot_with_parent: fire whenever visit_value_bits returns None for a nan-boxed target in from-space with a plausible header (live or genuinely forwarded). Zero hits across all cycles of 3 crashing runs. The mark_addr/classify/move_young path never drops or fails to rewrite a live edge it is shown.

(d) All GC-visible roots are clean. Post-rewrite, per cycle: shadow stack + module globals (visit_mutable_root_slots) + every registered runtime side-table scanner (MUTABLE_ROOT_SCANNERS, walked read-only via RuntimeRootVisitor::for_copy) checked for from-space referents. Zero stale entries, every cycle, every run.

(e) The runtime store choke point never stores a detectably-dangling pointer. Probe in runtime_store_jsvalue_slot (backtrace on storing a nan-boxed pointer to arena memory with an implausible header): zero hits — with the caveat that recycled-Eden targets can carry plausible-looking foreign headers, so this filter has false negatives; taken with (b)–(d) it still means no runtime-helper store path is the injector.

3. What is left — and it explains every observation

At the cycle where the shared closure dies/moves, no heap slot, no root, no side table holds its address (otherwise (b)/(c)/(d) would have caught it — the fromspace scan sees every heap holder and I extended it to every root surface). The stale address re-enters the heap afterwards, via mutator stores that bypass runtime_store_jsvalue_slot — i.e. codegen-inlined stores — sourced from a raw value that survived a moving minor un-rewritten.

Where can compiled code hold a heap value that no GC surface sees? A register / unspilled temporary live across the loop back-edge safepoint — the only place evacuating minors run (js_gc_loop_safepoint → gc_safepoint_moving_minor, gc/policy.rs:1638; alloc-point triggers defer to the next back edge). The documented soundness argument is "at a back-edge … every live value is a named local on the shadow stack: a precise-root safepoint". The crash is that guarantee failing for at least one value.

This is the only hypothesis consistent with all of:

the kill switch: PERRY_GC_MOVING_LOOP_POLLS=0 removes precisely the back-edge collections (and with no evacuation nothing moves, so a stale raw value is harmless);
gc: default-on evacuating young-gen scavenge (moving minor at precise safepoints) #7019 (default-on evacuating minor) being the trigger;
offenders visible only post-death in every scanned cycle (the injection happens in mutator windows between cycles);
Old holders with ever_dirty=false (a barrier-less codegen store of an old→young edge would also explain the unbarriered slots — same codegen path, same fix domain);
one shared young target referenced from field[1] of ~250 shape-identical fields=3 holders: the holders are zod's _zod records {def, constr, traits} (node_modules/zod/src/v4/core/core.ts $constructor), field[1] = constr, the shared target = the $constructor closure _ — init stores constr: _ into a fresh record on every new Schema(), so one stale copy of _'s address fans out to every record built after the move;
my minimal zod repro (800× z.string()) not crashing while the registry (many distinct schema classes, deep loops) does — the window is a loop back edge in/around the $constructor/init path (inst[k] = proto[k].bind(inst) allocates every iteration, forcing safepoint polls mid-loop).

4. Fix spec (codegen owner)

The fix is in perry-codegen's shadow-stack spill/reload contract around js_gc_loop_safepoint polls, not in the runtime GC:

Audit values live across a back-edge poll that codegen keeps in registers (or reads once and reuses after the poll) instead of reloading from their shadow-stack/capture-slot homes after the safepoint. Prime suspects in the zod path: the capture read of _ used for constr: in init, and receiver/prototype temporaries in the proto[k].bind(inst) loop.
Confirming probe: --trace llvm / --trace hir --focus init on the bundled zod $constructor; check whether any heap-valued temp crosses the js_gc_loop_safepoint call without a post-call reload. A runtime tripwire also works: in gc_safepoint_moving_minor, after the minor, poison (or log) any unbound shadow entries vs. live register state — or simply compile with back-edge polls off for the zod module and watch the crash disappear (module-scoped bisection).
The same codegen path likely also skips the old→young write barrier for those stores (the ever_dirty=false Old holders), so the spill audit should cover barrier emission on the inlined store too.

I did NOT attempt this codegen fix: it is outside the runtime-GC scope I can validate with the tools at hand, and the guardrail (never ship a speculative heap/GC edit — extended: or a speculative codegen-liveness edit) applies. Runtime-side "fixes" are now proven to be the wrong layer — please do not spend another round there.

5. Two real (but non-crashing) latent bugs found on the way — worth independent tickets

layout_note_slot's POINTER_FREE→SIDE_MASK upgrade (gc/layout.rs ~L765) mints a mask containing only the just-stored slot, silently un-covering any pointer already present in another field. Faithful fix: rebuild from actual field words (implemented + tested here, no behavior change on this crash).
SHAPE_LAYOUTS (TYPED_LAYOUTS stores per-class-constant layout masks per OBJECT — O(objects) memory (272MB on churn bench) + hashmap insert on every new #6893) is keyed by keys_array address and never invalidated on free/relocation — a reused address can serve a stale shared descriptor to a different shape. (Fix A targeted this; not the crash driver.)

6. Scorecard

experiment	result
base, `iovalkey+FORCE_EVACUATE`	5/5 crash (~20s each, deterministic under load)
+ choke-point coverage validation (both arms) + note_slot mask rebuild + conservative promotion	4/4 – 5/5 crash
cycle census (`PERRY_GC_DIAG`)	695/695 copying-fast-path minors
offender slots enumerated in-cycle	315/315
visit-noop on plausible from-space target	0
stale from-space refs in roots/globals/all registered scanners, post-rewrite	0
dangling stores via `runtime_store_jsvalue_slot`	0
`PERRY_GC_MOVING_LOOP_POLLS=0`	clean

The stopgap (#7161, gate the evacuating minor off by default) remains the correct unblock until the back-edge safepoint liveness gap is closed. All probes reverted; branch clean at e04796621.

fix(gc): disable evacuating minor by default pending #7154 (#7161)

fix(gc): record layout+barrier on raw pointer publishes into traced p…

test(gc): #7154 pointer-publish invariant regression tests

test(gc): strengthen #7154 weakmap barrier test (old entry -> young v…

docs: changelog fragment for #7154

test(gc): register #7154 pointer-publish gap test in the GC x repsel …

docs(gc): record that #7154's corpus member is live only on the movin…

refactor(codegen): Layer 1 rooting migration slice 6 — the multi-poin…

feat(gc)!: seeded GC-schedule fuzzing (PERRY_GC_SCHEDULE_SEED)

fix(gc): moving-loop back-edge polls default ON again (#7161 stopgap …

fix(gc): restore evacuation at precise safepoints — the pacing half of …

feat(gc)!: seeded GC-schedule fuzzing (PERRY_GC_SCHEDULE_SEED)

feat(gc): seeded GC-schedule fuzzing (PERRY_GC_SCHEDULE_SEED)

feat(gc): seeded GC-schedule fuzzing so a failing schedule is reprodu…

Make a rare garbage-collector bug reproducible on demand (PERRY_GC_SC…

gc: diagnostics, two rooting fixes, and the ungated corpus/lowering c…

fix(gc): root raw pointers in dynamic-object get, add a PROCESS_EMITT…

batch: land #8848, #8849, #8850, #8852, #8853 (#8857)

perf(codegen): collapse the generic property-get tower to two exits

perf(codegen): build a rest bundle the way an array literal is built

GC: evacuating minor drops an old-to-young field[1] edge, crashing with 'value is not a function' #7154

Description

Summary

Relationship to #7138

Activity

jdalton commented on Aug 1, 2026

Barrier-trap split done — it is not case (a) barrier-skip or (b) snapshot-drop. The remembered set is doing its job; the residual bug is in the copying collector's evacuation of the shared young target.

Split result (genuine = type != 3; type=3 strings are the accepted conservative-scan FPs)

What the offenders are

Refined root cause / why a minimal remembered-set edit is unsound

Suggested next diagnostic step (GC owner)

jdalton commented on Aug 1, 2026

Live-trace evacuation probe done — it is enumerated-but-no-op, and the target is a dangling pointer to an already-reclaimed young closure. classify is behaving correctly; the real miss is one cycle upstream. Recommending the stopgap.

Result: enumerated-but-no-op (the "never-enumerated" alternative is ruled out for the observed cycles)

The field[1] value is a nanboxed pointer to reclaimed young memory (garbage header)

Why this proves the miss is one cycle upstream

Severity is higher than the title implies

Confirmed stopgap

Recommendation (GC owner)

jdalton commented on Aug 1, 2026

New, decisive evidence

The linchpin: conservative enumeration does NOT save the closure

Exact next probe + fix spec (for the GC owner)

Verification scorecard (this round)

proggeramlug commented on Aug 1, 2026

One root cause traced and fixed — but it is not the whole of #7154. Keep the stopgap.

The invariant that was broken

Two sites broke it

How it was traced

Why the stopgap should stay

jdalton commented on Aug 1, 2026

1. The directed fixes do not fix it

2. Systematic elimination — each layer measured in the live process

3. What is left — and it explains every observation

4. Fix spec (codegen owner)

5. Two real (but non-crashing) latent bugs found on the way — worth independent tickets

6. Scorecard

63 remaining items

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Split result (genuine = `type != 3`; type=3 strings are the accepted conservative-scan FPs)

Live-trace evacuation probe done — it is enumerated-but-no-op, and the target is a dangling pointer to an already-reclaimed young closure. `classify` is behaving correctly; the real miss is one cycle upstream. Recommending the stopgap.