jit: #73 Layer-2 + pc_map retirement — retire model/carrier Python-pc channels and the runtime py→jit inversion table (S1–S7, W1–W3b) by youknowone · Pull Request #602 · pyre-lang/pyre

youknowone · 2026-07-16T15:24:36Z

Continues the #73 pc_map-removal epic (follow-up to #589). Layer-2 slices S1–S6: every remaining model/carrier Python-pc channel in the walker is deleted; the Python coordinate is now derived once at the interpreter-flush boundary from a carried JitCode coordinate, or replaced outright by a certified jitcode-keyed twin read.

S1 — InlineParentFrame.call_py_pc (2 commits): thread call_jitcode_pc through every producer, then delete the py field; the flush boundary derives py via the same python_pc_for_jitcode_pc the producers ran (pure code motion). Unresolvable jitcode_index declines the flush defensively.

S2 — WalkContext.entry_py_pc (2 commits): replaced by EntryPyPc { Py(u32), Jit(usize) }; two eager backxlat_py_pc producer calls and one inline inversion expansion deleted. A marker-derived flip was considered and rejected (raw producing coordinate carried instead — census precedent shows resume markers diverge from trivia-normalized coordinates).

S3 — inline-abort carrier py fields (2 commits): InlineAbortCarrier/MidBodyPayload lose call_py_pc/post_call_py_pc/callee_py_pc; the eager outer-CALL inversion block becomes a carried (jitcode_index, op.pc) coordinate resolved at the trace.rs flush boundary. Entry leg observed live (22 fires per backend, 0 divergence).

S4 — FOR_ITER inflight body_pc (2 commits): InflightForiterBody { Py, Jit }; the fbw_foriter_body_pc_from_op_pc helper and its 3 producer calls deleted; resolution happens just before the keyed lookup at all 5 delivery sites (203 resolved fires, 0 divergence). Resolution failure = no match (conservative refuse-to-deliver; legacy bypass runs the iteration once).

S5 — InlineParentFrame.resume_py_pc (2 commits): replaced by a required ParentResumeCoord { Backxlat, CallFallthrough }; the two vable consumers derive at the boundary, and 4 py-keyed table reads flip to their certified jitcode-keyed twins (depth_for_jitcode_pc_pred, pcdep_for_jitcode_pc, depth_trivia_for_jitcode_pc, result_color_for_jitcode_pc_pred) — census 6,164/6,164 equal across both backends. Missing marker falls back to legacy py; missing metadata declines with GuardResumeCoordinateUnavailable.

S6 — boundary dedup + frontier censuses (2 commits): the duplicated WALK_END restart-pc inversion in trace.rs is derived once; report-only censuses for the recipe result-color read (0 fires — gate left in place) and the multi-frame callee diagnostic inversion (diagnostic-only; census re-confirmed plain resume marker ≠ trivia-normalized coordinate, 89/1,673 divergence on both backends).

With this the mechanical Layer-2 frontier is exhausted: no model or carrier struct stores a Python pc. Remaining py reads are boundary derivations (single inversion points) plus the vstack_cur_pypc mirror, which retires with the walker-as-tracer core (Layer 3).

Verification per slice: codex-implemented, independently re-verified — check.py dynasm 208/208 + cranelift 208/208 (audit-off), audit-on runs green (PYRE_PCMAP_*_AUDIT), cargo test -p pyre-jit-trace -p pyre-jit standalone green. Only defensive declines added; no panic paths.

S7 — guard-capture py narrowing (3 commits): the dead resume_py_pc stamp and the encode_after_residual_call_pc py-word encode are deleted; three guard-capture depth reads flip to the trivia/marker jitcode-keyed twins (censuses 5,198+5,566 and 579+649 comparisons, all equal; post-flip corpus audits 9,089+1,193 / 9,707+1,311 all equal); the MidBody producer depth/pcdep reads flip to their predecessor twins.

W1 — jitcode-keyed pivot table + resolver rewire (1 commit): py_floor_by_jit_pc — a piecewise-constant boundary table (jit start → containing Python pc) built at metadata finalize — replaces the runtime inversion implementation of python_pc_for_jitcode_pc, flipping all ~45 call sites at once. Certified by an exhaustive build-time sweep under PYRE_PCMAP_PIVOT_AUDIT: 684 JitCodes / 859,032 jitcode bytes, pivot == legacy at every byte, both backends.

W2 — floor-only refactor + direct-consumer flips (3 commits): the pivot is reduced to floor boundaries (block-head precedence stays in block_head_py_by_jit_pc); vstack_containing_py_pc and the two MidBody anchors flip to floor-segment equivalents (0 divergence); a report-only census instruments the two guard-snapshot entry-anchor legs.

W3a — entry-anchor completion + residue censuses (3 commits): the guard-arm entry anchor flips to the floor-segment start (census 435/612 fires, 0 divergence); n_py_instrs replaces the four length-only table reads; the fallthrough leg gets an inverse-query candidate census whose miss set provably coincides with the legacy-None set (26/26 per backend, count arithmetic), plus fire counters on every remaining fallback arm (all zero except the 26-fire py-value entry arms).

W3b — runtime inversion-table deletion (2 commits, net −452 lines): the fallthrough entry anchor flips to the inverse query; PyJitCodeMetadata.first_jit_pc_by_py_pc and carryfwd_resume_pc are deleted along with python_pc_for_jitcode_pc_legacy, pc_map_marker_for, the legacy anchor helpers, and the audit oracles. The codewriter keeps a build-time local first-op array to construct the pivot and twins. The runtime now holds no Python-pc→jitcode inversion table and no runtime inversion search; Python coordinates resolve via the jitcode-keyed py_floor_by_jit_pc / block_head_py_by_jit_pc tables — the same shape as PyPy baking Python pcs into jitcode as constants.

Verification for S7/W1–W3b: same per-slice battery (both backends audit-off + PYRE_PCMAP_PIVOT_AUDIT=1 audit-on, corpus 208–210/210 green throughout, MidBody workloads, cargo test with the known standalone-flake rerun).

🤖 Generated with Claude Code

Summary by CodeRabbit

Bug Fixes
- Improved JIT execution reliability when resuming code after nested calls, exceptions, and inlined operations.
- Strengthened handling of loop iteration state and restart points, helping prevent incorrect execution during complex control flow.
- Corrected constant-loading dispatch behavior in traced code.
- Improved stack, liveness, and value restoration during JIT transitions.
Diagnostics
- Added runtime auditing and validation for code-position mappings to make difficult JIT issues easier to detect and investigate.

coderabbitai · 2026-07-16T15:24:48Z

Walkthrough

The PR refactors JIT dispatch and resume handling around native JIT-code coordinates, introduces floor-segment PC metadata, updates FOR_ITER and inline-abort tracking, and expands PC-map audits and tests.

Changes

JIT coordinate and resume refactor

Layer / File(s)	Summary
PC-map metadata and floor segments `pyre/pyre-jit-trace/src/pyjitcode.rs`, `pyre/pyre-jit/src/jit/codewriter.rs`, `pyre/pyre-jit-trace/src/state.rs`	Metadata now stores `n_py_instrs` and `py_floor_by_jit_pc`; code generation builds these tables and liveness helpers use the revised shape.
Entry and snapshot coordinate propagation `pyre/pyre-jit-trace/src/jitcode_dispatch.rs`	Entry PCs, outer active-box metadata, inline parent frames, guard reconciliation, and diagnostics preserve JIT-code coordinates until Python-PC resolution.
FOR_ITER native-body tracking `pyre/pyre-jit-trace/src/jitcode_dispatch.rs`	FOR_ITER capture, attempt, matching, and delivery use `InflightForiterBody` and resolver-based native body identities.
Inline abort coordinate plumbing `pyre/pyre-jit-trace/src/jitcode_dispatch.rs`	Abort carriers, outer-resume latches, structural anchors, and midbody payloads now carry JIT-code indices and PCs.
Abort flush and walk-end commits `pyre/pyre-jit-trace/src/trace.rs`, `pyre/pyre-jit-trace/src/trace_opcode.rs`, `pyre/pyre-jit-trace/src/jitcode_dispatch.rs`	Trace flushing resolves native carriers into Python flush coordinates, updates walk-end state, and refreshes fixtures and audit-related tests.

Estimated code review effort: 5 (Critical) | ~120 minutes

Possibly related issues

youknowone/pyre issue 368 — Concerns the same PC-coordinate and resume-handling area, though this PR retains the broader pc_map APIs.
youknowone/pyre issue 369 — Directly overlaps the propagation of native JIT-code PCs through resume, liveness, and abort paths.

Possibly related PRs

youknowone/pyre#378 — Updates the same PyJitCodeMetadata fields and related test fixtures.
youknowone/pyre#471 — Directly relates to inline-abort resume latching and outer-call flushing.
youknowone/pyre#569 — Refactors the same JIT-code coordinate and resume-mapping layer.

Suggested reviewers: lifthrasiir

Poem

A rabbit hops through JIT-code lanes,
Carrying PCs where memory reigns.
Floor maps bloom, resumes align,
FOR_ITER follows a native sign.
Abort paths flush with careful cheer—
“The coordinate trail is clear!” 🐇

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly matches the main change: retiring Python-PC channels and the runtime py→jit inversion table.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch rewrite-tracer

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ast-grep (0.44.1)

pyre/pyre-jit-trace/src/jitcode_dispatch.rs

ast-grep timed out on this file

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

github-actions · 2026-07-16T15:39:51Z

🤖 Codex parity review

Static analysis of this diff vs the local RPython/PyPy sources (commit ca843b7).
Updated: 2026-07-17T08:47:54.387Z

Files in the reviewed diff

pyre/pyre-jit-trace/src/jitcode_dispatch.rs
pyre/pyre-jit-trace/src/pyjitcode.rs
pyre/pyre-jit-trace/src/state.rs
pyre/pyre-jit-trace/src/trace.rs
pyre/pyre-jit-trace/src/trace_opcode.rs
pyre/pyre-jit/src/jit/codewriter.rs

1. Regressions to PyPy parity introduced by this patch

pyre/pyre-jit-trace/src/jitcode_dispatch.rs:801 ↔ rpython/jit/metainterp/pyjitpl.py:737 — “Every root entry currently passes None… None disables the field-based specialization gate.” Removing the production arm entry drops the STORE_SUBSCR specialization route, whereas PyPy directly records and heap-caches array stores through execute_setarrayitem_gc.

2. Other mismatches introduced by this patch

None.