codegen: give the oversized-unit per-function arm its own cap (measured: do not retune it)

changelog.d/8144-oversized-unit-per-function-cap.md

@@ -0,0 +1,94 @@
+**#8132 follow-up — measured NEGATIVE result: the oversized-unit opt-level
+threshold is not retuned, and here is the evidence for why not.**
+
+`oversized_opt_flag` routes a codegen unit past `PERRY_LL_O0_THRESHOLD_BYTES`
+(6 MiB of IR) to `-Os` unless it looks like the `#4880` "one giant generated
+function" pathology, in which case only `-O0` finishes in practical time. It has
+two arms: an average bytes-per-function cap (256 KB) and a per-function arm for
+the unit's widest body. The per-function arm did not have a threshold of its
+own — it **borrowed `ll_o0_threshold_bytes()`**, a *whole-unit* constant
+answering a different question. `[email protected]`'s bundled `jsonwebtoken` lowers to
+a unit whose widest body is 3.5 MB, misses that 6 MiB cutoff, takes `-Os`, and
+falls into the RS4GC blow-up #8132 describes.
+
+The obvious fix is to lower the cap. Measured, it is a bad trade.
+
+### There is no separating band
+
+Compiling the 17 largest `[email protected]` `dist/compiled/**` bundles produced **52
+oversized codegen units**. Their widest function runs **751 KB to 11.1 MB,
+continuously** — deciles 0.73 / 1.3 / 1.6 / 1.9 / 2.1 / 2.2 / 2.7 / 3.4 / 3.8 /
+5.4 MB. There is no gap between "large bundle of ordinary functions" and
+"monolith", because in real webpack output *every* oversized unit is
+monolith-bearing: the bundler emits each chunk's module factory as one function.
+Every one of the 52 averages 19-67 KB/fn, i.e. all 52 clear the 256 KB average
+cap, so the per-function arm is the only arm that ever fires on this shape.
+
+| per-function cap | units routed to `-O0` |
+|---|---|
+| 256 KB (the average cap) | 52 / 52 (100%) |
+| 1 MiB | 50 / 52 (96%) |
+| 1.5 MB (needed to catch both #8132 units) | 44 / 52 (85%) |
+| 2 MiB | 33 / 52 (64%) |
+| 4 MiB | 10 / 52 (19%) |
+| **6 MiB (today, unchanged)** | **3 / 52 (6%)** |
+
+### The price of reclassifying, on the #8132 bundle itself
+
+One compiler, one fixture (`dist/compiled/jsonwebtoken/index.js`, sha256
+`056c2ddd…a6b9`), arms selected only by the new env var, `PERRY_RS4GC=0`
+(see below for why):
+
+| arm | `__text` | dylib total | clang CPU | wall |
+|---|---|---|---|---|
+| both units `-Os` (today) | 2,798,984 | 4,272,080 | 43.76 s | 48.6 s |
+| widest arm at 2 MiB (one unit each way) | 4,294,952 | 5,775,232 | 29.61 s | 35.7 s |
+| both units `-O0` | 5,452,344 | 6,931,088 | 16.49 s | 25.3 s |
+
+**`-Os` → `-O0` is +94.8% `__text` and +62.2% total, to buy −62.3% clang CPU.**
+That confirms the "30-50% less `__text`" figure the existing comment claims, and
+it is the cost that would land on 85-96% of oversized units under any cap low
+enough to help #8132. Dropping a unit to `-O0` takes its ~400 ordinary functions
+down with the one monolith; that is the whole trade.
+
+### And it would not fix #8132 anyway
+
+On `origin/main` (`0a1e78e5f`, `perry-dev` profile, LLVM 22.1.4, arm64 macOS)
+**neither arm produces a binary for that bundle with statepoints enabled.** Both
+die with `SIGBUS` at the same site — `AArch64TargetLowering::LowerCall` under
+`SelectionDAGBuilder::LowerAsSTATEPOINT` — reproduced 4/4: `-Os` at 292 s,
+`-O0` at 120-208 s. It is not a stack overflow; the faulting address lands in
+reserved, never-mapped space, and raising the codegen worker stack from the
+default 2 MiB to 1 GiB changed nothing. The text backend fails differently
+(`clang: unterminated attribute group` on the 414 MB rendered unit). Only
+`PERRY_RS4GC=0` compiles it, in 20 s. So issue #8132's recorded `-O0` result
+(2m46.5s, 40 MB dylib) does not reproduce on that commit, and routing the unit
+to `-O0` swaps a hang for a crash rather than unblocking #8040.
+
+### What this change does land
+
+Nothing that alters a default:
+
+- `DEFAULT_LL_O0_MAX_FN_BYTES` (6 MiB) gives the per-function arm its own
+  constant instead of borrowing the whole-unit one, so raising
+  `PERRY_LL_O0_THRESHOLD_BYTES` — a knob about unit *size* — stops silently
+  moving the monolith cap too. Same value, same behaviour today.
+- `PERRY_LL_O0_MAX_FN_BYTES` overrides it, keyed into the object cache
+  alongside the other codegen env vars (#6394), so a project that *wants* the
+  trade can take it (`PERRY_LL_O0_MAX_FN_BYTES=1048576`) without a rebuild and
+  without imposing it on everyone.
+- The oversized-unit diagnostic now prints the **widest** function next to the
+  average. The average is precisely the statistic that cannot explain the
+  routing decision, and without the widest figure the corpus measurement above
+  is not possible.
+
+Two tests bracket the cutoff with absolute sizes, and are verified discriminating
+by sabotage: deleting the arm and raising the cap to 16 MiB each turn the
+monolith test red; lowering the cap to 256 KB turns the `-Os` test red. An
+earlier version that derived its fixtures from `ll_o0_max_fn_bytes()` stayed
+green under the 256 KB sabotage — vacuous, and caught by running the sabotage
+rather than reasoning about it.
+
+Files: `crates/perry-codegen/src/linker.rs`,
+`crates/perry-codegen/src/linker_tests.rs`,
+`crates/perry/src/commands/compile/object_cache.rs` (+ its tests).

crates/perry-codegen/src/linker.rs

@@ -350,6 +350,47 @@ fn ll_size_opt_max_fn_bytes() -> usize {
         .unwrap_or(DEFAULT_LL_SIZE_OPT_MAX_FN_BYTES)
 }
 
+/// For an oversized unit, the size of its **widest single function** above
+/// which we drop the whole unit to `-O0`.
+///
+/// This is the per-function counterpart of the average cap above, and it is the
+/// arm that can actually see the `#4880` shape. An average cannot: a bundle is
+/// hundreds of ordinary functions plus one generated monolith, and the ordinary
+/// ones dilute the monolith out of the mean. `[email protected]`'s bundled
+/// `jsonwebtoken` is exactly that — 468 functions averaging 28 KB with one at
+/// 3.5 MB — so its average sits nine times UNDER the average cap while one
+/// body is a monolith (#8132).
+///
+/// **The default is deliberately unchanged at 6 MiB, and lowering it is a
+/// measured bad trade.** The arm previously borrowed `ll_o0_threshold_bytes()`,
+/// which is a *whole-unit* constant answering a different question; it now has
+/// its own name and its own override so the two stop moving together. What it
+/// is NOT is a retune, because the corpus does not support one. Over the 52
+/// oversized codegen units Perry produces from `[email protected]`'s 17 largest
+/// `dist/compiled/**` bundles, the widest function runs from 751 KB to 11.1 MB
+/// *continuously* (deciles 0.73/1.3/1.6/1.9/2.1/2.2/2.7/3.4/3.8/5.4 MB) — there
+/// is no empty band between "ordinary bundle" and "monolith", because in real
+/// webpack output every oversized unit is monolith-bearing. (Their averages run
+/// 19-67 KB/fn, so all 52 clear the average cap and this arm is the only one
+/// that ever fires.) A cap low enough to catch #8132's two units (3.5 MB and
+/// 1.5 MB) is ≤ 1.5 MB, which reclassifies 44 of 52; 1 MiB reclassifies 50 of
+/// 52. Measured on that same bundle, reclassifying costs **+95% `__text`
+/// (2.80 MB → 5.45 MB) to buy −62% clang CPU (43.8 s → 16.5 s)**. Whether that
+/// trade is worth making is a product call; `PERRY_LL_O0_MAX_FN_BYTES` makes it
+/// one env var instead of a rebuild, and the default declines to make it on
+/// everybody's behalf.
+///
+/// `0` disables this arm entirely (leaving only the average cap);
+/// `PERRY_LL_SIZE_OPT` still overrides both.
+const DEFAULT_LL_O0_MAX_FN_BYTES: usize = 6 * 1024 * 1024;
+
+fn ll_o0_max_fn_bytes() -> usize {
+    std::env::var("PERRY_LL_O0_MAX_FN_BYTES")
+        .ok()
+        .and_then(|s| s.trim().parse::<usize>().ok())
+        .unwrap_or(DEFAULT_LL_O0_MAX_FN_BYTES)
+}
+
 /// Decide the clang opt flag for an oversized unit: `-Os` when the unit is many
 /// ordinary functions (size-optimize, big `__text` win), `-O0` when it is a
 /// pathological few-giant-function monolith (`#4880`). `ll_fn_count` is the
@@ -368,7 +409,8 @@ fn oversized_opt_flag(
     // hundreds of small functions dilute one pathological generated function's
     // average: that sent a 20+ MiB body through -Os in the Claude bundle and
     // spent minutes in LLVM where the same body finishes in seconds at -O0.
-    if max_fn_bytes.is_some_and(|bytes| bytes > ll_o0_threshold_bytes()) {
+    let max_fn_cap = ll_o0_max_fn_bytes();
+    if max_fn_cap > 0 && max_fn_bytes.is_some_and(|bytes| bytes > max_fn_cap) {
         return "-O0";
     }
     let avg_fn_bytes = ll_byte_size / ll_fn_count.max(1);
@@ -414,15 +456,25 @@ fn build_clang_compile_plan(
     let o0_threshold = ll_o0_threshold_bytes();
     let opt_flag = if o0_threshold > 0 && ll_byte_size > o0_threshold {
         let flag = oversized_opt_flag(ll_byte_size, ll_fn_count, max_fn_bytes);
+        // `widest` is the quantity the -O0 arm actually decides on, so print
+        // it: an average alone cannot explain why a unit was routed, and a
+        // corpus sweep needs the per-unit distribution to retune the cap.
+        // `?` when the caller could not supply it (the textual clang path,
+        // which has no per-function sizes).
         eprintln!(
             "perry: module IR is {:.1} MB (> {:.1} MB), {} functions \
-             (~{:.0} KB/fn); compiling at {} instead of -O3 so LLVM's -O1+ \
-             pipeline doesn't blow up on oversized functions (#4880). Override \
-             with PERRY_LL_O0_THRESHOLD_BYTES / PERRY_LL_SIZE_OPT.",
+             (~{:.0} KB/fn avg, widest {}); compiling at {} instead of -O3 so \
+             LLVM's -O1+ pipeline doesn't blow up on oversized functions \
+             (#4880). Override with PERRY_LL_O0_THRESHOLD_BYTES / \
+             PERRY_LL_O0_MAX_FN_BYTES / PERRY_LL_SIZE_OPT.",
             ll_byte_size as f64 / (1024.0 * 1024.0),
             o0_threshold as f64 / (1024.0 * 1024.0),
             ll_fn_count,
             (ll_byte_size as f64 / ll_fn_count.max(1) as f64) / 1024.0,
+            match max_fn_bytes {
+                Some(bytes) => format!("{:.0} KB", bytes as f64 / 1024.0),
+                None => "?".to_string(),
+            },
             flag,
         );
         flag

crates/perry-codegen/src/linker_tests.rs

@@ -232,6 +232,67 @@ fn compile_plan_keeps_o0_for_oversized_giant_function_monolith() {
     assert!(!plan.clang_args.contains(&"-Os".to_string()));
 }
 
+/// A unit with hundreds of ordinary functions and ONE outsized body: 19 MB of
+/// IR over 400 functions is 48 KB/fn, five times UNDER the 256 KB AVERAGE cap,
+/// so the average arm always says `-Os` here and only the per-function arm can
+/// decide. Shape taken from `[email protected]`'s bundled `jsonwebtoken` (#8132).
+///
+/// Sizes are absolute on purpose. Deriving them from `ll_o0_max_fn_bytes()`
+/// makes the fixtures move with the constant, and then lowering the cap keeps
+/// both tests green — measured, not assumed: that version survived a sabotage
+/// run at a 256 KB cap.
+fn plan_for_unit_with_widest_function(widest_bytes: usize) -> ClangCompilePlan {
+    build_clang_compile_plan(
+        PathBuf::from("clang"),
+        PathBuf::from("/tmp/input.ll"),
+        PathBuf::from("/tmp/output.o"),
+        None,
+        19 * 1024 * 1024,
+        400,
+        Some(widest_bytes),
+        false,
+        false,
+    )
+}
+
+#[test]
+fn compile_plan_drops_to_o0_when_one_function_is_a_monolith() {
+    // 8 MB in one body is past any reading of "ordinary", and the unit's
+    // average cannot see it. Deleting the per-function arm, or raising its cap
+    // above 8 MB, turns this red.
+    let plan = plan_for_unit_with_widest_function(8 * 1024 * 1024);
+    assert!(
+        plan.clang_args.contains(&"-O0".to_string()),
+        "a unit carrying an 8 MB function must compile at -O0 despite a \
+         48 KB/fn average; got {:?}",
+        plan.clang_args
+    );
+    assert!(!plan.clang_args.contains(&"-Os".to_string()));
+    assert!(!plan.clang_args.contains(&"-O3".to_string()));
+}
+
+#[test]
+fn compile_plan_keeps_os_for_the_8132_bundle_shape() {
+    // 3.5 MB is the widest body Perry lowers `[email protected]`'s bundled
+    // `jsonwebtoken` to (#8132), and this pins the deliberate decision NOT to
+    // catch it. Dropping that unit to -O0 takes its 400 ordinary functions
+    // with it: measured on that bundle, +95% __text (2.80 MB -> 5.45 MB) to
+    // buy -62% clang CPU (43.8 s -> 16.5 s). Nothing in the corpus separates
+    // "monolith" from "ordinary bundle": the 52 oversized units measured over
+    // [email protected]'s bundles run continuously from 751 KB to 11.1 MB, so a cap
+    // low enough to catch this one (<= 1.5 MB) reclassifies 44 of those 52.
+    // Lowering DEFAULT_LL_O0_MAX_FN_BYTES turns this red, which is the point.
+    let plan = plan_for_unit_with_widest_function(3557 * 1024);
+    assert!(
+        plan.clang_args.contains(&"-Os".to_string()),
+        "the #8132 bundle shape keeps -Os by default; lowering the cap is a \
+         measured size regression, not a free win. got {:?}",
+        plan.clang_args
+    );
+    assert!(!plan.clang_args.contains(&"-O0".to_string()));
+    assert!(!plan.clang_args.contains(&"-O3".to_string()));
+}
+
 #[test]
 fn compile_plan_skips_native_tuning_for_explicit_target() {
     let plan = build_clang_compile_plan(

crates/perry/src/commands/compile/build_cache.rs

@@ -96,6 +96,7 @@ const BUILD_CACHE_ENV_VARS: &[&str] = &[
     "PERRY_LLVM_LIB",
     "PERRY_LLVM_OPT",
     "PERRY_LL_O0_THRESHOLD_BYTES",
+    "PERRY_LL_O0_MAX_FN_BYTES",
     "PERRY_LL_SIZE_OPT",
     "PERRY_LL_SIZE_OPT_MAX_FN_BYTES",
     "PERRY_OUTLINE_METHOD_DISPATCH",

crates/perry/src/commands/compile/object_cache.rs

@@ -916,6 +916,10 @@ fn compute_object_cache_key_with_env(
             .as_deref()
             .unwrap_or(""),
     );
+    h.field(
+        "env_ll_o0_max_fn_bytes",
+        env_var("PERRY_LL_O0_MAX_FN_BYTES").as_deref().unwrap_or(""),
+    );
     h.field(
         "env_entry_symbol",
         env_var("PERRY_ENTRY_SYMBOL").as_deref().unwrap_or(""),

crates/perry/src/commands/compile/object_cache/object_cache_tests.rs

@@ -616,6 +616,7 @@ fn key_changes_with_codegen_env_vars() {
         "PERRY_LL_O0_THRESHOLD_BYTES",
         "PERRY_LL_SIZE_OPT",
         "PERRY_LL_SIZE_OPT_MAX_FN_BYTES",
+        "PERRY_LL_O0_MAX_FN_BYTES",
         "PERRY_ENTRY_SYMBOL",
         "PERRY_CODEGEN_UNITS",
         "PERRY_CODEGEN_UNIT_BYTES",