Three bugs measured on an AMD APU (26 GiB RAM) running LLaDA2.2-flash Q4_K_S: print_usage segfault, no mmap path, compute-buffer estimate OOMs a 22 GiB box

[com-junkawasaki]

We ran LLaDA2.2-flash (103B MoE, the Akicou GGUF Q4_K_S = 59,086,787,040 B) on two small AMD boxes — Ryzen 7 7735HS (16 threads, 26 GiB) and Ryzen 5 6600H (12 threads, 22 GiB) — through your LLaDA2.2 MoE support. It runs end-to-end and produces correct text after three local patches. Report in case upstream wants them; all three were measured, not inferred.

1. print_usage segfaults (undefined behavior, one-line fix)

tools/main-cli.cpp: print_usage(prog) calls fprintf(stderr, "…Usage: %s [options]…") with the %s format but no corresponding argument. Any invocation prints usage and segfaults:

• before fix: ./diffuse-cli --help → SIGSEGV (RC 139) on both boxes
• after fix (pass prog): RC 0, usage prints correctly

Reproduce: build either native or -DGGML_NATIVE=OFF, run --help.

2. No mmap path — the 59 GB GGUF cannot load on a 26 GiB box

diffuse_model_load_impl uses gguf_init_params{ no_alloc = false }, so every tensor is read into a heap buffer: 59.1 GB of allocations before the first token. Nothing at or under 32 GiB RAM can load the model, even though only ~13B params are active per token.

Local patch (kept on the box, not sent as a diff yet): no_alloc = true + mmap() the file + point each tensor at base + data_offset + tensor_offset (446/446 tensors wired), with madvise selectable at run time (DIFFUSE_MMAP_MADV=random|willneed|none).

Result: the engine runs with max RSS ~23.5 GiB while mapping a 55.03 GiB file, reading ~4.3 GB per forward (2.2–2.6M major faults per forward).

3. Compute-buffer estimate is ~5.5x the real working set → OOM on 22 GiB

diffuse_compute_buf_size() multiplies per-layer scratch by n_layer using the dense feed_forward_length (9216) for every layer. But LLaDA2.2-flash has first_k_dense_replace = 1: only layer 0 is dense, layers 1–31 route through 1024-wide experts (expert_feed_forward_length).

Measured on the 22 GiB box (before the fix): the engine asked for 16,275 MB and was OOM-killed mid-generation (dmesg: oom-kill … task=diffuse-cli total-vm 70 GB, anon-rss 7.8 GB).

Fix: size MoE layers by model->moe_intermediate (1024) instead of hp.n_ff. This dropped the request to 11,967 MB and the 26 GiB box completes generation. The 22 GiB box still OOMs (swap-full) — its available memory is ~7.5 GiB — so we also documented that the minimum practical box for this checkpoint is ~26 GiB with the mmap path, and that steps acts as a budget, not a guarantee (n_steps=8 actually runs 10–11).

Measurements with the patches (26 GiB box, 16 threads, temp 0)

config	tok/s	tokens	wall	peak RSS
n=8, steps=8, thr=0.95	0.032	8	249 s	23.4 GiB
n=16, steps=16	0.018	16	886 s	23.6 GiB
n=32, steps=32, madv=none	0.026	32	1209 s	23.6 GiB
n=32, steps=8, thr=0.7	0.088	32	364 s	23.7 GiB

(threshold 0.7 = 3–4.5× faster but visibly repetitive output; decode is entirely NVMe-bandwidth-bound at ~4.3 GB/forward.)

Full ledger and reproduction steps: https://github.com/com-junkawasaki/root/blob/main/90-docs/adr/2609131058-dllm-vs-ar-cpu-boxes-llada22-measured.edn

Happy to turn any of the three into a PR if you tell me which shape you want (1 is trivial; 2 and 3 touch the loader/scheduler and I'd rather match your intended design for n_gpu_layers handling).

---

This repo is using Opire - what does it mean? 👇

💵 Everyone can add rewards for this issue commenting /reward 100 (replace 100 with the amount).
🕵️‍♂️ If someone starts working on this issue to earn the rewards, they can comment /try to let everyone know!
🙌 And when they open the PR, they can comment /claim #6 either in the PR description or in a PR's comment.

🪙 Also, everyone can tip any user commenting /tip 20 @com-junkawasaki (replace 20 with the amount, and @com-junkawasaki with the user to tip).

📖 If you want to learn more, check out our documentation.