Fix nonfinite ARM FP16 accumulation in shipped CPU variants #174

Scope and classification

P2 confirmed numerical defect in the published v0.8.0-3 macOS ARM64 library and both optimized Linux ARM64 CPU modules. Finite synthetic inputs produce infinity while the baseline and controls remain finite. A focused owner repair is being prepared. Public-model/app impact and whether this is a newly introduced regression remain unmeasured; release scope must keep those limits explicit.

Upstream#31734 reports a public model-free F16-row/F32-column ggml_mul_mat test on native-built M4Max/raw13ec8f614: the F32 result becomes infinity while the double reference remains finite. The official generic Mac binary does not reproduce the reported model behavior. Private model weights are neither available nor required for the synthetic check. CPU-only F16 attention/cache behavior must be distinguished from Metal, SVE/F32 paths and ordinary lower-magnitude inputs.

Exact layers and reachability evidence

• Release candidate core PR #1746, head d14513aea015f747b889a81450d21204b5c801b1, pins native v0.8.0-3, owner 703c0f5 / upstream 8642541b255bfb5dc39cd351a178be09013a55d6. No consumer pin changed during this investigation.
• At that pin, simd-mappings.h uses FP16 vector accumulators/vfmaq_f16 and FP16 reduction; vec.cpp uses the macro path without SVE.
• The current Linux ARM variant contract ships armv8.2-a+dotprod+fp16 and armv8.6-a+dotprod+fp16+i8mm without SVE; Android has the corresponding android_armv8.2_2 path.
• Historical published-module disassembly found296 FP16 FMLA instructions in each inspectedv0.7.0-3 optimized Linux module versus zero in baseline. That is reachability/history evidence, not a freshv0.8.0-3 binary reproduction.

Closed #126 measured performance and introduced variants; open #137 concerns throughput selection. Neither owns finite/extreme-value correctness across actual selected packaged variants. #112 retains broader raw-upgrade qualification.

Published-artifact reproduction — November 7

All 11 model-free F16-row/F32-column graph cases completed with compute status 0 and one thread. At magnitudes 80/90/100, affected modules returned infinity against finite references 592448/749817/925700. The other eight cases, including low-value, cancellation and F32 controls, passed.

The Mac archive is 632946dbe37964c1172b753ddfed7754d5f7398c7078f73f57f464e759d2b586 (5652934 bytes); the Linux archive is 069ddd5b63b2b17601425700c9a03d50ef9a171ea728080e79d7be39b9cd1dc8 (13869003 bytes). Both were rehashed against the verified public release. Linux used the same base library, 1f294a3aa55d302741862f35fd28107fc2554a9a60ead0748619e248a00a1bdd, with every bound symbol checked against the explicitly selected module in a fresh offline local ARM64 container. All owned containers were verified absent afterward.

The pinned CPU attention implementation reaches this dot product with default F16 KV caches, including small-query flash attention. Quantized weight storage does not establish immunity. The reported extreme inputs remain synthetic: no public model, app, Android, iOS, Windows ARM64 or SVE execution is claimed here.

Bounded acceptance

• Use exact published module hashes plus actual selected-provider/ISA evidence; run the public synthetic operation and low-magnitude/mixed-sign controls against baseline, optimized non-SVE and available SVE/F32 paths. Compare finite output to F32/double reference; do not infer all ARM or all quantized models are affected.
• Cover affected attention/F16-K-cache and flash-attention on/off paths using a suitable existing public model only if the model-free check establishes exposure. Record unavailable host/variant cases explicitly.
• If reproduced, qualify a bounded F32-accumulation fix or typed safe restriction with causal failure/restoration tests and original upstream linkage. A source-only or unmerged proposed fix is not delivered.
• Preserve ARM baseline compatibility, runtime selection, FP16/FHM/SVE containment and representative performance. Do not raise the minimum ISA or globally disable optimized variants without evidence.
• Publish qualified corrected bundles/export/manifests/Apple checks if required, then coordinate consumer pin/docs/validation. Existing public/main safety claims remain unchanged until exact artifact evidence.

The latest phase executed only local model-free Mac/Linux probes. It changed no runtime pin, published no package and used no paid cloud/device job. The owner repair and durable regression tests are being prepared separately; source readiness, corrected artifact qualification and consumer adoption remain separate gates.