Skip to content

Xcode 26.5-built MLX-C runtime returns incorrect BF16 gather/RMSNorm/matmul result #115

Description

@pd95

While porting Apertus to the MLX-based Ollama runner, we observed corrupted output after rebuilding the runtime with Xcode 26.5.
I reduced this to a standalone MLX-C repro with constant tensors and no model/application code.

The same repro passes when MLX-C/MLX is built with Xcode 26.4, but fails when built with Xcode 26.5.

Result Matrix

MLX-C pinned ref fba4470 + Xcode 26.5: FAIL, first=512
MLX-C main + Xcode 26.5:               FAIL, first=512
MLX-C pinned ref fba4470 + Xcode 26.4: PASS, first=1024

Full pinned MLX-C ref:

fba4470b89073180056c9ea46c443051375f7399

The included run used CMAKE_OSX_DEPLOYMENT_TARGET=26.0.

Repro Operation

The repro computes:

BF16 take_axis/gather -> fast_rms_norm -> BF16 matmul

with constant tensors:

embedding:      [131072, 4096] BF16, all 0.5
RMSNorm weight: [4096] BF16, all 1.0
q_proj weight:  [4096, 4096] BF16, all 0.25

Expected:

4096 * (0.5 / sqrt(0.5^2 + 1e-5)) * 0.25 ~= 1023.9795

Observed with Xcode 26.5-built runtime:

first=512.00000000
status=FAIL

Observed with Xcode 26.4-built runtime:

first=1024.00000000
status=PASS

The 512 result is equivalent to the matmul consuming the unnormalized 0.5 input rather than the RMSNorm output.

Toolchain Details

From summary.log:

Xcode 26.4 SDK: 26.4
Metal Toolchain: com.apple.MobileAsset.MetalToolchain-v17.5.188.0.mGsQWM
Result: PASS

Xcode 26.5 SDK: 26.5
Metal Toolchain: com.apple.MobileAsset.MetalToolchain-v17.6.42.0.1BIDXR
Result: FAIL

Both toolchains report:

Apple metal version 32023.883 (metalfe-32023.883)
Target: air64-apple-darwin25.5.0

Reproduction

The attached archive contains:

mlx-c-bug/run.sh
mlx-c-bug/repro.c
mlx-c-bug/README.md
mlx-c-bug/results/20260514-193925/summary.log
per-case repro logs, commit refs, runtime file lists, and checksums

Run:

./mlx-c-bug/run.sh --clean-work

The script clones ml-explore/mlx-c, builds MLX-C/MLX directly with the selected Xcode versions, collects the produced runtime files, compiles repro.c against libmlxc.dylib, and runs the constant tensor check.

Question

Is this expected to be an Xcode/Metal Toolchain regression, or is there a build flag/configuration needed for MLX-C with Xcode 26.5 and deployment target 26.0?

mlx-c-bug-repro-20260514.zip

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions