Skip to content

[Metal] gather_qmm improvement - #4572

Merged
nastya236 merged 3 commits into
mainfrom
gather-qmm-tiling
Sep 28, 2026
Merged

nastya236 merged 3 commits into
mainfrom
gather-qmm-tiling

Conversation

@nastya236

Copy link
Copy Markdown
Collaborator

Same as in #4567

Given #4567 I asked Claude code to do the same for gather_qmm.

qwen3.5-397b-a17b:

Format N Before (ms) After (ms) Speedup
affine 512 8.08497 6.24240 1.30×
affine 713 8.75225 6.61468 1.32×
affine 1024 9.76828 7.38171 1.32×
affine 2048 13.55017 11.41791 1.19×
affine 4123 21.36925 16.61189 1.29×
affine 8192 36.69587 28.45843 1.29×
affine 16384 65.42287 53.63210 1.22×
nvfp4 512 8.23671 6.29423 1.31×
nvfp4 713 8.81961 6.66266 1.32×
nvfp4 1024 9.88744 7.52525 1.31×
nvfp4 2048 13.53901 11.54110 1.17×
nvfp4 4123 21.34495 16.66406 1.28×
nvfp4 8192 36.87975 28.73714 1.28×
nvfp4 16384 65.41614 54.00811 1.21×
mxfp8 512 12.06362 9.16402 1.32×
mxfp8 713 12.67015 9.46002 1.34×
mxfp8 1024 13.90283 10.22280 1.36×
mxfp8 2048 19.21845 14.19822 1.35×
mxfp8 4123 28.60970 19.87559 1.44×
mxfp8 8192 47.65744 33.30466 1.43×
mxfp8 16384 80.14714 60.24049 1.33×

@zcbenz zcbenz left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants