Regarding the Hadamard transform accuracy drop for nvfp4 with RTN in the paper

[TheTinyTeddy]

Many thanks for the great work!

I was wondering regarding the Hadamard transform accuracy drop for nvfp4 with RTN in the paper, is it because of the block size of hadamard is too small (16 vs 32 in INT4 and MXFP4) such that the outlier smoothing characteristic of the HT has been weakened? Also, could randomness (i.e. RHT) help with the result?

Furthermore, would you consider decrease the Quantization block size/Hadamard transformaton size from 32 to 16 for INT4 as a additional option to verify it?

Kind regards

[Godofnothing]

Hi, @TheTinyTeddy. Thank you for the interest in your work.

Concerning the accuracy drop for Hadamard transform applied to RTN for NVFP as discussed in the paper - it is because of better loss asymptotic of more heavy-tailed Laplace distribution for small group sizes compared to Gaussian. Hadamard transform turns Laplace-like distribution into Gaussian. Also note, that FP grid is non-uniform with more precision around zero and the Hadamard smearing effect is detrimental.

We have explored Hadamards for integer quantization as well and it helps consistent with prior work.