Repository navigation
Conversation
Lanes with x or y in {0, inf, nan} were fixed one at a time by a scalar
copy of powf's special-case code, after the vector core had run on all
four lanes, and an exact +0 also sent the vector through
v_powf_x_is_neg_or_small. Exact zeros are common in clamped image data.
Replace the scalar fallback with a vector version of the same decision
tree, and skip the core when every lane is special. When the only
special lanes have non-negative x and finite nonzero y, and no other
lane has negative or subnormal x, a new helper, v_powf_xspecial, fixes
them inline. It computes 1 / (x * x) for negative y as an exponent-field
flip, and special_case uses it too, so pow (+-0, y < 0) no longer raises
FE_DIVBYZERO. Vector math does not guarantee exception flags. Special
lanes no longer count as negative or subnormal, and the special-lane and
x < 0x1p-126 tests share one branch and one vector-to-general transfer.
Neoverse N1 (built with GCC 16.1 at -O3), ns per element, y = 0.45:
- clean inputs: 5.84 -> 5.77 (1.01x)
- 25% negative lanes: 7.04 -> 7.23 (0.97x)
- 50% zero lanes: 8.10 -> 6.94 (1.17x)
- all zeros: 8.97 -> 1.52 (5.90x; scalar powf: 3.02)
- all -0: 9.63 -> 2.55 (3.78x; scalar powf: 3.35)
- one negative and one -0 lane per vector: 8.04 -> 8.77 (0.92x), as
special_case now runs its odd-integer test in vector code where the
old code used scalar code for the -0 lane.
There was a problem hiding this comment.
Thank you for your contribution to Arm's optimized routines. Contribution to this repository does require a signed contributor's license agreement. See contributor-agreement.pdf for details
|
Thank you very much for this PR. This showcases some very good attempts at improving performance of special cases, that I think we should try to integrate, but I have additional concerns. First, it looks like the current change is overly focused on cases that are considered rare in the design of AOR. Second, it is important that we leave the more general case of "clean" inputs as undisturbed as possible, as well as the common negative and subnormal path. At present I've also measured regression just above the noise level (~2%) on the faster paths (including positive, negative and subnormals), which could be acceptable, but also some more significant ones for negative or subnormals with inf/nan lanes (6%) and negative y (17%). What these number suggest is that it is hard to optimise for one set of special cases without damaging performance of other cases. And it is not obvious which case is most common in general. Finally, a blocker IMO, but I'm open to discussion, is that the increased code complexity and size is too big for the overall benefit. There may be a way to share some of this logic with powr, which could mitigate this cost though. There have been attempts in the past to optimize special cases but never successfully attained a satisfactory balance in overall performance (and in code complexity). The best compromise so far was to keep the simple scalar fallback, but I'm open to a vectorised solutions if it performs better overall (and some cases regress within reason). |
|
@ptomsich Is this optimisation intended to improve performance on a particular benchmark? If so which benchmark and how much gain do you observe? |
ocio in SPEC2026:
|
This has been signed and filed. |
Lanes with x or y in {0, inf, nan} were fixed one at a time by a scalar copy of powf's special-case code, after the vector core had run on all four lanes, and an exact +0 also sent the vector through v_powf_x_is_neg_or_small. Exact zeros are common in clamped image data.
Replace the scalar fallback with a vector version of the same decision tree, and skip the core when every lane is special. When the only special lanes have non-negative x and finite nonzero y, and no other lane has negative or subnormal x, a new helper, v_powf_xspecial, fixes them inline. It computes 1 / (x * x) for negative y as an exponent-field flip, and special_case uses it too, so pow (+-0, y < 0) no longer raises FE_DIVBYZERO. Vector math does not guarantee exception flags. Special lanes no longer count as negative or subnormal, and the special-lane and x < 0x1p-126 tests share one branch and one vector-to-general transfer.
Neoverse N1 (built with GCC 16.1 at -O3), ns per element, y = 0.45: