Fun, safe math considerations

[raphlinus]

This issue is a proposal for how to systematically deal with the tradeoff between precise semantics and performance. This tradeoff affects rounding behavior, especially in multiply-add combinations, and also NaN handling.

The general principle is as follows: the short, ergonomic name is a method with relaxed semantics, optimized for performance. Guarantees are explicitly weaker than, say, the Rust core language.

When applications require it, we will also add a _precise version, which is guaranteed to match Rust semantics, but may have degraded performance.

Here are more specific details:

mul_add may in general be implemented as a * b + c or as fused multiply-add. We'll see the former in x86 versions lower than x86_64-v3, and also in WASM SIMD (until relaxed_simd lands). On aarch64, I'm still researching the situation; both vmla (multiply-add) and vfma (fused multiply-add) instructions exist. The former may be higher performance in some implementations, in which case that will be the choice for mul_add, but of course the latter for mul_add_precise.

min and max may have different NaN handling. I believe this will affect mostly Intel. The _precise variants will be polyfilled by a compare and select combination on Intel, and vminnm/vmaxnm on aarch64. If needed, we can also implement minimum and maximum, which are vmin and vmax respectively on aarch64.

round will typically be implemented as round-ties-even semantics, but that's not guaranteed. If we need more precision, round_ties_even will guarantee ties-even semantics, and, if needed, round_precise will guarantee the same semantics as round, ie away from 0.0.

[raphlinus]

The Arm Cortex-A78 Core Software Optimization Guide shows vfma as strictly higher performance than vmla - 4 cycles latency as opposed to 5, with the same throughput. Neoverse V2 is the same. Based on this, I'm leaning toward using vfma in all cases, but am certainly open to more data. As another data point, unfortunately LLVM appears not to fuse vmul and vadd into a single vmla; I believe this is a valid optimization, but not 100% sure.

[awxkee]

mul_add may in general be implemented as a * b + c or as fused multiply-add. We'll see the former in x86 versions lower than x86_64-v3, and also in WASM SIMD (until relaxed_simd lands). On aarch64, I'm still researching the situation; both vmla (multiply-add) and vfma (fused multiply-add) instructions exist. The former may be higher performance in some implementations, in which case that will be the choice for mul_add, but of course the latter for mul_add_precise

Rust will call libm's fmaf when FMA is not available, instead of using a * b + c. This can severely hurt performance, since most libraries implement fmaf as a software fallback.

unfortunately LLVM appears not to fuse vmul and vadd into a single vmla; I believe this is a valid optimization, but not 100% sure.

This is not allowed without fast-math, because it changes the result. If fast-math is enabled LLVM is happy to fuse everything available, as it fuses on C++ everything from scratch.

[raphlinus]

This is not allowed without fast-math, because it changes the result. If fast-math is enabled LLVM is happy to fuse everything available, as it fuses on C++ everything from scratch.

Actually I did more experimentation, and it does the reverse, rewriting a vmla into a mul-add sequence (at least on the aarch64-apple-darwin target). I believe this transformation is valid because vmla (unlike vfma) does not fuse, it rounds the intermediate result.

[awxkee]

I believe this is Rust intrinsic implementation bug ( or an undocumented feature ), because it’s not really clear why instruction separation occurs.

[raphlinus]

It's possible it's a bug but I think the most likely explanation is that LLVM's cost model predicts that the mul-add sequence will have lower or equal cost to the vmla. In any case, I have no reason to believe it's an incorrect transformation. Certainly there are many instances where LLVM substitutes a different instruction sequence when its cost model indicates, and when that model is correct, that's entirely a good thing. I haven't done the benchmark here but that might be worth trying.

[awxkee]

I got it. This is an instruction on 32-bit platforms ( that is used ), but sequence on aarch64, because there is no vmla on aarch64.