Commit d3cd0b4
committed
Round exact ties to nearest-even when converting to float8_e8m0fnu
Converting a float/double/bfloat16 to float8_e8m0fnu (the mantissa-less,
exponent-only scale type) rounded every exact tie up instead of to
nearest-even, contradicting the type's round_style (round_to_nearest)
and the RoundBitsToNearestEven routine used for the conversion. For
example 3.0 (an exact tie between 2.0 and 4.0) converted to 4.0 rather
than 2.0.
RoundBitsToNearestEven derives the round-to-even ("last kept") bit as
bit `roundoff` of the mantissa. For a mantissa-less target that bit is
the always-1 implicit bit, so ties always rounded up; the result's true
least-significant bit is the exponent LSB, which the implicit bit carries
into during packing.
Add an overload of RoundBitsToNearestEven that accepts the last-kept bit
explicitly, and pass the exponent LSB for the kToMantissaBits == 0 case.
Targets with mantissa bits are unchanged. 126 of the 254 e8m0 tie points
were previously mis-rounded.1 parent 8f7a061 commit d3cd0b4
3 files changed
Lines changed: 53 additions & 4 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
27 | 27 | | |
28 | 28 | | |
29 | 29 | | |
| 30 | + | |
| 31 | + | |
30 | 32 | | |
31 | 33 | | |
32 | 34 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1263 | 1263 | | |
1264 | 1264 | | |
1265 | 1265 | | |
1266 | | - | |
| 1266 | + | |
| 1267 | + | |
1267 | 1268 | | |
1268 | 1269 | | |
1269 | 1270 | | |
| |||
1272 | 1273 | | |
1273 | 1274 | | |
1274 | 1275 | | |
1275 | | - | |
| 1276 | + | |
| 1277 | + | |
| 1278 | + | |
| 1279 | + | |
| 1280 | + | |
| 1281 | + | |
| 1282 | + | |
1276 | 1283 | | |
1277 | 1284 | | |
1278 | 1285 | | |
| 1286 | + | |
| 1287 | + | |
| 1288 | + | |
| 1289 | + | |
| 1290 | + | |
1279 | 1291 | | |
1280 | 1292 | | |
1281 | 1293 | | |
| |||
1462 | 1474 | | |
1463 | 1475 | | |
1464 | 1476 | | |
1465 | | - | |
1466 | | - | |
| 1477 | + | |
| 1478 | + | |
| 1479 | + | |
| 1480 | + | |
| 1481 | + | |
| 1482 | + | |
| 1483 | + | |
| 1484 | + | |
| 1485 | + | |
| 1486 | + | |
| 1487 | + | |
| 1488 | + | |
| 1489 | + | |
| 1490 | + | |
| 1491 | + | |
| 1492 | + | |
| 1493 | + | |
1467 | 1494 | | |
1468 | 1495 | | |
1469 | 1496 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
430 | 430 | | |
431 | 431 | | |
432 | 432 | | |
| 433 | + | |
| 434 | + | |
| 435 | + | |
| 436 | + | |
| 437 | + | |
| 438 | + | |
| 439 | + | |
| 440 | + | |
| 441 | + | |
| 442 | + | |
| 443 | + | |
| 444 | + | |
| 445 | + | |
| 446 | + | |
| 447 | + | |
| 448 | + | |
| 449 | + | |
| 450 | + | |
| 451 | + | |
| 452 | + | |
433 | 453 | | |
434 | 454 | | |
435 | 455 | | |
| |||
0 commit comments