[libc][fuzzing] Improve printf long double fuzzing

[michaelrj-google]

Previously we only checked the long double value of the provided double.
This meant we didn't check any values near the edge of the long double
range, which meant we were missing several bugs. This patch adds a
second long double value to check which is generated separately from the
double value and can be anywhere in the range.

[llvmbot]

@llvm/pr-subscribers-libc

Author: Michael Jones (michaelrj-google)

Changes

Previously we only checked the long double value of the provided double.
This meant we didn't check any values near the edge of the long double
range, which meant we were missing several bugs. This patch adds a
second long double value to check which is generated separately from the
double value and can be anywhere in the range.

---

Full diff: https://github.com/llvm/llvm-project/pull/172113.diff

1 Files Affected:

• (modified) libc/fuzzing/stdio/printf_float_conv_fuzz.cpp (+11-5)

diff --git a/libc/fuzzing/stdio/printf_float_conv_fuzz.cpp b/libc/fuzzing/stdio/printf_float_conv_fuzz.cpp
index eefe78c2920d3..f5cfb263cbcb8 100644
--- a/libc/fuzzing/stdio/printf_float_conv_fuzz.cpp
+++ b/libc/fuzzing/stdio/printf_float_conv_fuzz.cpp
@@ -87,10 +87,12 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
   // data = raw_data;
   // size = sizeof(raw_data);
   double num = 0.0;
+  long double ld_num = 0.0L;
   int prec = 0;
   int width = 0;
 
   LIBC_NAMESPACE::fputil::FPBits<double>::StorageType raw_num = 0;
+  LIBC_NAMESPACE::fputil::FPBits<long double>::StorageType ld_raw_num = 0;
 
   // Copy as many bytes of data as will fit into num, prec, and with. Any extras
   // are ignored.
@@ -101,16 +103,19 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
       prec = (prec << 8) + data[cur];
     } else if (cur < sizeof(raw_num) + sizeof(prec) + sizeof(width)) {
       width = (width << 8) + data[cur];
+    } else if (cur < sizeof(raw_num) + sizeof(prec) + sizeof(width) +
+                         sizeof(ld_raw_num)) {
+      ld_raw_num = (ld_raw_num << 8) + data[cur];
     }
   }
 
   num = LIBC_NAMESPACE::fputil::FPBits<double>(raw_num).get_val();
+  ld_num = LIBC_NAMESPACE::fputil::FPBits<long double>(ld_raw_num).get_val();
 
-  // While we could create a "ld_raw_num" from additional bytes, it's much
-  // easier to stick with simply casting num to long double. This avoids the
-  // issues around 80 bit long doubles, especially unnormal and pseudo-denormal
-  // numbers, which MPFR doesn't handle well.
-  long double ld_num = static_cast<long double>(num);
+  // checking the same value in double and long double could help find
+  // mismatches. Mostly this is here to match previous behavior where this was
+  // the only long double value checked.
+  long double num_as_ld = static_cast<long double>(num);
 
   if (width > MAX_SIZE) {
     width = MAX_SIZE;
@@ -130,6 +135,7 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
     TestResult result;
     if (fmt_arr[cur_fmt][fmt_len - 2] == 'L') {
       result = test_vals<long double>(fmt_arr[cur_fmt], ld_num, prec, width);
+      result = test_vals<long double>(fmt_arr[cur_fmt], num_as_ld, prec, width);
     } else {
       result = test_vals<double>(fmt_arr[cur_fmt], num, prec, width);
     }

[michaelrj-google]

I'm currently letting the fuzzer run on my workstation, I'll probably let it run over the weekend before merging.

[vonosmas]

Arguably, this also needed to start exercising long double printing code early on, before the data grows big enough to start producing non-empty ld_raw_num data?

[michaelrj-google]

updated the comment.