I have a benchmark which compares three functions for throughput. The slowest function (recv_throughput/contended_subchannel_recv) takes around 360 seconds warming up. The other functions take around 5-6 seconds warming up. Since the slowest function is nothing like 60 times slower than the others, I think there may be a problem with the warming up phase.
I am using v0.8.1 of criterion.rs on Linux.
Here's an example run:
Running benches/recv_throughput.rs (target/release/deps/recv_throughput-b6f1fb7439e8afdc)
Gnuplot not found, using plotters backend
Benchmarking recv_throughput/ipc_channel_recv: Collecting 100 samples in estimated 5.0060 s (34k iterecv_throughput/ipc_channel_recv
time: [145.08 µs 145.38 µs 145.70 µs]
thrpt: [343.17 Kelem/s 343.92 Kelem/s 344.64 Kelem/s]
change:
time: [+1.0792% +1.9783% +2.6259%] (p = 0.00 < 0.05)
thrpt: [−2.5587% −1.9399% −1.0676%]
Performance has regressed.
Benchmarking recv_throughput/uncontended_subchannel_recv: Collecting 100 samples in estimated 5.0268recv_throughput/uncontended_subchannel_recv
time: [361.49 µs 362.31 µs 363.22 µs]
thrpt: [137.66 Kelem/s 138.00 Kelem/s 138.32 Kelem/s]
change:
time: [−2.2085% −1.7972% −1.3766%] (p = 0.00 < 0.05)
thrpt: [+1.3958% +1.8301% +2.2584%]
Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
3 (3.00%) high mild
4 (4.00%) high severe
Benchmarking recv_throughput/contended_subchannel_recv: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 34.8s, or reduce sample count to 10.
Benchmarking recv_throughput/contended_subchannel_recv: Collecting 100 samples in estimated 34.758 srecv_throughput/contended_subchannel_recv
time: [420.20 µs 428.50 µs 438.38 µs]
thrpt: [114.06 Kelem/s 116.69 Kelem/s 118.99 Kelem/s]
change:
time: [−85.018% −76.502% −48.480%] (p = 0.00 < 0.05)
thrpt: [+94.098% +325.57% +567.46%]
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
4 (4.00%) high mild
4 (4.00%) high severe
I have a benchmark which compares three functions for throughput. The slowest function (
recv_throughput/contended_subchannel_recv) takes around 360 seconds warming up. The other functions take around 5-6 seconds warming up. Since the slowest function is nothing like 60 times slower than the others, I think there may be a problem with the warming up phase.I am using v0.8.1 of criterion.rs on Linux.
Here's an example run: