Skip to content

benchmark_results.md

Latest commit

 

History

History
197 lines (144 loc) · 13.9 KB

benchmark_results.md

File metadata and controls

197 lines (144 loc) · 13.9 KB
Raw

Benchmark Results & Analysis

*** Generated by Gemini Pro 2.5 for your convenience based on the raw benchmark data.

*Note: Benchmarks run on Windows 11, 12th Gen Intel Core i5-12600K, .NET 9.0.4, with SwitchMediator v1.13.0.

These benchmarks compare SwitchMediator (a source-generated mediator) against the popular MediatR library across various scenarios. The goal is to quantify the performance benefits offered by SwitchMediator's compile-time generation approach.

Key Observations & Conclusions

1. Startup Performance (_Startup methods)

  • Observation: SwitchMediator's startup is staggeringly faster than MediatR's, and the performance gap widens exponentially as the number of handlers (N) increases.
    • N=25: ~1.6 µs (Switch) vs ~162 µs (MediatR) -> ~102x faster
    • N=100: ~4.0 µs (Switch) vs ~1.3 ms (MediatR) -> ~335x faster
    • N=600: ~19.9 µs (Switch) vs ~33.5 ms (MediatR) -> ~1688x faster
  • Conclusion: This is a primary advantage of source generation. SwitchMediator completely avoids MediatR's runtime assembly scanning, resulting in near-instantaneous startup that scales incredibly well. This is crucial for:
    • Applications with a large number of handlers.
    • Serverless functions or scenarios requiring fast cold starts.
    • AOT (Ahead-of-Time) compilation scenarios where reflection is problematic.
    • Faster developer feedback loops during testing and debugging.
  • Allocation: Startup allocations are also orders of magnitude lower for SwitchMediator (~147 KB vs ~17 MB at N=600 -> ~117x less), reducing memory pressure during application initialization.

2. Request/Response Dispatch (_Send methods - No Pipeline)

  • Observation: SwitchMediator's Send operation is consistently faster across different handler counts (N).
    • Performance is stable regardless of N.
    • Speedup is consistently ~2.4x - 3.0x faster than MediatR (e.g., ~20 ns vs ~56 ns at N=600).
  • Conclusion: The source-generated dispatch mechanism provides a significant, constant-time speed advantage for core request/response handling compared to MediatR's approach.
  • Allocation: SwitchMediator allocates 3x less memory per Send operation (96 B vs 288 B). This reduction is vital for high-throughput applications, minimizing GC pressure and improving overall efficiency. The 96 B likely represents near-optimal allocation (Task, request/response objects).

3. Notification Dispatch (_Publish methods)

  • Observation: SwitchMediator's Publish is significantly faster and scales well.
    • Consistently ~4.8x - 5.1x faster than MediatR (e.g., ~20 ns vs ~96 ns at N=600).
  • Conclusion: SwitchMediator uses a highly efficient, source-generated mechanism for identifying and invoking notification handlers.
  • Allocation: This is a major highlight: SwitchMediator achieves ZERO memory allocation for the Publish operation itself (the notification object was pre-allocated for the test). MediatR allocates 592 B per publish. This makes SwitchMediator extremely compelling for event-driven systems or scenarios with frequent notifications, as it completely avoids GC impact from the dispatch process.

4. Pipeline Performance (_Send_WithPipeline methods)

  • Observation: SwitchMediator remains faster when executing requests through open generic pipeline behaviors, with the advantage holding as the number of behaviors (B) increases.
    • B=1: ~541 ns vs ~788 ns -> ~1.46x faster
    • B=5: ~1.56 µs vs ~1.95 µs -> ~1.25x faster
  • Conclusion: The source-generated pipeline handling in SwitchMediator introduces less overhead per step compared to MediatR's pipeline resolution and execution.
  • Allocation: SwitchMediator consistently allocates less memory during pipeline execution (~1.6x - 2.0x less than MediatR - e.g., 312 B vs 640 B for B=1). This reinforces the lower overhead per pipeline step and contributes to better performance in complex request flows.

Overall Conclusion

The benchmark results clearly demonstrate that SwitchMediator's source-generation strategy delivers substantial performance improvements and significant memory allocation reductions compared to MediatR across all core operations:

  • Startup: Orders of magnitude faster (up to ~1688x), with excellent scaling and ~117x less allocation.
  • Send: ~3x faster with 3x less allocation.
  • Publish: ~5x faster with zero allocation.
  • Pipelines: Faster execution (~1.25-1.45x) with lower allocation per step (~1.6-2.0x less).

SwitchMediator is positioned as the high-performance, low-allocation choice for implementing the mediator pattern in modern .NET applications, particularly beneficial in scenarios demanding fast startup times, low latency, high throughput, and minimal GC pressure.

Raw Benchmark Data


SwitchMediator v1.13.0

BenchmarkDotNet v0.14.0, Windows 11 (10.0.26100.3775)
12th Gen Intel Core i5-12600K, 1 CPU, 16 logical and 10 physical cores
.NET SDK 9.0.203
  [Host]     : .NET 9.0.4 (9.0.425.16305), X64 RyuJIT AVX2
  DefaultJob : .NET 9.0.4 (9.0.425.16305), X64 RyuJIT AVX2
Method N Mean Error StdDev Gen0 Gen1 Allocated
MediatR_Publish 25 92.10 ns 1.121 ns 0.936 ns 0.0565 - 592 B
SwitchMediator_Publish 25 19.08 ns 0.071 ns 0.066 ns - - -
MediatR_Send 25 54.84 ns 1.057 ns 0.989 ns 0.0275 - 288 B
SwitchMediator_Send 25 23.12 ns 0.053 ns 0.050 ns 0.0092 - 96 B
MediatR_Startup 25 161,727.22 ns 691.492 ns 646.822 ns 61.0352 1.4648 640385 B
SwitchMediator_Startup 25 1,589.41 ns 6.579 ns 6.154 ns 1.1005 0.0324 11512 B
Method N Mean Error StdDev Gen0 Gen1 Allocated
MediatR_Publish 100 101.01 ns 0.250 ns 0.234 ns 0.0565 - 592 B
SwitchMediator_Publish 100 19.78 ns 0.058 ns 0.054 ns - - -
MediatR_Send 100 60.65 ns 0.106 ns 0.099 ns 0.0275 - 288 B
SwitchMediator_Send 100 20.13 ns 0.034 ns 0.032 ns 0.0092 - 96 B
MediatR_Startup 100 1,345,423.47 ns 5,475.594 ns 4,853.971 ns 234.3750 15.6250 2488491 B
SwitchMediator_Startup 100 4,010.25 ns 18.276 ns 16.201 ns 3.0518 0.1984 31904 B
Method N Mean Error StdDev Gen0 Gen1 Allocated
MediatR_Publish 600 96.23 ns 0.192 ns 0.170 ns 0.0565 - 592 B
SwitchMediator_Publish 600 20.11 ns 0.078 ns 0.069 ns - - -
MediatR_Send 600 55.74 ns 0.071 ns 0.063 ns 0.0275 - 288 B
SwitchMediator_Send 600 20.09 ns 0.099 ns 0.088 ns 0.0092 - 96 B
MediatR_Startup 600 33,505,761.90 ns 150,263.383 ns 133,204.575 ns 1600.0000 400.0000 17127661 B
SwitchMediator_Startup 600 19,852.04 ns 64.766 ns 60.582 ns 13.9771 2.8076 146616 B
Method B Mean Error StdDev Gen0 Allocated
MediatR_Send_WithPipeline 1 787.8 ns 4.73 ns 4.19 ns 0.0610 640 B
SwitchMediator_Send_WithPipeline 1 541.4 ns 6.42 ns 5.69 ns 0.0296 312 B
Method B Mean Error StdDev Gen0 Allocated
MediatR_Send_WithPipeline 5 1.954 μs 0.0073 μs 0.0068 μs 0.1564 1.63 KB
SwitchMediator_Send_WithPipeline 5 1.556 μs 0.0155 μs 0.0137 μs 0.0992 1.02 KB 

SwitchMediator v1.12.1

BenchmarkDotNet v0.14.0, macOS Ventura 13.7.5 (22H527) [Darwin 22.6.0]
Intel Core i7-7820HQ CPU 2.90GHz (Kaby Lake), 1 CPU, 8 logical and 4 physical cores
.NET SDK 9.0.202
  [Host]     : .NET 9.0.3 (9.0.325.11113), X64 RyuJIT AVX2
  DefaultJob : .NET 9.0.3 (9.0.325.11113), X64 RyuJIT AVX2
Method N Mean Error StdDev Gen0 Allocated
MediatR_Publish 25 471.39 ns 1.894 ns 1.679 ns 0.1411 592 B
SwitchMediator_Publish 25 71.74 ns 0.413 ns 0.387 ns - -
MediatR_Send 25 273.20 ns 2.562 ns 2.397 ns 0.0687 288 B
SwitchMediator_Send 25 83.01 ns 0.351 ns 0.293 ns 0.0229 96 B
MediatR_Startup 25 675,057.70 ns 3,292.076 ns 2,918.339 ns 154.2969 645961 B
SwitchMediator_Startup 25 7,258.13 ns 38.090 ns 29.738 ns 2.4719 10360 B
Method N Mean Error StdDev Median Gen0 Gen1 Allocated
MediatR_Publish 100 473.63 ns 1.939 ns 1.719 ns 473.22 ns 0.1411 - 592 B
SwitchMediator_Publish 100 73.19 ns 0.140 ns 0.124 ns 73.18 ns - - -
MediatR_Send 100 236.12 ns 3.978 ns 7.758 ns 232.65 ns 0.0687 - 288 B
SwitchMediator_Send 100 85.73 ns 0.355 ns 0.332 ns 85.79 ns 0.0229 - 96 B
MediatR_Startup 100 4,786,158.05 ns 23,124.573 ns 21,630.740 ns 4,783,637.26 ns 609.3750 15.6250 2597164 B
SwitchMediator_Startup 100 37,267.24 ns 121.801 ns 107.973 ns 37,280.91 ns 7.3242 - 30752 B
Method N Mean Error StdDev Gen0 Gen1 Allocated
MediatR_Publish 600 503.12 ns 2.910 ns 2.430 ns 0.1411 - 592 B
SwitchMediator_Publish 600 75.12 ns 0.436 ns 0.408 ns - - -
MediatR_Send 600 230.97 ns 0.367 ns 0.344 ns 0.0687 - 288 B
SwitchMediator_Send 600 82.48 ns 0.289 ns 0.256 ns 0.0229 - 96 B
MediatR_Startup 600 113,893,678.89 ns 884,434.379 ns 827,300.467 ns 5666.6667 333.3333 24237781 B
SwitchMediator_Startup 600 779,252.40 ns 1,331.257 ns 1,180.124 ns 34.1797 - 145465 B
Method B Mean Error StdDev Gen0 Allocated
MediatR_Send_WithPipeline 1 1.699 μs 0.0031 μs 0.0027 μs 0.1526 640 B
SwitchMediator_Send_WithPipeline 1 1.238 μs 0.0052 μs 0.0049 μs 0.0744 312 B
Method B Mean Error StdDev Gen0 Allocated
MediatR_Send_WithPipeline 5 5.492 μs 0.0639 μs 0.0566 μs 0.3967 1.63 KB
SwitchMediator_Send_WithPipeline 5 4.539 μs 0.0871 μs 0.0855 μs 0.2518 1.02 KB