New benchmark suite #761

ardatan · 2025-09-02T14:36:10Z

No description provided.

github-actions · 2025-09-02T15:40:18Z

Overview for: `constant-vus-subgraphs-delay`

This scenario runs 4 subgraphs and a GraphQL gateway with Federation spec, and runs a heavy query. It's being executed with a constant amount of VUs over a fixed amount of time. It measure things like memory usage, CPU usage, average RPS. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was running 50 VUs over 60s

Comparison

Gateway	RPS ⬇️	Requests	Duration	Notes
grafbase	168	13638 total, 0 failed	avg: 222ms, p95: 235ms	✅
hive-router	167	13535 total, 0 failed	avg: 224ms, p95: 234ms	✅
cosmo	160	13030 total, 0 failed	avg: 232ms, p95: 254ms	✅
apollo-router	124	10166 total, 0 failed	avg: 298ms, p95: 343ms	✅
hive-gateway	111	8545 total, 0 failed	avg: 353ms, p95: 467ms	✅
hive-gateway-bun	111	8548 total, 0 failed	avg: 353ms, p95: 446ms	✅

Summary for: `grafbase`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 40614      ✗ 0    
     data_received..................: 1.2 GB  15 MB/s
     data_sent......................: 16 MB   196 kB/s
     http_req_blocked...............: avg=20.49µs  min=1.36µs   med=3.18µs   max=8.93ms   p(90)=4.71µs   p(95)=5.67µs   p(99.9)=6.57ms  
     http_req_connecting............: avg=16.44µs  min=0s       med=0s       max=8.89ms   p(90)=0s       p(95)=0s       p(99.9)=6.55ms  
     http_req_duration..............: avg=221.62ms min=205.86ms med=220.17ms max=630.19ms p(90)=231.29ms p(95)=234.7ms  p(99.9)=512.67ms
       { expected_response:true }...: avg=221.62ms min=205.86ms med=220.17ms max=630.19ms p(90)=231.29ms p(95)=234.7ms  p(99.9)=512.67ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 13638
     http_req_receiving.............: avg=85.47µs  min=30.81µs  med=57.37µs  max=58.01ms  p(90)=101.98µs p(95)=130.4µs  p(99.9)=2.27ms  
     http_req_sending...............: avg=61.7µs   min=6.33µs   med=12.67µs  max=235.67ms p(90)=21.98µs  p(95)=37.62µs  p(99.9)=3.41ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=221.47ms min=205.79ms med=220.06ms max=621.84ms p(90)=231.15ms p(95)=234.59ms p(99.9)=511.31ms
     http_reqs......................: 13638   168.161741/s
     iteration_duration.............: avg=222.01ms min=206.03ms med=220.5ms  max=644.86ms p(90)=231.56ms p(95)=234.98ms p(99.9)=536.53ms
     iterations.....................: 13538   166.928703/s
     success_rate...................: 100.00% ✓ 13538      ✗ 0    
     vus............................: 31      min=0        max=50 
     vus_max........................: 50      min=50       max=50

Performance Overview

HTTP Overview

Summary for: `hive-router`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 40305      ✗ 0    
     data_received..................: 1.2 GB  15 MB/s
     data_sent......................: 16 MB   194 kB/s
     http_req_blocked...............: avg=19.34µs  min=1.17µs   med=2.18µs   max=9ms      p(90)=3.17µs   p(95)=4.58µs   p(99.9)=6.62ms  
     http_req_connecting............: avg=16.68µs  min=0s       med=0s       max=8.97ms   p(90)=0s       p(95)=0s       p(99.9)=6.6ms   
     http_req_duration..............: avg=223.59ms min=159.57ms med=222.76ms max=616.11ms p(90)=231.65ms p(95)=234.03ms p(99.9)=520.25ms
       { expected_response:true }...: avg=223.59ms min=159.57ms med=222.76ms max=616.11ms p(90)=231.65ms p(95)=234.03ms p(99.9)=520.25ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 13535
     http_req_receiving.............: avg=75.2µs   min=24.18µs  med=41.96µs  max=11.59ms  p(90)=87.3µs   p(95)=253.23µs p(99.9)=1.86ms  
     http_req_sending...............: avg=87.2µs   min=5.44µs   med=9.12µs   max=380.59ms p(90)=28.79µs  p(95)=94.43µs  p(99.9)=4.1ms   
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=223.43ms min=159.52ms med=222.68ms max=615.37ms p(90)=231.5ms  p(95)=233.85ms p(99.9)=512.34ms
     http_reqs......................: 13535   167.074757/s
     iteration_duration.............: avg=223.92ms min=159.7ms  med=223ms    max=625.07ms p(90)=231.83ms p(95)=234.18ms p(99.9)=526.46ms
     iterations.....................: 13435   165.840366/s
     success_rate...................: 100.00% ✓ 13435      ✗ 0    
     vus............................: 40      min=0        max=50 
     vus_max........................: 50      min=50       max=50

Performance Overview

HTTP Overview

Summary for: `cosmo`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 38790      ✗ 0    
     data_received..................: 1.1 GB  14 MB/s
     data_sent......................: 15 MB   187 kB/s
     http_req_blocked...............: avg=20.69µs  min=1.48µs   med=2.58µs   max=9.12ms   p(90)=3.63µs   p(95)=4.45µs   p(99.9)=6.85ms  
     http_req_connecting............: avg=17.53µs  min=0s       med=0s       max=9.09ms   p(90)=0s       p(95)=0s       p(99.9)=6.84ms  
     http_req_duration..............: avg=232.07ms min=164.67ms med=232.03ms max=635.27ms p(90)=248.02ms p(95)=253.98ms p(99.9)=522.1ms 
       { expected_response:true }...: avg=232.07ms min=164.67ms med=232.03ms max=635.27ms p(90)=248.02ms p(95)=253.98ms p(99.9)=522.1ms 
     http_req_failed................: 0.00%   ✓ 0          ✗ 13030
     http_req_receiving.............: avg=272.78µs min=30.72µs  med=60.79µs  max=340.72ms p(90)=169.65µs p(95)=415.66µs p(99.9)=25.27ms 
     http_req_sending...............: avg=91.25µs  min=6.51µs   med=10.73µs  max=205.76ms p(90)=27.33µs  p(95)=89.85µs  p(99.9)=3.18ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=231.71ms min=164.35ms med=231.79ms max=620.97ms p(90)=247.61ms p(95)=253.36ms p(99.9)=509.92ms
     http_reqs......................: 13030   160.793756/s
     iteration_duration.............: avg=232.53ms min=164.84ms med=232.31ms max=641.42ms p(90)=248.32ms p(95)=254.28ms p(99.9)=536.32ms
     iterations.....................: 12930   159.559728/s
     success_rate...................: 100.00% ✓ 12930      ✗ 0    
     vus............................: 36      min=0        max=50 
     vus_max........................: 50      min=50       max=50

Performance Overview

HTTP Overview

Summary for: `apollo-router`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 30198      ✗ 0    
     data_received..................: 892 MB  11 MB/s
     data_sent......................: 12 MB   145 kB/s
     http_req_blocked...............: avg=63.83µs  min=1.8µs    med=2.81µs   max=17.15ms  p(90)=4.16µs   p(95)=5.13µs   p(99.9)=14.55ms 
     http_req_connecting............: avg=60.09µs  min=0s       med=0s       max=16.88ms  p(90)=0s       p(95)=0s       p(99.9)=14.44ms 
     http_req_duration..............: avg=297.89ms min=160.66ms med=298.84ms max=721.82ms p(90)=333.74ms p(95)=342.84ms p(99.9)=622.37ms
       { expected_response:true }...: avg=297.89ms min=160.66ms med=298.84ms max=721.82ms p(90)=333.74ms p(95)=342.84ms p(99.9)=622.37ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 10166
     http_req_receiving.............: avg=98.89µs  min=34.42µs  med=52.07µs  max=149.97ms p(90)=104.23µs p(95)=187.83µs p(99.9)=1.46ms  
     http_req_sending...............: avg=94.94µs  min=7.68µs   med=11.03µs  max=208.5ms  p(90)=28.37µs  p(95)=130.06µs p(99.9)=3.92ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=297.69ms min=160.6ms  med=298.7ms  max=713.82ms p(90)=333.62ms p(95)=342.72ms p(99.9)=609.72ms
     http_reqs......................: 10166   124.741718/s
     iteration_duration.............: avg=299.1ms  min=160.86ms med=299.39ms max=736.33ms p(90)=334.09ms p(95)=343.18ms p(99.9)=679.48ms
     iterations.....................: 10066   123.51467/s
     success_rate...................: 100.00% ✓ 10066      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50

Performance Overview

HTTP Overview

Summary for: `hive-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 25335      ✗ 0   
     data_received..................: 751 MB  9.8 MB/s
     data_sent......................: 9.9 MB  130 kB/s
     http_req_blocked...............: avg=42.99µs  min=1.58µs   med=4.12µs   max=11.96ms p(90)=5.92µs   p(95)=6.81µs   p(99.9)=10.15ms 
     http_req_connecting............: avg=37.55µs  min=0s       med=0s       max=11.93ms p(90)=0s       p(95)=0s       p(99.9)=10.12ms 
     http_req_duration..............: avg=353.27ms min=160.56ms med=351.41ms max=1.05s   p(90)=431.96ms p(95)=467.12ms p(99.9)=948.45ms
       { expected_response:true }...: avg=353.27ms min=160.56ms med=351.41ms max=1.05s   p(90)=431.96ms p(95)=467.12ms p(99.9)=948.45ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 8545
     http_req_receiving.............: avg=102.19µs min=35.49µs  med=77.64µs  max=89.07ms p(90)=125.99µs p(95)=153.87µs p(99.9)=1.82ms  
     http_req_sending...............: avg=61.19µs  min=7.72µs   med=16.06µs  max=98.8ms  p(90)=24.64µs  p(95)=40.47µs  p(99.9)=2.65ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s      p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=353.11ms min=160.48ms med=351.31ms max=1.05s   p(90)=431.72ms p(95)=466.92ms p(99.9)=947.66ms
     http_reqs......................: 8545    111.479501/s
     iteration_duration.............: avg=356.08ms min=231.58ms med=352.97ms max=1.1s    p(90)=432.9ms  p(95)=468.9ms  p(99.9)=988.78ms
     iterations.....................: 8445    110.174884/s
     success_rate...................: 100.00% ✓ 8445       ✗ 0   
     vus............................: 50      min=0        max=50
     vus_max........................: 50      min=50       max=50

Performance Overview

HTTP Overview

Summary for: `hive-gateway-bun`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 25344      ✗ 0   
     data_received..................: 751 MB  9.8 MB/s
     data_sent......................: 9.9 MB  129 kB/s
     http_req_blocked...............: avg=89.56µs  min=1.73µs   med=4.75µs   max=20.79ms  p(90)=6.43µs   p(95)=7.49µs   p(99.9)=18.61ms 
     http_req_connecting............: avg=83.24µs  min=0s       med=0s       max=20.72ms  p(90)=0s       p(95)=0s       p(99.9)=18.5ms  
     http_req_duration..............: avg=353.09ms min=160.67ms med=350.74ms max=889.92ms p(90)=422.36ms p(95)=445.66ms p(99.9)=800.25ms
       { expected_response:true }...: avg=353.09ms min=160.67ms med=350.74ms max=889.92ms p(90)=422.36ms p(95)=445.66ms p(99.9)=800.25ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 8548
     http_req_receiving.............: avg=119.84µs min=37.74µs  med=88.8µs   max=146.05ms p(90)=136.82µs p(95)=164.61µs p(99.9)=2.52ms  
     http_req_sending...............: avg=124.36µs min=8µs      med=18.81µs  max=215.61ms p(90)=29.75µs  p(95)=44.59µs  p(99.9)=2.73ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=352.85ms min=160.54ms med=350.59ms max=889.37ms p(90)=422.18ms p(95)=445.46ms p(99.9)=798.74ms
     http_reqs......................: 8548    111.29116/s
     iteration_duration.............: avg=355.94ms min=192.27ms med=351.92ms max=931.7ms  p(90)=423.19ms p(95)=446.64ms p(99.9)=882.29ms
     iterations.....................: 8448    109.989205/s
     success_rate...................: 100.00% ✓ 8448       ✗ 0   
     vus............................: 50      min=0        max=50
     vus_max........................: 50      min=50       max=50

Performance Overview

HTTP Overview

github-actions · 2025-09-02T15:40:19Z

Overview for: `constant-vus-subgraphs-delay-resources`

This scenario runs 4 subgraphs and a GraphQL gateway with Federation spec, and runs a heavy query. It's being executed with a constant amount of VUs over a fixed amount of time. It measure things like memory usage, CPU usage, average RPS. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was running 50 VUs over 60s

Comparison

Gateway	RPS ⬇️	Requests	Duration	Notes
hive-router	169	13742 total, 0 failed	avg: 220ms, p95: 230ms	✅
grafbase	167	13597 total, 0 failed	avg: 222ms, p95: 236ms	✅
cosmo	166	13489 total, 0 failed	avg: 224ms, p95: 248ms	✅
apollo-router	135	11072 total, 0 failed	avg: 273ms, p95: 316ms	❌ 1 unexpected GraphQL errors
hive-gateway-bun	130	10019 total, 0 failed	avg: 301ms, p95: 396ms	✅
hive-gateway	129	9905 total, 0 failed	avg: 304ms, p95: 377ms	✅

Summary for: `hive-router`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 40926      ✗ 0    
     data_received..................: 1.2 GB  15 MB/s
     data_sent......................: 16 MB   198 kB/s
     http_req_blocked...............: avg=19.29µs  min=1.19µs   med=2.04µs   max=8.95ms   p(90)=3.12µs   p(95)=4.46µs   p(99.9)=6.65ms  
     http_req_connecting............: avg=16.73µs  min=0s       med=0s       max=8.92ms   p(90)=0s       p(95)=0s       p(99.9)=6.63ms  
     http_req_duration..............: avg=220.03ms min=117.85ms med=219.03ms max=610.97ms p(90)=227.1ms  p(95)=229.59ms p(99.9)=522.82ms
       { expected_response:true }...: avg=220.03ms min=117.85ms med=219.03ms max=610.97ms p(90)=227.1ms  p(95)=229.59ms p(99.9)=522.82ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 13742
     http_req_receiving.............: avg=70.27µs  min=23.79µs  med=41.22µs  max=9.61ms   p(90)=84.23µs  p(95)=240.85µs p(99.9)=1.34ms  
     http_req_sending...............: avg=163.05µs min=5.31µs   med=8.54µs   max=378.88ms p(90)=29.7µs   p(95)=94.15µs  p(99.9)=6.71ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=219.8ms  min=113.7ms  med=218.95ms max=610.28ms p(90)=226.95ms p(95)=229.4ms  p(99.9)=503.48ms
     http_reqs......................: 13742   169.867718/s
     iteration_duration.............: avg=220.32ms min=118.05ms med=219.25ms max=619.41ms p(90)=227.27ms p(95)=229.76ms p(99.9)=530.13ms
     iterations.....................: 13642   168.631597/s
     success_rate...................: 100.00% ✓ 13642      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50

Performance Overview

HTTP Overview

Summary for: `grafbase`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 40491      ✗ 0    
     data_received..................: 1.2 GB  15 MB/s
     data_sent......................: 16 MB   195 kB/s
     http_req_blocked...............: avg=19.96µs  min=1.32µs   med=2.96µs   max=8.67ms   p(90)=4.15µs   p(95)=4.95µs   p(99.9)=6.44ms  
     http_req_connecting............: avg=16.16µs  min=0s       med=0s       max=8.64ms   p(90)=0s       p(95)=0s       p(99.9)=6.42ms  
     http_req_duration..............: avg=222.27ms min=205.56ms med=220.86ms max=642.26ms p(90)=232.45ms p(95)=235.83ms p(99.9)=523.31ms
       { expected_response:true }...: avg=222.27ms min=205.56ms med=220.86ms max=642.26ms p(90)=232.45ms p(95)=235.83ms p(99.9)=523.31ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 13597
     http_req_receiving.............: avg=77.21µs  min=28.44µs  med=53.13µs  max=4.11ms   p(90)=95.04µs  p(95)=129µs    p(99.9)=2.37ms  
     http_req_sending...............: avg=88.78µs  min=6.18µs   med=12µs     max=306.85ms p(90)=19.41µs  p(95)=35.57µs  p(99.9)=2.61ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=222.1ms  min=205.52ms med=220.76ms max=640.91ms p(90)=232.32ms p(95)=235.7ms  p(99.9)=515.81ms
     http_reqs......................: 13597   167.688893/s
     iteration_duration.............: avg=222.66ms min=205.68ms med=221.15ms max=668.41ms p(90)=232.73ms p(95)=236.12ms p(99.9)=552.34ms
     iterations.....................: 13497   166.455614/s
     success_rate...................: 100.00% ✓ 13497      ✗ 0    
     vus............................: 24      min=0        max=50 
     vus_max........................: 50      min=50       max=50

Performance Overview

HTTP Overview

Summary for: `cosmo`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 40167      ✗ 0    
     data_received..................: 1.2 GB  15 MB/s
     data_sent......................: 16 MB   193 kB/s
     http_req_blocked...............: avg=48.43µs  min=1.35µs   med=2.61µs   max=18.76ms  p(90)=3.78µs   p(95)=4.7µs    p(99.9)=14.01ms 
     http_req_connecting............: avg=44.94µs  min=0s       med=0s       max=18.73ms  p(90)=0s       p(95)=0s       p(99.9)=13.93ms 
     http_req_duration..............: avg=224.28ms min=116.54ms med=222.98ms max=603.4ms  p(90)=242.55ms p(95)=248.19ms p(99.9)=497.23ms
       { expected_response:true }...: avg=224.28ms min=116.54ms med=222.98ms max=603.4ms  p(90)=242.55ms p(95)=248.19ms p(99.9)=497.23ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 13489
     http_req_receiving.............: avg=470.11µs min=32.19µs  med=61.91µs  max=34.87ms  p(90)=343.73µs p(95)=740µs    p(99.9)=25.84ms 
     http_req_sending...............: avg=94.23µs  min=6.05µs   med=10.89µs  max=225.29ms p(90)=30.09µs  p(95)=117.11µs p(99.9)=6.01ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=223.71ms min=112.89ms med=222.47ms max=600.81ms p(90)=241.83ms p(95)=247.37ms p(99.9)=491.68ms
     http_reqs......................: 13489   166.323917/s
     iteration_duration.............: avg=224.69ms min=116.72ms med=223.33ms max=629.08ms p(90)=242.9ms  p(95)=248.45ms p(99.9)=519.81ms
     iterations.....................: 13389   165.090883/s
     success_rate...................: 100.00% ✓ 13389      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50

Performance Overview

HTTP Overview

Summary for: `apollo-router`

K6 Output

     ✓ response code was 200
     ✗ no graphql errors
      ↳  99% — ✓ 10971 / ✗ 1
     ✓ valid response structure

     checks.........................: 99.99% ✓ 32915      ✗ 1    
     data_received..................: 972 MB 12 MB/s
     data_sent......................: 13 MB  158 kB/s
     http_req_blocked...............: avg=45.48µs  min=1.71µs   med=2.93µs   max=16.74ms  p(90)=4.45µs   p(95)=5.47µs   p(99.9)=12.88ms 
     http_req_connecting............: avg=40.7µs   min=0s       med=0s       max=16.54ms  p(90)=0s       p(95)=0s       p(99.9)=12.76ms 
     http_req_duration..............: avg=273.36ms min=151.85ms med=274.45ms max=680.72ms p(90)=307.64ms p(95)=315.99ms p(99.9)=579.8ms 
       { expected_response:true }...: avg=273.36ms min=151.85ms med=274.45ms max=680.72ms p(90)=307.64ms p(95)=315.99ms p(99.9)=579.8ms 
     http_req_failed................: 0.00%  ✓ 0          ✗ 11072
     http_req_receiving.............: avg=119.88µs min=30.34µs  med=54.1µs   max=12.36ms  p(90)=158.78µs p(95)=427.81µs p(99.9)=4.09ms  
     http_req_sending...............: avg=122.39µs min=7.06µs   med=11.43µs  max=222.51ms p(90)=45.64µs  p(95)=134.79µs p(99.9)=4.66ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=273.11ms min=151.77ms med=274.29ms max=677.86ms p(90)=307.4ms  p(95)=315.77ms p(99.9)=577ms   
     http_reqs......................: 11072  135.900792/s
     iteration_duration.............: avg=274.28ms min=152.03ms med=275.02ms max=697.92ms p(90)=307.95ms p(95)=316.46ms p(99.9)=624.85ms
     iterations.....................: 10972  134.673364/s
     success_rate...................: 99.99% ✓ 10971      ✗ 1    
     vus............................: 50     min=0        max=50 
     vus_max........................: 50     min=50       max=50

Performance Overview

HTTP Overview

Summary for: `hive-gateway-bun`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 29757     ✗ 0    
     data_received..................: 880 MB  12 MB/s
     data_sent......................: 12 MB   152 kB/s
     http_req_blocked...............: avg=36.03µs  min=1.6µs    med=3.83µs   max=11.13ms  p(90)=5.61µs   p(95)=6.8µs    p(99.9)=9.21ms  
     http_req_connecting............: avg=29.45µs  min=0s       med=0s       max=11.09ms  p(90)=0s       p(95)=0s       p(99.9)=9.18ms  
     http_req_duration..............: avg=300.96ms min=160.88ms med=303.27ms max=936.82ms p(90)=368.68ms p(95)=396.28ms p(99.9)=801.87ms
       { expected_response:true }...: avg=300.96ms min=160.88ms med=303.27ms max=936.82ms p(90)=368.68ms p(95)=396.28ms p(99.9)=801.87ms
     http_req_failed................: 0.00%   ✓ 0         ✗ 10019
     http_req_receiving.............: avg=117.29µs min=35.93µs  med=78.67µs  max=117.27ms p(90)=132.4µs  p(95)=174.39µs p(99.9)=3.24ms  
     http_req_sending...............: avg=69.13µs  min=7.01µs   med=15.75µs  max=153.59ms p(90)=33.91µs  p(95)=47.61µs  p(99.9)=3.27ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=300.78ms min=160.75ms med=303.16ms max=935.38ms p(90)=368.45ms p(95)=396.12ms p(99.9)=790.91ms
     http_reqs......................: 10019   130.72193/s
     iteration_duration.............: avg=302.89ms min=189.06ms med=304.33ms max=965.83ms p(90)=369.54ms p(95)=397.04ms p(99.9)=856.12ms
     iterations.....................: 9919    129.41719/s
     success_rate...................: 100.00% ✓ 9919      ✗ 0    
     vus............................: 50      min=0       max=50 
     vus_max........................: 50      min=50      max=50

Performance Overview

HTTP Overview

Summary for: `hive-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 29415      ✗ 0   
     data_received..................: 870 MB  11 MB/s
     data_sent......................: 12 MB   150 kB/s
     http_req_blocked...............: avg=31.24µs  min=1.65µs   med=4.46µs   max=9.95ms   p(90)=6.04µs   p(95)=7.23µs   p(99.9)=7.96ms  
     http_req_connecting............: avg=24.72µs  min=0s       med=0s       max=9.91ms   p(90)=0s       p(95)=0s       p(99.9)=7.94ms  
     http_req_duration..............: avg=304.38ms min=162.07ms med=298ms    max=822.6ms  p(90)=349.55ms p(95)=376.89ms p(99.9)=722.12ms
       { expected_response:true }...: avg=304.38ms min=162.07ms med=298ms    max=822.6ms  p(90)=349.55ms p(95)=376.89ms p(99.9)=722.12ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 9905
     http_req_receiving.............: avg=120.88µs min=31.76µs  med=91.65µs  max=10.56ms  p(90)=155.44µs p(95)=197.77µs p(99.9)=4.05ms  
     http_req_sending...............: avg=114.87µs min=7.73µs   med=18.98µs  max=236.49ms p(90)=37.69µs  p(95)=64.76µs  p(99.9)=4.84ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=304.14ms min=161.94ms med=297.83ms max=821.2ms  p(90)=349.38ms p(95)=376.52ms p(99.9)=720.8ms 
     http_reqs......................: 9905    129.16641/s
     iteration_duration.............: avg=306.46ms min=166.97ms med=298.57ms max=877.13ms p(90)=350.35ms p(95)=377.62ms p(99.9)=802.3ms 
     iterations.....................: 9805    127.862357/s
     success_rate...................: 100.00% ✓ 9805       ✗ 0   
     vus............................: 50      min=0        max=50
     vus_max........................: 50      min=50       max=50

Performance Overview

HTTP Overview

github-actions · 2025-09-02T15:40:25Z

Overview for: `constant-vus-over-time`

This scenario runs 4 subgraphs and a GraphQL gateway with Federation spec, and runs a heavy query. It's being executed with a constant amount of VUs over a fixed amount of time. It measure things like memory usage, CPU usage, average RPS. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was running 50 VUs over 60s

Comparison

Gateway	RPS ⬇️	Requests	Duration	Notes
hive-router	1788	107732 total, 0 failed	avg: 28ms, p95: 49ms	✅
cosmo	690	41719 total, 0 failed	avg: 72ms, p95: 105ms	✅
grafbase	584	35294 total, 0 failed	avg: 85ms, p95: 110ms	✅
apollo-router	380	23148 total, 0 failed	avg: 129ms, p95: 169ms	✅
hive-gateway-bun	150	9215 total, 0 failed	avg: 326ms, p95: 451ms	✅
hive-gateway	142	8756 total, 0 failed	avg: 343ms, p95: 507ms	✅

Summary for: `hive-router`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 322896      ✗ 0     
     data_received..................: 9.4 GB  157 MB/s
     data_sent......................: 125 MB  2.1 MB/s
     http_req_blocked...............: avg=5.33µs   min=1.19µs  med=2.72µs  max=8.89ms   p(90)=4.07µs   p(95)=4.91µs   p(99.9)=88.09µs
     http_req_connecting............: avg=2.07µs   min=0s      med=0s      max=8.86ms   p(90)=0s       p(95)=0s       p(99.9)=0s     
     http_req_duration..............: avg=27.64ms  min=1.91ms  med=25.16ms max=443.33ms p(90)=42.92ms  p(95)=49.12ms  p(99.9)=84.7ms 
       { expected_response:true }...: avg=27.64ms  min=1.91ms  med=25.16ms max=443.33ms p(90)=42.92ms  p(95)=49.12ms  p(99.9)=84.7ms 
     http_req_failed................: 0.00%   ✓ 0           ✗ 107732
     http_req_receiving.............: avg=123.75µs min=24.79µs med=44.29µs max=150.31ms p(90)=119.85µs p(95)=363.44µs p(99.9)=13.03ms
     http_req_sending...............: avg=87.82µs  min=5.71µs  med=10.66µs max=256.99ms p(90)=32.5µs   p(95)=130.16µs p(99.9)=12.35ms
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s      max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s     
     http_req_waiting...............: avg=27.43ms  min=1.87ms  med=25ms    max=442.25ms p(90)=42.56ms  p(95)=48.74ms  p(99.9)=83.3ms 
     http_reqs......................: 107732  1788.095273/s
     iteration_duration.............: avg=27.86ms  min=5.25ms  med=25.36ms max=452.18ms p(90)=43.13ms  p(95)=49.33ms  p(99.9)=85.7ms 
     iterations.....................: 107632  1786.43551/s
     success_rate...................: 100.00% ✓ 107632      ✗ 0     
     vus............................: 50      min=50        max=50  
     vus_max........................: 50      min=50        max=50

Performance Overview

HTTP Overview

Summary for: `cosmo`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 124857     ✗ 0    
     data_received..................: 3.7 GB  61 MB/s
     data_sent......................: 49 MB   803 kB/s
     http_req_blocked...............: avg=8.86µs   min=1.18µs  med=2.68µs  max=9.36ms   p(90)=3.78µs   p(95)=4.5µs    p(99.9)=1.34ms  
     http_req_connecting............: avg=5.58µs   min=0s      med=0s      max=9.32ms   p(90)=0s       p(95)=0s       p(99.9)=1.32ms  
     http_req_duration..............: avg=71.69ms  min=2.8ms   med=70.61ms max=505.9ms  p(90)=97.03ms  p(95)=104.82ms p(99.9)=286.59ms
       { expected_response:true }...: avg=71.69ms  min=2.8ms   med=70.61ms max=505.9ms  p(90)=97.03ms  p(95)=104.82ms p(99.9)=286.59ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 41719
     http_req_receiving.............: avg=160.91µs min=28.27µs med=69.65µs max=115.05ms p(90)=143.72µs p(95)=355.48µs p(99.9)=15.21ms 
     http_req_sending...............: avg=67.3µs   min=5.52µs  med=11.05µs max=411.58ms p(90)=28.77µs  p(95)=118.22µs p(99.9)=2.79ms  
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s      max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=71.46ms  min=2.72ms  med=70.43ms max=493.58ms p(90)=96.8ms   p(95)=104.61ms p(99.9)=267.13ms
     http_reqs......................: 41719   690.33038/s
     iteration_duration.............: avg=72.1ms   min=5.95ms  med=70.89ms max=527.61ms p(90)=97.28ms  p(95)=105.11ms p(99.9)=300.54ms
     iterations.....................: 41619   688.675665/s
     success_rate...................: 100.00% ✓ 41619      ✗ 0    
     vus............................: 50      min=50       max=50 
     vus_max........................: 50      min=50       max=50

Performance Overview

HTTP Overview

Summary for: `grafbase`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 105582     ✗ 0    
     data_received..................: 3.1 GB  51 MB/s
     data_sent......................: 41 MB   680 kB/s
     http_req_blocked...............: avg=12.63µs  min=1.26µs  med=3.24µs  max=9.93ms   p(90)=4.33µs   p(95)=5.58µs   p(99.9)=3.04ms  
     http_req_connecting............: avg=7.28µs   min=0s      med=0s      max=9.9ms    p(90)=0s       p(95)=0s       p(99.9)=2.8ms   
     http_req_duration..............: avg=84.67ms  min=2.55ms  med=83.49ms max=499.46ms p(90)=103.79ms p(95)=110.08ms p(99.9)=300.26ms
       { expected_response:true }...: avg=84.67ms  min=2.55ms  med=83.49ms max=499.46ms p(90)=103.79ms p(95)=110.08ms p(99.9)=300.26ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 35294
     http_req_receiving.............: avg=161.93µs min=26.4µs  med=58.19µs max=99.17ms  p(90)=173.77µs p(95)=416.49µs p(99.9)=8.5ms   
     http_req_sending...............: avg=106.42µs min=5.73µs  med=12.42µs max=374.37ms p(90)=85.94µs  p(95)=128.35µs p(99.9)=6.95ms  
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s      max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=84.4ms   min=2.5ms   med=83.23ms max=487.27ms p(90)=103.48ms p(95)=109.7ms  p(99.9)=292.45ms
     http_reqs......................: 35294   584.826733/s
     iteration_duration.............: avg=85.26ms  min=22.38ms med=83.88ms max=528.71ms p(90)=104.12ms p(95)=110.39ms p(99.9)=314.18ms
     iterations.....................: 35194   583.169718/s
     success_rate...................: 100.00% ✓ 35194      ✗ 0    
     vus............................: 50      min=50       max=50 
     vus_max........................: 50      min=50       max=50

Performance Overview

HTTP Overview

Summary for: `apollo-router`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 69144      ✗ 0    
     data_received..................: 2.0 GB  33 MB/s
     data_sent......................: 27 MB   443 kB/s
     http_req_blocked...............: avg=12.3µs   min=1.37µs  med=2.84µs   max=7.88ms   p(90)=4.26µs   p(95)=4.92µs   p(99.9)=4.47ms  
     http_req_connecting............: avg=8.84µs   min=0s      med=0s       max=7.85ms   p(90)=0s       p(95)=0s       p(99.9)=4.45ms  
     http_req_duration..............: avg=129.41ms min=5.91ms  med=128.33ms max=597.67ms p(90)=159.32ms p(95)=169.15ms p(99.9)=417.18ms
       { expected_response:true }...: avg=129.41ms min=5.91ms  med=128.33ms max=597.67ms p(90)=159.32ms p(95)=169.15ms p(99.9)=417.18ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 23148
     http_req_receiving.............: avg=83.58µs  min=29.54µs med=52.16µs  max=228.21ms p(90)=98.43µs  p(95)=125.62µs p(99.9)=1.16ms  
     http_req_sending...............: avg=64.32µs  min=7.03µs  med=11.46µs  max=391.28ms p(90)=19.64µs  p(95)=33.59µs  p(99.9)=1.89ms  
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=129.26ms min=5.86ms  med=128.23ms max=569.68ms p(90)=159.21ms p(95)=169.04ms p(99.9)=408.26ms
     http_reqs......................: 23148   380.922413/s
     iteration_duration.............: avg=130.26ms min=31.64ms med=128.68ms max=658.6ms  p(90)=159.61ms p(95)=169.45ms p(99.9)=439.69ms
     iterations.....................: 23048   379.276817/s
     success_rate...................: 100.00% ✓ 23048      ✗ 0    
     vus............................: 50      min=50       max=50 
     vus_max........................: 50      min=50       max=50

Performance Overview

HTTP Overview

Summary for: `hive-gateway-bun`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 27345      ✗ 0   
     data_received..................: 809 MB  13 MB/s
     data_sent......................: 11 MB   175 kB/s
     http_req_blocked...............: avg=84.45µs  min=1.64µs  med=3.55µs   max=19.41ms  p(90)=5.19µs   p(95)=5.94µs   p(99.9)=17.92ms
     http_req_connecting............: avg=79.48µs  min=0s      med=0s       max=19.38ms  p(90)=0s       p(95)=0s       p(99.9)=17.88ms
     http_req_duration..............: avg=325.81ms min=7.49ms  med=322.37ms max=1.15s    p(90)=423.27ms p(95)=450.91ms p(99.9)=1.03s  
       { expected_response:true }...: avg=325.81ms min=7.49ms  med=322.37ms max=1.15s    p(90)=423.27ms p(95)=450.91ms p(99.9)=1.03s  
     http_req_failed................: 0.00%   ✓ 0          ✗ 9215
     http_req_receiving.............: avg=81.86µs  min=33.15µs med=67.56µs  max=4.36ms   p(90)=112.54µs p(95)=132.69µs p(99.9)=1.94ms 
     http_req_sending...............: avg=85.27µs  min=7.15µs  med=14.37µs  max=108.12ms p(90)=21.9µs   p(95)=37.64µs  p(99.9)=7.08ms 
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s     
     http_req_waiting...............: avg=325.64ms min=7.43ms  med=322.28ms max=1.15s    p(90)=423.13ms p(95)=450.33ms p(99.9)=1.02s  
     http_reqs......................: 9215    150.489732/s
     iteration_duration.............: avg=329.85ms min=76.64ms med=323.63ms max=1.21s    p(90)=424.46ms p(95)=451.79ms p(99.9)=1.06s  
     iterations.....................: 9115    148.856637/s
     success_rate...................: 100.00% ✓ 9115       ✗ 0   
     vus............................: 48      min=48       max=50
     vus_max........................: 50      min=50       max=50

Performance Overview

HTTP Overview

Summary for: `hive-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 25968      ✗ 0   
     data_received..................: 769 MB  13 MB/s
     data_sent......................: 10 MB   166 kB/s
     http_req_blocked...............: avg=37.77µs  min=1.64µs   med=3.76µs   max=10.8ms   p(90)=5.46µs   p(95)=6.23µs   p(99.9)=9.46ms  
     http_req_connecting............: avg=33.02µs  min=0s       med=0s       max=10.76ms  p(90)=0s       p(95)=0s       p(99.9)=9.44ms  
     http_req_duration..............: avg=342.88ms min=7.72ms   med=331.34ms max=1.11s    p(90)=476.09ms p(95)=506.94ms p(99.9)=936.52ms
       { expected_response:true }...: avg=342.88ms min=7.72ms   med=331.34ms max=1.11s    p(90)=476.09ms p(95)=506.94ms p(99.9)=936.52ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 8756
     http_req_receiving.............: avg=83.79µs  min=33.99µs  med=68.35µs  max=3.12ms   p(90)=115.94µs p(95)=139.06µs p(99.9)=1.51ms  
     http_req_sending...............: avg=54.55µs  min=7.13µs   med=15.01µs  max=136.26ms p(90)=22.74µs  p(95)=38.1µs   p(99.9)=1.39ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=342.74ms min=7.63ms   med=331.18ms max=1.11s    p(90)=475.87ms p(95)=506.87ms p(99.9)=935.67ms
     http_reqs......................: 8756    142.90833/s
     iteration_duration.............: avg=347.27ms min=154.11ms med=334.51ms max=1.13s    p(90)=477.06ms p(95)=508.97ms p(99.9)=950.68ms
     iterations.....................: 8656    141.276211/s
     success_rate...................: 100.00% ✓ 8656       ✗ 0   
     vus............................: 50      min=0        max=50
     vus_max........................: 50      min=50       max=50

Performance Overview

HTTP Overview

github-actions · 2025-09-02T15:41:52Z

Overview for: `ramping-vus`

This scenario runs 4 subgraphs and a GraphQL gateway with Federation spec, and runs a heavy query. We are running a heavy load of concurrent VUs to measure response time and other stats, during stress. It measure things like memory usage, CPU usage, response times. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was running 500 VUs over 60s

Comparison

Gateway	RPS ⬇️	Requests	Duration	Notes
hive-router	1996	124235 total, 0 failed	avg: 112ms, p95: 278ms	✅
cosmo	713	45450 total, 0 failed	avg: 309ms, p95: 689ms	✅
grafbase	510	32361 total, 0 failed	avg: 435ms, p95: 969ms	✅
apollo-router	406	27254 total, 0 failed	avg: 517ms, p95: 1175ms	❌ 10 unexpected GraphQL errors
hive-gateway-bun	155	10817 total, 0 failed	avg: 1232ms, p95: 2488ms	✅
hive-gateway	149	10301 total, 0 failed	avg: 1286ms, p95: 2546ms	✅

Summary for: `hive-router`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 369705      ✗ 0     
     data_received..................: 11 GB   175 MB/s
     data_sent......................: 145 MB  2.3 MB/s
     http_req_blocked...............: avg=356.05µs min=1.06µs  med=2.47µs   max=349.16ms p(90)=3.81µs   p(95)=4.54µs   p(99.9)=133.51ms
     http_req_connecting............: avg=352.7µs  min=0s      med=0s       max=349.08ms p(90)=0s       p(95)=0s       p(99.9)=133.15ms
     http_req_duration..............: avg=112.16ms min=1.69ms  med=99.5ms   max=419.38ms p(90)=229.81ms p(95)=277.57ms p(99.9)=384.5ms 
       { expected_response:true }...: avg=112.16ms min=1.69ms  med=99.5ms   max=419.38ms p(90)=229.81ms p(95)=277.57ms p(99.9)=384.5ms 
     http_req_failed................: 0.00%   ✓ 0           ✗ 124235
     http_req_receiving.............: avg=455.35µs min=22.54µs med=40.73µs  max=140.07ms p(90)=172.12µs p(95)=377.85µs p(99.9)=68.11ms 
     http_req_sending...............: avg=339.22µs min=5.12µs  med=9.88µs   max=147.22ms p(90)=27.7µs   p(95)=123.6µs  p(99.9)=59.09ms 
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=111.37ms min=1.64ms  med=98.82ms  max=413.09ms p(90)=228.02ms p(95)=275.79ms p(99.9)=382.49ms
     http_reqs......................: 124235  1996.08097/s
     iteration_duration.............: avg=113.67ms min=1.8ms   med=100.86ms max=721.46ms p(90)=231.87ms p(95)=279.75ms p(99.9)=397.37ms
     iterations.....................: 123235  1980.013992/s
     success_rate...................: 100.00% ✓ 123235      ✗ 0     
     vus............................: 71      min=0         max=495 
     vus_max........................: 500     min=500       max=500

Performance Overview

HTTP Overview

Summary for: `cosmo`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 133350     ✗ 0    
     data_received..................: 4.0 GB  63 MB/s
     data_sent......................: 53 MB   830 kB/s
     http_req_blocked...............: avg=93.16µs  min=1.24µs  med=2.91µs   max=190.26ms p(90)=4.16µs   p(95)=5.39µs   p(99.9)=26.54ms 
     http_req_connecting............: avg=88.61µs  min=0s      med=0s       max=190.22ms p(90)=0s       p(95)=0s       p(99.9)=26.48ms 
     http_req_duration..............: avg=308.99ms min=2.86ms  med=286.55ms max=1.1s     p(90)=610.44ms p(95)=688.66ms p(99.9)=976.93ms
       { expected_response:true }...: avg=308.99ms min=2.86ms  med=286.55ms max=1.1s     p(90)=610.44ms p(95)=688.66ms p(99.9)=976.93ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 45450
     http_req_receiving.............: avg=475.76µs min=27.05µs med=59.14µs  max=165.03ms p(90)=213.8µs  p(95)=476.45µs p(99.9)=64.01ms 
     http_req_sending...............: avg=154.49µs min=5.54µs  med=11.28µs  max=145.69ms p(90)=31.42µs  p(95)=129.55µs p(99.9)=32.27ms 
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=308.36ms min=2.78ms  med=285.77ms max=1.1s     p(90)=609.35ms p(95)=687.33ms p(99.9)=974.54ms
     http_reqs......................: 45450   713.120644/s
     iteration_duration.............: avg=316.38ms min=3.13ms  med=294.32ms max=1.14s    p(90)=614.16ms p(95)=691.67ms p(99.9)=979.14ms
     iterations.....................: 44450   697.430421/s
     success_rate...................: 100.00% ✓ 44450      ✗ 0    
     vus............................: 94      min=0        max=500
     vus_max........................: 500     min=500      max=500

Performance Overview

HTTP Overview

Summary for: `grafbase`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 94083      ✗ 0    
     data_received..................: 2.8 GB  45 MB/s
     data_sent......................: 38 MB   595 kB/s
     http_req_blocked...............: avg=230.86µs min=1.29µs  med=3.22µs   max=206.31ms p(90)=4.87µs   p(95)=9.11µs   p(99.9)=69.08ms
     http_req_connecting............: avg=222.42µs min=0s      med=0s       max=206.24ms p(90)=0s       p(95)=0s       p(99.9)=65.85ms
     http_req_duration..............: avg=434.58ms min=2.54ms  med=420.49ms max=1.53s    p(90)=874.85ms p(95)=969.44ms p(99.9)=1.27s  
       { expected_response:true }...: avg=434.58ms min=2.54ms  med=420.49ms max=1.53s    p(90)=874.85ms p(95)=969.44ms p(99.9)=1.27s  
     http_req_failed................: 0.00%   ✓ 0          ✗ 32361
     http_req_receiving.............: avg=273.61µs min=27.96µs med=52.9µs   max=196.85ms p(90)=164.07µs p(95)=470.72µs p(99.9)=43.15ms
     http_req_sending...............: avg=247.75µs min=5.81µs  med=12.3µs   max=177.57ms p(90)=41.03µs  p(95)=136.54µs p(99.9)=46.3ms 
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s     
     http_req_waiting...............: avg=434.06ms min=2.5ms   med=419.74ms max=1.53s    p(90)=874.22ms p(95)=968.97ms p(99.9)=1.27s  
     http_reqs......................: 32361   510.789321/s
     iteration_duration.............: avg=449.48ms min=2.97ms  med=439.59ms max=1.58s    p(90)=880.84ms p(95)=974.62ms p(99.9)=1.27s  
     iterations.....................: 31361   495.005219/s
     success_rate...................: 100.00% ✓ 31361      ✗ 0    
     vus............................: 76      min=0        max=496
     vus_max........................: 500     min=500      max=500

Performance Overview

HTTP Overview

Summary for: `apollo-router`

K6 Output

     ✓ response code was 200
     ✗ no graphql errors
      ↳  99% — ✓ 26244 / ✗ 10
     ✓ valid response structure

     checks.........................: 99.98% ✓ 78752      ✗ 10   
     data_received..................: 2.4 GB 36 MB/s
     data_sent......................: 32 MB  474 kB/s
     http_req_blocked...............: avg=75.33µs  min=1.49µs  med=2.9µs    max=89.9ms  p(90)=4.57µs   p(95)=6.71µs   p(99.9)=24.86ms
     http_req_connecting............: avg=70.03µs  min=0s      med=0s       max=89.83ms p(90)=0s       p(95)=0s       p(99.9)=24.72ms
     http_req_duration..............: avg=517.48ms min=6.11ms  med=479.01ms max=1.74s   p(90)=1.05s    p(95)=1.17s    p(99.9)=1.63s  
       { expected_response:true }...: avg=517.48ms min=6.11ms  med=479.01ms max=1.74s   p(90)=1.05s    p(95)=1.17s    p(99.9)=1.63s  
     http_req_failed................: 0.00%  ✓ 0          ✗ 27254
     http_req_receiving.............: avg=110.98µs min=27.88µs med=51.2µs   max=81.47ms p(90)=100.62µs p(95)=170.62µs p(99.9)=10.09ms
     http_req_sending...............: avg=98.16µs  min=6.26µs  med=11.46µs  max=85.22ms p(90)=27.39µs  p(95)=112.49µs p(99.9)=19.27ms
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s      p(90)=0s       p(95)=0s       p(99.9)=0s     
     http_req_waiting...............: avg=517.27ms min=6.06ms  med=478.85ms max=1.74s   p(90)=1.05s    p(95)=1.17s    p(99.9)=1.63s  
     http_reqs......................: 27254  406.780457/s
     iteration_duration.............: avg=537.55ms min=6.33ms  med=503ms    max=1.74s   p(90)=1.06s    p(95)=1.18s    p(99.9)=1.63s  
     iterations.....................: 26254  391.854924/s
     success_rate...................: 99.96% ✓ 26244      ✗ 10   
     vus............................: 62     min=0        max=497
     vus_max........................: 500    min=500      max=500

Performance Overview

HTTP Overview

Summary for: `hive-gateway-bun`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 29451      ✗ 0    
     data_received..................: 950 MB  14 MB/s
     data_sent......................: 13 MB   186 kB/s
     http_req_blocked...............: avg=51.53µs min=1.62µs  med=3.62µs  max=36.27ms p(90)=5.6µs    p(95)=22.19µs  p(99.9)=9.97ms
     http_req_connecting............: avg=44.65µs min=0s      med=0s      max=36.21ms p(90)=0s       p(95)=0s       p(99.9)=9.92ms
     http_req_duration..............: avg=1.23s   min=7.19ms  med=1.21s   max=12.88s  p(90)=2.26s    p(95)=2.48s    p(99.9)=11.83s
       { expected_response:true }...: avg=1.23s   min=7.19ms  med=1.21s   max=12.88s  p(90)=2.26s    p(95)=2.48s    p(99.9)=11.83s
     http_req_failed................: 0.00%   ✓ 0          ✗ 10817
     http_req_receiving.............: avg=95.73µs min=30.18µs med=59.63µs max=43.44ms p(90)=109.49µs p(95)=159.26µs p(99.9)=3.36ms
     http_req_sending...............: avg=74.89µs min=6.93µs  med=13.6µs  max=51.24ms p(90)=35.46µs  p(95)=60.36µs  p(99.9)=14.2ms
     http_req_tls_handshaking.......: avg=0s      min=0s      med=0s      max=0s      p(90)=0s       p(95)=0s       p(99.9)=0s    
     http_req_waiting...............: avg=1.23s   min=7.12ms  med=1.21s   max=12.88s  p(90)=2.26s    p(95)=2.48s    p(99.9)=11.83s
     http_reqs......................: 10817   155.824425/s
     iteration_duration.............: avg=1.35s   min=8.17ms  med=1.35s   max=12.9s   p(90)=2.29s    p(95)=2.51s    p(99.9)=12.06s
     iterations.....................: 9817    141.418913/s
     success_rate...................: 100.00% ✓ 9817       ✗ 0    
     vus............................: 94      min=0        max=500
     vus_max........................: 500     min=500      max=500

Performance Overview

HTTP Overview

Summary for: `hive-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 27903      ✗ 0    
     data_received..................: 905 MB  13 MB/s
     data_sent......................: 12 MB   178 kB/s
     http_req_blocked...............: avg=53.95µs min=1.52µs  med=3.4µs   max=34.46ms p(90)=5.74µs   p(95)=23.46µs  p(99.9)=11.43ms
     http_req_connecting............: avg=46.17µs min=0s      med=0s      max=33.97ms p(90)=0s       p(95)=0s       p(99.9)=11.38ms
     http_req_duration..............: avg=1.28s   min=6.97ms  med=1.29s   max=13.95s  p(90)=2.33s    p(95)=2.54s    p(99.9)=12.85s 
       { expected_response:true }...: avg=1.28s   min=6.97ms  med=1.29s   max=13.95s  p(90)=2.33s    p(95)=2.54s    p(99.9)=12.85s 
     http_req_failed................: 0.00%   ✓ 0          ✗ 10301
     http_req_receiving.............: avg=91.46µs min=28.71µs med=61.67µs max=18.79ms p(90)=112.15µs p(95)=156.08µs p(99.9)=3.1ms  
     http_req_sending...............: avg=50.04µs min=6.92µs  med=13.49µs max=20.28ms p(90)=35.4µs   p(95)=58.14µs  p(99.9)=7.29ms 
     http_req_tls_handshaking.......: avg=0s      min=0s      med=0s      max=0s      p(90)=0s       p(95)=0s       p(99.9)=0s     
     http_req_waiting...............: avg=1.28s   min=6.92ms  med=1.29s   max=13.95s  p(90)=2.33s    p(95)=2.54s    p(99.9)=12.85s 
     http_reqs......................: 10301   149.072777/s
     iteration_duration.............: avg=1.42s   min=8.25ms  med=1.44s   max=14.02s  p(90)=2.36s    p(95)=2.58s    p(99.9)=12.95s 
     iterations.....................: 9301    134.601097/s
     success_rate...................: 100.00% ✓ 9301       ✗ 0    
     vus............................: 83      min=0        max=500
     vus_max........................: 500     min=500      max=500

Performance Overview

HTTP Overview

Copilot

Pull Request Overview

This PR introduces a comprehensive website for displaying GraphQL Federation Gateway performance benchmarks. The website allows users to compare different GraphQL gateways (Apollo, Cosmo, Grafbase, and Hive) across various performance metrics through interactive charts and data tables.

New React-based website: Complete frontend application built with React, TypeScript, and Tailwind CSS
Interactive benchmark visualization: Charts showing RPS, latency, CPU, memory, and reliability metrics over time
Data-driven comparison tables: Summary tables with performance statistics and compatibility scores

Reviewed Changes

Copilot reviewed 111 out of 152 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
website/src/main.tsx	React application entry point with root rendering
website/src/lib/data.ts	Core data processing logic for CSV benchmark data
website/src/lib/utils.ts	Utility functions for CSS class management
website/src/components/ui/*	Reusable UI components (charts, tables, cards, buttons)
website/src/components/*-chart.tsx	Specialized chart components for different metrics
website/src/components/intro.tsx	Summary tables and test descriptions
website/src/app.tsx	Main application component with test selection and metric visualization
website/src/index.css	Tailwind CSS configuration and custom styling
website/package.json	Project dependencies and build scripts
website/index.html	HTML template with meta tags and SEO configuration
website/constant-data.csv	Benchmark data for constant load testing

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

website/src/lib/data.ts

website/src/components/intro.tsx

gateways/mercurius/install.sh

grafana/json-dashboards/cadvisor-exporter.json

ardatan marked this pull request as ready for review September 3, 2025 13:29

ardatan requested a review from Copilot September 3, 2025 13:53

Copilot AI reviewed Sep 3, 2025

View reviewed changes

website/src/lib/data.ts Show resolved Hide resolved

website/src/lib/data.ts Show resolved Hide resolved

website/src/components/intro.tsx Outdated Show resolved Hide resolved

website/src/components/intro.tsx Outdated Show resolved Hide resolved

Urigo requested review from dotansimha and kamilkisiela September 3, 2025 14:03

ardatan changed the title ~~New benchmarks~~ New benchmark suite Sep 3, 2025

ardatan added 20 commits September 3, 2025 21:47

New benchmarks

d9ab453

Go

5e8407c

..

b012bd3

Make install

13c5293

Wait for

efcc34a

Maibn

5715da5

Trigger

b4115d7

Only install the gw

df41160

Wait subgraphs just before the actual run

4436027

No need to log in to the registry

7a84c10

..

5b76ab5

More timeout

f52dc72

..

28591a0

LFG

2f0c771

LFG

01e8f78

Run bun

0dd3c14

Fix bun

58ce8ba

Trigger

b2b5253

Try again

9616fa8

Go

eeb4aad

ardatan added 16 commits September 3, 2025 21:47

Double JIT

4aa1b51

Lets go

c2587bf

Lets go

2113408

Lets go

6366ae4

Toolkit

9ed1a20

Lets go

716b70a

Lets go

b853167

Lets go

f83d2db

Stats json

729a390

Lets go

9e3bc55

Generate report do that

e41c7fd

lETS go

818a269

Lets go

81c4649

..

b2df1ed

Lets go

fb97cde

Lets go

aa047ac

ardatan force-pushed the new-- branch from eda523f to aa047ac Compare September 3, 2025 18:47

ardatan and others added 6 commits September 3, 2025 21:48

Bump Hive GW

12cdec5

Runner

4ea580c

added website pipeline

006a3d3

now with the file

6031236

fix grafana screenshot

bd85f14

latest hive-router, fix screenshots again

9347942

dotansimha approved these changes Sep 4, 2025

View reviewed changes

kamilkisiela reviewed Sep 4, 2025

View reviewed changes

gateways/mercurius/install.sh Outdated Show resolved Hide resolved

kamilkisiela reviewed Sep 4, 2025

View reviewed changes

grafana/json-dashboards/cadvisor-exporter.json Outdated Show resolved Hide resolved

dotansimha added 2 commits September 4, 2025 11:30

remove mercurius, remove apollo-gateway (deprecated), no yarn

4ac2961

remove cadvisor

6d18ca1

dotansimha merged commit 73efd34 into main Sep 4, 2025
29 checks passed

dotansimha deleted the new-- branch September 4, 2025 08:46

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

New benchmark suite #761

New benchmark suite #761

Uh oh!

ardatan commented Sep 2, 2025

Uh oh!

github-actions bot commented Sep 2, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Sep 2, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Sep 2, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Sep 2, 2025 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

New benchmark suite #761

New benchmark suite #761

Uh oh!

Conversation

ardatan commented Sep 2, 2025

Uh oh!

github-actions bot commented Sep 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview for: constant-vus-subgraphs-delay

Comparison

Uh oh!

github-actions bot commented Sep 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview for: constant-vus-subgraphs-delay-resources

Comparison

Uh oh!

github-actions bot commented Sep 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview for: constant-vus-over-time

Comparison

Uh oh!

github-actions bot commented Sep 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview for: ramping-vus

Comparison

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

github-actions bot commented Sep 2, 2025 •

edited

Loading

Overview for: `constant-vus-subgraphs-delay`

github-actions bot commented Sep 2, 2025 •

edited

Loading

Overview for: `constant-vus-subgraphs-delay-resources`

github-actions bot commented Sep 2, 2025 •

edited

Loading

Overview for: `constant-vus-over-time`

github-actions bot commented Sep 2, 2025 •

edited

Loading

Overview for: `ramping-vus`