Skip to content

Conversation

ardatan
Copy link
Member

@ardatan ardatan commented Sep 4, 2025

No description provided.

Copy link

github-actions bot commented Sep 4, 2025

Overview for: constant-vus-over-time

This scenario runs 4 subgraphs and a GraphQL gateway with Federation spec, and runs a heavy query. It's being executed with a constant amount of VUs over a fixed amount of time. It measure things like memory usage, CPU usage, average RPS. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was running 50 VUs over 60s

Comparison

Comparison

Gateway RPS ⬇️ Requests Duration Notes
hive-router 1905 114791 total, 0 failed avg: 26ms, p95: 46ms
cosmo 640 38720 total, 0 failed avg: 77ms, p95: 112ms
grafbase 534 32287 total, 0 failed avg: 93ms, p95: 121ms
apollo-router 366 22302 total, 0 failed avg: 134ms, p95: 176ms
hive-gateway 170 10410 total, 0 failed avg: 288ms, p95: 430ms
hive-gateway-bun 166 10171 total, 0 failed avg: 295ms, p95: 424ms
apollo-gateway 123 7538 total, 0 failed avg: 399ms, p95: 468ms
Summary for: `hive-router`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 344073      ✗ 0     
     data_received..................: 10 GB   167 MB/s
     data_sent......................: 134 MB  2.2 MB/s
     http_req_blocked...............: avg=6.01µs   min=1.15µs  med=2.5µs   max=14.22ms  p(90)=3.62µs   p(95)=4.27µs   p(99.9)=57.27µs
     http_req_connecting............: avg=2.95µs   min=0s      med=0s      max=14.19ms  p(90)=0s       p(95)=0s       p(99.9)=0s     
     http_req_duration..............: avg=25.92ms  min=1.86ms  med=23.55ms max=426.18ms p(90)=40.13ms  p(95)=46.33ms  p(99.9)=80.61ms
       { expected_response:true }...: avg=25.92ms  min=1.86ms  med=23.55ms max=426.18ms p(90)=40.13ms  p(95)=46.33ms  p(99.9)=80.61ms
     http_req_failed................: 0.00%   ✓ 0           ✗ 114791
     http_req_receiving.............: avg=112.47µs min=22.97µs med=41.96µs max=53.4ms   p(90)=106.46µs p(95)=328.72µs p(99.9)=11.61ms
     http_req_sending...............: avg=82.15µs  min=5.18µs  med=9.95µs  max=281.99ms p(90)=29.04µs  p(95)=120.44µs p(99.9)=10.83ms
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s      max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s     
     http_req_waiting...............: avg=25.73ms  min=1.82ms  med=23.4ms  max=415.14ms p(90)=39.75ms  p(95)=45.98ms  p(99.9)=78.8ms 
     http_reqs......................: 114791  1905.897554/s
     iteration_duration.............: avg=26.14ms  min=4.33ms  med=23.74ms max=455.15ms p(90)=40.33ms  p(95)=46.54ms  p(99.9)=80.85ms
     iterations.....................: 114691  1904.237234/s
     success_rate...................: 100.00% ✓ 114691      ✗ 0     
     vus............................: 50      min=50        max=50  
     vus_max........................: 50      min=50        max=50  

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `cosmo`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 115860     ✗ 0    
     data_received..................: 3.4 GB  56 MB/s
     data_sent......................: 45 MB   745 kB/s
     http_req_blocked...............: avg=10.25µs  min=1.29µs  med=2.78µs  max=9.87ms   p(90)=4.33µs   p(95)=5.07µs   p(99.9)=2.49ms  
     http_req_connecting............: avg=6.69µs   min=0s      med=0s      max=9.83ms   p(90)=0s       p(95)=0s       p(99.9)=2.29ms  
     http_req_duration..............: avg=77.22ms  min=2.87ms  med=76.32ms max=515.55ms p(90)=104.41ms p(95)=112.25ms p(99.9)=295.93ms
       { expected_response:true }...: avg=77.22ms  min=2.87ms  med=76.32ms max=515.55ms p(90)=104.41ms p(95)=112.25ms p(99.9)=295.93ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 38720
     http_req_receiving.............: avg=168.33µs min=28.43µs med=76.65µs max=70.26ms  p(90)=153.89µs p(95)=377.95µs p(99.9)=15.63ms 
     http_req_sending...............: avg=43.02µs  min=5.75µs  med=11.44µs max=116.77ms p(90)=28.95µs  p(95)=122.37µs p(99.9)=2.49ms  
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s      max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=77ms     min=2.78ms  med=76.11ms max=503.7ms  p(90)=104.18ms p(95)=112.07ms p(99.9)=295.64ms
     http_reqs......................: 38720   640.819788/s
     iteration_duration.............: avg=77.69ms  min=6.62ms  med=76.6ms  max=544.16ms p(90)=104.68ms p(95)=112.55ms p(99.9)=321.47ms
     iterations.....................: 38620   639.164779/s
     success_rate...................: 100.00% ✓ 38620      ✗ 0    
     vus............................: 50      min=50       max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `grafbase`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 96561      ✗ 0    
     data_received..................: 2.8 GB  47 MB/s
     data_sent......................: 38 MB   622 kB/s
     http_req_blocked...............: avg=18.64µs  min=1.49µs  med=3.64µs  max=14.9ms   p(90)=5.94µs   p(95)=8.58µs   p(99.9)=5.67ms  
     http_req_connecting............: avg=12.13µs  min=0s      med=0s      max=14.86ms  p(90)=0s       p(95)=0s       p(99.9)=5.64ms  
     http_req_duration..............: avg=92.57ms  min=2.57ms  med=91.31ms max=491.01ms p(90)=114.08ms p(95)=121.17ms p(99.9)=311.42ms
       { expected_response:true }...: avg=92.57ms  min=2.57ms  med=91.31ms max=491.01ms p(90)=114.08ms p(95)=121.17ms p(99.9)=311.42ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 32287
     http_req_receiving.............: avg=180.26µs min=30.25µs med=70.62µs max=24.1ms   p(90)=204.23µs p(95)=461.24µs p(99.9)=9.97ms  
     http_req_sending...............: avg=110.03µs min=6.19µs  med=13.81µs max=182.13ms p(90)=117.9µs  p(95)=146.8µs  p(99.9)=9.66ms  
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s      max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=92.28ms  min=2.53ms  med=91.08ms max=481.1ms  p(90)=113.76ms p(95)=120.64ms p(99.9)=293.53ms
     http_reqs......................: 32287   534.519095/s
     iteration_duration.............: avg=93.24ms  min=24.41ms med=91.75ms max=526.84ms p(90)=114.45ms p(95)=121.6ms  p(99.9)=333.55ms
     iterations.....................: 32187   532.863571/s
     success_rate...................: 100.00% ✓ 32187      ✗ 0    
     vus............................: 50      min=50       max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-router`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 66606      ✗ 0    
     data_received..................: 2.0 GB  32 MB/s
     data_sent......................: 26 MB   427 kB/s
     http_req_blocked...............: avg=21.14µs  min=1.87µs  med=3.02µs   max=14.64ms  p(90)=4.73µs   p(95)=5.46µs   p(99.9)=8.35ms  
     http_req_connecting............: avg=17.34µs  min=0s      med=0s       max=14.6ms   p(90)=0s       p(95)=0s       p(99.9)=8.33ms  
     http_req_duration..............: avg=134.35ms min=6.1ms   med=133.52ms max=575.01ms p(90)=166ms    p(95)=175.95ms p(99.9)=419.44ms
       { expected_response:true }...: avg=134.35ms min=6.1ms   med=133.52ms max=575.01ms p(90)=166ms    p(95)=175.95ms p(99.9)=419.44ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 22302
     http_req_receiving.............: avg=94.87µs  min=37.88µs med=56.35µs  max=296.91ms p(90)=105.4µs  p(95)=135.2µs  p(99.9)=1.91ms  
     http_req_sending...............: avg=91.36µs  min=8.37µs  med=12.41µs  max=340.16ms p(90)=21.43µs  p(95)=36.56µs  p(99.9)=9.83ms  
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=134.16ms min=5.98ms  med=133.4ms  max=550.67ms p(90)=165.85ms p(95)=175.79ms p(99.9)=403.24ms
     http_reqs......................: 22302   366.938385/s
     iteration_duration.............: avg=135.23ms min=21.38ms med=133.88ms max=650.51ms p(90)=166.31ms p(95)=176.29ms p(99.9)=434.63ms
     iterations.....................: 22202   365.293069/s
     success_rate...................: 100.00% ✓ 22202      ✗ 0    
     vus............................: 50      min=50       max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 30930      ✗ 0    
     data_received..................: 914 MB  15 MB/s
     data_sent......................: 12 MB   198 kB/s
     http_req_blocked...............: avg=67.43µs  min=1.72µs  med=4.11µs   max=18.92ms  p(90)=5.78µs   p(95)=6.63µs   p(99.9)=16.11ms 
     http_req_connecting............: avg=61.74µs  min=0s      med=0s       max=18.89ms  p(90)=0s       p(95)=0s       p(99.9)=16.08ms 
     http_req_duration..............: avg=288.25ms min=6.93ms  med=275.79ms max=770.8ms  p(90)=385.51ms p(95)=430.32ms p(99.9)=616.58ms
       { expected_response:true }...: avg=288.25ms min=6.93ms  med=275.79ms max=770.8ms  p(90)=385.51ms p(95)=430.32ms p(99.9)=616.58ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 10410
     http_req_receiving.............: avg=129.75µs min=35.14µs med=78.23µs  max=323.24ms p(90)=128.85µs p(95)=154.05µs p(99.9)=1.72ms  
     http_req_sending...............: avg=80.54µs  min=7.3µs   med=16.28µs  max=322.63ms p(90)=24.64µs  p(95)=40.27µs  p(99.9)=2.81ms  
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=288.04ms min=6.88ms  med=275.63ms max=733.91ms p(90)=385.18ms p(95)=429.86ms p(99.9)=601.82ms
     http_reqs......................: 10410   170.160492/s
     iteration_duration.............: avg=291.6ms  min=61.51ms med=276.63ms max=823.53ms p(90)=386.97ms p(95)=431.47ms p(99.9)=720.24ms
     iterations.....................: 10310   168.525905/s
     success_rate...................: 100.00% ✓ 10310      ✗ 0    
     vus............................: 27      min=27       max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway-bun`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 30213      ✗ 0    
     data_received..................: 893 MB  15 MB/s
     data_sent......................: 12 MB   193 kB/s
     http_req_blocked...............: avg=64.83µs  min=1.91µs  med=3.9µs    max=20.11ms  p(90)=5.52µs   p(95)=6.2µs    p(99.9)=17.65ms 
     http_req_connecting............: avg=59.88µs  min=0s      med=0s       max=20.08ms  p(90)=0s       p(95)=0s       p(99.9)=17.6ms  
     http_req_duration..............: avg=294.91ms min=6.98ms  med=283.59ms max=721.81ms p(90)=385.5ms  p(95)=423.71ms p(99.9)=650.09ms
       { expected_response:true }...: avg=294.91ms min=6.98ms  med=283.59ms max=721.81ms p(90)=385.5ms  p(95)=423.71ms p(99.9)=650.09ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 10171
     http_req_receiving.............: avg=87.57µs  min=34.09µs med=68.2µs   max=12.52ms  p(90)=115.75µs p(95)=135.9µs  p(99.9)=2.11ms  
     http_req_sending...............: avg=94.12µs  min=8.16µs  med=15.18µs  max=199.8ms  p(90)=22.7µs   p(95)=39.66µs  p(99.9)=9.7ms   
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=294.73ms min=6.91ms  med=283.39ms max=721.64ms p(90)=385.28ms p(95)=423.55ms p(99.9)=649.87ms
     http_reqs......................: 10171   166.294652/s
     iteration_duration.............: avg=298.35ms min=31.43ms med=284.43ms max=782.43ms p(90)=386.91ms p(95)=425.3ms  p(99.9)=723.23ms
     iterations.....................: 10071   164.659663/s
     success_rate...................: 100.00% ✓ 10071      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 22314      ✗ 0   
     data_received..................: 662 MB  11 MB/s
     data_sent......................: 8.8 MB  143 kB/s
     http_req_blocked...............: avg=41.27µs  min=1.32µs   med=2.95µs   max=11.5ms   p(90)=3.98µs   p(95)=4.63µs   p(99.9)=10.02ms 
     http_req_connecting............: avg=37.99µs  min=0s       med=0s       max=11.47ms  p(90)=0s       p(95)=0s       p(99.9)=10ms    
     http_req_duration..............: avg=398.8ms  min=7.41ms   med=399.16ms max=763.46ms p(90)=451.93ms p(95)=468.34ms p(99.9)=680.74ms
       { expected_response:true }...: avg=398.8ms  min=7.41ms   med=399.16ms max=763.46ms p(90)=451.93ms p(95)=468.34ms p(99.9)=680.74ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 7538
     http_req_receiving.............: avg=80.81µs  min=28.77µs  med=50.37µs  max=171.95ms p(90)=84.22µs  p(95)=97.28µs  p(99.9)=318.86µs
     http_req_sending...............: avg=95.14µs  min=6.01µs   med=11.63µs  max=198.54ms p(90)=15.36µs  p(95)=19.61µs  p(99.9)=5.43ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=398.62ms min=7.36ms   med=399.05ms max=762.49ms p(90)=451.87ms p(95)=468.16ms p(99.9)=673.27ms
     http_reqs......................: 7538    123.340179/s
     iteration_duration.............: avg=404.39ms min=133.28ms med=399.76ms max=781.87ms p(90)=452.5ms  p(95)=468.75ms p(99.9)=701.61ms
     iterations.....................: 7438    121.703934/s
     success_rate...................: 100.00% ✓ 7438       ✗ 0   
     vus............................: 30      min=30       max=50
     vus_max........................: 50      min=50       max=50

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview

Copy link

github-actions bot commented Sep 4, 2025

Overview for: ramping-vus

This scenario runs 4 subgraphs and a GraphQL gateway with Federation spec, and runs a heavy query. We are running a heavy load of concurrent VUs to measure response time and other stats, during stress. It measure things like memory usage, CPU usage, response times. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was running 500 VUs over 60s

Comparison

Comparison

Gateway RPS ⬇️ Requests Duration Notes
hive-router 1963 122156 total, 0 failed avg: 114ms, p95: 281ms
cosmo 658 42223 total, 0 failed avg: 333ms, p95: 743ms
grafbase 502 31839 total, 0 failed avg: 442ms, p95: 979ms
apollo-router 392 26340 total, 0 failed avg: 535ms, p95: 1204ms ❌ 24 unexpected GraphQL errors
hive-gateway 162 11153 total, 0 failed avg: 1201ms, p95: 2398ms
hive-gateway-bun 161 11144 total, 0 failed avg: 1201ms, p95: 2386ms
apollo-gateway 154 10563 total, 0 failed avg: 1232ms, p95: 2622ms
Summary for: `hive-router`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 363468      ✗ 0     
     data_received..................: 11 GB   172 MB/s
     data_sent......................: 142 MB  2.3 MB/s
     http_req_blocked...............: avg=391.31µs min=1.14µs  med=2.53µs   max=311.52ms p(90)=3.94µs   p(95)=4.67µs   p(99.9)=146.39ms
     http_req_connecting............: avg=387.49µs min=0s      med=0s       max=311.48ms p(90)=0s       p(95)=0s       p(99.9)=146.32ms
     http_req_duration..............: avg=113.92ms min=1.67ms  med=102.37ms max=415.08ms p(90)=234.02ms p(95)=281.47ms p(99.9)=372.37ms
       { expected_response:true }...: avg=113.92ms min=1.67ms  med=102.37ms max=415.08ms p(90)=234.02ms p(95)=281.47ms p(99.9)=372.37ms
     http_req_failed................: 0.00%   ✓ 0           ✗ 122156
     http_req_receiving.............: avg=474.22µs min=22.71µs med=41.58µs  max=153.4ms  p(90)=143.77µs p(95)=390.43µs p(99.9)=68.33ms 
     http_req_sending...............: avg=344.05µs min=5.18µs  med=10.12µs  max=116.76ms p(90)=27.27µs  p(95)=127.44µs p(99.9)=62.99ms 
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=113.11ms min=1.62ms  med=101.58ms max=393.9ms  p(90)=232.05ms p(95)=279.74ms p(99.9)=369.44ms
     http_reqs......................: 122156  1963.741841/s
     iteration_duration.............: avg=115.5ms  min=1.8ms   med=103.71ms max=688.18ms p(90)=236.11ms p(95)=283.58ms p(99.9)=385.55ms
     iterations.....................: 121156  1947.666152/s
     success_rate...................: 100.00% ✓ 121156      ✗ 0     
     vus............................: 70      min=0         max=495 
     vus_max........................: 500     min=500       max=500 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `cosmo`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 123669     ✗ 0    
     data_received..................: 3.7 GB  58 MB/s
     data_sent......................: 49 MB   766 kB/s
     http_req_blocked...............: avg=123.37µs min=1.34µs  med=3.02µs   max=214.16ms p(90)=4.96µs   p(95)=7.1µs    p(99.9)=30.27ms 
     http_req_connecting............: avg=117.06µs min=0s      med=0s       max=213.98ms p(90)=0s       p(95)=0s       p(99.9)=30.18ms 
     http_req_duration..............: avg=332.89ms min=2.88ms  med=314.37ms max=1.29s    p(90)=657.55ms p(95)=742.92ms p(99.9)=1.05s   
       { expected_response:true }...: avg=332.89ms min=2.88ms  med=314.37ms max=1.29s    p(90)=657.55ms p(95)=742.92ms p(99.9)=1.05s   
     http_req_failed................: 0.00%   ✓ 0          ✗ 42223
     http_req_receiving.............: avg=1.43ms   min=29.66µs med=64.66µs  max=434.04ms p(90)=194.5µs  p(95)=529.03µs p(99.9)=281.91ms
     http_req_sending...............: avg=174.09µs min=5.88µs  med=11.84µs  max=143.44ms p(90)=31.42µs  p(95)=130.61µs p(99.9)=32.81ms 
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=331.29ms min=2.76ms  med=313.01ms max=1.29s    p(90)=653.27ms p(95)=737.9ms  p(99.9)=1.05s   
     http_reqs......................: 42223   658.13241/s
     iteration_duration.............: avg=341.51ms min=3.22ms  med=323.43ms max=1.3s     p(90)=661.73ms p(95)=745.37ms p(99.9)=1.06s   
     iterations.....................: 41223   642.545351/s
     success_rate...................: 100.00% ✓ 41223      ✗ 0    
     vus............................: 68      min=0        max=496
     vus_max........................: 500     min=500      max=500

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `grafbase`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 92517      ✗ 0    
     data_received..................: 2.8 GB  44 MB/s
     data_sent......................: 37 MB   585 kB/s
     http_req_blocked...............: avg=139.54µs min=1.46µs  med=3.24µs   max=167.59ms p(90)=5.02µs   p(95)=9.14µs   p(99.9)=37.35ms
     http_req_connecting............: avg=130.46µs min=0s      med=0s       max=167.41ms p(90)=0s       p(95)=0s       p(99.9)=35.69ms
     http_req_duration..............: avg=442.11ms min=2.68ms  med=427.96ms max=1.43s    p(90)=878.97ms p(95)=978.87ms p(99.9)=1.3s   
       { expected_response:true }...: avg=442.11ms min=2.68ms  med=427.96ms max=1.43s    p(90)=878.97ms p(95)=978.87ms p(99.9)=1.3s   
     http_req_failed................: 0.00%   ✓ 0          ✗ 31839
     http_req_receiving.............: avg=199.47µs min=26.15µs med=53.04µs  max=82.76ms  p(90)=158.17µs p(95)=422.66µs p(99.9)=23.76ms
     http_req_sending...............: avg=163.66µs min=6.2µs   med=12.6µs   max=80.77ms  p(90)=39.36µs  p(95)=133.94µs p(99.9)=29.05ms
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s     
     http_req_waiting...............: avg=441.74ms min=2.64ms  med=427.32ms max=1.43s    p(90)=878.68ms p(95)=978.63ms p(99.9)=1.29s  
     http_reqs......................: 31839   502.226086/s
     iteration_duration.............: avg=457.21ms min=2.9ms   med=445.6ms  max=1.43s    p(90)=884.44ms p(95)=983.02ms p(99.9)=1.3s   
     iterations.....................: 30839   486.452159/s
     success_rate...................: 100.00% ✓ 30839      ✗ 0    
     vus............................: 82      min=0        max=498
     vus_max........................: 500     min=500      max=500

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-router`

K6 Output

     ✓ response code was 200
     ✗ no graphql errors
      ↳  99% — ✓ 25316 / ✗ 24
     ✓ valid response structure

     checks.........................: 99.96% ✓ 75996      ✗ 24   
     data_received..................: 2.3 GB 34 MB/s
     data_sent......................: 31 MB  458 kB/s
     http_req_blocked...............: avg=63.38µs  min=1.77µs  med=2.96µs   max=158.72ms p(90)=4.82µs  p(95)=7µs      p(99.9)=14.74ms
     http_req_connecting............: avg=58.37µs  min=0s      med=0s       max=158.65ms p(90)=0s      p(95)=0s       p(99.9)=14.68ms
     http_req_duration..............: avg=535.19ms min=6.14ms  med=496.41ms max=1.83s    p(90)=1.07s   p(95)=1.2s     p(99.9)=1.68s  
       { expected_response:true }...: avg=535.19ms min=6.14ms  med=496.41ms max=1.83s    p(90)=1.07s   p(95)=1.2s     p(99.9)=1.68s  
     http_req_failed................: 0.00%  ✓ 0          ✗ 26340
     http_req_receiving.............: avg=96.9µs   min=30.45µs med=53.75µs  max=150.75ms p(90)=102.1µs p(95)=156.59µs p(99.9)=3.41ms 
     http_req_sending...............: avg=82.19µs  min=7.33µs  med=11.77µs  max=49.97ms  p(90)=28.21µs p(95)=72.35µs  p(99.9)=13.5ms 
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s      p(95)=0s       p(99.9)=0s     
     http_req_waiting...............: avg=535.02ms min=6.09ms  med=496.33ms max=1.83s    p(90)=1.07s   p(95)=1.2s     p(99.9)=1.68s  
     http_reqs......................: 26340  392.615542/s
     iteration_duration.............: avg=556.68ms min=6.54ms  med=517.63ms max=1.83s    p(90)=1.08s   p(95)=1.21s    p(99.9)=1.69s  
     iterations.....................: 25340  377.709864/s
     success_rate...................: 99.90% ✓ 25316      ✗ 24   
     vus............................: 68     min=0        max=496
     vus_max........................: 500    min=500      max=500

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 30459      ✗ 0    
     data_received..................: 980 MB  14 MB/s
     data_sent......................: 13 MB   194 kB/s
     http_req_blocked...............: avg=58.22µs  min=1.8µs   med=3.76µs  max=31.17ms p(90)=5.97µs   p(95)=24.22µs  p(99.9)=13.91ms
     http_req_connecting............: avg=50.87µs  min=0s      med=0s      max=31.02ms p(90)=0s       p(95)=0s       p(99.9)=13.84ms
     http_req_duration..............: avg=1.2s     min=6.2ms   med=1.18s   max=12.87s  p(90)=2.21s    p(95)=2.39s    p(99.9)=11.55s 
       { expected_response:true }...: avg=1.2s     min=6.2ms   med=1.18s   max=12.87s  p(90)=2.21s    p(95)=2.39s    p(99.9)=11.55s 
     http_req_failed................: 0.00%   ✓ 0          ✗ 11153
     http_req_receiving.............: avg=100.36µs min=33.27µs med=61.29µs max=24.38ms p(90)=112.25µs p(95)=158.59µs p(99.9)=4.14ms 
     http_req_sending...............: avg=61.43µs  min=7µs     med=13.99µs max=30.13ms p(90)=35.46µs  p(95)=52.98µs  p(99.9)=11.88ms
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s      max=0s      p(90)=0s       p(95)=0s       p(99.9)=0s     
     http_req_waiting...............: avg=1.2s     min=6.15ms  med=1.18s   max=12.87s  p(90)=2.21s    p(95)=2.39s    p(99.9)=11.55s 
     http_reqs......................: 11153   162.57534/s
     iteration_duration.............: avg=1.31s    min=7.09ms  med=1.31s   max=12.88s  p(90)=2.24s    p(95)=2.42s    p(99.9)=11.63s 
     iterations.....................: 10153   147.998514/s
     success_rate...................: 100.00% ✓ 10153      ✗ 0    
     vus............................: 104     min=0        max=500
     vus_max........................: 500     min=500      max=500

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway-bun`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 30432      ✗ 0    
     data_received..................: 979 MB  14 MB/s
     data_sent......................: 13 MB   192 kB/s
     http_req_blocked...............: avg=74.24µs  min=1.75µs  med=3.74µs  max=40.45ms p(90)=6.16µs   p(95)=24.53µs  p(99.9)=19.55ms
     http_req_connecting............: avg=64.81µs  min=0s      med=0s      max=40.29ms p(90)=0s       p(95)=0s       p(99.9)=18.89ms
     http_req_duration..............: avg=1.2s     min=6.42ms  med=1.18s   max=12.71s  p(90)=2.19s    p(95)=2.38s    p(99.9)=11.17s 
       { expected_response:true }...: avg=1.2s     min=6.42ms  med=1.18s   max=12.71s  p(90)=2.19s    p(95)=2.38s    p(99.9)=11.17s 
     http_req_failed................: 0.00%   ✓ 0          ✗ 11144
     http_req_receiving.............: avg=102.21µs min=30.59µs med=67.57µs max=38.57ms p(90)=119.23µs p(95)=165.05µs p(99.9)=3.44ms 
     http_req_sending...............: avg=67.39µs  min=6.9µs   med=14.38µs max=35.43ms p(90)=35.58µs  p(95)=61.3µs   p(99.9)=11.48ms
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s      max=0s      p(90)=0s       p(95)=0s       p(99.9)=0s     
     http_req_waiting...............: avg=1.2s     min=6.35ms  med=1.18s   max=12.71s  p(90)=2.19s    p(95)=2.38s    p(99.9)=11.17s 
     http_reqs......................: 11144   161.469575/s
     iteration_duration.............: avg=1.31s    min=8.13ms  med=1.31s   max=12.73s  p(90)=2.22s    p(95)=2.41s    p(99.9)=11.35s 
     iterations.....................: 10144   146.980201/s
     success_rate...................: 100.00% ✓ 10144      ✗ 0    
     vus............................: 78      min=0        max=500
     vus_max........................: 500     min=500      max=500

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 28689      ✗ 0    
     data_received..................: 928 MB  14 MB/s
     data_sent......................: 13 MB   185 kB/s
     http_req_blocked...............: avg=37.76µs min=1.4µs   med=3.25µs  max=53.76ms p(90)=5.26µs  p(95)=11.07µs  p(99.9)=3.63ms
     http_req_connecting............: avg=31.74µs min=0s      med=0s      max=53.55ms p(90)=0s      p(95)=0s       p(99.9)=3.59ms
     http_req_duration..............: avg=1.23s   min=7.25ms  med=1.19s   max=15.86s  p(90)=2.21s   p(95)=2.62s    p(99.9)=14.35s
       { expected_response:true }...: avg=1.23s   min=7.25ms  med=1.19s   max=15.86s  p(90)=2.21s   p(95)=2.62s    p(99.9)=14.35s
     http_req_failed................: 0.00%   ✓ 0          ✗ 10563
     http_req_receiving.............: avg=73.5µs  min=27.36µs med=53.17µs max=6.87ms  p(90)=99.69µs p(95)=125.13µs p(99.9)=1.49ms
     http_req_sending...............: avg=34.8µs  min=6.02µs  med=12.47µs max=18.61ms p(90)=25.31µs p(95)=43.69µs  p(99.9)=4.16ms
     http_req_tls_handshaking.......: avg=0s      min=0s      med=0s      max=0s      p(90)=0s      p(95)=0s       p(99.9)=0s    
     http_req_waiting...............: avg=1.23s   min=7.21ms  med=1.19s   max=15.86s  p(90)=2.21s   p(95)=2.62s    p(99.9)=14.35s
     http_reqs......................: 10563   154.552987/s
     iteration_duration.............: avg=1.36s   min=8.03ms  med=1.34s   max=15.87s  p(90)=2.26s   p(95)=2.68s    p(99.9)=14.44s
     iterations.....................: 9563    139.921444/s
     success_rate...................: 100.00% ✓ 9563       ✗ 0    
     vus............................: 95      min=0        max=500
     vus_max........................: 500     min=500      max=500

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview

Copy link

github-actions bot commented Sep 4, 2025

Overview for: constant-vus-subgraphs-delay-resources

This scenario runs 4 subgraphs and a GraphQL gateway with Federation spec, and runs a heavy query. It's being executed with a constant amount of VUs over a fixed amount of time. It measure things like memory usage, CPU usage, average RPS. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was running 50 VUs over 60s

Comparison

Comparison

Gateway RPS ⬇️ Requests Duration Notes
hive-router 169 13754 total, 0 failed avg: 220ms, p95: 229ms
grafbase 168 13703 total, 0 failed avg: 221ms, p95: 233ms
cosmo 166 13506 total, 0 failed avg: 224ms, p95: 246ms
hive-gateway-bun 147 11267 total, 0 failed avg: 268ms, p95: 416ms
hive-gateway 144 11040 total, 0 failed avg: 273ms, p95: 428ms
apollo-router 136 11075 total, 0 failed avg: 273ms, p95: 313ms
apollo-gateway 118 9646 total, 0 failed avg: 313ms, p95: 348ms
Summary for: `hive-router`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 40962      ✗ 0    
     data_received..................: 1.2 GB  15 MB/s
     data_sent......................: 16 MB   198 kB/s
     http_req_blocked...............: avg=18.82µs  min=1.2µs    med=2.06µs   max=8.83ms   p(90)=3.14µs   p(95)=4.52µs   p(99.9)=6.46ms  
     http_req_connecting............: avg=16.25µs  min=0s       med=0s       max=8.79ms   p(90)=0s       p(95)=0s       p(99.9)=6.44ms  
     http_req_duration..............: avg=219.53ms min=161.65ms med=218.88ms max=591.22ms p(90)=226.78ms p(95)=229.25ms p(99.9)=498.04ms
       { expected_response:true }...: avg=219.53ms min=161.65ms med=218.88ms max=591.22ms p(90)=226.78ms p(95)=229.25ms p(99.9)=498.04ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 13754
     http_req_receiving.............: avg=72.85µs  min=23.85µs  med=41.28µs  max=16.94ms  p(90)=83.19µs  p(95)=246.6µs  p(99.9)=1.91ms  
     http_req_sending...............: avg=62.23µs  min=5.5µs    med=8.61µs   max=365.19ms p(90)=27.79µs  p(95)=92.16µs  p(99.9)=3.63ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=219.39ms min=161.6ms  med=218.81ms max=590.51ms p(90)=226.65ms p(95)=229.09ms p(99.9)=485.42ms
     http_reqs......................: 13754   169.884397/s
     iteration_duration.............: avg=219.84ms min=161.8ms  med=219.08ms max=610.64ms p(90)=226.94ms p(95)=229.41ms p(99.9)=521.67ms
     iterations.....................: 13654   168.649234/s
     success_rate...................: 100.00% ✓ 13654      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `grafbase`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 40809      ✗ 0    
     data_received..................: 1.2 GB  15 MB/s
     data_sent......................: 16 MB   197 kB/s
     http_req_blocked...............: avg=20.87µs  min=1.28µs   med=2.88µs   max=9.19ms   p(90)=4µs      p(95)=4.74µs   p(99.9)=6.78ms  
     http_req_connecting............: avg=17.02µs  min=0s       med=0s       max=9.16ms   p(90)=0s       p(95)=0s       p(99.9)=6.76ms  
     http_req_duration..............: avg=220.54ms min=205.22ms med=218.96ms max=662.22ms p(90)=229.81ms p(95)=233.23ms p(99.9)=540.43ms
       { expected_response:true }...: avg=220.54ms min=205.22ms med=218.96ms max=662.22ms p(90)=229.81ms p(95)=233.23ms p(99.9)=540.43ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 13703
     http_req_receiving.............: avg=83.27µs  min=25.82µs  med=52.17µs  max=126.36ms p(90)=93.22µs  p(95)=121.73µs p(99.9)=2.2ms   
     http_req_sending...............: avg=47.39µs  min=5.98µs   med=12.04µs  max=170.1ms  p(90)=18.24µs  p(95)=34.36µs  p(99.9)=2.62ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=220.41ms min=205.17ms med=218.86ms max=646.22ms p(90)=229.68ms p(95)=233.09ms p(99.9)=534.87ms
     http_reqs......................: 13703   168.971892/s
     iteration_duration.............: avg=220.92ms min=205.37ms med=219.25ms max=681.14ms p(90)=230.09ms p(95)=233.54ms p(99.9)=568.99ms
     iterations.....................: 13603   167.73879/s
     success_rate...................: 100.00% ✓ 13603      ✗ 0    
     vus............................: 32      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `cosmo`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 40218      ✗ 0    
     data_received..................: 1.2 GB  15 MB/s
     data_sent......................: 16 MB   193 kB/s
     http_req_blocked...............: avg=51.92µs  min=1.47µs   med=2.79µs   max=18.19ms  p(90)=4.39µs   p(95)=5.45µs   p(99.9)=14.81ms 
     http_req_connecting............: avg=48.06µs  min=0s       med=0s       max=18.04ms  p(90)=0s       p(95)=0s       p(99.9)=14.74ms 
     http_req_duration..............: avg=223.71ms min=167.39ms med=222.63ms max=613.99ms p(90)=240.52ms p(95)=246.11ms p(99.9)=512.52ms
       { expected_response:true }...: avg=223.71ms min=167.39ms med=222.63ms max=613.99ms p(90)=240.52ms p(95)=246.11ms p(99.9)=512.52ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 13506
     http_req_receiving.............: avg=508.48µs min=32.32µs  med=71.93µs  max=325.33ms p(90)=370.88µs p(95)=826.99µs p(99.9)=28.62ms 
     http_req_sending...............: avg=116.44µs min=5.79µs   med=11.44µs  max=336.67ms p(90)=31.47µs  p(95)=120.94µs p(99.9)=4.62ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=223.08ms min=167.27ms med=222.17ms max=609.66ms p(90)=239.9ms  p(95)=245.58ms p(99.9)=490.68ms
     http_reqs......................: 13506   166.225026/s
     iteration_duration.............: avg=224.16ms min=167.66ms med=222.96ms max=633.87ms p(90)=240.86ms p(95)=246.39ms p(99.9)=546.6ms 
     iterations.....................: 13406   164.994276/s
     success_rate...................: 100.00% ✓ 13406      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway-bun`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 33501      ✗ 0    
     data_received..................: 990 MB  13 MB/s
     data_sent......................: 13 MB   171 kB/s
     http_req_blocked...............: avg=28.37µs  min=1.5µs    med=3.61µs   max=10.26ms  p(90)=5.26µs   p(95)=6.26µs   p(99.9)=8.14ms  
     http_req_connecting............: avg=22.8µs   min=0s       med=0s       max=10.21ms  p(90)=0s       p(95)=0s       p(99.9)=8.12ms  
     http_req_duration..............: avg=267.71ms min=159.77ms med=252.74ms max=807.73ms p(90)=306.92ms p(95)=416.05ms p(99.9)=749.37ms
       { expected_response:true }...: avg=267.71ms min=159.77ms med=252.74ms max=807.73ms p(90)=306.92ms p(95)=416.05ms p(99.9)=749.37ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 11267
     http_req_receiving.............: avg=106.44µs min=31.22µs  med=73.82µs  max=15.13ms  p(90)=125.39µs p(95)=169.04µs p(99.9)=4.63ms  
     http_req_sending...............: avg=82.31µs  min=7.02µs   med=14.67µs  max=217.16ms p(90)=30.14µs  p(95)=46.05µs  p(99.9)=3.53ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=267.52ms min=159.59ms med=252.61ms max=806.59ms p(90)=306.43ms p(95)=415.93ms p(99.9)=749.28ms
     http_reqs......................: 11267   147.267303/s
     iteration_duration.............: avg=269.12ms min=171.92ms med=253.29ms max=837.13ms p(90)=309.23ms p(95)=416.87ms p(99.9)=767.92ms
     iterations.....................: 11167   145.960235/s
     success_rate...................: 100.00% ✓ 11167      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 32820      ✗ 0    
     data_received..................: 970 MB  13 MB/s
     data_sent......................: 13 MB   168 kB/s
     http_req_blocked...............: avg=29.25µs  min=1.42µs   med=3.69µs   max=10.48ms  p(90)=5.51µs   p(95)=6.57µs   p(99.9)=8.26ms  
     http_req_connecting............: avg=23.5µs   min=0s       med=0s       max=10.44ms  p(90)=0s       p(95)=0s       p(99.9)=8.19ms  
     http_req_duration..............: avg=273.13ms min=160.69ms med=259.43ms max=880.43ms p(90)=318.59ms p(95)=427.54ms p(99.9)=767.4ms 
       { expected_response:true }...: avg=273.13ms min=160.69ms med=259.43ms max=880.43ms p(90)=318.59ms p(95)=427.54ms p(99.9)=767.4ms 
     http_req_failed................: 0.00%   ✓ 0          ✗ 11040
     http_req_receiving.............: avg=117.04µs min=30.73µs  med=77.35µs  max=103.91ms p(90)=137.53µs p(95)=186.38µs p(99.9)=3.95ms  
     http_req_sending...............: avg=96.26µs  min=6.62µs   med=15.23µs  max=222.96ms p(90)=34.35µs  p(95)=52.58µs  p(99.9)=3.79ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=272.91ms min=160.58ms med=259.26ms max=878.93ms p(90)=318.37ms p(95)=427.02ms p(99.9)=766.59ms
     http_reqs......................: 11040   144.282313/s
     iteration_duration.............: avg=274.63ms min=186.48ms med=259.96ms max=939.74ms p(90)=319.85ms p(95)=428.83ms p(99.9)=819.22ms
     iterations.....................: 10940   142.975408/s
     success_rate...................: 100.00% ✓ 10940      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-router`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 32925      ✗ 0    
     data_received..................: 972 MB  12 MB/s
     data_sent......................: 13 MB   158 kB/s
     http_req_blocked...............: avg=57.17µs  min=1.26µs   med=2.7µs    max=19.64ms  p(90)=3.95µs   p(95)=4.83µs   p(99.9)=13.99ms 
     http_req_connecting............: avg=52.08µs  min=0s       med=0s       max=19.61ms  p(90)=0s       p(95)=0s       p(99.9)=13.97ms 
     http_req_duration..............: avg=272.78ms min=148.26ms med=273.42ms max=682.76ms p(90)=304.09ms p(95)=313.1ms  p(99.9)=590.27ms
       { expected_response:true }...: avg=272.78ms min=148.26ms med=273.42ms max=682.76ms p(90)=304.09ms p(95)=313.1ms  p(99.9)=590.27ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 11075
     http_req_receiving.............: avg=133.56µs min=31.95µs  med=51.94µs  max=115.62ms p(90)=157.03µs p(95)=432.51µs p(99.9)=5.16ms  
     http_req_sending...............: avg=105.79µs min=6.48µs   med=11.05µs  max=197.94ms p(90)=46.37µs  p(95)=128.68µs p(99.9)=5.8ms   
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=272.54ms min=145.66ms med=273.21ms max=661.58ms p(90)=303.67ms p(95)=312.73ms p(99.9)=571.84ms
     http_reqs......................: 11075   136.15131/s
     iteration_duration.............: avg=273.72ms min=148.43ms med=273.92ms max=710.67ms p(90)=304.39ms p(95)=313.43ms p(99.9)=619.74ms
     iterations.....................: 10975   134.921952/s
     success_rate...................: 100.00% ✓ 10975      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 28638      ✗ 0   
     data_received..................: 847 MB  10 MB/s
     data_sent......................: 11 MB   137 kB/s
     http_req_blocked...............: avg=78.49µs  min=1.42µs   med=4.35µs   max=23.94ms  p(90)=6.29µs   p(95)=7.45µs   p(99.9)=16.79ms 
     http_req_connecting............: avg=72.38µs  min=0s       med=0s       max=23.89ms  p(90)=0s       p(95)=0s       p(99.9)=16.76ms 
     http_req_duration..............: avg=313.13ms min=212.25ms med=310.4ms  max=840.16ms p(90)=337.4ms  p(95)=348.13ms p(99.9)=726.82ms
       { expected_response:true }...: avg=313.13ms min=212.25ms med=310.4ms  max=840.16ms p(90)=337.4ms  p(95)=348.13ms p(99.9)=726.82ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 9646
     http_req_receiving.............: avg=106.58µs min=33.18µs  med=87.39µs  max=5.8ms    p(90)=144.63µs p(95)=175.19µs p(99.9)=2.76ms  
     http_req_sending...............: avg=154.36µs min=6.46µs   med=17.64µs  max=232.48ms p(90)=33.68µs  p(95)=47µs     p(99.9)=5.6ms   
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=312.87ms min=212.15ms med=310.24ms max=838.53ms p(90)=337.18ms p(95)=347.59ms p(99.9)=725.4ms 
     http_reqs......................: 9646    118.106576/s
     iteration_duration.............: avg=314.7ms  min=218.63ms med=310.95ms max=859.84ms p(90)=337.97ms p(95)=348.66ms p(99.9)=783.79ms
     iterations.....................: 9546    116.882166/s
     success_rate...................: 100.00% ✓ 9546       ✗ 0   
     vus............................: 50      min=0        max=50
     vus_max........................: 50      min=50       max=50

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview

Copy link

github-actions bot commented Sep 4, 2025

Overview for: constant-vus-subgraphs-delay

This scenario runs 4 subgraphs and a GraphQL gateway with Federation spec, and runs a heavy query. It's being executed with a constant amount of VUs over a fixed amount of time. It measure things like memory usage, CPU usage, average RPS. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was running 50 VUs over 60s

Comparison

Comparison

Gateway RPS ⬇️ Requests Duration Notes
grafbase 168 13679 total, 0 failed avg: 221ms, p95: 233ms
hive-router 167 13538 total, 0 failed avg: 223ms, p95: 234ms
cosmo 156 12652 total, 0 failed avg: 239ms, p95: 267ms
hive-gateway 135 10413 total, 0 failed avg: 290ms, p95: 415ms
hive-gateway-bun 131 10047 total, 0 failed avg: 300ms, p95: 426ms
apollo-router 123 10031 total, 0 failed avg: 301ms, p95: 344ms ❌ 1 unexpected GraphQL errors
apollo-gateway 87 7149 total, 0 failed avg: 424ms, p95: 465ms
Summary for: `grafbase`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 40737      ✗ 0    
     data_received..................: 1.2 GB  15 MB/s
     data_sent......................: 16 MB   196 kB/s
     http_req_blocked...............: avg=45.84µs  min=1.43µs   med=3.04µs   max=22.18ms  p(90)=4.4µs    p(95)=5.4µs    p(99.9)=13.89ms 
     http_req_connecting............: avg=41.86µs  min=0s       med=0s       max=22.15ms  p(90)=0s       p(95)=0s       p(99.9)=13.83ms 
     http_req_duration..............: avg=220.8ms  min=205.55ms med=219.35ms max=614.01ms p(90)=229.42ms p(95)=232.68ms p(99.9)=508.55ms
       { expected_response:true }...: avg=220.8ms  min=205.55ms med=219.35ms max=614.01ms p(90)=229.42ms p(95)=232.68ms p(99.9)=508.55ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 13679
     http_req_receiving.............: avg=83.74µs  min=30.93µs  med=58.98µs  max=8.57ms   p(90)=105.19µs p(95)=136.32µs p(99.9)=2.1ms   
     http_req_sending...............: avg=95.49µs  min=6.25µs   med=12.56µs  max=217.04ms p(90)=22.46µs  p(95)=36.59µs  p(99.9)=7.5ms   
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=220.62ms min=205.27ms med=219.23ms max=613.5ms  p(90)=229.27ms p(95)=232.51ms p(99.9)=501.06ms
     http_reqs......................: 13679   168.69253/s
     iteration_duration.............: avg=221.22ms min=205.7ms  med=219.65ms max=660.84ms p(90)=229.68ms p(95)=233ms    p(99.9)=543.74ms
     iterations.....................: 13579   167.459307/s
     success_rate...................: 100.00% ✓ 13579      ✗ 0    
     vus............................: 24      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-router`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 40314      ✗ 0    
     data_received..................: 1.2 GB  15 MB/s
     data_sent......................: 16 MB   195 kB/s
     http_req_blocked...............: avg=32.36µs  min=1.14µs   med=2.19µs   max=14.08ms  p(90)=3.25µs   p(95)=4.52µs   p(99.9)=11.07ms 
     http_req_connecting............: avg=29.58µs  min=0s       med=0s       max=14.04ms  p(90)=0s       p(95)=0s       p(99.9)=11.04ms 
     http_req_duration..............: avg=223.2ms  min=161.09ms med=222.38ms max=595.07ms p(90)=230.97ms p(95)=233.56ms p(99.9)=502.72ms
       { expected_response:true }...: avg=223.2ms  min=161.09ms med=222.38ms max=595.07ms p(90)=230.97ms p(95)=233.56ms p(99.9)=502.72ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 13538
     http_req_receiving.............: avg=75.66µs  min=24.26µs  med=44.03µs  max=8.86ms   p(90)=89.28µs  p(95)=263.37µs p(99.9)=1.89ms  
     http_req_sending...............: avg=145.62µs min=5.56µs   med=9.53µs   max=333.05ms p(90)=30.1µs   p(95)=99.78µs  p(99.9)=7.48ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=222.98ms min=160.6ms  med=222.27ms max=589.67ms p(90)=230.83ms p(95)=233.34ms p(99.9)=481.62ms
     http_reqs......................: 13538   167.378448/s
     iteration_duration.............: avg=223.54ms min=161.27ms med=222.61ms max=602.33ms p(90)=231.16ms p(95)=233.76ms p(99.9)=513.09ms
     iterations.....................: 13438   166.142088/s
     success_rate...................: 100.00% ✓ 13438      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `cosmo`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 37656      ✗ 0    
     data_received..................: 1.1 GB  14 MB/s
     data_sent......................: 15 MB   182 kB/s
     http_req_blocked...............: avg=54.3µs   min=1.47µs   med=2.66µs   max=17.35ms  p(90)=3.93µs   p(95)=5.01µs   p(99.9)=15.07ms 
     http_req_connecting............: avg=50.75µs  min=0s       med=0s       max=17.29ms  p(90)=0s       p(95)=0s       p(99.9)=14.99ms 
     http_req_duration..............: avg=238.55ms min=156.26ms med=238.19ms max=610.01ms p(90)=260.23ms p(95)=266.58ms p(99.9)=509.17ms
       { expected_response:true }...: avg=238.55ms min=156.26ms med=238.19ms max=610.01ms p(90)=260.23ms p(95)=266.58ms p(99.9)=509.17ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 12652
     http_req_receiving.............: avg=489.68µs min=32.36µs  med=64.59µs  max=70.49ms  p(90)=241µs    p(95)=494.77µs p(99.9)=35.52ms 
     http_req_sending...............: avg=87.15µs  min=6.35µs   med=10.86µs  max=208.41ms p(90)=29.84µs  p(95)=122.3µs  p(99.9)=5.57ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=237.97ms min=155ms    med=237.72ms max=605.71ms p(90)=259.14ms p(95)=265.64ms p(99.9)=503.96ms
     http_reqs......................: 12652   156.310824/s
     iteration_duration.............: avg=239.1ms  min=156.45ms med=238.56ms max=644.92ms p(90)=260.5ms  p(95)=266.89ms p(99.9)=551.83ms
     iterations.....................: 12552   155.07536/s
     success_rate...................: 100.00% ✓ 12552      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 30939      ✗ 0    
     data_received..................: 915 MB  12 MB/s
     data_sent......................: 12 MB   158 kB/s
     http_req_blocked...............: avg=29.37µs  min=1.68µs   med=3.65µs   max=10.2ms   p(90)=5.36µs   p(95)=6.15µs   p(99.9)=8.24ms  
     http_req_connecting............: avg=24.12µs  min=0s       med=0s       max=10.16ms  p(90)=0s       p(95)=0s       p(99.9)=8.22ms  
     http_req_duration..............: avg=289.57ms min=160.25ms med=273.16ms max=841.48ms p(90)=390.43ms p(95)=414.92ms p(99.9)=736.9ms 
       { expected_response:true }...: avg=289.57ms min=160.25ms med=273.16ms max=841.48ms p(90)=390.43ms p(95)=414.92ms p(99.9)=736.9ms 
     http_req_failed................: 0.00%   ✓ 0          ✗ 10413
     http_req_receiving.............: avg=110.41µs min=37.1µs   med=72.33µs  max=237.94ms p(90)=119.26µs p(95)=143.23µs p(99.9)=1.96ms  
     http_req_sending...............: avg=105.37µs min=7.29µs   med=14.57µs  max=237.9ms  p(90)=22.83µs  p(95)=38.16µs  p(99.9)=2.87ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=289.35ms min=160.13ms med=273.06ms max=810.57ms p(90)=390.04ms p(95)=414.72ms p(99.9)=727.21ms
     http_reqs......................: 10413   135.766384/s
     iteration_duration.............: avg=291.33ms min=175.76ms med=273.59ms max=899.99ms p(90)=391.57ms p(95)=415.75ms p(99.9)=789.5ms 
     iterations.....................: 10313   134.462567/s
     success_rate...................: 100.00% ✓ 10313      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway-bun`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 29841      ✗ 0    
     data_received..................: 882 MB  12 MB/s
     data_sent......................: 12 MB   153 kB/s
     http_req_blocked...............: avg=36.54µs  min=1.97µs   med=4.1µs    max=12.16ms  p(90)=5.86µs   p(95)=6.82µs   p(99.9)=9.87ms  
     http_req_connecting............: avg=30.41µs  min=0s       med=0s       max=12.12ms  p(90)=0s       p(95)=0s       p(99.9)=9.84ms  
     http_req_duration..............: avg=300.2ms  min=160.7ms  med=282.4ms  max=876.31ms p(90)=400.86ms p(95)=425.73ms p(99.9)=748.82ms
       { expected_response:true }...: avg=300.2ms  min=160.7ms  med=282.4ms  max=876.31ms p(90)=400.86ms p(95)=425.73ms p(99.9)=748.82ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 10047
     http_req_receiving.............: avg=108.46µs min=38.01µs  med=79.46µs  max=112.82ms p(90)=128.14µs p(95)=159.36µs p(99.9)=3.06ms  
     http_req_sending...............: avg=50.26µs  min=8.72µs   med=16.88µs  max=117.5ms  p(90)=27.73µs  p(95)=43.12µs  p(99.9)=1.34ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=300.04ms min=160.61ms med=282.26ms max=876.03ms p(90)=400.75ms p(95)=425.64ms p(99.9)=737.89ms
     http_reqs......................: 10047   131.137339/s
     iteration_duration.............: avg=302.19ms min=166.14ms med=282.94ms max=925.83ms p(90)=401.8ms  p(95)=426.48ms p(99.9)=885.64ms
     iterations.....................: 9947    129.8321/s
     success_rate...................: 100.00% ✓ 9947       ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-router`

K6 Output

     ✓ response code was 200
     ✗ no graphql errors
      ↳  99% — ✓ 9930 / ✗ 1
     ✓ valid response structure

     checks.........................: 99.99% ✓ 29792      ✗ 1    
     data_received..................: 880 MB 11 MB/s
     data_sent......................: 12 MB  143 kB/s
     http_req_blocked...............: avg=23.78µs  min=1.73µs   med=2.75µs   max=8.1ms    p(90)=4.28µs   p(95)=5.35µs   p(99.9)=6.54ms  
     http_req_connecting............: avg=20.26µs  min=0s       med=0s       max=8.06ms   p(90)=0s       p(95)=0s       p(99.9)=6.52ms  
     http_req_duration..............: avg=301.18ms min=178.95ms med=302.08ms max=691.82ms p(90)=335.1ms  p(95)=344.49ms p(99.9)=609.58ms
       { expected_response:true }...: avg=301.18ms min=178.95ms med=302.08ms max=691.82ms p(90)=335.1ms  p(95)=344.49ms p(99.9)=609.58ms
     http_req_failed................: 0.00%  ✓ 0          ✗ 10031
     http_req_receiving.............: avg=91.4µs   min=36.85µs  med=56.35µs  max=5.73ms   p(90)=112.29µs p(95)=231.66µs p(99.9)=1.57ms  
     http_req_sending...............: avg=117.41µs min=7.87µs   med=11.43µs  max=340.02ms p(90)=29.11µs  p(95)=136.21µs p(99.9)=2.17ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=300.97ms min=178.85ms med=301.92ms max=690.76ms p(90)=334.91ms p(95)=344.33ms p(99.9)=607.99ms
     http_reqs......................: 10031  123.133565/s
     iteration_duration.............: avg=302.46ms min=179.21ms med=302.66ms max=708.29ms p(90)=335.47ms p(95)=344.92ms p(99.9)=659.13ms
     iterations.....................: 9931   121.906035/s
     success_rate...................: 99.98% ✓ 9930       ✗ 1    
     vus............................: 50     min=0        max=50 
     vus_max........................: 50     min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 21147     ✗ 0   
     data_received..................: 628 MB  7.7 MB/s
     data_sent......................: 8.3 MB  102 kB/s
     http_req_blocked...............: avg=47.35µs  min=1.39µs   med=2.98µs   max=11.91ms  p(90)=4.39µs   p(95)=5.25µs   p(99.9)=10.41ms 
     http_req_connecting............: avg=43.8µs   min=0s       med=0s       max=11.87ms  p(90)=0s       p(95)=0s       p(99.9)=10.38ms 
     http_req_duration..............: avg=423.97ms min=212.07ms med=425.66ms max=909.35ms p(90)=453.2ms  p(95)=465.01ms p(99.9)=834.81ms
       { expected_response:true }...: avg=423.97ms min=212.07ms med=425.66ms max=909.35ms p(90)=453.2ms  p(95)=465.01ms p(99.9)=834.81ms
     http_req_failed................: 0.00%   ✓ 0         ✗ 7149
     http_req_receiving.............: avg=62.22µs  min=28.36µs  med=51.86µs  max=982.43µs p(90)=93.07µs  p(95)=108.56µs p(99.9)=343.79µs
     http_req_sending...............: avg=75.01µs  min=6.07µs   med=12.28µs  max=217.36ms p(90)=18.62µs  p(95)=23.53µs  p(99.9)=3.32ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=423.83ms min=211.73ms med=425.56ms max=909.16ms p(90)=453.1ms  p(95)=464.91ms p(99.9)=834ms   
     http_reqs......................: 7149    87.475182/s
     iteration_duration.............: avg=427.47ms min=270.5ms  med=426.1ms  max=934.25ms p(90)=453.85ms p(95)=465.42ms p(99.9)=917.73ms
     iterations.....................: 7049    86.251582/s
     success_rate...................: 100.00% ✓ 7049      ✗ 0   
     vus............................: 50      min=0       max=50
     vus_max........................: 50      min=50      max=50

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview

@ardatan ardatan force-pushed the impr branch 4 times, most recently from 46f02f9 to bd659d2 Compare September 4, 2025 11:37
@ardatan ardatan marked this pull request as ready for review September 4, 2025 12:10
@ardatan ardatan requested a review from dotansimha September 4, 2025 12:15
@dotansimha dotansimha merged commit 13fca1b into main Sep 4, 2025
33 checks passed
@dotansimha dotansimha deleted the impr branch September 4, 2025 12:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants