Skip to content

Conversation

renovate[bot]
Copy link
Contributor

@renovate renovate bot commented Sep 29, 2025

Coming soon: The Renovate bot (GitHub App) will be renamed to Mend. PRs from Renovate will soon appear from 'Mend'. Learn more here.

This PR contains the following updates:

Package Type Update Change
docker/login-action action digest 184bdaa -> 5e57cd1

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Enabled.

Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.


  • If you want to rebase/retry this PR, check this box

This PR was generated by Mend Renovate. View the repository job log.

@renovate renovate bot added the dependencies Pull requests that update a dependency file label Sep 29, 2025
Copy link

💻 Website Preview

The latest changes are available as preview in: https://29ce24f7.federation-gateway-benchmark.pages.dev

Copy link

Overview for: constant-vus-over-time

This scenario runs 4 subgraphs and a GraphQL gateway with Federation spec, and runs a heavy query. It's being executed with a constant amount of VUs over a fixed amount of time. It measure things like memory usage, CPU usage, average RPS. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was running 50 VUs over 60s

Comparison

Comparison

Gateway RPS ⬇️ Requests Duration Notes
hive-router 1807 108934 total, 0 failed avg: 27ms, p95: 49ms
grafbase 1519 91723 total, 0 failed avg: 32ms, p95: 58ms
cosmo 605 36580 total, 0 failed avg: 82ms, p95: 120ms
apollo-router 362 22006 total, 0 failed avg: 136ms, p95: 177ms
hive-gateway 306 18760 total, 0 failed avg: 160ms, p95: 219ms
hive-gateway-bun 300 18305 total, 0 failed avg: 164ms, p95: 234ms
apollo-gateway 118 7272 total, 0 failed avg: 413ms, p95: 486ms
Summary for: `hive-router`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 326502      ✗ 0     
     data_received..................: 9.6 GB  159 MB/s
     data_sent......................: 127 MB  2.1 MB/s
     http_req_blocked...............: avg=5.06µs   min=1.08µs  med=2.51µs  max=9.24ms   p(90)=3.64µs   p(95)=4.25µs   p(99.9)=92.65µs
     http_req_connecting............: avg=2.1µs    min=0s      med=0s      max=9.2ms    p(90)=0s       p(95)=0s       p(99.9)=0s     
     http_req_duration..............: avg=27.28ms  min=1.91ms  med=24.61ms max=419.51ms p(90)=42.6ms   p(95)=48.91ms  p(99.9)=87.41ms
       { expected_response:true }...: avg=27.28ms  min=1.91ms  med=24.61ms max=419.51ms p(90)=42.6ms   p(95)=48.91ms  p(99.9)=87.41ms
     http_req_failed................: 0.00%   ✓ 0           ✗ 108934
     http_req_receiving.............: avg=125.33µs min=21.89µs med=41.96µs max=64.94ms  p(90)=106.97µs p(95)=341.64µs p(99.9)=15.56ms
     http_req_sending...............: avg=100.98µs min=5.24µs  med=9.91µs  max=275.35ms p(90)=29.39µs  p(95)=124.91µs p(99.9)=14.46ms
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s      max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s     
     http_req_waiting...............: avg=27.06ms  min=1.86ms  med=24.43ms max=418.13ms p(90)=42.16ms  p(95)=48.5ms   p(99.9)=86.01ms
     http_reqs......................: 108934  1807.960748/s
     iteration_duration.............: avg=27.55ms  min=4.22ms  med=24.84ms max=470.57ms p(90)=42.85ms  p(95)=49.15ms  p(99.9)=88.21ms
     iterations.....................: 108834  1806.301063/s
     success_rate...................: 100.00% ✓ 108834      ✗ 0     
     vus............................: 50      min=50        max=50  
     vus_max........................: 50      min=50        max=50  

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `grafbase`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 274869      ✗ 0    
     data_received..................: 8.1 GB  134 MB/s
     data_sent......................: 107 MB  1.8 MB/s
     http_req_blocked...............: avg=6.52µs   min=1.33µs  med=3.24µs  max=9.52ms   p(90)=4.66µs   p(95)=5.56µs   p(99.9)=130.01µs
     http_req_connecting............: avg=2.59µs   min=0s      med=0s      max=9.48ms   p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_duration..............: avg=32.41ms  min=2.62ms  med=28.68ms max=475.71ms p(90)=50.7ms   p(95)=58.12ms  p(99.9)=101.27ms
       { expected_response:true }...: avg=32.41ms  min=2.62ms  med=28.68ms max=475.71ms p(90)=50.7ms   p(95)=58.12ms  p(99.9)=101.27ms
     http_req_failed................: 0.00%   ✓ 0           ✗ 91723
     http_req_receiving.............: avg=175.18µs min=28.69µs med=51.78µs max=150.37ms p(90)=154.35µs p(95)=432.15µs p(99.9)=19.85ms 
     http_req_sending...............: avg=127.46µs min=5.69µs  med=12.26µs max=52.27ms  p(90)=39.73µs  p(95)=153.03µs p(99.9)=19.8ms  
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s      max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=32.11ms  min=2.57ms  med=28.48ms max=459.91ms p(90)=50.07ms  p(95)=57.38ms  p(99.9)=100.34ms
     http_reqs......................: 91723   1519.679043/s
     iteration_duration.............: avg=32.73ms  min=5.31ms  med=28.98ms max=525ms    p(90)=51ms     p(95)=58.43ms  p(99.9)=101.98ms
     iterations.....................: 91623   1518.022229/s
     success_rate...................: 100.00% ✓ 91623       ✗ 0    
     vus............................: 50      min=50        max=50 
     vus_max........................: 50      min=50        max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `cosmo`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 109440     ✗ 0    
     data_received..................: 3.2 GB  53 MB/s
     data_sent......................: 43 MB   704 kB/s
     http_req_blocked...............: avg=11.32µs min=1.41µs  med=2.94µs  max=11.21ms  p(90)=4.66µs   p(95)=5.87µs   p(99.9)=2.66ms  
     http_req_connecting............: avg=7.49µs  min=0s      med=0s      max=11.17ms  p(90)=0s       p(95)=0s       p(99.9)=2.63ms  
     http_req_duration..............: avg=81.69ms min=2.88ms  med=80.55ms max=537.8ms  p(90)=111.05ms p(95)=120.03ms p(99.9)=319.88ms
       { expected_response:true }...: avg=81.69ms min=2.88ms  med=80.55ms max=537.8ms  p(90)=111.05ms p(95)=120.03ms p(99.9)=319.88ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 36580
     http_req_receiving.............: avg=182.1µs min=33.39µs med=80.94µs max=57.22ms  p(90)=166.34µs p(95)=429.73µs p(99.9)=16.69ms 
     http_req_sending...............: avg=78.22µs min=6.71µs  med=12.06µs max=283.56ms p(90)=31.2µs   p(95)=139.5µs  p(99.9)=4.46ms  
     http_req_tls_handshaking.......: avg=0s      min=0s      med=0s      max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=81.43ms min=2.8ms   med=80.32ms max=536.06ms p(90)=110.72ms p(95)=119.66ms p(99.9)=315.69ms
     http_reqs......................: 36580   605.194747/s
     iteration_duration.............: avg=82.25ms min=5.99ms  med=80.92ms max=583.01ms p(90)=111.36ms p(95)=120.35ms p(99.9)=340.42ms
     iterations.....................: 36480   603.540305/s
     success_rate...................: 100.00% ✓ 36480      ✗ 0    
     vus............................: 50      min=50       max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-router`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 65718      ✗ 0    
     data_received..................: 1.9 GB  32 MB/s
     data_sent......................: 26 MB   421 kB/s
     http_req_blocked...............: avg=12.98µs  min=1.91µs  med=2.93µs   max=7.81ms   p(90)=4.54µs   p(95)=5.61µs   p(99.9)=4.71ms  
     http_req_connecting............: avg=9.07µs   min=0s      med=0s       max=7.78ms   p(90)=0s       p(95)=0s       p(99.9)=4.52ms  
     http_req_duration..............: avg=136.12ms min=5.99ms  med=134.9ms  max=632.43ms p(90)=167.51ms p(95)=177.49ms p(99.9)=449.67ms
       { expected_response:true }...: avg=136.12ms min=5.99ms  med=134.9ms  max=632.43ms p(90)=167.51ms p(95)=177.49ms p(99.9)=449.67ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 22006
     http_req_receiving.............: avg=82.56µs  min=37.73µs med=55.52µs  max=9.85ms   p(90)=110.11µs p(95)=143.12µs p(99.9)=1.92ms  
     http_req_sending...............: avg=67.19µs  min=7.62µs  med=11.83µs  max=386.95ms p(90)=21.95µs  p(95)=37.6µs   p(99.9)=2.29ms  
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=135.97ms min=5.94ms  med=134.79ms max=631.32ms p(90)=167.34ms p(95)=177.35ms p(99.9)=439.33ms
     http_reqs......................: 22006   362.127639/s
     iteration_duration.............: avg=137.05ms min=20.7ms  med=135.31ms max=659.78ms p(90)=167.86ms p(95)=177.92ms p(99.9)=467.14ms
     iterations.....................: 21906   360.482053/s
     success_rate...................: 100.00% ✓ 21906      ✗ 0    
     vus............................: 50      min=50       max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 55980      ✗ 0    
     data_received..................: 1.6 GB  27 MB/s
     data_sent......................: 22 MB   356 kB/s
     http_req_blocked...............: avg=19.59µs  min=1.21µs  med=2.81µs   max=11.45ms  p(90)=4.47µs   p(95)=5.5µs    p(99.9)=7.02ms  
     http_req_connecting............: avg=15.57µs  min=0s      med=0s       max=11.42ms  p(90)=0s       p(95)=0s       p(99.9)=6.99ms  
     http_req_duration..............: avg=159.83ms min=6.56ms  med=148.39ms max=578.73ms p(90)=180.11ms p(95)=219.01ms p(99.9)=492.85ms
       { expected_response:true }...: avg=159.83ms min=6.56ms  med=148.39ms max=578.73ms p(90)=180.11ms p(95)=219.01ms p(99.9)=492.85ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 18760
     http_req_receiving.............: avg=103.44µs min=25.64µs med=46.79µs  max=196.6ms  p(90)=101.39µs p(95)=154.44µs p(99.9)=4.95ms  
     http_req_sending...............: avg=82.17µs  min=5.61µs  med=11.23µs  max=405.88ms p(90)=32µs     p(95)=118.15µs p(99.9)=5ms     
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=159.64ms min=6.51ms  med=148.24ms max=578.5ms  p(90)=179.92ms p(95)=218.35ms p(99.9)=492.28ms
     http_reqs......................: 18760   306.373289/s
     iteration_duration.............: avg=161.02ms min=39.47ms med=148.76ms max=618.55ms p(90)=180.7ms  p(95)=220.5ms  p(99.9)=499.67ms
     iterations.....................: 18660   304.740169/s
     success_rate...................: 100.00% ✓ 18660      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway-bun`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 54615      ✗ 0    
     data_received..................: 1.6 GB  26 MB/s
     data_sent......................: 21 MB   350 kB/s
     http_req_blocked...............: avg=20.08µs  min=1.1µs   med=2.71µs   max=11.68ms  p(90)=4.61µs   p(95)=5.73µs   p(99.9)=7.3ms   
     http_req_connecting............: avg=16.28µs  min=0s      med=0s       max=11.64ms  p(90)=0s       p(95)=0s       p(99.9)=7.25ms  
     http_req_duration..............: avg=163.53ms min=6.8ms   med=151.31ms max=634.3ms  p(90)=184.45ms p(95)=233.61ms p(99.9)=516.86ms
       { expected_response:true }...: avg=163.53ms min=6.8ms   med=151.31ms max=634.3ms  p(90)=184.45ms p(95)=233.61ms p(99.9)=516.86ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 18305
     http_req_receiving.............: avg=107.26µs min=26.55µs med=46.89µs  max=221.4ms  p(90)=107.05µs p(95)=185.32µs p(99.9)=3.33ms  
     http_req_sending...............: avg=77.38µs  min=5.65µs  med=11.38µs  max=362.67ms p(90)=33.09µs  p(95)=128.99µs p(99.9)=5.62ms  
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=163.34ms min=6.74ms  med=151.17ms max=633.31ms p(90)=184.19ms p(95)=230.54ms p(99.9)=516.37ms
     http_reqs......................: 18305   300.598419/s
     iteration_duration.............: avg=164.84ms min=52.7ms  med=151.75ms max=691.16ms p(90)=185ms    p(95)=237.64ms p(99.9)=551.3ms 
     iterations.....................: 18205   298.956254/s
     success_rate...................: 100.00% ✓ 18205      ✗ 0    
     vus............................: 50      min=50       max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 21516      ✗ 0   
     data_received..................: 639 MB  11 MB/s
     data_sent......................: 8.5 MB  138 kB/s
     http_req_blocked...............: avg=44.81µs  min=1.42µs   med=2.85µs   max=11.39ms  p(90)=4.65µs   p(95)=5.28µs   p(99.9)=10.22ms 
     http_req_connecting............: avg=38.8µs   min=0s       med=0s       max=11.36ms  p(90)=0s       p(95)=0s       p(99.9)=9.62ms  
     http_req_duration..............: avg=413.24ms min=7.43ms   med=412.93ms max=825.46ms p(90)=467.89ms p(95)=485.54ms p(99.9)=740.24ms
       { expected_response:true }...: avg=413.24ms min=7.43ms   med=412.93ms max=825.46ms p(90)=467.89ms p(95)=485.54ms p(99.9)=740.24ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 7272
     http_req_receiving.............: avg=80.39µs  min=29.62µs  med=51.7µs   max=111.82ms p(90)=98.85µs  p(95)=115.32µs p(99.9)=599.67µs
     http_req_sending...............: avg=94.97µs  min=6.18µs   med=11.71µs  max=173.86ms p(90)=19.87µs  p(95)=23.11µs  p(99.9)=4.47ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=413.07ms min=7.37ms   med=412.84ms max=796.4ms  p(90)=467.58ms p(95)=485.4ms  p(99.9)=732.76ms
     http_reqs......................: 7272    118.980746/s
     iteration_duration.............: avg=419.43ms min=184.33ms med=413.69ms max=857.77ms p(90)=468.77ms p(95)=486.39ms p(99.9)=772.42ms
     iterations.....................: 7172    117.344597/s
     success_rate...................: 100.00% ✓ 7172       ✗ 0   
     vus............................: 31      min=31       max=50
     vus_max........................: 50      min=50       max=50

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview

Copy link

Overview for: constant-vus-subgraphs-delay

This scenario runs 4 subgraphs and a GraphQL gateway with Federation spec, and runs a heavy query. It's being executed with a constant amount of VUs over a fixed amount of time. It measure things like memory usage, CPU usage, average RPS. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was running 50 VUs over 60s

Comparison

Comparison

Gateway RPS ⬇️ Requests Duration Notes
grafbase 172 13947 total, 0 failed avg: 216ms, p95: 228ms
hive-router 167 13590 total, 0 failed avg: 222ms, p95: 234ms
cosmo 157 12748 total, 0 failed avg: 237ms, p95: 262ms
hive-gateway 148 11341 total, 0 failed avg: 266ms, p95: 318ms
hive-gateway-bun 146 11268 total, 0 failed avg: 268ms, p95: 406ms
apollo-router 129 10545 total, 0 failed avg: 287ms, p95: 337ms
apollo-gateway 85 6963 total, 0 failed avg: 434ms, p95: 480ms
Summary for: `grafbase`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 41541      ✗ 0    
     data_received..................: 1.2 GB  15 MB/s
     data_sent......................: 16 MB   200 kB/s
     http_req_blocked...............: avg=19.6µs   min=1.32µs   med=2µs      max=9.45ms   p(90)=3.63µs   p(95)=5.19µs   p(99.9)=6.82ms  
     http_req_connecting............: avg=16.86µs  min=0s       med=0s       max=9.42ms   p(90)=0s       p(95)=0s       p(99.9)=6.8ms   
     http_req_duration..............: avg=216.46ms min=161.63ms med=215.48ms max=629.5ms  p(90)=225.49ms p(95)=228.38ms p(99.9)=528.46ms
       { expected_response:true }...: avg=216.46ms min=161.63ms med=215.48ms max=629.5ms  p(90)=225.49ms p(95)=228.38ms p(99.9)=528.46ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 13947
     http_req_receiving.............: avg=88.76µs  min=28.43µs  med=46.39µs  max=13.86ms  p(90)=97.35µs  p(95)=270.99µs p(99.9)=7.56ms  
     http_req_sending...............: avg=138.88µs min=6.1µs    med=8.58µs   max=382.56ms p(90)=32.48µs  p(95)=98.39µs  p(99.9)=8.96ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=216.23ms min=159.97ms med=215.35ms max=628.09ms p(90)=225.29ms p(95)=228.11ms p(99.9)=509.96ms
     http_reqs......................: 13947   172.045489/s
     iteration_duration.............: avg=216.76ms min=161.87ms med=215.75ms max=650.87ms p(90)=225.69ms p(95)=228.6ms  p(99.9)=540.62ms
     iterations.....................: 13847   170.811923/s
     success_rate...................: 100.00% ✓ 13847      ✗ 0    
     vus............................: 6       min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-router`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 40470      ✗ 0    
     data_received..................: 1.2 GB  15 MB/s
     data_sent......................: 16 MB   195 kB/s
     http_req_blocked...............: avg=18.99µs  min=1.16µs   med=2.17µs   max=8.83ms   p(90)=3.1µs    p(95)=4.25µs   p(99.9)=6.54ms  
     http_req_connecting............: avg=16.42µs  min=0s       med=0s       max=8.79ms   p(90)=0s       p(95)=0s       p(99.9)=6.52ms  
     http_req_duration..............: avg=222.36ms min=159.9ms  med=221.43ms max=611.05ms p(90)=230.81ms p(95)=233.71ms p(99.9)=518.82ms
       { expected_response:true }...: avg=222.36ms min=159.9ms  med=221.43ms max=611.05ms p(90)=230.81ms p(95)=233.71ms p(99.9)=518.82ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 13590
     http_req_receiving.............: avg=72.95µs  min=23.97µs  med=42.11µs  max=10.48ms  p(90)=84.66µs  p(95)=226.24µs p(99.9)=2.38ms  
     http_req_sending...............: avg=138.61µs min=5.37µs   med=9.21µs   max=365.94ms p(90)=27.49µs  p(95)=94.19µs  p(99.9)=5.45ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=222.15ms min=159.77ms med=221.35ms max=610.39ms p(90)=230.66ms p(95)=233.5ms  p(99.9)=503.09ms
     http_reqs......................: 13590   167.97819/s
     iteration_duration.............: avg=222.73ms min=160.23ms med=221.7ms  max=623.09ms p(90)=231ms    p(95)=233.89ms p(99.9)=530.14ms
     iterations.....................: 13490   166.742148/s
     success_rate...................: 100.00% ✓ 13490      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `cosmo`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 37944      ✗ 0    
     data_received..................: 1.1 GB  14 MB/s
     data_sent......................: 15 MB   183 kB/s
     http_req_blocked...............: avg=22.65µs  min=1.29µs   med=2.58µs   max=10.15ms  p(90)=3.62µs   p(95)=4.54µs   p(99.9)=7.57ms  
     http_req_connecting............: avg=19.26µs  min=0s       med=0s       max=10.12ms  p(90)=0s       p(95)=0s       p(99.9)=7.54ms  
     http_req_duration..............: avg=236.9ms  min=120.37ms med=236.85ms max=636.88ms p(90)=256.85ms p(95)=262.27ms p(99.9)=547.27ms
       { expected_response:true }...: avg=236.9ms  min=120.37ms med=236.85ms max=636.88ms p(90)=256.85ms p(95)=262.27ms p(99.9)=547.27ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 12748
     http_req_receiving.............: avg=371.49µs min=32.69µs  med=57.57µs  max=44.47ms  p(90)=157.26µs p(95)=451.41µs p(99.9)=32.28ms 
     http_req_sending...............: avg=56.81µs  min=6.14µs   med=10.7µs   max=107.82ms p(90)=28.66µs  p(95)=119.43µs p(99.9)=4ms     
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=236.47ms min=120.19ms med=236.43ms max=635.93ms p(90)=256.35ms p(95)=261.66ms p(99.9)=546.06ms
     http_reqs......................: 12748   157.42542/s
     iteration_duration.............: avg=237.5ms  min=120.61ms med=237.28ms max=663.63ms p(90)=257.22ms p(95)=262.63ms p(99.9)=583.68ms
     iterations.....................: 12648   156.190517/s
     success_rate...................: 100.00% ✓ 12648      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 33723      ✗ 0    
     data_received..................: 996 MB  13 MB/s
     data_sent......................: 13 MB   173 kB/s
     http_req_blocked...............: avg=27.97µs  min=871ns    med=2.31µs   max=11.14ms  p(90)=3.93µs   p(95)=5.4µs    p(99.9)=8.98ms  
     http_req_connecting............: avg=24.43µs  min=0s       med=0s       max=11.1ms   p(90)=0s       p(95)=0s       p(99.9)=8.95ms  
     http_req_duration..............: avg=266.24ms min=127.31ms med=255.82ms max=976.97ms p(90)=274.26ms p(95)=317.79ms p(99.9)=961.07ms
       { expected_response:true }...: avg=266.24ms min=127.31ms med=255.82ms max=976.97ms p(90)=274.26ms p(95)=317.79ms p(99.9)=961.07ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 11341
     http_req_receiving.............: avg=95.92µs  min=22.76µs  med=44.12µs  max=18.08ms  p(90)=109.71µs p(95)=389.56µs p(99.9)=4.61ms  
     http_req_sending...............: avg=80.3µs   min=4.24µs   med=9.94µs   max=216.84ms p(90)=32.37µs  p(95)=144.34µs p(99.9)=5.6ms   
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=266.06ms min=127.25ms med=255.68ms max=976.91ms p(90)=273.97ms p(95)=316.69ms p(99.9)=961.02ms
     http_reqs......................: 11341   148.32792/s
     iteration_duration.............: avg=267.52ms min=127.58ms med=256.17ms max=980.43ms p(90)=274.69ms p(95)=322.94ms p(99.9)=961.38ms
     iterations.....................: 11241   147.020029/s
     success_rate...................: 100.00% ✓ 11241      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway-bun`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 33504      ✗ 0    
     data_received..................: 990 MB  13 MB/s
     data_sent......................: 13 MB   171 kB/s
     http_req_blocked...............: avg=32.36µs  min=1.16µs   med=2.75µs   max=12.05ms  p(90)=4.27µs   p(95)=5.35µs   p(99.9)=9.65ms  
     http_req_connecting............: avg=28.68µs  min=0s       med=0s       max=12.02ms  p(90)=0s       p(95)=0s       p(99.9)=9.63ms  
     http_req_duration..............: avg=268.14ms min=137.58ms med=258.31ms max=715.35ms p(90)=281.33ms p(95)=406.44ms p(99.9)=598.13ms
       { expected_response:true }...: avg=268.14ms min=137.58ms med=258.31ms max=715.35ms p(90)=281.33ms p(95)=406.44ms p(99.9)=598.13ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 11268
     http_req_receiving.............: avg=105.18µs min=26.13µs  med=47.27µs  max=135.74ms p(90)=103.53µs p(95)=316.23µs p(99.9)=5.09ms  
     http_req_sending...............: avg=143.17µs min=5.48µs   med=11.22µs  max=212.33ms p(90)=34.52µs  p(95)=143.24µs p(99.9)=11.55ms 
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=267.89ms min=137.53ms med=258.16ms max=714.49ms p(90)=281.06ms p(95)=405.53ms p(99.9)=597.15ms
     http_reqs......................: 11268   146.752601/s
     iteration_duration.............: avg=269.46ms min=137.82ms med=258.69ms max=735.57ms p(90)=281.78ms p(95)=408.92ms p(99.9)=667.3ms 
     iterations.....................: 11168   145.450217/s
     success_rate...................: 100.00% ✓ 11168      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-router`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 31335      ✗ 0    
     data_received..................: 925 MB  11 MB/s
     data_sent......................: 12 MB   151 kB/s
     http_req_blocked...............: avg=25.75µs  min=1.75µs   med=2.74µs   max=9.64ms   p(90)=4.05µs   p(95)=4.99µs   p(99.9)=7.67ms  
     http_req_connecting............: avg=22.33µs  min=0s       med=0s       max=9.6ms    p(90)=0s       p(95)=0s       p(99.9)=7.65ms  
     http_req_duration..............: avg=286.81ms min=174.41ms med=286.22ms max=746.45ms p(90)=326.82ms p(95)=336.79ms p(99.9)=649.36ms
       { expected_response:true }...: avg=286.81ms min=174.41ms med=286.22ms max=746.45ms p(90)=326.82ms p(95)=336.79ms p(99.9)=649.36ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 10545
     http_req_receiving.............: avg=103.32µs min=34.33µs  med=51.04µs  max=216.5ms  p(90)=101.43µs p(95)=207.73µs p(99.9)=1.4ms   
     http_req_sending...............: avg=105.11µs min=7.64µs   med=11.06µs  max=265.15ms p(90)=26.33µs  p(95)=136.56µs p(99.9)=3.56ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=286.6ms  min=174.33ms med=286.1ms  max=729.08ms p(90)=326.68ms p(95)=336.56ms p(99.9)=642.84ms
     http_reqs......................: 10545   129.525611/s
     iteration_duration.............: avg=287.88ms min=174.66ms med=286.94ms max=776.81ms p(90)=327.22ms p(95)=337.14ms p(99.9)=664.69ms
     iterations.....................: 10445   128.297298/s
     success_rate...................: 100.00% ✓ 10445      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 20589     ✗ 0   
     data_received..................: 612 MB  7.5 MB/s
     data_sent......................: 8.1 MB  99 kB/s
     http_req_blocked...............: avg=49.31µs  min=1.4µs    med=3.23µs   max=12.1ms   p(90)=5.21µs   p(95)=5.95µs   p(99.9)=10.65ms 
     http_req_connecting............: avg=43.79µs  min=0s       med=0s       max=12.07ms  p(90)=0s       p(95)=0s       p(99.9)=10.41ms 
     http_req_duration..............: avg=434.14ms min=212.03ms med=436.66ms max=989.01ms p(90)=463.66ms p(95)=479.95ms p(99.9)=924.54ms
       { expected_response:true }...: avg=434.14ms min=212.03ms med=436.66ms max=989.01ms p(90)=463.66ms p(95)=479.95ms p(99.9)=924.54ms
     http_req_failed................: 0.00%   ✓ 0         ✗ 6963
     http_req_receiving.............: avg=87.5µs   min=32µs     med=60.65µs  max=107.21ms p(90)=108.94µs p(95)=128.47µs p(99.9)=389.98µs
     http_req_sending...............: avg=106.2µs  min=6.43µs   med=13.48µs  max=225.53ms p(90)=22.47µs  p(95)=32.21µs  p(99.9)=4.58ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=433.95ms min=211.93ms med=436.56ms max=988.16ms p(90)=463.51ms p(95)=479.78ms p(99.9)=923.7ms 
     http_reqs......................: 6963    85.183038/s
     iteration_duration.............: avg=438.07ms min=269.68ms med=437.14ms max=1.02s    p(90)=464.16ms p(95)=480.5ms  p(99.9)=989.13ms
     iterations.....................: 6863    83.959671/s
     success_rate...................: 100.00% ✓ 6863      ✗ 0   
     vus............................: 50      min=0       max=50
     vus_max........................: 50      min=50      max=50

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview

Copy link

Overview for: constant-vus-subgraphs-delay-resources

This scenario runs 4 subgraphs and a GraphQL gateway with Federation spec, and runs a heavy query. It's being executed with a constant amount of VUs over a fixed amount of time. It measure things like memory usage, CPU usage, average RPS. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was running 50 VUs over 60s

Comparison

Comparison

Gateway RPS ⬇️ Requests Duration Notes
grafbase 173 14040 total, 0 failed avg: 215ms, p95: 226ms
cosmo 169 13704 total, 0 failed avg: 221ms, p95: 240ms
hive-router 168 13641 total, 0 failed avg: 221ms, p95: 232ms
hive-gateway 167 12787 total, 0 failed avg: 236ms, p95: 260ms
hive-gateway-bun 163 12512 total, 0 failed avg: 241ms, p95: 266ms
apollo-router 136 11131 total, 0 failed avg: 271ms, p95: 314ms ❌ 1 unexpected GraphQL errors
apollo-gateway 124 10151 total, 0 failed avg: 298ms, p95: 336ms
Summary for: `grafbase`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 41820      ✗ 0    
     data_received..................: 1.2 GB  15 MB/s
     data_sent......................: 16 MB   202 kB/s
     http_req_blocked...............: avg=18.55µs  min=1.08µs   med=1.86µs   max=8.78ms   p(90)=3.24µs   p(95)=4.72µs   p(99.9)=6.46ms  
     http_req_connecting............: avg=15.61µs  min=0s       med=0s       max=8.75ms   p(90)=0s       p(95)=0s       p(99.9)=6.35ms  
     http_req_duration..............: avg=215.01ms min=158.23ms med=214.32ms max=609.91ms p(90)=223.37ms p(95)=225.73ms p(99.9)=513.75ms
       { expected_response:true }...: avg=215.01ms min=158.23ms med=214.32ms max=609.91ms p(90)=223.37ms p(95)=225.73ms p(99.9)=513.75ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 14040
     http_req_receiving.............: avg=83.37µs  min=27.2µs   med=44.49µs  max=23.32ms  p(90)=90.05µs  p(95)=247.69µs p(99.9)=6.58ms  
     http_req_sending...............: avg=130.56µs min=5.49µs   med=8.29µs   max=369.89ms p(90)=27.14µs  p(95)=98.26µs  p(99.9)=8.8ms   
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=214.79ms min=158.18ms med=214.22ms max=609.11ms p(90)=223.21ms p(95)=225.54ms p(99.9)=495.61ms
     http_reqs......................: 14040   173.607036/s
     iteration_duration.............: avg=215.31ms min=158.46ms med=214.56ms max=622.06ms p(90)=223.56ms p(95)=225.94ms p(99.9)=520.7ms 
     iterations.....................: 13940   172.370519/s
     success_rate...................: 100.00% ✓ 13940      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `cosmo`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 40812      ✗ 0    
     data_received..................: 1.2 GB  15 MB/s
     data_sent......................: 16 MB   197 kB/s
     http_req_blocked...............: avg=23.3µs   min=1.33µs   med=2.66µs   max=10.28ms  p(90)=3.87µs   p(95)=5.14µs   p(99.9)=7.71ms  
     http_req_connecting............: avg=19.65µs  min=0s       med=0s       max=10.24ms  p(90)=0s       p(95)=0s       p(99.9)=7.69ms  
     http_req_duration..............: avg=220.67ms min=160.92ms med=219.09ms max=624.15ms p(90)=234.24ms p(95)=239.93ms p(99.9)=531.43ms
       { expected_response:true }...: avg=220.67ms min=160.92ms med=219.09ms max=624.15ms p(90)=234.24ms p(95)=239.93ms p(99.9)=531.43ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 13704
     http_req_receiving.............: avg=387.51µs min=31.44µs  med=63.37µs  max=175.98ms p(90)=359.4µs  p(95)=759.97µs p(99.9)=22.48ms 
     http_req_sending...............: avg=104.4µs  min=6.3µs    med=11.11µs  max=332.99ms p(90)=32.46µs  p(95)=127.25µs p(99.9)=4.43ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=220.18ms min=160.79ms med=218.65ms max=622.85ms p(90)=233.67ms p(95)=239.4ms  p(99.9)=498.59ms
     http_reqs......................: 13704   169.025206/s
     iteration_duration.............: avg=221.11ms min=161.25ms med=219.45ms max=649.35ms p(90)=234.55ms p(95)=240.41ms p(99.9)=555.18ms
     iterations.....................: 13604   167.791806/s
     success_rate...................: 100.00% ✓ 13604      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-router`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 40623      ✗ 0    
     data_received..................: 1.2 GB  15 MB/s
     data_sent......................: 16 MB   196 kB/s
     http_req_blocked...............: avg=17.92µs  min=1.14µs   med=2µs      max=8.5ms    p(90)=3.26µs   p(95)=4.56µs   p(99.9)=6.15ms  
     http_req_connecting............: avg=15.4µs   min=0s       med=0s       max=8.47ms   p(90)=0s       p(95)=0s       p(99.9)=6.13ms  
     http_req_duration..............: avg=221.26ms min=157.15ms med=220.29ms max=613.02ms p(90)=228.93ms p(95)=231.63ms p(99.9)=519.15ms
       { expected_response:true }...: avg=221.26ms min=157.15ms med=220.29ms max=613.02ms p(90)=228.93ms p(95)=231.63ms p(99.9)=519.15ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 13641
     http_req_receiving.............: avg=78.85µs  min=24.04µs  med=43.42µs  max=13.01ms  p(90)=89.88µs  p(95)=244.81µs p(99.9)=3.72ms  
     http_req_sending...............: avg=144.63µs min=5.41µs   med=8.35µs   max=382.82ms p(90)=28.01µs  p(95)=100.4µs  p(99.9)=10.9ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=221.03ms min=157.1ms  med=220.18ms max=612.01ms p(90)=228.78ms p(95)=231.44ms p(99.9)=503.38ms
     http_reqs......................: 13641   168.795122/s
     iteration_duration.............: avg=221.61ms min=157.39ms med=220.54ms max=625.47ms p(90)=229.12ms p(95)=231.82ms p(99.9)=536.69ms
     iterations.....................: 13541   167.557712/s
     success_rate...................: 100.00% ✓ 13541      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 38061      ✗ 0    
     data_received..................: 1.1 GB  15 MB/s
     data_sent......................: 15 MB   195 kB/s
     http_req_blocked...............: avg=24.77µs  min=1.16µs   med=2.6µs    max=11.3ms   p(90)=4.17µs   p(95)=5.51µs   p(99.9)=8.21ms  
     http_req_connecting............: avg=21.34µs  min=0s       med=0s       max=11.26ms  p(90)=0s       p(95)=0s       p(99.9)=8.16ms  
     http_req_duration..............: avg=235.83ms min=159.96ms med=228.6ms  max=696.88ms p(90)=247.92ms p(95)=260.34ms p(99.9)=635.59ms
       { expected_response:true }...: avg=235.83ms min=159.96ms med=228.6ms  max=696.88ms p(90)=247.92ms p(95)=260.34ms p(99.9)=635.59ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 12787
     http_req_receiving.............: avg=87.23µs  min=26.16µs  med=45.47µs  max=13.24ms  p(90)=98.49µs  p(95)=157.9µs  p(99.9)=6.17ms  
     http_req_sending...............: avg=76.82µs  min=5.59µs   med=10.86µs  max=166.1ms  p(90)=32.34µs  p(95)=115.05µs p(99.9)=4.2ms   
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=235.66ms min=159.88ms med=228.47ms max=695.73ms p(90)=247.73ms p(95)=259.96ms p(99.9)=634.96ms
     http_reqs......................: 12787   167.327297/s
     iteration_duration.............: avg=236.77ms min=169.1ms  med=229ms    max=709.58ms p(90)=248.23ms p(95)=260.73ms p(99.9)=640.65ms
     iterations.....................: 12687   166.018724/s
     success_rate...................: 100.00% ✓ 12687      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway-bun`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 37236      ✗ 0    
     data_received..................: 1.1 GB  14 MB/s
     data_sent......................: 15 MB   191 kB/s
     http_req_blocked...............: avg=24.07µs  min=1.09µs   med=2.55µs   max=10.63ms  p(90)=4.12µs   p(95)=5.41µs   p(99.9)=7.76ms  
     http_req_connecting............: avg=20.57µs  min=0s       med=0s       max=10.59ms  p(90)=0s       p(95)=0s       p(99.9)=7.71ms  
     http_req_duration..............: avg=240.75ms min=157.75ms med=233.02ms max=738.77ms p(90)=253.88ms p(95)=266.08ms p(99.9)=645.84ms
       { expected_response:true }...: avg=240.75ms min=157.75ms med=233.02ms max=738.77ms p(90)=253.88ms p(95)=266.08ms p(99.9)=645.84ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 12512
     http_req_receiving.............: avg=89.4µs   min=24.9µs   med=45.01µs  max=11.14ms  p(90)=102.78µs p(95)=246.18µs p(99.9)=4.64ms  
     http_req_sending...............: avg=84.41µs  min=5.44µs   med=10.76µs  max=304.89ms p(90)=33.04µs  p(95)=125.58µs p(99.9)=3.81ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=240.58ms min=157.7ms  med=232.91ms max=738.69ms p(90)=253.74ms p(95)=265.94ms p(99.9)=645.79ms
     http_reqs......................: 12512   163.811666/s
     iteration_duration.............: avg=241.79ms min=157.96ms med=233.38ms max=775.51ms p(90)=254.32ms p(95)=266.77ms p(99.9)=687.51ms
     iterations.....................: 12412   162.50243/s
     success_rate...................: 100.00% ✓ 12412      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-router`

K6 Output

     ✓ response code was 200
     ✗ no graphql errors
      ↳  99% — ✓ 11030 / ✗ 1
     ✓ valid response structure

     checks.........................: 99.99% ✓ 33092      ✗ 1    
     data_received..................: 977 MB 12 MB/s
     data_sent......................: 13 MB  159 kB/s
     http_req_blocked...............: avg=25.59µs  min=1.47µs   med=2.9µs    max=9.11ms   p(90)=4.18µs   p(95)=5.37µs   p(99.9)=7.21ms  
     http_req_connecting............: avg=20.95µs  min=0s       med=0s       max=9.08ms   p(90)=0s       p(95)=0s       p(99.9)=7.19ms  
     http_req_duration..............: avg=271.44ms min=149.96ms med=272.54ms max=683.78ms p(90)=305.31ms p(95)=314.44ms p(99.9)=597.88ms
       { expected_response:true }...: avg=271.44ms min=149.96ms med=272.54ms max=683.78ms p(90)=305.31ms p(95)=314.44ms p(99.9)=597.88ms
     http_req_failed................: 0.00%  ✓ 0          ✗ 11131
     http_req_receiving.............: avg=127.28µs min=31.08µs  med=51.32µs  max=8.26ms   p(90)=158.42µs p(95)=449.87µs p(99.9)=4.64ms  
     http_req_sending...............: avg=128.05µs min=6.33µs   med=11.22µs  max=275.58ms p(90)=88.2µs   p(95)=143.95µs p(99.9)=5.39ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=271.18ms min=149.9ms  med=272.33ms max=682.67ms p(90)=305.08ms p(95)=314.18ms p(99.9)=596.82ms
     http_reqs......................: 11131  136.734254/s
     iteration_duration.............: avg=272.44ms min=150.22ms med=273.21ms max=709ms    p(90)=305.75ms p(95)=314.84ms p(99.9)=617.94ms
     iterations.....................: 11031  135.505845/s
     success_rate...................: 99.99% ✓ 11030      ✗ 1    
     vus............................: 50     min=0        max=50 
     vus_max........................: 50     min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 30153      ✗ 0    
     data_received..................: 892 MB  11 MB/s
     data_sent......................: 12 MB   145 kB/s
     http_req_blocked...............: avg=31.32µs  min=1.42µs   med=3.64µs   max=10.3ms   p(90)=5.59µs   p(95)=6.57µs   p(99.9)=8.25ms  
     http_req_connecting............: avg=26.14µs  min=0s       med=0s       max=10.27ms  p(90)=0s       p(95)=0s       p(99.9)=8.23ms  
     http_req_duration..............: avg=297.72ms min=210.94ms med=295.38ms max=813.7ms  p(90)=324.95ms p(95)=335.93ms p(99.9)=720.29ms
       { expected_response:true }...: avg=297.72ms min=210.94ms med=295.38ms max=813.7ms  p(90)=324.95ms p(95)=335.93ms p(99.9)=720.29ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 10151
     http_req_receiving.............: avg=93.78µs  min=30.5µs   med=71.71µs  max=35.39ms  p(90)=121.36µs p(95)=153.29µs p(99.9)=2.9ms   
     http_req_sending...............: avg=116.4µs  min=6.35µs   med=14.41µs  max=225.99ms p(90)=29.78µs  p(95)=44.69µs  p(99.9)=5.53ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=297.51ms min=210.83ms med=295.23ms max=811.47ms p(90)=324.72ms p(95)=335.71ms p(99.9)=718.59ms
     http_reqs......................: 10151   124.333732/s
     iteration_duration.............: avg=299.05ms min=225.38ms med=296ms    max=841.77ms p(90)=325.49ms p(95)=336.52ms p(99.9)=762.34ms
     iterations.....................: 10051   123.108889/s
     success_rate...................: 100.00% ✓ 10051      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview

Copy link

Overview for: ramping-vus

This scenario runs 4 subgraphs and a GraphQL gateway with Federation spec, and runs a heavy query. We are running a heavy load of concurrent VUs to measure response time and other stats, during stress. It measure things like memory usage, CPU usage, response times. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was running 500 VUs over 60s

Comparison

Comparison

Gateway RPS ⬇️ Requests Duration Notes
hive-router 1774 110677 total, 0 failed avg: 126ms, p95: 306ms
grafbase 1542 97327 total, 0 failed avg: 143ms, p95: 330ms
cosmo 716 45618 total, 0 failed avg: 308ms, p95: 684ms
apollo-router 391 26293 total, 0 failed avg: 537ms, p95: 1253ms ❌ 18 unexpected GraphQL errors, non-compatible response structure (1)
hive-gateway 270 18705 total, 0 failed avg: 749ms, p95: 1683ms
hive-gateway-bun 238 16879 total, 0 failed avg: 829ms, p95: 1794ms
apollo-gateway 151 10395 total, 0 failed avg: 1244ms, p95: 2584ms
Summary for: `hive-router`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 329031      ✗ 0     
     data_received..................: 9.7 GB  156 MB/s
     data_sent......................: 129 MB  2.1 MB/s
     http_req_blocked...............: avg=532.13µs min=1.15µs  med=2.61µs   max=373.9ms  p(90)=4.12µs   p(95)=4.95µs   p(99.9)=184.28ms
     http_req_connecting............: avg=528.56µs min=0s      med=0s       max=373.84ms p(90)=0s       p(95)=0s       p(99.9)=184.18ms
     http_req_duration..............: avg=125.75ms min=1.72ms  med=113.87ms max=444.33ms p(90)=257.96ms p(95)=305.84ms p(99.9)=399.15ms
       { expected_response:true }...: avg=125.75ms min=1.72ms  med=113.87ms max=444.33ms p(90)=257.96ms p(95)=305.84ms p(99.9)=399.15ms
     http_req_failed................: 0.00%   ✓ 0           ✗ 110677
     http_req_receiving.............: avg=553.94µs min=23.46µs med=43.65µs  max=155.77ms p(90)=104.82µs p(95)=413.98µs p(99.9)=77ms    
     http_req_sending...............: avg=463.79µs min=5.35µs  med=10.3µs   max=145.29ms p(90)=23.67µs  p(95)=134.04µs p(99.9)=78.83ms 
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=124.74ms min=1.67ms  med=112.76ms max=422.35ms p(90)=255.73ms p(95)=303.65ms p(99.9)=393.96ms
     http_reqs......................: 110677  1774.991916/s
     iteration_duration.............: avg=127.74ms min=1.87ms  med=115.64ms max=763.52ms p(90)=260.5ms  p(95)=308.57ms p(99.9)=443.42ms
     iterations.....................: 109677  1758.95433/s
     success_rate...................: 100.00% ✓ 109677      ✗ 0     
     vus............................: 78      min=0         max=493 
     vus_max........................: 500     min=500       max=500 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `grafbase`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 288981      ✗ 0    
     data_received..................: 8.6 GB  136 MB/s
     data_sent......................: 113 MB  1.8 MB/s
     http_req_blocked...............: avg=683.62µs min=1.19µs  med=3.02µs   max=382.86ms p(90)=4.29µs   p(95)=5.02µs   p(99.9)=220.12ms
     http_req_connecting............: avg=678.03µs min=0s      med=0s       max=382.8ms  p(90)=0s       p(95)=0s       p(99.9)=220.06ms
     http_req_duration..............: avg=142.96ms min=2.5ms   med=132.52ms max=491.99ms p(90)=282.06ms p(95)=330.11ms p(99.9)=425.23ms
       { expected_response:true }...: avg=142.96ms min=2.5ms   med=132.52ms max=491.99ms p(90)=282.06ms p(95)=330.11ms p(99.9)=425.23ms
     http_req_failed................: 0.00%   ✓ 0           ✗ 97327
     http_req_receiving.............: avg=651.01µs min=23.96µs med=46.53µs  max=180.98ms p(90)=105.53µs p(95)=428.16µs p(99.9)=85.09ms 
     http_req_sending...............: avg=582.15µs min=5.09µs  med=11.72µs  max=144.46ms p(90)=30.72µs  p(95)=138.4µs  p(99.9)=84.02ms 
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=141.72ms min=2.46ms  med=131.2ms  max=433.71ms p(90)=279.18ms p(95)=328.27ms p(99.9)=419.2ms 
     http_reqs......................: 97327   1542.421836/s
     iteration_duration.............: avg=145.5ms  min=2.66ms  med=134.92ms max=895.43ms p(90)=285.02ms p(95)=333.04ms p(99.9)=516.69ms
     iterations.....................: 96327   1526.574005/s
     success_rate...................: 100.00% ✓ 96327       ✗ 0    
     vus............................: 65      min=0         max=497
     vus_max........................: 500     min=500       max=500

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `cosmo`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 133854     ✗ 0    
     data_received..................: 4.0 GB  63 MB/s
     data_sent......................: 53 MB   834 kB/s
     http_req_blocked...............: avg=117.85µs min=1.23µs med=2.72µs   max=209.36ms p(90)=3.99µs   p(95)=5.28µs   p(99.9)=38.35ms 
     http_req_connecting............: avg=113.32µs min=0s     med=0s       max=209.27ms p(90)=0s       p(95)=0s       p(99.9)=38.29ms 
     http_req_duration..............: avg=307.74ms min=2.61ms med=293.58ms max=1.26s    p(90)=606.66ms p(95)=684.06ms p(99.9)=1.02s   
       { expected_response:true }...: avg=307.74ms min=2.61ms med=293.58ms max=1.26s    p(90)=606.66ms p(95)=684.06ms p(99.9)=1.02s   
     http_req_failed................: 0.00%   ✓ 0          ✗ 45618
     http_req_receiving.............: avg=827.36µs min=25.5µs med=56.92µs  max=185.78ms p(90)=180.88µs p(95)=489.03µs p(99.9)=107.04ms
     http_req_sending...............: avg=190.09µs min=5.53µs med=10.95µs  max=140.71ms p(90)=28.98µs  p(95)=125.64µs p(99.9)=41.16ms 
     http_req_tls_handshaking.......: avg=0s       min=0s     med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=306.72ms min=2.55ms med=292.79ms max=1.26s    p(90)=605.22ms p(95)=682.85ms p(99.9)=1.01s   
     http_reqs......................: 45618   716.577697/s
     iteration_duration.............: avg=315.18ms min=2.8ms  med=303.14ms max=1.26s    p(90)=610.69ms p(95)=687.45ms p(99.9)=1.02s   
     iterations.....................: 44618   700.869475/s
     success_rate...................: 100.00% ✓ 44618      ✗ 0    
     vus............................: 92      min=0        max=500
     vus_max........................: 500     min=500      max=500

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-router`

K6 Output

     ✓ response code was 200
     ✗ no graphql errors
      ↳  99% — ✓ 25275 / ✗ 18
     ✗ valid response structure
      ↳  99% — ✓ 25292 / ✗ 1

     checks.........................: 99.97% ✓ 75860      ✗ 19   
     data_received..................: 2.3 GB 34 MB/s
     data_sent......................: 31 MB  456 kB/s
     http_req_blocked...............: avg=67.58µs  min=1.09µs  med=2.81µs   max=112.5ms  p(90)=4.72µs   p(95)=6.88µs   p(99.9)=15.11ms
     http_req_connecting............: avg=62.39µs  min=0s      med=0s       max=112.3ms  p(90)=0s       p(95)=0s       p(99.9)=15.05ms
     http_req_duration..............: avg=536.58ms min=6.19ms  med=484.6ms  max=1.92s    p(90)=1.1s     p(95)=1.25s    p(99.9)=1.72s  
       { expected_response:true }...: avg=536.58ms min=6.19ms  med=484.6ms  max=1.92s    p(90)=1.1s     p(95)=1.25s    p(99.9)=1.72s  
     http_req_failed................: 0.00%  ✓ 0          ✗ 26293
     http_req_receiving.............: avg=104.42µs min=28.12µs med=52.42µs  max=50.58ms  p(90)=102.28µs p(95)=152.63µs p(99.9)=6.98ms 
     http_req_sending...............: avg=93.44µs  min=4.94µs  med=11.21µs  max=101.14ms p(90)=27.13µs  p(95)=107.13µs p(99.9)=15.23ms
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s     
     http_req_waiting...............: avg=536.38ms min=6.13ms  med=484.27ms max=1.92s    p(90)=1.1s     p(95)=1.25s    p(99.9)=1.72s  
     http_reqs......................: 26293  391.23648/s
     iteration_duration.............: avg=558.16ms min=6.4ms   med=511.19ms max=1.92s    p(90)=1.11s    p(95)=1.25s    p(99.9)=1.72s  
     iterations.....................: 25293  376.356607/s
     success_rate...................: 99.92% ✓ 25275      ✗ 18   
     vus............................: 73     min=0        max=495
     vus_max........................: 500    min=500      max=500

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 53115      ✗ 0    
     data_received..................: 1.6 GB  24 MB/s
     data_sent......................: 22 MB   317 kB/s
     http_req_blocked...............: avg=165.24µs min=1.3µs   med=2.77µs   max=140.18ms p(90)=4.64µs  p(95)=7.62µs   p(99.9)=43.6ms 
     http_req_connecting............: avg=160.16µs min=0s      med=0s       max=140.01ms p(90)=0s      p(95)=0s       p(99.9)=43.52ms
     http_req_duration..............: avg=749.41ms min=6.79ms  med=692.44ms max=4.71s    p(90)=1.48s   p(95)=1.68s    p(99.9)=3.7s   
       { expected_response:true }...: avg=749.41ms min=6.79ms  med=692.44ms max=4.71s    p(90)=1.48s   p(95)=1.68s    p(99.9)=3.7s   
     http_req_failed................: 0.00%   ✓ 0          ✗ 18705
     http_req_receiving.............: avg=172.54µs min=25.43µs med=43.83µs  max=112.55ms p(90)=91.8µs  p(95)=154.98µs p(99.9)=25.58ms
     http_req_sending...............: avg=223.69µs min=5.74µs  med=11.04µs  max=119.79ms p(90)=31.76µs p(95)=121.42µs p(99.9)=37.97ms
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s      p(95)=0s       p(99.9)=0s     
     http_req_waiting...............: avg=749.01ms min=6.71ms  med=692.31ms max=4.71s    p(90)=1.48s   p(95)=1.68s    p(99.9)=3.69s  
     http_reqs......................: 18705   270.296313/s
     iteration_duration.............: avg=792.22ms min=7.13ms  med=723.68ms max=4.72s    p(90)=1.49s   p(95)=1.69s    p(99.9)=3.74s  
     iterations.....................: 17705   255.845828/s
     success_rate...................: 100.00% ✓ 17705      ✗ 0    
     vus............................: 72      min=0        max=500
     vus_max........................: 500     min=500      max=500

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway-bun`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 47637      ✗ 0    
     data_received..................: 1.5 GB  21 MB/s
     data_sent......................: 20 MB   280 kB/s
     http_req_blocked...............: avg=183.99µs min=1.19µs  med=3.15µs   max=217.56ms p(90)=5.93µs   p(95)=9.6µs    p(99.9)=34.31ms
     http_req_connecting............: avg=176.14µs min=0s      med=0s       max=217.49ms p(90)=0s       p(95)=0s       p(99.9)=34.16ms
     http_req_duration..............: avg=828.8ms  min=7.19ms  med=768.62ms max=5.83s    p(90)=1.56s    p(95)=1.79s    p(99.9)=4.31s  
       { expected_response:true }...: avg=828.8ms  min=7.19ms  med=768.62ms max=5.83s    p(90)=1.56s    p(95)=1.79s    p(99.9)=4.31s  
     http_req_failed................: 0.00%   ✓ 0          ✗ 16879
     http_req_receiving.............: avg=251.78µs min=28.38µs med=50.58µs  max=178.05ms p(90)=116.72µs p(95)=387.22µs p(99.9)=32.6ms 
     http_req_sending...............: avg=269.67µs min=6.09µs  med=11.97µs  max=168.53ms p(90)=39.5µs   p(95)=139.53µs p(99.9)=40.87ms
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s     
     http_req_waiting...............: avg=828.28ms min=7.12ms  med=767.49ms max=5.83s    p(90)=1.56s    p(95)=1.79s    p(99.9)=4.31s  
     http_reqs......................: 16879   238.655162/s
     iteration_duration.............: avg=881.4ms  min=8.15ms  med=844.37ms max=5.84s    p(90)=1.58s    p(95)=1.81s    p(99.9)=4.5s   
     iterations.....................: 15879   224.515986/s
     success_rate...................: 100.00% ✓ 15879      ✗ 0    
     vus............................: 104     min=0        max=500
     vus_max........................: 500     min=500      max=500

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 28185      ✗ 0    
     data_received..................: 913 MB  13 MB/s
     data_sent......................: 12 MB   181 kB/s
     http_req_blocked...............: avg=36.65µs min=1.24µs med=3.42µs  max=20.6ms  p(90)=6.14µs   p(95)=15.64µs  p(99.9)=6.81ms
     http_req_connecting............: avg=29.37µs min=0s     med=0s      max=20.39ms p(90)=0s       p(95)=0s       p(99.9)=6.75ms
     http_req_duration..............: avg=1.24s   min=7.28ms med=1.23s   max=15.97s  p(90)=2.19s    p(95)=2.58s    p(99.9)=14.74s
       { expected_response:true }...: avg=1.24s   min=7.28ms med=1.23s   max=15.97s  p(90)=2.19s    p(95)=2.58s    p(99.9)=14.74s
     http_req_failed................: 0.00%   ✓ 0          ✗ 10395
     http_req_receiving.............: avg=88.24µs min=27.1µs med=61.28µs max=17.87ms p(90)=114.65µs p(95)=146.67µs p(99.9)=2.94ms
     http_req_sending...............: avg=46.91µs min=6.04µs med=13.67µs max=18.04ms p(90)=33.14µs  p(95)=55.39µs  p(99.9)=7.66ms
     http_req_tls_handshaking.......: avg=0s      min=0s     med=0s      max=0s      p(90)=0s       p(95)=0s       p(99.9)=0s    
     http_req_waiting...............: avg=1.24s   min=7.23ms med=1.23s   max=15.97s  p(90)=2.19s    p(95)=2.58s    p(99.9)=14.74s
     http_reqs......................: 10395   151.427729/s
     iteration_duration.............: avg=1.37s   min=8.44ms med=1.35s   max=15.98s  p(90)=2.21s    p(95)=2.65s    p(99.9)=14.87s
     iterations.....................: 9395    136.860367/s
     success_rate...................: 100.00% ✓ 9395       ✗ 0    
     vus............................: 111     min=0        max=500
     vus_max........................: 500     min=500      max=500

Performance Overview

Performance Overview

HTTP Overview

no-image-available

@ardatan ardatan merged commit 21bfc81 into main Sep 29, 2025
34 checks passed
@ardatan ardatan deleted the renovate/docker-login-action-digest branch September 29, 2025 15:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant