Skip to content

Conversation

renovate[bot]
Copy link
Contributor

@renovate renovate bot commented Oct 6, 2025

Note

Mend has cancelled the proposed renaming of the Renovate GitHub app being renamed to mend[bot].

This notice will be removed on 2025-10-07.


This PR contains the following updates:

Update Change
lockFileMaintenance All locks refreshed

🔧 This Pull Request updates lock files to use the latest dependency versions.


Configuration

📅 Schedule: Branch creation - "before 4am on monday" (UTC), Automerge - At any time (no schedule defined).

🚦 Automerge: Enabled.

Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

👻 Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.


  • If you want to rebase/retry this PR, check this box

This PR was generated by Mend Renovate. View the repository job log.

@renovate renovate bot added the dependencies Pull requests that update a dependency file label Oct 6, 2025
Copy link
Contributor Author

renovate bot commented Oct 6, 2025

⚠️ Artifact update problem

Renovate failed to update artifacts related to this branch. You probably do not want to merge this PR as-is.

♻ Renovate will retry this branch, including artifacts, only when one of the following happens:

  • any of the package files in this branch needs updating, or
  • the branch becomes conflicted, or
  • you click the rebase/retry checkbox if found above, or
  • you rename this PR's title to start with "rebase!" to trigger it manually

The artifact failure details are included below:

File name: Cargo.lock
Command failed: cargo update --config net.git-fetch-with-cli=true --manifest-path subgraphs/Cargo.toml
error: failed to acquire package cache lock

Caused by:
  failed to open: /home/ubuntu/.cargo/.package-cache

Caused by:
  failed to create directory `/home/ubuntu/.cargo`

Caused by:
  File exists (os error 17)

File name: Cargo.lock
Command failed: cargo update --config net.git-fetch-with-cli=true --manifest-path toolkit/Cargo.toml
error: failed to acquire package cache lock

Caused by:
  failed to open: /home/ubuntu/.cargo/.package-cache

Caused by:
  failed to create directory `/home/ubuntu/.cargo`

Caused by:
  File exists (os error 17)

Copy link

github-actions bot commented Oct 6, 2025

💻 Website Preview

The latest changes are available as preview in: https://598cfcad.federation-gateway-benchmark.pages.dev

Copy link

github-actions bot commented Oct 6, 2025

Overview for: constant-vus-over-time

This scenario runs 4 subgraphs and a GraphQL gateway with Federation spec, and runs a heavy query. It's being executed with a constant amount of VUs over a fixed amount of time. It measure things like memory usage, CPU usage, average RPS. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was running 50 VUs over 60s

Comparison

Comparison

Gateway RPS ⬇️ Requests Duration Notes
hive-router 1797 108299 total, 0 failed avg: 27ms, p95: 49ms
grafbase 1610 97191 total, 0 failed avg: 31ms, p95: 54ms
cosmo 698 42208 total, 0 failed avg: 71ms, p95: 104ms
apollo-router 388 23596 total, 0 failed avg: 127ms, p95: 165ms
hive-gateway 301 18450 total, 0 failed avg: 163ms, p95: 241ms
hive-gateway-bun 301 18357 total, 0 failed avg: 163ms, p95: 346ms
apollo-gateway 116 7112 total, 0 failed avg: 423ms, p95: 498ms
Summary for: `hive-router`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 324597      ✗ 0     
     data_received..................: 9.5 GB  158 MB/s
     data_sent......................: 126 MB  2.1 MB/s
     http_req_blocked...............: avg=5.04µs   min=1.19µs med=2.52µs  max=8.93ms   p(90)=3.64µs   p(95)=4.24µs   p(99.9)=84.49µs
     http_req_connecting............: avg=2.08µs   min=0s     med=0s      max=8.9ms    p(90)=0s       p(95)=0s       p(99.9)=0s     
     http_req_duration..............: avg=27.45ms  min=1.85ms med=24.65ms max=429.56ms p(90)=43.18ms  p(95)=49.45ms  p(99.9)=85.6ms 
       { expected_response:true }...: avg=27.45ms  min=1.85ms med=24.65ms max=429.56ms p(90)=43.18ms  p(95)=49.45ms  p(99.9)=85.6ms 
     http_req_failed................: 0.00%   ✓ 0           ✗ 108299
     http_req_receiving.............: avg=124.54µs min=23.5µs med=41.99µs max=52.98ms  p(90)=100.44µs p(95)=329.33µs p(99.9)=15.46ms
     http_req_sending...............: avg=100.88µs min=5.28µs med=10.01µs max=336.6ms  p(90)=28.28µs  p(95)=123.57µs p(99.9)=15.66ms
     http_req_tls_handshaking.......: avg=0s       min=0s     med=0s      max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s     
     http_req_waiting...............: avg=27.23ms  min=1.81ms med=24.49ms max=428.06ms p(90)=42.76ms  p(95)=49.01ms  p(99.9)=81.38ms
     http_reqs......................: 108299  1797.292667/s
     iteration_duration.............: avg=27.72ms  min=3.63ms med=24.89ms max=485.51ms p(90)=43.42ms  p(95)=49.71ms  p(99.9)=86.7ms 
     iterations.....................: 108199  1795.633102/s
     success_rate...................: 100.00% ✓ 108199      ✗ 0     
     vus............................: 50      min=50        max=50  
     vus_max........................: 50      min=50        max=50  

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `grafbase`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 291273      ✗ 0    
     data_received..................: 8.5 GB  142 MB/s
     data_sent......................: 113 MB  1.9 MB/s
     http_req_blocked...............: avg=5.92µs   min=1.17µs med=2.97µs  max=9.14ms   p(90)=4.07µs   p(95)=4.75µs   p(99.9)=114.51µs
     http_req_connecting............: avg=2.38µs   min=0s     med=0s      max=9.11ms   p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_duration..............: avg=30.58ms  min=2.57ms med=27.4ms  max=463.35ms p(90)=47.17ms  p(95)=53.91ms  p(99.9)=90.88ms 
       { expected_response:true }...: avg=30.58ms  min=2.57ms med=27.4ms  max=463.35ms p(90)=47.17ms  p(95)=53.91ms  p(99.9)=90.88ms 
     http_req_failed................: 0.00%   ✓ 0           ✗ 97191
     http_req_receiving.............: avg=150.83µs min=27.1µs med=47.41µs max=405.86ms p(90)=125.53µs p(95)=386.68µs p(99.9)=15.62ms 
     http_req_sending...............: avg=112.35µs min=5.56µs med=11.64µs max=208.08ms p(90)=35.52µs  p(95)=135.13µs p(99.9)=16.74ms 
     http_req_tls_handshaking.......: avg=0s       min=0s     med=0s      max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=30.32ms  min=2.52ms med=27.2ms  max=435.52ms p(90)=46.7ms   p(95)=53.39ms  p(99.9)=89.14ms 
     http_reqs......................: 97191   1610.94907/s
     iteration_duration.............: avg=30.89ms  min=6.01ms med=27.67ms max=486.49ms p(90)=47.46ms  p(95)=54.2ms   p(99.9)=92.36ms 
     iterations.....................: 97091   1609.291561/s
     success_rate...................: 100.00% ✓ 97091       ✗ 0    
     vus............................: 50      min=50        max=50 
     vus_max........................: 50      min=50        max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `cosmo`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 126324     ✗ 0    
     data_received..................: 3.7 GB  61 MB/s
     data_sent......................: 49 MB   813 kB/s
     http_req_blocked...............: avg=9.42µs   min=1.29µs  med=2.81µs  max=9.74ms   p(90)=3.87µs   p(95)=4.62µs   p(99.9)=1.4ms   
     http_req_connecting............: avg=5.87µs   min=0s      med=0s      max=9.71ms   p(90)=0s       p(95)=0s       p(99.9)=1.29ms  
     http_req_duration..............: avg=70.79ms  min=2.83ms  med=69.7ms  max=508.22ms p(90)=96.29ms  p(95)=103.98ms p(99.9)=291.17ms
       { expected_response:true }...: avg=70.79ms  min=2.83ms  med=69.7ms  max=508.22ms p(90)=96.29ms  p(95)=103.98ms p(99.9)=291.17ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 42208
     http_req_receiving.............: avg=165.99µs min=28.91µs med=70.68µs max=118.15ms p(90)=145.63µs p(95)=389.91µs p(99.9)=16.6ms  
     http_req_sending...............: avg=64.75µs  min=6.02µs  med=11.34µs max=348.22ms p(90)=29.8µs   p(95)=126.67µs p(99.9)=4.04ms  
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s      max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=70.56ms  min=2.65ms  med=69.5ms  max=487.46ms p(90)=96ms     p(95)=103.61ms p(99.9)=274.2ms 
     http_reqs......................: 42208   698.730532/s
     iteration_duration.............: avg=71.26ms  min=7.26ms  med=70.03ms max=552.97ms p(90)=96.61ms  p(95)=104.27ms p(99.9)=298.55ms
     iterations.....................: 42108   697.075087/s
     success_rate...................: 100.00% ✓ 42108      ✗ 0    
     vus............................: 50      min=50       max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-router`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 70488      ✗ 0    
     data_received..................: 2.1 GB  34 MB/s
     data_sent......................: 27 MB   452 kB/s
     http_req_blocked...............: avg=12.14µs  min=1.79µs  med=2.8µs    max=7.82ms   p(90)=4.05µs   p(95)=4.76µs   p(99.9)=4.45ms  
     http_req_connecting............: avg=8.69µs   min=0s      med=0s       max=7.78ms   p(90)=0s       p(95)=0s       p(99.9)=4.43ms  
     http_req_duration..............: avg=126.96ms min=6ms     med=126.39ms max=600.43ms p(90)=155.8ms  p(95)=164.67ms p(99.9)=422.58ms
       { expected_response:true }...: avg=126.96ms min=6ms     med=126.39ms max=600.43ms p(90)=155.8ms  p(95)=164.67ms p(99.9)=422.58ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 23596
     http_req_receiving.............: avg=89.92µs  min=35.5µs  med=51.22µs  max=190.48ms p(90)=101.01µs p(95)=137.31µs p(99.9)=2.17ms  
     http_req_sending...............: avg=82.62µs  min=7.15µs  med=11.41µs  max=324.83ms p(90)=19.47µs  p(95)=39.86µs  p(99.9)=2.38ms  
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=126.79ms min=5.95ms  med=126.28ms max=599.22ms p(90)=155.63ms p(95)=164.51ms p(99.9)=410.18ms
     http_reqs......................: 23596   388.420077/s
     iteration_duration.............: avg=127.78ms min=28.51ms med=126.75ms max=616.51ms p(90)=156.17ms p(95)=165.02ms p(99.9)=429.6ms 
     iterations.....................: 23496   386.77395/s
     success_rate...................: 100.00% ✓ 23496      ✗ 0    
     vus............................: 50      min=50       max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 55050      ✗ 0    
     data_received..................: 1.6 GB  27 MB/s
     data_sent......................: 22 MB   350 kB/s
     http_req_blocked...............: avg=20.58µs  min=1.21µs  med=2.82µs   max=11.85ms  p(90)=4.55µs   p(95)=5.68µs   p(99.9)=7.3ms   
     http_req_connecting............: avg=16.31µs  min=0s      med=0s       max=11.81ms  p(90)=0s       p(95)=0s       p(99.9)=7.25ms  
     http_req_duration..............: avg=162.5ms  min=6.82ms  med=150.86ms max=679.75ms p(90)=183.9ms  p(95)=241.05ms p(99.9)=501.93ms
       { expected_response:true }...: avg=162.5ms  min=6.82ms  med=150.86ms max=679.75ms p(90)=183.9ms  p(95)=241.05ms p(99.9)=501.93ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 18450
     http_req_receiving.............: avg=115.93µs min=27.22µs med=50.07µs  max=370.41ms p(90)=113.05µs p(95)=191.41µs p(99.9)=3.87ms  
     http_req_sending...............: avg=140.56µs min=5.61µs  med=11.45µs  max=432.02ms p(90)=33.38µs  p(95)=126.3µs  p(99.9)=5.41ms  
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=162.25ms min=6.77ms  med=150.71ms max=679.48ms p(90)=183.72ms p(95)=239.93ms p(99.9)=500.66ms
     http_reqs......................: 18450   301.272716/s
     iteration_duration.............: avg=163.74ms min=39.41ms med=151.26ms max=721.34ms p(90)=184.29ms p(95)=258.25ms p(99.9)=517.82ms
     iterations.....................: 18350   299.639802/s
     success_rate...................: 100.00% ✓ 18350      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway-bun`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 54771      ✗ 0    
     data_received..................: 1.6 GB  27 MB/s
     data_sent......................: 21 MB   350 kB/s
     http_req_blocked...............: avg=19.03µs  min=1.17µs  med=2.75µs   max=11.32ms  p(90)=4.52µs   p(95)=5.64µs   p(99.9)=7.03ms  
     http_req_connecting............: avg=15.26µs  min=0s      med=0s       max=11.28ms  p(90)=0s       p(95)=0s       p(99.9)=6.98ms  
     http_req_duration..............: avg=163.08ms min=6.87ms  med=151.02ms max=659.88ms p(90)=184.8ms  p(95)=346.49ms p(99.9)=488.62ms
       { expected_response:true }...: avg=163.08ms min=6.87ms  med=151.02ms max=659.88ms p(90)=184.8ms  p(95)=346.49ms p(99.9)=488.62ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 18357
     http_req_receiving.............: avg=94.61µs  min=26.47µs med=46.76µs  max=16.15ms  p(90)=104.26µs p(95)=223.73µs p(99.9)=4.94ms  
     http_req_sending...............: avg=89.59µs  min=5.57µs  med=11.45µs  max=222.26ms p(90)=33.12µs  p(95)=130.34µs p(99.9)=5.5ms   
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=162.9ms  min=6.81ms  med=150.89ms max=659.18ms p(90)=184.61ms p(95)=346.24ms p(99.9)=478.67ms
     http_reqs......................: 18357   301.207232/s
     iteration_duration.............: avg=164.42ms min=69.11ms med=151.44ms max=716.66ms p(90)=185.38ms p(95)=347.71ms p(99.9)=610.67ms
     iterations.....................: 18257   299.566401/s
     success_rate...................: 100.00% ✓ 18257      ✗ 0    
     vus............................: 50      min=50       max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 21036      ✗ 0   
     data_received..................: 625 MB  10 MB/s
     data_sent......................: 8.3 MB  135 kB/s
     http_req_blocked...............: avg=43.99µs  min=1.34µs   med=3.07µs   max=11.91ms  p(90)=5.18µs   p(95)=5.93µs   p(99.9)=9.91ms  
     http_req_connecting............: avg=40.28µs  min=0s       med=0s       max=11.88ms  p(90)=0s       p(95)=0s       p(99.9)=9.86ms  
     http_req_duration..............: avg=422.69ms min=7.58ms   med=421.99ms max=858.39ms p(90)=480.2ms  p(95)=497.91ms p(99.9)=769.99ms
       { expected_response:true }...: avg=422.69ms min=7.58ms   med=421.99ms max=858.39ms p(90)=480.2ms  p(95)=497.91ms p(99.9)=769.99ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 7112
     http_req_receiving.............: avg=70.66µs  min=29.19µs  med=57.4µs   max=1.42ms   p(90)=109.96µs p(95)=127.34µs p(99.9)=498.16µs
     http_req_sending...............: avg=120.63µs min=6.24µs   med=12.49µs  max=168.31ms p(90)=21.82µs  p(95)=25.92µs  p(99.9)=7.17ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=422.49ms min=7.5ms    med=421.9ms  max=857.06ms p(90)=480.02ms p(95)=497.8ms  p(99.9)=768.81ms
     http_reqs......................: 7112    116.230865/s
     iteration_duration.............: avg=429.08ms min=139.65ms med=422.68ms max=874.59ms p(90)=481.13ms p(95)=498.73ms p(99.9)=795.28ms
     iterations.....................: 7012    114.596573/s
     success_rate...................: 100.00% ✓ 7012       ✗ 0   
     vus............................: 39      min=39       max=50
     vus_max........................: 50      min=50       max=50

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview

Copy link

github-actions bot commented Oct 6, 2025

Overview for: ramping-vus

This scenario runs 4 subgraphs and a GraphQL gateway with Federation spec, and runs a heavy query. We are running a heavy load of concurrent VUs to measure response time and other stats, during stress. It measure things like memory usage, CPU usage, response times. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was running 500 VUs over 60s

Comparison

Comparison

Gateway RPS ⬇️ Requests Duration Notes
hive-router 1848 115065 total, 0 failed avg: 121ms, p95: 289ms
grafbase 1523 96751 total, 0 failed avg: 144ms, p95: 339ms
cosmo 696 44427 total, 0 failed avg: 316ms, p95: 691ms
apollo-router 390 26239 total, 0 failed avg: 537ms, p95: 1206ms ❌ 17 unexpected GraphQL errors, non-compatible response structure (1)
hive-gateway-bun 271 18843 total, 0 failed avg: 746ms, p95: 1641ms
hive-gateway 266 18513 total, 0 failed avg: 757ms, p95: 1621ms
apollo-gateway 156 10698 total, 0 failed avg: 1223ms, p95: 2659ms
Summary for: `hive-router`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 342195      ✗ 0     
     data_received..................: 10 GB   162 MB/s
     data_sent......................: 134 MB  2.2 MB/s
     http_req_blocked...............: avg=506.33µs min=1.13µs  med=2.5µs    max=350.56ms p(90)=3.85µs   p(95)=4.49µs   p(99.9)=178.62ms
     http_req_connecting............: avg=502.98µs min=0s      med=0s       max=350.48ms p(90)=0s       p(95)=0s       p(99.9)=178.56ms
     http_req_duration..............: avg=120.98ms min=1.6ms   med=110.23ms max=426.56ms p(90)=242.3ms  p(95)=288.72ms p(99.9)=381.09ms
       { expected_response:true }...: avg=120.98ms min=1.6ms   med=110.23ms max=426.56ms p(90)=242.3ms  p(95)=288.72ms p(99.9)=381.09ms
     http_req_failed................: 0.00%   ✓ 0           ✗ 115065
     http_req_receiving.............: avg=542.19µs min=22.57µs med=40.93µs  max=135.75ms p(90)=92.19µs  p(95)=383.51µs p(99.9)=74.54ms 
     http_req_sending...............: avg=448.44µs min=5.06µs  med=10.13µs  max=138.53ms p(90)=20.98µs  p(95)=124.57µs p(99.9)=67.47ms 
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=119.99ms min=1.56ms  med=109.35ms max=394.48ms p(90)=240.25ms p(95)=286.78ms p(99.9)=378.27ms
     http_reqs......................: 115065  1848.831682/s
     iteration_duration.............: avg=122.83ms min=1.77ms  med=111.57ms max=771.14ms p(90)=244.76ms p(95)=291.34ms p(99.9)=415.9ms 
     iterations.....................: 114065  1832.763966/s
     success_rate...................: 100.00% ✓ 114065      ✗ 0     
     vus............................: 70      min=0         max=496 
     vus_max........................: 500     min=500       max=500 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `grafbase`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 287253      ✗ 0    
     data_received..................: 8.5 GB  134 MB/s
     data_sent......................: 113 MB  1.8 MB/s
     http_req_blocked...............: avg=705.54µs min=1.16µs med=3.04µs   max=406.08ms p(90)=4.41µs   p(95)=5.29µs   p(99.9)=224.73ms
     http_req_connecting............: avg=701.52µs min=0s     med=0s       max=406.02ms p(90)=0s       p(95)=0s       p(99.9)=224.66ms
     http_req_duration..............: avg=143.75ms min=2.54ms med=132.71ms max=489.22ms p(90)=285.25ms p(95)=338.77ms p(99.9)=428.84ms
       { expected_response:true }...: avg=143.75ms min=2.54ms med=132.71ms max=489.22ms p(90)=285.25ms p(95)=338.77ms p(99.9)=428.84ms
     http_req_failed................: 0.00%   ✓ 0           ✗ 96751
     http_req_receiving.............: avg=652.94µs min=24µs   med=48.23µs  max=161.92ms p(90)=111.98µs p(95)=437.8µs  p(99.9)=85.44ms 
     http_req_sending...............: avg=610.29µs min=5.29µs med=11.73µs  max=157.92ms p(90)=31.29µs  p(95)=140.95µs p(99.9)=86.99ms 
     http_req_tls_handshaking.......: avg=0s       min=0s     med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=142.49ms min=2.49ms med=131.3ms  max=444.33ms p(90)=282.56ms p(95)=335.96ms p(99.9)=424.93ms
     http_reqs......................: 96751   1523.245464/s
     iteration_duration.............: avg=146.34ms min=2.81ms med=134.97ms max=871.9ms  p(90)=288.22ms p(95)=342.37ms p(99.9)=542.84ms
     iterations.....................: 95751   1507.501487/s
     success_rate...................: 100.00% ✓ 95751       ✗ 0    
     vus............................: 85      min=0         max=495
     vus_max........................: 500     min=500       max=500

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `cosmo`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 130281     ✗ 0    
     data_received..................: 3.9 GB  61 MB/s
     data_sent......................: 52 MB   811 kB/s
     http_req_blocked...............: avg=151.56µs min=1.27µs  med=2.83µs   max=316.85ms p(90)=4.24µs   p(95)=5.74µs   p(99.9)=40.64ms 
     http_req_connecting............: avg=146.54µs min=0s      med=0s       max=316.68ms p(90)=0s       p(95)=0s       p(99.9)=40.19ms 
     http_req_duration..............: avg=315.91ms min=2.85ms  med=302.43ms max=1.31s    p(90)=609.68ms p(95)=691.2ms  p(99.9)=1.14s   
       { expected_response:true }...: avg=315.91ms min=2.85ms  med=302.43ms max=1.31s    p(90)=609.68ms p(95)=691.2ms  p(99.9)=1.14s   
     http_req_failed................: 0.00%   ✓ 0          ✗ 44427
     http_req_receiving.............: avg=1.9ms    min=27.67µs med=60.2µs   max=445.17ms p(90)=220.74µs p(95)=495.93µs p(99.9)=334.7ms 
     http_req_sending...............: avg=194.52µs min=5.93µs  med=11.36µs  max=121.97ms p(90)=31.99µs  p(95)=133.8µs  p(99.9)=37.64ms 
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=313.81ms min=2.76ms  med=301.7ms  max=1.17s    p(90)=607.72ms p(95)=687.92ms p(99.9)=983.72ms
     http_reqs......................: 44427   696.625266/s
     iteration_duration.............: avg=323.81ms min=3.11ms  med=311.7ms  max=1.31s    p(90)=612.82ms p(95)=695.16ms p(99.9)=1.15s   
     iterations.....................: 43427   680.945043/s
     success_rate...................: 100.00% ✓ 43427      ✗ 0    
     vus............................: 98      min=0        max=500
     vus_max........................: 500     min=500      max=500

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-router`

K6 Output

     ✓ response code was 200
     ✗ no graphql errors
      ↳  99% — ✓ 25222 / ✗ 17
     ✗ valid response structure
      ↳  99% — ✓ 25238 / ✗ 1

     checks.........................: 99.97% ✓ 75699      ✗ 18   
     data_received..................: 2.3 GB 34 MB/s
     data_sent......................: 31 MB  456 kB/s
     http_req_blocked...............: avg=102.97µs min=1.63µs  med=3.12µs   max=107.47ms p(90)=4.95µs   p(95)=7.08µs   p(99.9)=36.78ms
     http_req_connecting............: avg=97.43µs  min=0s      med=0s       max=107.41ms p(90)=0s       p(95)=0s       p(99.9)=36.71ms
     http_req_duration..............: avg=537.33ms min=6.03ms  med=504.97ms max=1.88s    p(90)=1.07s    p(95)=1.2s     p(99.9)=1.72s  
       { expected_response:true }...: avg=537.33ms min=6.03ms  med=504.97ms max=1.88s    p(90)=1.07s    p(95)=1.2s     p(99.9)=1.72s  
     http_req_failed................: 0.00%  ✓ 0          ✗ 26239
     http_req_receiving.............: avg=136.3µs  min=30.75µs med=53.67µs  max=75.73ms  p(90)=106.65µs p(95)=252.11µs p(99.9)=15.35ms
     http_req_sending...............: avg=130.23µs min=7.57µs  med=11.85µs  max=94.53ms  p(90)=31.08µs  p(95)=125.49µs p(99.9)=23.11ms
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s     
     http_req_waiting...............: avg=537.06ms min=5.98ms  med=504.67ms max=1.88s    p(90)=1.07s    p(95)=1.2s     p(99.9)=1.72s  
     http_reqs......................: 26239  390.585497/s
     iteration_duration.............: avg=559.14ms min=6.52ms  med=529.41ms max=1.88s    p(90)=1.08s    p(95)=1.21s    p(99.9)=1.73s  
     iterations.....................: 25239  375.699811/s
     success_rate...................: 99.93% ✓ 25222      ✗ 17   
     vus............................: 71     min=0        max=495
     vus_max........................: 500    min=500      max=500

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway-bun`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 53529      ✗ 0    
     data_received..................: 1.7 GB  24 MB/s
     data_sent......................: 22 MB   318 kB/s
     http_req_blocked...............: avg=130.98µs min=1.23µs  med=2.72µs   max=208.55ms p(90)=4.67µs  p(95)=7.76µs   p(99.9)=28.86ms
     http_req_connecting............: avg=125.49µs min=0s      med=0s       max=208.37ms p(90)=0s      p(95)=0s       p(99.9)=28.81ms
     http_req_duration..............: avg=745.97ms min=6.86ms  med=691.94ms max=4.53s    p(90)=1.47s   p(95)=1.64s    p(99.9)=3.76s  
       { expected_response:true }...: avg=745.97ms min=6.86ms  med=691.94ms max=4.53s    p(90)=1.47s   p(95)=1.64s    p(99.9)=3.76s  
     http_req_failed................: 0.00%   ✓ 0          ✗ 18843
     http_req_receiving.............: avg=207.57µs min=23.98µs med=43.75µs  max=137.44ms p(90)=96.87µs p(95)=302.4µs  p(99.9)=32.71ms
     http_req_sending...............: avg=221.85µs min=5.62µs  med=10.89µs  max=146.74ms p(90)=33.19µs p(95)=128.09µs p(99.9)=45.09ms
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s      p(95)=0s       p(99.9)=0s     
     http_req_waiting...............: avg=745.54ms min=6.79ms  med=691.38ms max=4.53s    p(90)=1.47s   p(95)=1.64s    p(99.9)=3.76s  
     http_reqs......................: 18843   271.656507/s
     iteration_duration.............: avg=788.16ms min=8.08ms  med=741.87ms max=4.55s    p(90)=1.49s   p(95)=1.64s    p(99.9)=3.79s  
     iterations.....................: 17843   257.239667/s
     success_rate...................: 100.00% ✓ 17843      ✗ 0    
     vus............................: 86      min=0        max=500
     vus_max........................: 500     min=500      max=500

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 52539      ✗ 0    
     data_received..................: 1.6 GB  23 MB/s
     data_sent......................: 22 MB   312 kB/s
     http_req_blocked...............: avg=93.45µs  min=1.24µs  med=2.83µs   max=77.47ms p(90)=4.66µs  p(95)=7.36µs   p(99.9)=23.54ms
     http_req_connecting............: avg=88.31µs  min=0s      med=0s       max=77.41ms p(90)=0s      p(95)=0s       p(99.9)=23.49ms
     http_req_duration..............: avg=757.11ms min=6.84ms  med=704.62ms max=5.22s   p(90)=1.45s   p(95)=1.62s    p(99.9)=3.99s  
       { expected_response:true }...: avg=757.11ms min=6.84ms  med=704.62ms max=5.22s   p(90)=1.45s   p(95)=1.62s    p(99.9)=3.99s  
     http_req_failed................: 0.00%   ✓ 0          ✗ 18513
     http_req_receiving.............: avg=130.6µs  min=24.97µs med=44.2µs   max=96.75ms p(90)=88.8µs  p(95)=150.36µs p(99.9)=16.35ms
     http_req_sending...............: avg=178.29µs min=5.73µs  med=10.98µs  max=64.93ms p(90)=28.93µs p(95)=96.27µs  p(99.9)=27.93ms
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s      p(90)=0s      p(95)=0s       p(99.9)=0s     
     http_req_waiting...............: avg=756.8ms  min=6.79ms  med=704.12ms max=5.22s   p(90)=1.45s   p(95)=1.62s    p(99.9)=3.99s  
     http_reqs......................: 18513   266.279645/s
     iteration_duration.............: avg=800.62ms min=7.01ms  med=783.97ms max=5.23s   p(90)=1.47s   p(95)=1.62s    p(99.9)=4.05s  
     iterations.....................: 17513   251.896258/s
     success_rate...................: 100.00% ✓ 17513      ✗ 0    
     vus............................: 88      min=0        max=500
     vus_max........................: 500     min=500      max=500

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 29094      ✗ 0    
     data_received..................: 940 MB  14 MB/s
     data_sent......................: 13 MB   187 kB/s
     http_req_blocked...............: avg=26.44µs min=1.29µs  med=3.27µs  max=36.47ms p(90)=5.69µs   p(95)=10.91µs  p(99.9)=2.53ms
     http_req_connecting............: avg=20.33µs min=0s      med=0s      max=36.39ms p(90)=0s       p(95)=0s       p(99.9)=2.47ms
     http_req_duration..............: avg=1.22s   min=7.29ms  med=1.16s   max=15.03s  p(90)=2.19s    p(95)=2.65s    p(99.9)=14.24s
       { expected_response:true }...: avg=1.22s   min=7.29ms  med=1.16s   max=15.03s  p(90)=2.19s    p(95)=2.65s    p(99.9)=14.24s
     http_req_failed................: 0.00%   ✓ 0          ✗ 10698
     http_req_receiving.............: avg=80.65µs min=29.12µs med=55.3µs  max=9.64ms  p(90)=105.58µs p(95)=133.05µs p(99.9)=2.2ms 
     http_req_sending...............: avg=43.5µs  min=6.13µs  med=12.89µs max=27.66ms p(90)=31.08µs  p(95)=50.76µs  p(99.9)=6.21ms
     http_req_tls_handshaking.......: avg=0s      min=0s      med=0s      max=0s      p(90)=0s       p(95)=0s       p(99.9)=0s    
     http_req_waiting...............: avg=1.22s   min=7.24ms  med=1.16s   max=15.03s  p(90)=2.19s    p(95)=2.65s    p(99.9)=14.24s
     http_reqs......................: 10698   156.378276/s
     iteration_duration.............: avg=1.34s   min=7.94ms  med=1.32s   max=15.05s  p(90)=2.24s    p(95)=2.74s    p(99.9)=14.4s 
     iterations.....................: 9698    141.760752/s
     success_rate...................: 100.00% ✓ 9698       ✗ 0    
     vus............................: 98      min=0        max=500
     vus_max........................: 500     min=500      max=500

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview

Copy link

github-actions bot commented Oct 6, 2025

Overview for: constant-vus-subgraphs-delay-resources

This scenario runs 4 subgraphs and a GraphQL gateway with Federation spec, and runs a heavy query. It's being executed with a constant amount of VUs over a fixed amount of time. It measure things like memory usage, CPU usage, average RPS. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was running 50 VUs over 60s

Comparison

Comparison

Gateway RPS ⬇️ Requests Duration Notes
grafbase 173 14041 total, 0 failed avg: 215ms, p95: 226ms
hive-router 170 13794 total, 0 failed avg: 219ms, p95: 230ms
hive-gateway-bun 166 12725 total, 0 failed avg: 237ms, p95: 261ms
hive-gateway 165 12663 total, 0 failed avg: 238ms, p95: 265ms
cosmo 164 13396 total, 0 failed avg: 225ms, p95: 251ms
apollo-router 136 11095 total, 0 failed avg: 272ms, p95: 314ms ❌ 1 unexpected GraphQL errors
apollo-gateway 119 9740 total, 0 failed avg: 310ms, p95: 385ms
Summary for: `grafbase`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 41823     ✗ 0    
     data_received..................: 1.2 GB  15 MB/s
     data_sent......................: 16 MB   202 kB/s
     http_req_blocked...............: avg=19.01µs  min=1.17µs   med=1.91µs   max=8.84ms   p(90)=3.56µs   p(95)=5.19µs   p(99.9)=6.39ms  
     http_req_connecting............: avg=15.57µs  min=0s       med=0s       max=8.8ms    p(90)=0s       p(95)=0s       p(99.9)=6.36ms  
     http_req_duration..............: avg=215.07ms min=157.23ms med=214.44ms max=599.86ms p(90)=223.66ms p(95)=226.4ms  p(99.9)=503.09ms
       { expected_response:true }...: avg=215.07ms min=157.23ms med=214.44ms max=599.86ms p(90)=223.66ms p(95)=226.4ms  p(99.9)=503.09ms
     http_req_failed................: 0.00%   ✓ 0         ✗ 14041
     http_req_receiving.............: avg=82.95µs  min=25.37µs  med=47.69µs  max=16.89ms  p(90)=97.73µs  p(95)=251.99µs p(99.9)=2.81ms  
     http_req_sending...............: avg=101.53µs min=5.4µs    med=8.34µs   max=364.34ms p(90)=26.42µs  p(95)=96.24µs  p(99.9)=8.04ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=214.89ms min=157.16ms med=214.31ms max=599.14ms p(90)=223.51ms p(95)=226.21ms p(99.9)=491.89ms
     http_reqs......................: 14041   173.57775/s
     iteration_duration.............: avg=215.38ms min=157.5ms  med=214.67ms max=625.84ms p(90)=223.88ms p(95)=226.62ms p(99.9)=522.23ms
     iterations.....................: 13941   172.34153/s
     success_rate...................: 100.00% ✓ 13941     ✗ 0    
     vus............................: 50      min=0       max=50 
     vus_max........................: 50      min=50      max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-router`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 41082      ✗ 0    
     data_received..................: 1.2 GB  15 MB/s
     data_sent......................: 16 MB   198 kB/s
     http_req_blocked...............: avg=18.91µs  min=1.17µs   med=1.91µs   max=9ms      p(90)=3.06µs   p(95)=4.08µs   p(99.9)=6.67ms  
     http_req_connecting............: avg=16.51µs  min=0s       med=0s       max=8.96ms   p(90)=0s       p(95)=0s       p(99.9)=6.65ms  
     http_req_duration..............: avg=219.42ms min=159.73ms med=218.51ms max=607.9ms  p(90)=226.93ms p(95)=229.66ms p(99.9)=509.56ms
       { expected_response:true }...: avg=219.42ms min=159.73ms med=218.51ms max=607.9ms  p(90)=226.93ms p(95)=229.66ms p(99.9)=509.56ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 13794
     http_req_receiving.............: avg=70.61µs  min=24.15µs  med=41.16µs  max=9.53ms   p(90)=81.29µs  p(95)=237.62µs p(99.9)=2.26ms  
     http_req_sending...............: avg=95.43µs  min=5.43µs   med=8.09µs   max=377.42ms p(90)=27.4µs   p(95)=94.1µs   p(99.9)=6.46ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=219.25ms min=159.69ms med=218.41ms max=607.19ms p(90)=226.8ms  p(95)=229.47ms p(99.9)=499.28ms
     http_reqs......................: 13794   170.422413/s
     iteration_duration.............: avg=219.74ms min=159.86ms med=218.75ms max=616.97ms p(90)=227.13ms p(95)=229.85ms p(99.9)=518.32ms
     iterations.....................: 13694   169.186931/s
     success_rate...................: 100.00% ✓ 13694      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway-bun`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 37875      ✗ 0    
     data_received..................: 1.1 GB  15 MB/s
     data_sent......................: 15 MB   194 kB/s
     http_req_blocked...............: avg=22.31µs  min=1.12µs   med=2.58µs   max=10.05ms  p(90)=4.05µs   p(95)=5.26µs   p(99.9)=7.34ms  
     http_req_connecting............: avg=18.91µs  min=0s       med=0s       max=9.99ms   p(90)=0s       p(95)=0s       p(99.9)=7.32ms  
     http_req_duration..............: avg=237.06ms min=154.64ms med=228.93ms max=780.45ms p(90)=249.06ms p(95)=261.12ms p(99.9)=653.09ms
       { expected_response:true }...: avg=237.06ms min=154.64ms med=228.93ms max=780.45ms p(90)=249.06ms p(95)=261.12ms p(99.9)=653.09ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 12725
     http_req_receiving.............: avg=95.06µs  min=23.55µs  med=43.99µs  max=142.56ms p(90)=97.51µs  p(95)=238.71µs p(99.9)=4.07ms  
     http_req_sending...............: avg=80.55µs  min=5.29µs   med=10.51µs  max=207.12ms p(90)=32.66µs  p(95)=126.58µs p(99.9)=4.12ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=236.89ms min=154.55ms med=228.8ms  max=767.02ms p(90)=248.77ms p(95)=260.97ms p(99.9)=648.14ms
     http_reqs......................: 12725   166.349621/s
     iteration_duration.............: avg=238.03ms min=154.9ms  med=229.3ms  max=802.53ms p(90)=249.42ms p(95)=261.46ms p(99.9)=678.2ms 
     iterations.....................: 12625   165.042354/s
     success_rate...................: 100.00% ✓ 12625      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 37689      ✗ 0    
     data_received..................: 1.1 GB  15 MB/s
     data_sent......................: 15 MB   192 kB/s
     http_req_blocked...............: avg=21.37µs  min=1.15µs   med=2.55µs   max=9.43ms   p(90)=4.13µs   p(95)=5.37µs   p(99.9)=6.99ms  
     http_req_connecting............: avg=18.06µs  min=0s       med=0s       max=9.4ms    p(90)=0s       p(95)=0s       p(99.9)=6.96ms  
     http_req_duration..............: avg=238.36ms min=160.34ms med=230.78ms max=754.16ms p(90)=252.57ms p(95)=265.37ms p(99.9)=636.74ms
       { expected_response:true }...: avg=238.36ms min=160.34ms med=230.78ms max=754.16ms p(90)=252.57ms p(95)=265.37ms p(99.9)=636.74ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 12663
     http_req_receiving.............: avg=86.45µs  min=25.54µs  med=45.37µs  max=77.26ms  p(90)=97.37µs  p(95)=160.44µs p(99.9)=2.58ms  
     http_req_sending...............: avg=80.27µs  min=5.43µs   med=10.73µs  max=107.4ms  p(90)=32.42µs  p(95)=120.19µs p(99.9)=6.52ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=238.2ms  min=160.26ms med=230.64ms max=753.44ms p(90)=252.37ms p(95)=265.16ms p(99.9)=635.4ms 
     http_reqs......................: 12663   165.468884/s
     iteration_duration.............: avg=239.33ms min=173.22ms med=231.16ms max=777.99ms p(90)=252.88ms p(95)=265.81ms p(99.9)=660.81ms
     iterations.....................: 12563   164.162172/s
     success_rate...................: 100.00% ✓ 12563      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `cosmo`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 39888      ✗ 0    
     data_received..................: 1.2 GB  15 MB/s
     data_sent......................: 16 MB   192 kB/s
     http_req_blocked...............: avg=24.21µs  min=1.42µs   med=3.01µs   max=10.74ms  p(90)=4.64µs   p(95)=6µs      p(99.9)=8.14ms  
     http_req_connecting............: avg=20.09µs  min=0s       med=0s       max=10.71ms  p(90)=0s       p(95)=0s       p(99.9)=8.11ms  
     http_req_duration..............: avg=225.22ms min=121.06ms med=223.84ms max=666.68ms p(90)=245.03ms p(95)=250.99ms p(99.9)=542.06ms
       { expected_response:true }...: avg=225.22ms min=121.06ms med=223.84ms max=666.68ms p(90)=245.03ms p(95)=250.99ms p(99.9)=542.06ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 13396
     http_req_receiving.............: avg=541.88µs min=30.48µs  med=66.84µs  max=239.13ms p(90)=413.69µs p(95)=1.07ms   p(99.9)=28.72ms 
     http_req_sending...............: avg=96.16µs  min=5.9µs    med=11.62µs  max=187.28ms p(90)=33.7µs   p(95)=135.32µs p(99.9)=4.2ms   
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=224.58ms min=119.72ms med=223.3ms  max=665.23ms p(90)=244.38ms p(95)=250.37ms p(99.9)=540.92ms
     http_reqs......................: 13396   164.911727/s
     iteration_duration.............: avg=225.69ms min=121.37ms med=224.24ms max=687.23ms p(90)=245.37ms p(95)=251.34ms p(99.9)=577.63ms
     iterations.....................: 13296   163.680675/s
     success_rate...................: 100.00% ✓ 13296      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-router`

K6 Output

     ✓ response code was 200
     ✗ no graphql errors
      ↳  99% — ✓ 10994 / ✗ 1
     ✓ valid response structure

     checks.........................: 99.99% ✓ 32984      ✗ 1    
     data_received..................: 973 MB 12 MB/s
     data_sent......................: 13 MB  159 kB/s
     http_req_blocked...............: avg=22.27µs  min=1.49µs   med=2.73µs   max=7.76ms   p(90)=4.08µs   p(95)=5.07µs   p(99.9)=6.16ms  
     http_req_connecting............: avg=18.24µs  min=0s       med=0s       max=7.73ms   p(90)=0s       p(95)=0s       p(99.9)=6.14ms  
     http_req_duration..............: avg=272.25ms min=158.13ms med=273.15ms max=685.01ms p(90)=305.43ms p(95)=314.2ms  p(99.9)=599.15ms
       { expected_response:true }...: avg=272.25ms min=158.13ms med=273.15ms max=685.01ms p(90)=305.43ms p(95)=314.2ms  p(99.9)=599.15ms
     http_req_failed................: 0.00%  ✓ 0          ✗ 11095
     http_req_receiving.............: avg=133.12µs min=29.64µs  med=51.54µs  max=20.02ms  p(90)=161.37µs p(95)=466.25µs p(99.9)=6.42ms  
     http_req_sending...............: avg=118.93µs min=6.36µs   med=11.16µs  max=225.96ms p(90)=80.76µs  p(95)=143.85µs p(99.9)=4.55ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=272ms    min=158.02ms med=272.94ms max=683.34ms p(90)=305.2ms  p(95)=313.78ms p(99.9)=598.29ms
     http_reqs......................: 11095  136.355694/s
     iteration_duration.............: avg=273.25ms min=158.42ms med=273.84ms max=702.69ms p(90)=305.87ms p(95)=314.67ms p(99.9)=645.38ms
     iterations.....................: 10995  135.126711/s
     success_rate...................: 99.99% ✓ 10994      ✗ 1    
     vus............................: 50     min=0        max=50 
     vus_max........................: 50     min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 28920      ✗ 0   
     data_received..................: 856 MB  11 MB/s
     data_sent......................: 11 MB   139 kB/s
     http_req_blocked...............: avg=34.71µs  min=1.38µs   med=3.24µs   max=11.57ms  p(90)=4.91µs   p(95)=5.76µs   p(99.9)=9.44ms  
     http_req_connecting............: avg=30.11µs  min=0s       med=0s       max=11.53ms  p(90)=0s       p(95)=0s       p(99.9)=9.42ms  
     http_req_duration..............: avg=310.19ms min=210.58ms med=333.49ms max=803.06ms p(90)=375.53ms p(95)=384.84ms p(99.9)=689.3ms 
       { expected_response:true }...: avg=310.19ms min=210.58ms med=333.49ms max=803.06ms p(90)=375.53ms p(95)=384.84ms p(99.9)=689.3ms 
     http_req_failed................: 0.00%   ✓ 0          ✗ 9740
     http_req_receiving.............: avg=82.18µs  min=31.48µs  med=63.31µs  max=5.89ms   p(90)=111.33µs p(95)=137.25µs p(99.9)=2.08ms  
     http_req_sending...............: avg=86.92µs  min=6.29µs   med=13.34µs  max=166.46ms p(90)=22.79µs  p(95)=38.75µs  p(99.9)=5.68ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=310.02ms min=210.45ms med=333.32ms max=801.75ms p(90)=375.37ms p(95)=384.63ms p(99.9)=687.8ms 
     http_reqs......................: 9740    119.337949/s
     iteration_duration.............: avg=311.69ms min=223.68ms med=334.62ms max=812.39ms p(90)=376.05ms p(95)=385.37ms p(99.9)=720.54ms
     iterations.....................: 9640    118.112714/s
     success_rate...................: 100.00% ✓ 9640       ✗ 0   
     vus............................: 50      min=0        max=50
     vus_max........................: 50      min=50       max=50

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview

Copy link

github-actions bot commented Oct 6, 2025

Overview for: constant-vus-subgraphs-delay

This scenario runs 4 subgraphs and a GraphQL gateway with Federation spec, and runs a heavy query. It's being executed with a constant amount of VUs over a fixed amount of time. It measure things like memory usage, CPU usage, average RPS. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was running 50 VUs over 60s

Comparison

Comparison

Gateway RPS ⬇️ Requests Duration Notes
grafbase 172 13990 total, 0 failed avg: 216ms, p95: 227ms
hive-router 166 13490 total, 0 failed avg: 224ms, p95: 236ms
cosmo 156 12704 total, 0 failed avg: 238ms, p95: 264ms
hive-gateway 149 11447 total, 0 failed avg: 263ms, p95: 295ms
hive-gateway-bun 148 11387 total, 0 failed avg: 265ms, p95: 394ms
apollo-router 125 10236 total, 0 failed avg: 295ms, p95: 340ms
apollo-gateway 86 7059 total, 0 failed avg: 429ms, p95: 469ms
Summary for: `grafbase`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 41670      ✗ 0    
     data_received..................: 1.2 GB  15 MB/s
     data_sent......................: 16 MB   201 kB/s
     http_req_blocked...............: avg=19.17µs  min=1.18µs   med=1.92µs   max=9.24ms   p(90)=3.35µs   p(95)=4.83µs   p(99.9)=6.71ms  
     http_req_connecting............: avg=16.56µs  min=0s       med=0s       max=9.21ms   p(90)=0s       p(95)=0s       p(99.9)=6.69ms  
     http_req_duration..............: avg=216.05ms min=162.19ms med=215.13ms max=608.25ms p(90)=224.82ms p(95)=227.46ms p(99.9)=516.84ms
       { expected_response:true }...: avg=216.05ms min=162.19ms med=215.13ms max=608.25ms p(90)=224.82ms p(95)=227.46ms p(99.9)=516.84ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 13990
     http_req_receiving.............: avg=77.59µs  min=25.8µs   med=44.72µs  max=17.25ms  p(90)=88.52µs  p(95)=227.64µs p(99.9)=3.01ms  
     http_req_sending...............: avg=142.35µs min=6.07µs   med=8.52µs   max=374.66ms p(90)=29.73µs  p(95)=93.96µs  p(99.9)=9.22ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=215.83ms min=162.14ms med=215.03ms max=606.79ms p(90)=224.66ms p(95)=227.28ms p(99.9)=486.55ms
     http_reqs......................: 13990   172.749204/s
     iteration_duration.............: avg=216.38ms min=162.43ms med=215.41ms max=635.06ms p(90)=225.03ms p(95)=227.7ms  p(99.9)=547.46ms
     iterations.....................: 13890   171.514399/s
     success_rate...................: 100.00% ✓ 13890      ✗ 0    
     vus............................: 10      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-router`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 40170      ✗ 0    
     data_received..................: 1.2 GB  15 MB/s
     data_sent......................: 16 MB   194 kB/s
     http_req_blocked...............: avg=19.97µs  min=1.2µs    med=2.24µs   max=9.02ms   p(90)=3.32µs   p(95)=4.47µs   p(99.9)=6.73ms  
     http_req_connecting............: avg=17.28µs  min=0s       med=0s       max=8.99ms   p(90)=0s       p(95)=0s       p(99.9)=6.72ms  
     http_req_duration..............: avg=224.04ms min=126.67ms med=223.06ms max=624.07ms p(90)=232.86ms p(95)=235.99ms p(99.9)=529.67ms
       { expected_response:true }...: avg=224.04ms min=126.67ms med=223.06ms max=624.07ms p(90)=232.86ms p(95)=235.99ms p(99.9)=529.67ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 13490
     http_req_receiving.............: avg=74.56µs  min=24.35µs  med=43.19µs  max=9.93ms   p(90)=85.07µs  p(95)=215.97µs p(99.9)=3.71ms  
     http_req_sending...............: avg=149.08µs min=5.51µs   med=9.15µs   max=378.74ms p(90)=27.09µs  p(95)=94.82µs  p(99.9)=6.93ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=223.82ms min=122.58ms med=222.96ms max=623.29ms p(90)=232.69ms p(95)=235.78ms p(99.9)=514.12ms
     http_reqs......................: 13490   166.712448/s
     iteration_duration.............: avg=224.4ms  min=126.83ms med=223.32ms max=632.67ms p(90)=233.07ms p(95)=236.21ms p(99.9)=540.8ms 
     iterations.....................: 13390   165.476626/s
     success_rate...................: 100.00% ✓ 13390      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `cosmo`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 37812      ✗ 0    
     data_received..................: 1.1 GB  14 MB/s
     data_sent......................: 15 MB   182 kB/s
     http_req_blocked...............: avg=21.64µs  min=1.41µs   med=2.59µs   max=9.28ms   p(90)=3.77µs   p(95)=4.73µs   p(99.9)=7.04ms  
     http_req_connecting............: avg=18.38µs  min=0s       med=0s       max=9.25ms   p(90)=0s       p(95)=0s       p(99.9)=7.02ms  
     http_req_duration..............: avg=237.65ms min=136.07ms med=236.92ms max=652.16ms p(90)=257.8ms  p(95)=264.1ms  p(99.9)=537.33ms
       { expected_response:true }...: avg=237.65ms min=136.07ms med=236.92ms max=652.16ms p(90)=257.8ms  p(95)=264.1ms  p(99.9)=537.33ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 12704
     http_req_receiving.............: avg=296.75µs min=31.65µs  med=58.74µs  max=249.15ms p(90)=167.14µs p(95)=456.03µs p(99.9)=26.78ms 
     http_req_sending...............: avg=102.08µs min=7.45µs   med=10.72µs  max=249.24ms p(90)=29.49µs  p(95)=125.8µs  p(99.9)=5.14ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=237.25ms min=135.93ms med=236.61ms max=633.62ms p(90)=257.4ms  p(95)=263.6ms  p(99.9)=536.19ms
     http_reqs......................: 12704   156.629837/s
     iteration_duration.............: avg=238.21ms min=136.37ms med=237.3ms  max=658.67ms p(90)=258.17ms p(95)=264.45ms p(99.9)=552.98ms
     iterations.....................: 12604   155.39692/s
     success_rate...................: 100.00% ✓ 12604      ✗ 0    
     vus............................: 11      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 34041      ✗ 0    
     data_received..................: 1.0 GB  13 MB/s
     data_sent......................: 13 MB   174 kB/s
     http_req_blocked...............: avg=28.69µs  min=1.11µs   med=2.64µs   max=11.55ms  p(90)=4.2µs    p(95)=5.84µs   p(99.9)=9.17ms  
     http_req_connecting............: avg=25.19µs  min=0s       med=0s       max=11.52ms  p(90)=0s       p(95)=0s       p(99.9)=9.14ms  
     http_req_duration..............: avg=263.22ms min=154.22ms med=254.01ms max=879.67ms p(90)=272.03ms p(95)=294.98ms p(99.9)=776.06ms
       { expected_response:true }...: avg=263.22ms min=154.22ms med=254.01ms max=879.67ms p(90)=272.03ms p(95)=294.98ms p(99.9)=776.06ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 11447
     http_req_receiving.............: avg=97.24µs  min=25.91µs  med=46.33µs  max=44.79ms  p(90)=108.13µs p(95)=387.57µs p(99.9)=3.27ms  
     http_req_sending...............: avg=96.38µs  min=5.27µs   med=11µs     max=234.92ms p(90)=37.03µs  p(95)=148.17µs p(99.9)=6.73ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=263.03ms min=150.35ms med=253.91ms max=878.9ms  p(90)=271.91ms p(95)=293.78ms p(99.9)=774.97ms
     http_reqs......................: 11447   149.952069/s
     iteration_duration.............: avg=264.57ms min=154.53ms med=254.39ms max=903.92ms p(90)=272.4ms  p(95)=296.33ms p(99.9)=812.06ms
     iterations.....................: 11347   148.6421/s
     success_rate...................: 100.00% ✓ 11347      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway-bun`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 33861      ✗ 0    
     data_received..................: 1.0 GB  13 MB/s
     data_sent......................: 13 MB   173 kB/s
     http_req_blocked...............: avg=28.89µs  min=1.19µs   med=2.63µs   max=11.43ms  p(90)=4.12µs   p(95)=5.36µs   p(99.9)=9.07ms  
     http_req_connecting............: avg=24.98µs  min=0s       med=0s       max=11.4ms   p(90)=0s       p(95)=0s       p(99.9)=9.04ms  
     http_req_duration..............: avg=265ms    min=129.76ms med=255.79ms max=709.71ms p(90)=275.43ms p(95)=394.21ms p(99.9)=599.77ms
       { expected_response:true }...: avg=265ms    min=129.76ms med=255.79ms max=709.71ms p(90)=275.43ms p(95)=394.21ms p(99.9)=599.77ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 11387
     http_req_receiving.............: avg=96.09µs  min=25.72µs  med=45.12µs  max=19.51ms  p(90)=101.87µs p(95)=383.03µs p(99.9)=4.31ms  
     http_req_sending...............: avg=120.06µs min=5.53µs   med=10.94µs  max=224.61ms p(90)=36.59µs  p(95)=152.74µs p(99.9)=6.16ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=264.79ms min=129.7ms  med=255.64ms max=708.94ms p(90)=275.16ms p(95)=392.42ms p(99.9)=598.72ms
     http_reqs......................: 11387   148.602918/s
     iteration_duration.............: avg=266.32ms min=129.98ms med=256.16ms max=731.98ms p(90)=275.88ms p(95)=396.28ms p(99.9)=661.97ms
     iterations.....................: 11287   147.297896/s
     success_rate...................: 100.00% ✓ 11287      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-router`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 30408      ✗ 0    
     data_received..................: 898 MB  11 MB/s
     data_sent......................: 12 MB   146 kB/s
     http_req_blocked...............: avg=23.3µs   min=1.61µs   med=2.73µs   max=7.98ms   p(90)=4.04µs   p(95)=4.96µs   p(99.9)=6.4ms   
     http_req_connecting............: avg=19.9µs   min=0s       med=0s       max=7.95ms   p(90)=0s       p(95)=0s       p(99.9)=6.39ms  
     http_req_duration..............: avg=295.28ms min=180.99ms med=296.02ms max=718.44ms p(90)=331.26ms p(95)=340.14ms p(99.9)=622.91ms
       { expected_response:true }...: avg=295.28ms min=180.99ms med=296.02ms max=718.44ms p(90)=331.26ms p(95)=340.14ms p(99.9)=622.91ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 10236
     http_req_receiving.............: avg=86.65µs  min=36.48µs  med=52.99µs  max=6.4ms    p(90)=102.93µs p(95)=218.99µs p(99.9)=1.59ms  
     http_req_sending...............: avg=180.67µs min=7.48µs   med=11µs     max=371.73ms p(90)=28.27µs  p(95)=137.17µs p(99.9)=4.54ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=295.02ms min=180.89ms med=295.86ms max=717.65ms p(90)=331.08ms p(95)=339.94ms p(99.9)=621.82ms
     http_reqs......................: 10236   125.736281/s
     iteration_duration.............: avg=296.49ms min=181.28ms med=296.67ms max=733.45ms p(90)=331.69ms p(95)=340.59ms p(99.9)=670.77ms
     iterations.....................: 10136   124.507908/s
     success_rate...................: 100.00% ✓ 10136      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 20877     ✗ 0   
     data_received..................: 620 MB  7.6 MB/s
     data_sent......................: 8.2 MB  101 kB/s
     http_req_blocked...............: avg=43.21µs  min=1.38µs   med=3.1µs    max=11.48ms  p(90)=4.78µs   p(95)=5.55µs   p(99.9)=9.92ms  
     http_req_connecting............: avg=39.47µs  min=0s       med=0s       max=11.45ms  p(90)=0s       p(95)=0s       p(99.9)=9.89ms  
     http_req_duration..............: avg=428.61ms min=210.91ms med=430.54ms max=949.3ms  p(90)=457.97ms p(95)=469.47ms p(99.9)=847.24ms
       { expected_response:true }...: avg=428.61ms min=210.91ms med=430.54ms max=949.3ms  p(90)=457.97ms p(95)=469.47ms p(99.9)=847.24ms
     http_req_failed................: 0.00%   ✓ 0         ✗ 7059
     http_req_receiving.............: avg=78.43µs  min=29.93µs  med=52.88µs  max=101.88ms p(90)=95.4µs   p(95)=112.28µs p(99.9)=578.91µs
     http_req_sending...............: avg=138.55µs min=6.27µs   med=12.64µs  max=311.01ms p(90)=19.65µs  p(95)=25.82µs  p(99.9)=1.97ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=428.39ms min=210.69ms med=430.45ms max=937.65ms p(90)=457.89ms p(95)=469.15ms p(99.9)=845.85ms
     http_reqs......................: 7059    86.479938/s
     iteration_duration.............: avg=432.27ms min=256.78ms med=431.02ms max=987.03ms p(90)=458.88ms p(95)=469.99ms p(99.9)=956.44ms
     iterations.....................: 6959    85.254836/s
     success_rate...................: 100.00% ✓ 6959      ✗ 0   
     vus............................: 50      min=0       max=50
     vus_max........................: 50      min=50      max=50

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview

@ardatan ardatan merged commit 229775d into main Oct 6, 2025
34 of 35 checks passed
@ardatan ardatan deleted the renovate/lock-file-maintenance branch October 6, 2025 11:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant