Use successful requests for calculating RPS by enisdenjo · Pull Request #659 · graphql-hive/graphql-gateways-benchmark

enisdenjo · 2025-04-16T19:10:36Z

No description provided.

github-actions · 2025-04-16T19:17:05Z

Overview for: `federation/constant-vus-subgraphs-delay-resources`

This scenario runs 4 subgraphs and a GraphQL gateway with Federation spec, and runs a heavy query. It's being executed with a constant amount of VUs over a fixed amount of time. It measure things like memory usage, CPU usage, average RPS. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was running 500 VUs over 30s

Comparison

Gateway	Successful RPS ⬇️	Requests	Duration	Notes
cosmo	183	5780 total, 0 failed	avg: 1323ms, p95: 3017ms	✅
grafbase	163	5141 total, 0 failed	avg: 1595ms, p95: 4215ms	✅
apollo-router	161	5147 total, 2 failed	avg: 1508ms, p95: 3456ms	❌ 2 failed requests, 2 non-200 responses, 2 unexpected GraphQL errors, non-compatible response structure (1)
hive-gateway-bun	88	3083 total, 0 failed	avg: 5178ms, p95: 8721ms	✅
apollo-server	81	2916 total, 0 failed	avg: 5576ms, p95: 33702ms	✅
hive-gateway	73	2690 total, 0 failed	avg: 6071ms, p95: 33948ms	✅
mercurius	72	2398 total, 0 failed	avg: 6489ms, p95: 10834ms	✅

Summary for: `cosmo`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 17280      ✗ 0    
     data_received..................: 507 MB  16 MB/s
     data_sent......................: 6.9 MB  218 kB/s
     http_req_blocked...............: avg=3.79ms  min=1.41µs  med=3.14µs   max=1.8s   p(90)=11.98µs  p(95)=26.32ms
     http_req_connecting............: avg=3.18ms  min=0s      med=0s       max=1.8s   p(90)=0s       p(95)=25.06ms
     http_req_duration..............: avg=1.32s   min=3.75ms  med=1.17s    max=6.24s  p(90)=2.29s    p(95)=3.01s  
       { expected_response:true }...: avg=1.32s   min=3.75ms  med=1.17s    max=6.24s  p(90)=2.29s    p(95)=3.01s  
     http_req_failed................: 0.00%   ✓ 0          ✗ 5780 
     http_req_receiving.............: avg=209.6ms min=33.88µs med=76.02µs  max=5.08s  p(90)=847.53ms p(95)=1.39s  
     http_req_sending...............: avg=21.32ms min=7.38µs  med=14.36µs  max=3.91s  p(90)=1.16ms   p(95)=17.01ms
     http_req_tls_handshaking.......: avg=0s      min=0s      med=0s       max=0s     p(90)=0s       p(95)=0s     
     http_req_waiting...............: avg=1.09s   min=3.68ms  med=998.31ms max=4.32s  p(90)=1.72s    p(95)=2.21s  
     http_reqs......................: 5780    183.53909/s
     iteration_duration.............: avg=2.63s   min=50.18ms med=2.42s    max=12.91s p(90)=4.59s    p(95)=5.45s  
     iterations.....................: 5760    182.904007/s
     vus............................: 219     min=219      max=500
     vus_max........................: 500     min=500      max=500

Performance Overview

Subgraphs Overview

HTTP Overview

Summary for: `grafbase`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 15363      ✗ 0    
     data_received..................: 452 MB  14 MB/s
     data_sent......................: 6.1 MB  194 kB/s
     http_req_blocked...............: avg=3.94ms   min=1.43µs  med=3.86µs   max=2.08s  p(90)=108.73µs p(95)=16.01ms
     http_req_connecting............: avg=3.02ms   min=0s      med=0s       max=2.08s  p(90)=0s       p(95)=12.82ms
     http_req_duration..............: avg=1.59s    min=3.16ms  med=1.28s    max=8.21s  p(90)=3.4s     p(95)=4.21s  
       { expected_response:true }...: avg=1.59s    min=3.16ms  med=1.28s    max=8.21s  p(90)=3.4s     p(95)=4.21s  
     http_req_failed................: 0.00%   ✓ 0          ✗ 5141 
     http_req_receiving.............: avg=485.68ms min=34.5µs  med=89.33µs  max=7.61s  p(90)=1.85s    p(95)=3.14s  
     http_req_sending...............: avg=36.78ms  min=8.11µs  med=17.92µs  max=4.33s  p(90)=1.18ms   p(95)=10.47ms
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s     p(90)=0s       p(95)=0s     
     http_req_waiting...............: avg=1.07s    min=3.1ms   med=971.67ms max=4.77s  p(90)=2.19s    p(95)=2.33s  
     http_reqs......................: 5141    163.729779/s
     iteration_duration.............: avg=2.97s    min=24.21ms med=2.55s    max=15.45s p(90)=5.81s    p(95)=6.86s  
     iterations.....................: 5121    163.092822/s
     vus............................: 97      min=97       max=500
     vus_max........................: 500     min=500      max=500

Performance Overview

Subgraphs Overview

HTTP Overview

Summary for: `apollo-router`

K6 Output

     ✗ response code was 200
      ↳  99% — ✓ 5125 / ✗ 2
     ✗ no graphql errors
      ↳  99% — ✓ 5125 / ✗ 2
     ✗ valid response structure
      ↳  99% — ✓ 5125 / ✗ 1

     █ setup

     checks.........................: 99.96% ✓ 15375      ✗ 5    
     data_received..................: 451 MB 14 MB/s
     data_sent......................: 6.1 MB 192 kB/s
     http_req_blocked...............: avg=3.53ms   min=1.51µs  med=3.55µs  max=2.16s  p(90)=107.71µs p(95)=14.83ms
     http_req_connecting............: avg=3.1ms    min=0s      med=0s      max=2.16s  p(90)=0s       p(95)=14.69ms
     http_req_duration..............: avg=1.5s     min=6.8ms   med=1.27s   max=5.94s  p(90)=2.94s    p(95)=3.45s  
       { expected_response:true }...: avg=1.5s     min=6.8ms   med=1.27s   max=5.94s  p(90)=2.94s    p(95)=3.45s  
     http_req_failed................: 0.03%  ✓ 2          ✗ 5145 
     http_req_receiving.............: avg=408.43ms min=0s      med=85.22µs max=4.64s  p(90)=1.55s    p(95)=2.31s  
     http_req_sending...............: avg=30.01ms  min=8µs     med=15.95µs max=3.61s  p(90)=418.42µs p(95)=10.28ms
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s      max=0s     p(90)=0s       p(95)=0s     
     http_req_waiting...............: avg=1.06s    min=6.73ms  med=1.01s   max=3.19s  p(90)=1.84s    p(95)=2.09s  
     http_reqs......................: 5147   161.993561/s
     iteration_duration.............: avg=2.97s    min=30.71ms med=2.7s    max=11.14s p(90)=5.45s    p(95)=6.17s  
     iterations.....................: 5127   161.364094/s
     vus............................: 300    min=300      max=500
     vus_max........................: 500    min=500      max=500

Performance Overview

Subgraphs Overview

HTTP Overview

Summary for: `hive-gateway-bun`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 9189      ✗ 0    
     data_received..................: 271 MB  7.8 MB/s
     data_sent......................: 3.7 MB  105 kB/s
     http_req_blocked...............: avg=3.35ms  min=1.82µs   med=3.92µs   max=75.02ms  p(90)=12.2ms   p(95)=31.55ms 
     http_req_connecting............: avg=3.24ms  min=0s       med=0s       max=74.98ms  p(90)=11.26ms  p(95)=31.14ms 
     http_req_duration..............: avg=5.17s   min=17.62ms  med=4.85s    max=12.58s   p(90)=7.36s    p(95)=8.72s   
       { expected_response:true }...: avg=5.17s   min=17.62ms  med=4.85s    max=12.58s   p(90)=7.36s    p(95)=8.72s   
     http_req_failed................: 0.00%   ✓ 0         ✗ 3083 
     http_req_receiving.............: avg=78.62ms min=38.79µs  med=216.16µs max=3.16s    p(90)=10.88ms  p(95)=400.42ms
     http_req_sending...............: avg=1.29ms  min=9.7µs    med=22.12µs  max=736.28ms p(90)=599.75µs p(95)=2.85ms  
     http_req_tls_handshaking.......: avg=0s      min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=5.09s   min=17.41ms  med=4.77s    max=12.58s   p(90)=7.29s    p(95)=8.7s    
     http_reqs......................: 3083    88.362059/s
     iteration_duration.............: avg=5.29s   min=499.96ms med=4.96s    max=12.74s   p(90)=7.42s    p(95)=8.84s   
     iterations.....................: 3063    87.788837/s
     vus............................: 187     min=187     max=500
     vus_max........................: 500     min=500     max=500

Performance Overview

Subgraphs Overview

HTTP Overview

Summary for: `apollo-server`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 8688      ✗ 0    
     data_received..................: 256 MB  7.2 MB/s
     data_sent......................: 3.5 MB  97 kB/s
     http_req_blocked...............: avg=2.06ms   min=1.64µs   med=3.62µs   max=81.36ms  p(90)=8.52ms   p(95)=19.23ms 
     http_req_connecting............: avg=1.97ms   min=0s       med=0s       max=25.99ms  p(90)=8.25ms   p(95)=18.81ms 
     http_req_duration..............: avg=5.57s    min=11.9ms   med=1.65s    max=35.27s   p(90)=32.06s   p(95)=33.7s   
       { expected_response:true }...: avg=5.57s    min=11.9ms   med=1.65s    max=35.27s   p(90)=32.06s   p(95)=33.7s   
     http_req_failed................: 0.00%   ✓ 0         ✗ 2916 
     http_req_receiving.............: avg=609.24µs min=45µs     med=108.08µs max=130.61ms p(90)=236.99µs p(95)=545.71µs
     http_req_sending...............: avg=430.66µs min=9.43µs   med=20.1µs   max=122.72ms p(90)=620.94µs p(95)=1.65ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=5.57s    min=11.81ms  med=1.65s    max=35.27s   p(90)=32.06s   p(95)=33.7s   
     http_reqs......................: 2916    81.617048/s
     iteration_duration.............: avg=5.63s    min=396.81ms med=1.66s    max=35.28s   p(90)=32.11s   p(95)=33.71s  
     iterations.....................: 2896    81.05726/s
     vus............................: 69      min=69      max=500
     vus_max........................: 500     min=500     max=500

Performance Overview

Subgraphs Overview

HTTP Overview

Summary for: `hive-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 8010      ✗ 0    
     data_received..................: 236 MB  6.5 MB/s
     data_sent......................: 3.2 MB  88 kB/s
     http_req_blocked...............: avg=3.18ms   min=2.09µs   med=5.51µs   max=61.99ms  p(90)=16.8ms  p(95)=23.33ms
     http_req_connecting............: avg=3.07ms   min=0s       med=0s       max=44.47ms  p(90)=16.61ms p(95)=23ms   
     http_req_duration..............: avg=6.07s    min=15.44ms  med=2.44s    max=35.84s   p(90)=27.97s  p(95)=33.94s 
       { expected_response:true }...: avg=6.07s    min=15.44ms  med=2.44s    max=35.84s   p(90)=27.97s  p(95)=33.94s 
     http_req_failed................: 0.00%   ✓ 0         ✗ 2690 
     http_req_receiving.............: avg=1.49ms   min=49.55µs  med=134.43µs max=187.66ms p(90)=1.32ms  p(95)=3.33ms 
     http_req_sending...............: avg=429.67µs min=10.64µs  med=32.3µs   max=42.12ms  p(90)=1.06ms  p(95)=2.88ms 
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s      p(95)=0s     
     http_req_waiting...............: avg=6.06s    min=15.31ms  med=2.44s    max=35.84s   p(90)=27.97s  p(95)=33.94s 
     http_reqs......................: 2690    73.787641/s
     iteration_duration.............: avg=6.14s    min=357.77ms med=2.48s    max=35.86s   p(90)=28.06s  p(95)=33.98s 
     iterations.....................: 2670    73.239034/s
     vus............................: 31      min=31      max=500
     vus_max........................: 500     min=500     max=500

Performance Overview

Subgraphs Overview

HTTP Overview

Summary for: `mercurius`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 7134      ✗ 0    
     data_received..................: 211 MB  6.4 MB/s
     data_sent......................: 2.8 MB  87 kB/s
     http_req_blocked...............: avg=3.74ms   min=1.63µs   med=3.81µs   max=51.41ms  p(90)=19.79ms  p(95)=25.13ms
     http_req_connecting............: avg=3.52ms   min=0s       med=0s       max=51.37ms  p(90)=19.31ms  p(95)=24.03ms
     http_req_duration..............: avg=6.48s    min=11.32ms  med=6.6s     max=12.66s   p(90)=8.53s    p(95)=10.83s 
       { expected_response:true }...: avg=6.48s    min=11.32ms  med=6.6s     max=12.66s   p(90)=8.53s    p(95)=10.83s 
     http_req_failed................: 0.00%   ✓ 0         ✗ 2398 
     http_req_receiving.............: avg=26.68ms  min=34.94µs  med=101.73µs max=887.49ms p(90)=548.35µs p(95)=88.97ms
     http_req_sending...............: avg=396.73µs min=9.09µs   med=22.04µs  max=9.38ms   p(90)=1.54ms   p(95)=2.62ms 
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s     
     http_req_waiting...............: avg=6.46s    min=11.24ms  med=6.6s     max=12.66s   p(90)=8.53s    p(95)=10.82s 
     http_reqs......................: 2398    72.977878/s
     iteration_duration.............: avg=6.57s    min=454.25ms med=6.61s    max=12.67s   p(90)=8.55s    p(95)=10.87s 
     iterations.....................: 2378    72.369222/s
     vus............................: 255     min=255     max=500
     vus_max........................: 500     min=500     max=500

Performance Overview

Subgraphs Overview

HTTP Overview

github-actions · 2025-04-16T19:17:15Z

Overview for: `federation/constant-vus-over-time`

This scenario runs 4 subgraphs and a GraphQL gateway with Federation spec, and runs a heavy query. It's being executed with a constant amount of VUs over a fixed amount of time. It measure things like memory usage, CPU usage, average RPS. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was running 300 VUs over 30s

Comparison

Gateway	Successful RPS ⬇️	Requests	Duration	Notes
cosmo	187	5788 total, 0 failed	avg: 875ms, p95: 2185ms	✅
apollo-router	172	5378 total, 0 failed	avg: 966ms, p95: 2409ms	✅
grafbase	172	5293 total, 0 failed	avg: 945ms, p95: 2461ms	✅
apollo-server	85	2844 total, 0 failed	avg: 3308ms, p95: 27727ms	✅
hive-gateway-bun	85	2878 total, 0 failed	avg: 3290ms, p95: 5337ms	✅
hive-gateway	72	2443 total, 0 failed	avg: 3852ms, p95: 26439ms	✅
mercurius	71	2302 total, 0 failed	avg: 4001ms, p95: 5996ms	✅

Summary for: `cosmo`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 17304      ✗ 0    
     data_received..................: 508 MB  17 MB/s
     data_sent......................: 6.9 MB  223 kB/s
     http_req_blocked...............: avg=1.39ms   min=1.48µs  med=3.3µs    max=996.69ms p(90)=6.1µs    p(95)=1.22ms  
     http_req_connecting............: avg=875.86µs min=0s      med=0s       max=924.2ms  p(90)=0s       p(95)=910.71µs
     http_req_duration..............: avg=875.24ms min=3.29ms  med=743.82ms max=4.88s    p(90)=1.68s    p(95)=2.18s   
       { expected_response:true }...: avg=875.24ms min=3.29ms  med=743.82ms max=4.88s    p(90)=1.68s    p(95)=2.18s   
     http_req_failed................: 0.00%   ✓ 0          ✗ 5788 
     http_req_receiving.............: avg=217.79ms min=30.23µs med=79.42µs  max=4.25s    p(90)=980.78ms p(95)=1.49s   
     http_req_sending...............: avg=21.1ms   min=7.96µs  med=15.02µs  max=2.74s    p(90)=376.79µs p(95)=46.62ms 
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=636.34ms min=3.19ms  med=626.02ms max=2.43s    p(90)=1.02s    p(95)=1.2s    
     http_reqs......................: 5788    187.574853/s
     iteration_duration.............: avg=1.57s    min=29.06ms med=1.37s    max=7.63s    p(90)=3.04s    p(95)=3.67s   
     iterations.....................: 5768    186.926702/s
     vus............................: 300     min=300      max=300
     vus_max........................: 300     min=300      max=300

Performance Overview

Subgraphs Overview

HTTP Overview

Summary for: `apollo-router`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 16074      ✗ 0    
     data_received..................: 472 MB  15 MB/s
     data_sent......................: 6.4 MB  205 kB/s
     http_req_blocked...............: avg=1.49ms   min=1.39µs  med=3.09µs   max=1.45s p(90)=5.56µs   p(95)=365.04µs
     http_req_connecting............: avg=1.15ms   min=0s      med=0s       max=1.45s p(90)=0s       p(95)=239.24µs
     http_req_duration..............: avg=965.61ms min=6.32ms  med=751.33ms max=5.67s p(90)=2.02s    p(95)=2.4s    
       { expected_response:true }...: avg=965.61ms min=6.32ms  med=751.33ms max=5.67s p(90)=2.02s    p(95)=2.4s    
     http_req_failed................: 0.00%   ✓ 0          ✗ 5378 
     http_req_receiving.............: avg=326.99ms min=33.29µs med=82.66µs  max=4.81s p(90)=1.42s    p(95)=1.93s   
     http_req_sending...............: avg=31.27ms  min=8.06µs  med=14.39µs  max=2.56s p(90)=159.53µs p(95)=14.29ms 
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s    p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=607.33ms min=6.26ms  med=579.54ms max=2.22s p(90)=1.02s    p(95)=1.18s   
     http_reqs......................: 5378    172.256679/s
     iteration_duration.............: avg=1.71s    min=36.21ms med=1.47s    max=7.69s p(90)=3.44s    p(95)=4.13s   
     iterations.....................: 5358    171.616081/s
     vus............................: 140     min=140      max=300
     vus_max........................: 300     min=300      max=300

Performance Overview

Subgraphs Overview

HTTP Overview

Summary for: `grafbase`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 15819      ✗ 0    
     data_received..................: 465 MB  15 MB/s
     data_sent......................: 6.3 MB  205 kB/s
     http_req_blocked...............: avg=1.3ms    min=1.47µs  med=3.46µs   max=1.74s p(90)=5.83µs   p(95)=364.04µs
     http_req_connecting............: avg=1.09ms   min=0s      med=0s       max=1.74s p(90)=0s       p(95)=294.22µs
     http_req_duration..............: avg=945.14ms min=3.11ms  med=734.65ms max=6.18s p(90)=2.07s    p(95)=2.46s   
       { expected_response:true }...: avg=945.14ms min=3.11ms  med=734.65ms max=6.18s p(90)=2.07s    p(95)=2.46s   
     http_req_failed................: 0.00%   ✓ 0          ✗ 5293 
     http_req_receiving.............: avg=344.91ms min=32.43µs med=89.16µs  max=6s    p(90)=1.18s    p(95)=1.97s   
     http_req_sending...............: avg=19.7ms   min=7.45µs  med=16.1µs   max=3.77s p(90)=310.42µs p(95)=17.17ms 
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s    p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=580.52ms min=3.03ms  med=557.98ms max=2.26s p(90)=1.05s    p(95)=1.24s   
     http_reqs......................: 5293    172.393359/s
     iteration_duration.............: avg=1.71s    min=26.14ms med=1.4s     max=9.09s p(90)=3.48s    p(95)=4.12s   
     iterations.....................: 5273    171.741958/s
     vus............................: 300     min=300      max=300
     vus_max........................: 300     min=300      max=300

Performance Overview

Subgraphs Overview

HTTP Overview

Summary for: `apollo-server`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 8472      ✗ 0    
     data_received..................: 250 MB  7.5 MB/s
     data_sent......................: 3.4 MB  101 kB/s
     http_req_blocked...............: avg=558.04µs min=1.57µs   med=3.44µs   max=22.14ms p(90)=342.86µs p(95)=5.17ms  
     http_req_connecting............: avg=521.05µs min=0s       med=0s       max=10.93ms p(90)=219.41µs p(95)=4.97ms  
     http_req_duration..............: avg=3.3s     min=10.49ms  med=1.39s    max=32.89s  p(90)=2.37s    p(95)=27.72s  
       { expected_response:true }...: avg=3.3s     min=10.49ms  med=1.39s    max=32.89s  p(90)=2.37s    p(95)=27.72s  
     http_req_failed................: 0.00%   ✓ 0         ✗ 2844 
     http_req_receiving.............: avg=259.39µs min=44.59µs  med=106.63µs max=40.53ms p(90)=198.74µs p(95)=342.34µs
     http_req_sending...............: avg=250.25µs min=9.01µs   med=18.02µs  max=43.5ms  p(90)=142.26µs p(95)=2.52ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s      p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=3.3s     min=10.38ms  med=1.39s    max=32.88s  p(90)=2.37s    p(95)=27.72s  
     http_reqs......................: 2844    85.261306/s
     iteration_duration.............: avg=3.34s    min=376.49ms med=1.4s     max=32.9s   p(90)=2.38s    p(95)=28.36s  
     iterations.....................: 2824    84.661719/s
     vus............................: 39      min=39      max=300
     vus_max........................: 300     min=300     max=300

Performance Overview

Subgraphs Overview

HTTP Overview

Summary for: `hive-gateway-bun`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 8574      ✗ 0    
     data_received..................: 253 MB  7.5 MB/s
     data_sent......................: 3.4 MB  101 kB/s
     http_req_blocked...............: avg=2.11ms   min=1.63µs  med=4.03µs   max=48.78ms  p(90)=426.99µs p(95)=20.04ms 
     http_req_connecting............: avg=2.04ms   min=0s      med=0s       max=45.2ms   p(90)=225.47µs p(95)=19.75ms 
     http_req_duration..............: avg=3.29s    min=16.98ms med=2.96s    max=7.88s    p(90)=4.96s    p(95)=5.33s   
       { expected_response:true }...: avg=3.29s    min=16.98ms med=2.96s    max=7.88s    p(90)=4.96s    p(95)=5.33s   
     http_req_failed................: 0.00%   ✓ 0         ✗ 2878 
     http_req_receiving.............: avg=58.76ms  min=40.01µs med=141.27µs max=2.34s    p(90)=20.16ms  p(95)=453.51ms
     http_req_sending...............: avg=830.84µs min=8.88µs  med=21.34µs  max=286.59ms p(90)=523.72µs p(95)=1.85ms  
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=3.23s    min=16.69ms med=2.92s    max=7.88s    p(90)=4.94s    p(95)=5.31s   
     http_reqs......................: 2878    85.340743/s
     iteration_duration.............: avg=3.37s    min=90.61ms med=3.04s    max=7.89s    p(90)=5.04s    p(95)=5.4s    
     iterations.....................: 2858    84.747687/s
     vus............................: 134     min=134     max=300
     vus_max........................: 300     min=300     max=300

Performance Overview

Subgraphs Overview

HTTP Overview

Summary for: `hive-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 7269      ✗ 0    
     data_received..................: 215 MB  6.4 MB/s
     data_sent......................: 2.9 MB  86 kB/s
     http_req_blocked...............: avg=2.94ms   min=1.95µs   med=4.93µs  max=84.27ms p(90)=3.07ms   p(95)=30.94ms
     http_req_connecting............: avg=2.8ms    min=0s       med=0s      max=81.37ms p(90)=2.74ms   p(95)=29.73ms
     http_req_duration..............: avg=3.85s    min=16.19ms  med=1.94s   max=33.03s  p(90)=3.88s    p(95)=26.43s 
       { expected_response:true }...: avg=3.85s    min=16.19ms  med=1.94s   max=33.03s  p(90)=3.88s    p(95)=26.43s 
     http_req_failed................: 0.00%   ✓ 0         ✗ 2443 
     http_req_receiving.............: avg=332.79µs min=39.51µs  med=119.6µs max=32.57ms p(90)=449.63µs p(95)=1.19ms 
     http_req_sending...............: avg=2.02ms   min=9.38µs   med=29.86µs max=42.79ms p(90)=3.39ms   p(95)=17.1ms 
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s      max=0s      p(90)=0s       p(95)=0s     
     http_req_waiting...............: avg=3.84s    min=16.04ms  med=1.94s   max=33.03s  p(90)=3.88s    p(95)=26.42s 
     http_reqs......................: 2443    72.591056/s
     iteration_duration.............: avg=3.9s     min=106.58ms med=1.96s   max=33.05s  p(90)=3.97s    p(95)=26.7s  
     iterations.....................: 2423    71.996778/s
     vus............................: 51      min=51      max=300
     vus_max........................: 300     min=300     max=300

Performance Overview

Subgraphs Overview

HTTP Overview

Summary for: `mercurius`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 6846      ✗ 0    
     data_received..................: 202 MB  6.3 MB/s
     data_sent......................: 2.7 MB  85 kB/s
     http_req_blocked...............: avg=1.25ms   min=1.91µs   med=4.63µs  max=29.98ms  p(90)=3.6ms    p(95)=12.68ms 
     http_req_connecting............: avg=1.21ms   min=0s       med=0s      max=29.79ms  p(90)=3.45ms   p(95)=12.43ms 
     http_req_duration..............: avg=4s       min=14.15ms  med=4.08s   max=8.01s    p(90)=4.96s    p(95)=5.99s   
       { expected_response:true }...: avg=4s       min=14.15ms  med=4.08s   max=8.01s    p(90)=4.96s    p(95)=5.99s   
     http_req_failed................: 0.00%   ✓ 0         ✗ 2302 
     http_req_receiving.............: avg=2.75ms   min=40.6µs   med=108.6µs max=340.98ms p(90)=295.13µs p(95)=781.69µs
     http_req_sending...............: avg=185.08µs min=10.11µs  med=27.07µs max=21.07ms  p(90)=375.94µs p(95)=783.19µs
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s      max=0s       p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=3.99s    min=14.05ms  med=4.08s   max=8.01s    p(90)=4.96s    p(95)=5.99s   
     http_reqs......................: 2302    71.485986/s
     iteration_duration.............: avg=4.06s    min=558.21ms med=4.1s    max=8.07s    p(90)=4.97s    p(95)=6.02s   
     iterations.....................: 2282    70.864909/s
     vus............................: 71      min=71      max=300
     vus_max........................: 300     min=300     max=300

Performance Overview

Subgraphs Overview

HTTP Overview

github-actions · 2025-04-16T19:17:17Z

Overview for: `federation/constant-vus-subgraphs-delay`

This scenario runs 4 subgraphs and a GraphQL gateway with Federation spec, and runs a heavy query. It's being executed with a constant amount of VUs over a fixed amount of time. It measure things like memory usage, CPU usage, average RPS. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was running 300 VUs over 30s

Comparison

Gateway	Successful RPS ⬇️	Requests	Duration	Notes
grafbase	166	5156 total, 0 failed	avg: 963ms, p95: 2454ms	✅
cosmo	161	5007 total, 0 failed	avg: 890ms, p95: 2346ms	✅
apollo-router	154	4813 total, 0 failed	avg: 1001ms, p95: 2279ms	✅
apollo-server	83	2795 total, 0 failed	avg: 3367ms, p95: 30318ms	✅
hive-gateway-bun	83	2744 total, 0 failed	avg: 3362ms, p95: 5421ms	✅
hive-gateway	73	2529 total, 0 failed	avg: 3753ms, p95: 24687ms	✅
mercurius	73	2343 total, 0 failed	avg: 3923ms, p95: 6401ms	✅

Summary for: `grafbase`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 15408      ✗ 0    
     data_received..................: 453 MB  15 MB/s
     data_sent......................: 6.1 MB  198 kB/s
     http_req_blocked...............: avg=1.22ms   min=1.43µs  med=3.37µs   max=1.57s    p(90)=6.14µs   p(95)=278.47µs
     http_req_connecting............: avg=700.97µs min=0s      med=0s       max=933.72ms p(90)=0s       p(95)=154.28µs
     http_req_duration..............: avg=963.2ms  min=3.11ms  med=752.57ms max=5.74s    p(90)=2.13s    p(95)=2.45s   
       { expected_response:true }...: avg=963.2ms  min=3.11ms  med=752.57ms max=5.74s    p(90)=2.13s    p(95)=2.45s   
     http_req_failed................: 0.00%   ✓ 0          ✗ 5156 
     http_req_receiving.............: avg=371.69ms min=34.04µs med=119.25µs max=5.36s    p(90)=1.4s     p(95)=1.78s   
     http_req_sending...............: avg=19.95ms  min=8µs     med=15.58µs  max=2.7s     p(90)=279.91µs p(95)=15.92ms 
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=571.56ms min=3.04ms  med=468.77ms max=2.55s    p(90)=1.2s     p(95)=1.52s   
     http_reqs......................: 5156    166.966811/s
     iteration_duration.............: avg=1.76s    min=25ms    med=1.47s    max=8.75s    p(90)=3.61s    p(95)=4.51s   
     iterations.....................: 5136    166.31915/s
     vus............................: 300     min=300      max=300
     vus_max........................: 300     min=300      max=300

Performance Overview

Subgraphs Overview

HTTP Overview

Summary for: `cosmo`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 14961      ✗ 0    
     data_received..................: 439 MB  14 MB/s
     data_sent......................: 5.9 MB  192 kB/s
     http_req_blocked...............: avg=2.54ms   min=1.49µs  med=3.56µs   max=2.29s p(90)=6.21µs   p(95)=17.18ms
     http_req_connecting............: avg=2.4ms    min=0s      med=0s       max=2.29s p(90)=0s       p(95)=16.85ms
     http_req_duration..............: avg=890.34ms min=4.17ms  med=697.41ms max=4.64s p(90)=1.9s     p(95)=2.34s  
       { expected_response:true }...: avg=890.34ms min=4.17ms  med=697.41ms max=4.64s p(90)=1.9s     p(95)=2.34s  
     http_req_failed................: 0.00%   ✓ 0          ✗ 5007 
     http_req_receiving.............: avg=342.53ms min=40.73µs med=105.83µs max=4.01s p(90)=1.35s    p(95)=1.75s  
     http_req_sending...............: avg=14.24ms  min=8.92µs  med=16.37µs  max=2.45s p(90)=276.05µs p(95)=3.37ms 
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s    p(90)=0s       p(95)=0s     
     http_req_waiting...............: avg=533.56ms min=3.97ms  med=509.34ms max=2.01s p(90)=997.7ms  p(95)=1.06s  
     http_reqs......................: 5007    161.321826/s
     iteration_duration.............: avg=1.82s    min=24.91ms med=1.51s    max=9.73s p(90)=3.75s    p(95)=4.42s  
     iterations.....................: 4987    160.677441/s
     vus............................: 19      min=19       max=300
     vus_max........................: 300     min=300      max=300

Performance Overview

Subgraphs Overview

HTTP Overview

Summary for: `apollo-router`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 14379      ✗ 0    
     data_received..................: 422 MB  14 MB/s
     data_sent......................: 5.7 MB  183 kB/s
     http_req_blocked...............: avg=816.78µs min=1.73µs  med=4.03µs   max=669.22ms p(90)=6.93µs   p(95)=2.32ms
     http_req_connecting............: avg=573.24µs min=0s      med=0s       max=150ms    p(90)=0s       p(95)=1.95ms
     http_req_duration..............: avg=1s       min=7.16ms  med=846.58ms max=5.57s    p(90)=1.96s    p(95)=2.27s 
       { expected_response:true }...: avg=1s       min=7.16ms  med=846.58ms max=5.57s    p(90)=1.96s    p(95)=2.27s 
     http_req_failed................: 0.00%   ✓ 0          ✗ 4813 
     http_req_receiving.............: avg=359.35ms min=34.29µs med=129.18µs max=5.08s    p(90)=1.28s    p(95)=1.6s  
     http_req_sending...............: avg=19.5ms   min=7.78µs  med=21.36µs  max=2.96s    p(90)=256.96µs p(95)=7.41ms
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s    
     http_req_waiting...............: avg=621.67ms min=7.07ms  med=580.73ms max=2.27s    p(90)=1.11s    p(95)=1.2s  
     http_reqs......................: 4813    154.526067/s
     iteration_duration.............: avg=1.89s    min=52.65ms med=1.64s    max=9.76s    p(90)=3.75s    p(95)=4.33s 
     iterations.....................: 4793    153.883948/s
     vus............................: 73      min=73       max=300
     vus_max........................: 300     min=300      max=300

Performance Overview

Subgraphs Overview

HTTP Overview

Summary for: `apollo-server`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 8325      ✗ 0    
     data_received..................: 246 MB  7.4 MB/s
     data_sent......................: 3.3 MB  99 kB/s
     http_req_blocked...............: avg=574.85µs min=1.7µs    med=3.87µs   max=16.38ms p(90)=336.95µs p(95)=5.65ms  
     http_req_connecting............: avg=551.38µs min=0s       med=0s       max=13.73ms p(90)=221.55µs p(95)=5.55ms  
     http_req_duration..............: avg=3.36s    min=12.41ms  med=1.41s    max=32.94s  p(90)=2.25s    p(95)=30.31s  
       { expected_response:true }...: avg=3.36s    min=12.41ms  med=1.41s    max=32.94s  p(90)=2.25s    p(95)=30.31s  
     http_req_failed................: 0.00%   ✓ 0         ✗ 2795 
     http_req_receiving.............: avg=181.76µs min=45.08µs  med=111.91µs max=28.04ms p(90)=197.26µs p(95)=312.73µs
     http_req_sending...............: avg=148.85µs min=9.52µs   med=21.09µs  max=15.76ms p(90)=157.2µs  p(95)=678.46µs
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s      p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=3.36s    min=12.29ms  med=1.41s    max=32.94s  p(90)=2.25s    p(95)=30.31s  
     http_reqs......................: 2795    83.74718/s
     iteration_duration.............: avg=3.4s     min=210.76ms med=1.43s    max=32.95s  p(90)=2.29s    p(95)=30.79s  
     iterations.....................: 2775    83.147916/s
     vus............................: 33      min=33      max=300
     vus_max........................: 300     min=300     max=300

Performance Overview

Subgraphs Overview

HTTP Overview

Summary for: `hive-gateway-bun`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 8172      ✗ 0    
     data_received..................: 241 MB  7.3 MB/s
     data_sent......................: 3.3 MB  99 kB/s
     http_req_blocked...............: avg=361.31µs min=1.97µs   med=4.9µs    max=18.47ms  p(90)=212.6µs  p(95)=3.44ms  
     http_req_connecting............: avg=334.12µs min=0s       med=0s       max=8.36ms   p(90)=125.83µs p(95)=3.34ms  
     http_req_duration..............: avg=3.36s    min=17.47ms  med=3.13s    max=8.03s    p(90)=4.87s    p(95)=5.42s   
       { expected_response:true }...: avg=3.36s    min=17.47ms  med=3.13s    max=8.03s    p(90)=4.87s    p(95)=5.42s   
     http_req_failed................: 0.00%   ✓ 0         ✗ 2744 
     http_req_receiving.............: avg=73.31ms  min=39.38µs  med=216.15µs max=2.01s    p(90)=73.06ms  p(95)=635.54ms
     http_req_sending...............: avg=883.8µs  min=8.54µs   med=26.23µs  max=667.89ms p(90)=214.65µs p(95)=2.05ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=3.28s    min=17.31ms  med=3.08s    max=8.03s    p(90)=4.81s    p(95)=5.35s   
     http_reqs......................: 2744    83.408208/s
     iteration_duration.............: avg=3.43s    min=133.84ms med=3.19s    max=8.05s    p(90)=4.98s    p(95)=5.48s   
     iterations.....................: 2724    82.800277/s
     vus............................: 129     min=129     max=300
     vus_max........................: 300     min=300     max=300

Performance Overview

Subgraphs Overview

HTTP Overview

Summary for: `hive-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 7527      ✗ 0    
     data_received..................: 222 MB  6.5 MB/s
     data_sent......................: 3.0 MB  88 kB/s
     http_req_blocked...............: avg=965.76µs min=1.47µs   med=4.31µs   max=17.8ms   p(90)=1.27ms   p(95)=10.08ms
     http_req_connecting............: avg=942.58µs min=0s       med=0s       max=17.77ms  p(90)=1.07ms   p(95)=9.79ms 
     http_req_duration..............: avg=3.75s    min=15.05ms  med=1.9s     max=33.66s   p(90)=3.85s    p(95)=24.68s 
       { expected_response:true }...: avg=3.75s    min=15.05ms  med=1.9s     max=33.66s   p(90)=3.85s    p(95)=24.68s 
     http_req_failed................: 0.00%   ✓ 0         ✗ 2529 
     http_req_receiving.............: avg=843.18µs min=41.96µs  med=113.19µs max=179.37ms p(90)=424.26µs p(95)=1.33ms 
     http_req_sending...............: avg=210.96µs min=8.84µs   med=25.59µs  max=10.53ms  p(90)=283.42µs p(95)=1.2ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s     
     http_req_waiting...............: avg=3.75s    min=14.9ms   med=1.9s     max=33.66s   p(90)=3.85s    p(95)=24.68s 
     http_reqs......................: 2529    73.895193/s
     iteration_duration.............: avg=3.8s     min=132.46ms med=1.92s    max=33.67s   p(90)=3.93s    p(95)=24.75s 
     iterations.....................: 2509    73.31081/s
     vus............................: 23      min=23      max=300
     vus_max........................: 300     min=300     max=300

Performance Overview

Subgraphs Overview

HTTP Overview

Summary for: `mercurius`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 6969      ✗ 0    
     data_received..................: 206 MB  6.4 MB/s
     data_sent......................: 2.8 MB  87 kB/s
     http_req_blocked...............: avg=215.54µs min=2µs      med=4.26µs   max=12.35ms  p(90)=391.54µs p(95)=1.54ms  
     http_req_connecting............: avg=198.63µs min=0s       med=0s       max=12.31ms  p(90)=332.47µs p(95)=1.45ms  
     http_req_duration..............: avg=3.92s    min=11.52ms  med=3.98s    max=8.81s    p(90)=5.06s    p(95)=6.4s    
       { expected_response:true }...: avg=3.92s    min=11.52ms  med=3.98s    max=8.81s    p(90)=5.06s    p(95)=6.4s    
     http_req_failed................: 0.00%   ✓ 0         ✗ 2343 
     http_req_receiving.............: avg=5.95ms   min=42.78µs  med=106.84µs max=810.71ms p(90)=284.86µs p(95)=692.9µs 
     http_req_sending...............: avg=89.23µs  min=9.63µs   med=24.14µs  max=5.38ms   p(90)=162.16µs p(95)=446.73µs
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=3.91s    min=11.41ms  med=3.98s    max=8.81s    p(90)=5.06s    p(95)=6.4s    
     http_reqs......................: 2343    73.279384/s
     iteration_duration.............: avg=3.98s    min=275.99ms med=4s       max=8.83s    p(90)=5.11s    p(95)=6.42s   
     iterations.....................: 2323    72.653866/s
     vus............................: 221     min=221     max=300
     vus_max........................: 300     min=300     max=300

Performance Overview

Subgraphs Overview

HTTP Overview

github-actions · 2025-04-16T19:18:11Z

Overview for: `federation/ramping-vus`

This scenario runs 4 subgraphs and a GraphQL gateway with Federation spec, and runs a heavy query. We are running a heavy load of concurrent VUs to measure response time and other stats, during stress. It measure things like memory usage, CPU usage, response times. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was trying to reach 2000 concurrent VUs over 60s

Comparison

Gateway	duration(p95)⬇️	Successful RPS	Requests	Durations	Notes
grafbase	5028ms	165	11603 total, 0 failed	avg: 2239ms, p95: 5029ms, max: 11003ms, med: 2134ms	✅
apollo-router	7737ms	160	11910 total, 492 failed	avg: 2488ms, p95: 7737ms, max: 17665ms, med: 1973ms	❌ 492 failed requests, 492 non-200 responses, 492 unexpected GraphQL errors
cosmo	8382ms	175	12517 total, 0 failed	avg: 2860ms, p95: 8382ms, max: 17760ms, med: 2425ms	✅
hive-gateway-bun	29940ms	87	7228 total, 0 failed	avg: 12107ms, p95: 29940ms, max: 38181ms, med: 10507ms	✅
mercurius	43175ms	52	4834 total, 0 failed	avg: 21875ms, p95: 43175ms, max: 44060ms, med: 19009ms	✅
hive-gateway	53098ms	83	7388 total, 0 failed	avg: 12826ms, p95: 53099ms, max: 58305ms, med: 3449ms	✅
apollo-server	60000ms	74	7504 total, 557 failed	avg: 12576ms, p95: 60000ms, max: 60392ms, med: 2325ms	❌ 557 failed requests, 557 non-200 responses, 557 unexpected GraphQL errors

Summary for: `grafbase`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 34749      ✗ 0     
     data_received..................: 1.0 GB  15 MB/s
     data_sent......................: 14 MB   196 kB/s
     http_req_blocked...............: avg=342.37ms min=1.64µs  med=3.91µs  max=6.84s  p(90)=1.57s    p(95)=2.6s    
     http_req_connecting............: avg=340.53ms min=0s      med=0s      max=6.84s  p(90)=1.56s    p(95)=2.6s    
     http_req_duration..............: avg=2.23s    min=4.02ms  med=2.13s   max=11s    p(90)=4.19s    p(95)=5.02s   
       { expected_response:true }...: avg=2.23s    min=4.02ms  med=2.13s   max=11s    p(90)=4.19s    p(95)=5.02s   
     http_req_failed................: 0.00%   ✓ 0          ✗ 11603 
     http_req_receiving.............: avg=169.31ms min=31.53µs med=77.74µs max=9.58s  p(90)=546.14ms p(95)=884.57ms
     http_req_sending...............: avg=123.23ms min=8.53µs  med=19.79µs max=6.89s  p(90)=306.05ms p(95)=824.8ms 
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s      max=0s     p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=1.94s    min=3.93ms  med=1.79s   max=8.4s   p(90)=3.81s    p(95)=4.53s   
     http_reqs......................: 11603   165.448481/s
     iteration_duration.............: avg=6.01s    min=10.85ms med=5.28s   max=28.36s p(90)=12.65s   p(95)=14.78s  
     iterations.....................: 11583   165.163299/s
     vus............................: 4       min=4        max=1986
     vus_max........................: 2000    min=2000     max=2000

Performance Overview

Subgraphs Overview

HTTP Overview

Summary for: `apollo-router`

K6 Output

     ✗ response code was 200
      ↳  95% — ✓ 11398 / ✗ 492
     ✗ no graphql errors
      ↳  95% — ✓ 11398 / ✗ 492
     ✓ valid response structure

     █ setup

     checks.........................: 97.20% ✓ 34194      ✗ 984   
     data_received..................: 1.0 GB 14 MB/s
     data_sent......................: 14 MB  199 kB/s
     http_req_blocked...............: avg=453.27ms min=1.52µs med=3.83µs  max=8.91s  p(90)=1.48s    p(95)=2.91s   
     http_req_connecting............: avg=446.66ms min=0s     med=0s      max=8.91s  p(90)=1.45s    p(95)=2.89s   
     http_req_duration..............: avg=2.48s    min=0s     med=1.97s   max=17.66s p(90)=5.13s    p(95)=7.73s   
       { expected_response:true }...: avg=2.47s    min=6.62ms med=1.95s   max=17.66s p(90)=5.12s    p(95)=7.8s    
     http_req_failed................: 4.13%  ✓ 492        ✗ 11418 
     http_req_receiving.............: avg=732ms    min=0s     med=89.51µs max=9.56s  p(90)=2.21s    p(95)=4.71s   
     http_req_sending...............: avg=208.32ms min=0s     med=19.96µs max=9.46s  p(90)=481.26ms p(95)=995.28ms
     http_req_tls_handshaking.......: avg=0s       min=0s     med=0s      max=0s     p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=1.54s    min=0s     med=1.37s   max=10.33s p(90)=2.97s    p(95)=3.5s    
     http_reqs......................: 11910  167.899898/s
     iteration_duration.............: avg=6.19s    min=30.2ms med=5.23s   max=30.16s p(90)=12.79s   p(95)=15.37s  
     iterations.....................: 11890  167.61795/s
     vus............................: 81     min=58       max=1990
     vus_max........................: 2000   min=2000     max=2000

Performance Overview

Subgraphs Overview

HTTP Overview

Summary for: `cosmo`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 37491      ✗ 0     
     data_received..................: 1.1 GB  15 MB/s
     data_sent......................: 15 MB   209 kB/s
     http_req_blocked...............: avg=376.01ms min=1.51µs  med=4.03µs  max=10.33s p(90)=1.14s    p(95)=3.11s   
     http_req_connecting............: avg=370.44ms min=0s      med=0s      max=10.33s p(90)=1.11s    p(95)=3.09s   
     http_req_duration..............: avg=2.85s    min=3.3ms   med=2.42s   max=17.76s p(90)=5.2s     p(95)=8.38s   
       { expected_response:true }...: avg=2.85s    min=3.3ms   med=2.42s   max=17.76s p(90)=5.2s     p(95)=8.38s   
     http_req_failed................: 0.00%   ✓ 0          ✗ 12517 
     http_req_receiving.............: avg=281.43ms min=34.36µs med=88.89µs max=10.91s p(90)=679.98ms p(95)=1.29s   
     http_req_sending...............: avg=171.62ms min=7.67µs  med=22.07µs max=8.85s  p(90)=312.78ms p(95)=960.01ms
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s      max=0s     p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=2.4s     min=3.21ms  med=1.92s   max=10.3s  p(90)=4.7s     p(95)=5.14s   
     http_reqs......................: 12517   175.767894/s
     iteration_duration.............: avg=6.02s    min=30.6ms  med=5s      max=27.15s p(90)=12.34s   p(95)=14.1s   
     iterations.....................: 12497   175.487047/s
     vus............................: 269     min=70       max=1999
     vus_max........................: 2000    min=2000     max=2000

Performance Overview

Subgraphs Overview

HTTP Overview

Summary for: `hive-gateway-bun`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 21624     ✗ 0     
     data_received..................: 635 MB  7.7 MB/s
     data_sent......................: 8.6 MB  104 kB/s
     http_req_blocked...............: avg=6.2ms    min=2.24µs   med=5.58µs   max=437.62ms p(90)=2.72ms   p(95)=42.46ms
     http_req_connecting............: avg=6.1ms    min=0s       med=0s       max=437.52ms p(90)=2.47ms   p(95)=42.29ms
     http_req_duration..............: avg=12.1s    min=17.62ms  med=10.5s    max=38.18s   p(90)=26s      p(95)=29.94s 
       { expected_response:true }...: avg=12.1s    min=17.62ms  med=10.5s    max=38.18s   p(90)=26s      p(95)=29.94s 
     http_req_failed................: 0.00%   ✓ 0         ✗ 7228  
     http_req_receiving.............: avg=113.66ms min=41.99µs  med=240.64µs max=9.2s     p(90)=4.28ms   p(95)=36.17ms
     http_req_sending...............: avg=3.41ms   min=9.29µs   med=35.18µs  max=875.19ms p(90)=823.85µs p(95)=19.52ms
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s     
     http_req_waiting...............: avg=11.99s   min=17.41ms  med=10.42s   max=37.9s    p(90)=25.99s   p(95)=29.93s 
     http_reqs......................: 7228    87.837618/s
     iteration_duration.............: avg=12.25s   min=189.55ms med=10.61s   max=38.19s   p(90)=26.26s   p(95)=30.26s 
     iterations.....................: 7208    87.59457/s
     vus............................: 100     min=53      max=2000
     vus_max........................: 2000    min=2000    max=2000

Performance Overview

Subgraphs Overview

HTTP Overview

Summary for: `mercurius`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 14441     ✗ 0     
     data_received..................: 424 MB  4.6 MB/s
     data_sent......................: 5.9 MB  64 kB/s
     http_req_blocked...............: avg=180.01µs min=1.56µs  med=4.51µs   max=38.54ms p(90)=438.92µs p(95)=582.19µs
     http_req_connecting............: avg=149µs    min=0s      med=0s       max=38.36ms p(90)=367.47µs p(95)=489.11µs
     http_req_duration..............: avg=21.87s   min=11.17ms med=19s      max=44.06s  p(90)=41.76s   p(95)=43.17s  
       { expected_response:true }...: avg=21.87s   min=11.17ms med=19s      max=44.06s  p(90)=41.76s   p(95)=43.17s  
     http_req_failed................: 0.00%   ✓ 0         ✗ 4834  
     http_req_receiving.............: avg=25.69ms  min=34.98µs med=107.71µs max=1.97s   p(90)=657.37µs p(95)=4.45ms  
     http_req_sending...............: avg=48.46µs  min=9.14µs  med=28µs     max=4.71ms  p(90)=63.12µs  p(95)=79.5µs  
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s      p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=21.84s   min=11.05ms med=19s      max=43.99s  p(90)=41.4s    p(95)=43.09s  
     http_reqs......................: 4834    52.280526/s
     iteration_duration.............: avg=22.01s   min=141.2ms med=19.84s   max=44.47s  p(90)=42.22s   p(95)=43.25s  
     iterations.....................: 4813    52.053408/s
     vus............................: 118     min=60      max=1999
     vus_max........................: 2000    min=2000    max=2000

Performance Overview

Subgraphs Overview

HTTP Overview

Summary for: `hive-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     █ setup

     checks.........................: 100.00% ✓ 22104     ✗ 0     
     data_received..................: 649 MB  7.3 MB/s
     data_sent......................: 8.8 MB  99 kB/s
     http_req_blocked...............: avg=1.4ms    min=1.88µs  med=5.01µs   max=142.87ms p(90)=714.61µs p(95)=5.12ms  
     http_req_connecting............: avg=1.36ms   min=0s      med=0s       max=139.66ms p(90)=558.85µs p(95)=4.84ms  
     http_req_duration..............: avg=12.82s   min=14.63ms med=3.44s    max=58.3s    p(90)=46.45s   p(95)=53.09s  
       { expected_response:true }...: avg=12.82s   min=14.63ms med=3.44s    max=58.3s    p(90)=46.45s   p(95)=53.09s  
     http_req_failed................: 0.00%   ✓ 0         ✗ 7388  
     http_req_receiving.............: avg=970.69µs min=41.79µs med=118.13µs max=292.11ms p(90)=918.02µs p(95)=2.59ms  
     http_req_sending...............: avg=768.45µs min=9.38µs  med=30.16µs  max=255.26ms p(90)=125.38µs p(95)=818.38µs
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=12.82s   min=14.49ms med=3.44s    max=58.3s    p(90)=46.45s   p(95)=53.09s  
     http_reqs......................: 7388    83.043566/s
     iteration_duration.............: avg=12.89s   min=60.21ms med=3.48s    max=58.33s   p(90)=46.51s   p(95)=53.13s  
     iterations.....................: 7368    82.818759/s
     vus............................: 25      min=25      max=2000
     vus_max........................: 2000    min=2000    max=2000

Performance Overview

Subgraphs Overview

HTTP Overview

Summary for: `apollo-server`

K6 Output

     ✗ response code was 200
      ↳  92% — ✓ 6927 / ✗ 557
     ✗ no graphql errors
      ↳  92% — ✓ 6927 / ✗ 557
     ✓ valid response structure

     █ setup

     checks.........................: 94.91% ✓ 20781     ✗ 1114  
     data_received..................: 611 MB 6.6 MB/s
     data_sent......................: 8.9 MB 96 kB/s
     http_req_blocked...............: avg=458.82µs min=1.68µs  med=4.24µs   max=162.35ms p(90)=373.12µs p(95)=557.83µs
     http_req_connecting............: avg=428.56µs min=0s      med=0s       max=162.28ms p(90)=298.17µs p(95)=464.6µs 
     http_req_duration..............: avg=12.57s   min=11.19ms med=2.32s    max=1m0s     p(90)=56.5s    p(95)=1m0s    
       { expected_response:true }...: avg=8.77s    min=11.19ms med=2.22s    max=59.98s   p(90)=37.88s   p(95)=43.32s  
     http_req_failed................: 7.42%  ✓ 557       ✗ 6947  
     http_req_receiving.............: avg=230.39µs min=0s      med=111.58µs max=123.8ms  p(90)=222.39µs p(95)=333.61µs
     http_req_sending...............: avg=225.63µs min=9.36µs  med=22.83µs  max=101.85ms p(90)=67.86µs  p(95)=108.6µs 
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=12.57s   min=11.1ms  med=2.32s    max=1m0s     p(90)=56.5s    p(95)=1m0s    
     http_reqs......................: 7504   80.963876/s
     iteration_duration.............: avg=12.62s   min=60.56ms med=2.34s    max=1m0s     p(90)=56.54s   p(95)=1m0s    
     iterations.....................: 7484   80.748087/s
     vus............................: 44     min=44      max=2000
     vus_max........................: 2000   min=2000    max=2000

Performance Overview

Subgraphs Overview

HTTP Overview

enisdenjo added 3 commits April 16, 2025 21:09

succ rps

4cc307a

format

093c58b

note successful

8c409ff

enisdenjo marked this pull request as draft April 16, 2025 19:20

enisdenjo and others added 9 commits April 16, 2025 21:28

ms to s

98e8fd1

successful rps also in ramping

d16bd23

Update results for scenario: constant-vus-over-time

8bff7fd

Update results for scenario: constant-vus-subgraphs-delay

a4c4a66

Update results for scenario: constant-vus-subgraphs-delay-resources

3f745d8

Update results for scenario: ramping-vus

2129827

format and update

9ec7ce1

use alpha hive gw

227994a

another alpha

65d0200

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use successful requests for calculating RPS#659

Use successful requests for calculating RPS#659
enisdenjo wants to merge 12 commits intomainfrom
succ-reps

enisdenjo commented Apr 16, 2025

Uh oh!

github-actions bot commented Apr 16, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Apr 16, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Apr 16, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Apr 16, 2025 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

enisdenjo commented Apr 16, 2025

Uh oh!

github-actions bot commented Apr 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview for: federation/constant-vus-subgraphs-delay-resources

Comparison

Uh oh!

github-actions bot commented Apr 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview for: federation/constant-vus-over-time

Comparison

Uh oh!

github-actions bot commented Apr 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview for: federation/constant-vus-subgraphs-delay

Comparison

Uh oh!

github-actions bot commented Apr 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview for: federation/ramping-vus

Comparison

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

github-actions bot commented Apr 16, 2025 •

edited

Loading

Overview for: `federation/constant-vus-subgraphs-delay-resources`

github-actions bot commented Apr 16, 2025 •

edited

Loading

Overview for: `federation/constant-vus-over-time`

github-actions bot commented Apr 16, 2025 •

edited

Loading

Overview for: `federation/constant-vus-subgraphs-delay`

github-actions bot commented Apr 16, 2025 •

edited

Loading

Overview for: `federation/ramping-vus`