Skip to content

Conversation

@enisdenjo
Copy link
Member

Related graphql-hive/gateway#1629

Also

  • Updated Hive Router and Grafbase versions to latest
  • No JIT on Hive Gateway with Bun

@github-actions
Copy link

github-actions bot commented Oct 26, 2025

💻 Website Preview

The latest changes are available as preview in: https://a1fc3c50.federation-gateway-benchmark.pages.dev

@github-actions
Copy link

github-actions bot commented Oct 26, 2025

Overview for: constant-vus-over-time

This scenario runs 4 subgraphs and a GraphQL gateway with Federation spec, and runs a heavy query. It's being executed with a constant amount of VUs over a fixed amount of time. It measure things like memory usage, CPU usage, average RPS. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was running 50 VUs over 60s

Comparison

Comparison

Gateway RPS ⬇️ Requests Duration Notes
hive-router 1719 103638 total, 0 failed avg: 29ms, p95: 50ms
grafbase 1466 88611 total, 0 failed avg: 34ms, p95: 59ms
hive-gateway 661 40113 total, 0 failed avg: 74ms, p95: 99ms
cosmo 645 39025 total, 0 failed avg: 77ms, p95: 113ms
hive-gateway-bun 627 38093 total, 0 failed avg: 79ms, p95: 102ms
apollo-router 369 22448 total, 0 failed avg: 133ms, p95: 174ms ❌ 1 unexpected GraphQL errors
apollo-gateway 118 7247 total, 0 failed avg: 415ms, p95: 489ms
Summary for: `hive-router`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 310614      ✗ 0     
     data_received..................: 9.1 GB  151 MB/s
     data_sent......................: 121 MB  2.0 MB/s
     http_req_blocked...............: avg=5.25µs   min=1.12µs  med=2.55µs  max=9.06ms   p(90)=3.63µs   p(95)=4.22µs   p(99.9)=83.53µs
     http_req_connecting............: avg=2.21µs   min=0s      med=0s      max=9.02ms   p(90)=0s       p(95)=0s       p(99.9)=0s     
     http_req_duration..............: avg=28.69ms  min=1.89ms  med=26.26ms max=499.82ms p(90)=43.71ms  p(95)=50.19ms  p(99.9)=91.8ms 
       { expected_response:true }...: avg=28.69ms  min=1.89ms  med=26.26ms max=499.82ms p(90)=43.71ms  p(95)=50.19ms  p(99.9)=91.8ms 
     http_req_failed................: 0.00%   ✓ 0           ✗ 103638
     http_req_receiving.............: avg=118.31µs min=22.32µs med=41.68µs max=129.49ms p(90)=100.33µs p(95)=330.93µs p(99.9)=12.89ms
     http_req_sending...............: avg=106.09µs min=4.93µs  med=9.88µs  max=446.6ms  p(90)=29.13µs  p(95)=123.81µs p(99.9)=14.96ms
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s      max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s     
     http_req_waiting...............: avg=28.47ms  min=1.85ms  med=26.1ms  max=481.11ms p(90)=43.32ms  p(95)=49.78ms  p(99.9)=89.96ms
     http_reqs......................: 103638  1719.783953/s
     iteration_duration.............: avg=28.97ms  min=4.15ms  med=26.5ms  max=543.72ms p(90)=43.95ms  p(95)=50.43ms  p(99.9)=92.29ms
     iterations.....................: 103538  1718.124538/s
     success_rate...................: 100.00% ✓ 103538      ✗ 0     
     vus............................: 50      min=50        max=50  
     vus_max........................: 50      min=50        max=50  

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `grafbase`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 265533      ✗ 0    
     data_received..................: 7.8 GB  129 MB/s
     data_sent......................: 103 MB  1.7 MB/s
     http_req_blocked...............: avg=6.92µs   min=1.23µs  med=3.28µs  max=10.22ms  p(90)=4.96µs   p(95)=6.06µs   p(99.9)=128.18µs
     http_req_connecting............: avg=2.81µs   min=0s      med=0s      max=10.18ms  p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_duration..............: avg=33.55ms  min=3.2ms   med=29.91ms max=528.99ms p(90)=51.57ms  p(95)=58.82ms  p(99.9)=105.53ms
       { expected_response:true }...: avg=33.55ms  min=3.2ms   med=29.91ms max=528.99ms p(90)=51.57ms  p(95)=58.82ms  p(99.9)=105.53ms
     http_req_failed................: 0.00%   ✓ 0           ✗ 88611
     http_req_receiving.............: avg=174.78µs min=28.65µs med=51.63µs max=258.47ms p(90)=171.14µs p(95)=438.82µs p(99.9)=19.4ms  
     http_req_sending...............: avg=129.53µs min=5.96µs  med=12.34µs max=358.09ms p(90)=43.23µs  p(95)=155.94µs p(99.9)=19.26ms 
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s      max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=33.25ms  min=3.09ms  med=29.69ms max=511.33ms p(90)=50.99ms  p(95)=58.17ms  p(99.9)=104.42ms
     http_reqs......................: 88611   1466.58496/s
     iteration_duration.............: avg=33.88ms  min=8.01ms  med=30.2ms  max=579.1ms  p(90)=51.87ms  p(95)=59.12ms  p(99.9)=106.21ms
     iterations.....................: 88511   1464.929878/s
     success_rate...................: 100.00% ✓ 88511       ✗ 0    
     vus............................: 50      min=50        max=50 
     vus_max........................: 50      min=50        max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 120039     ✗ 0    
     data_received..................: 3.5 GB  58 MB/s
     data_sent......................: 47 MB   769 kB/s
     http_req_blocked...............: avg=10.49µs min=1.11µs  med=2.66µs  max=11.44ms  p(90)=4.03µs   p(95)=4.95µs   p(99.9)=2.06ms  
     http_req_connecting............: avg=7.07µs  min=0s      med=0s      max=11.41ms  p(90)=0s       p(95)=0s       p(99.9)=2.03ms  
     http_req_duration..............: avg=74.49ms min=4.76ms  med=72.57ms max=573.49ms p(90)=91.31ms  p(95)=98.75ms  p(99.9)=328.78ms
       { expected_response:true }...: avg=74.49ms min=4.76ms  med=72.57ms max=573.49ms p(90)=91.31ms  p(95)=98.75ms  p(99.9)=328.78ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 40113
     http_req_receiving.............: avg=96.95µs min=26.21µs med=47µs    max=227.42ms p(90)=103.87µs p(95)=235.26µs p(99.9)=4.74ms  
     http_req_sending...............: avg=82.9µs  min=5.53µs  med=10.96µs max=460.98ms p(90)=30.84µs  p(95)=129.8µs  p(99.9)=5.22ms  
     http_req_tls_handshaking.......: avg=0s      min=0s      med=0s      max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=74.31ms min=4.7ms   med=72.43ms max=561.11ms p(90)=91.15ms  p(95)=98.54ms  p(99.9)=318.96ms
     http_reqs......................: 40113   661.521896/s
     iteration_duration.............: avg=75.01ms min=12.67ms med=72.88ms max=614.93ms p(90)=91.62ms  p(95)=99.08ms  p(99.9)=357.44ms
     iterations.....................: 40013   659.87275/s
     success_rate...................: 100.00% ✓ 40013      ✗ 0    
     vus............................: 50      min=50       max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `cosmo`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 116775     ✗ 0    
     data_received..................: 3.4 GB  57 MB/s
     data_sent......................: 45 MB   751 kB/s
     http_req_blocked...............: avg=10.86µs  min=1.41µs  med=3.04µs  max=10.45ms  p(90)=4.41µs   p(95)=5.43µs   p(99.9)=2.7ms   
     http_req_connecting............: avg=6.86µs   min=0s      med=0s      max=10.41ms  p(90)=0s       p(95)=0s       p(99.9)=2.31ms  
     http_req_duration..............: avg=76.57ms  min=2.65ms  med=75.56ms max=519.21ms p(90)=104.06ms p(95)=112.76ms p(99.9)=305.16ms
       { expected_response:true }...: avg=76.57ms  min=2.65ms  med=75.56ms max=519.21ms p(90)=104.06ms p(95)=112.76ms p(99.9)=305.16ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 39025
     http_req_receiving.............: avg=160.04µs min=29.49µs med=77.5µs  max=235.02ms p(90)=155µs    p(95)=407.17µs p(99.9)=12.24ms 
     http_req_sending...............: avg=56.93µs  min=5.75µs  med=11.84µs max=325.57ms p(90)=30.46µs  p(95)=131.6µs  p(99.9)=3.78ms  
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s      max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=76.35ms  min=2.56ms  med=75.38ms max=501.42ms p(90)=103.82ms p(95)=112.54ms p(99.9)=295.41ms
     http_reqs......................: 39025   645.966896/s
     iteration_duration.............: avg=77.07ms  min=2.91ms  med=75.89ms max=574.86ms p(90)=104.38ms p(95)=113.08ms p(99.9)=319.55ms
     iterations.....................: 38925   644.311631/s
     success_rate...................: 100.00% ✓ 38925      ✗ 0    
     vus............................: 50      min=50       max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway-bun`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 113979     ✗ 0    
     data_received..................: 3.3 GB  55 MB/s
     data_sent......................: 44 MB   730 kB/s
     http_req_blocked...............: avg=11.43µs  min=1.22µs  med=2.83µs  max=11.83ms  p(90)=4.47µs   p(95)=5.69µs   p(99.9)=2.52ms  
     http_req_connecting............: avg=7.71µs   min=0s      med=0s      max=11.79ms  p(90)=0s       p(95)=0s       p(99.9)=2.49ms  
     http_req_duration..............: avg=78.5ms   min=5.09ms  med=76.55ms max=558.89ms p(90)=94.49ms  p(95)=102.01ms p(99.9)=348.36ms
       { expected_response:true }...: avg=78.5ms   min=5.09ms  med=76.55ms max=558.89ms p(90)=94.49ms  p(95)=102.01ms p(99.9)=348.36ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 38093
     http_req_receiving.............: avg=102.82µs min=27.73µs med=53.8µs  max=143.86ms p(90)=115.97µs p(95)=261.01µs p(99.9)=4.29ms  
     http_req_sending...............: avg=82.21µs  min=5.53µs  med=11.25µs max=387.58ms p(90)=31.62µs  p(95)=130.87µs p(99.9)=5.54ms  
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s      max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=78.31ms  min=5.04ms  med=76.39ms max=557.53ms p(90)=94.31ms  p(95)=101.8ms  p(99.9)=320.48ms
     http_reqs......................: 38093   627.776717/s
     iteration_duration.............: avg=79ms     min=15.69ms med=76.87ms max=613.65ms p(90)=94.8ms   p(95)=102.29ms p(99.9)=355.52ms
     iterations.....................: 37993   626.128706/s
     success_rate...................: 100.00% ✓ 37993      ✗ 0    
     vus............................: 50      min=50       max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-router`

K6 Output

     ✓ response code was 200
     ✗ no graphql errors
      ↳  99% — ✓ 22347 / ✗ 1
     ✓ valid response structure

     checks.........................: 99.99% ✓ 67043      ✗ 1    
     data_received..................: 2.0 GB 32 MB/s
     data_sent......................: 26 MB  430 kB/s
     http_req_blocked...............: avg=14.73µs  min=1.92µs  med=3.1µs    max=9.59ms   p(90)=4.49µs   p(95)=5.32µs   p(99.9)=5.46ms  
     http_req_connecting............: avg=10.65µs  min=0s      med=0s       max=9.55ms   p(90)=0s       p(95)=0s       p(99.9)=5.24ms  
     http_req_duration..............: avg=133.41ms min=5.96ms  med=132.59ms max=587.82ms p(90)=163.71ms p(95)=173.52ms p(99.9)=425.11ms
       { expected_response:true }...: avg=133.41ms min=5.96ms  med=132.59ms max=587.82ms p(90)=163.71ms p(95)=173.52ms p(99.9)=425.11ms
     http_req_failed................: 0.00%  ✓ 0          ✗ 22448
     http_req_receiving.............: avg=80.87µs  min=36.29µs med=55.44µs  max=6.71ms   p(90)=107.27µs p(95)=136.48µs p(99.9)=1.58ms  
     http_req_sending...............: avg=81.84µs  min=7.5µs   med=12.07µs  max=405.96ms p(90)=20.35µs  p(95)=36.59µs  p(99.9)=2.29ms  
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=133.25ms min=5.92ms  med=132.47ms max=586.89ms p(90)=163.61ms p(95)=173.35ms p(99.9)=411.66ms
     http_reqs......................: 22448  369.431316/s
     iteration_duration.............: avg=134.33ms min=27.75ms med=133.02ms max=631.43ms p(90)=164.06ms p(95)=173.88ms p(99.9)=455.16ms
     iterations.....................: 22348  367.785596/s
     success_rate...................: 99.99% ✓ 22347      ✗ 1    
     vus............................: 50     min=50       max=50 
     vus_max........................: 50     min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 21441      ✗ 0   
     data_received..................: 637 MB  10 MB/s
     data_sent......................: 8.4 MB  138 kB/s
     http_req_blocked...............: avg=44.07µs  min=1.37µs  med=2.9µs    max=11.81ms  p(90)=4.41µs   p(95)=5.08µs   p(99.9)=9.89ms  
     http_req_connecting............: avg=39.28µs  min=0s      med=0s       max=11.38ms  p(90)=0s       p(95)=0s       p(99.9)=9.65ms  
     http_req_duration..............: avg=414.88ms min=7.41ms  med=413.58ms max=898.15ms p(90)=470.94ms p(95)=488.58ms p(99.9)=849.98ms
       { expected_response:true }...: avg=414.88ms min=7.41ms  med=413.58ms max=898.15ms p(90)=470.94ms p(95)=488.58ms p(99.9)=849.98ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 7247
     http_req_receiving.............: avg=74.36µs  min=29.32µs med=50.26µs  max=91.24ms  p(90)=91.42µs  p(95)=106.96µs p(99.9)=568.7µs 
     http_req_sending...............: avg=131.41µs min=6.32µs  med=11.71µs  max=269.62ms p(90)=18.92µs  p(95)=22.21µs  p(99.9)=4.45ms  
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=414.68ms min=7.36ms  med=413.47ms max=898.02ms p(90)=470.78ms p(95)=488.43ms p(99.9)=849.1ms 
     http_reqs......................: 7247    118.469572/s
     iteration_duration.............: avg=421.25ms min=86.41ms med=414.24ms max=943.46ms p(90)=471.51ms p(95)=489.47ms p(99.9)=904.06ms
     iterations.....................: 7147    116.834832/s
     success_rate...................: 100.00% ✓ 7147       ✗ 0   
     vus............................: 42      min=42       max=50
     vus_max........................: 50      min=50       max=50

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview

@github-actions
Copy link

github-actions bot commented Oct 26, 2025

Overview for: ramping-vus

This scenario runs 4 subgraphs and a GraphQL gateway with Federation spec, and runs a heavy query. We are running a heavy load of concurrent VUs to measure response time and other stats, during stress. It measure things like memory usage, CPU usage, response times. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was running 500 VUs over 60s

Comparison

Comparison

Gateway RPS ⬇️ Requests Duration Notes
hive-router 1769 110149 total, 0 failed avg: 126ms, p95: 308ms
grafbase 1457 92132 total, 0 failed avg: 151ms, p95: 356ms
cosmo 673 43029 total, 0 failed avg: 327ms, p95: 727ms
hive-gateway-bun 603 39757 total, 0 failed avg: 353ms, p95: 727ms ❌ non-compatible response structure (1)
hive-gateway 586 38778 total, 0 failed avg: 362ms, p95: 737ms ❌ non-compatible response structure (1)
apollo-router 392 26266 total, 0 failed avg: 537ms, p95: 1232ms ❌ 26 unexpected GraphQL errors
apollo-gateway 153 10553 total, 0 failed avg: 1232ms, p95: 2561ms
Summary for: `hive-router`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 327447      ✗ 0     
     data_received..................: 9.7 GB  155 MB/s
     data_sent......................: 128 MB  2.1 MB/s
     http_req_blocked...............: avg=476.29µs min=1.11µs  med=2.58µs   max=334.26ms p(90)=4.03µs   p(95)=4.79µs   p(99.9)=171.62ms
     http_req_connecting............: avg=472.4µs  min=0s      med=0s       max=329.88ms p(90)=0s       p(95)=0s       p(99.9)=171.57ms
     http_req_duration..............: avg=126.43ms min=1.54ms  med=113.84ms max=459.81ms p(90)=260.48ms p(95)=307.85ms p(99.9)=396.97ms
       { expected_response:true }...: avg=126.43ms min=1.54ms  med=113.84ms max=459.81ms p(90)=260.48ms p(95)=307.85ms p(99.9)=396.97ms
     http_req_failed................: 0.00%   ✓ 0           ✗ 110149
     http_req_receiving.............: avg=517.29µs min=22.61µs med=41.94µs  max=149.52ms p(90)=98.62µs  p(95)=394.74µs p(99.9)=76.19ms 
     http_req_sending...............: avg=440.33µs min=5.1µs   med=10.17µs  max=131.97ms p(90)=22.25µs  p(95)=128.58µs p(99.9)=71.55ms 
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=125.48ms min=1.48ms  med=112.87ms max=420.52ms p(90)=258.37ms p(95)=305.71ms p(99.9)=393.02ms
     http_reqs......................: 110149  1769.459393/s
     iteration_duration.............: avg=128.38ms min=1.8ms   med=115.54ms max=825.38ms p(90)=263.15ms p(95)=309.86ms p(99.9)=427.42ms
     iterations.....................: 109149  1753.395158/s
     success_rate...................: 100.00% ✓ 109149      ✗ 0     
     vus............................: 72      min=0         max=495 
     vus_max........................: 500     min=500       max=500 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `grafbase`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 273396      ✗ 0    
     data_received..................: 8.1 GB  128 MB/s
     data_sent......................: 107 MB  1.7 MB/s
     http_req_blocked...............: avg=781.55µs min=1.25µs med=3.15µs   max=586.86ms p(90)=4.77µs   p(95)=5.94µs   p(99.9)=239.9ms 
     http_req_connecting............: avg=775.48µs min=0s     med=0s       max=586.81ms p(90)=0s       p(95)=0s       p(99.9)=239.38ms
     http_req_duration..............: avg=150.95ms min=2.57ms med=138.16ms max=526.72ms p(90)=299.09ms p(95)=355.89ms p(99.9)=466.72ms
       { expected_response:true }...: avg=150.95ms min=2.57ms med=138.16ms max=526.72ms p(90)=299.09ms p(95)=355.89ms p(99.9)=466.72ms
     http_req_failed................: 0.00%   ✓ 0           ✗ 92132
     http_req_receiving.............: avg=681.61µs min=25µs   med=50.53µs  max=151.23ms p(90)=126.76µs p(95)=470.74µs p(99.9)=90.94ms 
     http_req_sending...............: avg=659.28µs min=5.49µs med=12.1µs   max=152.97ms p(90)=32.83µs  p(95)=150.7µs  p(99.9)=92.51ms 
     http_req_tls_handshaking.......: avg=0s       min=0s     med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=149.61ms min=2.49ms med=136.71ms max=491.6ms  p(90)=296.47ms p(95)=352.94ms p(99.9)=462.88ms
     http_reqs......................: 92132   1457.559682/s
     iteration_duration.............: avg=153.82ms min=2.74ms med=140.68ms max=949.37ms p(90)=302.93ms p(95)=360.07ms p(99.9)=554.96ms
     iterations.....................: 91132   1441.739341/s
     success_rate...................: 100.00% ✓ 91132       ✗ 0    
     vus............................: 70      min=0         max=495
     vus_max........................: 500     min=500       max=500

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `cosmo`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 126087     ✗ 0    
     data_received..................: 3.8 GB  59 MB/s
     data_sent......................: 50 MB   784 kB/s
     http_req_blocked...............: avg=100.73µs min=1.21µs  med=3.16µs   max=222.54ms p(90)=5.04µs   p(95)=6.91µs   p(99.9)=33.48ms 
     http_req_connecting............: avg=95.65µs  min=0s      med=0s       max=222.36ms p(90)=0s       p(95)=0s       p(99.9)=33.34ms 
     http_req_duration..............: avg=326.59ms min=2.78ms  med=310.69ms max=1.35s    p(90)=640.96ms p(95)=726.66ms p(99.9)=1.12s   
       { expected_response:true }...: avg=326.59ms min=2.78ms  med=310.69ms max=1.35s    p(90)=640.96ms p(95)=726.66ms p(99.9)=1.12s   
     http_req_failed................: 0.00%   ✓ 0          ✗ 43029
     http_req_receiving.............: avg=701.72µs min=28.21µs med=63.81µs  max=155.31ms p(90)=185.07µs p(95)=499.63µs p(99.9)=107.58ms
     http_req_sending...............: avg=158.25µs min=5.79µs  med=11.89µs  max=98.59ms  p(90)=31.86µs  p(95)=137.82µs p(99.9)=26ms    
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=325.73ms min=2.72ms  med=309.19ms max=1.35s    p(90)=640.31ms p(95)=726.2ms  p(99.9)=1.12s   
     http_reqs......................: 43029   673.022014/s
     iteration_duration.............: avg=334.88ms min=3.09ms  med=319.97ms max=1.35s    p(90)=644.38ms p(95)=730.19ms p(99.9)=1.12s   
     iterations.....................: 42029   657.380888/s
     success_rate...................: 100.00% ✓ 42029      ✗ 0    
     vus............................: 62      min=0        max=498
     vus_max........................: 500     min=500      max=500

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway-bun`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✗ valid response structure
      ↳  99% — ✓ 38756 / ✗ 1

     checks.........................: 99.99% ✓ 116270     ✗ 1    
     data_received..................: 3.5 GB 53 MB/s
     data_sent......................: 46 MB  703 kB/s
     http_req_blocked...............: avg=163.38µs min=1.2µs   med=2.81µs   max=210.25ms p(90)=4.24µs   p(95)=5.79µs   p(99.9)=52.94ms 
     http_req_connecting............: avg=157.38µs min=0s      med=0s       max=210.07ms p(90)=0s       p(95)=0s       p(99.9)=52.81ms 
     http_req_duration..............: avg=353.34ms min=4.68ms  med=345.15ms max=1.3s     p(90)=659.17ms p(95)=726.76ms p(99.9)=993.27ms
       { expected_response:true }...: avg=353.34ms min=4.68ms  med=345.15ms max=1.3s     p(90)=659.17ms p(95)=726.76ms p(99.9)=993.27ms
     http_req_failed................: 0.00%  ✓ 0          ✗ 39757
     http_req_receiving.............: avg=206.66µs min=26.75µs med=44.37µs  max=143.72ms p(90)=88.64µs  p(95)=374.45µs p(99.9)=29.14ms 
     http_req_sending...............: avg=270.65µs min=5.46µs  med=10.87µs  max=157.14ms p(90)=28.69µs  p(95)=131.65µs p(99.9)=49.26ms 
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=352.86ms min=4.63ms  med=344.74ms max=1.3s     p(90)=658.44ms p(95)=725.94ms p(99.9)=993.21ms
     http_reqs......................: 39757  603.943696/s
     iteration_duration.............: avg=363.04ms min=5.2ms   med=354.64ms max=1.38s    p(90)=664.05ms p(95)=729.61ms p(99.9)=1s      
     iterations.....................: 38757  588.752819/s
     success_rate...................: 99.99% ✓ 38756      ✗ 1    
     vus............................: 56     min=0        max=499
     vus_max........................: 500    min=500      max=500

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✗ valid response structure
      ↳  99% — ✓ 37777 / ✗ 1

     checks.........................: 99.99% ✓ 113333     ✗ 1    
     data_received..................: 3.4 GB 52 MB/s
     data_sent......................: 45 MB  684 kB/s
     http_req_blocked...............: avg=138.99µs min=1.29µs  med=2.99µs   max=143.74ms p(90)=4.64µs   p(95)=6.35µs   p(99.9)=42.47ms
     http_req_connecting............: avg=134.32µs min=0s      med=0s       max=143.68ms p(90)=0s       p(95)=0s       p(99.9)=42.42ms
     http_req_duration..............: avg=362.21ms min=4.81ms  med=357.97ms max=1.29s    p(90)=681.8ms  p(95)=737.24ms p(99.9)=1.08s  
       { expected_response:true }...: avg=362.21ms min=4.81ms  med=357.97ms max=1.29s    p(90)=681.8ms  p(95)=737.24ms p(99.9)=1.08s  
     http_req_failed................: 0.00%  ✓ 0          ✗ 38778
     http_req_receiving.............: avg=208.98µs min=26.67µs med=46.03µs  max=130.16ms p(90)=91.37µs  p(95)=363.39µs p(99.9)=37.79ms
     http_req_sending...............: avg=264.9µs  min=5.92µs  med=11.23µs  max=137.77ms p(90)=29.78µs  p(95)=134.33µs p(99.9)=46.62ms
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s     
     http_req_waiting...............: avg=361.74ms min=4.76ms  med=357.39ms max=1.29s    p(90)=680.91ms p(95)=736.38ms p(99.9)=1.08s  
     http_reqs......................: 38778  586.887262/s
     iteration_duration.............: avg=372.36ms min=5.13ms  med=367.59ms max=1.31s    p(90)=684.53ms p(95)=739.33ms p(99.9)=1.08s  
     iterations.....................: 37778  571.75272/s
     success_rate...................: 99.99% ✓ 37777      ✗ 1    
     vus............................: 66     min=0        max=496
     vus_max........................: 500    min=500      max=500

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-router`

K6 Output

     ✓ response code was 200
     ✗ no graphql errors
      ↳  99% — ✓ 25240 / ✗ 26
     ✓ valid response structure

     checks.........................: 99.96% ✓ 75772      ✗ 26   
     data_received..................: 2.3 GB 34 MB/s
     data_sent......................: 31 MB  457 kB/s
     http_req_blocked...............: avg=105.84µs min=1.41µs  med=3.04µs   max=164.57ms p(90)=4.8µs    p(95)=6.91µs   p(99.9)=28.73ms
     http_req_connecting............: avg=100.38µs min=0s      med=0s       max=164.41ms p(90)=0s       p(95)=0s       p(99.9)=28.6ms 
     http_req_duration..............: avg=536.52ms min=6.07ms  med=493.67ms max=1.92s    p(90)=1.08s    p(95)=1.23s    p(99.9)=1.7s   
       { expected_response:true }...: avg=536.52ms min=6.07ms  med=493.67ms max=1.92s    p(90)=1.08s    p(95)=1.23s    p(99.9)=1.7s   
     http_req_failed................: 0.00%  ✓ 0          ✗ 26266
     http_req_receiving.............: avg=124.33µs min=32.55µs med=52.69µs  max=118.48ms p(90)=103.21µs p(95)=183.48µs p(99.9)=8.42ms 
     http_req_sending...............: avg=156.06µs min=6.79µs  med=11.7µs   max=153.25ms p(90)=31.21µs  p(95)=122.98µs p(99.9)=31.26ms
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s     
     http_req_waiting...............: avg=536.24ms min=6.02ms  med=493.29ms max=1.91s    p(90)=1.08s    p(95)=1.23s    p(99.9)=1.7s   
     http_reqs......................: 26266  392.011072/s
     iteration_duration.............: avg=558.26ms min=6.29ms  med=518.11ms max=1.92s    p(90)=1.09s    p(95)=1.23s    p(99.9)=1.7s   
     iterations.....................: 25266  377.086414/s
     success_rate...................: 99.89% ✓ 25240      ✗ 26   
     vus............................: 61     min=0        max=497
     vus_max........................: 500    min=500      max=500

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 28659      ✗ 0    
     data_received..................: 927 MB  13 MB/s
     data_sent......................: 13 MB   183 kB/s
     http_req_blocked...............: avg=35.32µs min=1.13µs  med=3.92µs  max=32.32ms p(90)=6.85µs   p(95)=14.44µs p(99.9)=6.05ms
     http_req_connecting............: avg=28.71µs min=0s      med=0s      max=32.2ms  p(90)=0s       p(95)=0s      p(99.9)=5.99ms
     http_req_duration..............: avg=1.23s   min=7.38ms  med=1.15s   max=15.74s  p(90)=2.25s    p(95)=2.56s   p(99.9)=14.43s
       { expected_response:true }...: avg=1.23s   min=7.38ms  med=1.15s   max=15.74s  p(90)=2.25s    p(95)=2.56s   p(99.9)=14.43s
     http_req_failed................: 0.00%   ✓ 0          ✗ 10553
     http_req_receiving.............: avg=93.62µs min=26.98µs med=74.02µs max=11.75ms p(90)=123.41µs p(95)=154.1µs p(99.9)=1.65ms
     http_req_sending...............: avg=54.6µs  min=5.04µs  med=14.42µs max=34.73ms p(90)=29.15µs  p(95)=49.01µs p(99.9)=9.45ms
     http_req_tls_handshaking.......: avg=0s      min=0s      med=0s      max=0s      p(90)=0s       p(95)=0s      p(99.9)=0s    
     http_req_waiting...............: avg=1.23s   min=7.29ms  med=1.15s   max=15.74s  p(90)=2.25s    p(95)=2.56s   p(99.9)=14.43s
     http_reqs......................: 10553   153.013493/s
     iteration_duration.............: avg=1.36s   min=8.09ms  med=1.3s    max=15.75s  p(90)=2.3s     p(95)=2.61s   p(99.9)=14.56s
     iterations.....................: 9553    138.513968/s
     success_rate...................: 100.00% ✓ 9553       ✗ 0    
     vus............................: 66      min=0        max=500
     vus_max........................: 500     min=500      max=500

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview

@github-actions
Copy link

github-actions bot commented Oct 26, 2025

Overview for: constant-vus-subgraphs-delay

This scenario runs 4 subgraphs and a GraphQL gateway with Federation spec, and runs a heavy query. It's being executed with a constant amount of VUs over a fixed amount of time. It measure things like memory usage, CPU usage, average RPS. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was running 50 VUs over 60s

Comparison

Comparison

Gateway RPS ⬇️ Requests Duration Notes
grafbase 172 13990 total, 0 failed avg: 216ms, p95: 228ms
hive-router 165 13393 total, 0 failed avg: 225ms, p95: 238ms
cosmo 160 12997 total, 0 failed avg: 232ms, p95: 254ms
hive-gateway 141 11543 total, 0 failed avg: 262ms, p95: 272ms
hive-gateway-bun 137 11190 total, 0 failed avg: 270ms, p95: 294ms
apollo-router 123 10097 total, 0 failed avg: 299ms, p95: 345ms
apollo-gateway 85 6975 total, 0 failed avg: 434ms, p95: 478ms
Summary for: `grafbase`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 41670      ✗ 0    
     data_received..................: 1.2 GB  15 MB/s
     data_sent......................: 16 MB   201 kB/s
     http_req_blocked...............: avg=19.18µs  min=1.22µs   med=1.9µs    max=9.28ms   p(90)=3.34µs   p(95)=4.74µs   p(99.9)=6.78ms  
     http_req_connecting............: avg=16.64µs  min=0s       med=0s       max=9.24ms   p(90)=0s       p(95)=0s       p(99.9)=6.76ms  
     http_req_duration..............: avg=216.37ms min=110.77ms med=215.49ms max=621.4ms  p(90)=225.39ms p(95)=228.14ms p(99.9)=525.8ms 
       { expected_response:true }...: avg=216.37ms min=110.77ms med=215.49ms max=621.4ms  p(90)=225.39ms p(95)=228.14ms p(99.9)=525.8ms 
     http_req_failed................: 0.00%   ✓ 0          ✗ 13990
     http_req_receiving.............: avg=83.14µs  min=25.54µs  med=45.49µs  max=13.88ms  p(90)=94.4µs   p(95)=253.02µs p(99.9)=4.39ms  
     http_req_sending...............: avg=136.73µs min=6.16µs   med=8.56µs   max=374.38ms p(90)=32.5µs   p(95)=98.09µs  p(99.9)=8.74ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=216.15ms min=110.72ms med=215.38ms max=620.64ms p(90)=225.24ms p(95)=227.96ms p(99.9)=507.25ms
     http_reqs......................: 13990   172.589458/s
     iteration_duration.............: avg=216.69ms min=111.01ms med=215.75ms max=634.14ms p(90)=225.61ms p(95)=228.35ms p(99.9)=541.76ms
     iterations.....................: 13890   171.355795/s
     success_rate...................: 100.00% ✓ 13890      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-router`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 39879      ✗ 0    
     data_received..................: 1.2 GB  15 MB/s
     data_sent......................: 16 MB   193 kB/s
     http_req_blocked...............: avg=40.45µs  min=1.18µs   med=2.28µs   max=277.93ms p(90)=3.49µs   p(95)=4.88µs   p(99.9)=6.93ms  
     http_req_connecting............: avg=16.76µs  min=0s       med=0s       max=8.85ms   p(90)=0s       p(95)=0s       p(99.9)=6.6ms   
     http_req_duration..............: avg=225.36ms min=161.68ms med=224.42ms max=601.33ms p(90)=234.6ms  p(95)=237.52ms p(99.9)=503.13ms
       { expected_response:true }...: avg=225.36ms min=161.68ms med=224.42ms max=601.33ms p(90)=234.6ms  p(95)=237.52ms p(99.9)=503.13ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 13393
     http_req_receiving.............: avg=81.43µs  min=26.58µs  med=43.79µs  max=17.1ms   p(90)=93.35µs  p(95)=251.87µs p(99.9)=4.8ms   
     http_req_sending...............: avg=72.78µs  min=5.65µs   med=9.63µs   max=261.98ms p(90)=30.05µs  p(95)=99.22µs  p(99.9)=5.62ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=225.21ms min=158.61ms med=224.32ms max=600.64ms p(90)=234.46ms p(95)=237.35ms p(99.9)=502.31ms
     http_reqs......................: 13393   165.619805/s
     iteration_duration.............: avg=225.8ms  min=161.85ms med=224.69ms max=628.14ms p(90)=234.81ms p(95)=237.78ms p(99.9)=539.63ms
     iterations.....................: 13293   164.38319/s
     success_rate...................: 100.00% ✓ 13293      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `cosmo`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 38691      ✗ 0    
     data_received..................: 1.1 GB  14 MB/s
     data_sent......................: 15 MB   187 kB/s
     http_req_blocked...............: avg=21.07µs  min=1.36µs   med=2.7µs    max=9.31ms   p(90)=3.91µs   p(95)=4.97µs   p(99.9)=6.89ms  
     http_req_connecting............: avg=17.68µs  min=0s       med=0s       max=9.27ms   p(90)=0s       p(95)=0s       p(99.9)=6.87ms  
     http_req_duration..............: avg=232.16ms min=135.08ms med=231.74ms max=630.32ms p(90)=248.58ms p(95)=253.91ms p(99.9)=526.44ms
       { expected_response:true }...: avg=232.16ms min=135.08ms med=231.74ms max=630.32ms p(90)=248.58ms p(95)=253.91ms p(99.9)=526.44ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 12997
     http_req_receiving.............: avg=301.82µs min=29.98µs  med=62.4µs   max=270.17ms p(90)=170.79µs p(95)=480.25µs p(99.9)=25.31ms 
     http_req_sending...............: avg=66.86µs  min=6.18µs   med=10.88µs  max=255.4ms  p(90)=28.9µs   p(95)=121.52µs p(99.9)=3.5ms   
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=231.79ms min=135.02ms med=231.44ms max=629.39ms p(90)=248.06ms p(95)=253.55ms p(99.9)=525.7ms 
     http_reqs......................: 12997   160.545401/s
     iteration_duration.............: avg=232.72ms min=135.31ms med=232.12ms max=654.52ms p(90)=248.93ms p(95)=254.22ms p(99.9)=569.11ms
     iterations.....................: 12897   159.310152/s
     success_rate...................: 100.00% ✓ 12897      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 34329      ✗ 0    
     data_received..................: 1.0 GB  13 MB/s
     data_sent......................: 13 MB   165 kB/s
     http_req_blocked...............: avg=30.71µs  min=1.15µs   med=2.47µs   max=12.03ms  p(90)=3.94µs   p(95)=5.23µs   p(99.9)=9.74ms  
     http_req_connecting............: avg=27.49µs  min=0s       med=0s       max=11.98ms  p(90)=0s       p(95)=0s       p(99.9)=9.72ms  
     http_req_duration..............: avg=261.61ms min=174.45ms med=260.29ms max=693.26ms p(90)=269.18ms p(95)=272.11ms p(99.9)=598.68ms
       { expected_response:true }...: avg=261.61ms min=174.45ms med=260.29ms max=693.26ms p(90)=269.18ms p(95)=272.11ms p(99.9)=598.68ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 11543
     http_req_receiving.............: avg=89.63µs  min=25.87µs  med=45.48µs  max=111.69ms p(90)=94.53µs  p(95)=289.76µs p(99.9)=1.95ms  
     http_req_sending...............: avg=148.76µs min=5.53µs   med=10.28µs  max=333.27ms p(90)=32.91µs  p(95)=123.19µs p(99.9)=6.13ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=261.38ms min=170.48ms med=260.17ms max=691.74ms p(90)=269.01ms p(95)=271.94ms p(99.9)=589.03ms
     http_reqs......................: 11543   141.89038/s
     iteration_duration.............: avg=262.38ms min=174.74ms med=260.59ms max=737.25ms p(90)=269.47ms p(95)=272.39ms p(99.9)=623.08ms
     iterations.....................: 11443   140.661147/s
     success_rate...................: 100.00% ✓ 11443      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway-bun`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 33270      ✗ 0    
     data_received..................: 983 MB  12 MB/s
     data_sent......................: 13 MB   160 kB/s
     http_req_blocked...............: avg=30.91µs  min=1.15µs   med=2.55µs   max=11.69ms  p(90)=4.13µs   p(95)=5.42µs   p(99.9)=9.34ms  
     http_req_connecting............: avg=27.6µs   min=0s       med=0s       max=11.64ms  p(90)=0s       p(95)=0s       p(99.9)=9.32ms  
     http_req_duration..............: avg=270.2ms  min=176.21ms med=277.48ms max=943.65ms p(90)=289.05ms p(95)=293.64ms p(99.9)=847.74ms
       { expected_response:true }...: avg=270.2ms  min=176.21ms med=277.48ms max=943.65ms p(90)=289.05ms p(95)=293.64ms p(99.9)=847.74ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 11190
     http_req_receiving.............: avg=96.19µs  min=26µs     med=47.41µs  max=17.83ms  p(90)=119.65µs p(95)=351.62µs p(99.9)=3.58ms  
     http_req_sending...............: avg=139.25µs min=5.4µs    med=10.63µs  max=393.72ms p(90)=36.49µs  p(95)=127.19µs p(99.9)=10.92ms 
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=269.97ms min=176.07ms med=277.35ms max=942.42ms p(90)=288.86ms p(95)=293.37ms p(99.9)=846.22ms
     http_reqs......................: 11190   137.270992/s
     iteration_duration.............: avg=271.2ms  min=176.53ms med=277.82ms max=959.12ms p(90)=289.35ms p(95)=293.98ms p(99.9)=867.78ms
     iterations.....................: 11090   136.044263/s
     success_rate...................: 100.00% ✓ 11090      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-router`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 29991      ✗ 0    
     data_received..................: 886 MB  11 MB/s
     data_sent......................: 12 MB   144 kB/s
     http_req_blocked...............: avg=27.07µs  min=1.59µs   med=2.77µs   max=9.52ms   p(90)=4.11µs   p(95)=5.05µs   p(99.9)=7.57ms  
     http_req_connecting............: avg=23.58µs  min=0s       med=0s       max=9.48ms   p(90)=0s       p(95)=0s       p(99.9)=7.55ms  
     http_req_duration..............: avg=299.33ms min=149.08ms med=300.66ms max=710.63ms p(90)=336.14ms p(95)=344.88ms p(99.9)=629.96ms
       { expected_response:true }...: avg=299.33ms min=149.08ms med=300.66ms max=710.63ms p(90)=336.14ms p(95)=344.88ms p(99.9)=629.96ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 10097
     http_req_receiving.............: avg=93.47µs  min=37.1µs   med=52.7µs   max=36.33ms  p(90)=102.59µs p(95)=276.69µs p(99.9)=1.82ms  
     http_req_sending...............: avg=112.34µs min=6.76µs   med=11.18µs  max=289.7ms  p(90)=31.27µs  p(95)=138.79µs p(99.9)=3.76ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=299.12ms min=148.95ms med=300.49ms max=709.2ms  p(90)=335.97ms p(95)=344.53ms p(99.9)=628.9ms 
     http_reqs......................: 10097   123.775948/s
     iteration_duration.............: avg=300.61ms min=149.32ms med=301.21ms max=728.07ms p(90)=336.64ms p(95)=345.35ms p(99.9)=674.08ms
     iterations.....................: 9997    122.550079/s
     success_rate...................: 100.00% ✓ 9997       ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 20625     ✗ 0   
     data_received..................: 613 MB  7.5 MB/s
     data_sent......................: 8.1 MB  99 kB/s
     http_req_blocked...............: avg=45.83µs  min=1.43µs   med=3.13µs   max=11.48ms  p(90)=4.86µs   p(95)=5.6µs    p(99.9)=9.93ms  
     http_req_connecting............: avg=41.97µs  min=0s       med=0s       max=11.44ms  p(90)=0s       p(95)=0s       p(99.9)=9.9ms   
     http_req_duration..............: avg=433.58ms min=210.69ms med=437.02ms max=938.41ms p(90)=463.44ms p(95)=478.13ms p(99.9)=851.51ms
       { expected_response:true }...: avg=433.58ms min=210.69ms med=437.02ms max=938.41ms p(90)=463.44ms p(95)=478.13ms p(99.9)=851.51ms
     http_req_failed................: 0.00%   ✓ 0         ✗ 6975
     http_req_receiving.............: avg=90.16µs  min=32.94µs  med=54.98µs  max=102.22ms p(90)=101.42µs p(95)=119.64µs p(99.9)=693.17µs
     http_req_sending...............: avg=114.46µs min=6.56µs   med=12.59µs  max=211.24ms p(90)=20.82µs  p(95)=28.91µs  p(99.9)=5.89ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=433.38ms min=210.59ms med=436.94ms max=910.39ms p(90)=463.29ms p(95)=477.99ms p(99.9)=850.36ms
     http_reqs......................: 6975    85.382747/s
     iteration_duration.............: avg=437.34ms min=265.63ms med=437.57ms max=998.87ms p(90)=464.07ms p(95)=478.87ms p(99.9)=885.26ms
     iterations.....................: 6875    84.158622/s
     success_rate...................: 100.00% ✓ 6875      ✗ 0   
     vus............................: 50      min=0       max=50
     vus_max........................: 50      min=50      max=50

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview

@github-actions
Copy link

github-actions bot commented Oct 26, 2025

Overview for: constant-vus-subgraphs-delay-resources

This scenario runs 4 subgraphs and a GraphQL gateway with Federation spec, and runs a heavy query. It's being executed with a constant amount of VUs over a fixed amount of time. It measure things like memory usage, CPU usage, average RPS. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was running 50 VUs over 60s

Comparison

Comparison

Gateway RPS ⬇️ Requests Duration Notes
grafbase 174 14089 total, 0 failed avg: 214ms, p95: 226ms
hive-router 168 13644 total, 0 failed avg: 221ms, p95: 232ms
cosmo 165 13444 total, 0 failed avg: 225ms, p95: 248ms
hive-gateway 147 12054 total, 0 failed avg: 251ms, p95: 265ms
hive-gateway-bun 147 12051 total, 0 failed avg: 251ms, p95: 271ms
apollo-router 130 10681 total, 0 failed avg: 283ms, p95: 327ms ❌ 2 unexpected GraphQL errors
apollo-gateway 120 9823 total, 0 failed avg: 308ms, p95: 343ms
Summary for: `grafbase`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 41967      ✗ 0    
     data_received..................: 1.2 GB  15 MB/s
     data_sent......................: 16 MB   203 kB/s
     http_req_blocked...............: avg=18µs     min=1.24µs   med=1.91µs   max=8.57ms   p(90)=3.32µs   p(95)=4.21µs   p(99.9)=6.27ms  
     http_req_connecting............: avg=15.5µs   min=0s       med=0s       max=8.54ms   p(90)=0s       p(95)=0s       p(99.9)=6.26ms  
     http_req_duration..............: avg=214.47ms min=158ms    med=213.71ms max=607.51ms p(90)=222.9ms  p(95)=225.5ms  p(99.9)=510.31ms
       { expected_response:true }...: avg=214.47ms min=158ms    med=213.71ms max=607.51ms p(90)=222.9ms  p(95)=225.5ms  p(99.9)=510.31ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 14089
     http_req_receiving.............: avg=71.15µs  min=24.95µs  med=43.23µs  max=9.62ms   p(90)=84.28µs  p(95)=251.43µs p(99.9)=1.77ms  
     http_req_sending...............: avg=146.27µs min=5.34µs   med=8.08µs   max=364.04ms p(90)=24.34µs  p(95)=98.01µs  p(99.9)=8.68ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=214.25ms min=157.96ms med=213.62ms max=606.68ms p(90)=222.72ms p(95)=225.31ms p(99.9)=491.69ms
     http_reqs......................: 14089   174.102114/s
     iteration_duration.............: avg=214.77ms min=158.16ms med=213.97ms max=622.18ms p(90)=223.1ms  p(95)=225.69ms p(99.9)=528.82ms
     iterations.....................: 13989   172.866383/s
     success_rate...................: 100.00% ✓ 13989      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-router`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 40632      ✗ 0    
     data_received..................: 1.2 GB  15 MB/s
     data_sent......................: 16 MB   196 kB/s
     http_req_blocked...............: avg=19.33µs  min=1.21µs   med=2.01µs   max=8.8ms    p(90)=3.25µs   p(95)=4.52µs   p(99.9)=6.56ms  
     http_req_connecting............: avg=16.43µs  min=0s       med=0s       max=8.76ms   p(90)=0s       p(95)=0s       p(99.9)=6.54ms  
     http_req_duration..............: avg=221.21ms min=157.43ms med=220.32ms max=606.34ms p(90)=229.04ms p(95)=232.17ms p(99.9)=509.91ms
       { expected_response:true }...: avg=221.21ms min=157.43ms med=220.32ms max=606.34ms p(90)=229.04ms p(95)=232.17ms p(99.9)=509.91ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 13644
     http_req_receiving.............: avg=69.14µs  min=24.25µs  med=41.62µs  max=11.31ms  p(90)=85.31µs  p(95)=233.63µs p(99.9)=1.08ms  
     http_req_sending...............: avg=90.14µs  min=5.38µs   med=8.27µs   max=364.88ms p(90)=27.93µs  p(95)=93.77µs  p(99.9)=6.01ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=221.05ms min=157.38ms med=220.23ms max=605.61ms p(90)=228.93ms p(95)=232.01ms p(99.9)=500.2ms 
     http_reqs......................: 13644   168.857213/s
     iteration_duration.............: avg=221.54ms min=157.58ms med=220.57ms max=616.49ms p(90)=229.23ms p(95)=232.35ms p(99.9)=520.2ms 
     iterations.....................: 13544   167.61962/s
     success_rate...................: 100.00% ✓ 13544      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `cosmo`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 40032      ✗ 0    
     data_received..................: 1.2 GB  15 MB/s
     data_sent......................: 16 MB   193 kB/s
     http_req_blocked...............: avg=21.52µs  min=1.47µs   med=2.64µs   max=9.36ms   p(90)=3.93µs   p(95)=5.2µs    p(99.9)=7.01ms  
     http_req_connecting............: avg=17.81µs  min=0s       med=0s       max=9.32ms   p(90)=0s       p(95)=0s       p(99.9)=6.99ms  
     http_req_duration..............: avg=224.82ms min=130.92ms med=223.43ms max=638.6ms  p(90)=242.43ms p(95)=248.31ms p(99.9)=518.66ms
       { expected_response:true }...: avg=224.82ms min=130.92ms med=223.43ms max=638.6ms  p(90)=242.43ms p(95)=248.31ms p(99.9)=518.66ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 13444
     http_req_receiving.............: avg=440.1µs  min=32.5µs   med=60.48µs  max=153.33ms p(90)=309.07µs p(95)=734.83µs p(99.9)=26.76ms 
     http_req_sending...............: avg=95.31µs  min=6.58µs   med=10.92µs  max=284.67ms p(90)=31.59µs  p(95)=123.39µs p(99.9)=4.15ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=224.28ms min=126.67ms med=223.01ms max=623.52ms p(90)=241.82ms p(95)=247.82ms p(99.9)=504.64ms
     http_reqs......................: 13444   165.841584/s
     iteration_duration.............: avg=225.29ms min=131.16ms med=223.84ms max=647.84ms p(90)=242.78ms p(95)=248.59ms p(99.9)=553.37ms
     iterations.....................: 13344   164.608011/s
     success_rate...................: 100.00% ✓ 13344      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 35862      ✗ 0    
     data_received..................: 1.1 GB  13 MB/s
     data_sent......................: 14 MB   172 kB/s
     http_req_blocked...............: avg=26.72µs  min=1.21µs   med=2.57µs   max=11.35ms  p(90)=4.36µs   p(95)=5.59µs   p(99.9)=8.44ms  
     http_req_connecting............: avg=23.35µs  min=0s       med=0s       max=11.32ms  p(90)=0s       p(95)=0s       p(99.9)=8.38ms  
     http_req_duration..............: avg=250.78ms min=165.79ms med=248.95ms max=981.31ms p(90)=259.57ms p(95)=264.55ms p(99.9)=871.84ms
       { expected_response:true }...: avg=250.78ms min=165.79ms med=248.95ms max=981.31ms p(90)=259.57ms p(95)=264.55ms p(99.9)=871.84ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 12054
     http_req_receiving.............: avg=90.99µs  min=26.52µs  med=47.05µs  max=144.1ms  p(90)=98µs     p(95)=174.21µs p(99.9)=2.64ms  
     http_req_sending...............: avg=88.24µs  min=5.44µs   med=10.47µs  max=111.57ms p(90)=32.57µs  p(95)=106.35µs p(99.9)=6.56ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=250.61ms min=165.72ms med=248.84ms max=980.49ms p(90)=259.44ms p(95)=264.36ms p(99.9)=871.11ms
     http_reqs......................: 12054   147.823048/s
     iteration_duration.............: avg=251.51ms min=165.96ms med=249.26ms max=1s       p(90)=259.88ms p(95)=264.82ms p(99.9)=884.55ms
     iterations.....................: 11954   146.596708/s
     success_rate...................: 100.00% ✓ 11954      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `hive-gateway-bun`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 35853      ✗ 0    
     data_received..................: 1.1 GB  13 MB/s
     data_sent......................: 14 MB   172 kB/s
     http_req_blocked...............: avg=26.56µs  min=1.15µs   med=2.37µs   max=11.31ms  p(90)=3.9µs    p(95)=5.03µs   p(99.9)=8.64ms  
     http_req_connecting............: avg=23.41µs  min=0s       med=0s       max=11.27ms  p(90)=0s       p(95)=0s       p(99.9)=8.61ms  
     http_req_duration..............: avg=251.1ms  min=168.58ms med=249.17ms max=693.85ms p(90)=265.97ms p(95)=270.85ms p(99.9)=581.6ms 
       { expected_response:true }...: avg=251.1ms  min=168.58ms med=249.17ms max=693.85ms p(90)=265.97ms p(95)=270.85ms p(99.9)=581.6ms 
     http_req_failed................: 0.00%   ✓ 0          ✗ 12051
     http_req_receiving.............: avg=80.65µs  min=24.84µs  med=44.12µs  max=15.94ms  p(90)=92.72µs  p(95)=219.71µs p(99.9)=3.44ms  
     http_req_sending...............: avg=71.7µs   min=5.29µs   med=9.96µs   max=187.4ms  p(90)=31.08µs  p(95)=103.73µs p(99.9)=4.2ms   
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=250.94ms min=168.18ms med=249.02ms max=692.77ms p(90)=265.83ms p(95)=270.7ms  p(99.9)=580.69ms
     http_reqs......................: 12051   147.853553/s
     iteration_duration.............: avg=251.74ms min=168.75ms med=249.54ms max=704.46ms p(90)=266.25ms p(95)=271.21ms p(99.9)=598.31ms
     iterations.....................: 11951   146.626655/s
     success_rate...................: 100.00% ✓ 11951      ✗ 0    
     vus............................: 50      min=0        max=50 
     vus_max........................: 50      min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-router`

K6 Output

     ✓ response code was 200
     ✗ no graphql errors
      ↳  99% — ✓ 10579 / ✗ 2
     ✓ valid response structure

     checks.........................: 99.99% ✓ 31741      ✗ 2    
     data_received..................: 937 MB 12 MB/s
     data_sent......................: 12 MB  152 kB/s
     http_req_blocked...............: avg=18.92µs  min=1.69µs   med=3.18µs   max=6.11ms   p(90)=4.95µs   p(95)=6.04µs   p(99.9)=4.75ms  
     http_req_connecting............: avg=14.52µs  min=0s       med=0s       max=6.08ms   p(90)=0s       p(95)=0s       p(99.9)=4.73ms  
     http_req_duration..............: avg=283.12ms min=169.77ms med=284.19ms max=667.09ms p(90)=317.66ms p(95)=327.44ms p(99.9)=574.2ms 
       { expected_response:true }...: avg=283.12ms min=169.77ms med=284.19ms max=667.09ms p(90)=317.66ms p(95)=327.44ms p(99.9)=574.2ms 
     http_req_failed................: 0.00%  ✓ 0          ✗ 10681
     http_req_receiving.............: avg=116.01µs min=27.33µs  med=57.79µs  max=9.94ms   p(90)=145.02µs p(95)=442.11µs p(99.9)=3.6ms   
     http_req_sending...............: avg=97.28µs  min=7.09µs   med=11.95µs  max=285.83ms p(90)=37.39µs  p(95)=154.2µs  p(99.9)=3.16ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=282.9ms  min=169.7ms  med=284.04ms max=666.27ms p(90)=317.44ms p(95)=327.17ms p(99.9)=573.31ms
     http_reqs......................: 10681  130.797607/s
     iteration_duration.............: avg=284.14ms min=170.03ms med=284.78ms max=681.58ms p(90)=318.21ms p(95)=327.82ms p(99.9)=613.02ms
     iterations.....................: 10581  129.573025/s
     success_rate...................: 99.98% ✓ 10579      ✗ 2    
     vus............................: 50     min=0        max=50 
     vus_max........................: 50     min=50       max=50 

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview
Summary for: `apollo-gateway`

K6 Output

     ✓ response code was 200
     ✓ no graphql errors
     ✓ valid response structure

     checks.........................: 100.00% ✓ 29169      ✗ 0   
     data_received..................: 863 MB  11 MB/s
     data_sent......................: 11 MB   140 kB/s
     http_req_blocked...............: avg=34.23µs  min=1.35µs   med=3.19µs   max=11.73ms  p(90)=4.93µs   p(95)=5.84µs   p(99.9)=9.6ms   
     http_req_connecting............: avg=29.79µs  min=0s       med=0s       max=11.7ms   p(90)=0s       p(95)=0s       p(99.9)=9.57ms  
     http_req_duration..............: avg=307.55ms min=210.61ms med=305.49ms max=791.56ms p(90)=332.6ms  p(95)=343.09ms p(99.9)=723.25ms
       { expected_response:true }...: avg=307.55ms min=210.61ms med=305.49ms max=791.56ms p(90)=332.6ms  p(95)=343.09ms p(99.9)=723.25ms
     http_req_failed................: 0.00%   ✓ 0          ✗ 9823
     http_req_receiving.............: avg=80.77µs  min=30.78µs  med=59.45µs  max=9.34ms   p(90)=109.23µs p(95)=135.17µs p(99.9)=2.14ms  
     http_req_sending...............: avg=92.7µs   min=6.38µs   med=12.94µs  max=308.05ms p(90)=23.42µs  p(95)=38.87µs  p(99.9)=3.04ms  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s       p(99.9)=0s      
     http_req_waiting...............: avg=307.37ms min=210.52ms med=305.4ms  max=790.16ms p(90)=332.39ms p(95)=342.93ms p(99.9)=721.95ms
     http_reqs......................: 9823    120.447217/s
     iteration_duration.............: avg=309.02ms min=226.29ms med=306.05ms max=832.29ms p(90)=333.04ms p(95)=343.49ms p(99.9)=780.98ms
     iterations.....................: 9723    119.221041/s
     success_rate...................: 100.00% ✓ 9723       ✗ 0   
     vus............................: 50      min=0        max=50
     vus_max........................: 50      min=50       max=50

Performance Overview

Performance Overview

HTTP Overview

HTTP Overview

# Conflicts:
#	gateways/hive-gateway-bun/package.json
#	gateways/hive-gateway/package.json
#	package-lock.json
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant