Skip to content

Add walltime support for vitest plugin #49

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: cod-1066-tinybench-nodejs-support-for-the-walltime-executor
Choose a base branch
from

Conversation

GuillaumeLagrange
Copy link

@GuillaumeLagrange GuillaumeLagrange commented Jul 26, 2025

Code is functional barring one issue: timing sleep benchmarks have some very low outliers min value (microseconds when waiting multiple miliseconds)

This appear to be the case when running vitest benches with the default runner, so not sure how we should behave here
Reproduced

  • locally on my machine with vitest v1.2.2 (current dev dependency) and 3.2.4 (latest)
  • in ci with vitest v1.2.2

@GuillaumeLagrange GuillaumeLagrange force-pushed the cod-241-vitest-nodejs-support-for-the-walltime-executor branch from 124dd9a to b3ebc91 Compare July 26, 2025 17:29
Copy link

codspeed-hq bot commented Jul 26, 2025

CodSpeed WallTime Performance Report

Merging #49 will degrade performances by 14.29%

Comparing cod-241-vitest-nodejs-support-for-the-walltime-executor (c9d2169) with cod-1066-tinybench-nodejs-support-for-the-walltime-executor (bfb95e2)

Summary

❌ 1 regressions
✅ 26 untouched benchmarks
🆕 17 new benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

Benchmark BASE HEAD Change
🆕 fibo 30 N/A 109.5 ms N/A
switch 2 144 ns 168 ns -14.29%
🆕 fibo 10 N/A 8.1 µs N/A
🆕 fibo 15 N/A 78.3 µs N/A
🆕 long body N/A 235.7 µs N/A
🆕 short body N/A 24.9 µs N/A
🆕 end N/A 57.3 µs N/A
🆕 long body N/A 240 µs N/A
🆕 short body N/A 25.2 µs N/A
🆕 short body N/A 23.8 µs N/A
🆕 long body N/A 240.8 µs N/A
🆕 short body N/A 23.5 µs N/A
🆕 long body N/A 239.7 µs N/A
🆕 short body N/A 23.5 µs N/A
🆕 short body N/A 23.8 µs N/A
🆕 wait 1ms N/A 1 ms N/A
🆕 wait 1sec N/A 1 s N/A
🆕 wait 500ms N/A 500 ms N/A

Copy link

codspeed-hq bot commented Jul 26, 2025

CodSpeed Instrumentation Performance Report

Merging #49 will degrade performances by 66.79%

Comparing cod-241-vitest-nodejs-support-for-the-walltime-executor (c9d2169) with cod-1066-tinybench-nodejs-support-for-the-walltime-executor (bfb95e2)

Summary

⚡ 2 improvements
❌ 2 (👁 2) regressions
✅ 71 untouched benchmarks
🆕 7 new benchmarks
⁉️ 10 (👁 10) dropped benchmarks

Benchmarks breakdown

Benchmark BASE HEAD Change
test_iterative_fibo_100 465.2 µs 282.1 µs +64.87%
👁 fibo 10 14.9 µs N/A N/A
🆕 fibo 30 N/A 175 ms N/A
👁 short body 3 63.6 µs 191.6 µs -66.79%
👁 wait 1sec 12.3 ms 23 ms -46.42%
wait 500ms 12.3 ms 10.8 ms +13.31%
👁 end 123.5 µs N/A N/A
👁 one 108.6 µs N/A N/A
👁 two 108.7 µs N/A N/A
🆕 end N/A 207.9 µs N/A
🆕 one N/A 228.2 µs N/A
🆕 two N/A 200.5 µs N/A
👁 one 133 µs N/A N/A
👁 five 143.5 µs N/A N/A
👁 seven 141.2 µs N/A N/A
👁 three 141.4 µs N/A N/A
👁 two 125 µs N/A N/A
👁 one 325.6 µs N/A N/A
🆕 wait 1ms N/A 24.3 µs N/A
🆕 wait 1sec N/A 23 ms N/A
... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

@GuillaumeLagrange GuillaumeLagrange force-pushed the cod-241-vitest-nodejs-support-for-the-walltime-executor branch from 85f6528 to 7a1c502 Compare July 26, 2025 18:10
@GuillaumeLagrange GuillaumeLagrange force-pushed the cod-1066-tinybench-nodejs-support-for-the-walltime-executor branch from 1944230 to 022c470 Compare July 26, 2025 20:10
@GuillaumeLagrange GuillaumeLagrange force-pushed the cod-241-vitest-nodejs-support-for-the-walltime-executor branch from 7a1c502 to 8245f30 Compare July 26, 2025 20:10
@GuillaumeLagrange GuillaumeLagrange marked this pull request as ready for review July 26, 2025 20:14
@GuillaumeLagrange GuillaumeLagrange force-pushed the cod-241-vitest-nodejs-support-for-the-walltime-executor branch from 8245f30 to fc28669 Compare July 26, 2025 20:23
@adriencaccia adriencaccia force-pushed the cod-1066-tinybench-nodejs-support-for-the-walltime-executor branch from 7b3194b to 3086a30 Compare July 29, 2025 16:34
@adriencaccia adriencaccia force-pushed the cod-241-vitest-nodejs-support-for-the-walltime-executor branch from a1527d5 to 56710b1 Compare July 29, 2025 16:34
@adriencaccia adriencaccia force-pushed the cod-1066-tinybench-nodejs-support-for-the-walltime-executor branch from 3086a30 to 940276b Compare July 30, 2025 10:02
@adriencaccia adriencaccia force-pushed the cod-241-vitest-nodejs-support-for-the-walltime-executor branch 2 times, most recently from 9b6bc65 to e074ae9 Compare July 30, 2025 10:28
@adriencaccia adriencaccia force-pushed the cod-1066-tinybench-nodejs-support-for-the-walltime-executor branch from 940276b to bfb95e2 Compare July 30, 2025 12:39
@adriencaccia adriencaccia force-pushed the cod-241-vitest-nodejs-support-for-the-walltime-executor branch 2 times, most recently from 2ad03f2 to 0f4ab29 Compare July 30, 2025 14:44
@adriencaccia adriencaccia force-pushed the cod-241-vitest-nodejs-support-for-the-walltime-executor branch from 0f4ab29 to c9d2169 Compare July 30, 2025 14:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants