Skip to content

Bump node, bump tinybench, and add support of walltime runs for tinybench #48

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 12 commits into
base: main
Choose a base branch
from

Conversation

GuillaumeLagrange
Copy link

@GuillaumeLagrange GuillaumeLagrange commented Jul 24, 2025

Fix #44

@GuillaumeLagrange GuillaumeLagrange force-pushed the cod-1066-tinybench-nodejs-support-for-the-walltime-executor branch 3 times, most recently from 59f5f8c to b1db2f2 Compare July 24, 2025 16:13
Copy link

codspeed-hq bot commented Jul 24, 2025

CodSpeed Instrumentation Performance Report

Merging #48 will degrade performances by 96.36%

Comparing cod-1066-tinybench-nodejs-support-for-the-walltime-executor (bfb95e2) with main (6df11e8)

Summary

⚡ 10 improvements
❌ 22 (👁 22) regressions
✅ 50 untouched benchmarks
🆕 3 new benchmarks
⁉️ 4 (👁 4) dropped benchmarks

Benchmarks breakdown

Benchmark BASE HEAD Change
👁 RegExp#test 14 µs N/A N/A
👁 String#indexOf 10.3 µs N/A N/A
👁 switch 1 11.9 µs N/A N/A
👁 switch 2 12.3 µs N/A N/A
👁 switch 1 11.9 µs 23.6 µs -49.62%
👁 switch 2 12.4 µs 24.5 µs -49.56%
👁 test_iterative_fibo_10 16.5 µs 453.2 µs -96.36%
👁 test_iterative_fibo_100 24.8 µs 202.5 µs -87.74%
👁 test_recursive_cached_fibo_10 30.6 µs 45.7 µs -33.21%
👁 test_recursive_cached_fibo_20 35.7 µs 43.1 µs -17.36%
👁 test_recursive_cached_fibo_30 41.7 µs 216.6 µs -80.75%
👁 test sync baz 10 16.1 µs 209.6 µs -92.31%
👁 test sync baz 100 17.7 µs 30.4 µs -41.86%
test_recursive_fibo_10 34.6 µs 21.7 µs +59.59%
👁 test_iterative_fibo_10 16.5 µs 28.5 µs -42.15%
👁 test_iterative_fibo_100 24.7 µs 465.2 µs -94.68%
👁 test_recursive_cached_fibo_10 30.6 µs 45.2 µs -32.21%
👁 test_recursive_cached_fibo_20 35.6 µs 43 µs -17.18%
👁 test_recursive_cached_fibo_30 41.8 µs 46.7 µs -10.43%
👁 test sync baz 10 17 µs 38.3 µs -55.66%
... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

@GuillaumeLagrange GuillaumeLagrange force-pushed the cod-1066-tinybench-nodejs-support-for-the-walltime-executor branch 3 times, most recently from 065360b to 21234dd Compare July 25, 2025 07:58
Copy link

codspeed-hq bot commented Jul 25, 2025

CodSpeed WallTime Performance Report

Congrats! CodSpeed is installed 🎉

🆕 27 new benchmarks were detected.

You will start to see performance impacts in the reports once the benchmarks are run from your default branch.

Detected benchmarks

  • switch 1 (156 ns)
  • switch 2 (156 ns)
  • test_iterative_fibo_10 (228 ns)
  • test_iterative_fibo_100 (504 ns)
  • test_recursive_cached_fibo_10 (984 ns)
  • test_recursive_cached_fibo_20 (2.4 µs)
  • test_recursive_cached_fibo_30 (3.1 µs)
  • test_recursive_fibo_10 (2.1 µs)
  • test_recursive_fibo_20 (248.3 µs)
  • test sync baz 10 (168 ns)
  • test sync baz 100 (276 ns)
  • test_iterative_fibo_10 (216 ns)
  • test_iterative_fibo_100 (504 ns)
  • test_recursive_cached_fibo_10 (996 ns)
  • test_recursive_cached_fibo_20 (2.3 µs)
  • test_recursive_cached_fibo_30 (3.1 µs)
  • test_recursive_fibo_10 (2.1 µs)
  • test_recursive_fibo_20 (248.1 µs)
  • test sync baz 10 (168 ns)
  • test sync baz 100 (288 ns)
  • switch 1 (156 ns)
  • switch 2 (156 ns)
  • switch 1 (144 ns)
  • switch 2 (144 ns)
  • wait 1ms (1 ms)
  • wait 1sec (1 s)
  • wait 500ms (500 ms)

@GuillaumeLagrange GuillaumeLagrange force-pushed the cod-1066-tinybench-nodejs-support-for-the-walltime-executor branch 4 times, most recently from 62e121a to 9db57da Compare July 25, 2025 09:04
Node 22 failed because of a rollup issue regarding assert keyword that I
was not able to fix yet
20.5.1 is chosen because of this:
egoist/esbuild-register#96
This prevents us from running our examples easily
@GuillaumeLagrange GuillaumeLagrange force-pushed the cod-1066-tinybench-nodejs-support-for-the-walltime-executor branch from 9db57da to 2b1bdc9 Compare July 25, 2025 09:13
@GuillaumeLagrange GuillaumeLagrange marked this pull request as ready for review July 25, 2025 09:15
@GuillaumeLagrange GuillaumeLagrange changed the title Cod 1066 tinybench nodejs support for the walltime executor Bump node, bump tinybench, and add support of walltime runs for tinybench Jul 25, 2025
Copy link
Member

@adriencaccia adriencaccia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!
Really eager to release this

@Nerivec
Copy link

Nerivec commented Jul 25, 2025

My two cents on the node issues: since you're at it, might be best to move away from esbuild-register, seems it's not maintained anymore? https://github.com/egoist/esbuild-register/commits/dev/

@GuillaumeLagrange
Copy link
Author

My two cents on the node issues: since you're at it, might be best to move away from esbuild-register, seems it's not maintained anymore? https://github.com/egoist/esbuild-register/commits/dev/

That's the plan, but I ran into issues making tsx work under our valgrind instrumentation for now, and do not have time to deep dive yet.
Definitely the long term fix is to migrate towards tsx or another modern equivalent of esbuild-register. Who knows, we might even be able to use node's native solution someday 👼

@GuillaumeLagrange GuillaumeLagrange force-pushed the cod-1066-tinybench-nodejs-support-for-the-walltime-executor branch 2 times, most recently from f000974 to 6129c06 Compare July 25, 2025 12:50
@GuillaumeLagrange GuillaumeLagrange force-pushed the cod-1066-tinybench-nodejs-support-for-the-walltime-executor branch from 6129c06 to b94dd64 Compare July 25, 2025 13:11
@GuillaumeLagrange GuillaumeLagrange force-pushed the cod-1066-tinybench-nodejs-support-for-the-walltime-executor branch 2 times, most recently from 1944230 to 022c470 Compare July 26, 2025 20:10
@adriencaccia adriencaccia force-pushed the cod-1066-tinybench-nodejs-support-for-the-walltime-executor branch 2 times, most recently from 3086a30 to 940276b Compare July 30, 2025 10:02
@adriencaccia adriencaccia force-pushed the cod-1066-tinybench-nodejs-support-for-the-walltime-executor branch from 940276b to bfb95e2 Compare July 30, 2025 12:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

tinybench-plugin doesn't work with tinybench@4
3 participants