Crowdsource benchmark results

[viktorklang]

Want to help verifying the performance improvements of Scala Futures?

• Clone this repo
• Shut down all other apps
• Start sbt on the root project on the master branch
• Run benches/jmh:runMain scala.future.BenchRunner -p threads=1 -p recursion=8192 -i 30 -wi 15 -f1 -t1 Benchmark
• Let the benchmarks finish running
• Gist the output of the benches and link it on this issue!

[alexandru]

Something wrong with that command:

sbt:scala-futures> benches/jmh:runMain scala.future.BenchRunner -p threads=1 -p recursion=8192 -i 30 -wi 15 -f1 -t1 Benchmarks
[info] Packaging /Users/alex/Projects/personale/scala-futures/target/scala-2.12/scala-futures_2.12-0.1-SNAPSHOT.jar ...
[info] Packaging /Users/alex/Projects/personale/scala-futures/benches/target/scala-2.12/benches_2.12-0.1-SNAPSHOT.jar ...
[info] Packaging /Users/alex/Projects/personale/scala-futures/benches/target/scala-2.12/benches_2.12-0.1-SNAPSHOT-tests.jar ...
[info] Done packaging.
Processing 33 classes from /Users/alex/Projects/personale/scala-futures/benches/target/scala-2.12/classes with "reflection" generator
Writing out Java source to /Users/alex/Projects/personale/scala-futures/benches/target/scala-2.12/src_managed/jmh and resources to /Users/alex/Projects/personale/scala-futures/benches/target/scala-2.12/resource_managed/jmh
[info] Done packaging.
[info] Done packaging.
[info] Compiling 136 Java sources to /Users/alex/Projects/personale/scala-futures/benches/target/scala-2.12/classes ...
[info] Done compiling.
[info] Packaging /Users/alex/Projects/personale/scala-futures/benches/target/scala-2.12/benches_2.12-0.1-SNAPSHOT-jmh.jar ...
[info] Done packaging.
[info] Running (fork) scala.future.BenchRunner -p threads=1 -p recursion=8192 -i 30 -wi 15 -f1 -t1 Benchmarks
[error] Exception in thread "main" No benchmarks to run; check the include/exclude regexps.
[error] 	at org.openjdk.jmh.runner.Runner.internalRun(Runner.java:268)
[error] 	at org.openjdk.jmh.runner.Runner.run(Runner.java:209)
[error] 	at scala.future.BenchRunner$.main(BenchRunner.scala:25)
[error] 	at scala.future.BenchRunner.main(BenchRunner.scala)
[error] java.lang.RuntimeException: Nonzero exit code returned from runner: 1
[error] 	at sbt.ForkRun.processExitCode$1(Run.scala:29)
[error] 	at sbt.ForkRun.run(Run.scala:38)
[error] 	at sbt.Defaults$.$anonfun$bgRunMainTask$6(Defaults.scala:1133)
[error] 	at sbt.Defaults$.$anonfun$bgRunMainTask$6$adapted(Defaults.scala:1128)
[error] 	at sbt.internal.BackgroundThreadPool.$anonfun$run$1(DefaultBackgroundJobService.scala:359)
[error] 	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
[error] 	at scala.util.Try$.apply(Try.scala:209)
[error] 	at sbt.internal.BackgroundThreadPool$BackgroundRunnable.run(DefaultBackgroundJobService.scala:282)
[error] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[error] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[error] 	at java.lang.Thread.run(Thread.java:745)
[error] (benches/jmh:runMain) Nonzero exit code returned from runner: 1
[error] Total time: 5 s, completed Mar 26, 2018 10:23:54 AM

[viktorklang]

@alexandru Yeah, you wrote "Benchmarks" instead of "Benchmark" :)

benches/jmh:runMain scala.future.BenchRunner -p threads=1 -p recursion=8192 -i 30 -wi 15 -f1 -t1 Benchmark

[adisesha]

I couldn't find any log file. Here is the console output.

[viktorklang]

@adisesha Thank you!

[danicheg]

Literally full output.

[sbjorn]

Benchmark run on a Ryzen 1700X.

[alexandru]

@viktorklang here's the output.

Test system is a MacBook Pro 2015, 2,2 GHz Intel Core i7, 16 GB 1600 MHz DDR3. I disabled all apps I could and the network connection, like I do whenever I do benchmarks ... display went off at some point though, not sure if that matters.

[alexandru]

I'm really happy you're optimizing Future btw 👍 🖖

[ktoso]

Terminal I ran this on decided to truncate output...
Let me know if you want the entire run's output and I can re-do it.

Results https://gist.github.com/ktoso/dbfddb7630ae2e917e92ab9f41e626d0
Same machine as #3

cpu: Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz (HT enabled)
memory: 16G DDR4 2400Mhz
(Dell XPS15, mid 2017)