Warmup in Alto Lab

[akoehn]

Original report by Alexander Koller (Bitbucket: akoller, GitHub: akoller).

---

When a task in Alto Lab has warmup > 0, it will attempt to run the warmup and the experiment at the same time. This defeats the purpose of warmup, and may mess up time measurements.

Cause of the problem: In the warmup iteration (CommandLineInterface.java:256), the call to program.run() returns before all warmup tasks are completed. In fact, it already returns before all warmup tasks have been submitted to the ForkJoinPool in Program.java. These tasks are still submitted to the pool. It is almost as if program#run were called in its own separate thread, and the task submissions took place in the background. But I can find no place where a new thread is being spawned.

This happens both in verbose mode and non-verbose, so the problem is probably not related to the use of the ConsoleProgressBar.

Workaround: Don't use warmup for now.