diff --git a/chapters/2-Measuring-Performance/2-1 Noise In Modern Systems.md b/chapters/2-Measuring-Performance/2-1 Noise In Modern Systems.md index 0e00eec6c6..b2978563d2 100644 --- a/chapters/2-Measuring-Performance/2-1 Noise In Modern Systems.md +++ b/chapters/2-Measuring-Performance/2-1 Noise In Modern Systems.md @@ -1,10 +1,10 @@ ## Noise in Modern Systems {#sec:secFairExperiments} -There are many features in hardware and software that are designed to increase performance, but not all of them have deterministic behavior. Let's consider Dynamic Frequency Scaling (DFS), a feature that allows a CPU to increase its frequency for a short time interval, making it run significantly faster. However, the CPU can't stay in the "overclocked" mode for a long time, so later, it decreases its frequency back to the base value. DFS usually depends a lot on a core temperature, which makes it hard to predict the impact on our experiments. +There are many features in hardware and software that are designed to increase performance, but not all of them have deterministic behavior. Let's consider Dynamic Frequency Scaling (DFS), a feature that allows a CPU to increase its frequency making far above the base frequency, allowing it run significantly faster. DFS is also frequently reffered to as the *turbo* mode. Unfortunately, a CPU cannot stay in the turbo mode for a long time, otherwise it may face the risk of overheating. So later, it decreases its frequency to stay within its thermal limits. DFS usually depends a lot on the current system load and external factors, such as core temperature, which makes it hard to predict the impact on our experiments. -If we start two runs of the benchmark, one right after another on a "cold" processor,[^1] the first run could possibly work for some time in the "overclocked" mode and then decrease its frequency back to the base level. However, it's possible that the second run might not have this advantage and will operate at the base frequency without entering "turbo mode". Even though we run the exact same version of the program two times, the environment in which they run is not the same. Figure @fig:FreqScaling shows a situation where dynamic frequency scaling can cause variance in measurements. As you can see, the first run is 200 milliseconds faster than the second run due to the fact that it was running on a higher frequency in the beginning. Such a scenario can frequently happen when you benchmark software on a laptop since laptops have limited heat dissipation. +Figure @fig:FreqScaling shows a typical example where DFS can cause variance in measurements. In our scenario, we started two runs of the benchmark, one right after another on a "cold" processor.[^1] During the first second, the first iteration of the benchmark was running on the maximum turbo frequency of 4.4 Ghz but later it has to decrease its frequency a bit below 4 Ghz. Second run did not have the advantage of boosting the CPU frequency and did not enter the turbo mode. Even though we ran the exact same version of a program two times, the environment in which they ran was not the same. As you can see, the first run is 200 milliseconds faster than the second run due to the fact that it was running on a higher frequency in the beginning. Such a scenario can frequently happen when you benchmark software on a laptop since laptops have limited heat dissipation. -![Variance in performance caused by frequency scaling: the first run is 1 second faster than the second.](../../img/measurements/FreqScaling.jpg){#fig:FreqScaling width=90%} +![Variance in performance caused by dynamic frequency scaling: the first run is 200 milliseconds faster than the second.](../../img/measurements/FreqScaling.jpg){#fig:FreqScaling width=90%} Remember that even running Windows task manager or Linux `top` programs, can affect measurements since an additional CPU core will be activated and assigned to it. This might affect the frequency of the core that is running the actual benchmark. diff --git a/img/measurements/FreqScaling.jpg b/img/measurements/FreqScaling.jpg index 44e0cd7430..2b972dd0a4 100644 Binary files a/img/measurements/FreqScaling.jpg and b/img/measurements/FreqScaling.jpg differ