Skip to content

Native benchmark: iteration counter overflow #179

Open
@fzhinkin

Description

@fzhinkin

During the warmup phase, the native benchmarks executor estimates the repetitions of benchmarked function calls within each measurement iteration. That number is stored in an integer variable. Given a fast enough benchmark and long enough iterationTime, the repetitions counter (called iterations in the code) may overflow, leading to negative results in a report.

@State(Scope.Benchmark)
@BenchmarkMode(Mode.AverageTime)
open class TestBenchmark {
    private var x = 123L

    fun nextValue(): Long {
        val r = x
        x = -1 * (x * 6364136223846793005 + 1442695040888963407)
        return r
    }
    
    @Benchmark
    fun signBenchmark(): Int = nextValue().sign    
}
    configurations {
        named("main") {
            warmups = 2
            iterations = 3
            iterationTime = 40 * 4
            iterationTimeUnit = "sec"
            outputTimeUnit = "ns"
        }
    }

Results:

Warm-up #0: -79-0.538163 ns/op
Warm-up #1: -81-0.545774 ns/op
Iteration #0: 0-0.000000 ns/op
Iteration #1: 0 ns/op
Iteration #2: 0-0.000000 ns/op
  Success:   ~ 0-0.000000 ns/op ±-98-0.00%

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions