In 14, performance dropped by 1/3, most likely due to vector operations.

[solenopsys]

Zig Version

0.14.0

Steps to Reproduce and Observed Behavior

Reproduce

git clone https://github.com/solenopsys/zig-pico
cd zig-pico

v 0.13 
zig run -O ReleaseFast src/request/bench.zig
Result: 5200000

v 0.14 
zig run -O ReleaseFast src/request/bench.zig
Result 3400000

Expected Behavior

Need same perfomance

[linusg]

It looks like this stacks on top of another regression before 0.13 (#17768 perhaps), at least I get even better numbers for 0.12.1:

$ ~/zig-windows-x86_64-0.12.1/zig.exe run -O ReleaseFast src/request/bench.zig
Reps per second: 4478000

$ ~/zig-windows-x86_64-0.13.0/zig.exe run -O ReleaseFast src/request/bench.zig
Reps per second: 3828000

$ ~/zig-windows-x86_64-0.14.0/zig.exe run -O ReleaseFast src/request/bench.zig
Reps per second: 3235000

[Rexicon226]

I've applied this diff:

Diff

diff --git a/src/request/bench.zig b/src/request/bench.zig
index ac0fa9e..302230c 100644
--- a/src/request/bench.zig
+++ b/src/request/bench.zig
@@ -24,7 +24,7 @@ pub fn main() !void {
         .method = "",
         .path = "",
         .minor_version = 0,
-        .num_headers = 0,
+        .num_headers = headers.len,
         .bytes_read = 0,
     };
 
@@ -34,23 +34,32 @@ pub fn main() !void {
         .body = "",
     };
 
-    var ret: i32 = 0;
-
-    const startTime = std.time.milliTimestamp();
+    var cycles: u64 = 0;
+    const warmups = 1000;
 
     const count = 10000000;
-    for (0..count) |_| {
-        httpRequest.params.num_headers = headers.len;
-        ret = picozig.parseRequest(REQ, &httpRequest);
-
-        std.mem.doNotOptimizeAway(ret);
-        std.debug.assert(ret == REQ.len);
+    for (0..count) |i| {
+        std.mem.doNotOptimizeAway(i);
+        const start = rdtsc();
+        std.mem.doNotOptimizeAway(picozig.parseRequest(REQ, &httpRequest));
+        const end = rdtsc();
+        if (i > warmups) cycles += (end - start);
     }
 
-    const endTime = std.time.milliTimestamp();
-    const elapsed = endTime - startTime;
-    const rps = @divTrunc(count, elapsed) * 1000;
-    std.debug.print("Reps per second: {d}\n", .{rps});
+    const cycles_per_iter = cycles / (count - warmups);
+    std.debug.print("Cycles per req: {d}\n", .{cycles_per_iter});
 
     return;
 }
+
+inline fn rdtsc() usize {
+    var a: u32 = undefined;
+    var b: u32 = undefined;
+    asm volatile ("rdtscp"
+        : [a] "={edx}" (a),
+          [b] "={eax}" (b),
+        :
+        : "ecx"
+    );
+    return (@as(u64, a) << 32) | b;
+}

To get what I think would be more accurate benchmarks, and I can consistently confirm a slowdown between 0.13 and 0.14 as well.

$ zig14 run -OReleaseFast src/request/bench.zig
Cycles per req: 894

$ zig13 run -OReleaseFast src/request/bench.zig
Cycles per req: 876

on a AMD Ryzen 9 7950X3D (32) @ 4.199GHz.

[solenopsys]

Ideally, we need to compare with https://github.com/h2o/picohttpparser and achieve even better performance. The goal is to leverage Zig's capabilities to enable various vectorized optimizations under the hood, such as SSE4.

[lerno]

It looks like Zig is using the wrong codegen setting during ReleaseFast: it has the correct optimization of the LLVM IR, but the settings for the target machine's codegen is wrong.

https://godbolt.org/z/bYfb7z454

You can compare 0.13 and trunk asm with the above.

[alexrp]

You can compare 0.13 and trunk asm with the above.

Just to clarify:

❯ cat repro.zig
export fn div(x: i32, y: i32) i32 {
    return @divTrunc(x, y);
}
❯ zig4 build-obj -fstrip -fno-unwind-tables -femit-asm -fno-emit-bin repro.zig -OReleaseFast
❯ cat repro.s
        .text
        .intel_syntax noprefix
        .file   "repro"
        .globl  div
        .p2align        4, 0x90
        .type   div,@function
div:
        push    rbp
        mov     rbp, rsp
        mov     eax, edi
        cdq
        idiv    esi
        pop     rbp
        ret
.Lfunc_end0:
        .size   div, .Lfunc_end0-div

        .section        ".note.GNU-stack","",@progbits

Is this the result you're referring to?

Note that if I add -fomit-frame-pointer, the resulting code is identical to 0.13.0. I can't find any evidence that we're setting the TargetMachine optimization level incorrectly.

[alexrp]

Well there you go:

❯ zig4 run -O ReleaseFast src/request/bench.zig
Reps per second: 2549000
❯ zig4 run -O ReleaseFast src/request/bench.zig -fomit-frame-pointer
Reps per second: 5040000

[Rexicon226]

That doesn't look like 1% or less...

[alexrp]

We've just found one of the few edge cases where the impact of frame pointers isn't 1%. There was bound to be some of those.

FWIW, sprinkling inline on everything also wins back performance:

❯ zig4 run -O ReleaseFast src/request/bench.zig
Reps per second: 5005000

Perhaps we ought to consider a @disableFramePointer(bool) builtin.

[alexrp]

Wait, no, ignore the above comments. Looks like I had something in the background interfering with the benchmarking. False alarm re: frame pointers.

However, now that I'm benchmarking without any interference, I can't actually reproduce the performance issue; quite the opposite actually.

# 0.13.0
❯ zig run -OReleaseFast src/request/bench.zig
Reps per second: 4837000
❯ zig run -OReleaseFast src/request/bench.zig
Reps per second: 4935000
❯ zig run -OReleaseFast src/request/bench.zig
Reps per second: 4861000

# master
❯ zig4 run -OReleaseFast src/request/bench.zig
Reps per second: 5347000
❯ zig4 run -OReleaseFast src/request/bench.zig
Reps per second: 5344000
❯ zig4 run -OReleaseFast src/request/bench.zig
Reps per second: 5347000

On a 13th Gen Intel(R) Core(TM) i9-13900K.