`std.Build.Step.Compile.create` with `Options.use_llvm = false` breaks fuzz testing

[Paladynee]

Zig Version

0.15.0-dev.97+677b2d62e

Steps to Reproduce and Observed Behavior

similar to #20990 in nature.

$ zig version
0.15.0-dev.97+677b2d62e

Minimal repro:

1. zig init
2. in build.zig change exe_unit_tests:

const exe_unit_tests = b.addTest(.{
    .root_module = exe_mod,
    .use_llvm = false,
});

3. zig build test --fuzz
   Observed behavior:

eurydice@serenity:~/coreign/minimal$ zig build test --fuzz
info: web interface listening at http://127.0.0.1:37931/
info: hint: pass --port 37931 to use this same port next time
thread 59931 panic: start index 1 is larger than end index 0
/snap/zig/13959/lib/std/Build/Fuzz/WebServer.zig:685:17: 0x159270e in addEntryPoint (build)
        for (pcs[1..], 1..) |elem_addr, i| {
                ^
/snap/zig/13959/lib/std/Build/Fuzz/WebServer.zig:566:56: 0x154148d in coverageRun (build)
            .entry_point => |entry_point| addEntryPoint(ws, entry_point.coverage_id, entry_point.addr) catch |err| switch (err) {
                                                       ^
/snap/zig/13959/lib/std/Thread.zig:510:13: 0x1507afd in callFn__anon_71977 (build)
            @call(.auto, f, args);
            ^
/snap/zig/13959/lib/std/Thread.zig:1403:30: 0x14c3ee1 in entryFn (build)
                return callFn(f, self.fn_args);
                             ^
/snap/zig/13959/lib/std/os/linux/x86_64.zig:126:5: 0x14bc6c1 in clone (build)
    asm volatile (
    ^
???:?:?: 0x0 in ??? (???)
error: the following build command crashed:
/home/eurydice/coreign/minimal/.zig-cache/o/fdcab04bb682fef78333aab175a3a144/build /snap/zig/13959/zig /snap/zig/13959/lib /home/eurydice/coreign/minimal /home/eurydice/coreign/minimal/.zig-cache /home/eurydice/.cache/zig --seed 0x6a556cf9 -Zbf398f7019233e9f test --fuzz

Expected Behavior

Expected to see normal fuzzer results:

eurydice@serenity:~/coreign/minimal$ zig build test --fuzz
info: web interface listening at http://127.0.0.1:39513/
info: hint: pass --port 39513 to use this same port next time
[0/1] Fuzzing
└─ main.test.fuzz example

[Paladynee]

Updated with minimal reproduction.

[presentfactory]

Note you do not actually need .use_llvm = false to trigger this. If you just run zig init then zig build test --fuzz it will give you the same error. I guess this is because use_llvm is set to false by default though because setting it to true oddly makes it work. I'm on 0.15.0-dev.767+201c0f54a for reference.

[gooncreeper]

The underlying issue here is that the self-hosted x86 backend does not provide the fuzzer the __sancov_cntrs and __sancov_pcs1 sections.

The missing __sancov_pcs1 section is the culprit for the apparent issue as the fuzzer gets zero for the start and end of the section making it seem there are 0 pcs (which cannot normally happen.) As shown in the stack trace, the web server assumes at least one 1 pc.

As @presentfactory pointed out, if you build with use_llvm = true, it will work since it does not use the self-hosted backend. This is also true for release modes; you should run fuzz testing on ReleaseSafe anyways for the performance benefits (and rerun on Debug with bad inputs.)

[travisstaloch]

As @presentfactory pointed out, if you build with use_llvm = true, it will work since it does not use the self-hosted backend. This is also true for release modes

This is happening with the llvm backend too.

I just tried this with 0.16.0-dev.319+4d1b15bd9 and i'm seeing the crash with debug and test_exe.use_llvm = true,. Same with a release safe build. Same when setting exe.use_llvm = true,

$ zig version
0.16.0-dev.319+4d1b15bd9
$ mkdir asdf && cd asdf
$ zig init
$ zig build test --fuzz --release=safe --webui=[::1]:45589
info(web_server): web interface listening at http://[::1]:45589/
thread 542504 panic: start index 1 is larger than end index 0
~/zig/stage3/lib/zig/std/Build/Fuzz.zig:428:17: 0x1455573 in addEntryPoint (std.zig)
        for (pcs[1..], 1..) |elem_addr, i| {
                ^
~/zig/stage3/lib/zig/std/Build/Fuzz.zig:313:56: 0x13d9355 in coverageRun (std.zig)
            .entry_point => |entry_point| addEntryPoint(fuzz, entry_point.coverage_id, entry_point.addr) catch |err| switch (err) {
                                                       ^
~/zig/stage3/lib/zig/std/Thread.zig:511:13: 0x134d120 in callFn__anon_82012 (std.zig)
            @call(.auto, f, args);
            ^
~/zig/stage3/lib/zig/std/Thread.zig:1383:30: 0x12c1bd8 in entryFn (std.zig)
                return callFn(f, self.fn_args);
                             ^
~/zig/stage3/lib/zig/std/os/linux/x86_64.zig:119:5: 0x1248975 in clone (std.zig)
    asm volatile (
    ^
???:?:?: 0x0 in ??? (???)
error: the following build command crashed:
.zig-cache/o/8d664209a4672f0195975a80ab4764c3/build ~/zig/stage3/bin/zig ~/zig/stage3/lib/zig ~/fuzzing-example/asdf .zig-cache /home/travis/.cache/zig --seed 0xbeeebfb8 -Zb102de79da5bd393 test --fuzz --release=safe --webui=[::1]:45589

This is a fresh local zig build. Same crash with zigup downloaded 0.16.0-dev.254+6dd0270a1.

[gooncreeper]

$ zig build test --fuzz --release=safe --webui=[::1]:45589

If you pass -Doptimize=ReleaseSafe instead of --release=safe with the example you gave it works fine.

For the other examples you listed, you will have to give more complete reproductions. Also you may want to pass use_llvm through std.Build.TestOptions instead of modifying it directly.

[travisstaloch]

If you pass -Doptimize=ReleaseSafe instead of --release=safe with the example you gave it works fine.

For the other examples you listed, you will have to give more complete reproductions. Also you may want to pass use_llvm through std.Build.TestOptions instead of modifying it directly.

Thanks for the response. I just tried again and things seem to be working fine. I guess there was some kind of issue on my end. I did notice afterward a bunch of orphaned test processes still running. Not sure how that happened. But maybe that was it. Anyway sorry for the noise.

Btw, --release=safe seems to work fine too.