CC @fmeum too

Related: #8241

The Jazzer bug caused a crash if the fuzz target wasn't executed with a -seed argument. Both the bad build check and all our internal tests did that for the sake of reproducibility.

Not sure what to do about this particular bug type - we now have an internal one covering it.

Could we perhaps add one extra check (for libFuzzer engines) where we:

• Try running with -runs=1
• Try reproducing with an empty input

None of these should result in a crash if it's a good build.

Could we perhaps add one extra check (for libFuzzer engines) where we:

• Try running with -runs=1
• Try reproducing with an empty input

None of these should result in a crash if it's a good build.

IMO the check worked as it should. There was nothing wrong with Jazzer projects and reporting the bugs there instead of when we update Jazzer is not great UX. Ideally we wcould make seperate tests that run during updating

Could we perhaps add one extra check (for libFuzzer engines) where we:

• Try running with -runs=1
• Try reproducing with an empty input

None of these should result in a crash if it's a good build.

IMO the check worked as it should. There was nothing wrong with Jazzer projects and reporting the bugs there instead of when we update Jazzer is not great UX. Ideally we wcould make seperate tests that run during updating

If we don't pin, yes.

If we had these bad build checks and we pin, then we can catch them with our trial_build checks right?

Even ignoring Jazzer, making sure a target doesn't crash with an empty input is pretty valid right?

Right, I think we should absolutely pin though

Even ignoring Jazzer, making sure a target doesn't crash with an empty input is pretty valid right?

I guess. I think our existing bad build check probably cover this 99% of the time and just happened to miss it here.

Indeed. Let's:

1. Keep pinning Jazzer
2. Add a test to make sure that running against with an empty input does not crash in some way to bad_build_check.

@fmeum you mentioned being able to help with this. Would you be able to help us with a PR here? Thanks!

@oliverchang With empty input, do you mean an invocation of the form fuzz_target empty_file? If so, that would have caught the issue and I can add it in a PR.

@oliverchang With empty input, do you mean an invocation of the form fuzz_target empty_file? If so, that would have caught the issue and I can add it in a PR.

Yep!