[infra] Check that fuzzers don't fail on an empty input file

[fmeum]

Adds a new bad build check for libfuzzer targets that runs the fuzzer on
an empty file without any further arguments. This is specifically meant
to catch issues with argument parsing in libFuzzer derivates such as
Jazzer.

Fixes #8276

[fmeum]

@oliverchang I don't see how the test failure is related to my change as it diffs cloud build steps. Do you know what could cause it?

[jonathanmetzman]

@oliverchang I'm not sure I like this. I think it's super tailored to catching the last problem (which should be caught through pinning and tests there) and I don't want to add overhead to bad_build_check which is used in ClusterFuzzLite.

[jonathanmetzman]

There may be fuzzers that are slow to startup and frankly I think any amount of time spent on this in ClusterFuzzLite is too much, it doesn't belong there and I think we should do something like this elsewhere.

[oliverchang]

@oliverchang I'm not sure I like this. I think it's super tailored to catching the last problem (which should be caught through pinning and tests there) and I don't want to add overhead to bad_build_check which is used in ClusterFuzzLite.

I think this is still important. We have a lot of wrapper scripts now with languages like Java, Python and we need to make sure reproducing works.

Even ignoring the last issue -- it seems to make perfect sense to me that a target should not crash with an empty input.

Re ClusterFuzzLite slowness -- is this something we can fix there instead? Does bad build check make sense there in general?

[oliverchang]

@fmeum could you please merge master into this PR to see if that fixes the infra tests?

[fmeum]

Will do.

@oliverchang If this check is mostly meant to apply to Jazzer, Atheris and potential future wrappers, maybe limiting the check to fuzzing languages other than C/C++ could help with @jonathanmetzman's concerns?

[oliverchang]

Thanks @fmeum! @jonathanmetzman WDYT?

[jonathanmetzman]

I'm not sold on this change, i don't think it's necessary and it slows things down.

[oliverchang]

Spoke with @jonathanmetzman offline here. The main concern was slowing down CFLite -- we could just gate this behind a flag and let CFLite opt out.

[fmeum]

@oliverchang Some of the checks fail with:

2022-09-09T14:50:01.9843353Z �[0m�[91mERROR: /src/centipede/BUILD:502:8: Compiling blob_file_test.cc failed: (Exit 1): clang-15 failed: error executing command /usr/local/bin/clang-15 -U_FORTIFY_SOURCE -fstack-protector -Wall -Wthread-safety -Wself-assign -Wunused-but-set-parameter -Wno-free-nonheap-object -fcolor-diagnostics -fno-omit-frame-pointer -g0 -O2 ... (remaining 53 arguments skipped)
2022-09-09T14:50:01.9844040Z 
2022-09-09T14:50:01.9844480Z Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
2022-09-09T14:50:01.9916525Z �[0m�[91mblob_file_test.cc:50:5: error: use of undeclared identifier 'EXPECT_OK'
2022-09-09T14:50:01.9916996Z     EXPECT_OK(appender->Open(path));

I'm also running into this locally.

[oliverchang]

@oliverchang Some of the checks fail with:

2022-09-09T14:50:01.9843353Z �[0m�[91mERROR: /src/centipede/BUILD:502:8: Compiling blob_file_test.cc failed: (Exit 1): clang-15 failed: error executing command /usr/local/bin/clang-15 -U_FORTIFY_SOURCE -fstack-protector -Wall -Wthread-safety -Wself-assign -Wunused-but-set-parameter -Wno-free-nonheap-object -fcolor-diagnostics -fno-omit-frame-pointer -g0 -O2 ... (remaining 53 arguments skipped)
2022-09-09T14:50:01.9844040Z 
2022-09-09T14:50:01.9844480Z Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
2022-09-09T14:50:01.9916525Z �[0m�[91mblob_file_test.cc:50:5: error: use of undeclared identifier 'EXPECT_OK'
2022-09-09T14:50:01.9916996Z     EXPECT_OK(appender->Open(path));

I'm also running into this locally.

Please merge master. This should be fixed by #8467

[fmeum]

@oliverchang Looks good now.

[oliverchang]

Should this be -n instead?

[fmeum]

Good catch, fixed.

[oliverchang]

I think this may still break due to the -u argument passed to the hashbang bash command.

You'll need something like ${MORE_BAD_BUILD_CHECKS:-} instead

[oliverchang]

Thanks! I'll merge this tomorrow since it's close to EOD for me.