Skip to content

Commit

Permalink
Preliminary support for libfuzzer
Browse files Browse the repository at this point in the history
  • Loading branch information
@smoelius
smoelius committed May 1, 2023
1 parent 97c1d65 commit fe92156
Show file tree
Hide file tree
Showing 58 changed files with 1,643 additions and 546 deletions.
63 changes: 55 additions & 8 deletions .github/workflows/ci.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -99,12 +99,47 @@ jobs:
cargo clean && cargo +nightly udeps --features=test-fuzz/auto_concretize --all-targets
test:
runs-on: ubuntu-latest

strategy:
matrix:
serde_format: [bincode, cbor, cbor4ii]
toolchain: [stable, nightly]
include:
- fuzzer: aflplusplus
serde_format: bincode
environment: ubuntu-latest
toolchain: stable
- fuzzer: aflplusplus
serde_format: cbor
environment: ubuntu-latest
toolchain: nightly
- fuzzer: aflplusplus
serde_format: cbor4ii
environment: macos-latest
toolchain: stable
- fuzzer: aflplusplus-persistent
serde_format: bincode
environment: macos-latest
toolchain: nightly
- fuzzer: aflplusplus-persistent
serde_format: cbor
environment: ubuntu-latest
toolchain: stable
- fuzzer: aflplusplus-persistent
serde_format: cbor4ii
environment: ubuntu-latest
toolchain: nightly
- fuzzer: libfuzzer
serde_format: bincode
environment: macos-latest
toolchain: stable
- fuzzer: libfuzzer
serde_format: cbor
environment: macos-latest
toolchain: nightly
- fuzzer: libfuzzer
serde_format: cbor4ii
environment: ubuntu-latest
toolchain: stable

runs-on: ${{ matrix.environment }}

steps:
- uses: actions/checkout@v3
Expand All @@ -113,6 +148,7 @@ jobs:
run: rustup default ${{ matrix.toolchain }}

- name: Install llvm
if: ${{ matrix.environment == 'ubuntu-latest' }}
run: sudo apt-get install llvm

# smoelius: The Substrate tests require `protoc`.
Expand All @@ -123,6 +159,15 @@ jobs:
- name: Install afl
run: cargo install afl

- name: Run afl-system-config
run: |
sudo "$HOME"/.local/share/afl.rs/rustc-*/afl.rs-*/afl/bin/afl-system-config
if [[ ${{ matrix.environment }} = 'macos-latest' ]]; then
SL=/System/Library; PL=com.apple.ReportCrash
launchctl unload -w ${SL}/LaunchAgents/${PL}.plist
sudo launchctl unload -w ${SL}/LaunchDaemons/${PL}.Root.plist
fi
# smoelius: The wasm32 target is needed for some Substrate tests, regardless of the toolchain
# used to build test-fuzz.
- name: Add wasm32 target
Expand All @@ -136,18 +181,18 @@ jobs:
AUTO_CONCRETIZE=
IGNORED=
SHUFFLE=
if [[ ${{ matrix.toolchain }} = nightly ]]; then
if [[ ${{ matrix.toolchain }} = 'nightly' ]]; then
AUTO_CONCRETIZE='--features=test-fuzz/auto_concretize'
SHUFFLE='-Z unstable-options --shuffle --test-threads=1'
fi
if [[ ${{ github.event_name }} = schedule ]]; then
if [[ ${{ github.event_name }} = 'schedule' ]]; then
IGNORED='--ignored'
fi
cargo test --features=test-fuzz/serde_${{ matrix.serde_format }} "$AUTO_CONCRETIZE" -- --nocapture $IGNORED $SHUFFLE
env:
AFL_I_DONT_CARE_ABOUT_MISSING_CRASHES: 1
RUST_BACKTRACE: 1
RUST_LOG: warn
TEST_FUZZ_FUZZER: ${{ matrix.fuzzer }}

test-uninstalled-cargo-afl:
runs-on: ubuntu-latest
Expand All @@ -162,13 +207,14 @@ jobs:
run: |
OUTPUT="$(cargo run -p cargo-test-fuzz -- test-fuzz -p test-fuzz-examples --no-run 2>&1 1>/dev/null || true)"
echo "$OUTPUT"
echo "$OUTPUT" | grep '^Error: Could not determine `cargo-afl` version. Is it installed? Try `cargo install afl`.$'
echo "$OUTPUT" | grep 'Could not determine `cargo-afl` version. Is it installed? Try `cargo install afl`.'
test-incompatible-cargo-afl:
runs-on: ubuntu-latest

env:
RUSTUP_TOOLCHAIN: nightly
TEST_FUZZ_FUZZER: aflplusplus-persistent

steps:
- uses: actions/checkout@v3
Expand All @@ -190,6 +236,7 @@ jobs:

env:
RUSTUP_TOOLCHAIN: nightly
TEST_FUZZ_FUZZER: aflplusplus-persistent

steps:
- uses: actions/checkout@v3
Expand Down
90 changes: 46 additions & 44 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -248,50 +248,52 @@ The `cargo test-fuzz` command is used to interact with fuzz targets, and to mani
#### Options

```
--backtrace Display backtraces
--consolidate Move one target's crashes, hangs, and work queue to its corpus; to
consolidate all targets, use --consolidate-all
--display <OBJECT> Display concretizations, corpus, crashes, `impl` concretizations,
hangs, or work queue. By default, corpus uses an uninstrumented fuzz
target; the others use an instrumented fuzz target. To display the
corpus with instrumentation, use --display corpus-instrumented.
[possible values: concretizations, corpus, corpus-instrumented,
crashes, hangs, impl-concretizations, queue]
--exact Target name is an exact name rather than a substring
--exit-code Exit with 0 if the time limit was reached, 1 for other programmatic
aborts, and 2 if an error occurred; implies --no-ui, does not imply
--run-until-crash or -- -V <SECONDS>
--features <FEATURES> Space or comma separated list of features to activate
--list List fuzz targets
--manifest-path <PATH> Path to Cargo.toml
--no-default-features Do not activate the `default` feature
--no-instrumentation Compile without instrumentation (for testing build process)
--no-run Compile, but don't fuzz
--no-ui Disable user interface
-p, --package <PACKAGE> Package containing fuzz target
--persistent Enable persistent mode fuzzing
--pretty-print Pretty-print debug output when displaying/replaying
--replay <OBJECT> Replay corpus, crashes, hangs, or work queue. By default, corpus uses
an uninstrumented fuzz target; the others use an instrumented fuzz
target. To replay the corpus with instrumentation, use --replay
corpus-instrumented. [possible values: concretizations, corpus,
corpus-instrumented, crashes, hangs, impl-concretizations, queue]
--reset Clear fuzzing data for one target, but leave corpus intact; to reset
all targets, use --reset-all
--resume Resume target's last fuzzing session
--run-until-crash Stop fuzzing once a crash is found
--test <NAME> Integration test containing fuzz target
--timeout <TIMEOUT> Number of milliseconds to consider a hang when fuzzing or replaying
(equivalent to -- -t <TIMEOUT> when fuzzing)
--verbose Show build output when displaying/replaying
-h, --help Print help
-V, --version Print version
To fuzz at most <SECONDS> of time, use:
cargo test-fuzz ... -- -V <SECONDS>
Try `cargo afl fuzz --help` to see additional fuzzer options.
--backtrace Display backtraces
--consolidate Move one target's crashes, hangs, and work queue to its corpus; to
consolidate all targets, use --consolidate-all
--display <OBJECT> Display concretizations, corpus, crashes, `impl` concretizations,
hangs, or work queue. By default, corpus uses an uninstrumented
fuzz target; the others use an instrumented fuzz target. To
display the corpus with instrumentation, use --display
corpus-instrumented. [possible values: concretizations, corpus,
corpus-instrumented, crashes, hangs, impl-concretizations, queue]
--exact Target name is an exact name rather than a substring
--exit-code Exit with 0 if the time limit was reached, 1 for other
programmatic aborts, and 2 if an error occurred; implies --no-ui,
does not imply --run-until-crash or --max-total-time <SECONDS>
--features <FEATURES> Space or comma separated list of features to activate
--fuzzer <FUZZER> Fuzz using <FUZZER> [possible values: aflplusplus,
aflplusplus-persistent, libfuzzer]
--list List fuzz targets
--manifest-path <PATH> Path to Cargo.toml
--max-total-time <SECONDS> Fuzz at most <SECONDS> of time (equivalent to -- -V <SECONDS> for
aflplusplus, and -- --max_total_time <SECONDS> for libfuzzer)
--no-default-features Do not activate the `default` feature
--no-instrumentation Compile without instrumentation (for testing build process)
--no-run Compile, but don't fuzz
--no-ui Disable user interface
-p, --package <PACKAGE> Package containing fuzz target
--pretty-print Pretty-print debug output when displaying/replaying
--replay <OBJECT> Replay corpus, crashes, hangs, or work queue. By default, corpus
uses an uninstrumented fuzz target; the others use an instrumented
fuzz target. To replay the corpus with instrumentation, use
--replay corpus-instrumented. [possible values: concretizations,
corpus, corpus-instrumented, crashes, hangs, impl-concretizations,
queue]
--reset Clear fuzzing data for one target, but leave corpus intact; to
reset all targets, use --reset-all
--resume Resume target's last fuzzing session
--run-until-crash Stop fuzzing once a crash is found
--test <NAME> Integration test containing fuzz target
--timeout <TIMEOUT> Number of milliseconds to consider a hang when fuzzing or
replaying (equivalent to -- -t <TIMEOUT> when fuzzing with
aflplusplus, and -- -timeout <TIMEOUT/1000> when fuzzing with
libfuzzer)
--verbose Show build output when displaying/replaying
-h, --help Print help
-V, --version Print version
Try `cargo afl fuzz --help` to see additional AFLplusplus options.
```

### Convenience functions and macros
Expand Down
8 changes: 6 additions & 2 deletions cargo-test-fuzz/Cargo.toml
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[package]
name = "cargo-test-fuzz"
version = "3.0.5"
edition = "2018"
edition = "2021"

description = "cargo-test-fuzz"

Expand All @@ -17,18 +17,22 @@ path = "src/bin/cargo_test_fuzz.rs"
doctest = false

[dependencies]
anyhow = "1.0"
anyhow = { version = "1.0", features = ["backtrace"] }
bitflags = "2.2"
cargo-fuzz = { git = "https://github.com/trail-of-forks/cargo-fuzz", features = ["no-manifest-check"] }
cargo_metadata = "0.15"
clap = { version = "4.2", features = ["cargo", "derive", "wrap_help"] }
env_logger = "0.10"
fs_extra = "1.3"
heck = "0.4"
lazy_static = "1.4"
log = "0.4"
once_cell = "1.16"
paste = "1.0"
remain = "0.2"
semver = "1.0"
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
strum_macros = "0.24"
subprocess = "0.2"

Expand Down
Loading

0 comments on commit fe92156

Please sign in to comment.