thermostatter/aflplusplus repository overview

⁠American Fuzzy Lop plus plus (afl++)

Release Version: 2.68c⁠

Github Version: 3.00a

Repository: https://github.com/AFLplusplus/AFLplusplus⁠

afl++ is maintained by:

• Marc "van Hauser" Heuse [email protected]⁠,
• Heiko "hexcoder-" Eißfeldt [email protected]⁠,
• Andrea Fioraldi [email protected]⁠ and
• Dominik Maier [email protected]⁠.

Originally developed by Michał "lcamtuf" Zalewski.

afl++ is a superior fork to Google's afl - more speed, more and better mutations, more and better instrumentation, custom module support, etc.

If you want to use afl++ for your academic work, check the papers page⁠ on the website.

⁠Major changes in afl++ 3.0

With afl++ 3.0 we introduced changes that break some previous afl and afl++ behaviours and defaults:

• There are no llvm_mode and gcc_plugin subdirectories anymore and there is only one compiler: afl-cc. All previous compilers now symlink to this one compiler. All instrumentation source code is now in the instrumentation/ folder.
• The gcc_plugin was replaced with a new version submitted by AdaCore that supports more features. thank you!
• qemu_mode got upgraded to QEMU 5.1, but to be able to build this a current ninja build tool version and python3 setuptools are required. qemu_mode also got new options like snapshotting, instrumenting specific shared libraries, etc. Additionally QEMU 5.1 supports more CPU targets so this is really worth it.
• When instrumenting targets, afl-cc will not supersede optimizations. This allows to fuzz targets as same as they are built for debug or release.
• afl-fuzz:
  • if neither -M or -S is specified, -S default is assumed, so more fuzzers can easily be added later
  • -i input directory option now descends into subdirectories. It also does not fatal on crashes and too large files, instead it skips them and uses them for splicing mutations
  • -m none is now default, set memory limits (in MB) with e.g. -m 250
  • deterministic fuzzing is now disabled by default (unless using -M) and can be enabled with -D
  • a caching of testcases can now be performed and can be modified by editing config.h for TESTCASE_CACHE or by specifying the env variable AFL_TESTCACHE_SIZE (in MB). Good values are between 50-500 (default: 50).
• utils/ got renamed to utils/

⁠Contents

1. Features⁠
2. How to compile and install afl++⁠
3. How to fuzz a target⁠
4. Fuzzing binary-only targets⁠
5. Good examples and writeups of afl++ usages⁠
6. Branches⁠
7. Want to help?⁠
8. Detailed help and description of afl++⁠

⁠Important features of afl++

afl++ supports llvm up to version 12, very fast binary fuzzing with QEMU 5.1 with laf-intel and redqueen, unicorn mode, gcc plugin, full *BSD, Solaris and Android support and much, much, much more.

1. default for LLVM >= 9.0, env var for older version due an efficiency bug in llvm <= 8
2. GCC creates non-performant code, hence it is disabled in gcc_plugin
3. (currently unassigned)
4. with pcguard mode and LTO mode for LLVM >= 11
5. upcoming, development in the branch
6. not compatible with LTO instrumentation and needs at least LLVM >= 4.1
7. automatic in LTO mode with LLVM >= 11, an extra pass for all LLVM version that writes to a file to use with afl-fuzz' -x

Among others, the following features and patches have been integrated:

• NeverZero patch for afl-gcc, instrumentation, qemu_mode and unicorn_mode which prevents a wrapping map value to zero, increases coverage
• Persistent mode, deferred forkserver and in-memory fuzzing for qemu_mode
• Unicorn mode which allows fuzzing of binaries from completely different platforms (integration provided by domenukk)
• The new CmpLog instrumentation for LLVM and QEMU inspired by Redqueen⁠
• Win32 PE binary-only fuzzing with QEMU and Wine
• AFLfast's power schedules by Marcel Böhme: https://github.com/mboehme/aflfast⁠
• The MOpt mutator: https://github.com/puppet-meteor/MOpt-AFL⁠
• LLVM mode Ngram coverage by Adrian Herrera https://github.com/adrianherrera/afl-ngram-pass⁠
• LAF-Intel/CompCov support for instrumentation, qemu_mode and unicorn_mode (with enhanced capabilities)
• Radamsa and honggfuzz mutators (as custom mutators).
• QBDI mode to fuzz android native libraries via Quarkslab's QBDI⁠ framework
• Frida and ptrace mode to fuzz binary-only libraries, etc.

So all in all this is the best-of afl that is out there :-)

For new versions and additional information, check out: https://github.com/AFLplusplus/AFLplusplus⁠

To compare notes with other users or get notified about major new features, send a mail to [email protected]⁠.

See docs/QuickStartGuide.md⁠ if you don't have time to read this file - however this is not recommended!

⁠Branches

The following branches exist:

• stable/trunk⁠ : stable state of afl++ - it is synced from dev from time to time when we are satisfied with its stability
• dev⁠ : development state of afl++ - bleeding edge and you might catch a checkout which does not compile or has a bug. We only accept PRs in dev!!
• (any other) : experimental branches to work on specific features or testing new functionality or changes.

For releases, please see the Releases⁠ tab.

⁠Help wanted

We were happy to be part of Google Summer of Code 2020⁠ and we will try to participate again in 2021!

We have several ideas we would like to see in AFL++ to make it even better. However, we already work on so many things that we do not have the time for all the big ideas.

This can be your way to support and contribute to AFL++ - extend it to do something cool.

We have an idea list in docs/ideas.md⁠.

For everyone who wants to contribute (and send pull requests) please read CONTRIBUTING.md⁠ before your submit.

⁠Building and installing afl++

An easy way to install afl++ with everything compiled is available via docker: You can use the Dockerfile⁠ (which has gcc-10 and clang-11 - hence afl-clang-lto is available!) or just pull directly from the docker hub:

docker pull aflplusplus/aflplusplus
docker run -ti -v /location/of/your/target:/src aflplusplus/aflplusplus

This image is automatically generated when a push to the stable repo happens. You will find your target source code in /src in the container.

If you want to build afl++ yourself you have many options. The easiest choice is to build and install everything:

sudo apt install build-essential python3-dev automake flex bison libglib2.0-dev libpixman-1-dev clang python3-setuptools llvm
make distrib
sudo make install

It is recommended to install the newest available gcc, clang and llvm-dev possible in your distribution!

Note that "make distrib" also builds instrumentation, qemu_mode, unicorn_mode and more. If you just want plain afl++ then do "make all", however compiling and using at least instrumentation is highly recommended for much better results - hence in this case

make source-only

is what you should choose.

These build targets exist:

• all: just the main afl++ binaries
• binary-only: everything for binary-only fuzzing: qemu_mode, unicorn_mode, libdislocator, libtokencap
• source-only: everything for source code fuzzing: instrumentation, libdislocator, libtokencap
• distrib: everything (for both binary-only and source code fuzzing)
• man: creates simple man pages from the help option of the programs
• install: installs everything you have compiled with the build options above
• clean: cleans everything compiled, not downloads (unless not on a checkout)
• deepclean: cleans everything including downloads
• code-format: format the code, do this before you commit and send a PR please!
• tests: runs test cases to ensure that all features are still working as they should
• unit: perform unit tests (based on cmocka)
• help: shows these build options

Unless you are on Mac OS X⁠ you can also build statically linked versions of the afl++ binaries by passing the STATIC=1 argument to make:

make all STATIC=1

These build options exist:

• STATIC - compile AFL++ static
• ASAN_BUILD - compiles with memory sanitizer for debug purposes
• DEBUG - no optimization, -ggdb3, all warnings and -Werror
• PROFILING - compile with profiling information (gprof)
• INTROSPECTION - compile afl-fuzz with mutation introspection
• NO_PYTHON - disable python support
• NO_SPLICING - disables splicing mutation in afl-fuzz, not recommended for normal fuzzing
• AFL_NO_X86 - if compiling on non-intel/amd platforms
• LLVM_CONFIG - if your distro doesn't use the standard name for llvm-config (e.g. Debian)

e.g.: make ASAN_BUILD=1

⁠Good examples and writeups

Here are some good writeups to show how to effectively use AFL++:

• https://aflplus.plus/docs/tutorials/libxml2_tutorial/⁠
• https://bananamafia.dev/post/gb-fuzz/⁠
• https://securitylab.github.com/research/fuzzing-challenges-solutions-1⁠
• https://securitylab.github.com/research/fuzzing-software-2⁠
• https://securitylab.github.com/research/fuzzing-sockets-FTP⁠
• https://securitylab.github.com/research/fuzzing-sockets-FreeRDP⁠

If you are interested in fuzzing structured data (where you define what the structure is), these links have you covered:

• Superion for afl++: https://github.com/adrian-rt/superion-mutator⁠
• libprotobuf raw: https://github.com/bruce30262/libprotobuf-mutator_fuzzing_learning/tree/master/4_libprotobuf_aflpp_custom_mutator⁠
• libprotobuf for old afl++ API: https://github.com/thebabush/afl-libprotobuf-mutator⁠

If you find other good ones, please send them to us :-)

⁠How to fuzz with afl++

The following describes how to fuzz with a target if source code is available. If you have a binary-only target please skip to [#Instrumenting binary-only apps](#Instrumenting binary-only apps)

Fuzzing source code is a three-step process.

1. Compile the target with a special compiler that prepares the target to be fuzzed efficiently. This step is called "instrumenting a target".
2. Prepare the fuzzing by selecting and optimizing the input corpus for the target.
3. Perform the fuzzing of the target by randomly mutating input and assessing if a generated input was processed in a new path in the target binary.

⁠1. Instrumenting that target

⁠a) Selecting the best afl++ compiler for instrumenting the target

afl++ comes with a central compiler afl-cc that incorporates various different kinds of compiler targets and and instrumentation options. The following evaluation flow will help you to select the best possible.

It is highly recommended to have the newest llvm version possible installed, anything below 9 is not recommended.

+--------------------------------+
| clang/clang++ 11+ is available | --> use LTO mode (afl-clang-lto/afl-clang-lto++)
+--------------------------------+     see [instrumentation/README.lto.md](instrumentation/README.lto.md)
    |
    | if not, or if the target fails with LTO afl-clang-lto/++
    |
    v
+---------------------------------+
| clang/clang++ 3.3+ is available | --> use LLVM mode (afl-clang-fast/afl-clang-fast++)
+---------------------------------+     see [instrumentation/README.llvm.md](instrumentation/README.llvm.md)
    |
    | if not, or if the target fails with LLVM afl-clang-fast/++
    |
    v
 +--------------------------------+
 | if you want to instrument only | -> use GCC_PLUGIN mode (afl-gcc-fast/afl-g++-fast)
 | parts of the target            |    see [instrumentation/README.gcc_plugin.md](instrumentation/README.gcc_plugin.md) and
 +--------------------------------+    [instrumentation/README.instrument_list.md](instrumentation/README.instrument_list.md)
    |
    | if not, or if you do not have a gcc with plugin support
    |
    v
   use GCC mode (afl-gcc/afl-g++) (or afl-clang/afl-clang++ for clang)

Clickable README links for the chosen compiler:

• LTO mode - afl-clang-lto⁠
• LLVM mode - afl-clang-fast⁠
• GCC_PLUGIN mode - afl-gcc-fast⁠
• GCC mode (afl-gcc) has no README as it has no own features

You can select the mode for the afl-cc compiler by:

1. passing --afl-MODE command line options to the compiler via CFLAGS/CXXFLAGS/CPPFLAGS
2. use a symlink to afl-cc: afl-gcc, afl-g++, afl-clang, afl-clang++, afl-clang-fast, afl-clang-fast++, afl-clang-lto, afl-clang-lto++, afl-gcc-fast, afl-g++-fast
3. using the environment variable AFL_CC_COMPILER with MODE

MODE can be one of: LTO (afl-clang-lto*), LLVM (afl-clang-fast*), GCC_PLUGIN (afl-g*-fast) or GCC (afl-gcc/afl-g++).

Because no afl specific command-line options are accepted (beside the --afl-MODE command), the compile-time tools make fairly broad use of environment variables, which can be listed with afl-cc -hh or by reading docs/env_variables.md⁠.

⁠b) Selecting instrumentation options

The following options are available when you instrument with LTO mode (afl-clang-fast/afl-clang-lto):

• Splitting integer, string, float and switch comparisons so afl++ can easier solve these. This is an important option if you do not have a very good and large input corpus. This technique is called laf-intel or COMPCOV. To use this set the following environment variable before compiling the target: export AFL_LLVM_LAF_ALL=1 You can read more about this in instrumentation/README.laf-intel.md⁠
• A different technique (and usually a better one than laf-intel) is to instrument the target so that any compare values in the target are sent to afl++ which then tries to put these values into the fuzzing data at different locations. This technique is very fast and good - if the target does not transform input data before comparison. Therefore this technique is called input to state or redqueen. If you want to use this technique, then you have to compile the target twice, once specifically with/for this mode, and pass this binary to afl-fuzz via the -c parameter. Note that you can compile also just a cmplog binary and use that for both however there will be a performance penality. You can read more about this in instrumentation/README.cmplog.md⁠

If you use LTO, LLVM or GCC_PLUGIN mode (afl-clang-fast/afl-clang-lto/afl-gcc-fast) you have the option to selectively only instrument parts of the target that you are interested in:

• To instrument only those parts of the target that you are interested in create a file with all the filenames of the source code that should be instrumented. For afl-clang-lto and afl-gcc-fast - or afl-clang-fast if either the clang version is below 7 or the CLASSIC instrumentation is used - just put one filename or function per line (no directory information necessary for filenames9, and either set export AFL_LLVM_ALLOWLIST=allowlist.txt or export AFL_LLVM_DENYLIST=denylist.txt - depending on if you want per default to instrument unless noted (DENYLIST) or not perform instrumentation unless requested (ALLOWLIST). NOTE: During optimization functions might be inlined and then would not match! See instrumentation/README.instrument_list.md⁠ For afl-clang-fast > 6.0 or if PCGUARD instrumentation is used then use the llvm sancov allow-list feature: http://clang.llvm.org/docs/SanitizerCoverage.html⁠ The llvm sancov format works with the allowlist/denylist feature of afl++ however afl++'s format is more flexible.

There are many more options and modes available however these are most of the time less effective. See:

• instrumentation/README.ctx.md⁠
• instrumentation/README.ngram.md⁠
• instrumentation/README.instrim.md⁠

afl++ performs "never zero" counting in its bitmap. You can read more about this here:

• instrumentation/README.neverzero.md⁠

⁠c) Modify the target

If the target has features that make fuzzing more difficult, e.g. checksums, HMAC, etc. then modify the source code so that this is removed. This can even be done for operational source code by eliminating these checks within this specific defines:

#ifdef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
  // say that the checksum or HMAC was fine - or whatever is required
  // to eliminate the need for the fuzzer to guess the right checksum
  return 0;
#endif

All afl++ compilers will set this preprocessor definition automatically.

⁠d) Instrument the target

In this step the target source code is compiled so that it can be fuzzed.

Basically you have to tell the target build system that the selected afl++ compiler is used. Also - if possible - you should always configure the build system such that the target is compiled statically and not dynamically. How to do this is described below.

Then build the target. (Usually with make)

NOTE: sometimes configure and build systems are fickle and do not like stderr output (and think this means a test failure) - which is something afl++ like to do to show statistics. It is recommended to disable them via export AFL_QUIET=1.

⁠configure

For configure build systems this is usually done by: CC=afl-clang-fast CXX=afl-clang-fast++ ./configure --disable-shared

Note that if you are using the (better) afl-clang-lto compiler you also have to set AR to llvm-ar[-VERSION] and RANLIB to llvm-ranlib[-VERSION] - as is described in instrumentation/README.lto.md⁠.

⁠cmake

For cmake build systems this is usually done by: mkdir build; cmake -DCMAKE_C_COMPILERC=afl-cc -DCMAKE_CXX_COMPILER=afl-c++ ..

Note that if you are using the (better) afl-clang-lto compiler you also have to set AR to llvm-ar[-VERSION] and RANLIB to llvm-ranlib[-VERSION] - as is described in instrumentation/README.lto.md⁠.

⁠meson

For meson you have to set the afl++ compiler with the very first command! CC=afl-cc CXX=afl-c++ meson

⁠other build systems or if configure/cmake didn't work

Sometimes cmake and configure do not pick up the afl++ compiler, or the ranlib/ar that is needed - because this was just not foreseen by the developer of the target. Or they have non-standard options. Figure out if there is a non-standard way to set this, otherwise set up the build normally and edit the generated build environment afterwards manually to point it to the right compiler (and/or ranlib and ar).

⁠d) Better instrumentation

If you just fuzz a target program as-is you are wasting a great opportunity for much more fuzzing speed.

This requires the usage of afl-clang-lto or afl-clang-fast.

This is the so-called persistent mode, which is much, much faster but requires that you code a source file that is specifically calling the target functions that you want to fuzz, plus a few specific afl++ functions around it. See instrumentation/README.persistent_mode.md⁠ for details.

Basically if you do not fuzz a target in persistent mode then you are just doing it for a hobby and not professionally :-)

⁠2. Preparing the fuzzing campaign

As you fuzz the target with mutated input, having as diverse inputs for the target as possible improves the efficiency a lot.

⁠a) Collect inputs

Try to gather valid inputs for the target from wherever you can. E.g. if it is the PNG picture format try to find as many png files as possible, e.g. from reported bugs, test suites, random downloads from the internet, unit test case data - from all kind of PNG software.

If the input format is not known, you can also modify a target program to write normal data it receives and processes to a file and use these.

⁠b) Making the input corpus unique

Use the afl++ tool afl-cmin to remove inputs from the corpus that do not produce a new path in the target.

Put all files from step a) into one directory, e.g. INPUTS.

If the target program is to be called by fuzzing as bin/target -d INPUTFILE the run afl-cmin like this: afl-cmin -i INPUTS -o INPUTS_UNIQUE -- bin/target -d @@ Note that the INPUTFILE argument that the target program would read from has to be set as @@.

If the target reads from stdin instead, just omit the @@ as this is the default.

⁠c) Minimizing all corpus files

The shorter the input files that still traverse the same path within the target, the better the fuzzing will be. This is done with afl-tmin however it is a long process as this has to be done for every file:

mkdir input
cd INPUTS_UNIQUE
for i in *; do
  afl-tmin -i "$i" -o "../input/$i" -- bin/target -d @@
done

This step can also be parallelized, e.g. with parallel

⁠Done!

The INPUTS_UNIQUE/ directory from step b) - or even better the directory input/ if you minimized the corpus in step c) - is the resulting input corpus directory to be used in fuzzing! :-)

⁠3. Fuzzing the target

In this final step we fuzz the target. There are not that many important options to run the target - unless you want to use many CPU cores/threads for the fuzzing, which will make the fuzzing much more useful.

If you just use one CPU for fuzzing, then you are fuzzing just for fun and not seriously :-)

Pro tip: load the afl++ snapshot module⁠ before the start of afl-fuzz as this improves performance by a x2 speed increase (less if you use a persistent mode harness)!

⁠a) Running afl-fuzz

Before you do even a test run of afl-fuzz execute sudo afl-system-config (on the ho