Fuzzing test on Windows

Enclave NO.0X7971

2024-07-29

fuzz

What is fuzz testing

Fuzz testing or fuzzing is an automated software testing method that injects invalid, malformed, unexpected or random data as an input to a system to reveal software defects and vulnerabilities.

Why fuzz testing

Fuzz testing is valuable for:

Software that receives inputs from untrusted sources (security);
Sanity checking the equivalence of two complex algorithms (correctness);
Verifying the stability of a high-volume API that takes complex inputs (stability), e.g. a decompressor, even if all the inputs are trusted.

Fuzz testing finds programming errors unrelated to the project requirements. Errors are always errors, e.g. memory leaks or buffer overflows. Developers will rarely write tests to try and catch these types of bugs, and these are exactly the types of bugs that lead to security holes and reliability problems. Fuzzing is a great and very effective way to find these bugs before they hit you in production or the bad guys exploit them.

Fuzzing is typically used to find the following kinds of bugs

Bugs specific to C/C++ that require the sanitizers to catch:
- Use-after-free, buffer overflows
- Uses of uninitialized memory
- Memory leaks
Arithmetic bugs:
Div-by-zero, int/float overflows, invalid bitwise shifts
Plain crashes:
NULL dereferences, Uncaught exceptions
Concurrency bugs:
Data races, Deadlocks
Resource usage bugs:
Memory exhaustion, hangs or infinite loops, infinite recursion (stack overflows)
Logical bugs:
- Discrepancies between two implementations of the same protocol (example)
- Round-trip consistency bugs (e.g. compress the input, decompress back, - compare with the original)
- Assertion failures

Most of these are exactly the kinds of bugs that attackers use to produce exploits, from denial-of-service through to full remote code execution.

Fuzzing tools

libFuzzer

libFuzzer is a library for coverage-guided fuzz testing. It is supported by Clang (starting from 6.0).

Steps to use libFuzzer:
- Install llvm-clang in your system.
- write an entry function to invoke the fuzz function.
- compile your code with “-fsanitize=address, -fsanitize=fuzzer” option to turn on fuzzing and ASAN.
- Run the executable file you get with the options you input.
Reference:
- libFuzzer offical documents
- libFuzzer tutorial

AFL

FuzzTest

Tool usage

libFuzzer

Write fuzzer code
refer to libFuzzer tutorial, easy and clear.
Build the fuzzer
- Under Windows:
  1
  clang -fsanitize=address -fsanitize=fuzzer -g ./source.c ./dependent.c -o ./out.exe -O2
  1. -fsanitize=address means opening the address sanitizer (In windows, only ASAN supported)
  2. -fsanitize=fuzzer means using the libfuzzer in llvm suite
  3. -g means showing the detail of the compiling process
  1
  clang -fsanitize=address -fsanitize=fuzzer -fprofile-instr-generate -fcoverage-mapping -g ./source.c ./dependent.c -o ./out.exe -O2
  1. -fporfile-instr-generate and -fcoverage-mapping is to enable the coverage while compiling the code
  2. If you want to visualize the result, you can refer to SourceBasedCodeCoverage, follow the steps they provide
- Under MacOS:
  same as on Windows
- Under Linux:
  same as on Windows

Use fuzzer program

Build a corpus:
corpus is a set of valid and invalid input files designated by users, it can help the fuzzer program generate more usable testing cases. For example, if your program is a PE file parser, the corpus can be some valid PE files and some txt files. The corpus of libfuzzer is acutally a directory with desired input inside. AND, it’s also OK if you run the fuzzer program without a corpus, but the performance of libfuzzer won’t be very beautiful.
Merge the corpora:
If you have several corpora generated from the same file, the you can run the cmd below
1
./fuzzer.exe -merge=1 ./corpus_1/ ./corpus_2/
to merge two corpora to one, in the example above, the coveraged path in ./corpus_2 will be merged to ./corpus_1.

Running the fuzzer:

multi-process fuzzing

1	./fuzzer.exe -detect_leaks=1 -jobs=4 -workers=4 -rss_limit_mb=32000 -runs=50

-detect_leaks=1, try to detect memory leaks during fuzzing
-jobs=10, number of fuzzing jobs to run to completion
-workers=4, ideally will distribute 4 cpu core to execute 4 process processing the fuzzing tasks, but acually if your cpu only has let’s say 4 cores, then the practical running processes will only be min(jobs, cpu core/2) = 2.

The relationship between job and worker:
        job No.1    __ worker NO.1
                    |_ worker NO.2
                    |_ worker NO.3
                    |_ worker No.4

        job No.2    __ worker NO.1
                    |_ worker NO.2
                    |_ worker NO.3
                    |_ worker No.4

Result analysis

libFuzzer

Sample result analysis (without error found)

INFO: Seed: 1523017872
INFO: Loaded 1 modules (16 guards): [0x744e60, 0x744ea0),
INFO: -max_len is not provided, using 64
INFO: A corpus is not provided, starting from an empty corpus
#0    READ units: 1
#1    INITED cov: 3 ft: 2 corp: 1/1b exec/s: 0 rss: 24Mb
#3811 NEW    cov: 4 ft: 3 corp: 2/2b exec/s: 0 rss: 25Mb L: 1 MS: 5 ChangeBit-ChangeByte-ChangeBit-ShuffleBytes-ChangeByte-
#3827 NEW    cov: 5 ft: 4 corp: 3/4b exec/s: 0 rss: 25Mb L: 2 MS: 1 CopyPart-
#3963 NEW    cov: 6 ft: 5 corp: 4/6b exec/s: 0 rss: 25Mb L: 2 MS: 2 ShuffleBytes-ChangeBit-
#4167 NEW    cov: 7 ft: 6 corp: 5/9b exec/s: 0 rss: 25Mb L: 3 MS: 1 InsertByte-

INFO analysis:

Seed: the currently using random seed.
-max_len: the maximum length of the random input, can be assigned by user or automatically assigned by the program.
corpus: the dictionary directory which can help the program generate more fitted input.

Formal round analysis:

#Number: Which round of fuzzing is going on.
Event code: READ, INITED, NEW … refer to output code
cov: Total number of code blocks or edges covered by executing the current corpus
ft: libFuzzer uses different signals to evaluate the code coverage: edge coverage, edge counters, value profiles, indirect caller/callee pairs, etc. These signals combined are called features (ft:)
corp: Number of entries in the current in-memory test corpus and its size in bytes
exec/s: Number of fuzzer iterations per second
rss: Current memory consumption
L: Size of the new input in bytes
MS: Count and list of the mutation operations used to generate the input

Real case result analysis (with error found)

BC277F8184738644C504ACDFF22048E0BC277F8184738644C504ACDFF22048E0B05556AC7363B8E560A317DA6654A49DBC277F8184738644C504ACDFF22048E0==15656==ERROR: AddressSanitizer failed to allocate 0x61635000 (1633898496) bytes of LargeMmapAllocator (error code: 1455)
==15656==Dumping process modules:
        0x7ff786c60000-0x7ff7877e5000 C:\Project\out.exe
        0x7ff8ba770000-0x7ff8ba782000 C:\WINDOWS\SYSTEM32\kernel.appcore.dll
        0x7ff8bd940000-0x7ff8bdc36000 C:\WINDOWS\System32\KERNELBASE.dll
        0x7ff8beb80000-0x7ff8bec1e000 C:\WINDOWS\System32\msvcrt.dll
        0x7ff8bfc90000-0x7ff8bfdb3000 C:\WINDOWS\System32\RPCRT4.dll
        0x7ff8c0150000-0x7ff8c020d000 C:\WINDOWS\System32\KERNEL32.DLL
        0x7ff8c0250000-0x7ff8c0448000 C:\WINDOWS\SYSTEM32\ntdll.dll
AddressSanitizer: CHECK failed: sanitizer_common.cpp:61 "((0 && "unable to mmap")) != (0)" (0x0, 0x0) (tid=77952)
    #0 0x7ff786c91d98 in __asan::CheckUnwind C:\src\llvm_package_18.1.8\llvm-project\compiler-rt\lib\asan\asan_rtl.cpp:69
    #1 0x7ff786c76b24 in __sanitizer::CheckFailed(char const *, int, char const *, unsigned __int64, unsigned __int64) C:\src\llvm_package_18.1.8\llvm-project\compiler-rt\lib\sanitizer_common\sanitizer_termination.cpp:86
    #2 0x7ff786c87246 in __sanitizer::ReportMmapFailureAndDie(unsigned __int64, char const *, char const *, unsigned int, bool) C:\src\llvm_package_18.1.8\llvm-project\compiler-rt\lib\sanitizer_common\sanitizer_common.cpp:61
    #3 0x7ff786c76f31 in __sanitizer::ReturnNullptrOnOOMOrDie C:\src\llvm_package_18.1.8\llvm-project\compiler-rt\lib\sanitizer_common\sanitizer_win.cpp:172
    #4 0x7ff786c76f31 in __sanitizer::MmapOrDieOnFatalError(unsigned __int64, char const *) C:\src\llvm_package_18.1.8\llvm-project\compiler-rt\lib\sanitizer_common\sanitizer_win.cpp:1e_18.1.8\llvm-project\compiler-rt\lib\sanitizer_common\sanitizer_allocator_secondary.h:97
    #6 0x7ff786caf1bb in __sanitizer::CombinedAllocator<__sanitizer::SizeClassAllocator64<__asan::AP64<__sanitizer::LocalAddressSpaceView> >,__san, struct __sanitizer::LocalAddressSpacitizer::LargeMmapAllocatorPtrArrayDynamic>::Allocate C:\src\llvm_package_18.1.8\llvm-project\compiler-rt\lib\sanitizer_common\sanitizer_allocator_sanitizer_common\sanitizer_allocator_scombined.h:71
    #7 0x7ff786caf1bb in __asan::Allocator::Allocate(unsigned __int64, unsigned __int64, struct __sanitizer::BufferedStackTrace *, enum __asan::Alitizer::LargeMmapAllocatorPtrArrayDynalocType, bool) C:\src\llvm_package_18.1.8\llvm-project\compiler-rt\lib\asan\asan_allocator.cpp:582
    #8 0x7ff786caee99 in __asan::asan_malloc(unsigned __int64, struct __sanitizer::BufferedStackTrace *) C:\src\llvm_package_18.1.8\llvm-project\clocType, bool) C:\src\llvm_package_18.ompiler-rt\lib\asan\asan_allocator.cpp:1000
    #9 0x7ff786c98553 in malloc C:\src\llvm_package_18.1.8\llvm-project\compiler-rt\lib\asan\asan_malloc_win.cpp:99                               ompiler-rt\lib\asan\asan_allocator.cpp
    #10 0x7ff786c62c77 in xxxxxx_x_x_x_x C:\xxxxx_xxxxxx.c:221      
    #11 0x7ff786c611ee in LLVMFuzzerTestOneInput C:\playxxxxxxx.c:46
    #12 0x7ff786cdc729 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const *, unsigned __int64) C:\src\llvm_package_18.1.8\llvm-project\compiler-rt\lib\fuzzer\FuzzerLoop.cpp:614
    #13 0x7ff786cdd3eb in fuzzer::Fuzzer::TryDetectingAMemoryLeak(unsigned char const *, unsigned __int64, bool) C:\src\llvm_package_18.1.8\llvm-pr-rt\lib\fuzzer\FuzzerLoop.cpp:614    roject\compiler-rt\lib\fuzzer\FuzzerLoop.cpp:690                                                                                                  roject\compiler-rt\lib\fuzzer\FuzzerLo
    #14 0x7ff786cdd717 in fuzzer::Fuzzer::MutateAndTestOne(void) C:\src\llvm_package_18.1.8\llvm-project\compiler-rt\lib\fuzzer\FuzzerLoop.cpp:762

    #15 0x7ff786cde2e5 in fuzzer::Fuzzer::Loop(class std::vector<struct fuzzer::SizedFile, class std::allocator<struct fuzzer::SizedFile>> &) C:\src\llvm_package_18.1.8\llvm-project\corc\llvm_package_18.1.8\llvm-project\compiler-rt\lib\fuzzer\FuzzerLoop.cpp:905
    #16 0x7ff786cfaffc in fuzzer::FuzzerDriver(int *, char ***, int (__cdecl *)(unsigned char const *, unsigned __int64)) C:\src\llvm_package_18.1.8\llvm-project\compiler-rt\lib\fuzzer.8\llvm-project\compiler-rt\lib\fuzzer\FuzzerDriver.cpp:914
    #17 0x7ff786cb6d62 in main C:\src\llvm_package_18.1.8\llvm-project\compiler-rt\lib\fuzzer\FuzzerMain.cpp:20
    #18 0x7ff786d07b6b in invoke_main D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:78
    #19 0x7ff786d07b6b in __scrt_common_main_seh D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288
    #20 0x7ff8c0167343  (C:\WINDOWS\System32\KERNEL32.DLL+0x180017343)
    #21 0x7ff8c029cc90  (C:\WINDOWS\SYSTEM32\ntdll.dll+0x18004cc90)

MS: 3 ChangeByte-InsertRepeatedBytes-CrossOver-; base unit: 093656ecd97bcd7d4fac99048da35ccc334c11d3
artifact_prefix='./'; Test unit written to ./crash-197275936eb03f16ab89f9eabe73b665728c3c32

Error Analysis: the mechanism of libfuzzer of finding errors is actually using address sanitizers to check if there’s any illegal memory occuring. If yes, then the libfuzzer will break and return the error message.
The way we analyze error messages are given as below:

Find out the error type, it is “AddressSanitizer failed to allocate” in this case.
Locate the error function in our testing code.
Comment the code and execute the fuzz again to see if the error happens again.
Organize your logic and see if it’s a false negative or real error.