Fuzz testing

Fuzzing or fuzz testing is an automated software testing technique that involves providing invalid, unexpected, or random data as inputs to a computer program. The program is then monitored for exceptions such as crashes, failing built-in code assertions, or potential memory leaks. Typically, fuzzers are used to test programs that take structured inputs. This structure is specified, e.g., in a file format or protocol and distinguishes valid from invalid input. An effective fuzzer generates semi-valid inputs that are 'valid enough' in that they are not directly rejected by the parser, but do create unexpected behaviors deeper in the program and are 'invalid enough' to expose corner cases that have not been properly dealt with. Fuzzing or fuzz testing is an automated software testing technique that involves providing invalid, unexpected, or random data as inputs to a computer program. The program is then monitored for exceptions such as crashes, failing built-in code assertions, or potential memory leaks. Typically, fuzzers are used to test programs that take structured inputs. This structure is specified, e.g., in a file format or protocol and distinguishes valid from invalid input. An effective fuzzer generates semi-valid inputs that are 'valid enough' in that they are not directly rejected by the parser, but do create unexpected behaviors deeper in the program and are 'invalid enough' to expose corner cases that have not been properly dealt with. For the purpose of security, input that crosses a trust boundary is often the most interesting. For example, it is more important to fuzz code that handles the upload of a file by any user than it is to fuzz the code that parses a configuration file that is accessible only to a privileged user. Testing programs with random inputs dates back to the 1950s when data was still stored on punched cards. Programmers would use punched cards that were pulled from the trash or card decks of random numbers as input to computer programs. If an execution revealed undesired behavior, a bug had been detected and was fixed. The execution of random inputs is also called random testing or monkey testing. In 1981, Duran and Ntafos formally investigated the effectiveness of testing a program with random inputs. While random testing had been widely perceived to be the worst means of testing a program, the authors could show that it is a cost-effective alternative to more systematic testing techniques. In 1983, Steve Capps developed 'The Monkey', a tool that would generate random inputs for classic Mac OS applications, such as MacPaint. The figurative 'monkey' refers to the infinite monkey theorem which states that a monkey hitting keys at random on a typewriter keyboard for an infinite amount of time will eventually type out the entire works of Shakespeare. In the case of testing, the monkey would write the particular sequence of inputs that will trigger a crash. The term 'fuzzing' originates from a 1988 class project, taught by Barton Miller at the University of Wisconsin. To fuzz test a Unix utility meant to automatically generate random files and command-line parameters for the utility. The project was designed to test the reliability of Unix programs by executing a large number of random inputs in quick succession until they crashed. It also provided early debugging tools to determine the cause and category of each detected failure. To allow other researchers to conduct similar experiments with other software, the source code of the tools, the test procedures, and the raw result data were made publicly available. Later, the term fuzzing was not limited only to command-line utilities. In 1991, the crashme tool was released, which was intended to test the robustness of Unix and Unix-like operating systems by executing random machine instructions. In 1995, a fuzzer was used to test GUI-based tools (such as the X Window System), network protocols, and system library APIs.