An Exploratory Study of Higher Order Mutation Testing in Aspect-Oriented Programming

Higher order mutation testing is a relatively new area of research. Researchers have claimed that higher order mutants have the potential to increase test effectiveness and reduce test effort. However, these claims have not been evaluated in the context of the aspect-oriented programming paradigm, which introduces new and unique concepts and constructs, and consequently new testing challenges. We present an exploratory study of higher order mutation testing in the context of AspectJ, which is the most widely used aspect-oriented programming language. Based on published fault-models, we propose four approaches to constructing higher order mutants in AspectJ programs. We evaluate the approaches in terms of their ability to create higher order mutants that result in higher test effectiveness and lower test effort compared to first order mutants. The approaches (1) insert two faults in a single base class or two faults in a single aspect, (2) insert two faults in two different base classes, (3) insert two faults in two different aspects, and (4)~insert one fault in a base class and one in an aspect. We developed a prototype tool that automates the process of generating, compiling, and executing higher order mutants. The first approach produced a larger percentage of higher order mutants that were harder to kill than the constituent first order mutants as compared to the last three approaches. The first approach lowered the total number of mutants to be compiled and executed to a greater extent than the last three approaches. The last three approaches produced a lower density of equivalent mutants but the absolute number of equivalent mutants is greater for higher order mutants than for first order mutants.