Metamorphic Testing of AI-based Applications: A Critical Review

Metamorphic testing is the youngest testing ap-proach among other members of the testing family. It is de-signed to test software, which are complex in nature and it is difficult to compute test oracle for them against a given set of inputs. Metamorphic testing approach tests the software with the help of metamorphic relations that guide the tester to check if the observed output can be produced after applying a certain input. Since its first appearance, a lot of research has been done to check its effectiveness on different complex families of software applications like search engines, compilers, artificial intelligence (AI) and so on. Artificial intelligence has gained immense attention due to its successfully application in many of the computer science and even other domains like medical science, social science, economic, and so on. AI-based applications are quite complex in nature as compared to other conventional software applications and because of that they are hard to test. We have selected specifically testing of AI-based applications for this research study. Although all the researchers claim to propose the best set of metamorphic relations to test AI-based applications but that still needs to be verified. In this study, we have performed a critical review supported by rigorous set of parameters that we have prepared after thorough literature survey. The survey shows that researchers have applied metamorphic testing on applications that are either based on Genetic Algorithm (GA) or Machine Learning (ML). Our analysis has helped us identifying the strengths and weaknesses of the proposed approaches. Research still needs to be done to design a generalized set of metamorphic rules that can test a family of AI applications rather than just one. The findings are supported by strong arguments and justified with logical reasoning. The identified problem domains can be targeted by the researchers in future to further enhance the capabilities of metamorphic testing and its range of applications.