On the Exploratory Testing of Mobile Apps

While the literature acknowledges that mobile apps present different testing challenges and automated solutions have been pursued, it lacks a better understanding of how pervasive practices of manual testing (namely Exploratory Testing - ET) can be more effectively applied. This paper aims to investigate the use of ET in mobile apps. With this study, we intend to have a better understanding of how exploratory testing is employed, its effectiveness, and its usage in an ample and diverse range of apps. To do so, we conducted two studies. The first study was conducted for the purpose of applying ET to apps with diverse contexts and available on Google Play in order to analyze whether testers actually explore all possible scenarios that apps may display. The second study, also applied the ET, however in two apps that were developed by a software development company; this study has the objective of applying the ET in order to identify bugs of different levels, that often cannot be revealed using other techniques. As expected the first study revealed that there are several test scenarios that are not exploited by the testers, yet the 40 participants revealed on average 5 bugs in 1.5h of test sessions. The second study revealed 64 bugs and 21 issues in two apps. Such revealed bugs are of different criticality and category. ET has shown to be a promising technique to uncover bugs, though test professionals can be better guided to explore their apps and search for bugs in scenarios related to mobile specific events.