Risk-based integration testing of software product lines

Software product lines (SPL) capture commonalities and variabilities of product families and, thus, enable mass customization of product variants according to customers desired configurations. However, they introduce new challenges to software testing due to a potentially large number of variants. While each variant should be tested, testing resources are limited and, thus, a retest of all, partially redundant, test cases for each variant is not feasible in SPL testing. Coping with these issues has been a major research focus in recent years, leading to different testing approaches. However, risk-based testing has not gained much attention in the SPL domain while being a successful approach for single-software systems. In this paper, we propose a novel risk-based testing approach for SPL integration testing. We incrementally test SPLs by stepping from one variant to the next. For each variant, we automatically compute failure probabilities and failure impacts for its architectural components. To avoid a computational overhead of generating and analyzing each variant, we exploit the variability between variants defined as deltas to focus on important changes. We evaluate our approach using an automotive case study, showing that the risk-based technique leads to positive results compared to random and delta-oriented testing.