Dynamic test prioritization of product lines: An application on configurable simulation models

Product line testing is challenging due to the potentially huge number of configurations. Several approaches have tackled this challenge; most of them focused on reducing the number of tested products by selecting a representative subset. However, little attention has been paid to product line test optimization using test results, while tests are executed. This paper aims at optimizing the testing process of product lines by increasing the fault detection rate. To this end we propose a dynamic test prioritization approach. In contrast to traditional static test prioritization, our dynamic test prioritization leverages information of tests being executed in specific products. Processing this information, the initially prioritized tests are rearranged in order to find non-discovered faults. The proposed approach is valid for any kind of product lines, but we have adapted it to the context of configurable simulation models, an area where testing is especially time-consuming and optimization methods are paramount. The approach was empirically evaluated by employing two case studies. The results of this evaluation reveal that the proposed test prioritization approach improves both the static prioritization algorithm and the selected baseline technique. The results provide a basis for suggesting that the proposed dynamic test prioritization approach is appropriate to optimize the testing process of product lines.