Runtime Prioritization with the Classification Tree Method for Test Automation

Even for automated testing, it can be necessary to prioritize test cases. One way is to order test cases by their expected runtime. However, the manual prediction and assignment of expected runtimes to test cases requires in depth knowledge of the system under test and remains time consuming and error prone. We propose an approach for automated runtime prioritization with the classification tree method. The idea is to measure the runtime of a few chosen test cases to predict the runtimes for the other test cases. This allows for better usage of limit test execution times in highly automated testing environments. We identify several challenges and propose possible solutions.