Software testability and its application to avionic software

Randomly generated black-box testing is an established yet controversial method of estimating software reliability. Unfortunately, as software applications have required higher reliabilities, practical difficulties with black-box testing have become increasingly problematic. These practical problems are particularly acute in life-critical avionics software, where requirements of failures per hour of system reliability can translate into a probability of failure (pof) of perhaps lo-' or less for each individual execution of the software. This paper describes the application of one type of testability analysis called "sensitivity analysis" to B-737 avionics software; one application of sensitivity analysis is to quantify whether software testing is capable of detecting faults in a particular program and thus whether we can be confident that a tested program is not hiding faults. We do so by finding the testabilities of the individual statements of the program, and then use those statement testabilities to find the testabilities of the functions and modules. For the B-737 system 'we analyzed, we were able to isolate those functions that are more prone to hide errors during system/reliability testing. have begun to build orders-of-magnitude more complex systems while our testing technologies are no more advanced. Thus the same problems that we had in past years when testing a 1000 line program are compounded when we apply those techniques to a 10M line program today. In short, we quickly concede that we are building software systems that are destined to be inadequately tested. Since we know this a pn'ori, it suggests to us that we need to look for techniques that aid the testing process where the process is known to be weak. In this paper, we discuss one such technique: a method that quantifies the dynamic testability of a system that is undergoing system/reliability testing. Software testing is performed for generally two reasons: (1) detect faults so that they can be fixed, and (2) reliability estimation. The goal of our research is to strengthen the software testing process as it applies to reliability estimation. Our research is less concerned with fault detection, even though it is true that a function that is more likely to hide faults during system testing may also be more likely to hide faults during unit testing. Also, we assume that the software is close-to-a-correct-version before our dynamic analysis would be applied.