Stem — A Project on Software Test and Evaluation Methods

STEM is an acronym for Software Test and Evaluation Methodologies. The objective of the STEM project was to evaluate a number of fault detection, fault prediction and failure estimation methods by applying them to the documented programs produced in PODS (Project on Diverse Software, IEEE Trans. S. Eng. Sept 1986) which contained known faults. At the time this paper was written, the STEM project was still in progress, but some of the results obtained have been surprizing. For the particular programs used in this project it was found that: Of the many types of test data examined, comparison testing with uniform random data seemed to be the most effective (even when compared with specially designed tests). Non-identical bugs in diverse programs generally conformed with the independent failure assumption and around 5% were negatively correlated.