Reliability of conformance tests

Conformance testing is considered from a statistical point of view. An s-confidence interval is found for the reliability that an implementation of a software package complies with specifications of a standard. Determination of whether it complies depends on a conformance test, which is written directly from the standard. Although the conformance test is written directly from the standard it does not test all possible software parameter-settings that invoke the standard. Thus, statistical inference is necessary. A general s-confidence interval for the reliability is given when the specification requires that the implementation passes all the tests in the conformance test suite. The conformance test is made of disjoint homogeneous partitions. The failure probability of the software is based on a weighted linear combination of the partition failure probabilities. An example is included.