Measuring software misinterpretation

Abstract In practice, when assessing the understandability of a software-related product, one is often concerned not only with the degree to which the information is comprehended but also with the degree to which it is misinterpreted. For example, because requirements specifications are used during the planning phase of the software life-cycle, how much the specification is misinterpreted is often as important as how much of it is comprehended. Low comprehension can be recognized before design proceeds. However, misinterpretations can easily go unnoticed until later in the life-cycle, when they are more expensive to resolve. This distinction between comprehension and misinterpretation is difficult to address experimentally because of measurement difficulties. Construction of an operational definition of the degree to which information is misinterpreted is difficult to achieve using traditional methods. In this paper a technique is described that can be used to measure both software misinterpretation and comprehension. The results of an experiment are presented in which the measurement technique was used to assess misinterpretation and comprehension of requirements specifications. The impact of specification language on misinterpretation was different from its impact on comprehension in the experiment, underscoring the value of measuring both misinterpretation and comprehension.