Uncertainty-Wise Testing of Cyber-Physical Systems

Abstract As compared with classical software/system testing, uncertainty-wise testing explicitly addresses known uncertainty about the behavior of a System Under Test (SUT), its operating environment, and interactions between the SUT and its operational environment, across all testing phases, including test design, test generation, test optimization, and test execution, with the aim to mainly achieve the following two goals. First, uncertainty-wise testing aims to ensure that the SUT deals with known uncertainty adequately. Second, uncertainty-wise testing should be also capable of learning new (previously unknown) uncertainties such that the SUT's implementation can be improved to guard against newly learned uncertainties during its operation. The necessity to integrate uncertainty in testing is becoming imperative because of the emergence of new types of intelligent and communicating software-based systems such as Cyber-Physical Systems (CPSs). Intrinsically, such systems are exposed to uncertainty because of their interactions with highly indeterminate physical environments. In this chapter, we provide our understanding and experience of uncertainty-wise testing from the aspects of uncertainty-wise model-based testing, uncertainty-wise modeling and evolution of test ready models, and uncertainty-wise multiobjective test optimization, in the context of testing CPSs under uncertainty. Furthermore, we present our vision about this new testing paradigm and its plausible future research directions.