Assessing Risk Estimations for Cyber-Security Using Expert Judgment

In this paper, we show a use-case of structured expert judgment to assess the risk of a cyber-security attack. We showcase the process of elicitating unknown and uncertain values using multiple experts and combining these judgments by weighing the experts based on their performance. The performance of an expert is assessed using the information and calibration score calculated from the judgments of calibration questions. The judgments are stated with three-points estimates of minimum, most likely, and maximum value, which serve as input for the PERT probability distribution. For the use-case, the input values frequency, vulnerability, and impact were asked. The combined results are propagated along an attack path to calculate the risk of a cyber-security attack. This was done using RISKEE, a tool for assessing risk in cyber-security and implementing the combination of expert judgments and propagation of the values in an attack-tree. It uses an attack graph to model the attack paths and applies probability distributions for the input values to consider the uncertainty of predictions and expert judgments. We also describe experiences and lessons-learned for conducting an expert elicitation to acquire input values for estimating risks in cyber-security.