Testing Best Practices to Reduce the Overconfidence Bias in Multi-criteria Decision Analysis

This paper explores the effectiveness of several methods to reduce the overconfidence bias when eliciting continuous probability distributions in the context of multicriteria decision analysis. We examine the effectiveness of using a fixed value method (as opposed to the standard fixed probability method) and the use of counterfactuals and hypothetical bets to increase the range of the distributions and to correct possible median displacements. The results show that the betting procedure to correct the median is quite effective, but the methods to increase the range of estimates have only a have small, but positive effect.