A Monte-Carlo Approach to the Value of Information in Crowdsourcing Quality Control Tasks

In the process of decision-making, the purpose of computing value of information (VOI) is to guide information collection process under uncertain environment, improve the quality of decision-making, and ultimately achieve the optimal decision. In the field of artificial intelligence, MDP is a basic theoretical model for modeling and planning decision problems, and also a major research area of sequential decision-making. In this paper, we presents a novel framework at a specific type of optimal uncertain sequential decision problems that need achieve the best trade-off between decision qualities and cost. We apply it to quality control in crowdsourcing task. Because of the combinatorial challenge of the state space when calculating the optimal policy of the general Markov decision model, this paper considers a more efficient approximation method: A Monte-Carlo Tree method computing the value of information (BMCT) based on belief states.