A hybrid approach for automatic generation of named entity distractors for multiple choice questions

Assessment plays an important role in learning and Multiple Choice Questions (MCQs) are quite popular in large-scale evaluations. Technology-enabled learning necessitates a smart assessment. Therefore, automatic MCQ generation became increasingly popular in the last two decades. Despite a large amount of research effort, system generated MCQs are not useful in real educational applications. This is because of the inability to produce diverse and human-alike distractors. Distractors are the wrong choices given along with the correct answer (key) to befuddle the examinee. In several domains, the MCQs deal with names or named entities. However, existing literature is not adequate in generating quality named entity distractors. In this paper, we present a method for automatic generation of named entity distractors. The technique uses a combination of statistical and semantic similarities. To compute the statistical similarity, a set of class-specific attributes are defined are their values are extracted from the web. Semantic similarity is computed using a predicate-argument extraction based method. The proposed technique is tested in cricket domain because of the availability of a large number of web resources and MCQs for dataset preparation. An evaluation strategy is proposed along with a set of metrics. A set of human evaluators performed the evaluation and they found that the average closeness value of the distractors as 2.3. This value indicates that 2.3 out of 3 system-generated distractors are as good as human-generated distractors. Two good distractors make an MCQ usable in real assessment. So, the proposed technique is capable of generating high-quality distractors.