Improving Question Generation with the Teacher’s Implicit Feedback

Although current Question Generation systems can be used to automatically generate questions for students’ assessments, these need validation and, often, manual corrections. However, this information is never used to improve the performance of QG systems, where it can play an important role. In this work, we present a system, GEN, that learns from such (implicit) feedback in a online learning setting. Following an example-based approach, it takes as input a small set of sentence/question pairs and creates patterns which are then applied to learning materials. Each generated question, after being corrected by the teacher, is used as a new seed in the next iteration, so more patterns are created each time. We also take advantage of the corrections made by the teacher to score the patterns and therefore rank the generated questions. We measure the teacher’s effort in post-editing required and show that GEN improves over time, reducing from 70% to 30% in average corrections needed per question.