Improving Learning Outcomes Using an Intelligent Tutoring System

Frequent feedback synchronized with the lesson plan has been shown to improve student learning outcomes in several studies. With increasing enrolments coupled with reductions in education funding, instructors are left with fewer resources to provide frequent feedback or to develop enhanced teaching techniques. Multiple choice questions based assessments, while easier to administer, are considered less reliable than traditional free-form assessments. Intelligent tutoring systems (ITSs) based assessments can improve the speed of assessment while providing immediate feedback to reinforce lectures and free-up resources for instructors. This paper first proposes an ITS design for multi-part problems where all questions are posed at once. Next, the paper evaluates student perception of this new ITS through a survey and a focus group. Following this, the paper evaluates the effectiveness of this ITS on a test score (an indicator of learning outcomes). Finally, the paper lists lessons learned that would be useful to the education community at large. The study indicates that the ITS was received well by the students and that the time taken to complete each attempt of an ITS assignment was comparable to a paper-based assessment. Additionally, the analysis of test scores indicates that the proposed ITS can lead to improvements in student learning. It may be noted that the proposed ITS differs from commercial ITSs by posing all questions at once. However, the pedagogical advantage of offering all questions at once was not evaluated in this study. Further, the proposed ITS was not compared to other ITSs and the observed benefits may not be unique to this implementation. As such, the current findings are locally encouraging and important.