Impact of Tutor Errors on Student Engagement in a Dialog Based Intelligent Tutoring System

Accurate classification of learner responses is a critical component of dialog based tutoring systems (DBT). Errors in identifying the intent and context of responses can have cascading effects on the ongoing interaction thereby affecting the learning experience and outcome. In this paper we attempt to quantify the impact of Tutor misclassifications on student behavior by analyzing differences across our hypothesized conditions namely, no-misclassification vs. misclassification using various dialog metrics. We find that not only are there significant changes in behavior across the two groups but that Tutor errors related to misunderstanding of Intent - although fewer in occurrence, appear to have a higher impact than a misclassification of a valid student answer. We also see some evidence of the effectiveness of scaffolds like FITBs in sustaining dialog thereby mitigating the effects of a Tutor error.