Evaluation of an intelligent team tutoring system for a collaborative two-person problem: Surveillance

Abstract This paper describes the development and evaluation of an Intelligent Team Tutoring System (ITTS) for pairs of learners working collaboratively to monitor an area. In the Surveillance Team Tutor (STT), learners performed a surveillance task in a virtual environment, communicating to track hostile moving soldiers. This collaborative problem solving task required significant communication to achieve the common goal of perfect surveillance. In a pilot evaluation, 16 two-person teams performed the task within one of three feedback conditions (Individual, Team, or None) across four trials each. The STT used a unique approach to filtering feedback so that teams in both individual and team conditions received a similar amount of feedback. In one performance measure, Team condition participants made fewer errors in one task than those in other conditions, though at a potential cost of mental workload. Feedback condition also significantly affected participants' subjective rating of both their own performance and their teammate's. This ITTS is one of the first automated team tutoring systems that provided real-time feedback during task execution. Recommendations are offered for the design of the optimal team task for future ITTSs that offer tutoring for small teams performing collaborative problem solving.