Computer adaptive assessment and its use in the development of a student model for blended learning

This paper presents an overview of our work on the development and testing of an automated feedback tool based on Computer-Adaptive Testing. Computer-adaptive tests (CATs) are software applications that adapt the presentation of test questions to the learner's proficiency level, so that those performing well are given more difficult questions and vice versa. In this paper, we present and describe the development of the models used in a feedback tool based on this approach. The model includes a proficiency level estimation based on Item Response Theory and also a questions' database. The questions in the database are classified according to topic area and difficulty level. The difficulty level is initially set by expert evaluation based upon Bloom's taxonomy and adapted according to students' performance over time. The output from our adaptive test is a continuously updated student model that estimates proficiency in each of the domain areas covered in the test, relating not only to performance, but also to cognitive ability, based on Bloom's levels. Earlier work has shown that the approach we adopt is reliable and fair to students and provides useful and important measures of ability. Potentially these measures may be used, not only in formative and summative assessment, but also to help in the delivery of learning or remedial activities based on individual ability. We describe our student model based on adaptive testing and show how it was used to provide automated feedback for students in a summative assessment context. The evaluation of our feedback tool by groups of learners and teachers suggested that our approach was a valid one, capable of providing useful advice for individual development. A survey of staff attitude supported this view. The results of these evaluations are presented in this paper. In the concluding section of the paper we suggest ways that the student profiles created by our method are likely to be useful in a variety of learning contexts.