A Critical Interpretative Synthesis: The Integration of Automated Writing Evaluation into Classroom Writing Instruction

Abstract Automated Writing Evaluation (AWE) is computer-generated scoring and feedback that is used for both assessment and instructional purposes. Much controversy has surrounded AWE, especially in high-stakes tests such as TOEFL, and much of the discussion has centered around the scoring and feedback capabilities of AWE and the effects of AWE on text quality. Relatively little attention has been directed towards the ways that AWE is used or could be used as an instructional tool in the writing classroom. Through a critical interpretative synthesis of existing research, this study provides an overview of what is currently known about the integration of AWE into classroom writing instruction. The synthesis found that that there are numerous purposes for using AWE stated in existing research, some of which do not accord with objectives commonly associated with AWE; that teachers had varied and creative ways of integrating AWE in their classrooms; and that, although students generally seemed to enjoy using AWE, at the times when the sample studies were conducted, there appeared to be many limitations in the feedback provided by AWE systems. The study discusses these findings in terms of criticisms that have been leveled against AWE and links this discussion to broader considerations of the relationship between literacy, technology and pedagogy.