A study of the use of pairwise comparison in the context of social online moderation

Traditional moderation of student assessments is often carried out with groups of teachers working face-to-face in a specified location making judgements concerning the quality of representations of achievement. This traditional model has relied little on modern information communications technologies and has been logistically challenging. We argue that social online moderation, coupled with the use of analytical and pairwise scoring methods and technologies, can provide better moderation outcomes and highly valuable professional learning experiences improving teachers’ understandings of assessment standards. This paper reports on a component of a study involving Visual Arts teachers from rural schools making comparative judgements of digitised student artworks. We report the teachers’ observations of the social online moderation processes, including the quality and standard of the digitised artworks, the effectiveness of the pairwise comparison process, the functionality of the online tools, and the concept of using online scoring for moderation and standard setting purposes.