Comparing the comparators: how should the quality of education offered by online universities be evaluated?

Comparing universities and courses is of interest to a variety of stakeholders including potential students, policy makers, news and media organisations, ranking providers, and universities themselves. There are a range of existing university ranking schemes that provide comparisons (e.g. Academic Ranking of World Universities) but typically these are designed with face-to-face teaching and learning in mind. There is also a growing number of quality assurance tools and approaches aimed at ensuring the quality of online education This paper reports an analysis of a range of quality assurance systems for online institutions and courses, together with variety of ranking systems targeted at conventional universities with the aim of establishing what they measure, and the metrics used to assess quality The findings from this analytical study was compared with a meta-analysis of the literature about how students choose their university in order to understand how these instruments influence current student opinion and can be used more proactively by universities themselves. The findings reveal: (1) current ranking systems are of limited value for most potential undergraduate students, particularly with reference to online education (2) comparison systems that can be of value to students from a variety of different backgrounds are likely to be complex to set up and run (3) quality indicators that promote both formative and summative evaluation may be beneficial to both institutions and students. The data that support the findings of this study are available from the corresponding author upon reasonable request. Acknowledgments This work was supported by a grant reference 2016-ES01-KA203-025182 from the EU Erasmus+ programme, 1st October 2016 - 30th September 2018.