Investigating research quality and the review process: some suggestions for improvement

Arguably, the most important opportunity to acquire the standards and norms of the discipline and develop researchers‟ judgement is the peer review process – but this depends on the quality of the reviews. „Good‟ feedback has been identified as being timely, specific and relevant. Yet often reviews lack these basic qualities. In this paper we report an investigation of the peer review process at a recent engineering education conference. Authors at the conference were given the chance to rate their reviews, we subsequently analysed both the nature of the reviews and authors‟ responses. We found that reviewers generally do a poor job of applying criteria, leaving some authors feeling that the review process does not offer them enough help in improving their papers. On the other hand, authors showed some of the same tendencies we see in students to take criticism personally and hence reject it. We conclude by discussing some strategies that might be implemented to help both parties.