Time to "refine" clinical ward assessments? Or time to retrain?

AIM: To investigate the reliability and intra-professional variation of senior and junior doctors in the assessment of a junior doctor's clinical skills via video simulation. METHODS: Simulation video was created showing 4 clinical scenarios. This video was shown to consultants, registrars and junior doctors in various forms at Auckland City Hospital. Participants evaluated each scenario against a modified version of the current assessment form used by the Medical Council of New Zealand. RESULTS: 103 Respondents completed the survey: 22 Senior Medical Officers, 17 registrars (PGY3+), 43 junior doctors (PGY1-2) and 21 undergraduates (medical students). Statistical significance between groups was reached only for Question 6 in which Senior Medical Officers rated communication skills and respect for patients lower than postgraduate students (p=0.005). Large variability was noted in ratings for 'presentation of history' and 'clinical knowledge'. CONCLUSION: There is marked variation between Senior Medical Officers in the assessment of a junior doctor's clinical practice as demonstrated by the use of a simulation video. This variation is of potential major concern. Quality training methods of assessors may need to be implemented for standardisation of assessment if a summative component exists.