New machine-marked tests for selection into core medical training: evidence from two validation studies

This study examined whether two machine- marked tests (MMTs; a clinical problem-solving test and sit- uational judgement test), previously validated for selection into UK general practice (GP) training, could provide a valid methodology for shortlisting into core medical training (CMT). A longitudinal design was used to examine the MMTs' psychometric properties in CMT samples, and corre- lations between MMT scores and CMT interview outcomes. Independent samples from two years were used: in 2008, a retrospective analysis was conducted (n1,711), while in 2009, CMT applicants completed the MMTs for evaluation purposes (n2,265). Both MMTs showed good reliability in CMT samples, similar to GP samples. Both MMTs were good predictors of CMT interview performance (r0.56, p 0.001 in 2008; r0.61, p 0.001 in 2009) and offered incremental validity over the current shortlisting process. The GP MMTs offer an appropriate measurement method- ology for selection into CMT, representing a significant inno- vation for selection methodology.