Development of a surgical trainer assessment questionnaire

BACKGROUND: The assessment of surgical trainees has become well-established in practice; however, this is not yet the case for surgical trainers. The aim of this study was to develop a trainer assessment questionnaire (OxTrAQ) based on the 12 key trainer attributes identified by a previous systematic review, to then explore this questionnaire's potential domains and test its reliability. METHODS: The OxTrAQ was piloted at two surgical training centres in two surgical sub-specialities. At each centre, every trainee assessed every trainer while under their clinical supervision. RESULTS: Internal consistency (Cronbach's alpha) was calculated for this 12-item questionnaire as 0.958. Factor analysis revealed two domains which accounted for 77.997% of the variance; these were the general domain (nine items) and the independence domain (three items). Inter-observer reliability was shown by the intraclass correlation coefficients (ICCs) (average measures) for the overall OxTrAQ score, general domain score and independence domain score which were 0.951, 0.945 and 0.955 for centre 1 and 0.857, 0.819 and 0.883 for centre 2, respectively. Intra-observer reliability was shown by the ICCs (average measures) for the overall OxTrAQ score, general domain score and independence domain score which were 0.925, 0.913 and 0.940, respectively. Construct validity was demonstrated by a high correlation (Spearman rho = 0.886, P = 0.019) between trainer scores and the scores predicted by the trainers themselves. CONCLUSIONS: The OxTrAQ is a simple trainer assessment questionnaire which has demonstrated both high inter- and intra-observer reliability.