Assessment of the Non-Technical Skills for Surgeons (NOTSS) framework in the USA.

BACKGROUND: Surgeons' non-technical skills are important for patient safety. The Non-Technical Skills for Surgeons assessment tool was developed in the UK and recently adapted to the US surgical context (NOTSS-US). The aim of this study was to evaluate the reliability and distribution of non-technical skill ratings given by attending (consultant) surgeons who underwent brief online training. METHODS: Attending surgeons across six specialties at a large US academic medical centre underwent a 10-min online training, then rated 60-s standardized videos of simulated operations. Intraclass correlation coefficient (ICC), and mean(s.d.) values for NOTSS-US ratings were determined for each non-technical skill category (score range 1-5, where 1 indicates poor, 3 average and 5 excellent) and for total NOTSS-US score (range 4-20; sum of 4 category scores). Outcomes were adjusted for rater characteristics including sex, specialty and clinical experience. RESULTS: A total of 8889 ratings were submitted by 81 surgeon raters on 30 simulated intraoperative videos. The mean(s.d.) total NOTSS-US score for all videos was 9.5(4.8) of 20. The within-video ICC for total NOTSS-US score was 0.64 (95 per cent c.i. 0.57 to 0.70). For individual non-technical skill categories, the ICC was highest for social skills (communication/teamwork: 0.63, 95 per cent c.i. 0.56 to 0.71; leadership: 0.64, 0.55 to 0.72) and lowest for cognitive skills (situation awareness: 0.54, 0.45 to 0.62; decision-making: 0.50, 0.41 to 0.59). Women gave higher total NOTSS-US scores than men (adjusted mean difference 0.93, 95 per cent c.i. 0.44 to 1.43; P = 0.001). CONCLUSION: After brief online training, the inter-rater reliability of the NOTSS-US assessment tool achieved moderate strength among trained surgeons rating simulated intraoperative videos.