Validation of a novel assessment tool identifying proficiency in Transurethral Bladder Tumour Resection: The OSATURBS assessment tool.

BACKGROUND Competence in transurethral bladder tumour resection (TURB) is critical in bladder cancer management and should be ensured before independent practice. OBJECTIVE Develop an assessment tool for TURB and explore validity evidence in a clinical context. DESIGN, SETTING, AND PARTICIPANTS July 2019-March 2021, a total of 33 volunteer doctors from three hospitals were included. Participants performed two TURB procedures on patients with bladder tumours. A newly developed assessment tool (OSATURBS) was used for direct observation assessment, self-assessment, and blinded video-assessment. Outcome measurements and statistical analysis: Cronbach's alpha and Pearson's r were calculated for across items internal consistency reliability, inter-rater reliability, and test-retest reliability. Correlation between OSATURBS scores and the operative experience was calculated with Pearson's r and a pass/fail score was established. Differences in assessment scores were explored with paired t-test and independent samples t-test. RESULTS AND LIMITATIONS The internal consistency reliability across items Cronbach's alpha was 0.94 (n = 260, p < 0.001). Inter-rater reliability = 0.80 (n = 64, p < 0.001). Test-retest correlation was high, r = 0.71 (n = 32, p < 0.001). Relation to TURB experience was high, r = 0.71 (n = 32, p < 0.001). Pass/fail score = 19 points. Direct observation assessments were strongly correlated with video ratings (r = 0.85, p < 0.001) but with a significant social bias with lower scores for inexperienced and higher scores for experienced participants. Participants tended to overestimate their own performances. CONCLUSIONS OSATURBS assessment tool for TURB can be used for assessment of surgical proficiency in the clinical setting. Direct observation assessment and self-assessment are biased, and blinded video-assessment of TURB performances is advised.