OC-021 Changes in scoring of direct observation of procedural skills (dops) forms in endoscopy training and their impact on competence assessment

Introduction DOPS are validated tools for assessing competence in endoscopy. Previously, DOPS were scored on a 4-point competence-based scale, with scores of 3 and 4 signifying competence. In July 2016, the DOPS rating scale changed to a supervision-based scale that has been shown to be more reliable, [1] with 4 ratings from maximal supervision, up to competent without supervision. We aimed to assess whether changes to the rating scale have affected distribution of scores and hence demonstrate validity. Method Scores were collected from DOPS for gastroscopy (1934), sigmoidoscopy (517), colonoscopy (2296) and polypectomy (370) in the 6 months before July 2016 (old DOPS) and after (new DOPS), for trainees at early stages of training (total procedures Results 5117 DOPS (77.7% new and 22.3% old) were included for analysis. Overall, there were variations in distributions of scores (p Graph ). Compared to new DOPS, scores of 1 were underutilised on old DOPS (0.6% vs 3.0%, p Conclusion Endoscopy assessors are applying a greater range of scores using a new DOPS rating scale based on degree of supervision, in two cohorts of trainees matched for experience. This indicates better construct validity with the new rating scale. Further work is underway to determine the reliability of the new DOPS to inform summative assessment and certification for UK endoscopy trainees. Reference . Crossley Jet al. Med Educ2011Jun;45(6):560–9. Disclosure of Interest None Declared