Analysing reliability of surgical planning and navigation systems

Abstract To avoid human errors in complex surgical work systems, it is mandatory to assess usability and reliability of new planning and navigation systems prior to its introduction into clinical routine. Clinical usability analysis includes guideline and user-based evaluation. In a user-based study, interaction of 25 expert and novice surgeons with a commercial CAS system was observed and assessed based on a FMEA under consideration of human errors. In total 152 incidents were observed. Ninety-nine (65%) were rated to be critical for the process as such, whereas 46 (30%) were found to be critical for the patient. Comparing interaction times of novices and expert surgeons have been the basis for an assessment of systems learnability. For novices preoperative planning revealed to be more difficult to learn than the intraoperative navigation. Moreover, a multidimensional questionnaire was used to document subjective user assessment of systems error tolerance, learnability as well as user satisfaction. Although novices obviously encountered severe problems during the tests, they rated learnability significantly better than experts. In contrast, experts' rating matched the findings of the test observers.