Inter-rater reliability of the American Society of Anesthesiologists physical status rating for emergency gastrointestinal surgery

Aim The American Society of Anesthesiologists Physical Status (ASA-PS) classification system is used worldwide and has also been incorporated into various prediction rules. However, concerns have been raised regarding inter-rater agreement in various surgical fields. Although emergency gastrointestinal surgery is relatively common and associated with high postoperative mortality, a reliability study has not yet been undertaken in this field. The aim of the present study was to investigate the inter-rater reliability of ASA-PS for emergency gastrointestinal surgery. Methods Three sets of scenarios were generated for each ASA-PS class (2E, 3E, and 4E) in emergency gastrointestinal surgery, resulting in nine scenarios. These scenarios described the preoperative profiles of patients in one hospital. Two or three anesthesiologists from 18 other hospitals provided scores for ASA-PS for each scenario. Results Fifty anesthesiologists scored the ASA-PS class. Between 66% and 90% of these anesthesiologists assigned the same ratings as the reference ratings for the individual scenarios. Inter-rater reliability was assessed using Fleiss’ kappa (95% confidence interval) of 0.55 (0.54–0.56, P < 0.001) and an intraclass correlation coefficient (95% confidence interval) of 0.79 (0.63–0.93, P < 0.001). Conclusion The results of the present study revealed the consistency of ASA-PS ratings between anesthesiologists for emergency gastrointestinal surgery. The ASA-PS may serve as a reliable variable in the prediction rules for this field.