Interrater Variability of a Severity-of-Illness Score in Critically Ill Adults

BACKGROUND: Critically ill patients often require sedation and analgesia. Scales have been developed to provide clinicians with sedation targets. Daily interruption of continuous infusions of sedatives and sedation protocols improve patients' outcomes. However, perceived instability of a patient's condition can prevent implementation of appropriate sedation targets and daily interruption of sedation. OBJECTIVE: To evaluate the interrater variability of a severity-of-illness score developed to help nurses determine patient-specific sedation targets and identify candidates for daily interruption of sedation. METHODS: The severity-of-illness score was implemented as part of an institutional protocol, and bedside nurses in the medical intensive care unit were taught how to determine and use the score. Bedside nurses recorded the score daily in patients' medical records. For study purposes, a study nurse who made rounds with the medical team and a pulmonary/critical care fellow physician also independently determined the score. RESULTS: A total of 38 assessments of severity-of-illness scores in 10 different patients were made during the study period. For the 24 assessments made by all 3 observers, the kappa coefficient for agreement for the severity-of-illness score was 0.58. CONCLUSIONS: The severity-of-illness score had good interrater variability as a tool for determining sedation targets and identifying candidates for daily interruption of sedation. Future study on how use of the score affects sedative dosing and outcomes is needed.