Inter-observer agreement in clinical assessment of children with acute dyspnea in the emergency department

Background: A large number of clinical dyspnea scores have been developed to reduce the substantial variation between providers in assessing dyspnea severity. However, the aspect of inter-observer agreement of these scoring systems has rarely been studied. Objective: To prospectively determine the inter-observer agreement of common clinical findings in children aged 0 to 16 years presenting with acute dyspnea to the emergency department by comparing live assessments with video assessments. Methods: Two observers examined each patient independently and assessed severity of dyspnea by scoring respiratory rate, retractions, prolonged expirium, nasal flaring, mental status and a general assessment of dyspnea on a Likert scale (1-10). After clinical examination, children were recorded on video with breath sounds recorded simultaneously. These video recordings were independently assessed by 2 pediatric pulmonologists by scoring the same items. Results: Thirty-six patients were enrolled, six of whom were analyzed separately because overall assessment of dyspnea was considered low by both observers. Inter-observer agreement for live assessments was fair to good (kappa 0.23-0.81), and agreement for video assessments was considerably worse (kappa 0.05-0.32). Conclusions: Inter-observer agreement was poor for individual clinical findings, and modest for overall assessment of dyspnea severity. Live assessments showed higher agreement than video assessments. When studying inter-observer agreement of clinical findings in children with dyspnea, live assessments are preferred over videotaped assessments. More research is needed focusing on factors which may reduce inter-observer variation.