Interobserver Agreement of Lung Ultrasound Findings of COVID-19

Background: Lung ultrasound (LUS) may be used in the diagnostic evaluation of patients with COVID-19. An abnormal LUS is associated with increased risk for ICU admission in COVID-19. Previously described LUS manifestations for COVID-19 include B-lines, consolidations, and pleural thickening. The interrater reliability (IRR) of these findings for COVID-19 is unknown. Research Question: What is the interrater reliability of lung ultrasound findings in patients with RT-PCR confirmed COVID-19? Study Design and Methods: This study was conducted at conducted at two academic medical centers between 03/2020-06/2020. Nine physicians (hospitalists: n=4; emergency medicine: n=5) independently evaluated n=20 LUS scans (n=180 independent observations) collected from RT-PCR confirmed COVID-19 patients. These studies were randomly selected from an image database consisting of COVID-19 patients evaluated in the emergency department with portable ultrasound devices. Physicians were blinded to any patient information or previous LUS interpretation. Kappa values were used to calculate IRR. Results: There was substantial IRR on the following items: normal LUS scan (K=0.79 [95% CI: 0.72-0.87]), presence of B-lines (K=0.79 [95% CI: 0.72-0.87]), >=3 B-lines observed (K=0.72 [95% CI: 0.64-0.79]). Moderate IRR was observed for the presence of any consolidation (K=0.57 [95% CI: 0.50-0.64]), subpleural consolidation (K=0.49 [95% CI: 0.42-0.56]), and presence of effusion (K=0.49 [95% CI: 0.41-0.56]). Fair IRR was observed for pleural thickening (K=0.23 [95% CI: 0.15-0.30]). Interpretation: Many LUS manifestations for COVID-19 appear to have moderate to substantial IRR across providers from multiple specialties utilizing differing portable devices. The most reliable LUS findings with COVID-19 may include the presence/count of B-lines or determining if a scan is normal. Clinical protocols for LUS with COVID-19 may require additional observers for the confirmation of less reliable findings such as consolidations.