Deep Learning Method for Video-Based Data to Classify Peripheral Edema Grades

Introduction Peripheral edema is a common precursor to decompensation for patients with heart failure (HF), and commonly measured with the pitting test. Video recordings of the pitting peripheral edema test can capture details of skin topography and time-evolution of tissue recovery with much greater precision than a human eye. Video-based deep learning can potentially overcome the subjectivity of the peripheral edema test and assist patients in identifying objective changes in their peripheral edema. We have previously shown that our deep learning model can accurately estimate peripheral edema grade with 98% accuracy using physical edema models. The objective of this study was to evaluate feasibility of collecting videos of peripheral edema from hospitalized patients with heart failure who have various skin colors and features for the purpose of refining the deep learning model for automated edema classification. Methods We enrolled 18 patients from February-April, 2019 from cardiac medical, surgical, and intensive care units at an urban university medical center. Patients were first screened for peripheral edema through the inpatient electronic medical record system. After meeting enrollment criteria and signing informed consent, trained personnel recorded 10 videos of the pitting test on patients with peripheral edema each day of their hospitalization using a Samsung Galaxy S9 phone. The personnel used one index finger to make a circular pit on the left and right side of three bony prominences (foot, ankle, and shin). Videos were taken freehand, under diffuse indoor light conditions and were 5 seconds to 2 minutes long, depending on the time it took for the skin/tissue to fully rebound from the pitting test. Each video was labeled independently with the edema grade by a research assistant and nurse, and downloaded from a secure endpoint (phone) to a central database for analysis. Results We collected 376 videos from 18 patients over an average of three days. Overall, the sample of participants was 16% female, 33% Black, 11% Hispanic and had grades of edema that ranged from grade 0 (12%), grade 1 (51%), grade 2 (18%), grade 3 (9%), and grade 4 (10%). The videos contained a wide variability of diverse skin tones and features to ensure that the method is robust and generalizable to a multiracial patient population (Figure 1). Discussion We established the feasibility of collecting videos of peripheral edema from hospitalized patients with HF. Deep learning holds promise for the automatic classification and objective measurement of clinical signs captured with video data.