Estimation of Heartbeat Peak Locations and Heartbeat Rate from Facial Video

Available systems for heartbeat signal estimations from facial video only provide an average of Heartbeat Rate (HR) over a period of time. However, physicians require Heartbeat Peak Locations (HPL) to assess a patient’s heart condition by detecting cardiac events and measuring different physiological parameters including HR and its variability. This paper proposes a new method of HPL estimation from facial video using Empirical Mode Decomposition (EMD), which provides clearly visible heartbeat peaks in a decomposed signal. The method also provides the notion of both color- and motion-based HR estimation by using HPLs. Moreover, it introduces a decision level fusion of color and motion information for better accuracy of multi-modal HR estimation. We have reported our results on the publicly available challenging database MAHNOB-HCI to demonstrate the success of our system in estimating HPL and HR from facial videos, even when there are voluntary internal and external head motions in the videos. The employed signal processing technique has resulted in a system that could significantly advance, among others, health-monitoring technologies.