Real-time pose invariant spontaneous smile detection using conditional random regression forests

Abstract Detecting spontaneous smile in unconstrained environment is a challenging problem mainly due to the large intra-class variations caused by head poses. This paper presents a real-time smile detection method based on conditional random regression forests. Since the relation between image patches and smile intensity is modelled conditional to head pose, the proposed smile detection method is not sensitive to head poses. To achieve high smile detection performance, techniques including regression forest, multiple-label dataset augmentation and non-informative patch removement are employed. Experimental results show that the proposed method achieves competitive performance to state-of-the-art deep neural network based methods on two challenging real-world datasets, although using hand-crafted features. A dynamical forest ensemble scheme is also presented to make a trade-off between smile detection performance and processing speed. In contrast to deep neural networks, the proposed method can run in real-time on general hardware without GPU.