An entertainment robot based on head pose estimation and facial expression recognition

In this paper, we propose a novel entertainment robot system that can act/react to humans. In the entertainment robots, how to act/react is the fundamental challenge. In general, it is very difficult for robots to automatically decide their behavior that human want to. Therefore entertainment robots with operator's assistance are quite effective for designing practical system. Our proposed system is based on the computer vision technologies: head pose estimation and facial expression recognition. Our system leads to the commercially available entertainment robot to act/react to humans based on puppeteering by the operators head motion instead of autonomous behavior.