Robot Awareness of Human Behaviors and Interacting with Humans by A Two-Step Identifying Approach combining LSTM and SSD

Robot cooperating with humans is a hot research topic recently. And the first challenge is how to specifically identify humans’ intention. In this paper, we propose a two-step approach based on LSTM and SSD for identifying humans’ specific intention. A LSTM with attention mechanism is firstly presented for recognizing humans’ actions based 3D skeleton data; then, the hand-held objects are detected by a SSD neural network. Finally, a human-robot interaction framework are constructed based on the proposed intention identification model and DMP model. The experiments show that the proposed method can effectively identifying humans’ intention. And the robot accomplishes to interact with the human with the built motion database and DMP.