Learning motor skills with non-rigid materials by reinforcement learning

This paper focuses on learning motor skills for anthropomorphic robots which must interact with non-rigid materials to perform tasks, such as wearing clothes, turning socks inside out, and bandaging. To learn such a motor skill, the task to be performed needs to be quantitatively defined using not only the state of the robot, but also the state of the non-rigid material. However, the non-rigid material is generally represented in a high dimensional configuration space (e.g., [1]) and obtaining such information in a real environment is difficult. In this paper we propose a novel learning framework for learning motor skills interacting with non-rigid materials by reinforcement learning that avoids these difficulties. Our learning framework focuses on the topological relationship between the configuration of the robot and the non-rigid material based on the consideration that most details of the material (e.g., wrinkles) are not important for performing the motor tasks. This focus allows us to define the task performance and provide reward signals based on a low-dimensional variable and to measure task performance in a real environment using reliable sensors. We constructed an experimental setting with an anthropomorphic dual-arm robot and a tailor-made T-shirt for the robot. To demonstrate the feasibility of the proposed method, we applied the method to have the robot perform the motor task of putting on the T-shirt. As a result of our learning framework, through trial and error the robot was able to acquire sequential movements that performed the goal of putting both arms into the corresponding sleeves of the T-shirt.