Task-based behavior generalization via manifold clustering

Machine learning algorithms can be expensive to deploy, in particular, those used in robotics applications that perform many variations of the same task. Solutions to one variation of a task may be found via Reinforcement Learning algorithms, and are typically modeled as a vector of N parameters encoding the robot's behavior policy. When N is large or executing robot trials is time-consuming, searching in the space of solutions becomes prohibitively expensive. In this paper, we introduce a method that allows robots to generalize behaviors by analyzing solutions to a small number of previously-trained related tasks. This allows for approximate policies for novel tasks to be rapidly estimated. We present a method that achieves this type of generalization by performing nonlinear regression directly on the policy manifold — i.e., the solution space spanned as we change the parameters describing tasks. Because tasks are typically described by few parameters, the corresponding policy manifold has few degrees of freedom, which leads to low-dimensional surfaces. We exploit this property to construct a function that maps task parameters to policy parameters (a parameterized skill). Our method uses manifold clustering techniques to deal with discontinuous manifolds, a challenging situation arising from physical obstacles or robot constraints. We evaluate our method on a set of robot manipulation tasks and show that it can efficiently estimate policies for novel tasks from a small number of training examples.