Planning Everyday Manipulation Tasks – Prediction-based Transformation of Structured Activity Descriptions

The field of autonomous robot manipulation experiences tremendous progress: the cost of robot platforms is decreasing substantially, sensor technology and perceptual capabilities are advancing rapidly, and we see an increasing sophistication of control mechanisms for manipulators. Researchers have also recently implemented robots that autonomously perform challenging manipulation tasks, such as making pancakes, folding clothes, baking cookies, and cutting salad. These developments lead us to the next big challenge: the investigation of control systems for robotic agents, such as robot co-workers and assistants that are capable of mastering human-scale everyday manipulation tasks. Robots mastering everyday manipulation tasks will have to perform tasks as general as “clean up”, “set the table”, and “put the bottle away/on the table”. Although such tasks are vaguely formulated the persons stating them have detailed expectations of how the robot should perform them. We believe that an essential planning capability of robotic agents mastering everyday activity will be their capability to reason about and predictively transform incomplete and ambiguous descriptions of various aspects of manipulation activities: the objects to be manipulated, the tools to be used, the locations where objects can be manipulated from, the motions and the grasps to be performed, etc. Vague descriptions of tasks and activities are not only a key challenge for robot planning but also an opportunity for more flexibility, robustness, generality, and robustness of robot control systems.