Rover maneuvering for autonomous vision-based dexterous manipulation

Manipulators mounted on-board rovers have limited dexterity due to power and weight constraints imposed by rover designs. However, to perform science operations, it is necessary to be able to position and orient these manipulators on science targets in order to carry out in-situ measurements. This article describes how we enhance manipulator dexterity using the rover mobility system. The lack of omni-directional driving capability and the constraints imposed by the mobility mechanism requires vehicle maneuvering to supplement the manipulators' motions. Target tracking using stereo vision is integrated with rover maneuvering to perform two types of operations: rock sample acquisition for return to earth and instrument placement for in-situ science measurements. We describe the computational architecture, tools, and algorithms that we developed for this task. We have successfully demonstrated these operations on a self-contained Mars Rover prototype, Rocky 7. We have demonstrated grasping a small rock sample from a distance of more than one meter away and placing an instrument on a boulder from a distance of more than five meters away.