Stereo-vision-based mixed reality: assistance for teleoperation

This paper deals with the use of mixed reality as a new assistance and training tool for performign teleoperation tasks in hostile environments. It describes the virtual reality techniques invovled and tackles the proble of scene registration using a man-machine cooperative and multisensory vision system. During a maintenance operation, a telerobotic task needs a perfect knowledge of the remote scne in which the robot operates. Therefore, the system provides the operator with pwoerful sensorial feedbacks as well as appropriate tools to build and update automatically the geometric model of the perceived scene. This local model is the world over which the robot is working. It also serves for mission traiing or planning an dpermits any viewpoint observation. We will describe here a new interactive approach combining image analysis and mixed reality technqiues for assisted 3D geometric semantic modling. We also tackle the problem of pose recovery and object tracking using a stereosocpic system mounted on a robot arm. The proposed model-based approach can be used for both real-time tracking and accurate static fitting of complex parametric curved objects. It therefore constitutes a unified tool for building and maintaining the local geometric model of the remote environment.