Application of Augmented Reality based on Sensing Data to Teleoperation System for Operator Support

In this paper, we propose a new system for operator support in teleoperation using Augmented Reality(AR) based on sensing data . In the proposed system, an 360-degree camera is equipped with a mobile machine in a remote place which is controlled by an operator, and the captured 360-degree image is displayed to the head mounted display put on the head of the operator. The operator can control the machine smoothly by watching what he needs for his current task by turning his head. In addition, the provided image around the machine has no blind spot because of the 360-degree image. However, a large transfer delay or a dropped frame occurs due to the enormous data size of the 360-degree image, which adversely influences the operator performance. In order to solve this problem, the AR image is actually provided to the operator, in which CG objects generated based on other sensor information are superimposed on the 360-degree image as AR objects. In this paper, the usefulness of the proposed method is shown through the experimental results on obstacle avoidance of a mobile robot by teleoperation.