AR video presentation using 3D LiDAR information for operator support in mobile robot teleoperation

In this paper, we propose a new teleoperation system of a mobile robot using Augmented Reality(AR) based on 3D LiDAR information for operator support. In the proposed system, an 360-degree camera is equipped with a robot which works in a remote place controlled by an operator, and the captured 360-degree image is displayed to the head mounted display put on the head of the operator. The operator can smoothly control the robot watching what he needs for his task by turning his head. In addition, the provided image around the robot has no blind spot because of the 360-degree image. However, the data size of the 360-degree image is enormous and it causes a large transfer delay or dropped frames which adversely influence the operator performance. In order to solve this problem, it is proposed that an AR image is actually provided to the operator, in which objects generated based on 3D LiDAR information are superimposed on the 360-degree image as AR objects. In this paper, the usefulness of the proposed method is shown through the experimental results on parking the robot remotely.