Integrating AI Planning for Telepresence with Time Delays

Teleoperation of remote robotic systems over time delays in the range of 2-10 seconds poses a unique set of challenges. In the context of a supervisory control system for the JSC Robonaut humanoid robot, we have developed an “intelligent assistant”, called the Task Level Assistant (TLA), that integrates an AI planner (JSHOP2) with execution monitoring of the state of both the human supervisor and the remote robot. The TLA reasons simultaneously about the world state on both sides of the time delay, which represents a novel application of this technology. The purpose of the assistant is to provide near real-time advice to the human supervisor about current and future activities, derived from a sequence of high-level goals to be achieved. To do this, the assistant must simultaneously monitor and react to various data sources, including actions taken by the supervisor who is issuing commands to the robot (e.g. with a data glove), actions taken by the robot as reported over the time delay, the environment of the robot, as currently perceived over the time delay, and the current sequence of goals. We have developed a “leader/follower” software architecture to handle the dual time-shifted streams of feedback from executing tasks. In this paper we describe the integrated planner and its executive, and how it operates in normal and anomaly situations.