Helping People on the Fly: Ad Hoc Teamwork for Human-Robot Teams.

We present the Bayesian Online Prediction for Ad hoc teamwork (BOPA), a novel algorithm for ad hoc teamwork which enables a robot to collaborate, on the fly, with human teammates without any pre-coordination protocol. Unlike previous works, BOPA relies only on state observations/transitions of the environment in order to identify the task being performed by a given teammate (without observing the teammate’s actions and environment’s reward signals). We evaluate BOPA in two distinct settings, namely (i) an empirical evaluation in a simulated environment with three different types of teammates, and (ii) an experimental evaluation in a real-world environment, deploying BOPA into an ad hoc robot with the goal of assisting a human teammate in completing a given task. Our results show that BOPA is effective at correctly identifying the target task, efficient at solving the correct task in optimal and near-optimal times, scalable by adapting to different problem sizes, and robust to non-optimal teammates, such as humans.