Collaborative Acceleration for Mixed Reality

A new generation of augmented reality (AR) devices, such as the Microsoft HoloLens, promises a user experience known as mixed reality (MR) that is more seamless, immersive, and intelligent than earlier AR technologies. However, this new experience comes with high computational costs, including exceptionally low latency and high quality requirements. While this cost could be offset in part through offloading, we also observe an increasing availability of on-device, task- specific accelerators. In this paper, we propose collaborative acceleration, a collaborative technique that utilizes the unique hardware accelerated capabilities of an MR device, in con- junction with an edge node, to partition an application’s core workflow according to the specific strengths of each device. To better understand the workloads of next gener- ation MR applications, we implement a concrete MR app on the HoloLens: an assistive tool to visually aid users in manipulating physical objects. Through our prototype, we find that offloading a subset of the app’s workload to an edge while also leveraging the strengths of the HoloLens delivers accurate enough results at a low latency. Our work provides an early glimpse into the system design challenges of MR, potentially the first “killer application” of edge offloading.