FaceDrive: Facial Expression Driven Operation to Control Virtual Supernumerary Robotic Arms

Supernumerary Robotic Arms (SRAs) can make physical activities easier, but require cooperation with the operator. To improve cooperation, we predict the operator’s intentions by using his/her Facial Expressions (FEs). We propose to map FEs to SRAs commands (e.g. grab, release). To measure FEs, we used a optical sensor-based approach (here inside a HMD), the sensors data are fed to a SVM classifying them in FEs. The SRAs can then carry out commands by predicting the operator’s FEs (and arguably, the operator’s intention). We made a Virtual reality Environment (VE) with SRAs and synchronizable avatar to investigate the most suitable mapping between FEs and SRAs. In SIGGRAPH Asia 2019, the user can manipulate virtual SRAs using his/her FEs.