Head-free, Human Gaze-driven Assistive Robotic System for Reaching and Grasping

Patients with limb dysfunction have limited mobility, which prevents them from performing daily activities. We have developed an assistive robot system with an intuitive head free gaze interface. The system consists of multiple modules, including 3D gaze estimation, head free coordinate transformation, intention recognition, and robot trajectory planning. The robotic assistive system obtains clues from the user’s gaze to decode their intentions and implement actions. This allows the user only needs to look at the objects to make the robot system reach, grasp, and bring them to the user. The 3D gaze estimation is evaluated with 5 subjects, showing an overall accuracy of 5.53±1.2 cm. The integrated system’s experimental results show that the success rate is 96% in the implementation of automatic trajectory planning, and the success rate is 92% in the implementation of fixation-based trajectory planning. Finally, the results and work required to improve the system are discussed.