End-to-End Hand Rehabilitation System with Single-Shot Gesture Classification for Stroke Patients

Rehabilitation of the hands is crucial for stroke survivors to regain their ability to perform activities of daily living. Various technologies were explored and found unmatured, expensive and uncomfortable. Existing devices to assist rehabilitation are typically costly, bulky and difficult to set up. Our proposed solution aims to provide an end-to-end hand rehabilitation system that can be produced at low cost with greater ease of use. It incorporates gamification to motivate stroke survivors to perform physical rehabilitation through an infra-red depth camera and computer system. MediaPipe was employed for hand detection and hand landmark extraction. A single-shot neural network model is proposed for hand gesture detection with an accuracy rate of 98%. Lastly, a visually interactive game was developed to promote engagement of the user during the performance of rehabilitation.