FootUI: Assisting People with Upper Body Motor Impairments to Use Smartphones with Foot Gestures on the Bed

Some people with upper body motor impairments but sound lower limbs usually use feet to interact with smartphones. However, touching the touchscreen with big toes is tiring, inefficient and easy to mistouch. In this paper, we propose FootUI, which leverages the phone camera to track users’ feet and translates the foot gestures to smartphone operations. This technique enables users to interact with smartphones while reclining on the bed and improves the comfort of users. We explore the usage scenario and foot gestures, define the mapping from foot gestures to smartphone operations and develop the prototype on smartphones, which includes the gesture tracking and recognition algorithm, and the user interface. Evaluation results show that FootUI is easy, efficient and interesting to use. Our work provides a novel input technique for people with upper body motor impairments but sound lower limbs.