A Home Appliance Control System with Hand Gesture based on Pose Estimation

Home appliances play a large role in our daily lives. However, with the increase of appliances, each require a dedicated remote controller, which can be cumbersome to users. Existing solutions have the shortcoming of requiring users to acquire devices that can be programmed to act as unified-remote control units. To overcome this challenge, we propose a system that utilizes hand-gestures to control various appliances. Our system provides various advantages, as users do not need to acquire dedicated devices, and controls can be carried out using intuitive hand gestures to perform various actions on the target appliance. We explain our approach and verify it by developing and integrating a system for controlling TV using various hand gestures. We conducted a performance and usability evaluations, which respectively gauged our systems accuracy under different operational and usage conditions, and investigated user impressions of our prototype. Results indicated that our system had generally high gesture-recognition accuracy, and users provided positive feedback about our system’s usability within common household usage contexts. Based on the results, we discuss various future research directions to extend our work.