A Human-Centered Approach to Designing Gestures for Natural User Interfaces

As technology matures, human-computer interfaces have changed to meet the needs of interacting with more complex systems in user-friendly ways. Gesture-based interfaces, a type of natural user interface (NUI), allow users to use their bodies to interact with computers or virtual/augmented reality (VR/AR) and offer a more natural and intuitive user experience; however, most gesture-based commands have been developed not by considering the user, but rather what is easiest for a computer to interpret. This methodology may lead to users having increased mental effort and low perceived system usability. In two studies, we took a user-centered approach to determine natural gestures for a set of common computer actions, such as selecting an object and moving an object. In Study One, participants performed gestures for a list of actions, and in Study Two a different group of participants rated how natural they found the gestures produced in Study One compared to a set of arbitrary gestures. Overall, participants produced very similar gestures for the actions in Study One, and these gestures were rated as most natural in Study Two. Taken together, this research provides a user-centered methodology for determining natural gestures and building a vocabulary of gestures for system designers to use when developing gesture-based NUIs.