Virtual keyboard for head mounted display-based wearable devices
4
Citation
14
Reference
10
Related Paper
Citation Trend
Abstract:
Wearable devices eliminate the need of physically taking out a mobile device before operating on it and are emerging as the next wave of mobile systems. Head-mounted display (HMD) is a key building block of wearable devices, and offers users immediate access to relevant information in a glance. However, most existing user input mechanisms accompanying HMDs are designed for interactive information exploration rather than for extended text entry. This paper describes the design, implementation and evaluation of a text input system for HMDs called Air Typing, which requires only a standard camera and is shown to be comparable in effectiveness to single-hand text input on tablet computers in a lab setting. Air Typing features a novel two-level virtual keyword layout, which substantially improves the typing speed by cutting down unnecessary hand movements during typing and greatly simplifies the associated image processing task by doing away with fine-grained matching between fingertips and keys. The current Air Typing prototype incorporates an OpenCV-based virtual key press detection algorithm that runs on the featured two-level virtual keyboard. In our tests, an experienced user's typing speeds of one-hand text input and of two-hand text input under Air Typing are 13 and 15 words per minute (WPM), respectively.Keywords:
Virtual keyboard
Words per minute
Input device
Wearable Technology
Text entry
Optical head-mounted display
Virtual and Augmented Reality deliver engaging interaction experiences that can transport and extend the capabilities of the user. To ensure these paradigms are more broadly usable and effective, however, it is necessary to also deliver many of the conventional functions of a smartphone or personal computer. It remains unclear how conventional input tasks, such as text entry, can best be translated into virtual and augmented reality. In this paper we examine the performance potential of four alternative text entry strategies in virtual reality (VR). These four strategies are selected to provide full coverage of two fundamental design dimensions: i) physical surface association; and ii) number of engaged fingers. Specifically, we examine typing with index fingers on a surface and in mid-air and typing using all ten fingers on a surface and in mid-air. The central objective is to evaluate the human performance potential of these four typing strategies without being constrained by current tracking and statistical text decoding limitations. To this end we introduce an auto-correction simulator that uses knowledge of the stimulus to emulate statistical text decoding within constrained experimental parameters and use high-precision motion tracking hardware to visualise and detect fingertip interactions. We find that alignment of the virtual keyboard with a physical surface delivers significantly faster entry rates over a mid-air keyboard. Also, users overwhelmingly fail to effectively engage all ten fingers in mid-air typing, resulting in slower entry rates and higher error rates compared to just using two index fingers. In addition to identifying the envelopes of human performance for the four strategies investigated, we also provide a detailed analysis of the underlying features that distinguish each strategy in terms of its performance and behaviour.
Text entry
Virtual keyboard
USable
Input device
Words per minute
Cite
Citations (64)
Wearable devices eliminate the need of physically taking out a mobile device before operating on it and are emerging as the next wave of mobile systems. Head-mounted display (HMD) is a key building block of wearable devices, and offers users immediate access to relevant information in a glance. However, most existing user input mechanisms accompanying HMDs are designed for interactive information exploration rather than for extended text entry. This paper describes the design, implementation and evaluation of a text input system for HMDs called Air Typing, which requires only a standard camera and is shown to be comparable in effectiveness to single-hand text input on tablet computers in a lab setting. Air Typing features a novel two-level virtual keyword layout, which substantially improves the typing speed by cutting down unnecessary hand movements during typing and greatly simplifies the associated image processing task by doing away with fine-grained matching between fingertips and keys. The current Air Typing prototype incorporates an OpenCV-based virtual key press detection algorithm that runs on the featured two-level virtual keyboard. In our tests, an experienced user's typing speeds of one-hand text input and of two-hand text input under Air Typing are 13 and 15 words per minute (WPM), respectively.
Virtual keyboard
Words per minute
Input device
Wearable Technology
Text entry
Optical head-mounted display
Cite
Citations (4)
Information technology plays a very important role in society. People with disabilities are often limited by slow text input speed despite the use of assistive devices. This study aimed to evaluate the effect of a dynamic on-screen keyboard (Custom Virtual Keyboard) and a word-prediction system (Sibylle) on text input speed in participants with functional tetraplegia. Ten participants tested four modes at home (static on-screen keyboard with and without word prediction and dynamic on-screen keyboard with and without word prediction) for 1 mo before choosing one mode and then using it for another month. Initial mean text input speed was around 23 characters per minute with the static keyboard and 12 characters per minute with the dynamic keyboard. The results showed that the dynamic keyboard reduced text input speed by 37% compared with the standard keyboard and that the addition of word prediction had no effect on text input speed. We suggest that current forms of dynamic keyboards and word prediction may not be suitable for increasing text input speed, particularly for subjects who use pointing devices. Future studies should evaluate the optimal ergonomic design of dynamic keyboards and the number and position of words that should be predicted.
Tetraplegia
Cite
Citations (20)
Virtual keyboard
Words per minute
Input device
Display size
Text entry
Cite
Citations (15)
Until recently text entry in virtual reality has been limited to using hand-held controllers. These techniques of text entry are feasible only for entering short texts like usernames and passwords. But recent improvements in virtual reality devices have paved the way to varied interactions in virtual environment and many of these tasks include annotation, text messaging, etc. These tasks require an effective way of text entry in virtual reality. We present an interactive midair text entry system in virtual reality which allows users to use their one or both hands as the means of entering text. Our system also allows users to enter text on a split keyboard using their two hands. We investigated user performance on these three conditions and found that users were slightly faster when they were using both hands. In this case, the mean entry rate was 16.4 words-per-minute (wpm). While using one hand, the entry rate was 16.1 wpm and using the split keyboard the entry rate was 14.7 wpm. The character error rates (CER) in these conditions were 0.74%, 0.79% and 1.41% respectively.
Text entry
Virtual keyboard
Words per minute
Optical head-mounted display
Word error rate
Cite
Citations (1)
We examined two vision-based interfaces (VBIs) for performance and user experience during character-based text entry using an on-screen virtual keyboard. Head-based VBI uses head motion to steer the computer pointer and mouth-opening gestures to select the keyboard keys. Gaze-based VBI utilizes gaze for pointing at the keys and an adjustable dwell for key selection. The results showed that after three sessions (45 min of typing in total), able-bodied novice participants (N = 34) typed significantly slower yet yielded significantly more accurate text with head-based VBI with gaze-based VBIs. The analysis of errors and corrective actions relative to the spatial layout of the keyboard revealed a difference in the error correction behavior of the participants when typing using both interfaces. We estimated the error correction cost for both interfaces and suggested implications for the future use and improvement of VBIs for hands-free text entry.
Text entry
Virtual keyboard
Words per minute
Input device
Fitts's law
Cite
Citations (2)
Integrated hand-tracking on modern virtual reality (VR) headsets can be readily exploited to deliver mid-air virtual input surfaces for text entry. These virtual input surfaces can closely replicate the experience of typing on a Qwerty keyboard on a physical touchscreen, thereby allowing users to leverage their pre-existing typing skills. However, the lack of passive haptic feedback, unconstrained user motion, and potential tracking inaccuracies or observability issues encountered in this interaction setting typically degrades the accuracy of user articulations. We present a comprehensive exploration of error-tolerant probabilistic hand-based input methods to support effective text input on a mid-air virtual Qwerty keyboard. Over three user studies we examine the performance potential of hand-based text input under both gesture and touch typing paradigms. We demonstrate typical entry rates in the range of 20 to 30 wpm and average peak entry rates of 40 to 45 wpm.
Touchscreen
Text entry
Virtual keyboard
Leverage (statistics)
Words per minute
Observability
Input device
Cite
Citations (9)
Text entry is a frequently used task in virtual reality (VR) applications, and controller is the most common interactive device in current VR systems. However, in terms of typing speed, there is still a gap between the existing controller-based text entry techniques and using a physical keyboard in reality, so it is important to improve the efficiency of the controller-based text entry. In this paper, we introduce Flower Text Entry, a single-controller text entry method based on a newly designed flower-shaped keyboard using hand 3D translation interaction for letters selection. We conduct user studies to optimize the keyboard design and the mapping between the interaction and selection, so as to evaluate our method. The results show that our method has high typing speed, lower error rate, and is very friendly to novices compared with the state-of-the-art controller-based text entry methods. After a short training, the novice group can type at 17.65 words per minute (WPM), and the potential expert group can type at 22.97 WPM. The highest typing speed is up to 30.80 WPM achieved by a potential expert participant.
Words per minute
Text entry
Virtual keyboard
Cite
Citations (10)
Text input is essential in a variety of uses in virtual and augmented reality (VR and AR). We present HMK: an effective text input method that mounts split keyboards on the left and right side of the head mounted display (HMD). Users who can touch-type are able to type using HMK by relying on their familiarity with the normal QWERTY keyboard. We develop custom keycaps to make it easier to find the home position. A study with three participants shows that users retain most of their normal keyboard typing skills. The participants achieved, on average, 34.7 words per minute (WPM) by the end of three days of use, retaining 81 percent of their regular entry speed.
Text entry
Optical head-mounted display
Words per minute
Virtual keyboard
Input device
Cite
Citations (11)
Cursor (databases)
Text entry
Virtual keyboard
Words per minute
Word error rate
Input device
Cite
Citations (11)