Using Digital Cameras for Text Input on Mobile Devices

This paper presents novel text input methods for mobile devices. In particular, it shows how users can capture text from books, newspapers, and other objects by using the digital cameras that are integrated into an increasing number of smartphones. The idea is to select text from a live video stream generated by these digital cameras. Different interaction patterns are discussed that allow users to select text: (1) automatic detection of the word closest to a fixed position on the mobile device's screen, (2) delayed movement of the text selection to improve the precision of the selection procedure, and (3) using the stylus to select words and word groups. The proposed interaction patterns are evaluated based on the results of a user study, and their application is shown in an example scenario. The presented research has practical relevance because it can improve the usability of mobile applications such as online searching