KnowWhereTM: An Audio/Spatial Interface for Blind People

The KnowWhereTM System was developed to present geographic information to blind people. The user's hand rests upon an illuminated surface which has been covered with a tactile grid and is viewed by a ceiling-mounted video camera. The video image is analyzed by specialized processors and the location of the user's fingertip on the light table is determined. An invisible virtual map is defined on the desk surface and the feature that the user is currently pointing to is signalled by a sound which serves to identify the feature or kind of feature that has been "touched." BACKGROUND The KnowWhereTM system grows out of a long abandoned dissertation project implemented in 1969.[1][2][3] That project, called Z-Meister, was intended to teach assembly language programming. It was built around an interpreter that was instrumented so that every program was accompanied by graphics and by auditory patterns generated on a Moog synthesizer, thus giving the user a sense of what their program was doing as it ran. Since then, we have used sounds as a debugging technique on a routine but unsystematic basis. The sensification of programs expanded to physically experiencing them and led to Dr. Krueger’s interactive environment and artificial reality concepts. From the beginning of the VIDEOPLACE system, sound has been used to substitute for touch when a participant’s video image came into contact with a graphic object.[1,2,3] The idea of using audio-touch to present information to blind people dates back to the beginning of his work, but did not resonate with anyone in the disabilities community until 1994 when Dr. Krueger and Dr. Gilden met and collaborated on this NIH sponsored SBIR Phase I project.[4] RELATED WORK Others have used sound in the human-computer interface to illustrate system functions [5][6][7] and for debugging [8][9]. In addition, sound has long been used to try to make information accessible to blind people starting with the audio oscilloscope.[10] More recently, both researchers and commercial entities has sought to make graphical user interfaces to blind people.[11][12]