Towards a model human cochlea: sensory substitution for crossmodal audio-tactile displays

We present a Model Human Cochlea (MHC): a sensory substitution technique for creating a crossmodal audio-touch display. This research is aimed at designing a chair-based interface to support deaf and hard of hearing users in experiencing musical content associated with film, and seeks to develop this multisensory crossmodal display as a framework for supporting research in enhancing sensory entertainment experiences for universal design. The MHC uses audio speakers as vibrotactile devices placed along the body to facilitate the expression of emotional elements that are associated with music. We present the results of our formative study, which compared the MHC to conventional audio speaker displays for communicating basic emotional information through touch. Results suggest that the separation of audio signals onto multiple vibrotactile channels is more effective at expressing emotional content than is possible using a complete audio signal as vibrotactile stimuli.