Optimal Time Window for the Integration of Spatial Audio-Visual Information in Virtual Environments

Sound duration and location may influence both auditory and visual perception with consequences for the judgement of both auditory-visual event location and integration. This study investigated audio-visual integration in a virtual environment using both short- and long-duration auditory stimuli with visual stimuli temporally offset from the start of the auditory stimulus, to investigate the effects of top-down neural effects on perception. Two tasks were used, an auditory localization task and a detection task (judgement of audio-visual synchrony). Eleven participants took part in the study using a HTC Vive Pro. The short-duration auditory stimuli (35-ms spatialized sound) and long-duration auditory stimuli (600-ms non-spatialized sound followed by 35 ms of spatialized sound) were presented at −60°, −30°, 0°, +30° and +60° degrees azimuth, with the visual stimulus presented synchronously or asynchronously with respect to the start of the auditory stimulus. Results showed that localization errors were larger for the longer-duration stimuli and judgements of audiovisual synchrony tended to be improved for stimuli presented at ±30°. Top-down neural processing can affect spatial localization and audio-visual processing. Auditory localization errors and audio-visual synchrony detection may reveal the effects of underlying neural feedback mechanisms that can be harnessed to optimize audio-visual experiences in virtual environments.