A Review of Multimodal Interaction in Intelligent Systems

Interest in the development of multimodal interaction modalities (e.g. visual, auditory, haptics etc.) has become more prominent. Despite this strong interest in multimodality, research on this domain remains to be quite a few. The present study aims to review the current literature regarding multimodal interaction in intelligent systems, specifically in extended realities (XR) and robots. 961 research articles gathered through engineering and science databases were subjected to rigorous screening. The articles were evaluated by multiple researchers based on its quality and relevance. 30 highly-relevant empirical research papers about multimodal interaction in the context of XR and robot systems were chosen and were subjected for review. Reviewed papers were summarized through descriptive statistics and qualitative analysis. It was revealed that although multimodal interaction is being utilized in various intelligent systems, there are still gaps in the literature that needs to be filled in order to completely understand the concept. The lack of user-centered and human factor-based investigation indicates that the current intelligent systems that require human interaction still need to be improved further. The results of this study can be utilized by system developers and designers in order to create systems that would better accommodate the needs and capabilities of its users.