Effective Sentiment Analysis for Multimodal Review Data on the Web

Multimodal review data on the web mainly involve videos, audios, and texts, which have been the major form of review data on E-business, social networks, etc. Extracting the sentimental opinions of the multimodal review data on the web is helpful for many web-based services, such as online customer services, product recommendation, and personalized web search. However, the multiple modalities of the multimodal review data on the web introduces new challenges for sentiment analysis. The key issue is how to make fusion of multimodal data to improve the effectiveness of sentiment analysis. In this paper, we present a novel two-staged self-attention-based approach for multimodal sentiment analysis. At the first stage, we perform an inter-utterance learning by using a self-attention neural network. At the second stage, we conduct a cross-model fusion to integrate information from different modalities. To the best of our knowledge, this is the first work that utilizes the self-attention network to achieve inter-utterance sentiment analysis for multimodal data. We conduct experiments on two public datasets and compare our proposal with several state-of-the-art methods. The results suggest the effectiveness and robustness of our proposal.