Visual Dialog with Targeted Objects

Visual Dialog aims to exchange information of the image between a questioner and an answerer through asking and answering questions alternately. To generate an accurate response to the target question requires to understand the visual information of the image according to the semantic information of the question and dialog history. However, existing methods paid attention to the visual feature of the whole image while ignoring that certain objects in the image always convey specific semantic. In this paper, we propose a novel visual dialog approach to focus on objects in the image which contain both visual information of attributes and semantic information of categories. Besides, we design several selection methods with different types of semantic guidance of the question and dialog history to select relevant objects. We evaluate the proposed approach on two datasets VisDial v0.9 and VisDial v1.0. And the performance improvement demonstrates the effectiveness of our approach.