Generating Diverse Conversation Responses by Creating and Ranking Multiple Candidates

Abstract This paper introduces our systems built for Track 2 of Dialog System Technology Challenge 7 (DSTC7). This challenge track aimed to evaluate the response generation methods using fully data-driven conversation models in a knowledge-grounded setting, where textual facts were provided as the knowledge for each context-response pair. The sequence-to-sequence models have achieved impressive results in machine translation and have also been widely used for end-to-end generative conversation modelling. However, they tended to output dull and repeated responses in previous studies. Our work aims to promote the diversity of end-to-end conversation response generation by adopting a two-stage pipeline. 1) Create multiple responses for an input context together with its textual facts. At this stage, two different models are designed, i.e., a variational generative (VariGen) model and a retrieval-based (Retrieval) model. 2) Rank and return the most relevant response by training a topic coherence discrimination (TCD) model for calculating ranking scores. In our experiments, we demonstrated the effectiveness of the response ranking strategy and the external textual knowledge for generating better responses. According to the official evaluation results, our Retrieval and VariGen systems ranked first and second respectively among all participant systems on Entropy metrics which measured the objective diversity of generated responses. Besides, the VariGen system ranked second on NIST and METEOR metrics which measured the objective quality of generated responses.