A Grammar-Aware Pointer Network for Abstractive Summarization

Pointer network (PN) has achieved breakthrough in recent years of text summarization research. But it only focuses on semantic relevance of the source sequence; in fact, the text should also comply with explicit grammar rules. The semantics and syntax are the two main granularities to research in this paper. In more detail, we proposed a grammar-aware pointer network (GPAN) for abstractive summarization, which not only tracks the key semantics of the original text, but also observes the syntax rules. To enforce the syntactic constraints, we get each word attached with the part-of-speech (POS) tag and syntactic dependency (DEP) tag and input them into the recurrent network when training the network. Then, we predict the POS and DEP at each decoder time step; by this way, we trying to let the model learn to track the grammar information of the ground truth. We evaluate our model on the benchmark dataset CNN/Daily Mail and GiGaword. The experimental results show that our model leads to significant improvements.