Seq2seq Model with RNN Attention for Abstractive Summarization

2019 
Conventional attention mechanism of seq2seq model is proposed to solve the translation alignment problem, which calculates the attention scores between encoding and decoding states independently for capturing their one-to-one relationship. However, it does not meet the requirements for information compression on abstract summarization. In this work, we propose a novel attention mechanism based on recurrent neural network (RNN), called RNN Attention (RNNA), which is more adaptive to the summary task. At each decoding time step, RNNA will establish a connection in multiple attentions by RNN for forming a many-to-one relationship between source and target, which achieves the ability of global information summarization. We evaluate our model on two benchmark datasets, LCSTS and English Gigaword. The result shows that our model's attention distribution is more extensive than the baseline model to cover more source information. Moreover, its ROUGE points also are improved and outperform the state-of-the-art models.
    • Correction
    • Source
    • Cite
    • Save
    • Machine Reading By IdeaReader
    28
    References
    0
    Citations
    NaN
    KQI
    []