关键词: 文本摘要生成, 混合注意力, 强化学习, 自然语言处理, 曝光偏差, 递归神经网络

Abstract: RNN-based sequence-to-sequence models have achieved good performance on abstractive summarization. However, these models have some shortcomings including repetitive and exposure bias. A model is presented based on mixed attention including temporal attention and decoding self-attention saving history attention and adding attention for the decoded word to optimize repetition problem. Reinforcement learning is used as a new training method to solve the problem of exposure bias, and modifying the loss function to improve the result. The proposed method is tested using CNN/Daily Mail data set by ROUGE, showing that mixed attention can improve the repetition problem, and the exposure bias can be eliminated by reinforcement learning, and the integrated model surpasses the advanced algorithm on the test set.

Key words: abstractive summarization, mixture attention, reinforcement learning, natural language processing, exposure bias, recursive neural network