Research on Topic-Aware Long Text Summarization Algorithm

doi:10.3778/j.issn.1002-8331.2103-0328

Abstract

Abstract: Summarization generation of long text is always a difficult problem in the field of automatic summarization. The existing methods have some problems such as low accuracy and redundancy in the process of processing long text. In view of the outstanding performance of the topic model in multi-document summarization, it is introduced into the long text summarization task. In addition, the current single extractive or abstractive method can not deal with the complex situation of long text. It proposes a hybrid summarization model for long text based on topic aware, which combines extractive and abstractive methods. The validity of the model is verified on TTNews and CNN/Daily Mail datasets. The ROUGE score of the model is 1 to 2 percentage points higher than that of the same type of model, resulting in a more readable summary.

Key words: topic model, long text summarization, hybrid model, pointer network

摘要： 长文本摘要生成一直是自动摘要领域的难题。现有方法在处理长文本的过程中，存在准确率低、冗余等问题。鉴于主题模型在多文档摘要中的突出表现，将其引入到长文本摘要任务中。另外，目前单一的抽取式或生成式方法都无法应对长文本的复杂情况。结合两种摘要方法，提出了一种针对长文本的基于主题感知的抽取式与生成式结合的混合摘要模型。并在TTNews和CNN/Daily Mail数据集上验证了模型的有效性，该模型生成摘要ROUGE分数与同类型模型相比提升了1~2个百分点，生成了可读性更高的摘要。

关键词: 主题模型, 长文本摘要, 混合模型, 指针网络

YANG Tao, XIE Qing, LIU Yongjian, LIU Pingfeng. Research on Topic-Aware Long Text Summarization Algorithm[J]. Computer Engineering and Applications, 2022, 58(20): 165-173.

杨涛, 解庆, 刘永坚, 刘平峰. 主题感知的长文本自动摘要算法[J]. 计算机工程与应用, 2022, 58(20): 165-173.

References

[1] SUTSKEVER I，VINYALS O，LE Q V.Sequence to sequence learning with neural networks[C]//Advances in Neural Information Processing Systems，2014：3104-3112.
[2] 黄佳佳，李鹏伟，彭敏，等.基于深度学习的主题模型研究[J].计算机学报，2020，43（5）：75-103.
HUANG J J，LI P W，PENG M，et al.Review of deep learning-based topic model[J].Chinese Journal of Computers，2020，43（5）：75-103.
[3] BLEI D M，NG A Y，JORDAN M I.Latent dirichlet allocation[J].Journal of Machine Learning Research，2003，3：993-1022.
[4] KINGMA D P，WELLING M.Auto-encoding variational bayes[C]//The 2nd International Conference on Learning Representations（ICLR），2013.
[5] CARTER C K，KOHN R.On Gibbs sampling for state space models[J].Biometrika，1994，81（3）：541-553.
[6] 王子璇，乐小虬，何远标.基于WMD语义相似度的Text-
Rank改进算法识别论文核心主题句研究[J].数据分析与知识发现，2017，1（4）：1-8.
WANG Z X，LE X Q，HE Y B.Recognizing core topic sentences with improved TextRank algorithm based on WMD semantic similarity[J].Data Analysis and Knowledge Discovery，2017，1（4）：1-8.
[7] MIAO Y，GREFENSTETTE E，BLUNSOM P.Discovering discrete latent topics with neural variational inference[C]//International Conference on Machine Learning，2017：2410-2419.
[8] DEVLIN J，CHANG M，LEE K，et al.Bert：pre-training of deep bidirectional transformers for language understanding[C]//North American Chapter of the Association for Computational Linguistics，2018：4171-4186.
[9] NARAYAN S，COHEN S B，LAPATA M.Ranking sentences for extractive summarization with reinforcement learning[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics：Human Language Technologies，2018：1747-1759.
[10] LIU Y，LAPATA M.Text summarization with pretrained encoders[C]//Proceedings of the 2019 Conference on Empiri-
cal Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing（EMNLP-IJCNLP），2019：3721-3731.
[11] ZHONG M，LIU P，CHEN Y，et al.Extractive summarization as text matching[C]//Proceedings of the Annual Meeting of the Association for Computational Linguistics，2020：6197-6208.
[12] RUSH A M，CHOPRA S，WESTON J.A neural attention model for abstractive sentence summarization[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing，2015：379-389.
[13] NALLAPATI R，ZHOU B，GULCEHRE C，et al.Abstractive text summarization using sequence-to-sequence RNNs and beyond[C]//Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning，2016：280-290.
[14] SEE A，LIU P J，MANNING C D.Get to the point：summarization with pointer-generator networks[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics，2017：1073-1083.
[15] WANG W，GAO Y，HUANG H，et al.Concept pointer network for abstractive summarization[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing（EMNLP-IJCNLP），2019：3067-3076.
[16] LIU L，LU Y，YANG M，et al.Generative adversarial network for abstractive text summarization[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2018.
[17] SRIVASTAVA A，SUTTON C.Autoencoding variational inference for topic models[C]//International Conference on Learning Representations，2017.
[18] LI S，ZHAO Z，HU R，et al.Analogical reasoning on Chinese morphological and semantic relations[C]//Meeting of the Association for Computational Linguistics，2018.
[19] CHEN Y，BANSAL M.Fast abstractive summarization with reinforce-selected sentence rewriting[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics（Volume 1：Long Papers），2018：675-686.
[20] GEHRMANN S，DENG Y，RUSH A M.Bottom-up abstractive summarization[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing，2018：4098-4109.
[21] HUANG S，WANG R，XIE Q，et al.An extraction-abstraction hybrid approach for long document summarization[C]//International Conference on Behavioral Economic and Socio Cultural Computing，2019：1-6.