Personalized Product Review Summary Generation Based on TRF-IM Model

doi:10.3778/j.issn.1002-8331.2107-0144

Abstract

Abstract: Aiming at the problems of insufficient contextual understanding of reviews, insufficient parallelism and long-distance text dependence in the process of generating a summary of the traditional model of hotel review summary generation, a personalized hotel review summary generation method based on TRF-IM（improved mask for transformer） model is proposed. This method uses the Transformer decoder structure to model the review summary task. By improving the masking method in the structure, the source review content can better learn the contextual semantic information. At the same time, user-type personalized word feature information is introduced to generate high-quality personalized hotel review summaries that meet user needs. The experimental results show that the proposed model achieves higher scores on the ROUGE indicator than the traditional model, and generates high-quality personalized hotel review summaries.

Key words: personalized hotel reviews, generative summary, Transformer, masking method, language mode

摘要： 针对传统的酒店评论摘要生成模型在生成摘要过程中存在对评论的上下文理解不够充分、并行能力不足和长距离文本依赖缺陷的问题，提出了一种基于TRF-IM（improved mask for transformer）模型的个性化酒店评论摘要生成方法。该方法利用Transformer译码器结构对评论摘要任务进行建模，通过改进其结构中的掩码方式，使得源评论内容都能够更好地学习到上下文语义信息；同时引入了用户类型的个性化词特征信息，使其生成高质量且满足用户需求的个性化酒店评论摘要。实验结果表明，该模型相比传统模型在ROUGE指标上取得了更高的分数，生成了高质量的个性化酒店评论摘要。

关键词: 个性化酒店评论, 生成式摘要, Transformer, 掩码方式, 语言模型

GAO Weijun, ZHU Jing, ZHAO Huayang, LI Lei. Personalized Product Review Summary Generation Based on TRF-IM Model[J]. Computer Engineering and Applications, 2023, 59(2): 135-142.

高玮军, 朱婧, 赵华洋, 李磊. 基于TRF-IM模型的个性化酒店评论摘要生成[J]. 计算机工程与应用, 2023, 59(2): 135-142.

References

[1] LIN Y，LAPATA M.Text summarization with pretrained encoders[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing（EMNLP-IJCNLP），2019：3730-3740.
[2] ZHON M，LIU P，CHEN Y，et al.Extractive summarization as text matching[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics，2020：6197-6208.
[3] WANG W，GAO Y，HUANG H Y，et al.Concept pointer network for abstractive summarization[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing（EMNLP-IJCNLP），2019：3076-3085.
[4] GAO S，REN Z，ZHAO Y，et al.Product-aware answer generation in e-commerce question-answering[C]//Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining，2019：429-437.
[5] YANG M，QU Q，SHEN Y，et al.Aspect and sentiment aware abstractive review summarization[C]//Proceedings of the 27th International Conference on Computational Linguistics，2018：1110-1120.
[6] LI H，ZHU J，ZHANG J，et al.Keywords-guided abstractive sentence summarization[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2020：8196-8203.
[7] VASWANI A，SHAZEER N，PARMAR N，et al.Attention is all you need[C]//Advances in Neural Information Processing Systems，2017：5998-6008.
[8] ERKAN G，RADEV D R.Lexrank：graph-based lexical centrality as salience in text summarization[J].Journal of Artificial Intelligence Research，2004，22：457-479.
[9] RUSH A M，CHOPRA S，WESTON J.A neural attention model for abstractive sentence summarization[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing，2015：379-389.
[10] COSTA F，OUYANG S，DOLOG P，et al.Automatic generation of natural language explanations[C]//Proceedings of the 23rd International Conference on Intelligent User Interfaces Companion，2018：1-2.
[11] TANG J，YANG Y，CARTON S，et al.Context-aware natural language generation with recurrent neural networks[J].arXiv：1611.09900，2016.
[12] DONG L，HUANG S，WEI F，et al.Learning to generate product reviews from attributes[C]//Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics：Volume 1，Long Papers，2017：623-632.
[13] 苏放，王晓宇，张治.基于注意力机制的评论摘要生成[J].北京邮电大学学报，2018，41（3）：7-13.
SU F，WANG X Y，ZHANG Z.Review summarization generation based on attention mechanism[J].Journal of Beijing University of Posts and Telecommunications，2018，41（3）：7-13.
[14] 周健，田萱，崔晓晖.基于改进Sequence-to-Sequence模型的文本摘要生成方法[J].计算机工程与应用，2019，55（1）：128-134.
ZHOU J，TIAN X，CUI X H.Generation method of text summarization based on advanced Sequence-to-Sequence model[J].Computer Engineering and Applications，2019，55（1）：128-134.
[15] AKHTAR N，ZUBAIR N，KUMAR A，et al.Aspect based sentiment oriented summarization of hotel reviews[J].Procedia Computer Science，2017，115：563-571.
[16] WANG W，ZHENG H T，LIU H.User preference-aware review generation[C]//Pacific-Asia Conference on Knowledge Discovery and Data Mining.Cham：Springer，2019：225-236.
[17] JIANG W，CHEN J，DING X，et al.Review summary generation in online systems：frameworks for supervised and unsupervised scenarios[J].ACM Transactions on the Web，2021，15（3）：1-33.
[18] GRAVES A.Sequence transduction with recurrent neural networks[J].arXiv：1211.3711v1，2012.
[19] NALLAPATI R，ZHOU B，DOS SANTOS C，et al.Abstractive text summarization using sequence-to-sequence RNNs and beyond[C]//Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning，2016：280-290.
[20] SEE A，LIU P J，MANNING C D.Get to the point：summarization with pointer-generator networks[J].arXiv：1704.

04368，2017.

[21] BRAZINSKAS A，LAPATA M，TITOV I.Unsupervised opinion summarization as copycat-review generation[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics，2020：5151-5169.