News Vector Representation Method Based on Semantic Relevance of Title and Body

doi:10.3778/j.issn.1002-8331.2108-0342

Abstract

Abstract: Aiming at the problems of large length of news body and complex semantic information of the news, this paper proposes a news vector representation method based on the semantic relevance of the title and the body（NRTA model）. Take news title as queries, excavate supplementary information of news title from multiple regions of the news body, pay attention to the semantics of the former text as well as the semantics of the latter text, and reduce the bias in the understanding of the news. Experiments on two real news recommendation datasets MIND and Adressa show that compared with the baseline method, the improvement of this method in each evaluation is between 0.86% and 3.95%, which verifies the importance of the latter semantics of the news body and further enriches the news vector representation.

Key words: news recommendation, semantic information of news body, news vector representation, attention mechanism

摘要： 针对新闻正文文本长度大、语义信息复杂的问题，提出了一种标题与正文语义融合的新闻向量表示方法（NRTA模型）。以新闻标题为查询，从正文的多个区域中挖掘标题的补充信息，关注前文语义的同时也关注后文语义，减少对新闻正文理解的偏差。在两个真实新闻推荐数据集MIND和Adressa上的实验表明，该方法较基线方法在各评价指标上的提升幅度在0.86%到3.95%之间，验证了正文后文语义信息的重要性，进一步丰富了新闻向量表示。

关键词: 新闻推荐, 正文语义信息, 向量表示, 注意力机制

LIAN Xiaoying, XUE Yuanhai, LIU Yue, SHEN Huawei. News Vector Representation Method Based on Semantic Relevance of Title and Body[J]. Computer Engineering and Applications, 2023, 59(4): 89-96.

连晓颖, 薛源海, 刘悦, 沈华伟. 标题与正文语义融合的新闻向量表示方法[J]. 计算机工程与应用, 2023, 59(4): 89-96.

References

[1] WU C H，WU F Z，GE S Y，et al.Neural news recommendation with multi-head self-attention[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing，2019：6390-6395.
[2] OKURA S，TAGAMI Y，ONO S，et al.Embedding-based news recommendation for millions of users[C]//Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining，2017：1933-1942.
[3] SU X，KHOSHGOFTAAR T M.A survey of collaborative filtering techniques[J].Advances in Artificial Intelligence，2009：421425.
[4] ?ZG?BEK ?，GULLA J A，ERDUR R C.A survey on challenges and methods in news recommendation[C]//Proceedings of the 10th International Conference on Web Information Systems and Technologies，2014：278-285.
[5] RUSSELL S，NORVIG P.Artificial intelligence：a modern approach[M].Upper Saddle River：Prentice Hall，2002.
[6] BLEI D M，NG A Y，JORDAN M I.Latent Dirichlet allocation[J].Journal of Machine Learning Research，2003，3：993-1022.
[7] LIAN J，ZHANG F，XIE X，et al.Towards better representation learning for personalized news recommendation：a multi-channel deep fusion approach[C]//Proceedings of the 27th International Joint Conference on Artificial Intelligence，2018：3805-3811.
[8] GUO H，TANG R，YE Y，et al.DeepFM：a factorization-machine based neural network for CTR prediction[J].arXiv：1703.04247，2017.
[9] CHENG H T，KOC L，HARMSEN J，et al.Wide & deep learning for recommender systems[C]//Proceedings of the 1st Workshop on Deep Learning for Recommender Systems，2016：7-10.
[10] VASWANI A，SHAZEER N，PARMAR N，et al.Attention is all you need[J].arXiv：1706.03762，2017.
[11] ZHU Q，ZHOU X，SONG Z，et al.DAN：deep attention neural network for news recommendation[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2019，33：5973-5980.
[12] WU C，WU F，AN M，et al.NPA：neural news recommendation with personalized attention[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining，2019：2576-2584.
[13] WU C，WU F，AN M，et al.Neural news recommendation with attentive multi-view learning[J].arXiv：1907.05576，2019.
[14] KIM Y.Convolutional neural networks for sentence classification[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing，2014：1746-1751.
[15] DAI Z，YANG Z，YANG Y，et al.Transformer-XL：attentive language models beyond a fixed-length context[J].arXiv：1901.02860，2019.
[16] WU F，QIAO Y，CHEN J H，et al.MIND：a large-scale dataset for news recommendation[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics，2020：3597-3606.
[17] GULLA J A，ZHANG L，LIU P，et al.The Adressa dataset for news recommendation[C]//Proceedings of the 2017 International Conference on Web Intelligence，2017：1042-1048.
[18] PENNINGTON J，SOCHER R，MANNING C D.GloVe：global vectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing，2014：1532-1543.
[19] MIKOLOV T，CHEN K，CORRADO G，et al.Efficient estimation of word representations in vector space[J].arXiv：1301.3781，2013.