Compound Convolutional and Self-Attention Network for Session-Based Recommendation

doi:10.3778/j.issn.1002-8331.2202-0128

Abstract

Abstract: Recently, methods based on convolutional neural networks（CNN） have shown potential in modeling conversational data, especially in extracting complex local behavioral interactions of users. Recurrent neural networks（RNNs） have difficulty for learning item dependencies from a distance, while self-attention（SA） structures have the ability to model the sequence of conversational events and capture interactions between distant items, which is a superior choice for session data. Therefore, a compound convolutional and self-attention network（CCNN-SA） architecture is proposed, which utilizes the complex local features extracted by two convolutional modules, and uses a multi-head self-attention structure to learn long-term interactions from conversational events, and this flexible and unified network architecture facilitates comprehensive modeling of various important features of conversational sequences. The proposed model is validated on two benchmark datasets from e-commerce, and the evaluation metrics Recall@20 and MRR@20, which characterize hit rate and prediction result ranking, are improved by 2.79% and 5.87% respectively on the YOOCHOOSE dataset, are improved by 2.17% and 6.43% respectively on the DIGINETIC dataset, which verifies the validity and rationality of the model.

Key words: compound convolution, multi-head self-attention, session-based recommendation

摘要： 最近，基于卷积神经网络（convolutional neural networks，CNN）的方法显示了在会话数据建模中的潜力，尤其是在提取用户复杂的局部行为交互方面。递归神经网络（recurrent neural networks，RNN）很难从远距离学习项目依赖，而自注意力（self-attention，SA）结构具有对会话事件的顺序进行建模并捕获远距离项目之间交互关系的能力，对于会话数据来说是一种优越的选择。因此提出了一种复合卷积自注意力架构（compound convolutional and self-attention network，CCNN-SA）。该架构利用了两个卷积模块提取的复杂局部特征，同时使用了多头自注意力结构从会话事件中学习长期交互关系，这种灵活而统一的网络体系结构有利于对会话序列的各种重要特征进行综合建模。在两个来自电子商务的基准数据集上对所提出的模型进行了验证，在表征命中率和预测结果排名的评估指标Recall@20和MRR@20上，对于YOOCHOOSE数据集分别提升了2.79%和5.87%，在DIGINETIC数据集上分别提升了2.17%和6.43%，验证了模型的有效性与合理性。

关键词: 复合卷积, 多头自注意力, 会话推荐

XIAO Yan, HUO Lin. Compound Convolutional and Self-Attention Network for Session-Based Recommendation[J]. Computer Engineering and Applications, 2023, 59(10): 104-113.

肖妍, 霍林. 基于复合卷积和自注意力的会话推荐[J]. 计算机工程与应用, 2023, 59(10): 104-113.

References

[1] ADOMAVICIUS G，TUZHILIN A.Toward the next generation of recommender systems：a survey of the state-of-the-art and possible extensions[J].IEEE Transactions on Knowledge and Data Engineering，2005，17（6）：734-749.
[2] SCHAFER J B，KONSTAN J，RIEDL J.Recommender systems in e-commerce[C]//Proceedings of the 1st ACM Conference on Electronic Commerce，1999：158-166.
[3] HIDASI B，KARATZOGLOU A，BALTRUNAS L，et al.Session-based recommendations with recurrent neural networks[J].arXiv：1511.06939，2015.
[4] TAN Y K，XU X，LIU Y.Improved recurrent neural networks for session-based recommendations[C]//Proceedings of the 1st Workshop on Deep Learning for Recommender Systems，2016：17-22.
[5] LECUN Y，BOSER B，DENKER J，et al.Handwritten digit recognition with a back-propagation network[C]//Proceedings of the 2nd International Conference on Neural Information Processing Systems，1989：396-404.
[6] CHEN Y.Convolutional neural network for sentence classification[D].University of Waterloo，2015.
[7] VASWANI A，SHAZEER N，PARMAR N，et al.Attention is all you need[C]//Advances in Neural Information Processing Systems 30，2017：6000-6010.
[8] KOREN Y，BELL R，VOLINSKY C.Matrix factorization techniques for recommender systems[J].Computer，2009，42（8）：30-37.
[9] REN Z，LIANG S，LI P，et al.Social collaborative viewpoint regression with explainable recommendations[C]//Proceedings of the 10th ACM International Conference on Web Search and Data Mining，2017：485-494.
[10] RENDLE S，FREUDENTHALER C，GANTNER Z，et al.BPR：Bayesian personalized ranking from implicit feedback[J].arXiv：1205.2618，2012.
[11] KOREN Y.Factorization meets the neighborhood：a multifaceted collaborative filtering model[C]//Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining，2008：426-434.
[12] LUDEWIG M，JANNACH D.Evaluation of session-based recommendation algorithms[J].User Modeling and User-Adapted Interaction，2018，28（4）：331-390.
[13] ODENTRANTZ J.Markov chains：Gibbs fields，Monte Carlo simulation，and queues[J].Technometrics，2012，42（4）：438-439.
[14] ZIMDARS A，CHICKERING D M，MEEK C.Using temporal data for making recommendations[J].arXiv：1301.
2320，2013.
[15] SHANI G，HECKERMAN D，BRAFMAN R I，et al.An MDP-based recommender system[J].Journal of Machine Learning Research，2005，6（9）：1265-1295.
[16] RENDLE S，FREUDENTHALER C，SCHMIDT-THIEME L.Factorizing personalized Markov chains for next-basket recommendation[C]//Proceedings of the 19th International Conference on World Wide Web，2010：811-820.
[17] FIGUEIREDO F，RIBEIRO B，ALMEIDA J M，et al.TribeFlow：mining & predicting user trajectories[C]//Proceedings of the 25th International Conference on World Wide Web，2016：695-706.
[18] HE K，ZHANG X，REN S，et al.Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition，2016：770-778.
[19] LI Z，ZHAO H，LIU Q，et al.Learning from history and present：next-item recommendation via discriminatively exploiting user behaviors[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining，2018：1734-1743.
[20] LI J，REN P，CHEN Z，et al.Neural attentive session-based recommendation[C]//Proceedings of the 2017 ACM on Conference on Information and Knowledge Management，2017：1419-1428.
[21] BAHDANAU D，CHO K，BENGIO Y.Neural machine translation by jointly learning to align and translate[J].arXiv：1409.0473，2014.
[22] LIU Q，ZENG Y，MOKHOSI R，et al.STAMP：short-term attention/memory priority model for session-based recommendation[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining，2018：1831-1839.
[23] 梁天安.双注意力个性化会话推荐模型研究[D]. 武汉：华中科技大学，2019.
LIANG T A.Research on personalized conversational recommendation model with dual attention[D].Wuhan：Huazhong University of Science and Technology，2019.
[24] WU S，TANG Y，ZHU Y，et al.Session-based recommendation with graph neural networks[C]//Proceedings of the 33rd AAAI Conference on Artificial Intelligence，2019：346-353.
[25] SONG B，CAO Y，ZHANG W，et al.Session-based recommendation with hierarchical memory networks[C]//Proceedings of the 28th ACM International Conference on Information and Knowledge Management，2019：2181-2184.
[26] FANG J.Session-based recommendation with self-attention networks[J].arXiv：2102.01922，2021.
[27] TUAN T X，PHUONG T M.3D convolutional networks for session-based recommendation with content features[C]//Proceedings of the 11th ACM Conference on Recommender Systems，2017：138-146.
[28] TANG J，WANG K.Personalized top-n sequential recommendation via convolutional sequence embedding[C]//Proceedings of the 11th ACM International Conference on Web Search and Data Mining，2018：565-573.
[29] YUAN F，KARATZOGLOU A，ARAPAKIS I，et al.A simple convolutional generative network for next item recommendation[C]//Proceedings of the 12th ACM International Conference on Web Search and Data Mining，2019：582-590.
[30] BACH N X，LONG D H，PHUONG T M.Recurrent convolutional networks for session-based recommendations[J].Neurocomputing，2020，411：247-258.
[31] ZHANG J，MA C，MU X，et al.Recurrent convolutional neural network for session-based recommendation[J].Neurocomputing，2021，437：157-167.
[32] JI Z，WU M，FENG Y，et al.Multi-channel convolutional neural network feature extraction for session based recommendation[J].Complexity，2021.DOI：10.1155/2021/6661901.
[33] BARKAN O，KOENIGSTEIN N.Item2Vec：neural item embedding for collaborative filtering[C]//2016 IEEE 26th International Workshop on Machine Learning for Signal Processing，2016：1-6.
[34] XIA X，YIN H，YU J，et al.Self-supervised hypergraph convolutional networks for session-based recommendation[C]//Proceedings of the 35th AAAI Conference on Artificial Intelligence，2021：4503-4511.