基于大语言模型的跨语言会话式商品推荐设计与实现

doi:10.3778/j.issn.1002-8331.2409-0335

摘要/Abstract

摘要： 利用文本等模态表征进行通用会话推荐系统的工作是学术界关注的焦点，在单语言环境下展现了优异性能，但其在跨语言环境下的研究仍属于空白区域。提出了一种跨语言会话推荐系统CLSRec，通过结合使用目标语言与大语言模型生成的中间语言，利用预训练模型生成的文本表征作为输入，摈弃传统物品ID依赖，将其进行文本白化后进行线性池化，提供给会话推荐骨架，最终得到用于推荐的评分。该方法分为预训练与微调两个学习阶段，在预训练阶段运用对比学习策略，从资源丰富语言数据提炼通用知识，在微调阶段学习目标语言数据集上的用户行为模式。采用两阶段学习策略，能够处理冷启动情景，在推荐目标上进行学习的同时，尽可能保留语义建模信息。在真实世界上的多语言电商数据集上进行实验，在三个不同语言数据集的NDCG@10、NDCG@50、Recall@10、Recall@50等指标上均实现了对目前基线的显著超越。还通过全面的对比实验，探究了影响多语言会话推荐系统的因素。

关键词: 跨语言推荐, 会话推荐, 大语言模型, 迁移学习, 对比学习

Abstract: Using modal representations like text to build universal session recommender systems has attracted the attention of the research community. However, these researches are based on the premise that the text information obtained by the recommender system is in the same language. This paper presents a cross-lingual session recommender system, named CLSRec. By using both the pivot language and the target language, the method uses the text representation as input, does not use the item ID, performs text whitening and linear pooling, and provides it to the session recommendation backbone, and finally obtains the score for recommendation. In the pre-training phase, contrastive learning technology is used to learn general knowledge of resource-rich language data. In the fine-tuning phase, user behavior patterns on the target language data set are learned to ensure the generalization of the method and handle cold start scenarios. While learning on the recommended target, the semantic modeling information is retained as much as possible. Experiments on real-world multilingual e-commerce datasets, the method is significantly higher than the current baseline on three different language data on NDCG@10, NDCG@50, Recall@10 and Recall@50. The paper also explores the factors that affect cross-lingual session recommender systems through comprehensive comparative experiments.

Key words: cross-lingual recommendation, session recommendation, large language models, transfer learning, contrastive learning

李奕杉, 孙培杰, 张敏. 基于大语言模型的跨语言会话式商品推荐设计与实现[J]. 计算机工程与应用, 2025, 61(24): 134-143.

LI Yishan, SUN Peijie, ZHANG Min. Cross-Lingual Session Recommender System Based on Large Language Models[J]. Computer Engineering and Applications, 2025, 61(24): 134-143.

参考文献

[1] HIDASI B, KARATZOGLOU A, BALTRUNAS L, et al. Session-based recommendations with recurrent neural networks[J]. arXiv:1511.06939, 2015.
[2] KANG W C, MCAULEY J. Self-attentive sequential recommendation[C]//Proceedings of the 2018 IEEE International Conference on Data Mining. Piscataway: IEEE, 2018: 197-206.
[3] LI C L, ZHAO M J, ZHANG H M, et al. RecGURU: adversarial learning of generalized user representations for cross-domain recommendation[C]//Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining. New York: ACM, 2022: 571-581.
[4] XIE R B, LIU Q, WANG L D, et al. Contrastive cross-domain recommendation in matching[C]//Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. New York: ACM, 2022: 4226-4236.
[5] HOU Y P, MU S L, ZHAO W X, et al. Towards universal sequ-ence representation learning for recommender systems[C]//Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. New York: ACM, 2022: 585-593.
[6] JIN W, MAO H T, LI Z, et al. Amazon-M2: a multilingual multi-locale shopping session dataset for recommendation and text generation[J]: arXiv: 2307. 09688, 2023.
[7] YANG S H, WANG C Y, LIU Y K, et al. Collaborative word-based pre-trained item representation for transferable recommendation[C]//Proceedings of the 2023 IEEE International Conference on Data Mining. Piscataway: IEEE, 2024: 728-737.
[8] ZHANG L Z, ZHOU X, ZENG Z W, et al. Are ID embeddings necessary? whitening pre-trained text embeddings for effective sequential recommendation[C]//Proceedings of the 2024 IEEE 40th International Conference on Data Engineering. Piscataway: IEEE, 2024: 530-543.
[9] GUO T C, YU L, SHIHADA B, et al. Few-shot news recommendation via cross-lingual transfer[C]//Proceedings of the ACM Web Conference 2023. New York: ACM, 2023: 1130-1140.
[10] DONKERS T, LOEPP B, ZIEGLER J. Sequential user-based recurrent neural network recommendations[C]//Proceedings of the Eleventh ACM Conference on Recommender Systems. New York: ACM, 2017: 152-160.
[11] LI J, REN P J, CHEN Z M, et al. Neural attentive session-based recommendation[C]//Proceedings of the 2017 ACM on Conference on Information and Knowledge Management. New York: ACM, 2017: 1419-1428.
[12] PENG B, REN Z Y, PARTHASARATHY S, et al. Ham: hybrid associations models for sequential recommendation [J]. IEEE Transactions on Knowledge and Data Engineering, 2022, 34(10): 4838-4853.
[13] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Advances in Neural Information Processing Systems, 2017.
[14] ZHOU K, WANG H, ZHAO W X, et al. S3-rec: self-supervised learning for sequential recommendation with mutual information maximization[C]//Proceedings of the 29th ACM International Conference on Information & Knowledge Man-agement. New York: ACM, 2020: 1893-1902.
[15] ZHANG T T, ZHAO P P, LIU Y C, et al. Feature-level deeper self-attention network for sequential recommendation[C]//Proceedings of the 28th International Joint Conference on Artificial Intelligence. New York: ACM, 2019: 4320-4326.
[16] WU C H, WU F Z, QI T, et al. Empowering news recommendation with pre-trained language models[C]//Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2021: 1652-1656.
[17] ZHOU X, SHEN Z Q. A tale of two graphs: freezing and denoising graph structures for multimodal recommendation[C]//Proceedings of the 31st ACM International Conference on Multimedia. New York: ACM, 2023: 935-943.
[18] DING H, MA Y, DEORAS A, et al. Zero-shot recommender systems[J]. arXiv:2105.08318, 2021.
[19] ZHANG Q, LI J J, JIA Q L, et al. UNBERT: user-news matching BERT for news recommendation[C]//Proceedings of the International Joint Conference on Artificial Intelligence, 2021.
[20] XIE Y Q, ZHOU P L, KIM S. Decoupled side information fusion for sequential recommendation[C]//Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2022: 1611-1621.
[21] ZHANG L Z, ZHOU X, ZENG Z W, et al. Multimodal pre-training for sequential recommendation via contrastive lea-rning[J]. ACM Transactions on Recommender Systems, 2024, 3(1): 1-23.
[22] ZHANG Q, LI J J, JIA Q L, et al. UNBERT: user-news matching BERT for news recommendation[C]//Proceedings of the International Joint Conference on Artificial Intelligence, 2021.
[23] LIU Q J, CHEN N, SAKAI T, et al. ONCE: boosting content-based recommendation with both open-and closed-source large language models[C]//Proceedings of the 17th ACM International Conference on Web Search and Data Mining. New York: ACM, 2024: 452-461.
[24] YUAN Z, YUAN F J, SONG Y, et al. Where to go next for recommender systems? ID-vs. modality-based recommender models revisited[C]//Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2023: 2639-2649.
[25] ZHU F, WANG Y, CHEN C C, et al. Cross-domain recommendation: challenges, progress, and prospects[J]. arXiv:2103. 01696, 2021.
[26] XU K, WANG Z L, ZHENG W, et al. A centralized-distributed transfer model for cross-domain recommendation based on multi-source heterogeneous transfer learning[C]//Proceedings of the 2022 IEEE International Conference on Data Mining. Piscataway: IEEE, 2023: 1269-1274.
[27] HU G N, ZHANG Y, YANG Q. CoNet: collaborative cross networks for cross-domain recommendation[C]//Proceedings of the 27th ACM International Conference on Information and Knowledge Management. New York: ACM, 2018: 667-676.
[28] YUAN F J, ZHANG G X, KARATZOGLOU A, et al. One person, one model, one world: learning continual user representation without forgetting[C]//Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2021: 696-705.
[29] SINGH A P, GORDON G J. Relational learning via collective matrix factorization[C]//Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2008: 650-658.
[30] ZHU F, CHEN C, WANG Y, et al. DTCDR: a framework for dual-target cross-domain recommendation[C]//Proceedings of the 28th ACM International Conference on Information and Knowledge Management, 2019: 1533-1542.
[31] HOU Y P, HE Z K, MCAULEY J, et al. Learning vector-quantized item representation for transferable sequential recommenders[C]//Proceedings of the ACM Web Conference 2023. New York: ACM, 2023: 1162-1171.
[32] ZHANG W X, HE R D, PENG H Y, et al. Cross-lingual aspect-based sentiment analysis with aspect term code-switching[C]//Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: ACL, 2021: 9220-9230.
[33] SU J, CAO J, LIU W, et al. Whitening sentence representations for better semantics and faster retrieval[J]. arXiv:2103. 15316, 2021
[34] SHAZEER N, MIRHOSEINI A, MAZIARZ K, et al. Outrageously large neural networks: the sparsely-gated mixture-of-experts layer[J]. arXiv:1701.06538, 2017.
[35] DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the North American Chapter of the Association for Computational Linguistics, 2019.
[36] WU S, TANG Y Y, ZHU Y Q, et al. Session-based recommendation with graph neural networks[C]//Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence. New York: ACM, 2019: 346-353.
[37] XU C F, ZHAO P P, LIU Y C, et al. Graph contextualized self-attention network for session-based recommendation[C]//Proceedings of the 28th International Joint Conference on Artificial Intelligence. New York: ACM, 2019: 3940-3946.
[38] RENDLE S, FREUDENTHALER C, SCHMIDT-THIEME L. Factorizing personalized Markov chains for next-basket recommendation[C]//Proceedings of the 19th International Conference on World Wide Web. New York: ACM, 2010: 811-820.
[39] JIANG A Q, SABLAYROLLES A, MENSCH A, et al. Mistral 7B[J]. arXiv:2310.06825, 2023.
[40] WANG L, YANG N, HUANG X, et al. Multilingual e5 text embeddings: a technical report[J]. arXiv:2402.05672, 2024.