Computer Engineering and Applications ›› 2022, Vol. 58 ›› Issue (9): 9-18.DOI: 10.3778/j.issn.1002-8331.2112-0382

• Research Hotspots and Reviews • Previous Articles     Next Articles

Survey on Attention Mechanisms in Deep Learning Recommendation Models

GAO Guangshang   

  1. School of Management, Guilin University of Technology, Guilin, Guangxi 541004, China
  • Online:2022-05-01 Published:2022-05-01

深度学习推荐模型中的注意力机制研究综述

高广尚   

  1. 桂林理工大学 商学院,广西 桂林 541004

Abstract: Aims to explore how the attention mechanism helps the recommendation model to dynamically focus on specific parts of the input that help to perform the current recommendation task. This paper analyzes the attention mechanism network framework and the weight calculation method of its input data, and then summarizes from the five perspectives of vanillaattention mechanism, co-attention mechanism, self-attention mechanism, hierarchical attention mechanism, and multi-head attention mechanism. Analyze how it uses key strategies, algorithms, or techniques to calculate the weight of the current input data, and use the calculated weights so that the recommendation model can focus on the necessary parts of the input at each step of the recommendation task, more effective user or item feature representation can be generated, and the operating efficiency and generalization ability of the recommendation model are improved. The attention mechanism can help the recommendation model assign different weights to each part of the input, extract more critical and important information, and enable the recommendation model to make more accurate judgments, and it will not bring more overhead to the calculation and storage of the recommendation model. Although the existing deep learning recommendation model with the attention mechanism can meet the needs of most recommendation tasks to a certain extent, it is certain that the uncertainty of human needs and the explosive growth of information under certain circumstances factors, it will still face the challenges of recommendation diversity, recommendation interpretability, and the integration of multiple auxiliary information.

Key words: attention mechanism, neural network, deep learning, deep learning recommendation model

摘要: 探讨注意力机制如何帮助推荐模型动态关注有助于执行当前推荐任务输入的特定部分。分析注意力机制网络框架及其输入数据的权重计算方法,分别从标准注意力机制、协同注意力机制、自注意力机制、层级注意力机制和多头注意力机制这五个角度出发,归纳分析其如何采用关键策略、算法或技术来计算当前输入数据的权重,并通过计算出的权重以使推荐模型可以在推荐任务的每个步骤上专注于输入的必要部分,从而产生更为有效的用户或物品特征表示,进而提高推荐模型的运行效率、泛化能力等。注意力机制可以帮助推荐模型对输入的每个部分赋予不同的权重,抽取出更加关键及重要的信息,使推荐模型做出更加准确的判断,同时不会对推荐模型的计算和存储带来更大的开销。尽管现有融合注意力机制的深度学习推荐模型能在一定程度上满足大部分推荐任务的需求,但可以肯定的是,在特定情况下人类需求的不确定性、信息的爆炸式增长这两个因素,将使得其仍然面临着推荐多样性、推荐可解释性和多种辅助信息融合等方面的挑战。

关键词: 注意力机制, 神经网络, 深度学习, 深度学习推荐模型