Computer Engineering and Applications ›› 2023, Vol. 59 ›› Issue (8): 192-199.DOI: 10.3778/j.issn.1002-8331.2112-0570

• Pattern Recognition and Artificial Intelligence • Previous Articles     Next Articles

Neural Recommendation Algorithm Using Combinations of Low and High-Order Features Based on Multi-Attention Mechanism

CUI Shaoguo, DU Xiao, YANG Zetian   

  1. School of Computer and Information Sciences, Chongqing Normal University, Chongqing 401331, China
  • Online:2023-04-15 Published:2023-04-15



  1. 重庆师范大学 计算机与信息科学学院,重庆 401331

Abstract: Aimed at the limitations of factorization machine in extracting only low-order combinatorial features, a deep neural recommendation method based on multi-attention mechanism(DeepNRM) is proposed based on multi-attention mechanism. Firstly, the factorization machine and multi-layer feed-forward neural network are used to extract low-order and high-order combined features from sparse and dense features respectively. Then it uses the attention network and the multi-head self-attention mechanism to automatically select key features from the combination of low-order and high-order features. Finally, it mixes low-order and high-order combined features based on their importance to produce recommendations. The algorithm model is experimentally validated on the common datasets of MovieLens and Criteo, and the ablation and comparative experimental results show that compared with the baseline model, there are 1.964 percentage points and 0.773 percentage points AUC improvements on the two datasets, respectively.

Key words: factorization machine, recommendation system, deep neural network, multi-head self-attention mechanism, feature extraction

摘要: 针对因子分解机仅提取低阶组合特征的局限性,提出了一种基于多注意力机制融合低阶和高阶组合特征的深度神经推荐算法(deep neural recommendation method,DeepNRM)。分别运用因子分解机和多层前馈神经网络从稀疏及稠密特征中提取低阶和高阶组合特征;采用注意力网络和多头自注意力机制从低阶和高阶组合特征中自动选取关键特征;将低、高阶组合特征根据重要性进行融合共同进行推荐。算法模型在MovieLens和Criteo公共数据集上进行了实验验证,消融和对比实验结果表明,提出的算法模型与基准模型相比在AUC指标上分别有1.964个百分点和0.773个百分点的提升。

关键词: 因子分解机, 推荐系统, 深度神经网络, 多头自注意力机制, 特征抽取