Rating Prediction Model Based on Variational Auto-Encoder

doi:10.3778/j.issn.1002-8331.2006-0440

Abstract

Abstract:

The deep learning model has the limitation of poor robustness. For example, the addition of specific noise in the picture will affect the classification and prediction results of the picture. Recently, some scholars have introduced deep learning into the recommendation system, so the influence of noise on the recommendation accuracy is also a problem in the recommendation system. Regarding the robustness of the deep recommendation model, this paper proposes a new rating prediction model REcommender Variational Auto-Encoder（REVAE） based on Variational Auto-Encoder（VAE）. In order to train the robustness of the model to noise interference, a layer of hidden layer representation is added on the traditional VAE, the posterior distribution is used to constrain the hidden layer representation, and the noise is added on the hidden layer. By reconstructing the input data, the recommended algorithm model with anti-noise ability is trained. The experimental results on the public Movielens dataset show that REVAE can effectively reduce the noise interference to the model, making the whole model more robust and having better recommendation effect than other rating prediction algorithms.

Key words: deep learning, recommendation system, Variational Auto-Encoder（VAE）, rating prediction

摘要：

深度学习模型具有鲁棒性差的局限性，常见的如在图片中增加特定的噪声会影响到图片的分类和预测结果。近期有学者将深度学习引入到推荐系统中，因此在推荐系统中也存在噪声对推荐精度影响的问题。针对深度推荐模型的鲁棒性问题，基于变分自编码器（Variational Auto-Encoder，VAE）提出了新的评分预测模型REVAE （REcommender Variational Auto-Encoder）。该模型为了训练模型对噪声干扰的鲁棒性，在传统的VAE上增加了一层隐层表示，利用后验分布对隐层表示进行约束，并在该隐层上增加了噪声，通过重构输入数据，训练得到具有抗噪能力的推荐算法模型。在公开的Movielens数据集上进行的实验结果表明，REVAE可以有效降低噪声对模型的干扰，使得整个模型更具有健壮性，相比其他评分预测算法具有更好的推荐效果。

关键词: 深度学习, 推荐系统, 变分自编码器（VAE）, 评分预测

CHEN Hai, QIAN Fulan, CHEN Jie, ZHAO Shu, ZHANG Yanping. Rating Prediction Model Based on Variational Auto-Encoder[J]. Computer Engineering and Applications, 2021, 57(22): 153-159.

陈海，钱付兰，陈洁，赵姝，张燕平. 基于变分自编码器的评分预测模型[J]. 计算机工程与应用, 2021, 57(22): 153-159.

References

[1] LECUN Y，BENGIO Y，HINTON G.Deep learning[J].Nature，2015，521（7553）：436-444.
[2] WANG J F，HAN P F.Adversarial training-based mean Bayesian personalized ranking for recommender system[J].IEEE Access，2020，8：7958-7968.
[3] HE X，HE Z，DU X，et al.Adversarial personalized ranking for recommendation[C]//41st International ACM SIGIR Conference on Research & Development in Information Retrieval.New York：ACM，2018：355-364.
[4] YUAN F，YAO L，BENATALLAH B.Adversarial collaborative auto-encoder for top-n recommendation[C]//2019 International Joint Conference on Neural Networks，Budapest，2019：1-8.
[5] WANG K，XU L，HUANG L，et al.Stacked discriminative denoising auto-encoder based recommender system[C]//International Conference on Intelligent Science and Big Data Engineering，Lanzhou.Cham：Springer，2018：276-286.
[6] ROSCA M，LAKSHMINARAYANAN B，WARDE-FARLEY D，et al.Variational approaches for auto-encoding generative adversarial networks[J].arXiv：1706.04987，2017.
[7] TAKAHASHI H，IWATA T，YAMANAKA Y，et al.Variational autoencoder with implicit optimal priors[J].arXiv：1809.05284v2，2019.
[8] LI Z，CHEN Y，SOMMER F T.Annealed denoising score matching：learning energy-based models in high-dimensional spaces[J].arXiv：1910.07762，2019.
[9] BIGDELI S A，LIN G，PORTENIER T，et al.Learning generative models using denoising density estimators[J].arXiv：2001.02728，2020.
[10] SALAKHUTDINOV R，MNIH A，HINTON G.Restricted Boltzmann machines for collaborative filtering[C]//24th International Conference on Machine Learning，Corvalis，2007：791-798.
[11] SEDHAIN S，MENON A K，SANNER S，et al.AutoRec：autoencoders meet collaboration filting[C]//International Conference on World Wide Web，Florence，2015：111-112.
[12] WU Y，DUBOIS C，ESTER M，et al.Collaborative denoising auto-encoders for top-N recommender systems[C]//International Conference on Web Search and Data Mining，San Francisco，2016：153-162.
[13] CHAE D K，KIM S W，LEE J T.Autoencoder-based personalized ranking framework unifying explicit and implicit feedback for accurate top-N recommendation[J].Knowledge Based System，2019，176：110-121.
[14] OUYANG Y，LIU W，RONG W，et al.Autoencoder-based collaborative filtering[C]//International Conference on Neural Information Processing，Kuching.Cham：Springer，2014：284-291.
[15] YI J，HUANG J，QIN J.Rating prediction in review-based recommendations via adversarial auto-encoder[C]//IEEE/WIC/ACM International Conference on Web Intelligence，Santiago，2018：144-149.
[16] KHAN Z A，ZUBAIR S，IMRAN K，et al.A new user rating-trend based collaborative denoising auto-encoder for top-N recommender system[J].IEEE Access，2019，7：141287-141310.
[17] SHENBIN I，ALEKSEEV A，TUTUBALINA E，et al.RecVAE：a new variational autoencoder for top-N recommendations with implicit feedback[C]//13th International Conference on Web Search and Data Mining，2020：528-536.
[18] KINGMA D P，WELLING M.Auto-encoding variational Bayes[J].arXiv：1312.6114，2013.
[19] SALAKHUTDINOV R.Probabilistic matrix factorization[C]//International Conference on Neural Information Processing System，Vancouver，2017：1257-1264.
[20] ZHANG S，YAO L，XU X，et al.Hybrid collaborative recommendation via semi-AutoEncoder[C]//International Conference on Neural Information Processing，Guangzhou，2017：185-193.
[21] SALAKHUTDINOV R，MNIH A.Bayesian probabilistic matrix factorization using Markov chain Monte Carlo[C]//International Conference on Machine Learning，New York，2008：880-887.
[22] GEORGIEV K，NAKOV P.A non-IID framework for collaborative filtering with restricted Boltzmann machines[C]//International Conference on Machine Learning，Atlanta，2013：1148-1156.
[23] QIAN F，MIN Q，ZHAO S，et al.Three-way decision collaborative recommendation algorithm based on user reputation[C]//International Joint Conference on Rough Sets.Cham：Springer，2019：424-438.
[24] PATEREK A L.Improving regularized singular value decomposition for collaborative filtering[C]//KDD Cup and Workshop，San Jose，2007：39-42.
[25] MAHESHWARI S，MAJUMDAR A.Hierarchical autoencoder for collaborative filtering[C]//2018 International Joint Conference on Neural Networks，2018：1-7.
[26] 程磊，高茂庭.结合时间加权和LDA聚类的混合推荐算法[J].计算机工程与应用，2019，55（11）：160-166.
CHENG L，GAO M T.Hybrid recommendation algorithm based on time weighted and LDA clustering[J].Computer Engineering and Applications，2019，55（11）：160-166.