关键词: 深度学习, 推荐系统, 协同深度学习, Spark

Abstract: Due to the strong feature learning ability and robustness in model fitting, Collaborative Deep Learning（CDL） solves the problem that the performance falls sharply in the case of data sparsity in recommendation systems. However, recommendation system cannot deal with big data because of the difficulty to maintain the training of model, which can bring up many problems that hard to predict. To solve the problem mentioned above, this paper proposes a model called “CDL with item private node”（CDL-i） on the basis of collaborative depth learning and its parallelization method. The private network node is added to improve the self-coding network in traditional CDL. In the case of sharing network parameters of the model, private bias for each projects are added to make the network more target to the content of the project learning parameters, so as to improve the model of project content detection in the recommendation system performance. In addition, the algorithm is parallelized by splitting the model, and a parallel training method of CDL-i is also proposed. The parallelized model is applied in Spark distributed cluster, and the parameters of each part of the model are optimized in parallel to enhance the data scale and extensibility of the model. The effectiveness and efficiency of the proposed algorithm are verified by experiments on multiple real data sets.

Key words: deep learning, recommender system, Collaborative Deep Learning（CDL）, Spark