Computer Engineering and Applications ›› 2020, Vol. 56 ›› Issue (1): 83-91.DOI: 10.3778/j.issn.1002-8331.1810-0142

Previous Articles     Next Articles

Improvement of Slope One Algorithm and Its Implementation on Big Data Platform

LIU Jiayao, WANG Jiabin   

  1. School of Engineering, Huaqiao University, Quanzhou, Fujian 362021, China
  • Online:2020-01-01 Published:2020-01-02

Slope One算法的改进及其在大数据平台的实现


  1. 华侨大学 工学院,福建 泉州 362021

Abstract: Aiming at the problem that the original Slope One algorithm ignores the similarities between the projects when calculating the value of the recommendation prediction, and the  recommendation inefficiency is low in the big data age, the clustering weighted Slope One recommendation algorithm based on the Spark platform is proposed. Firstly, the nearest neighbor set is generated by Canopy-K-medoids clustering algorithm. Then, in the nearest neighbor set, the Slope One algorithm is used to estimate the similarity between the weighted items. Finally, parallelization is implemented on the Spark platform. The experiments in film data set show that optimization algorithm based on Spark platform compared with the traditional Slope One algorithm and weighted similarity of Slope project One algorithm, improves the precision of recommendation.

Key words: Slope One algorithm, clustering, Spark platform, recommendation algorithm

摘要: 针对原始Slope One算法计算推荐预测值时忽略了项目之间的相似性,以及大数据时代下推荐效率低下的问题,提出基于Spark平台的聚类加权Slope One推荐算法。通过Canopy-K-medoids聚类算法生成最近邻居集合;在最近邻集中用Slope One算法上加权项目之间的相似性进行推荐预测;在Spark平台上实现并行化。通过在电影数据集上的实验得出,基于Spark平台的优化算法与传统Slope One算法、加权项目相似度的Slope One算法相比,提高了推荐精度。

关键词: Slope One算法, 聚类, Spark平台, 推荐算法