关键词: 大数据, DBSCAN算法, Apache Spark, 分布式计算

Abstract: DBSCAN algorithm is an excellent algorithm based on density. It can cluster arbitrary shape data and recognize noise data. In order to reduce the intervention of the input parameters neighborhood radius Eps and Minimum number of Points（MinPts）, a new method of calculating the Eps parameters is proposed. At the same time, in order to solve the performance problem of the traditional single machine DBSCAN algorithm in large data environment, the parallelization of the DBSCAN algorithm is realized based on the Spark framework. The experimental results show that the proposed DBSCAN algorithm has high accuracy and stability, and the parallel implementation of the DBSCAN algorithm has good parallel performance and is suitable for processing mass data clustering.

Key words: big data, DBSCAN, Apache Spark, distributed computing