关键词: 大数据, 频繁模式挖掘, Top-k, 模式树, 并行计算

Abstract: Under the environment of big data, the frequent pattern mining algorithm Apriori has some defects, including presetting minimum threshold and high time complexity when in data processing process. Therefore, the multistage mining strategy is adopted to realize the parallel frequent pattern mining algorithm（PTFP-Apriori）. Firstly, the preprocessed data is stored in a pattern tree, and the optimal threshold is got by the most frequent [K] model. Subsequently, according to the threshold, the frequent pattern that can’t grow up to be frequent patterns could be removed to reduce the computing scale. The RDD is used to accomplish the task of itemsets support counting and candidate itemsets generating. The experimental results show that the algorithm has higher effectivity and scalability than the traditional algorithm.

Key words: big data, frequent pattern mining, Top-k, pattern tree, parallel computing